fix: 增强 LLM 参数适配层 (#1317)

* fix: add llm generation parameter adaptation

* fix(review-feedback-1317): 补充官方来源链接或明确写成基于当前兼容测试的保守策略,并说明验证环境

* fix(review-feedback-1317): 补充官方文档/公告链接、当前依赖/运行时验证依据,以及 provider override 的预期扩展方式

* feat: 增强 LLM 参数错误自愈 (#1318)

* fix: add llm generation parameter adaptation

* fix: add llm parameter error recovery

* fix(review-feedback-1318): Scope recovery cache to the endpoint and making recovery caching

* fix(review-feedback-1318): Avoid forcing 1.0 for all default-only temperature errors

* fix(review-feedback-1318): 基于目标分支解决冲突,并确保解决后 diff 仍只包含本 PR 声称的 LLM 参数自愈增量

* fix(review-feedback-1318): Scope legacy Router recoveries to their endpoint

* fix(review-feedback-1318): 解决冲突并基于解决后的最终 diff 重新确认测试结果

* fix(review-feedback-1318): 基于目标 base 解决冲突,再重新确认 diff 与 CI

* fix(review-feedback-1318): 解决冲突后重新确认 diff 与 CI

* fix(review-feedback-1318): 补充覆盖 Analyzer legacy multi-key Router 的回归测试
This commit is contained in:
mumu
2026-05-16 21:31:28 +08:00
committed by GitHub
parent c169920b0b
commit 4aad40cdd4
16 changed files with 1554 additions and 195 deletions

View File

@@ -96,14 +96,18 @@ GEMINI_API_KEY=
# LLM_OLLAMA_MODELS=qwen3:8b
# 采样温度0.0-2.0,默认 0.70 确定性最高2 随机性最高)
# Kimi K2.6 兼容来源:
# 严格 temperature 兼容来源:
# - Moonshot API / 模型文档https://platform.kimi.ai/docs/guide/kimi-k2-6-quickstart
# - Moonshot 官方模型卡(评测默认 temperature = 1.0https://huggingface.co/moonshotai/Kimi-K2.6
# - OpenAI Chat Completions 规范https://platform.openai.com/docs/api-reference/chat/create
# - LiteLLM OpenAI-Compatible 规范https://docs.litellm.ai/docs/providers/openai_compatible
# 当前仓库运行时依赖约束litellm>=1.80.10,!=1.82.7,!=1.82.8,<2.0.0(显式排除 PyPI 事故版本,见 requirements.txt
# 因此 kimi-k2.6 会自动改用 1.0,避免 API 拒绝请求;其他模型和 fallback 仍使用你配置的 LLM_TEMPERATURE
# Web 设置页 / 桌面端导入不会静默清空或改写 LLM_TEMPERATURE只在真正发请求前按 Kimi 要求临时归一化
# 如果主模型切回非 Kimi原本配置的温度会自动恢复最小回滚方式是回退本次 Kimi 固定温度改动
# 因此 kimi-k2.6 会自动改用 1.0/0.6GPT-5 / o 系列等默认温度模型会省略 temperature,避免 API 拒绝请求。
# 若兼容平台返回明确的参数不支持错误,运行时会在当前请求内修正参数并重试一次;成功策略只做进程内缓存
# top_p、presence_penalty、frequency_penalty、seed 若返回“不支持参数”,同样会触发本次请求级别的修正与重试(不改写 LLM_TEMPERATURE
# 其他模型和 fallback 仍使用你配置的 LLM_TEMPERATURE。
# Web 设置页 / 桌面端导入不会静默清空或改写 LLM_TEMPERATURE只在真正发请求前临时适配模型参数。
# 如果主模型切回普通模型,原本配置的温度会自动恢复;最小回滚方式是回退本次 LLM 参数适配改动。
# LLM_TEMPERATURE=0.7
# --- 多渠道配置(可选,也可在 Web 设置页配置)---

View File

@@ -11,6 +11,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
<!-- 新条目格式:- [类型] 描述(类型取值:新功能/改进/修复/文档/测试/chore-->
<!-- 每条独立一行追加到本段末尾,无需分类标题,合并时冲突最小 -->
- [修复] 抽出 LiteLLM 生成参数适配层,对严格 temperature 模型按请求临时固定或省略参数,避免 GPT-5 / o 系列与 Kimi K2.6 拒绝默认温度请求。
- [改进] LiteLLM 参数错误支持一次请求内自动修正重试,并在成功后进程内缓存策略,降低新模型参数兼容问题的人工配置成本。
- [文档] 补充 Issue #1316 参数自愈改动的外部兼容依据、运行时配置清理边界与回滚证据;并在 `tests/test_system_config_service.py` 增加清理路径下 `LLM_TEMPERATURE` 保持不变的回归用例。
- [文档] 补充严格 temperature 兼容语义的官方来源、运行时依赖约束与 `LLM_TEMPERATURE` 回退/不回写路径说明。
- [修复] 统一 Windows 桌面安装包与自动更新元数据文件名,避免 Release 中出现重复安装包并阻断 `latest.yml` 指向不存在附件。
- [修复] 桌面端启动 WebUI 时为入口页增加 no-cache 响应头和版本化 cache-busting URL避免安装新版后 Electron 继续复用旧 WebUI 缓存。

View File

@@ -199,17 +199,20 @@ LITELLM_MODEL=ollama/qwen3:8b
- 如果当前环境没有任何有效 Agent 模型链路,问股页面会继续按失败语义返回,并直接展示后端真实配置诊断;补齐任一有效模型来源后即可恢复,无需额外执行配置迁移脚本。
- 推荐的新配置方式仍然是显式设置 `LITELLM_MODEL` / `AGENT_LITELLM_MODEL` 或使用 `LLM_CHANNELS`legacy provider keys 目前保留为兼容回退路径,方便旧 `.env`、本地 macOS 开发环境和历史部署平滑继续运行。
### Kimi K2.6 固定 temperature 兼容说明
### 严格 temperature 模型兼容说明
- Moonshot 官方说明 Kimi API 兼容 OpenAI 接口Base URL 使用 `https://api.moonshot.ai/v1`<https://platform.kimi.ai/docs/guide/kimi-k2-6-quickstart>
- LiteLLM 官方要求 OpenAI Compatible 渠道模型名使用 `openai/` 前缀:<https://docs.litellm.ai/docs/providers/openai_compatible>
- Moonshot 官方兼容性文档区分两种固定值:**thinking 模式固定 `1.0`non-thinking 模式固定 `0.6`**;传其它值会被接口拒绝:<https://platform.moonshot.ai/docs/guide/compatibility#parameters-differences-in-request-body>
- OpenAI Chat Completions 规范中 `temperature` 是可选参数;对 GPT-5 / o 系列等只接受默认温度的模型,本项目会在请求层省略 `temperature`,让服务端使用默认值,而不是改写你的 `LLM_TEMPERATURE`<https://platform.openai.com/docs/api-reference/chat/create>
- 当前仓库的运行时依赖约束是 `litellm>=1.80.10,!=1.82.7,!=1.82.8,<2.0.0`(见 `requirements.txt`本次兼容逻辑按该约束回归验证了主分析、大盘复盘、Agent 直连 LiteLLM以及系统设置页的渠道连通性测试。
- 因此本项目会在请求发出前按**实际请求模式**归一化 `kimi-k2.6` 及其 `kimi-k2.6-*` 变体:默认 / thinking 路径使用 `temperature=1.0`;如果你的 LiteLLM YAML 路由别名里显式写了 `litellm_params.extra_body.thinking.type: disabled`(或等价 non-thinking 配置),则自动切到 `temperature=0.6`。你在 `.env` 或 Web 设置里保存的 `LLM_TEMPERATURE` 不会被改写。
- `SystemConfigService` 在 Web 设置保存 / 桌面端 `.env` 导入时只更新你提交的 key不会因为切到 Kimi 静默清空、迁移或重写已有 `LLM_TEMPERATURE`;渠道测试请求里临时使用的 `1.0/0.6` 也不会回写到配置文件
- 非 Kimi 主模型、非 Kimi fallback 以及切回普通模型后的请求,仍继续使用你配置的温度;也就是说旧配置无需迁移,切换模型即可自动恢复原行为
- 如果兼容平台对未收录的新模型返回明确的参数错误(例如 `temperature` 不支持、只能使用默认 `1.0``top_p` 不支持),运行时会对**当前请求**做一次参数修正并重试;只有重试成功后才把该策略缓存在当前进程内。该缓存不会写回 `.env`,服务重启后会重新按配置与适配规则判断
- 对已经产生部分内容的流式响应,系统不会在半截输出后切换参数;仍沿用原有“同模型非流式重试 / fallback 模型”的稳定路径,避免拼接出不一致的回答
- `SystemConfigService` 在 Web 设置保存 / 桌面端 `.env` 导入时只更新你提交的 key不会因为切到严格 temperature 模型静默清空、迁移或重写已有 `LLM_TEMPERATURE`;渠道测试请求里的临时参数策略也不会回写到配置文件。
- 非严格主模型、非严格 fallback 以及切回普通模型后的请求,仍继续使用你配置的温度;也就是说旧配置无需迁移,切换模型即可自动恢复原行为。
- 本仓库兼容性回归覆盖见:`tests/test_llm_channel_config.py``tests/test_market_analyzer_generate_text.py``tests/test_agent_pipeline.py``tests/test_system_config_service.py`
- 最小回滚方式:直接回退本次 Kimi 固定温度相关改动,无需单独迁移已有 `LLM_TEMPERATURE` 配置。
- 最小回滚方式:直接回退本次 LLM 参数适配相关改动,无需单独迁移已有 `LLM_TEMPERATURE` 配置。
### 兼容性与回退复核清单(按 PR 审核口径)
@@ -224,6 +227,12 @@ LITELLM_MODEL=ollama/qwen3:8b
> **致命避坑说明**:如果你启用了 `LLM_CHANNELS`,那么你直接写在外面的 `DEEPSEEK_API_KEY` 或 `OPENAI_API_KEY` 将**全部失效(系统一律无视)**!二者**选其一即可**,千万不要既写了新手模式又写了渠道模式结果产生冲突。
> **Docker 注意**:如果你在 `docker compose environment:` 或 `docker run -e` 中显式传入 `LITELLM_MODEL`、`LLM_CHANNELS`、`LLM_DEEPSEEK_MODELS` 等变量,容器重启后这些环境变量会覆盖 Web 设置页写入的 `.env`,需要同步修改部署配置。
### 兼容依据与回退审计说明(本次 PR 适配说明)
- 官方与运行时兼容依据采用两层第一层为官方接口语义LiteLLM OpenAI-compatible 路由、OpenAI Chat Completions、Moonshot/Kimi 文档与官方模型说明);第二层为本仓库当前运行时语义(`litellm>=1.80.10,!=1.82.7,!=1.82.8,<2.0.0`)下的实际错误归类。
- 本次兼容恢复只使用“本地运行时错误归类 + 单请求修正重试 + 进程内缓存”策略,不写入 `.env`、不做配置迁移,仅在执行路径上动态规避不支持参数(`temperature``top_p``presence_penalty``frequency_penalty``seed`)。若要回退,不需要额外迁移命令,恢复旧值即可。
- 回归与证据:`tests/test_llm_param_recovery.py``tests/test_system_config_service.py``tests/test_llm_channel_config.py``tests/test_system_config_api.py``tests/test_market_analyzer_generate_text.py``tests/test_agent_pipeline.py`;桌面导入与运行时清理回退另有 `test_import_desktop_env_restores_runtime_models_after_cleanup` 直接覆盖。
---
## 方式三YAML 高级配置(适合老手自定义)
@@ -318,6 +327,7 @@ VISION_PROVIDER_PRIORITY=gemini,anthropic,openai
| **我写了好几家的Key为什么死活只有一个生效修改还没用** | 你把 **极简模式** 和 **渠道模式** 混着写了! | 想好一条路走到黑——只要简单就删掉 `LLM_CHANNELS` 开头的;想要丰富备用切换就要全部转投到 `LLM_CHANNELS` 下的编制里。 |
| **错误码报 400 或 401 或 Invalid API Key** | API Key 填错、少复制了一截、账号充值没到账、或者模型名字敲错(极度常见)。 | 1. 检查复制的 Key 前后是否有误填空格。<br> 2. 检查 Base URL 最后是不是少了一个 `/v1`。<br> 3. 检查模型名是否少写了 `openai/` 之类的前缀! |
| **Kimi K2.6 报 `invalid temperature`(可能提示只允许 `1.0` 或 `0.6`** | 该模型按 thinking / non-thinking 模式要求不同固定 temperature旧配置或调用入口可能还在传 `0.7`。 | 升级后系统会对 `kimi-k2.6` 默认 / thinking 请求自动使用 `temperature=1.0`;如果你在 LiteLLM YAML 路由里显式关闭 thinking则自动改用 `0.6`。模型名建议写成 `openai/kimi-k2.6` 并配合 Moonshot / 聚合平台的 OpenAI 兼容 Base URL 与 API Key。非 Kimi fallback 仍会继续使用你配置的 `LLM_TEMPERATURE`。 |
| **GPT-5 / o 系列报 `temperature` 不支持或只允许默认值** | 这类模型只接受服务端默认采样参数,但旧调用入口会显式传 `0.7`。 | 升级后请求层会省略 `temperature`,让服务端使用默认值;`.env` / Web 设置中的 `LLM_TEMPERATURE` 不会被改写,切回普通模型后仍按原值发送。 |
| **转圈转不停,最后报 Timeout / ConnectionRefused 等** | 1. 在国内使用国外原版(像 Google、OpenAI没开代理被墙了。<br>2. 你买的云服务器压根不能出境。 | 非常推荐使用**国内官方**如DeepSeek、阿里或者各种**兼容 OpenAI 的聚合中转接口**。因为中转站把网络问题帮你解决好了。 |
| **Ollama 报 404、`Could not get model info` 或 `api/generate/api/show`** | 误用 `OPENAI_BASE_URL` 配置 Ollama系统会错误拼接 URL | 改用 `OLLAMA_API_BASE=http://localhost:11434` 或渠道模式(`LLM_CHANNELS=ollama` + `LLM_OLLAMA_BASE_URL` |

View File

@@ -192,21 +192,30 @@ LITELLM_MODEL=ollama/qwen3:8b
- If the current environment has no valid Agent model path at all, the ask-stock page still returns a failure and now surfaces the backend's real configuration diagnosis. As soon as you restore any valid model source, the flow recovers without running any migration step.
- The recommended forward path is still to configure `LITELLM_MODEL` / `AGENT_LITELLM_MODEL` explicitly or move to `LLM_CHANNELS`; legacy provider keys remain a compatibility fallback for older `.env` files, local macOS development, and existing deployments.
### Kimi K2.6 Fixed-Temperature Compatibility Notes
### Strict Temperature Model Compatibility Notes
- Moonshot officially documents Kimi as an OpenAI-compatible API, with `https://api.moonshot.ai/v1` as the base URL: <https://platform.kimi.ai/docs/guide/kimi-k2-6-quickstart>
- LiteLLM officially requires the `openai/` prefix for OpenAI-compatible model routing: <https://docs.litellm.ai/docs/providers/openai_compatible>
- Moonshot's compatibility docs distinguish two fixed values: **thinking mode must use `1.0`, while non-thinking mode must use `0.6`**; other values are rejected by the API: <https://platform.moonshot.ai/docs/guide/compatibility#parameters-differences-in-request-body>
- The OpenAI Chat Completions API treats `temperature` as optional. For GPT-5 / o-series style models that only accept the provider default temperature, this project omits `temperature` at request time instead of rewriting your saved `LLM_TEMPERATURE`: <https://platform.openai.com/docs/api-reference/chat/create>
- The current runtime dependency constraint in this repository is `litellm>=1.80.10,!=1.82.7,!=1.82.8,<2.0.0` (see `requirements.txt`); this compatibility fix is regression-covered under that constraint across the main analyzer, market review, direct Agent LiteLLM calls, and the system-settings channel connectivity test path.
- This repository therefore normalizes `kimi-k2.6` and `kimi-k2.6-*` right before dispatch based on the **actual request mode**: default / thinking requests use `temperature=1.0`; if your LiteLLM YAML route alias explicitly sets `litellm_params.extra_body.thinking.type: disabled` (or an equivalent non-thinking override), it automatically switches to `temperature=0.6`. Your saved `LLM_TEMPERATURE` value in `.env` or the Web settings is not rewritten.
- `SystemConfigService` only updates keys that you actually submit when saving from the Web settings page or importing a desktop `.env`; switching to Kimi does not silently clear, migrate, or rewrite an existing `LLM_TEMPERATURE`. The temporary `1.0/0.6` used for Kimi channel tests is request-scoped and is not persisted back into the config file.
- Non-Kimi primary models, non-Kimi fallbacks, and any request after switching away from Kimi still use your configured temperature. Existing configs do not need migration; changing the model restores the original behavior automatically.
- If a compatible platform returns an explicit parameter error for a not-yet-profiled model, such as unsupported `temperature`, default-only `1.0`, or unsupported `top_p`, the runtime repairs the **current request** and retries once. The strategy is cached only in the current process after the retry succeeds; it is never written back to `.env`, and a service restart re-evaluates the configured rules normally.
- For streaming responses that already produced partial content, the runtime does not switch parameters mid-output. It keeps the existing same-model non-stream retry / fallback-model path to avoid stitching inconsistent answers together.
- `SystemConfigService` only updates keys that you actually submit when saving from the Web settings page or importing a desktop `.env`; switching to a strict-temperature model does not silently clear, migrate, or rewrite an existing `LLM_TEMPERATURE`. Temporary request-time parameter strategies are not persisted back into the config file.
- Non-strict primary models, non-strict fallbacks, and any request after switching back to a regular model still use your configured temperature. Existing configs do not need migration; changing the model restores the original behavior automatically.
- Repository-side compatibility coverage lives in `tests/test_llm_channel_config.py`, `tests/test_market_analyzer_generate_text.py`, `tests/test_agent_pipeline.py`, and `tests/test_system_config_service.py`.
- Minimal rollback: revert only the Kimi fixed-temperature change set; no separate `LLM_TEMPERATURE` migration is required.
- Minimal rollback: revert only the LLM generation-parameter adaptation change set; no separate `LLM_TEMPERATURE` migration is required.
> **Critical Warning**: If you enable `LLM_CHANNELS`, any standard `DEEPSEEK_API_KEY` or `OPENAI_API_KEY` declared independently will be **completely ignored**. **Use only one mode** to prevent configuration conflicts.
> **Docker note**: If `LITELLM_MODEL`, `LLM_CHANNELS`, `LLM_DEEPSEEK_MODELS`, or related variables are explicitly passed through `docker compose environment:` or `docker run -e`, they will override the `.env` written by the Web settings page after a container restart. Update the deployment environment at the same time.
### Compatibility evidence and rollback audit notes (for this recovery change)
- Compatibility is validated in two layers: first-party provider/API contract references (LiteLLM OpenAI-compatible routing, OpenAI Chat Completions, Moonshot/Kimi docs and model notes), and second the current runtime implementation in this repository under `litellm>=1.80.10,!=1.82.7,!=1.82.8,<2.0.0`.
- This recovery path is runtime-only and intentionally local: exception classification + one in-request repair retry + in-process cache. It does not rewrite `.env`, migrate saved config keys, or alter legacy values; it only omits/adjusts request parameters (`temperature`, `top_p`, `presence_penalty`, `frequency_penalty`, `seed`) for the current call. Rolling back requires no migration; restore previous settings and model/provider selection.
- Regression evidence for this path is in `tests/test_llm_param_recovery.py`, `tests/test_system_config_service.py`, `tests/test_llm_channel_config.py`, `tests/test_system_config_api.py`, `tests/test_market_analyzer_generate_text.py`, `tests/test_agent_pipeline.py`; desktop backup import restore is directly covered by `test_import_desktop_env_restores_runtime_models_after_cleanup`.
---
## Method 3: Advanced YAML Config (Expert Setup)
@@ -285,6 +294,7 @@ Afraid you got the config wrong? Type the following commands in your terminal to
| **I added multiple provider Keys, why is only one working?** | You mixed the **Simple Mode** and **Channels Mode**! | Choose one path. For simple setups, delete anything starting with `LLM_CHANNELS`. To use multi-model fallbacks, migrate all your Keys into the `LLM_CHANNELS` setup. |
| **Returns 400, 401, or Invalid API Key** | The API Key is wrong, copied incompletely, account lacks credits, or you mistyped the model name (extremely common). | 1. Ensure there are no spaces at the start/end of your Key.<br> 2. Ensure your Base URL ends with `/v1`.<br> 3. Check if you forgot the `openai/` prefix on the model name! |
| **Kimi K2.6 returns `invalid temperature` (it may say only `1.0` or `0.6` is allowed)** | The model requires different fixed temperatures for thinking vs non-thinking mode, while older config or call paths may still pass `0.7`. | After this fix, default / thinking `kimi-k2.6` requests automatically use `temperature=1.0`; if you explicitly disable thinking in a LiteLLM YAML route, the request automatically uses `0.6` instead. Prefer `openai/kimi-k2.6` with your Moonshot or relay OpenAI-compatible Base URL and API key. Non-Kimi fallbacks still keep your configured `LLM_TEMPERATURE`. |
| **GPT-5 / o-series returns that `temperature` is unsupported or only the default is allowed** | These models only accept the provider default sampling parameters, while older call paths may still send `0.7`. | The request layer now omits `temperature` so the provider default is used. Your `.env` / Web `LLM_TEMPERATURE` is not rewritten, and regular models keep using it after you switch back. |
| **Spins endlessly, eventually hits Timeout/ConnectionRefused** | You are using restricted APIs (like Google/OpenAI) in a blocked region without a proxy, or your cloud server lacks external internet access. | Highly recommend using **official regional APIs** (like DeepSeek) or **OpenAI-compatible relay platforms**. Third-party platforms bypass these network constraints. |
| **Ollama returns 404, `Could not get model info`, or `api/generate/api/show`** | Using `OPENAI_BASE_URL` for Ollama makes the system concatenate URLs incorrectly | Use `OLLAMA_API_BASE=http://localhost:11434` or channel mode (`LLM_CHANNELS=ollama` + `LLM_OLLAMA_BASE_URL`) instead |

View File

@@ -23,8 +23,9 @@ from src.config import (
get_configured_llm_models,
get_effective_agent_models_to_try,
get_effective_agent_primary_model,
normalize_litellm_temperature,
)
from src.llm.errors import call_litellm_with_param_recovery
from src.llm.generation_params import apply_litellm_generation_params
logger = logging.getLogger(__name__)
@@ -156,6 +157,7 @@ class LLMToolAdapter:
config = config or get_config()
self._config = config
self._router = None # litellm Router (multi-key primary model)
self._legacy_router_model_list: List[Dict[str, Any]] = []
self._litellm_available = False
self._register_custom_model_pricing()
self._init_litellm()
@@ -186,6 +188,7 @@ class LLMToolAdapter:
def _init_litellm(self) -> None:
"""Initialize litellm Router from channels / YAML / legacy keys."""
config = self._config
self._legacy_router_model_list = []
litellm_model = get_effective_agent_primary_model(config)
if not litellm_model:
logger.warning("Agent LLM: no effective primary model configured")
@@ -232,6 +235,7 @@ class LLMToolAdapter:
}
for k in keys
]
self._legacy_router_model_list = legacy_model_list
self._router = Router(
model_list=legacy_model_list,
routing_strategy="simple-shuffle",
@@ -404,12 +408,6 @@ class LLMToolAdapter:
call_kwargs: Dict[str, Any] = {
"model": model,
"messages": openai_messages,
"temperature": normalize_litellm_temperature(
model,
self._get_temperature() if temperature is None else temperature,
model_list=self._config.llm_model_list,
request_overrides={"extra_body": extra} if extra else None,
),
}
if max_tokens is not None:
call_kwargs["max_tokens"] = max_tokens
@@ -426,21 +424,53 @@ class LLMToolAdapter:
use_channel_router = self._has_channel_config()
_router_model_names = set(get_configured_llm_models(self._config.llm_model_list))
agent_primary_model = get_effective_agent_primary_model(self._config)
if use_channel_router and self._router and model in _router_model_names:
# Channel / YAML path: Router manages all models in its model_list
response = self._router.completion(**call_kwargs)
elif self._router and model == agent_primary_model and not use_channel_router:
# Legacy path: Router for primary model multi-key
response = self._router.completion(**call_kwargs)
else:
# Legacy/direct-env path: direct call (also handles direct-env
# providers like groq/ or bedrock/ that are not in the Router
# model_list even when channel mode is active)
uses_router = (
bool(use_channel_router and self._router and model in _router_model_names)
or bool(self._router and model == agent_primary_model and not use_channel_router)
)
recovery_model_list = self._config.llm_model_list
if self._router and model == agent_primary_model and not use_channel_router:
recovery_model_list = self._legacy_router_model_list or self._config.llm_model_list
if not uses_router:
keys = get_api_keys_for_model(model, self._config)
if keys:
call_kwargs["api_key"] = keys[0]
call_kwargs.update(extra_litellm_params(model, self._config))
response = litellm.completion(**call_kwargs)
call_kwargs = apply_litellm_generation_params(
call_kwargs,
model,
self._get_temperature() if temperature is None else temperature,
model_list=recovery_model_list,
)
if use_channel_router and self._router and model in _router_model_names:
# Channel / YAML path: Router manages all models in its model_list
response = call_litellm_with_param_recovery(
lambda kwargs: self._router.completion(**kwargs),
model=model,
call_kwargs=call_kwargs,
model_list=recovery_model_list,
logger=logger,
)
elif self._router and model == agent_primary_model and not use_channel_router:
# Legacy path: Router for primary model multi-key
response = call_litellm_with_param_recovery(
lambda kwargs: self._router.completion(**kwargs),
model=model,
call_kwargs=call_kwargs,
model_list=recovery_model_list,
logger=logger,
)
else:
# Legacy/direct-env path: direct call (also handles direct-env
# providers like groq/ or bedrock/ that are not in the Router
# model_list even when channel mode is active)
response = call_litellm_with_param_recovery(
lambda kwargs: litellm.completion(**kwargs),
model=model,
call_kwargs=call_kwargs,
model_list=self._config.llm_model_list,
logger=logger,
)
return self._parse_litellm_response(response, model)

View File

@@ -30,9 +30,10 @@ from src.config import (
get_api_keys_for_model,
get_config,
get_configured_llm_models,
normalize_litellm_temperature,
resolve_news_window_days,
)
from src.llm.generation_params import apply_litellm_generation_params
from src.llm.errors import call_litellm_with_param_recovery
from src.storage import persist_llm_usage
from src.data.stock_mapping import STOCK_NAME_MAP
from src.report_language import (
@@ -1810,6 +1811,7 @@ class GeminiAnalyzer:
self._use_legacy_default_prompt_override = use_legacy_default_prompt
self._resolved_prompt_state: Optional[Dict[str, Any]] = None
self._router = None
self._legacy_router_model_list: List[Dict[str, Any]] = []
self._litellm_available = False
self._init_litellm()
if not self._litellm_available:
@@ -1908,6 +1910,47 @@ class GeminiAnalyzer:
e.get('model_name', '').startswith('__legacy_') for e in config.llm_model_list
)
@staticmethod
def _legacy_router_provider_alias(model: str) -> str:
provider = model.split("/", 1)[0] if "/" in model else "openai"
return f"__legacy_{provider}__"
@staticmethod
def _build_legacy_router_model_list_from_config(
model: str,
model_list: List[Dict[str, Any]],
) -> List[Dict[str, Any]]:
"""Build legacy-router candidates from configured legacy llm_model_list entries."""
if not model:
return []
target_model = model
target_legacy_alias = GeminiAnalyzer._legacy_router_provider_alias(model)
legacy_entries: List[Dict[str, Any]] = []
for entry in model_list or []:
if not isinstance(entry, dict):
continue
model_name = str(entry.get("model_name") or "").strip()
if model_name != target_legacy_alias:
continue
params = entry.get("litellm_params")
if not isinstance(params, dict):
continue
api_key = str(params.get("api_key") or "").strip()
if not api_key or len(api_key) < 8:
continue
deployed_params = dict(params)
deployed_params["model"] = target_model
deployed_params["api_key"] = api_key
legacy_entries.append({
"model_name": target_model,
"litellm_params": deployed_params,
})
return legacy_entries
def _init_litellm(self) -> None:
"""Initialize litellm Router from channels / YAML / legacy keys."""
config = self._get_runtime_config()
@@ -1921,27 +1964,34 @@ class GeminiAnalyzer:
# --- Channel / YAML path: build Router from pre-built model_list ---
if self._has_channel_config(config):
model_list = config.llm_model_list
self._router = Router(
model_list=model_list,
routing_strategy="simple-shuffle",
num_retries=2,
)
unique_models = list(dict.fromkeys(
e['litellm_params']['model'] for e in model_list
))
logger.info(
f"Analyzer LLM: Router initialized from channels/YAML — "
f"{len(model_list)} deployment(s), models: {unique_models}"
)
return
try:
self._router = Router(
model_list=model_list,
routing_strategy="simple-shuffle",
num_retries=2,
)
except TypeError:
logger.debug("Analyzer LLM: Router constructor signature not compatible; fallback to direct mode")
self._router = None
else:
unique_models = list(dict.fromkeys(
e['litellm_params']['model'] for e in model_list
))
logger.info(
f"Analyzer LLM: Router initialized from channels/YAML — "
f"{len(model_list)} deployment(s), models: {unique_models}"
)
return
# --- Legacy path: build Router for multi-key, or use single key ---
keys = get_api_keys_for_model(litellm_model, config)
if len(keys) > 1:
# Build legacy Router for primary model multi-key load-balancing
legacy_model_list = self._build_legacy_router_model_list_from_config(
litellm_model,
config.llm_model_list,
)
if len(legacy_model_list) <= 1 and keys:
extra_params = extra_litellm_params(litellm_model, config)
legacy_model_list = [
configured_model_list = [
{
"model_name": litellm_model,
"litellm_params": {
@@ -1952,16 +2002,30 @@ class GeminiAnalyzer:
}
for k in keys
]
self._router = Router(
model_list=legacy_model_list,
routing_strategy="simple-shuffle",
num_retries=2,
)
logger.info(
f"Analyzer LLM: Legacy Router initialized with {len(keys)} keys "
f"for {litellm_model}"
)
elif keys:
if not legacy_model_list:
legacy_model_list = configured_model_list
elif len(legacy_model_list) < len(configured_model_list):
legacy_model_list = configured_model_list
if len(legacy_model_list) > 1:
self._legacy_router_model_list = legacy_model_list
try:
self._router = Router(
model_list=legacy_model_list,
routing_strategy="simple-shuffle",
num_retries=2,
)
except TypeError:
logger.debug("Analyzer LLM: Legacy Router constructor signature not compatible; using legacy model_list fallback")
self._router = None
else:
logger.info(
f"Analyzer LLM: Legacy Router initialized with {len(legacy_model_list)} keys "
f"for {litellm_model}"
)
return
if keys:
logger.info(f"Analyzer LLM: litellm initialized (model={litellm_model})")
else:
logger.info(
@@ -2206,6 +2270,11 @@ class GeminiAnalyzer:
effective_system_prompt = system_prompt or self.TEXT_SYSTEM_PROMPT
router_model_names = set(get_configured_llm_models(config.llm_model_list))
for model in models_to_try:
recovery_model_list = config.llm_model_list
legacy_router_model_list = getattr(self, "_legacy_router_model_list", None) or []
if legacy_router_model_list and model == config.litellm_model and not use_channel_router:
recovery_model_list = legacy_router_model_list
try:
model_short = model.split("/")[-1] if "/" in model else model
extra = get_thinking_extra_body(model_short)
@@ -2215,28 +2284,50 @@ class GeminiAnalyzer:
{"role": "system", "content": effective_system_prompt},
{"role": "user", "content": prompt},
],
"temperature": normalize_litellm_temperature(
model,
requested_temperature,
model_list=config.llm_model_list,
request_overrides={"extra_body": extra} if extra else None,
),
"max_tokens": max_tokens,
}
if extra:
call_kwargs["extra_body"] = extra
uses_router = (
(use_channel_router and self._router and model in router_model_names)
or (self._router and model == config.litellm_model and not use_channel_router)
)
if not uses_router:
try:
keys = get_api_keys_for_model(model, config)
except AttributeError:
keys = []
if keys:
call_kwargs["api_key"] = keys[0]
try:
call_kwargs.update(extra_litellm_params(model, config))
except AttributeError:
pass
call_kwargs = apply_litellm_generation_params(
call_kwargs,
model,
requested_temperature,
model_list=recovery_model_list,
)
_stream_text: Optional[str] = None
_stream_usage: Dict[str, Any] = {}
if stream:
try:
stream_response = self._dispatch_litellm_completion(
model,
{**call_kwargs, "stream": True},
config=config,
use_channel_router=use_channel_router,
router_model_names=router_model_names,
stream_response = call_litellm_with_param_recovery(
lambda kwargs: self._dispatch_litellm_completion(
model,
kwargs,
config=config,
use_channel_router=use_channel_router,
router_model_names=router_model_names,
),
model=model,
call_kwargs={**call_kwargs, "stream": True},
model_list=recovery_model_list,
cache_recovery=False,
logger=logger,
)
_stream_text, _stream_usage = self._consume_litellm_stream(
stream_response,
@@ -2272,12 +2363,18 @@ class GeminiAnalyzer:
response_validator(_stream_text)
return _stream_text, model, _stream_usage
response = self._dispatch_litellm_completion(
model,
call_kwargs,
config=config,
use_channel_router=use_channel_router,
router_model_names=router_model_names,
response = call_litellm_with_param_recovery(
lambda kwargs: self._dispatch_litellm_completion(
model,
kwargs,
config=config,
use_channel_router=use_channel_router,
router_model_names=router_model_names,
),
model=model,
call_kwargs=call_kwargs,
model_list=recovery_model_list,
logger=logger,
)
content = self._extract_completion_text(response)

View File

@@ -31,6 +31,7 @@ from src.notification_noise import (
parse_notification_quiet_hours,
validate_notification_timezone,
)
from src.llm import generation_params as llm_generation_params
logger = logging.getLogger(__name__)
@@ -61,20 +62,6 @@ _FALSEY_ENV_VALUES = {"0", "false", "no", "off"}
# These are compatibility examples; actual availability should be validated by Anspire console/model entitlement.
ANSPIRE_LLM_BASE_URL_DEFAULT = "https://open-gateway.anspire.cn/v6"
ANSPIRE_LLM_MODEL_DEFAULT = "Doubao-Seed-2.0-lite"
# Kimi K2.6 is consumed through Moonshot's OpenAI-compatible API in this
# repository. Official references:
# - https://platform.kimi.ai/docs/guide/kimi-k2-6-quickstart
# - https://platform.moonshot.ai/docs/guide/compatibility#parameters-differences-in-request-body
# - https://huggingface.co/moonshotai/Kimi-K2.6
# - https://docs.litellm.ai/docs/providers/openai_compatible
# Only the strict Kimi K2.6 family is normalized here; other models and
# fallbacks continue using the configured runtime temperature.
_FIXED_TEMPERATURE_LITELLM_MODELS: Dict[str, Dict[str, float]] = {
"kimi-k2.6": {
"thinking": 1.0,
"non_thinking": 0.6,
},
}
def _has_ntfy_topic_endpoint(value: Optional[str]) -> bool:
@@ -350,71 +337,7 @@ def resolve_litellm_wire_model(
model_list: Optional[List[Dict[str, Any]]] = None,
) -> str:
"""Resolve a router alias to its underlying LiteLLM wire model."""
normalized_model = (model or "").strip()
if not normalized_model or not model_list:
return normalized_model
model_entry = _resolve_litellm_model_list_entry(normalized_model, model_list)
if not model_entry:
return normalized_model
params = model_entry.get("litellm_params", {}) or {}
wire_model = str(params.get("model") or "").strip()
if wire_model:
return wire_model
return normalized_model
def _resolve_litellm_model_list_entry(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
) -> Optional[Dict[str, Any]]:
"""Return the Router model_list entry matching the configured alias."""
normalized_model = (model or "").strip()
if not normalized_model or not model_list:
return None
for entry in model_list:
model_name = str(entry.get("model_name") or "").strip()
if not model_name:
params = entry.get("litellm_params", {}) or {}
model_name = str(params.get("model") or "").strip()
if model_name == normalized_model:
return entry
return None
def _extract_thinking_config(payload: Optional[Dict[str, Any]]) -> Any:
"""Extract a thinking-mode flag from LiteLLM-style request kwargs."""
if not isinstance(payload, dict):
return None
extra_body = payload.get("extra_body")
if isinstance(extra_body, dict) and "thinking" in extra_body:
return extra_body.get("thinking")
if "thinking" in payload:
return payload.get("thinking")
return None
def _parse_thinking_enabled(value: Any) -> Optional[bool]:
"""Parse thinking-mode config into True/False/unknown."""
if value is None:
return None
if isinstance(value, bool):
return value
if isinstance(value, str):
normalized = value.strip().lower()
if normalized in {"enabled", "enable", "true", "1", "on", "thinking"}:
return True
if normalized in {"disabled", "disable", "false", "0", "off", "none", "non-thinking", "non_thinking"}:
return False
return None
if isinstance(value, dict):
if "enabled" in value:
return _parse_thinking_enabled(value.get("enabled"))
if "type" in value:
return _parse_thinking_enabled(value.get("type"))
return None
return llm_generation_params.resolve_litellm_wire_model(model, model_list)
def resolve_litellm_thinking_enabled(
@@ -423,19 +346,11 @@ def resolve_litellm_thinking_enabled(
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[bool]:
"""Resolve whether the outgoing LiteLLM request explicitly enables thinking."""
thinking_config = None
model_entry = _resolve_litellm_model_list_entry(model, model_list)
if model_entry:
thinking_config = _extract_thinking_config(model_entry)
entry_params = model_entry.get("litellm_params", {}) or {}
entry_thinking_config = _extract_thinking_config(entry_params)
if entry_thinking_config is not None:
thinking_config = entry_thinking_config
override_thinking_config = _extract_thinking_config(request_overrides)
if override_thinking_config is not None:
thinking_config = override_thinking_config
return _parse_thinking_enabled(thinking_config)
return llm_generation_params.resolve_litellm_thinking_enabled(
model,
model_list=model_list,
request_overrides=request_overrides,
)
def get_fixed_litellm_temperature(
@@ -444,24 +359,11 @@ def get_fixed_litellm_temperature(
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[float]:
"""Return a provider-mandated temperature for known strict models."""
normalized_model = resolve_litellm_wire_model(model, model_list).lower()
if not normalized_model:
return None
thinking_enabled = resolve_litellm_thinking_enabled(
return llm_generation_params.get_fixed_litellm_temperature(
model,
model_list=model_list,
request_overrides=request_overrides,
)
model_parts = [part for part in re.split(r"[/:\s]+", normalized_model) if part]
for model_name, temperatures in _FIXED_TEMPERATURE_LITELLM_MODELS.items():
if any(part == model_name or part.startswith(f"{model_name}-") for part in model_parts):
if thinking_enabled is False and temperatures.get("non_thinking") is not None:
return temperatures["non_thinking"]
if temperatures.get("thinking") is not None:
return temperatures["thinking"]
if temperatures.get("non_thinking") is not None:
return temperatures["non_thinking"]
return None
def normalize_litellm_temperature(
@@ -473,16 +375,13 @@ def normalize_litellm_temperature(
request_overrides: Optional[Dict[str, Any]] = None,
) -> float:
"""Normalize temperature before sending a LiteLLM request."""
fixed_temperature = get_fixed_litellm_temperature(
return llm_generation_params.normalize_litellm_temperature(
model,
temperature,
default=default,
model_list=model_list,
request_overrides=request_overrides,
)
if fixed_temperature is not None:
return fixed_temperature
if temperature is None:
return default
return float(temperature)
def resolve_unified_llm_temperature(model: str) -> float:

2
src/llm/__init__.py Normal file
View File

@@ -0,0 +1,2 @@
"""LLM runtime helpers."""

151
src/llm/errors.py Normal file
View File

@@ -0,0 +1,151 @@
# -*- coding: utf-8 -*-
"""LiteLLM error classification and one-shot parameter recovery."""
from __future__ import annotations
import re
from typing import Any, Callable, Dict, List, Optional
from src.llm.generation_params import (
GenerationParamRecovery,
apply_litellm_param_recovery,
remember_litellm_generation_param_recovery,
)
_UNSUPPORTED_PARAM_MARKERS = (
"unsupported",
"not supported",
"unrecognized",
"unknown parameter",
"not allowed",
"invalid parameter",
"does not support",
)
_TEMPERATURE_VALUE_PATTERN = r"-?\d+(?:\.\d+)?"
_ALLOWED_TEMPERATURE_PATTERNS = (
re.compile(
rf"\bonly\s+(?:the\s+)?(?:default\s+)?(?:temperature\s+)?(?:value\s+)?[\(`'\"]*(?P<value>{_TEMPERATURE_VALUE_PATTERN})(?!\w)"
),
re.compile(
rf"\bdefault(?:\s+temperature)?(?:\s+value)?\s*(?:is|=|:)\s*[\(`'\"]*(?P<value>{_TEMPERATURE_VALUE_PATTERN})(?!\w)"
),
)
def _collect_error_text(value: Any, seen: Optional[set] = None) -> List[str]:
if seen is None:
seen = set()
if value is None:
return []
value_id = id(value)
if value_id in seen:
return []
seen.add(value_id)
chunks = [str(value)]
if isinstance(value, BaseException):
chunks.extend(_collect_error_text(getattr(value, "args", None), seen))
if isinstance(value, dict):
for item in value.values():
chunks.extend(_collect_error_text(item, seen))
elif isinstance(value, (list, tuple, set)):
for item in value:
chunks.extend(_collect_error_text(item, seen))
else:
for attr in ("message", "body", "response", "llm_provider", "param"):
if hasattr(value, attr):
chunks.extend(_collect_error_text(getattr(value, attr), seen))
return chunks
def _normalized_error_text(error: BaseException) -> str:
return " ".join(chunk for chunk in _collect_error_text(error) if chunk).lower()
def _parse_allowed_temperature(text: str) -> Optional[float]:
for segment in re.split(r"(?<!\d)\.(?!\d)|[!?;\n]+", text):
if "only" not in segment:
continue
for pattern in _ALLOWED_TEMPERATURE_PATTERNS:
match = pattern.search(segment[segment.find("only") :])
if match is None:
continue
value = float(match.group("value"))
if 0 <= value <= 2:
return value
return None
def classify_litellm_generation_param_error(
error: BaseException,
) -> Optional[GenerationParamRecovery]:
"""Classify explicit provider parameter errors into a safe one-shot recovery."""
text = _normalized_error_text(error)
if not text:
return None
if "temperature" in text:
allowed_temperature = _parse_allowed_temperature(text)
if allowed_temperature is not None:
return GenerationParamRecovery(
set_params={"temperature": allowed_temperature},
reason="temperature_default_only",
)
if "only" in text and "default" in text:
return GenerationParamRecovery(
omit_params=("temperature",),
reason="temperature_default_only",
)
if any(marker in text for marker in _UNSUPPORTED_PARAM_MARKERS):
return GenerationParamRecovery(
omit_params=("temperature",),
reason="temperature_unsupported",
)
for param in ("top_p", "presence_penalty", "frequency_penalty", "seed"):
if param in text and any(marker in text for marker in _UNSUPPORTED_PARAM_MARKERS):
return GenerationParamRecovery(
omit_params=(param,),
reason=f"{param}_unsupported",
)
return None
def call_litellm_with_param_recovery(
call: Callable[[Dict[str, Any]], Any],
*,
model: str,
call_kwargs: Dict[str, Any],
model_list: Optional[List[Dict[str, Any]]] = None,
cache_recovery: bool = True,
logger: Optional[Any] = None,
log_label: str = "[LiteLLM]",
) -> Any:
"""Call LiteLLM once, then retry once for explicit generation-parameter errors."""
effective_kwargs = dict(call_kwargs)
try:
return call(effective_kwargs)
except Exception as exc:
recovery = classify_litellm_generation_param_error(exc)
if recovery is None:
raise
retry_kwargs = apply_litellm_param_recovery(effective_kwargs, recovery)
if retry_kwargs == effective_kwargs:
raise
if logger is not None:
logger.warning(
"%s %s generation parameter rejected (%s), retrying once with request-scoped recovery",
log_label,
model,
recovery.reason,
)
response = call(retry_kwargs)
if cache_recovery:
remember_litellm_generation_param_recovery(
model,
recovery,
model_list=model_list,
request_overrides=retry_kwargs,
)
return response

View File

@@ -0,0 +1,439 @@
# -*- coding: utf-8 -*-
"""LiteLLM generation-parameter compatibility helpers."""
from __future__ import annotations
import json
import re
from dataclasses import dataclass, field
from typing import Any, Dict, List, Mapping, Optional, Tuple
# Kimi K2.6 is consumed through Moonshot's OpenAI-compatible API in this
# repository. Official references:
# - https://platform.kimi.ai/docs/guide/kimi-k2-6-quickstart
# - https://platform.moonshot.ai/docs/guide/compatibility#parameters-differences-in-request-body
# - https://huggingface.co/moonshotai/Kimi-K2.6
# - https://docs.litellm.ai/docs/providers/openai_compatible
_FIXED_TEMPERATURE_LITELLM_MODELS: Dict[str, Dict[str, float]] = {
"kimi-k2.6": {
"thinking": 1.0,
"non_thinking": 0.6,
},
}
@dataclass(frozen=True)
class TemperatureDirective:
"""Request-scoped temperature strategy for one LiteLLM model call."""
temperature: Optional[float] = None
omit_temperature: bool = False
reason: str = ""
@dataclass(frozen=True)
class GenerationParamRecovery:
"""A learned request-parameter repair for a LiteLLM model call."""
omit_params: Tuple[str, ...] = ()
set_params: Mapping[str, Any] = field(default_factory=dict)
reason: str = ""
_GENERATION_PARAM_RECOVERY_CACHE: Dict[str, GenerationParamRecovery] = {}
_LITELLM_ENDPOINT_PARAM_KEYS = (
"api_base",
"base_url",
"api_version",
"api_type",
"azure_endpoint",
"azure_deployment",
"deployment_id",
"custom_llm_provider",
"organization",
"region_name",
"aws_region_name",
"vertex_project",
"vertex_location",
"extra_headers",
"headers",
"default_headers",
)
_LITELLM_ROUTING_PARAM_KEYS = ("model", *_LITELLM_ENDPOINT_PARAM_KEYS)
_SECRET_CACHE_FIELD_NAMES = {
"api_key",
"authorization",
"proxy_authorization",
"cookie",
"set-cookie",
"x-api-key",
"api-key",
"openai-api-key",
}
def _resolve_litellm_model_list_entry(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
) -> Optional[Dict[str, Any]]:
"""Return the Router model_list entry matching the configured alias."""
entries = _resolve_litellm_model_list_entries(model, model_list)
return entries[0] if entries else None
def _resolve_litellm_model_list_entries(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
) -> List[Dict[str, Any]]:
"""Return Router model_list entries matching the configured alias."""
normalized_model = (model or "").strip()
if not normalized_model or not model_list:
return []
entries: List[Dict[str, Any]] = []
for entry in model_list:
model_name = str(entry.get("model_name") or "").strip()
if not model_name:
params = entry.get("litellm_params", {}) or {}
model_name = str(params.get("model") or "").strip()
if model_name == normalized_model:
entries.append(entry)
return entries
def resolve_litellm_wire_model(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
) -> str:
"""Resolve a router alias to its underlying LiteLLM wire model."""
normalized_model = (model or "").strip()
if not normalized_model or not model_list:
return normalized_model
model_entry = _resolve_litellm_model_list_entry(normalized_model, model_list)
if not model_entry:
return normalized_model
params = model_entry.get("litellm_params", {}) or {}
wire_model = str(params.get("model") or "").strip()
if wire_model:
return wire_model
return normalized_model
def _extract_thinking_config(payload: Optional[Dict[str, Any]]) -> Any:
"""Extract a thinking-mode flag from LiteLLM-style request kwargs."""
if not isinstance(payload, dict):
return None
extra_body = payload.get("extra_body")
if isinstance(extra_body, dict) and "thinking" in extra_body:
return extra_body.get("thinking")
if "thinking" in payload:
return payload.get("thinking")
return None
def _parse_thinking_enabled(value: Any) -> Optional[bool]:
"""Parse thinking-mode config into True/False/unknown."""
if value is None:
return None
if isinstance(value, bool):
return value
if isinstance(value, str):
normalized = value.strip().lower()
if normalized in {"enabled", "enable", "true", "1", "on", "thinking"}:
return True
if normalized in {"disabled", "disable", "false", "0", "off", "none", "non-thinking", "non_thinking"}:
return False
return None
if isinstance(value, dict):
if "enabled" in value:
return _parse_thinking_enabled(value.get("enabled"))
if "type" in value:
return _parse_thinking_enabled(value.get("type"))
return None
def resolve_litellm_thinking_enabled(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[bool]:
"""Resolve whether the outgoing LiteLLM request explicitly enables thinking."""
thinking_config = None
model_entry = _resolve_litellm_model_list_entry(model, model_list)
if model_entry:
thinking_config = _extract_thinking_config(model_entry)
entry_params = model_entry.get("litellm_params", {}) or {}
entry_thinking_config = _extract_thinking_config(entry_params)
if entry_thinking_config is not None:
thinking_config = entry_thinking_config
override_thinking_config = _extract_thinking_config(request_overrides)
if override_thinking_config is not None:
thinking_config = override_thinking_config
return _parse_thinking_enabled(thinking_config)
def _model_parts(model: str) -> List[str]:
return [part for part in re.split(r"[/:\s]+", (model or "").lower()) if part]
def _matches_model_family(model: str, family: str) -> bool:
return any(part == family or part.startswith(f"{family}-") for part in _model_parts(model))
def _should_omit_litellm_temperature(model: str) -> bool:
"""Return whether a model family should rely on the provider default temperature."""
return any(
part.startswith(("gpt-5", "gpt5"))
or part in {"o1", "o3", "o4"}
or part.startswith(("o1-", "o3-", "o4-"))
for part in _model_parts(model)
)
def get_fixed_litellm_temperature(
model: str,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[float]:
"""Return a provider-mandated temperature for known strict models."""
normalized_model = resolve_litellm_wire_model(model, model_list).lower()
if not normalized_model:
return None
thinking_enabled = resolve_litellm_thinking_enabled(
model,
model_list=model_list,
request_overrides=request_overrides,
)
for model_name, temperatures in _FIXED_TEMPERATURE_LITELLM_MODELS.items():
if _matches_model_family(normalized_model, model_name):
if thinking_enabled is False and temperatures.get("non_thinking") is not None:
return temperatures["non_thinking"]
if temperatures.get("thinking") is not None:
return temperatures["thinking"]
if temperatures.get("non_thinking") is not None:
return temperatures["non_thinking"]
return None
def resolve_litellm_temperature_directive(
model: str,
*,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> TemperatureDirective:
"""Resolve the request-scoped temperature directive for a LiteLLM model."""
fixed_temperature = get_fixed_litellm_temperature(
model,
model_list=model_list,
request_overrides=request_overrides,
)
if fixed_temperature is not None:
return TemperatureDirective(
temperature=fixed_temperature,
reason="fixed_model_temperature",
)
wire_model = resolve_litellm_wire_model(model, model_list)
if _should_omit_litellm_temperature(wire_model):
return TemperatureDirective(
omit_temperature=True,
reason="provider_default_temperature",
)
return TemperatureDirective()
def normalize_litellm_temperature(
model: str,
temperature: Optional[float],
*,
default: float = 0.7,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> float:
"""Return the legacy float temperature normalization for callers that need it."""
fixed_temperature = get_fixed_litellm_temperature(
model,
model_list=model_list,
request_overrides=request_overrides,
)
if fixed_temperature is not None:
return fixed_temperature
if temperature is None:
return default
return float(temperature)
def _redact_recovery_cache_value(param_name: str, value: Any) -> Any:
if param_name.strip().lower() in _SECRET_CACHE_FIELD_NAMES:
return "<set>" if value else "<empty>"
if isinstance(value, Mapping):
return {
str(key): _redact_recovery_cache_value(str(key), nested_value)
for key, nested_value in sorted(value.items(), key=lambda item: str(item[0]))
}
if isinstance(value, (list, tuple)):
return [_redact_recovery_cache_value(param_name, item) for item in value]
return value
def _stable_recovery_cache_json(value: Mapping[str, Any]) -> str:
redacted = {
key: _redact_recovery_cache_value(key, val)
for key, val in sorted(value.items())
}
return json.dumps(redacted, sort_keys=True, separators=(",", ":"), default=str)
def _filter_litellm_routing_params(params: Mapping[str, Any]) -> Dict[str, Any]:
return {
key: params[key]
for key in _LITELLM_ROUTING_PARAM_KEYS
if key in params and params[key] not in (None, "")
}
def _request_endpoint_cache_scope(request_overrides: Optional[Dict[str, Any]]) -> Optional[str]:
if not isinstance(request_overrides, Mapping):
return None
routing_params = _filter_litellm_routing_params(request_overrides)
if not any(key in routing_params for key in _LITELLM_ENDPOINT_PARAM_KEYS):
return None
return _stable_recovery_cache_json(routing_params)
def _model_list_endpoint_cache_scope(
model: str,
model_list: Optional[List[Dict[str, Any]]],
) -> Optional[str]:
entries = _resolve_litellm_model_list_entries(model, model_list)
if not entries:
return "default"
fingerprints = set()
for entry in entries:
params = entry.get("litellm_params", {}) or {}
if not isinstance(params, Mapping):
params = {}
routing_params = _filter_litellm_routing_params(params)
if not routing_params:
routing_params = {"model": str(entry.get("model_name") or model).strip()}
fingerprints.add(_stable_recovery_cache_json(routing_params))
if len(fingerprints) != 1:
return None
return next(iter(fingerprints))
def _recovery_cache_key(
model: str,
*,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[str]:
wire_model = resolve_litellm_wire_model(model, model_list).strip().lower()
thinking_enabled = resolve_litellm_thinking_enabled(
model,
model_list=model_list,
request_overrides=request_overrides,
)
endpoint_scope = _request_endpoint_cache_scope(request_overrides)
if endpoint_scope is None:
endpoint_scope = _model_list_endpoint_cache_scope(model, model_list)
if endpoint_scope is None:
return None
return (
f"{wire_model or (model or '').strip().lower()}"
f"|thinking={thinking_enabled}"
f"|endpoint={endpoint_scope}"
)
def apply_litellm_param_recovery(
call_kwargs: Dict[str, Any],
recovery: GenerationParamRecovery,
) -> Dict[str, Any]:
"""Return kwargs with a learned parameter recovery applied."""
updated = dict(call_kwargs)
for param in recovery.omit_params:
updated.pop(param, None)
for param, value in recovery.set_params.items():
updated[param] = value
return updated
def get_cached_litellm_generation_param_recovery(
model: str,
*,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> Optional[GenerationParamRecovery]:
"""Return a process-local parameter recovery learned for this model call shape."""
key = _recovery_cache_key(
model,
model_list=model_list,
request_overrides=request_overrides,
)
if key is None:
return None
return _GENERATION_PARAM_RECOVERY_CACHE.get(key)
def remember_litellm_generation_param_recovery(
model: str,
recovery: GenerationParamRecovery,
*,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> None:
"""Remember a successful parameter recovery for later requests in this process."""
key = _recovery_cache_key(
model,
model_list=model_list,
request_overrides=request_overrides,
)
if key is None:
return
_GENERATION_PARAM_RECOVERY_CACHE[key] = recovery
def clear_litellm_generation_param_recovery_cache() -> None:
"""Clear process-local learned parameter recoveries. Intended for tests."""
_GENERATION_PARAM_RECOVERY_CACHE.clear()
def apply_litellm_generation_params(
call_kwargs: Dict[str, Any],
model: str,
temperature: Optional[float],
*,
default_temperature: float = 0.7,
model_list: Optional[List[Dict[str, Any]]] = None,
request_overrides: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
"""Return kwargs with model-compatible generation parameters applied."""
updated = dict(call_kwargs)
effective_overrides = request_overrides if request_overrides is not None else updated
directive = resolve_litellm_temperature_directive(
model,
model_list=model_list,
request_overrides=effective_overrides,
)
if directive.omit_temperature:
updated.pop("temperature", None)
elif directive.temperature is not None:
updated["temperature"] = directive.temperature
else:
updated["temperature"] = default_temperature if temperature is None else float(temperature)
cached_recovery = get_cached_litellm_generation_param_recovery(
model,
model_list=model_list,
request_overrides=updated,
)
if cached_recovery:
updated = apply_litellm_param_recovery(updated, cached_recovery)
return updated

View File

@@ -27,7 +27,6 @@ from src.config import (
channel_allows_empty_api_key,
get_configured_llm_models,
normalize_agent_litellm_model,
normalize_litellm_temperature,
normalize_news_strategy_profile,
normalize_llm_channel_model,
parse_env_bool,
@@ -43,6 +42,8 @@ from src.core.config_registry import (
get_field_definition,
get_registered_field_keys,
)
from src.llm.errors import call_litellm_with_param_recovery
from src.llm.generation_params import apply_litellm_generation_params
from src.notification_noise import validate_notification_timezone
from src.notification_sender.gotify_sender import resolve_gotify_message_endpoint
from src.notification_sender.ntfy_sender import resolve_ntfy_endpoint
@@ -724,10 +725,6 @@ class SystemConfigService:
call_kwargs: Dict[str, Any] = {
"model": resolved_model,
"messages": [{"role": "user", "content": "Reply with OK"}],
"temperature": normalize_litellm_temperature(
resolved_model,
self._get_runtime_llm_temperature(),
),
"max_tokens": 256, # Increased to allow MiniMax-M2.7 thinking process + response
"timeout": max(5.0, float(timeout_seconds)),
}
@@ -735,6 +732,11 @@ class SystemConfigService:
call_kwargs["api_key"] = selected_api_key
if base_url.strip():
call_kwargs["api_base"] = base_url.strip()
call_kwargs = apply_litellm_generation_params(
call_kwargs,
resolved_model,
self._get_runtime_llm_temperature(),
)
try:
import litellm
@@ -746,7 +748,13 @@ class SystemConfigService:
LLMToolAdapter._register_custom_model_pricing()
started_at = time.perf_counter()
response = litellm.completion(**call_kwargs)
response = call_litellm_with_param_recovery(
lambda kwargs: litellm.completion(**kwargs),
model=resolved_model,
call_kwargs=call_kwargs,
logger=logger,
log_label="[LLM channel test]",
)
latency_ms = int((time.perf_counter() - started_at) * 1000)
content, parse_error_code, parse_error, parse_reason = self._extract_llm_completion_content(response)
if parse_error_code:
@@ -1151,7 +1159,6 @@ class SystemConfigService:
call_kwargs: Dict[str, Any] = {
"model": resolved_model,
"messages": messages,
"temperature": normalize_litellm_temperature(resolved_model, 0.0),
"max_tokens": max_tokens,
"timeout": min(max(5.0, timeout), 10.0),
}
@@ -1161,6 +1168,11 @@ class SystemConfigService:
call_kwargs["api_base"] = base_url.strip()
if extra:
call_kwargs.update(extra)
call_kwargs = apply_litellm_generation_params(
call_kwargs,
resolved_model,
0.0,
)
return call_kwargs
@classmethod

View File

@@ -1687,6 +1687,172 @@ class TestAgentConstructionChain(unittest.TestCase):
self.assertEqual(result.content, "agent ok")
self.assertEqual(mock_completion.call_args.kwargs["temperature"], 0.6)
@patch("src.agent.llm_adapter.Router")
def test_llm_adapter_omits_temperature_for_gpt5_family(self, _mock_router):
"""Agent direct LiteLLM calls should omit temperature for strict default-temperature models."""
mock_cfg = SimpleNamespace(
agent_litellm_model="",
litellm_model="openai/gpt5.5-ferr",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=[],
deepseek_api_keys=[],
openai_base_url=None,
)
from src.agent.llm_adapter import LLMToolAdapter
adapter = LLMToolAdapter(config=mock_cfg)
adapter._router = None
response = SimpleNamespace(
choices=[
SimpleNamespace(
message=SimpleNamespace(
content="agent ok",
tool_calls=[],
)
)
],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=2, total_tokens=3),
)
with patch("src.agent.llm_adapter.litellm.completion", return_value=response) as mock_completion:
result = adapter._call_litellm_model(
[{"role": "user", "content": "hi"}],
[],
"openai/gpt5.5-ferr",
temperature=0.2,
)
self.assertEqual(result.content, "agent ok")
self.assertNotIn("temperature", mock_completion.call_args.kwargs)
@patch("src.agent.llm_adapter.Router")
def test_llm_adapter_recovers_from_unsupported_temperature(self, _mock_router):
"""Agent direct LiteLLM calls should retry once with a request-scoped parameter repair."""
from src.llm.generation_params import clear_litellm_generation_param_recovery_cache
clear_litellm_generation_param_recovery_cache()
mock_cfg = SimpleNamespace(
agent_litellm_model="",
litellm_model="openai/custom-temp-locked-agent",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=[],
deepseek_api_keys=[],
openai_base_url=None,
)
from src.agent.llm_adapter import LLMToolAdapter
adapter = LLMToolAdapter(config=mock_cfg)
adapter._router = None
response = SimpleNamespace(
choices=[
SimpleNamespace(
message=SimpleNamespace(
content="agent ok",
tool_calls=[],
)
)
],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=2, total_tokens=3),
)
with patch("src.agent.llm_adapter.litellm.completion") as mock_completion:
mock_completion.side_effect = [
RuntimeError("Unsupported parameter: temperature is not supported"),
response,
]
result = adapter._call_litellm_model(
[{"role": "user", "content": "hi"}],
[],
"openai/custom-temp-locked-agent",
temperature=0.2,
)
self.assertEqual(result.content, "agent ok")
self.assertEqual(mock_completion.call_args_list[0].kwargs["temperature"], 0.2)
self.assertNotIn("temperature", mock_completion.call_args_list[1].kwargs)
@patch("src.agent.llm_adapter.Router")
def test_llm_adapter_legacy_router_recovery_cache_is_scoped_to_endpoint(self, mock_router):
"""Legacy multi-key Router recoveries should not leak across base URLs."""
from src.llm.generation_params import clear_litellm_generation_param_recovery_cache
clear_litellm_generation_param_recovery_cache()
response = SimpleNamespace(
choices=[
SimpleNamespace(
message=SimpleNamespace(
content="agent ok",
tool_calls=[],
)
)
],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=2, total_tokens=3),
)
strict_router = MagicMock()
flex_router = MagicMock()
strict_router.completion.side_effect = [
RuntimeError("Unsupported parameter: temperature is not supported"),
response,
]
flex_router.completion.return_value = response
mock_router.side_effect = [strict_router, flex_router]
strict_cfg = SimpleNamespace(
agent_litellm_model="",
litellm_model="openai/shared-model",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=["sk-strict-key-1", "sk-strict-key-2"],
deepseek_api_keys=[],
openai_base_url="https://strict.example/v1",
)
flex_cfg = SimpleNamespace(
agent_litellm_model="",
litellm_model="openai/shared-model",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=["sk-flex-key-1", "sk-flex-key-2"],
deepseek_api_keys=[],
openai_base_url="https://flex.example/v1",
)
from src.agent.llm_adapter import LLMToolAdapter
strict_adapter = LLMToolAdapter(config=strict_cfg)
strict_result = strict_adapter._call_litellm_model(
[{"role": "user", "content": "hi"}],
[],
"openai/shared-model",
temperature=0.2,
)
flex_adapter = LLMToolAdapter(config=flex_cfg)
flex_result = flex_adapter._call_litellm_model(
[{"role": "user", "content": "hi"}],
[],
"openai/shared-model",
temperature=0.2,
)
self.assertEqual(strict_result.content, "agent ok")
self.assertEqual(flex_result.content, "agent ok")
self.assertEqual(strict_router.completion.call_args_list[0].kwargs["temperature"], 0.2)
self.assertNotIn("temperature", strict_router.completion.call_args_list[1].kwargs)
self.assertEqual(flex_router.completion.call_args.kwargs["temperature"], 0.2)
@patch("src.agent.llm_adapter.Router")
def test_llm_adapter_fallback_does_not_leak_kimi_fixed_temperature(self, _mock_router):
"""Non-Kimi fallbacks should keep the requested temperature after a Kimi failure."""

View File

@@ -14,6 +14,10 @@ from src.config import (
get_fixed_litellm_temperature,
normalize_litellm_temperature,
)
from src.llm.generation_params import (
apply_litellm_generation_params,
resolve_litellm_temperature_directive,
)
class LLMChannelConfigTestCase(unittest.TestCase):
@@ -479,6 +483,38 @@ class LLMChannelConfigTestCase(unittest.TestCase):
0.6,
)
def test_gpt5_family_temperature_is_omitted_at_request_build_time(self) -> None:
directive = resolve_litellm_temperature_directive("openai/gpt5.5-ferr")
self.assertTrue(directive.omit_temperature)
call_kwargs = apply_litellm_generation_params(
{"model": "openai/gpt5.5-ferr", "messages": [], "temperature": 0.2},
"openai/gpt5.5-ferr",
0.2,
)
self.assertNotIn("temperature", call_kwargs)
self.assertAlmostEqual(normalize_litellm_temperature("openai/gpt5.5-ferr", 0.2), 0.2)
def test_gpt5_temperature_directive_resolves_litellm_yaml_alias(self) -> None:
model_list = [
{
"model_name": "future_router",
"litellm_params": {"model": "openai/gpt-5.5"},
}
]
directive = resolve_litellm_temperature_directive("future_router", model_list=model_list)
call_kwargs = apply_litellm_generation_params(
{"model": "future_router", "messages": []},
"future_router",
0.2,
model_list=model_list,
)
self.assertTrue(directive.omit_temperature)
self.assertNotIn("temperature", call_kwargs)
@patch("src.config.setup_env")
@patch.object(Config, "_parse_litellm_yaml", return_value=[])
def test_local_openai_compatible_channel_defaults_to_openai_protocol(self, _mock_parse_yaml, _mock_setup_env) -> None:

View File

@@ -0,0 +1,202 @@
# -*- coding: utf-8 -*-
"""Tests for LiteLLM generation-parameter recovery."""
from src.llm.errors import (
call_litellm_with_param_recovery,
classify_litellm_generation_param_error,
)
from src.llm.generation_params import (
apply_litellm_generation_params,
clear_litellm_generation_param_recovery_cache,
)
def test_temperature_default_only_error_sets_temperature_to_one() -> None:
recovery = classify_litellm_generation_param_error(
RuntimeError(
"Unsupported value: 'temperature' does not support 0.7 with this model. "
"Only the default (1.0) value is supported."
)
)
assert recovery is not None
assert recovery.set_params == {"temperature": 1.0}
assert recovery.omit_params == ()
def test_temperature_default_only_error_uses_named_default_value() -> None:
recovery = classify_litellm_generation_param_error(
RuntimeError(
"Unsupported value: 'temperature' does not support 1.0 with this model. "
"Only `0.6` is allowed."
)
)
assert recovery is not None
assert recovery.set_params == {"temperature": 0.6}
assert recovery.omit_params == ()
def test_temperature_default_only_error_without_named_value_omits_temperature() -> None:
recovery = classify_litellm_generation_param_error(
RuntimeError(
"Unsupported value: 'temperature' does not support 0.7 with this model. "
"Only the default value is supported."
)
)
assert recovery is not None
assert recovery.set_params == {}
assert recovery.omit_params == ("temperature",)
def test_unsupported_temperature_error_retries_once_and_caches_recovery() -> None:
clear_litellm_generation_param_recovery_cache()
calls = []
def _call(kwargs):
calls.append(dict(kwargs))
if len(calls) == 1:
raise RuntimeError("Unsupported parameter: temperature is not supported")
return "ok"
result = call_litellm_with_param_recovery(
_call,
model="openai/custom-temp-locked",
call_kwargs={
"model": "openai/custom-temp-locked",
"messages": [],
"temperature": 0.7,
},
)
future_kwargs = apply_litellm_generation_params(
{"model": "openai/custom-temp-locked", "messages": []},
"openai/custom-temp-locked",
0.7,
)
assert result == "ok"
assert calls[0]["temperature"] == 0.7
assert "temperature" not in calls[1]
assert "temperature" not in future_kwargs
def test_recovery_cache_is_scoped_to_api_base() -> None:
clear_litellm_generation_param_recovery_cache()
calls = []
def _call(kwargs):
calls.append(dict(kwargs))
if len(calls) == 1:
raise RuntimeError("Unsupported parameter: temperature is not supported")
return "ok"
result = call_litellm_with_param_recovery(
_call,
model="openai/shared-model",
call_kwargs={
"model": "openai/shared-model",
"messages": [],
"api_base": "https://strict.example/v1",
"temperature": 0.7,
},
)
strict_kwargs = apply_litellm_generation_params(
{"model": "openai/shared-model", "messages": [], "api_base": "https://strict.example/v1"},
"openai/shared-model",
0.7,
)
flexible_kwargs = apply_litellm_generation_params(
{"model": "openai/shared-model", "messages": [], "api_base": "https://flex.example/v1"},
"openai/shared-model",
0.7,
)
assert result == "ok"
assert "temperature" not in strict_kwargs
assert flexible_kwargs["temperature"] == 0.7
def test_recovery_cache_skips_ambiguous_router_endpoints() -> None:
clear_litellm_generation_param_recovery_cache()
model_list = [
{
"model_name": "openai/shared-model",
"litellm_params": {
"model": "openai/shared-model",
"api_base": "https://strict.example/v1",
},
},
{
"model_name": "openai/shared-model",
"litellm_params": {
"model": "openai/shared-model",
"api_base": "https://flex.example/v1",
},
},
]
calls = []
def _call(kwargs):
calls.append(dict(kwargs))
if len(calls) == 1:
raise RuntimeError("Unsupported parameter: temperature is not supported")
return "ok"
result = call_litellm_with_param_recovery(
_call,
model="openai/shared-model",
call_kwargs={"model": "openai/shared-model", "messages": [], "temperature": 0.7},
model_list=model_list,
)
future_kwargs = apply_litellm_generation_params(
{"model": "openai/shared-model", "messages": []},
"openai/shared-model",
0.7,
model_list=model_list,
)
assert result == "ok"
assert future_kwargs["temperature"] == 0.7
def test_streaming_retry_does_not_cache_before_stream_is_consumed() -> None:
clear_litellm_generation_param_recovery_cache()
calls = []
def _broken_stream():
raise RuntimeError("stream failed during iteration")
yield # pragma: no cover
def _call(kwargs):
calls.append(dict(kwargs))
if len(calls) == 1:
raise RuntimeError("Unsupported parameter: temperature is not supported")
return _broken_stream()
stream = call_litellm_with_param_recovery(
_call,
model="openai/stream-model",
call_kwargs={
"model": "openai/stream-model",
"messages": [],
"temperature": 0.7,
"stream": True,
},
cache_recovery=False,
)
try:
list(stream)
except RuntimeError:
pass
else: # pragma: no cover
raise AssertionError("stream should fail during iteration")
future_kwargs = apply_litellm_generation_params(
{"model": "openai/stream-model", "messages": []},
"openai/stream-model",
0.7,
)
assert "temperature" not in calls[1]
assert future_kwargs["temperature"] == 0.7

View File

@@ -142,6 +142,149 @@ class TestAnalyzerGenerateText:
assert usage == {"prompt_tokens": 1, "completion_tokens": 2, "total_tokens": 3}
assert progress_updates == [3, 6]
def test_call_litellm_legacy_path_uses_legacy_model_list_for_param_recovery(self):
with patch("src.analyzer.get_config") as mock_cfg:
cfg = MagicMock()
cfg.litellm_model = "openai/gpt-4o-mini"
cfg.litellm_fallback_models = []
cfg.gemini_api_keys = []
cfg.anthropic_api_keys = []
cfg.deepseek_api_keys = []
cfg.openai_api_keys = ["sk-openai-legacy-a", "sk-openai-legacy-b"]
cfg.openai_base_url = None
cfg.llm_model_list = [
{
"model_name": "__legacy_openai__",
"litellm_params": {
"model": "__legacy_openai__",
"api_key": "sk-openai-legacy-a",
"api_base": "https://legacy-a.example/v1",
"extra_headers": {"x-tenant": "legacy-a"},
},
},
{
"model_name": "__legacy_openai__",
"litellm_params": {
"model": "__legacy_openai__",
"api_key": "sk-openai-legacy-b",
"api_base": "https://legacy-b.example/v1",
"extra_headers": {"x-tenant": "legacy-b"},
},
},
]
cfg.llm_temperature = 0.7
mock_cfg.return_value = cfg
from src.analyzer import GeminiAnalyzer
analyzer = GeminiAnalyzer()
analyzer._config_override = cfg
captured = {}
def _fake_call_litellm_with_param_recovery(call, **kwargs):
captured["model_list"] = kwargs.get("model_list")
return SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="ok"))],
usage=None,
)
with patch("src.analyzer.call_litellm_with_param_recovery", side_effect=_fake_call_litellm_with_param_recovery):
text, _, _ = analyzer._call_litellm("回归用例", {"max_tokens": 128, "temperature": 0.7})
assert text == "ok"
passed_model_list = captured.get("model_list")
assert passed_model_list is not None
assert len(passed_model_list) == 2
assert all(item["litellm_params"].get("model") == "openai/gpt-4o-mini" for item in passed_model_list)
assert [item["litellm_params"]["api_base"] for item in passed_model_list] == [
"https://legacy-a.example/v1",
"https://legacy-b.example/v1",
]
assert [item["litellm_params"]["extra_headers"] for item in passed_model_list] == [
{"x-tenant": "legacy-a"},
{"x-tenant": "legacy-b"},
]
@patch("src.analyzer.Router")
def test_analyzer_legacy_router_recovery_cache_is_scoped_by_api_base(self, mock_router):
"""Analyzer legacy recovery should not leak across same model different api_base."""
from src.analyzer import call_litellm_with_param_recovery as real_call
from src.llm.generation_params import clear_litellm_generation_param_recovery_cache
clear_litellm_generation_param_recovery_cache()
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="analyzer ok"))],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=2, total_tokens=3),
)
strict_router = MagicMock()
flex_router = MagicMock()
strict_router.completion.side_effect = [
RuntimeError("Unsupported parameter: temperature is not supported"),
response,
]
flex_router.completion.return_value = response
mock_router.side_effect = [strict_router, flex_router]
strict_cfg = SimpleNamespace(
litellm_model="openai/shared-model",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=["sk-strict-key-1", "sk-strict-key-2"],
deepseek_api_keys=[],
openai_base_url="https://strict.example/v1",
)
flex_cfg = SimpleNamespace(
litellm_model="openai/shared-model",
litellm_fallback_models=[],
llm_model_list=[],
llm_temperature=0.2,
gemini_api_keys=[],
anthropic_api_keys=[],
openai_api_keys=["sk-flex-key-1", "sk-flex-key-2"],
deepseek_api_keys=[],
openai_base_url="https://flex.example/v1",
)
captured_model_lists = []
def _fake_recovery(call, **kwargs):
captured_model_lists.append(kwargs.get("model_list"))
return real_call(call, **kwargs)
import src.analyzer as analyzer_module
from src.analyzer import GeminiAnalyzer
with patch.object(analyzer_module, "call_litellm_with_param_recovery", side_effect=_fake_recovery):
GeminiAnalyzer(config=strict_cfg)._call_litellm(
"prompt",
{"max_tokens": 128, "temperature": 0.2},
)
GeminiAnalyzer(config=flex_cfg)._call_litellm(
"prompt",
{"max_tokens": 128, "temperature": 0.2},
)
assert len(captured_model_lists) == 2
strict_model_list = captured_model_lists[0]
flex_model_list = captured_model_lists[1]
assert strict_model_list is not None
assert flex_model_list is not None
assert all(
item.get("litellm_params", {}).get("api_base") == "https://strict.example/v1"
for item in strict_model_list
)
assert all(
item.get("litellm_params", {}).get("api_base") == "https://flex.example/v1"
for item in flex_model_list
)
assert strict_router.completion.call_args_list[0].kwargs["temperature"] == 0.2
assert "temperature" not in strict_router.completion.call_args_list[1].kwargs
assert flex_router.completion.call_args.kwargs["temperature"] == 0.2
def test_call_litellm_stream_falls_back_to_non_stream_before_first_chunk(self):
analyzer = self._make_analyzer()
analyzer._config_override = SimpleNamespace(
@@ -315,6 +458,66 @@ class TestAnalyzerGenerateText:
call_kwargs = mock_dispatch.call_args.args[1]
assert call_kwargs["temperature"] == 0.6
def test_call_litellm_omits_temperature_for_gpt5_family(self):
analyzer = self._make_analyzer()
analyzer._config_override = SimpleNamespace(
litellm_model="openai/gpt5.5-ferr",
litellm_fallback_models=[],
llm_model_list=[],
)
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="ok"))],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=1, total_tokens=2),
)
with patch.object(analyzer, "_dispatch_litellm_completion", return_value=response) as mock_dispatch:
text, model_used, usage = analyzer._call_litellm(
"prompt",
{"max_tokens": 128, "temperature": 0.2},
)
assert text == "ok"
assert model_used == "openai/gpt5.5-ferr"
assert usage == {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
call_kwargs = mock_dispatch.call_args.args[1]
assert "temperature" not in call_kwargs
def test_call_litellm_recovers_from_temperature_default_error(self):
from src.llm.generation_params import clear_litellm_generation_param_recovery_cache
clear_litellm_generation_param_recovery_cache()
analyzer = self._make_analyzer()
analyzer._config_override = SimpleNamespace(
litellm_model="openai/custom-default-temp",
litellm_fallback_models=[],
llm_model_list=[],
)
response = SimpleNamespace(
choices=[SimpleNamespace(message=SimpleNamespace(content="ok"))],
usage=SimpleNamespace(prompt_tokens=1, completion_tokens=1, total_tokens=2),
)
calls = []
def _dispatch(model, call_kwargs, **_kwargs):
calls.append(dict(call_kwargs))
if len(calls) == 1:
raise RuntimeError(
"temperature=0.2 is unsupported. Only the default (1.0) value is supported."
)
return response
with patch.object(analyzer, "_dispatch_litellm_completion", side_effect=_dispatch):
text, model_used, usage = analyzer._call_litellm(
"prompt",
{"max_tokens": 128, "temperature": 0.2},
)
assert text == "ok"
assert model_used == "openai/custom-default-temp"
assert usage == {"prompt_tokens": 1, "completion_tokens": 1, "total_tokens": 2}
assert calls[0]["temperature"] == 0.2
assert calls[1]["temperature"] == 1.0
def test_call_litellm_keeps_user_temperature_for_non_kimi_fallback(self):
analyzer = self._make_analyzer()
analyzer._config_override = SimpleNamespace(

View File

@@ -1457,6 +1457,44 @@ class SystemConfigServiceTestCase(unittest.TestCase):
self.assertEqual(current_map["LITELLM_MODEL"], "openai/kimi-k2.6")
self.assertEqual(current_map["LLM_TEMPERATURE"], "0.42")
def test_update_runtime_model_cleanup_does_not_rewrite_temperature(self) -> None:
self._rewrite_env(
"STOCK_LIST=600519,000001",
"LLM_CHANNELS=deepseek",
"LLM_DEEPSEEK_PROTOCOL=deepseek",
"LLM_DEEPSEEK_BASE_URL=https://api.deepseek.com",
"LLM_DEEPSEEK_API_KEY=sk-test-value",
"LLM_DEEPSEEK_MODELS=deepseek-chat,deepseek-v4-flash",
"LITELLM_MODEL=deepseek/deepseek-chat",
"AGENT_LITELLM_MODEL=deepseek/deepseek-v4-flash",
"LLM_TEMPERATURE=0.42",
"LITELLM_FALLBACK_MODELS=deepseek/deepseek-v4-flash,cohere/command-r-plus",
"VISION_MODEL=deepseek/deepseek-chat",
)
response = self.service.update(
config_version=self.manager.get_config_version(),
items=[
{"key": "LLM_DEEPSEEK_MODELS", "value": "deepseek-v4-flash"},
{"key": "LITELLM_MODEL", "value": ""},
{"key": "AGENT_LITELLM_MODEL", "value": ""},
{"key": "LITELLM_FALLBACK_MODELS", "value": "deepseek/deepseek-v4-flash"},
{"key": "VISION_MODEL", "value": ""},
],
reload_now=False,
)
self.assertTrue(response["success"])
current_map = self.manager.read_config_map()
self.assertEqual(current_map["LLM_TEMPERATURE"], "0.42")
self.assertEqual(current_map["LITELLM_MODEL"], "")
self.assertEqual(current_map["AGENT_LITELLM_MODEL"], "")
self.assertEqual(current_map["VISION_MODEL"], "")
self.assertEqual(
current_map["LITELLM_FALLBACK_MODELS"],
"deepseek/deepseek-v4-flash",
)
@patch("litellm.completion")
def test_test_llm_channel_does_not_persist_normalized_kimi_temperature(self, mock_completion) -> None:
self._rewrite_env("LLM_TEMPERATURE=0.42")
@@ -1480,6 +1518,62 @@ class SystemConfigServiceTestCase(unittest.TestCase):
self.assertEqual(mock_completion.call_args.kwargs["temperature"], 1.0)
self.assertEqual(self.manager.read_config_map()["LLM_TEMPERATURE"], "0.42")
@patch("litellm.completion")
def test_test_llm_channel_omits_temperature_for_gpt5_family(self, mock_completion) -> None:
mock_completion.return_value = type(
"MockResponse",
(),
{
"choices": [type("Choice", (), {"message": type("Message", (), {"content": "OK"})()})()],
},
)()
payload = self.service.test_llm_channel(
name="primary",
protocol="openai",
base_url="https://api.example.com/v1",
api_key="sk-test-value",
models=["gpt5.5-ferr"],
)
self.assertTrue(payload["success"])
self.assertEqual(payload["resolved_model"], "openai/gpt5.5-ferr")
self.assertNotIn("temperature", mock_completion.call_args.kwargs)
@patch("litellm.completion")
@patch("src.services.system_config_service.Config._load_from_env")
def test_test_llm_channel_recovers_from_unsupported_temperature(
self,
mock_load_config,
mock_completion,
) -> None:
from src.llm.generation_params import clear_litellm_generation_param_recovery_cache
clear_litellm_generation_param_recovery_cache()
mock_load_config.return_value = SimpleNamespace(llm_temperature=0.42)
mock_completion.side_effect = [
RuntimeError("Unsupported parameter: temperature is not supported"),
type(
"MockResponse",
(),
{
"choices": [type("Choice", (), {"message": type("Message", (), {"content": "OK"})()})()],
},
)(),
]
payload = self.service.test_llm_channel(
name="primary",
protocol="openai",
base_url="https://api.example.com/v1",
api_key="sk-test-value",
models=["custom-temp-locked-settings"],
)
self.assertTrue(payload["success"])
self.assertEqual(mock_completion.call_args_list[0].kwargs["temperature"], 0.42)
self.assertNotIn("temperature", mock_completion.call_args_list[1].kwargs)
@patch("litellm.completion")
@patch("src.services.system_config_service.Config._load_from_env")
def test_test_llm_channel_uses_runtime_temperature_for_non_kimi_models(