feat(agent): per-category tool timeouts with graceful degradation (#1890) (#2134)

This commit is contained in:
lmx-2077
2026-08-17 21:41:19 +08:00
committed by GitHub
parent 5c964bf23b
commit cfd6b0a5fb
14 changed files with 1757 additions and 139 deletions

View File

@@ -507,6 +507,15 @@ AGENT_SKILLS=
# AGENT_PORTFOLIO_AGENT_TIMEOUT_S=0
# AGENT_SKILL_AGENT_TIMEOUT_S=0
# Agent 工具按类别默认超时0=关闭;未配置时回退到全局 tool_call_timeout_seconds 预算)
# 有效超时按 first-wins 解析:显式 per-run tool_call_timeout_seconds > 单工具显式 timeout_seconds > 类别默认值 > 无限制;
# 剩余 wall-clock 预算仅作不可突破的外层 capinf/nan/负数视为无限制。
# market 类工具get_market_indices / get_sector_rankings 等网络数据调用)复用 AGENT_DATA_TOOL_TIMEOUT_S无独立开关。
# AGENT_DATA_TOOL_TIMEOUT_S=0
# AGENT_SEARCH_TOOL_TIMEOUT_S=0
# AGENT_ANALYSIS_TOOL_TIMEOUT_S=0
# AGENT_ACTION_TOOL_TIMEOUT_S=0
# 策略专家并发数(仅 specialist 模式生效;范围 1-4默认 3
# AGENT_SKILL_CONCURRENCY=3

View File

@@ -15,6 +15,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
- [修复] 单股推送模式在未配置通知渠道时仍会落盘本地个股报告CLI 启动分析若因空股票列表、个股结果全失败或本地报告保存失败而未生成报告,会显式返回失败并记录原因。
- [修复] 合并推送模式下即使个股汇总报告落盘失败,仍会先发送已有的合并通知;仅启用大盘复盘但最终未生成任何复盘内容时,分析任务会显式返回失败。
- [新功能] Agent 工具调用支持按类别data/search/analysis/action/market配置默认超时并允许单工具声明 `timeout_seconds`;有效超时按 first-wins 优先级解析(显式 per-run `tool_call_timeout_seconds` > 单工具显式 `timeout_seconds` > 类别默认 > 无限制),剩余 wall-clock 预算仅作不可突破的外层 cap超时后返回结构化 `{"timeout": true}` 错误(标记 `retriable: false` 并写入 `non_retriable_tool_results` 防重试重复执行)供 Agent 继续执行而非中断循环fixes #1890)。
- [修复] Agent 工具注册表(`src/agent/factory.get_tool_registry`)由模块级缓存改为按「类别超时映射的值」比对失效,规避 CPython 回收对象后地址复用(`id(config)` 相同)导致配置 reload 后的 `Config` 被误判为未变、沿用过期超时的真 bug新增 `_coerce_config_timeout` 类型白名单,使调用方传入 `MagicMock` / 缺属性 stub / 脏字符串(如 `float(MagicMock())` 静默得到 1.0)时降级为「无类别限制」而非崩溃或强加 1 秒超时;`build_agent_executor(config)` / `build_agent_chat_executor(config)` 现已把调用方 `config` 透传给 `get_tool_registry(config)`(不再无参调用冻结首构 registry`main._reload_runtime_config``SystemConfigService._reload_runtime_singletons`(及 `update()``reload_now` 路径)在配置热重载时调用 `reset_tool_registry()` 强制重建;回归测试补充「传入新 config 后 registry 重建」「reload 后新超时应生效」及「builder 透传 config」三类场景#1890 的 review follow-up闭环 OR-COM-dd1e8fa7 / OR-COM-bff42110
- [修复] Agent 工具超时 review 闭环fixes #1890 的 4 个 blocker超时解析由 min 契约改为 first-wins显式 per-run `tool_call_timeout_seconds` > 单工具 `ToolDefinition.timeout_seconds` > 类别默认 > 无限制,剩余 wall-clock 预算只作不可突破的外层 capresearch 路径不再传 `tool_call_timeout_seconds` 以免覆盖类别限制);超时结果标记 `retriable: false` 并写入 `non_retriable_tool_results` 阻断 LLM 同调用重试重入,且超时触发时为仍在后台运行的 handler 武装协作取消信号(`is_tool_cancellation_requested()` 与既有 `check_tool_execution()` 检查点均响应handler 从不轮询则行为不变),作为 review 要求的「handler 内协作取消」缓解,规避 Python 线程无法 force-stop 导致的重复执行与副作用;`_coerce_config_timeout``inf`/`nan`/负数降级为「无限制」,根绝 `future.result(timeout=inf)` 触发 `OverflowError``get_tool_registry` / `reset_tool_registry``threading.Lock` 双检锁,且重建后返回本次构建的局部 registry而非全局缓存消除并发重建竞态与跨调用超时串扰`@tool` 装饰器将 `ToolPolicy.timeout_seconds` 折叠进 `ToolDefinition` 单一来源;统一单/并行工具超时包装(单一 executor + deadline 驱动的 wait loop消除并行路径嵌套 executor 与线程翻倍duration 精确到各工具自身超时值),并新增快慢工具混合并行回归;同步 `docs/full-guide_EN.md` 的超时环境变量文档;测试覆盖 first-wins、non-retriable、协作取消接线、finite 校验、缓存线程安全与快慢混合并行。
- [修复] 按最新 review 复核收敛 3 处正确性问题OR-COM-7f3d3f5b / 3d6b61f8 / a1e8b0c2`BaseAgent._filtered_registry()` 携带源 registry 的类别超时映射(工具子集仍生效类别上限,不再绕过 #1890 类别超时);并行批次 >5 时排队调用的 per-tool 超时自 worker 实际开始起算(不再提交即烧预算导致对未启动调用的假超时);`get_tool_registry()` 缓存命中快路径在锁内读取一致对(消除与 `reset_tool_registry()` 竞态返回 `None` 或错配 registry。新增对应回归测试。
## [3.30.0] - 2026-08-09
### 发布亮点

View File

@@ -246,6 +246,10 @@ daily_stock_analysis/
| `AGENT_BACKEND` | 现有问股 Chat 的运行方式:`auto`(推荐,保持默认模型)、`litellm``codex_app_server`(实验,仅 single-agent Chat | `auto` | 否 |
| `AGENT_GENERATION_BACKEND` | Agent Chat 生成后端Web 设置页仅暴露 `auto|litellm`,手写 local CLI backend 会返回 unsupported tool-calling 诊断 | `auto` | 否 |
| `AGENT_SKILL_CONCURRENCY` | `specialist` 模式策略专家 worker 并发上限,范围 `1-4`;最多选择 4 个策略,默认 3 个并发,第 4 个进入下一批次并共享整体超时预算 | `3` | 否 |
| `AGENT_DATA_TOOL_TIMEOUT_S` | Agent `data` 类工具默认超时秒数;同时作为 `market` 类工具(`get_market_indices` / `get_sector_rankings` 等网络数据调用)的类别默认;`0` 表示关闭,回退到全局预算;有效超时按 first-wins 解析:显式 per-run `tool_call_timeout_seconds` > 单工具显式 `timeout_seconds` > 类别默认 > 无限制,剩余 wall-clock 预算仅作不可突破的外层 cap`inf`/`nan`/负数降级为「无限制」;超时为 best-effort 软中断Python 线程无法被强制停止handler 可能在超时后继续运行,超时结果标记 `retriable: false` 并写入 `non_retriable_tool_results` 阻断重试,且为仍在后台的 handler 武装协作取消信号(`is_tool_cancellation_requested()` 与既有 `check_tool_execution()` 检查点均响应handler 从不轮询则行为不变)以尽量减小副作用 | `0` | 否 |
| `AGENT_SEARCH_TOOL_TIMEOUT_S` | Agent `search` 类工具默认超时秒数;`0` 表示关闭,回退到全局预算 | `0` | 否 |
| `AGENT_ANALYSIS_TOOL_TIMEOUT_S` | Agent `analysis` 类工具默认超时秒数;`0` 表示关闭,回退到全局预算 | `0` | 否 |
| `AGENT_ACTION_TOOL_TIMEOUT_S` | Agent `action` 类工具默认超时秒数;`0` 表示关闭,回退到全局预算 | `0` | 否 |
| `LITELLM_MODEL` | 主模型,格式 `provider/model`(如 `gemini/gemini-3.1-pro-preview`),推荐优先使用 | - | 否 |
| `AGENT_LITELLM_MODEL` | 「默认模型」问股的主模型(可选);留空继承主模型,无 provider 前缀按 `openai/<model>` 解析Codex 不使用此项 | - | 否 |
| `AGENT_CONTEXT_COMPRESSION_ENABLED` | 「默认模型」问股可见历史的 LLM 压缩开关Codex 使用最近 20 条可见对话且保留该配置 | `false` | 否 |

View File

@@ -217,6 +217,10 @@ Default schedule: Every weekday at **18:00 (Beijing Time)** automatic execution.
| `AGENT_BACKEND` | Runtime for the existing ask-stock Chat: `auto` (recommended, preserves the default model), `litellm`, or `codex_app_server` (experimental, single-agent Chat only) | `auto` | No |
| `AGENT_GENERATION_BACKEND` | Agent Chat generation backend. Web settings only expose `auto|litellm`; hand-written local CLI backends return an unsupported tool-calling diagnostic | `auto` | No |
| `AGENT_SKILL_CONCURRENCY` | Specialist-mode strategy worker concurrency cap, range `1-4`. Up to four strategies are selected; the default runs three concurrently and queues the fourth under the shared pipeline budget | `3` | No |
| `AGENT_DATA_TOOL_TIMEOUT_S` | Default timeout (seconds) for Agent `data`-category tools; also the category default for `market` tools (`get_market_indices` / `get_sector_rankings` and other network-backed data calls); `0` disables and falls back to the global budget. The effective timeout is resolved first-wins: explicit per-run `tool_call_timeout_seconds` > per-tool `timeout_seconds` > category default > no limit, with the remaining wall-clock budget as an unbreakable outer cap; `inf`/`nan`/negative degrade to "no limit". Timeout is a best-effort soft interrupt: Python threads cannot be force-stopped, so a handler may keep running after the timeout; the timed-out result is marked `retriable: false` and recorded in `non_retriable_tool_results` to block immediate retries, and the runner arms a cooperative-cancel signal (`is_tool_cancellation_requested()` and the existing `check_tool_execution()` checkpoints both honor it) to reduce side effects | `0` | No |
| `AGENT_SEARCH_TOOL_TIMEOUT_S` | Default timeout (seconds) for Agent `search`-category tools; `0` disables and falls back to the global budget | `0` | No |
| `AGENT_ANALYSIS_TOOL_TIMEOUT_S` | Default timeout (seconds) for Agent `analysis`-category tools; `0` disables and falls back to the global budget | `0` | No |
| `AGENT_ACTION_TOOL_TIMEOUT_S` | Default timeout (seconds) for Agent `action`-category tools; `0` disables and falls back to the global budget | `0` | No |
| `LITELLM_MODEL` | Primary model, format `provider/model` (e.g. `gemini/gemini-3.1-pro-preview`), recommended | - | No |
| `AGENT_LITELLM_MODEL` | Optional primary model for **Default model** ask-stock; empty inherits the primary model and bare names become `openai/<model>`; Codex does not use this setting | - | No |
| `AGENT_CONTEXT_COMPRESSION_ENABLED` | LLM compression for visible **Default model** ask-stock history; Codex uses the 20 most recent visible messages and retains this setting | `false` | No |

16
main.py
View File

@@ -1326,7 +1326,21 @@ def _reload_runtime_config() -> Config:
"""Reload config from the latest persisted `.env` values for scheduled runs."""
_reload_env_file_values_preserving_overrides()
Config.reset_instance()
return get_config()
new_config = get_config()
# Drop the module-level ToolRegistry so the next
# ``build_agent_executor`` / ``build_agent_chat_executor`` call rebuilds
# it against the freshly-loaded Config and picks up new
# ``AGENT_*_TOOL_TIMEOUT_S`` overrides (Issue #1890). Wrap in try/except
# so a future runtime reset helper cannot crash scheduled-job bootstrap.
try:
from src.agent.factory import reset_tool_registry
reset_tool_registry()
except Exception as exc: # pragma: no cover - defensive guard
logger.warning("Failed to reset tool registry during config reload: %s", exc)
return new_config
def _build_schedule_time_provider(default_schedule_time: str):

View File

@@ -257,7 +257,9 @@ class BaseAgent(ABC):
return self.tool_registry
from src.agent.tools.registry import ToolRegistry as TR
filtered = TR()
# Carry the source registry's category-timeout map so the filtered subset
# still enforces the per-category ceilings (review OR-COM-7f3d3f5b).
filtered = TR(category_timeout_map=self.tool_registry.category_timeout_map)
for name in self.tool_names:
tool_def = self.tool_registry.get(name)
if tool_def:

View File

@@ -26,6 +26,8 @@ Usage::
import copy
import logging
import math
import threading
from dataclasses import dataclass
from typing import List, Optional
@@ -37,6 +39,17 @@ logger = logging.getLogger(__name__)
# Module-level caches
# ---------------------------------------------------------------------------
_TOOL_REGISTRY = None
# Track the per-category timeout mapping ``_TOOL_REGISTRY`` was built from, so
# callers passing a freshly-reloaded ``Config`` always observe the right
# ceilings instead of inheriting the first build's values.
#
# Deliberately keyed by *value* rather than ``id(config)``: CPython reuses the
# address of a freed object, so a ``Config.reset_instance()`` + ``get_config()``
# round-trip can hand back a new instance with the *same* id() as the collected
# one, which would silently mark a changed config as "unchanged". This mirrors
# the ``_SKILL_MANAGER_CUSTOM_DIR`` value-comparison used by ``get_skill_manager``
# below. Pair with :func:`reset_tool_registry` for runtime config reload paths.
_CACHED_TIMEOUT_MAP: Optional[dict] = None
_SKILL_MANAGER_PROTOTYPE = None
# Sentinel used as initial value so None (i.e. no custom dir) compares as "changed"
# on the very first call, forcing a build rather than accidentally skipping it.
@@ -45,6 +58,20 @@ _SENTINEL = object()
# the cache if AGENT_SKILL_DIR changes at runtime (e.g. via config reload).
_SKILL_MANAGER_CUSTOM_DIR: object = _SENTINEL
# Guards the module-level ToolRegistry cache against concurrent rebuild/reset.
# ``get_tool_registry`` builds the registry *outside* this lock (see below) and
# only takes it to atomically assign the shared ``_TOOL_REGISTRY`` /
# ``_CACHED_TIMEOUT_MAP`` pair, so two requests with different ``Config``
# objects can never observe a registry built for the other's timeouts.
_tool_registry_lock = threading.Lock()
# Safety ceiling for any single per-category timeout value sourced from
# ``AGENT_*_TOOL_TIMEOUT_S``. ``inf``/``nan``/negative already degrade to
# "no limit"; this clips absurd-but-finite values so they can never reach
# ``future.result(timeout=...)`` and stall a request. The global
# ``tool_call_timeout_seconds`` budget is *not* subject to this clip.
_MAX_TOOL_TIMEOUT_S = 3600.0
@dataclass
class SkillPromptState:
@@ -190,12 +217,144 @@ def _should_use_legacy_default_prompt(
return getattr(bull_trend_skill, "source", None) == "builtin"
def get_tool_registry():
"""Return a cached ToolRegistry (built once, shared across requests)."""
global _TOOL_REGISTRY
if _TOOL_REGISTRY is not None:
return _TOOL_REGISTRY
def _coerce_config_timeout(config, field_name: str) -> float:
"""Read a timeout field off ``config`` as a float, defaulting to ``0.0``.
``get_tool_registry`` builds its category map from the *caller-supplied*
config, so it must tolerate partial config objects and test doubles: a
missing attribute raises ``AttributeError`` and ``MagicMock() > 0`` raises
``TypeError``. A plain ``float()`` conversion is not enough either --
``float(MagicMock())`` silently yields ``1.0``, which would quietly impose
a bogus 1-second ceiling on every tool. Only genuine numbers (or numeric
strings, since ``.env`` values arrive as text) are accepted; anything else
degrades to "no category limit" and defers to the global
``tool_call_timeout_seconds`` budget.
"""
raw_value = getattr(config, field_name, None)
if isinstance(raw_value, bool) or raw_value is None:
# ``True`` would coerce to a nonsensical 1-second ceiling.
return 0.0
if isinstance(raw_value, (int, float)):
value = float(raw_value)
if not math.isfinite(value):
# ``inf``/``nan`` are not valid ceilings; an ``inf`` would later
# raise ``OverflowError`` at ``future.result(timeout=inf)``.
logger.warning(
"[AgentFactory] Non-finite value for %s: %r, treating as no category limit",
field_name,
raw_value,
)
return 0.0
if value < 0:
logger.warning(
"[AgentFactory] Negative value for %s: %r, treating as no category limit",
field_name,
raw_value,
)
return 0.0
if value == 0.0:
# Explicitly disabled (the default) — silent, not a warning.
return 0.0
if value > _MAX_TOOL_TIMEOUT_S:
logger.warning(
"[AgentFactory] Clamping %s=%r to safety ceiling %ss",
field_name, value, _MAX_TOOL_TIMEOUT_S,
)
return float(_MAX_TOOL_TIMEOUT_S)
return value
if isinstance(raw_value, str):
try:
value = float(raw_value.strip())
except ValueError:
logger.warning(
"[AgentFactory] Invalid value for %s: %r, treating as no category limit",
field_name,
raw_value,
)
return 0.0
if not math.isfinite(value):
logger.warning(
"[AgentFactory] Non-finite value for %s: %r, treating as no category limit",
field_name,
raw_value,
)
return 0.0
if value < 0:
logger.warning(
"[AgentFactory] Negative value for %s: %r, treating as no category limit",
field_name,
raw_value,
)
return 0.0
if value == 0.0:
return 0.0
if value > _MAX_TOOL_TIMEOUT_S:
return float(_MAX_TOOL_TIMEOUT_S)
return value
# Mocks / arbitrary objects: never let them shape real timeout behaviour.
logger.debug(
"[AgentFactory] Non-numeric value for %s: %r, treating as no category limit",
field_name,
type(raw_value).__name__,
)
return 0.0
def _build_category_timeout_map(config) -> dict:
"""Map tool categories to their default timeout (seconds) from config.
Only explicit overrides (value > 0) are carried; 0.0 means "no category
limit, fall back to the global tool_call_timeout_seconds budget".
``market`` tools (get_market_indices / get_sector_rankings) are
network-backed data calls, so they share the ``data`` ceiling.
"""
data_timeout = _coerce_config_timeout(config, "agent_data_tool_timeout_s")
return {
k: v
for k, v in {
"data": data_timeout,
"search": _coerce_config_timeout(config, "agent_search_tool_timeout_s"),
"analysis": _coerce_config_timeout(config, "agent_analysis_tool_timeout_s"),
"action": _coerce_config_timeout(config, "agent_action_tool_timeout_s"),
"market": data_timeout,
}.items()
if v > 0
}
def reset_tool_registry() -> None:
"""Drop the cached :class:`ToolRegistry` so the next access rebuilds it.
Intended for runtime config reload paths
(``main._reload_runtime_config`` /
``SystemConfigService._reload_runtime_singletons``) so that newly-loaded
``AGENT_*_TOOL_TIMEOUT_S`` overrides actually take effect on subsequent
:func:`build_agent_executor` / :func:`build_agent_chat_executor` calls,
instead of being silently shadowed by the registry built at first import.
Guarded by ``_tool_registry_lock`` so a concurrent :func:`get_tool_registry`
cannot read a half-reset cache (``None`` registry with a stale map, or vice
versa).
"""
global _TOOL_REGISTRY, _CACHED_TIMEOUT_MAP
with _tool_registry_lock:
if _TOOL_REGISTRY is not None:
logger.info("[AgentFactory] ToolRegistry cache cleared; will rebuild on next access")
_TOOL_REGISTRY = None
_CACHED_TIMEOUT_MAP = None
def _build_tool_registry(category_timeout_map):
"""Construct and populate a :class:`ToolRegistry` from the tool modules.
Extracted from :func:`get_tool_registry` so the (potentially heavy) tool
module imports and registrations happen *outside* the cache lock — holding
the lock during imports would serialise startup and risk a deadlock if any
imported module ever imported this module back.
"""
from src.agent.tools.registry import ToolRegistry
from src.agent.tools.data_tools import ALL_DATA_TOOLS
from src.agent.tools.analysis_tools import ALL_ANALYSIS_TOOLS
@@ -203,13 +362,91 @@ def get_tool_registry():
from src.agent.tools.market_tools import ALL_MARKET_TOOLS
from src.agent.tools.backtest_tools import ALL_BACKTEST_TOOLS
registry = ToolRegistry()
for tool_fn in ALL_DATA_TOOLS + ALL_ANALYSIS_TOOLS + ALL_SEARCH_TOOLS + ALL_MARKET_TOOLS + ALL_BACKTEST_TOOLS:
registry = ToolRegistry(category_timeout_map=category_timeout_map or None)
for tool_fn in (
ALL_DATA_TOOLS
+ ALL_ANALYSIS_TOOLS
+ ALL_SEARCH_TOOLS
+ ALL_MARKET_TOOLS
+ ALL_BACKTEST_TOOLS
):
registry.register(tool_fn)
return registry
_TOOL_REGISTRY = registry
logger.info("[AgentFactory] ToolRegistry cached (%d tools)", len(registry._tools) if hasattr(registry, "_tools") else -1)
return _TOOL_REGISTRY
def get_tool_registry(config=None):
"""Return a :class:`ToolRegistry` bound to the lifetime of ``config``.
The first call (or the first call after :func:`reset_tool_registry`)
builds and caches a registry from ``config``. The cache is keyed by the
resolved per-category timeout *values*, so a config carrying different
``AGENT_*_TOOL_TIMEOUT_S`` settings (e.g. a freshly-reloaded ``Config``
singleton) rebuilds the registry, while an equivalent config reuses the
cached instance and avoids re-registering the whole tool surface.
Value comparison is intentional: ``id(config)`` is not a safe key because
CPython reuses the address of a collected object, so the instance returned
after ``Config.reset_instance()`` can share the previous instance's id and
would be misread as "unchanged".
When ``config`` is ``None`` the function falls back to ``get_config()``,
preserving the legacy contract for callers that do not yet know about
config reload.
Thread-safety: the fast-path cache hit reads both globals under the lock
(so the check and the return observe one consistent pair — see
``reset_tool_registry``), and the registry build runs outside the lock with
only the final assignment of the shared ``_TOOL_REGISTRY`` /
``_CACHED_TIMEOUT_MAP`` pair serialised (double-checked locking), so two
requests with different ``Config`` objects can never observe a registry built
for the other's timeouts. On a (re)build the function returns the registry it
just constructed for the caller's timeout map, so a concurrent rebuild cannot
leak a mismatched registry into this call's result.
"""
global _TOOL_REGISTRY, _CACHED_TIMEOUT_MAP
if config is None:
from src.config import get_config
config = get_config()
category_timeout_map = _build_category_timeout_map(config)
# Fast path: take the lock so the cache-hit check and the return observe one
# consistent (registry, timeout-map) pair — an unlocked check-then-use can
# race with ``reset_tool_registry()`` and return ``None`` or a registry built
# for the wrong timeout map (review OR-COM-a1e8b0c2).
with _tool_registry_lock:
if _TOOL_REGISTRY is not None and _CACHED_TIMEOUT_MAP == category_timeout_map:
return _TOOL_REGISTRY
# Build *outside* the lock (see ``_build_tool_registry`` rationale above).
registry = _build_tool_registry(category_timeout_map)
with _tool_registry_lock:
# Re-check under the lock: another thread may have installed an
# equivalent registry while we were importing the tool modules.
if _TOOL_REGISTRY is not None and _CACHED_TIMEOUT_MAP == category_timeout_map:
return _TOOL_REGISTRY
invalidated = _TOOL_REGISTRY is not None
_TOOL_REGISTRY = registry
_CACHED_TIMEOUT_MAP = category_timeout_map
if invalidated:
logger.info(
"[AgentFactory] ToolRegistry rebuilt for new category timeouts: %s",
category_timeout_map or "none (global budget only)",
)
else:
logger.info(
"[AgentFactory] ToolRegistry cached (%d tools, category timeouts: %s)",
len(registry._tools) if hasattr(registry, "_tools") else -1,
category_timeout_map or "none (global budget only)",
)
# Return the registry built for *this* call's timeout map (not the shared
# ``_TOOL_REGISTRY`` global). Under a concurrent rebuild for a different
# ``Config``/timeout map, this guarantees the caller gets the registry that
# matches the timeouts it requested, instead of a registry built for another
# request. The global is still assigned above so equivalent subsequent calls
# hit the cache fast path.
return registry
def get_skill_manager(config=None):
@@ -336,7 +573,11 @@ def build_agent_executor(config=None, skills: Optional[List[str]] = None):
from src.agent.llm_adapter import LLMToolAdapter
registry = get_tool_registry()
# Pass ``config`` explicitly so a reloaded ``Config`` instance observes
# the new per-category timeout map (Issue #1890 compatibility concern:
# the module-level ToolRegistry would otherwise keep the first build's
# timeouts even after ``_reload_runtime_config``).
registry = get_tool_registry(config)
prompt_state = resolve_skill_prompt_state(config, skills=skills)
skill_manager = prompt_state.skill_manager
logger.info(
@@ -406,7 +647,10 @@ def build_agent_chat_executor(config=None, skills: Optional[List[str]] = None):
if backend_id == "litellm" and arch == "multi":
return build_agent_executor(config, skills=skills)
registry = get_tool_registry()
# Pass ``config`` explicitly so a reloaded ``Config`` instance observes
# the new per-category timeout map (see ``build_agent_executor`` for the
# same rationale).
registry = get_tool_registry(config)
prompt_state = resolve_skill_prompt_state(config, skills=skills)
if backend_id == "litellm":
from src.agent.llm_adapter import LLMToolAdapter

View File

@@ -379,7 +379,10 @@ Token budget remaining: ~{remaining_budget}
llm_adapter=self.llm_adapter,
max_steps=4,
max_wall_clock_seconds=timeout_seconds,
tool_call_timeout_seconds=timeout_seconds,
# NOTE: do not pass tool_call_timeout_seconds here — under the
# first-wins contract it would override the per-tool / category
# limits, letting one tool consume the whole sub-question budget.
# The remaining wall-clock budget alone keeps tools bounded.
)
if not result.success and self._looks_like_timeout_error(result.error):
return {

View File

@@ -17,10 +17,12 @@ from __future__ import annotations
import json
import logging
import math
import re
import time
import threading
import contextvars
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError, as_completed
from concurrent.futures import FIRST_COMPLETED, ThreadPoolExecutor, wait
from dataclasses import dataclass, field
from typing import Any, Callable, Dict, List, Optional
@@ -30,6 +32,7 @@ from src.agent.protocols import StageFailureReason
from src.agent.stream_events import stream_event
from src.agent.tools.registry import ToolRegistry
from src.agent.tools.execution import (
TOOL_CANCEL_EVENT,
_build_tool_cache_key,
_guard_tool_stock_scope,
_is_non_retriable_tool_result,
@@ -348,7 +351,10 @@ def run_agent_loop(
progress_callback: Optional callback receiving progress dicts.
thinking_labels: Override map of tool_name → friendly label.
max_wall_clock_seconds: Optional overall timeout budget for the loop.
tool_call_timeout_seconds: Optional timeout for one parallel tool batch.
tool_call_timeout_seconds: Optional explicit per-run tool-call timeout.
Highest-priority (first-wins) in the per-tool timeout chain —
overrides per-tool declarations and category defaults — but is
capped by ``max_wall_clock_seconds`` when both are set.
emit_stage_events: Whether to emit the synthetic ``agent_loop``
stage lifecycle. Orchestrated business stages disable this so
``stage_start`` / ``stage_done`` only describe real stages.
@@ -509,13 +515,10 @@ def run_agent_loop(
assistant_msg["provider_blocks"] = response.provider_blocks
messages.append(assistant_msg)
# Execute tools (parallel when > 1)
effective_tool_timeout = tool_call_timeout_seconds
if remaining_timeout is not None:
effective_tool_timeout = min(
remaining_timeout,
tool_call_timeout_seconds if tool_call_timeout_seconds and tool_call_timeout_seconds > 0 else remaining_timeout,
)
# Execute tools (parallel when > 1). ``tool_call_timeout_seconds`` is
# the caller's explicit per-run override — highest priority in the
# first-wins chain (Issue #1890 contract) — while ``remaining_timeout``
# is the unbreakable outer wall-clock cap for this batch.
tool_results = _execute_tools(
response.tool_calls,
tool_registry,
@@ -523,7 +526,8 @@ def run_agent_loop(
progress_callback,
tool_calls_log,
non_retriable_tool_results,
tool_wait_timeout_seconds=effective_tool_timeout,
tool_call_timeout_seconds=tool_call_timeout_seconds,
tool_wait_timeout_seconds=remaining_timeout,
stock_scope=stock_scope,
)
@@ -601,6 +605,107 @@ def run_agent_loop(
# Internal tool execution
# ============================================================
def _coerce_positive_timeout(value) -> Optional[float]:
"""Coerce a timeout candidate to a positive finite float, else ``None``.
``None`` / non-numeric / non-positive / ``inf`` / ``nan`` all map to
``None`` ("no limit at this level"). Rejecting non-finite values prevents
an ``OverflowError`` from ``future.result(timeout=inf)`` and avoids the
undefined ordering a ``nan`` would produce.
"""
if value is None:
return None
try:
v = float(value)
except (TypeError, ValueError):
return None
if not math.isfinite(v) or v <= 0:
return None
return v
def _build_timeout_result_payload(
tool_name: str,
arguments: Dict[str, Any],
timeout_s: float,
non_retriable_tool_results: Optional[Dict[str, str]],
) -> str:
"""Build the timeout-shaped result string and record it as non-retriable.
A timed-out call is marked ``"retriable": False`` and inserted into
``non_retriable_tool_results`` so the LLM's retry of the *same* call reuses
the cached failure instead of spinning up a second (possibly side-effecting)
execution — a best-effort guard against duplicate work, since Python cannot
forcibly cancel an already-started tool thread.
"""
label = f"{timeout_s:.2f}s"
result_str = json.dumps({
"error": f"Tool execution timed out after {label}",
"timeout": True,
"retriable": False,
})
if non_retriable_tool_results is not None:
cache_key = _build_tool_cache_key(tool_name, arguments)
# Non-dict / missing ``arguments`` yields ``None`` here; skip the write so
# unrelated no-arg tool calls cannot collide on a shared ``None`` key.
if cache_key:
non_retriable_tool_results[cache_key] = result_str
return result_str
def _resolve_per_tool_timeout(
tool_call,
tool_registry: Optional[ToolRegistry],
explicit_timeout: Optional[float] = None,
wall_clock_budget: Optional[float] = None,
) -> Optional[float]:
"""Resolve the effective timeout for a single tool call (Issue #1890).
Precedence is *first-wins*, exactly as confirmed in the issue contract:
1. explicit per-run timeout (``tool_call_timeout_seconds``)
2. per-tool declaration (``ToolDefinition.timeout_seconds``)
3. category default (``AGENT_*_TOOL_TIMEOUT_S``)
4. none (no per-tool limit)
An earlier level is never lowered by a later one: an explicit
``tool_call_timeout_seconds`` can *relax* a stricter per-tool or category
default (the previous ``min()`` across all levels made that impossible), and
a per-tool declaration is never overridden by a smaller category default.
``wall_clock_budget`` (the remaining overall loop budget) is then applied as
an *unbreakable outer cap*: the winner can never exceed it, so a long
explicit timeout cannot blow past the caller's overall wall-clock budget.
"""
explicit = _coerce_positive_timeout(explicit_timeout)
budget = _coerce_positive_timeout(wall_clock_budget)
if tool_registry is None:
base = explicit
else:
tool_def = tool_registry.get(tool_call.name)
if tool_def is None:
base = explicit
else:
per_tool = _coerce_positive_timeout(getattr(tool_def, "timeout_seconds", None))
category = _coerce_positive_timeout(
tool_registry.category_default_timeout(tool_def.category)
)
# First-wins: explicit per-run > per-tool declaration > category default.
base = (
explicit
if explicit is not None
else (per_tool if per_tool is not None else category)
)
if base is None:
# No explicit/per-tool/category limit; the wall-clock budget governs.
return budget
if budget is not None:
return min(base, budget)
return base
def _execute_tools(
tool_calls,
tool_registry: ToolRegistry,
@@ -608,12 +713,23 @@ def _execute_tools(
progress_callback: Optional[Callable],
tool_calls_log: List[Dict[str, Any]],
non_retriable_tool_results: Optional[Dict[str, str]] = None,
tool_call_timeout_seconds: Optional[float] = None,
tool_wait_timeout_seconds: Optional[float] = None,
stock_scope: Optional[StockScope] = None,
) -> List[Dict[str, Any]]:
"""Execute one or more tool calls, returning ordered result dicts.
Single tools run inline; multiple tools run in parallel threads.
A single tool with no resolved timeout runs inline (no thread); otherwise
all tools share one executor whose per-tool timeouts are enforced by a
deadline-driven wait loop — there are no per-tool nested pools, so a batch
of N tools uses at most ``min(N, 5)`` threads (review: unify the single and
parallel timeout wrapping). ``tool_wait_timeout_seconds`` also caps the
whole batch as an outer ceiling.
``tool_call_timeout_seconds`` is the caller's explicit per-run override
(highest priority, first-wins) and ``tool_wait_timeout_seconds`` the
remaining wall-clock budget (outer cap) — both feed
:func:`_resolve_per_tool_timeout`.
"""
def _exec_single(tc_item):
@@ -624,50 +740,39 @@ def _execute_tools(
non_retriable_tool_results=non_retriable_tool_results,
)
results: List[Dict[str, Any]] = []
def _exec_with_deadline(tc_item, per_tool_timeout, deadline_holder):
"""Run one tool and record its per-tool deadline when the worker actually
starts. The batch pool is capped at ``min(N, 5)`` workers, so calls that
queue behind a full pool must not burn their timeout before execution
begins (review OR-COM-3d6b61f8) — otherwise a batch larger than five can
falsely time out a tool that never got a worker.
"""
if per_tool_timeout and per_tool_timeout > 0:
deadline_holder[0] = time.monotonic() + per_tool_timeout
return _exec_single(tc_item)
if len(tool_calls) == 1:
tc = tool_calls[0]
if progress_callback:
progress_callback(stream_event("tool_start", step=step, tool=tc.name))
timeout_triggered = False
if tool_wait_timeout_seconds and tool_wait_timeout_seconds > 0:
pool = ThreadPoolExecutor(max_workers=1)
ctx = contextvars.copy_context()
try:
future = pool.submit(ctx.run, _exec_single, tc)
try:
_, result_str, success, dur, cached, guard_result = future.result(timeout=tool_wait_timeout_seconds)
except FuturesTimeoutError:
timeout_triggered = True
future.cancel()
timeout_label = f"{tool_wait_timeout_seconds:.2f}s"
logger.warning("Tool '%s' timed out after %s at step %d", tc.name, timeout_label, step)
result_str = json.dumps({
"error": f"Tool execution timed out after {timeout_label}",
"timeout": True,
})
success = False
dur = round(tool_wait_timeout_seconds, 2)
cached = False
guard_result = None
finally:
pool.shutdown(wait=not timeout_triggered, cancel_futures=timeout_triggered)
def _record(tc_item, *, timed_out, timeout_s=None, out=None, guard_result=None):
"""Emit ``tool_done``, build the log entry, and append to ``results``."""
if out is not None:
_, result_str, success, dur, cached, guard_result = out
else:
_, result_str, success, dur, cached, guard_result = _exec_single(tc)
result_str = _build_timeout_result_payload(
tc_item.name, tc_item.arguments, timeout_s, non_retriable_tool_results,
)
success = False
dur = round(timeout_s, 2)
cached = False
if progress_callback:
progress_callback(stream_event("tool_done", step=step, tool=tc.name, success=success, duration=dur))
progress_callback(stream_event(
"tool_done", step=step, tool=tc_item.name, success=success, duration=dur,
))
log_entry = {
"step": step, "tool": tc.name, "arguments": tc.arguments,
"step": step, "tool": tc_item.name, "arguments": tc_item.arguments,
"success": success, "duration": dur, "result_length": len(result_str),
"cached": cached,
}
if tool_wait_timeout_seconds and tool_wait_timeout_seconds > 0 and not success:
try:
if json.loads(result_str).get("timeout") is True:
log_entry["timeout"] = True
except (TypeError, ValueError, json.JSONDecodeError):
pass
if timed_out:
log_entry["timeout"] = True
if guard_result is not None:
log_entry.update({
"guarded": True,
@@ -676,74 +781,109 @@ def _execute_tools(
"allowed_stock_codes": guard_result.get("allowed_stock_codes", []),
})
tool_calls_log.append(log_entry)
results.append({"tc": tc, "result_str": result_str})
else:
for tc in tool_calls:
if progress_callback:
progress_callback(stream_event("tool_start", step=step, tool=tc.name))
results.append({"tc": tc_item, "result_str": result_str})
pool = ThreadPoolExecutor(max_workers=min(len(tool_calls), 5))
timeout_triggered = False
try:
futures = {pool.submit(contextvars.copy_context().run, _exec_single, tc): tc for tc in tool_calls}
pending = set(futures)
for future in as_completed(
futures,
timeout=tool_wait_timeout_seconds if tool_wait_timeout_seconds and tool_wait_timeout_seconds > 0 else None,
):
pending.discard(future)
tc_item, result_str, success, dur, cached, guard_result = future.result()
if progress_callback:
progress_callback(stream_event("tool_done", step=step, tool=tc_item.name, success=success, duration=dur))
log_entry = {
"step": step, "tool": tc_item.name, "arguments": tc_item.arguments,
"success": success, "duration": dur, "result_length": len(result_str),
"cached": cached,
}
if guard_result is not None:
log_entry.update({
"guarded": True,
"expected_stock_code": guard_result.get("expected_stock_code"),
"requested_stock_code": guard_result.get("requested_stock_code"),
"allowed_stock_codes": guard_result.get("allowed_stock_codes", []),
})
tool_calls_log.append(log_entry)
results.append({"tc": tc_item, "result_str": result_str})
except FuturesTimeoutError:
timeout_triggered = True
timeout_label = (
f"{tool_wait_timeout_seconds:.2f}s"
if tool_wait_timeout_seconds is not None
else "the configured limit"
results: List[Dict[str, Any]] = []
if not tool_calls:
return results
plan = []
for tc in tool_calls:
per_tool_timeout = _resolve_per_tool_timeout(
tc, tool_registry, tool_call_timeout_seconds, tool_wait_timeout_seconds,
)
plan.append((tc, per_tool_timeout))
if progress_callback:
progress_callback(stream_event("tool_start", step=step, tool=tc.name))
# Fast path: a single tool with no resolved timeout runs inline (no thread).
if len(plan) == 1 and not (plan[0][1] and plan[0][1] > 0):
_record(plan[0][0], timed_out=False, out=_exec_single(plan[0][0]))
return results
pool = ThreadPoolExecutor(max_workers=min(len(plan), 5))
futures: Dict[Any, Any] = {}
cancel_of: Dict[Any, threading.Event] = {}
# ``deadline_of`` maps a Future to a single-element holder list; the worker
# writes the deadline into it when it *starts* running (see
# ``_exec_with_deadline``). ``None`` until then means "still queued / no
# per-tool timeout" and is never treated as expired.
deadline_of: Dict[Any, List[Optional[float]]] = {}
timeout_of: Dict[Any, float] = {}
timeout_triggered = False
try:
# One executor for the whole batch. Each future gets its own cancel
# event (keyed by Future, not by tool name, so two parallel calls to the
# same tool cannot shadow each other) and its own per-tool deadline.
for tc, per_tool_timeout in plan:
cancel_event = threading.Event()
ctx = contextvars.copy_context()
ctx.run(TOOL_CANCEL_EVENT.set, cancel_event)
deadline_holder: List[Optional[float]] = [None]
fut = pool.submit(
ctx.run, _exec_with_deadline, tc, per_tool_timeout, deadline_holder,
)
logger.warning("Tool batch timed out after %s at step %d", timeout_label, step)
for future, tc_item in futures.items():
if future in pending:
future.cancel()
result_str = json.dumps({
"error": f"Tool execution timed out after {timeout_label}",
"timeout": True,
})
if progress_callback:
progress_callback(stream_event(
"tool_done",
step=step,
tool=tc_item.name,
success=False,
duration=round(tool_wait_timeout_seconds or 0.0, 2),
))
tool_calls_log.append({
"step": step,
"tool": tc_item.name,
"arguments": tc_item.arguments,
"success": False,
"duration": round(tool_wait_timeout_seconds or 0.0, 2),
"result_length": len(result_str),
"cached": False,
"timeout": True,
})
results.append({"tc": tc_item, "result_str": result_str})
finally:
pool.shutdown(wait=not timeout_triggered, cancel_futures=timeout_triggered)
futures[fut] = tc
cancel_of[fut] = cancel_event
timeout_of[fut] = per_tool_timeout or 0.0
deadline_of[fut] = deadline_holder
batch_deadline = (
time.monotonic() + tool_wait_timeout_seconds
if tool_wait_timeout_seconds and tool_wait_timeout_seconds > 0
else None
)
pending = set(futures)
while pending:
now = time.monotonic()
deadlines = []
for f in pending:
holder = deadline_of.get(f)
if holder is not None and holder[0] is not None:
deadlines.append(holder[0])
if batch_deadline is not None:
deadlines.append(batch_deadline)
next_deadline = min(deadlines) if deadlines else None
if next_deadline is not None:
wait_timeout = max(0.0, next_deadline - now)
elif any(timeout_of.get(f, 0.0) > 0 for f in pending):
# A pending call has a per-tool timeout but its worker has not
# started yet (it is queued behind the capped pool). Poll briefly
# so a tool that starts and then hangs still gets its own timeout
# (review OR-COM-3d6b61f8) instead of being forgotten.
wait_timeout = 0.01
else:
# No deadline and no pending call has a timeout — nothing can time
# out, so block until a call completes (efficient for the default
# no-timeout path).
wait_timeout = None
done, _ = wait(pending, timeout=wait_timeout, return_when=FIRST_COMPLETED)
for fut in done:
pending.discard(fut)
_record(futures[fut], timed_out=False, out=fut.result())
now = time.monotonic()
for fut in list(pending):
holder = deadline_of.get(fut)
deadline = holder[0] if holder is not None else None
expired = (deadline is not None and now >= deadline) or (
batch_deadline is not None and now >= batch_deadline
)
if not expired:
continue
pending.discard(fut)
cancel_of[fut].set()
timeout_triggered = True
timeout_s = timeout_of.get(fut) or (tool_wait_timeout_seconds or 0.0)
logger.warning(
"Tool '%s' timed out after %.2fs at step %d",
futures[fut].name, timeout_s, step,
)
_record(futures[fut], timed_out=True, timeout_s=timeout_s)
finally:
# Do not wait when a timeout fired — the timed-out handler may still be
# running in a worker and would otherwise block this call indefinitely.
pool.shutdown(wait=not timeout_triggered, cancel_futures=True)
return results

View File

@@ -21,6 +21,39 @@ from src.agent.tools.registry import ToolRegistry
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Cooperative cancellation signal (best-effort timeouts, Issue #1890 / review)
# ---------------------------------------------------------------------------
# Python cannot forcibly stop an already-started tool thread, so when a tool
# times out its handler may keep running in the background. To give long or
# side-effecting handlers a way to honour the timeout, the runner arms a
# per-call ``threading.Event`` on timeout and publishes it through this
# contextvar for the duration of the call. Handlers MAY poll
# ``is_tool_cancellation_requested`` and abort early; the same signal is also
# honoured by the existing ``check_tool_execution()`` checkpoint that data /
# backtest / tool-surface handlers already call, so a timed-out handler stops at
# its next safe boundary even without opting into the helper. The signal is
# strictly opt-in by checkpoint: handlers that never call either are completely
# unaffected, and the runner only sets the event after a timeout has already
# fired — never during normal completion. This is the "in-handler cooperative
# cancel" mitigation requested in review, layered on top of the non-retriable
# cache (which blocks the LLM from re-launching the same call).
TOOL_CANCEL_EVENT: "contextvars.ContextVar[Optional[threading.Event]]" = contextvars.ContextVar(
"tool_cancel_event", default=None
)
def is_tool_cancellation_requested() -> bool:
"""Return True when the current tool call has been asked to cancel.
Tool handlers may call this cheaply inside long loops or before performing
an irreversible side effect, to honour a best-effort timeout. Returns False
unless the runner has armed the cancellation event for the running call.
"""
event = TOOL_CANCEL_EVENT.get()
return event is not None and event.is_set()
_SUMMARY_LIMIT = 500
_TOKEN_PATTERN = re.compile(
r"(?i)\b(?:sk|pk|ghp|gho|github_pat|xox[baprs]?|bearer)[-_a-z0-9]{12,}\b"
@@ -100,12 +133,20 @@ def reset_tool_execution_context(token: contextvars.Token) -> None:
def check_tool_execution() -> None:
"""Stop at a safe handler boundary when cancellation or deadline is reached."""
context = _ACTIVE_TOOL_CONTEXT.get()
if context is None:
return
if context.cancel_event is not None and context.cancel_event.is_set():
raise ToolExecutionCancelled("Tool execution was cancelled")
if context.deadline is not None and time.monotonic() >= context.deadline:
raise ToolExecutionDeadlineExceeded("Tool execution deadline was exceeded")
if context is not None:
if context.cancel_event is not None and context.cancel_event.is_set():
raise ToolExecutionCancelled("Tool execution was cancelled")
if context.deadline is not None and time.monotonic() >= context.deadline:
raise ToolExecutionDeadlineExceeded("Tool execution deadline was exceeded")
# Runner-armed cooperative cancel (Issue #1890): when a per-tool timeout
# fires, the runner arms ``TOOL_CANCEL_EVENT`` for the running call. Real
# tool handlers (data/backtest/tool-surface) poll this checkpoint rather than
# the opt-in ``is_tool_cancellation_requested()`` helper, so honour the signal
# here too — a still-running handler then aborts before its next side-effecting
# step instead of running to completion in the background thread.
cancel_event = TOOL_CANCEL_EVENT.get()
if cancel_event is not None and cancel_event.is_set():
raise ToolExecutionCancelled("Tool execution timed out (cooperative cancel)")
def serialize_tool_result(result: Any) -> str:

View File

@@ -44,6 +44,7 @@ class ToolPolicy:
policy_status: str = "unknown"
scope_dimensions: List[str] = field(default_factory=list)
cancellation_safe: bool = False
timeout_seconds: Optional[float] = None
@classmethod
def unknown(cls) -> "ToolPolicy":
@@ -58,6 +59,7 @@ class ToolPolicy:
permissions: Optional[List[str]] = None,
scope_dimensions: Optional[List[str]] = None,
cancellation_safe: bool = False,
timeout_seconds: Optional[float] = None,
) -> "ToolPolicy":
return cls(
read_only=read_only,
@@ -66,6 +68,7 @@ class ToolPolicy:
policy_status="declared",
scope_dimensions=list(scope_dimensions or []),
cancellation_safe=bool(cancellation_safe),
timeout_seconds=timeout_seconds,
)
def to_public_dict(self) -> Dict[str, Any]:
@@ -85,8 +88,9 @@ class ToolDefinition:
description: str
parameters: List[ToolParameter]
handler: Callable
category: str = "data" # data | analysis | search | action
category: str = "data" # data | analysis | search | action | market
policy: ToolPolicy = field(default_factory=ToolPolicy.unknown)
timeout_seconds: Optional[float] = None # Optional per-tool execution timeout
# ----- Multi-provider schema converters -----
@@ -172,8 +176,9 @@ class ToolRegistry:
registry.execute("get_realtime_quote", stock_code="600519")
"""
def __init__(self):
def __init__(self, category_timeout_map: Optional[Dict[str, float]] = None):
self._tools: Dict[str, ToolDefinition] = {}
self._category_timeouts: Dict[str, float] = dict(category_timeout_map or {})
# ----- Registration -----
@@ -215,6 +220,24 @@ class ToolRegistry:
def __contains__(self, name: str) -> bool:
return name in self._tools
def category_default_timeout(self, category: str) -> Optional[float]:
"""Return the configured default timeout (seconds) for a tool category.
Returns ``None`` when no default is configured, letting the caller fall
back to the global ``tool_call_timeout_seconds`` budget.
"""
return self._category_timeouts.get(category)
@property
def category_timeout_map(self) -> Dict[str, float]:
"""Return a copy of the per-category default timeout map.
Used by :meth:`ToolRegistry` consumers that build a filtered registry
(e.g. ``BaseAgent._filtered_registry``) so the category ceilings survive
the subset copy (review OR-COM-7f3d3f5b).
"""
return dict(self._category_timeouts)
# ----- Schema generation -----
def to_openai_tools(self) -> List[dict]:
@@ -315,6 +338,7 @@ def tool(
parameters: Optional[List[ToolParameter]] = None,
registry: Optional[ToolRegistry] = None,
policy: Optional[ToolPolicy] = None,
timeout_seconds: Optional[float] = None,
):
"""Decorator to register a function as an agent tool.
@@ -333,13 +357,29 @@ def tool(
if params is None:
params = _infer_parameters(func)
# Single source of truth for per-tool timeout is
# ``ToolDefinition.timeout_seconds`` (the field ``runner`` actually
# reads). If the caller supplied a ``timeout_seconds`` via the policy
# but omitted the explicit ``@tool(timeout_seconds=...)`` argument, fold
# the policy value in here so the two never diverge.
effective_timeout = timeout_seconds
resolved_policy = policy or ToolPolicy.unknown()
if effective_timeout is None and getattr(resolved_policy, "timeout_seconds", None) is not None:
effective_timeout = resolved_policy.timeout_seconds
logger.debug(
"Tool '%s': using ToolPolicy.timeout_seconds=%s as the effective per-tool timeout",
name,
effective_timeout,
)
tool_def = ToolDefinition(
name=name,
description=description,
parameters=params,
handler=func,
category=category,
policy=policy or ToolPolicy.unknown(),
policy=resolved_policy,
timeout_seconds=effective_timeout,
)
target_registry = registry or get_default_registry()

View File

@@ -1014,6 +1014,13 @@ class Config:
agent_decision_agent_timeout_s: float = 0
agent_portfolio_agent_timeout_s: float = 0
agent_skill_agent_timeout_s: float = 0
# Per-category default timeouts for agent tool calls (seconds).
# 0 / unset means "no category default" -> falls back to the global
# tool_call_timeout_seconds budget.
agent_data_tool_timeout_s: float = 0.0
agent_search_tool_timeout_s: float = 0.0
agent_analysis_tool_timeout_s: float = 0.0
agent_action_tool_timeout_s: float = 0.0
agent_skill_concurrency: int = 3
agent_risk_override: bool = True # Allow risk agent to veto buy signals
agent_deep_research_budget: int = 30000 # Max token budget for deep research
@@ -1946,6 +1953,22 @@ class Config:
os.getenv('AGENT_SKILL_AGENT_TIMEOUT_S'), 0,
field_name='AGENT_SKILL_AGENT_TIMEOUT_S', minimum=0,
),
agent_data_tool_timeout_s=parse_env_float(
os.getenv('AGENT_DATA_TOOL_TIMEOUT_S'), 0.0,
field_name='AGENT_DATA_TOOL_TIMEOUT_S', minimum=0.0,
),
agent_search_tool_timeout_s=parse_env_float(
os.getenv('AGENT_SEARCH_TOOL_TIMEOUT_S'), 0.0,
field_name='AGENT_SEARCH_TOOL_TIMEOUT_S', minimum=0.0,
),
agent_analysis_tool_timeout_s=parse_env_float(
os.getenv('AGENT_ANALYSIS_TOOL_TIMEOUT_S'), 0.0,
field_name='AGENT_ANALYSIS_TOOL_TIMEOUT_S', minimum=0.0,
),
agent_action_tool_timeout_s=parse_env_float(
os.getenv('AGENT_ACTION_TOOL_TIMEOUT_S'), 0.0,
field_name='AGENT_ACTION_TOOL_TIMEOUT_S', minimum=0.0,
),
agent_skill_concurrency=parse_env_int(
os.getenv('AGENT_SKILL_CONCURRENCY'),
3,

View File

@@ -313,9 +313,15 @@ class SystemConfigService:
@staticmethod
def _reload_runtime_singletons() -> None:
"""Reset runtime singleton services after config reload."""
from src.agent.factory import reset_tool_registry
from src.agent.tools.data_tools import reset_fetcher_manager
from src.search_service import reset_search_service
# Drop the module-level ToolRegistry so the next
# ``build_agent_executor`` / ``build_agent_chat_executor`` call
# rebuilds ``_TOOL_REGISTRY`` against the fresh Config and picks up
# new ``AGENT_*_TOOL_TIMEOUT_S`` overrides (Issue #1890).
reset_tool_registry()
reset_fetcher_manager()
reset_search_service()

File diff suppressed because it is too large Load Diff