* feat(runtime): partial notify and diagnostics after scheduler timeout
After a hard timeout, scan already-saved analyses and enrich last_error
with completed/pending counts; optional push via DSA_TIMEOUT_PARTIAL_NOTIFY.
Refs #2328
* test(runtime): cover timeout partial delivery helpers
Refs #2328
* docs: document DSA_TIMEOUT_PARTIAL_NOTIFY
Refs #2328
* fix(config): use switch ui_control for timeout partial notify
DSA_TIMEOUT_PARTIAL_NOTIFY used ui_control=toggle, which SystemConfigResponse rejects and broke GET /config in backend-tests 1/3.
* docs(runtime): document timeout partial fail-open for operators
Channel exceptions are swallowed after the analysis lock is released, so they cannot keep status.running true. Collect/import failures stay in warning logs because last_error cannot distinguish them from zero completions.
* feat: add bot index analysis entry
* test: add Bot index online smoke
* fix: guard Windows-only spawn flag in smoke tests for Linux CI
* fix: clean up worker tree on parent interrupt in smoke runner
* fix: surface interrupt cleanup failures in smoke runner
---------
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
* feat: make self-hosted SearXNG search timeout configurable
SEARXNG_TIMEOUT_SECONDS (default 10s, min 1) now controls the per-search
timeout for self-hosted SearXNG instances, threaded through all three
SearchService construction sites. Public-instance timeout stays fixed.
* fix: thread SEARXNG_TIMEOUT_SECONDS through subprocess rebuild and Actions
- persist searxng_timeout_seconds in _constructor_kwargs so the bounded
topic-news subprocess rebuild no longer silently falls back to 10s
- map SEARXNG_TIMEOUT_SECONDS (vars||secrets) in 00-daily-analysis.yml
- document the variable in full-guide/full-guide_EN data-source tables,
add CHANGELOG entry, and add regression tests for both paths
* fix: document SearXNG timeout environment contract
* fix: register SearXNG timeout setting
* fix: keep SearXNG timeout env-only
---------
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
Co-authored-by: zhulinsen <zhuls97@163.com>
* feat: add data capability contract
* fix(review-feedback-2289): preserve unknown status until availability is checked and Make
* fix(review-feedback-2289): Aggregate daily quality across supported markets and add kline
* fix(review-feedback-2289): Honor daily-source circuit breakers in quality selection and Make
* fix(review-feedback-2289): Scope market-overview quality by market and Do not select a news
* fix: align data capability with runtime routes
* fix: align data capability runtime coverage
* fix: align source and index capability routes
* fix: preserve runtime capability uncertainty
* fix: include US index capability routes
* fix: align realtime and monitor capabilities
* fix: remove unsupported Tushare index capability
* fix: align capabilities with runtime routes
* fix: model realtime and breaker routes
* fix: align US realtime request priority
* fix: align realtime circuit coverage
* fix: filter unavailable daily priorities
* fix: align US realtime capability claims
* fix: align executable US data routes
* feat: add one-shot index analysis via --stocks with structured AnalysisTarget pipeline
* fix: render index data sources in aggregate reports
* fix: refresh index registry before --stocks parsing and filter indices on CN holidays
---------
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
* feat: add Futu OpenD as an optional HK realtime and fundamental data source
Add FutuFetcher and FutuFundamentalAdapter behind FUTU_OPEND_HOST/PORT,
register the settings in Config and config_registry so the Web settings
page can expose host, port and HK realtime priority, and route HK
realtime quotes through a configurable futu/longbridge/akshare/yfinance
order while keeping A-share priority untouched. Include offline tests
for the adapter, config schema and HK routing/fallback, plus docs and
CHANGELOG entries.
* fix: wire Futu fundamentals into HK pipeline and restore quote supplementation
- _fetch_offshore_fundamental_bundle() prefers the Futu fundamental
adapter for HK when FUTU_OPEND_HOST is configured, and falls back to
yfinance when Futu is absent or returns no usable content.
- HK realtime priority loop now supplements missing quote fields
(volume_ratio / turnover_rate / pe/pb / market cap) from later
configured sources instead of returning after the first non-empty
quote, matching the US path's _supplement_quote behavior.
- capital_flow / boards blocks are filled from the Futu bundle for HK
instead of being hard-coded not_supported; status and missing_fields
aggregation updated accordingly.
- Add regression tests for partial-quote supplementation and Futu
fundamental bundle routing/fallback.
* test: expect boards block ok when bundle provides belong_boards
The Futu integration made the offshore boards block data-driven instead
of hard-coded not_supported; update the existing US/HK fundamental
context test to match (belong_boards from the bundle now surface as an
ok boards block).
* fix: preserve HK fallback_from metadata and normalize Futu quote timestamps
- HK realtime priority loop now records the failed preferred source token
and passes it as fallback_from when a later source takes over, so the
pipeline and analysis context can mark the quote as degraded.
- Futu snapshot update_time is a naive Beijing-time (UTC+8) string; attach
the +08:00 offset before storing provider_timestamp so stale_seconds /
is_stale / provider_timestamp freshness semantics are correct instead of
being parsed as UTC.
- Add regression tests for fallback_from propagation and timestamp
normalization.
* fix: normalize Futu belong_boards to name/type/code contract
OpenD owner_plate returns plate_code / plate_name / plate_type, but DSA
downstream consumers (notification, extract_board_detail_fields, market
structure) only read name/type/code. Map the fields in
FutuFundamentalAdapter._boards so HK Futu boards are actually consumed
instead of silently dropped, and add regression tests including an
end-to-end check through extract_board_detail_fields.
* fix: merge yfinance bundle when Futu fundamental returns partial blocks
Futu partial success (e.g. statements failed but static info worked) used
to short-circuit the whole bundle, silently dropping the growth/earnings
that the existing yfinance path could still provide. Now, when Futu
returns content but is missing growth or earnings, fetch the yfinance
bundle within the remaining budget and merge the missing blocks
(growth/earnings/institution/capital_flow/belong_boards), keeping
Futu-preferred values where both exist. Add regression test for the
partial-success merge path.
* fix: use field-level checks when deciding Futu-vs-yfinance growth/earnings
The previous merge condition only checked dict truthiness, so a truthy
growth/earnings shell (all-None core values or metadata-only keys such
as report_date/period/currency) would skip the yfinance supplement and
silently downgrade existing HK fundamentals. Add _earnings_block_has_values
(a core numeric field or a populated dividend is required) and reuse the
existing _has_meaningful_payload for growth; both the missing_core check
and the merge loop now use these. Add regression test for the
all-None-shell scenario.
* fix: fill HK fundamental field gaps from yfinance instead of block-level checks
Block-level meaningful checks still skipped the yfinance supplement when
Futu hit only part of the growth/earnings fields (e.g. revenue_yoy but
None net_profit_yoy, or earnings with only basic_eps), silently dropping
fields the main branch used to provide. Replace the missing_core decision
with a per-field gap list (growth: revenue_yoy/net_profit_yoy/gross_margin;
earnings.financial_report: revenue/net_profit_parent/basic_eps/gross_profit)
and make the merge field-level: keep Futu values, fill each missing field
from yfinance. Add regression tests for partial-hit and all-None shells.
* fix: normalize Futu dividends to the repo contract and treat dividend gaps as supplement triggers
Futu OpenD dividend_list carries raw fields (statement/ex_date/record_date)
which the notification/data_processing market-structure consumers do not
read; the repo contract is ttm_cash_dividend_per_share,
ttm_dividend_yield_pct and events[].cash_dividend_per_share /
ex_dividend_date / event_date. Normalize events in
FutuFundamentalAdapter._dividends_and_splits, compute TTM count/cash and
yield from the latest quote, and teach _field_gaps/_merge_bundles to treat
a dividend block that does not satisfy the contract as a gap so yfinance
supplements it. Also dedupe FUTU_OPEND_HOST/PORT in full-guide_EN.
* fix: read dividend yield price from UnifiedRealtimeQuote objects
FutuFetcher.get_realtime_quote returns a UnifiedRealtimeQuote dataclass,
not a dict, so the yield branch in _dividends_and_splits that guarded on
isinstance(quote, dict) never ran on the live Futu path, silently dropping
ttm_dividend_yield_pct while the contract check considered the dividend
block complete. Read price via getattr(quote, 'price', None) and keep the
dict fallback for other fetchers; add a regression test driving the real
UnifiedRealtimeQuote shape.
* fix: treat dividend blocks with TTM cash but no yield as supplement gaps
The repo contract consumes ttm_cash_dividend_per_share and
ttm_dividend_yield_pct together. When the Futu dividend path has events
and TTM cash but the extra realtime price snapshot failed (quote None /
no price), ttm_dividend_yield_pct cannot be computed and the block was
previously treated as complete, so yfinance was never consulted and the
notification rendered the yield as N/A.
_dividend_contract_has_values() now requires the paired yield whenever
TTM cash is present, so _field_gaps() triggers the yfinance supplement
and _merge_bundles() replaces the incomplete dividend block.
Add regression tests for the adapter-level gap shape (quote unavailable
leaves no yield) and the manager-level supplement path (Futu cash
without yield pulls yfinance and fills the yield).
* fix: skip unconfigured Futu in HK realtime routing
When FUTU_OPEND_HOST is not configured, the HK realtime priority loop
used to still attempt the futu source, record it as the failed primary,
and attach fallback_from='futu' to a successful quote from the next
enabled source (longbridge/akshare/yfinance). Consumers then wrongly
treated an enabled source's first success as degraded fallback data,
contradicting the documented contract that Futu only participates when
OpenD is configured.
The HK loop now checks FutuFetcher.has_configured_endpoint() once and
skips the futu token entirely when it is disabled, so no fallback_from
is written. Existing configured-Futu routing tests explicitly patch the
endpoint check; a new regression test asserts an unconfigured Futu is
never called and the enriched quote carries fallback_from=None.
* fix: release cached HK Futu fundamental fetcher in DataFetcherManager.close()
The HK Futu fundamental path lazily creates and caches its own
FutuFetcher (an OpenQuoteContext-backed OpenD connection) on
_futu_fundamental_fetcher, but close() only released the TickFlow
fetcher and the default fetchers snapshot. Explicit close / reload
paths therefore left the OpenD connection hanging.
close() now takes the cached _futu_fundamental_fetcher, clears the
reference and calls its close() best-effort. A regression test injects
an observable fetcher into _futu_fundamental_fetcher and asserts
close() invokes it and clears the attribute.
---------
Co-authored-by: BayMax local review <baymax-local@invalid>
* fix: render CJK text in Linux share images
* fix(review-feedback-2270): 补到 PR 附件或评论里，和本次渲染修复说明保持一致
* fix: preserve report language contract in share images
* feat: add canonical_id column and dual-write for stock daily data (#2207)
* fix: unify bare index codes to index canonical_id via matched_index (review OR-COR-4f9ffc38)
* docs: update docstrings for index-aware derivation and fix orphaned inspection-failure test (review OR-COR-4f9ffc38)
* fix: report actual market review backend
* fix: preserve exhausted fallback diagnostics
* fix(review-feedback-2241): update the attempt loop to retain the current model even when the and
* fix: preserve resolved market review provider
* fix(review-feedback-2241): Resolve aliased LiteLLM providers before recording
* fix(review-feedback-2241): preserve the failed LiteLLM route's resolved provider
* fix(review-feedback-2241): preserve template fallback for primary LiteLLM exhaustion and Use the
* fix(review-feedback-2241): preserve the fallback provider for unqualified response models
* fix(review-feedback-2241): Derive exhausted aliases from the router's last deployment and
* fix(review-feedback-2241): preserve the provider for gateway-owned slash model IDs
* fix(review-feedback-2241): preserve the explicit provider for unqualified failure models
* fix(review-feedback-2241): fix several earlier route-alias and router-failure diagnostics gaps,
* fix(review-feedback-2241): 评审结论 - 代码检查 ：当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要：Full
* fix(review-feedback-2241): 补一组成功路径回归：同一 alias 下首个 deployment 为 openai/~...、后续 deployment 为直连
* fix(review-feedback-2241): add a regression test that injects an analyzer exposing only is
* fix(review-feedback-2241): add a regression that asserts the legacy injected-analyzer path does
* fix(review-feedback-2241): 评审结论 - 代码检查 ：当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要：基于当前 HEAD
* fix(review-feedback-2241): 评审结论 - 代码检查 ：当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要：基于完整
* fix(review-feedback-2241): add a regression test that drives call litellm
* fix(issue-2201): [bug]-分析后无报告生成
* fix(review-feedback-2211): Keep the TTM cutoff anchored to the as-of date and Propagate report
* fix(review-feedback-2211): Move the empty-watchlist check before the trading-day return
* fix(review-feedback-2211): preserve the failure status, but defer returning until after the and
* fix(review-feedback-2211): preserve the failure result, but defer returning until after the
* fix(review-feedback-2211): Handle failures in the market-review-only CLI branch
* feat: 新增 STOCK_LIST 单条目解析契约（issue #2063 Phase 1）
新增 src/services/stock_list_parser.py 中的 parse_analysis_target() 单条目解析入口，
按 issue #2063 Phase 1 三段契约实现：
1. 前缀白名单指数：sh/sz 前缀且命中 IndexRegistry 的代码 → INDEX，
canonical_id 同步指数稳定 ID（sh000300→sh000300、sz399001→sz399001）；
2. 裸码默认个股：未带前缀的代码一律 STOCK，即使裸码与已知指数代码冲突
（000300、000016 等）也仅通过 matched_index 暴露冲突，不翻转 asset_type；
3. 前缀未命中降级为股票：sh/sz/bj/hk/us 前缀但 registry 未收录的代码 → STOCK，
不再产生 UNSUPPORTED，避免把 typo 或新代码误判为不可处理。
对外暴露 IndexRegistry、IndexEntry、AnalysisTarget、ParseStatus 以及
default_index_registry()（默认收录 sh000300、sh000016、sh000688、sz399001、
sz399006 五个核心指数），上层可注入自定义 registry 扩展白名单。
保留 split_stock_list() / serialize_stock_list() 两个 legacy helper 的签名与
行为不变，tests/test_stock_list_parser.py 在原有两个 legacy 测试的基础上新增
36 个测试覆盖三条契约、6 个 maintainer spec 样例（sh000300、sz399300、sh600519、
000300、000001、920xxx）及边界场景，全部通过。
* fix: 修复 stock_list_parser 三个 review blocker
修复 maintainer 在 PR #2094 (issue #2063 phase 1) review
中指出的三个解析正确性问题：
1. OR-COR-d24a4e9a — 美股 ticker 前缀冲突
_split_prefix 在切前缀时把 1-5 字母的裸美股 ticker 误判
为 (前缀, 剩余) 形式：SHOP -> (sh, OP)、HKD -> (hk, D)、
BJRI -> (bj, RI)、USM -> (us, M)、SHAK -> (sh, AK)、
USFD -> (us, FD)。修复：在切前缀扫描之前，整串为 1-5
ASCII 大写字母时直接短路返回 (None, token)，让它走裸码
分支的 US 路径。isupper() 是关键鉴别器——大写字母才符
合仓库 is_code_like 的 US ticker 形状 '^[A-Z]{1,5}$'，
混合大小写 (usAAPL) 与含数字码 (sh000300) 仍走前缀拆分
以保留 contract #3 的「prefix supplied → degrade to
stock」语义。
2. OR-COR-1b643ee6 — 裸 A 股 ETF 路由错误
_classify_bare_code 把 51/52/56/58/15/16/18 开头的 6 位
裸 ETF 码统一归到 'CN/STOCK'，_canonicalize_for_stock
随后合成出 'cn510300' / 'cn159915'，没有任何上游 fetcher
接受这种形式。修复：在 6 位分支增加与
data_provider/baostock_fetcher.py、data_provider/
yfinance_fetcher.py 以及 data_provider/base.py 中
ETF_PREFIXES 一致的前缀路由——51/52/56/58 -> SH，
15/16/18 -> SZ——canonical_id 直接生成 'sh510300' /
'sz159915'，可被 BaostockFetcher._convert_stock_code
原样接收（往返一致）。
3. OR-COR-403bd018 — 空 IndexRegistry 被静默覆盖
parse_analysis_target 用 'registry = registry or
default_index_registry()'，导致显式传入的 IndexRegistry([])
(空白名单配置) 被 falsy 短路替换为默认 registry，
sh000300 / sz399001 仍被 elevate 为 index，调用方配置
被破坏。修复：改为 'if registry is None: registry =
default_index_registry()' 显式 None 判断，空白名单被尊重
为合法的「不 whitelist 指数」配置——sh000300 在空 registry
下按 contract #3 degrade 为 stock；默认 registry 行为保持
不变 (sh000300 仍为 index)。
测试：新增 TestReviewBlockerRegressions 覆盖以上三条回归用
例 (USFD/SHAK/BJDX/SZKMY + 维持用例 SHOP/HKD/BJRI/USM/AAPL
/TSLA/BRK/A/Z；510300/159915/510050/520000/562000/588000/
159919/160000/164000/184000 + 与 BaostockFetcher 的往返一致
性；IndexRegistry([]) 下 sh000300/sz399001 degrade、默认
registry 下 sh000300 仍 elevate、自定义子集 registry 仅
匹配子集)。原 130 tests 保持全过，无回归。
属于 #2063 phase 1 收尾
* fix: make parse_analysis_target reuse normalize_code for lowercase/suffix inputs
PR #2094 (issue #2063 Phase 1) addressed three review blockers, but
maintainer flagged a fourth: lowercase/mixed-case/suffix inputs were
misrouted.
Before this commit:
- 'shop' / 'hkd' / 'aapl' (lowercase) was caught by the 2-char prefix
scan in _split_prefix and stripped to ('sh', 'op'), 'hk', 'd', etc.
- 'usAAPL' (mixed-case) flowed through to _classify_bare_code which
matched its alphanumeric pattern and routed to US.
- '600519.SH' / '00700.HK' / '7203.T' (suffix form) never had the suffix
stripped; '.'-containing tokens were discarded by the empty fallback
branch and the inner code was misclassified.
Fix: at the top of parse_analysis_target, call
stock_code_utils._normalize_code_and_exchange to uppercase, strip the
suffix and extract the exchange. The normalized form is then rewritten
via _EXCHANGE_NORMALIZER into the legacy sh600519/hk00700 prefix
canonical so _split_prefix stays the source of truth for the prefix →
contract flow. Special-case 000300.SH (index) is rebuilt into sh000300
so the index lookup still fires. Pure alphabetic suffix forms
(7203.T / 005930.KS / 2330.TW / 6505.TWO) short-circuit straight to
the Yahoo-style BASE.SUFFIX canonical.
Tests: new TestNormalizationReviewBlocker covers shop/hkd/aapl
uppercase, usBRK contract #3 degrade, USFD/USM bare-ticker
preservation (the isupper() short-circuit keeps them safe from prefix
scanning), all six CN/HK suffix forms, 000300.SH → sh000300 INDEX,
and all four JP/KR/TW alphabetic suffix forms. All 72 parser tests pass.
Refs: PR #2094 review blocker OR-COR-5f9691af.
* fix(#2063): 显式后缀 reject 不再静默改写为 sh<digits>
关闭 PR #2122 maintainer review 三个高置信度 correctness blocker：
- OR-COR-607f1395: 600519.BJ / 600000.HK / 1234567.SH 这类带显式
.SH/.SZ/.BJ/.HK 后缀、但与该交易所规则不匹配或位数非法的数字
代码，以前 parse_analysis_target 在 norm_code is None 分支里
无条件抽 raw 数字重建为 sh<digits>，把被规范化层明确拒绝的
输入静默变成上交所股票。修复：仅在 SH/SS alias 路径上、且
重建后的 sh<base> 在 IndexRegistry.find_by_prefixed_code 命中
时才重建（保留 000300.SH → sh000300 INDEX alias 语义）；其他
BJ/HK 不匹配或 SH 非识别 alias 一律返回 unsupported 并携带
可定位 reason。
- OR-COR-26596201: abc.SH 这种带显式交易所后缀、但主体本身不合法
的 token，以前 norm_code is None 后没在 reject 点终止，继续走
_split_prefix / _classify_bare_code，_classify_bare_code 对非
数字默认返回 US，导致 abc.SH 被静默改判为 US stock。修复：在
显式后缀 + base 非纯数字 + 无 registry alias 命中时直接
unsupported，终止后续分类。
- OR-COR-d6afd0d6: 1234567.SH / abc.SH 这类规范化层已经拒绝的
显式后缀输入以前仍会继续走后续分类，最终被当成有效股票而不是
unsupported。修复：同上，在 _split_explicit_exchange 命中且 raw
含 '.' 时强制走严格的 alias-lookup-or-reject 路径，阻断后续
_split_prefix / _classify_bare_code 误判。
非阻断建议同时处理：把 docs/CHANGELOG.md 中原放在 [3.28.0] 段的
STOCK_LIST 解析 [新功能] 条目移回 [Unreleased] 扁平列表，并新增
本轮 [修复] 条目描述显式后缀 reject 修复。
测试：新增 TestExplicitExchangeSuffixRejections 6 个用例覆盖
600519.BJ / 600000.HK / 1234567.SH / abc.SH 4 个 reject 路径，
以及 000300.SH / sh000300.SH 2 个 INDEX alias 仍命中的回归保
护。原 72 个 stock_list_parser 测试保持全过，无回归。
* fix(#2063): collapse 3 review blocker variants in parse_analysis_target
Round-2 review blocker closure for PR #2129 (issue #2063 Phase 1):
* OR-COR-d83a3580 — malformed mixed prefix+suffix tokens like
sh0x00300.SH no longer get reconstructed into a registered
index alias. The base_digits == base_after_prefix guard
requires raw to be a clean <prefix><6digits>.<suffix> shape;
any embedded hex-like garbage between the prefix and the digits
short-circuits to unsupported.
* OR-COR-b3e32200 — dotted-prefix tokens (SH.000999) now honour
contract #3 (prefix supplied → degrade to stock) instead of being
misrouted through strict-suffix reject. has_explicit_suffix
now requires both a literal . AND no dotted-prefix lead, so
SH.000999 falls through to _split_prefix and resolves as a
SH stock candidate (canonical sh.000999).
* OR-COR-e21e9de5 — foreign-exchange suffixes (.T / .KS / .KQ / .TW /
.TWO) whose base failed _valid_exchange_code are no longer
silently flipped to US stock by _classify_bare_code. A new
elif norm_exchange: branch returns unsupported with the
offending foreign suffix surfaced in exchange / reason.
Regression tests cover all 3 blocker categories alongside the
existing OR-COR-607f1395 family.
* fix(#2063): close OR-COR-6f4d6b12 + OR-COR-4b91e5a0 review blockers
OR-COR-6f4d6b12 (dotted-prefix invalid base silent acceptance):
parse_analysis_target previously let dotted-prefix inputs like
`SH.000999` / `BJ.600519` / `HK.600519` / `SS.000999` fall through
to `_split_prefix` after `_normalize_code_and_exchange` rejected the
base. The fall-through produced malformed canonical ids
(`sh.000999`, `bj.600519`, `hk.600519`), and for `SS.000999` —
since lowercase `ss` isn't a known exchange prefix — the bare-code
classifier silently flipped the token to `asset_type='stock',
exchange='US'`. Both outcomes mask user typos as round-trippable
canonical ids.
Fix: extend the explicit-suffix reject branch to also catch dotted-
prefix form (`<EXCHANGE>.base`) when `_split_explicit_exchange`
returned a token whose base failed `_valid_exchange_code`. The same
alias rebuild / reject logic now applies uniformly to strict-suffix
and dotted-prefix shapes.
OR-COR-4b91e5a0 (SZ index mixed prefix+suffix alias rejection):
`sz399001.SZ` and `sz399006.SZ` were rejected with "explicit exchange
suffix 'SZ' rejects base 'SZ399001'" despite the default registry
listing both as SZ indices. The alias rebuild whitelist was hardcoded
to {"SH", "SS"}, so the SZ mixed prefix+suffix path never got a
chance to rebuild into `sz399001` / `sz399006`.
Fix: extend the whitelist to {"SH", "SS", "SZ"}. The existing
`base_after_prefix` / `base_digits` / `clean_alias_shape` checks
already guard against malformed hex-like garbage (OR-COR-d83a3580)
and require the rebuilt base to actually hit the registry; SZ now
rebuilds symmetrically to SH/SS.
Test changes:
- `test_dotted_prefix_degrades_to_stock_per_contract3` is replaced
by `test_dotted_prefix_with_invalid_base_is_unsupported` covering
SH.000999 / BJ.600519 / HK.600519 / SS.000999. `SZ.000001` is
removed because SZ is now in the rebuild whitelist and resolves
through the index/alias path (it's a legitimate SZ stock alias).
- New `test_sz_mixed_prefix_suffix_resolves_through_index_alias`
covers sz399001.SZ / sz399006.SZ rebuilding into the canonical
SZ index ids with the right display_name.
Local: tests/test_stock_list_parser.py 91 passed; tests/test_stock_code_utils.py 96 passed; tests/test_agent_executor.py 63 passed.
* docs(changelog): 与代码行为对齐 PR #2129 dotted-prefix 描述
reviewer 指出当前 Unreleased 条目写着 SH.000999 按 contract #3 降级为该交易所股票候选，但 src/services/stock_list_parser.py 与 tests/test_stock_list_parser.py 已把 SH.000999 / BJ.600519 / HK.600519 / SS.000999 固定为 unsupported，白名单交易所的合法 dotted-prefix（如 sz399001.SZ / sz399006.SZ）才命中 INDEX。
同步修正文案：
- invalid base dotted-prefix 不被 strict-suffix 误拒、亦不静默降级为畸形 canonical stock，统一返回 unsupported
- 白名单交易所的合法 dotted-prefix（如 sz399001.SZ / sz399006.SZ）才命中 INDEX
非阻断修订，不改代码。
* fix(#2063): close PR #2129 review blocker OR-COR-9c3d2c44 (lowercase us bare ticker / lowercase explicit-prefix)
reviewer 在新 head ec7959aa 上发现新 blocker：parse_analysis_target()
在 norm_exchange == '' 的 elif 分支用 raw.startswith('us') 判定显式前缀，
但全小写 bare US ticker（如 usfd / usm）也命中这个分支并被错误剥前缀，
得到 canonical_id='FD' / 'M'，造成 USFD / USM 真实 ticker 被无声改写为
不同 symbol。
修复：
1. lowercase us 前缀分支仅在 raw 是 mixed case（既非全 lower 也非全 upper）
时才进入剥离分支。全小写 bare ticker 由 normalize 层处理为合规 uppercase
bare US ticker（^[A-Z]{1,5}$ 形态）。全大写裸 ticker 由 _split_prefix 短路。
2. _canonicalize_for_stock US 分支统一把 bare upper()，让 lowercase 显式前缀
形态（usaapl / usshop）也得到大写 canonical id，与 mixed-case 显式前缀
（usAAPL → AAPL）与 bare lowercase（aapl → AAPL）一致。
测试：
- tests/test_stock_list_parser.py 新增 4 个 regression case：
- usfd → USFD / usm → USM（blocker close）
- usaapl → AAPL / usshop → SHOP（non-blocking 修复）
完整本地测试：
- tests/test_stock_list_parser.py 95 passed
- tests/test_stock_code_utils.py 96 passed
- tests/test_stock_code_bse.py 17 passed
- tests/test_stock_index_loader.py 14 passed
- tests/test_agent_executor.py 63 passed
- 共 285 passed
文档：docs/CHANGELOG.md [Unreleased] 段追加新条目描述 OR-COR-9c3d2c44。
* fix(#2063): close PR #2129 review blocker OR-COR-2f0d1a7e (lowercase us-prefix split bifurcation) via Phase 1 contract unification
Phase 1 maintainer clarification (issue #2063, 2026-08-01): the "us"
exchange prefix is case-insensitive on the prefix itself, but the ticker
base following an "us" prefix must arrive in canonical uppercase US
symbol shape. Fully lowercase "us"-prefixed tokens are surfaced as
"unsupported" rather than silently rewritten — closes two prior blockers
under one consistent rule without a US ticker whitelist:
- OR-COR-9c3d2c44 (closed): "usfd" / "usm" were silently upper-cased to
bare "USFD" / "USM" (i.e. synthesised different US tickers like
US Foods / USM Holdings when the user may have meant something else).
- OR-COR-2f0d1a7e (new): "usibm" / "usamd" / "usge" / "usbk" were silently
length-dependent — 5-letter lowercase "us"-prefix inputs fell back
to bare treaty producing "USIBM" / "USAMD" / "USGE" / "USBK", while
longer lowercase "usmsft" / "usmeta" split cleanly to "MSFT" / "META".
After this commit:
| input | behaviour |
|-------|-----------|
| "usAAPL" / "usBRK" / "usFD" / "usM" (mixed case) | explicit "us" prefix, splits to "AAPL" / "BRK" / "FD" / "M" |
| "USAAPL" (all upper, >5) | explicit "us" prefix, splits to "AAPL" |
| "USFD" / "USM" (all upper, ≤5) | bare US ticker, preserved as "USFD" / "USM" |
| "usfd" / "usm" / "usibm" / "usamd" / "usge" / "usbk" / "usaapl" / "usshop" (all lower) | "unsupported" with diagnostic "must use uppercase ticker base" |
| "aapl" / "shop" (no "us" prefix) | bare lowercase ticker normalized to "AAPL" / "SHOP" (unchanged) |
Implementation:
- parse_analysis_target() early-return guard: when raw.isalpha() and
raw.startswith("us") and len(raw) > 2 and raw.islower(), surface as
ParseStatus.UNSUPPORTED with exchange="US" and a human-readable
unsupported_reason mentioning the canonical uppercase forms
("usAAPL" / "usBRK" / "USFD").
- Simplified the elif norm_code and norm_exchange == "": branch to
re-split only mixed-case "us"-prefix inputs (all-lowercase is already
short-circuited by the early guard; all-uppercase stays on the bare
ticker short-circuit in _split_prefix ≤5 letters or splits as
explicit prefix >5 letters).
- Updated parse_analysis_target() docstring to reflect the new
contract: exchange prefix is case-insensitive, but "us"-prefix base
must be uppercase US symbol shape, while bare codes may be either
case (the normalizer upper-cases them).
Tests:
- Renamed test_lowercase_us_ticker_regression →
test_lowercase_us_prefix_is_unsupported with 8 parametrized cases
("usfd" / "usm" / "usibm" / "usamd" / "usge" / "usbk" / "usaapl" /
"usshop"). All assert asset_type == UNSUPPORTED, exchange == "US",
normalized_prefix is None, and unsupported_reason mentioning
the uppercase base hint. 289 parser/code-utils/index-loader/agent
tests pass locally (the lone test_multi_agent.py::TestIntelAgent
PostProcess::test_repairs_json_and_caches_intel_context failure
reproduces on HEAD pre-patch and is unrelated — issue #2131
AlphaSift test pollution).
CHANGELOG:
- Replaced the OR-COR-9c3d2c44 close bullet with a single consolidated
entry that closes both OR-COR-9c3d2c44 and OR-COR-2f0d1a7e under one
contract rule from issue #2063.
* fix(#2063): close PR #2129 review blocker OR-COR-7b45f5c1 (mixed-case
us prefix with lowercase base) by extending guard to "case-insensitive
us prefix + base must be uppercase"
OpenReview Bot on PR #2129 head e642648c9d358ffde41c4e29750f55bf27745418
flagged a remaining correctness gap in the August 1, 2026 contract
implementation: the new early-return guard only rejected fully
lowercase "us"-prefixed tokens, so mixed-case prefixes with lowercase
bases such as "Usfd", "USibm", and "Usaapl" still slipped through and
were silently rewritten into different US tickers ("USFD", "USIBM")
or accepted as explicit prefixes with lowercase bases ("us"+"aapl")
instead of returning "unsupported" under the stated Phase 1 contract.
Root cause:
- parse_analysis_target() docstring (src/services/stock_list_parser.py:
451-456) and the new guard comments claim the "us" prefix is
case-insensitive AND the ticker base must be uppercase. The
implementation only fired the reject path when raw.startswith("us")
AND raw.islower() were both true (src/services/stock_list_parser.py:
522-527). Mixed-case prefixes like "Usfd" / "USibm" / "Usaapl" have
raw.islower()==False, so they bypassed the guard.
- The downstream elif branch (src/services/stock_list_parser.py:763-767)
used raw.startswith("us") (case-sensitive) for its recovery split,
so those mixed-case prefixes were re-canonicalised to bare "USFD" /
"USIBM" (when norm_code was non-empty and ≤5 letters fit the bare
treaty) or accepted as explicit "us" prefix with lowercase base
(when norm_code was None and the downstream _split_prefix split the
prefix from the lowercase base).
After this commit the contract is enforced uniformly:
| input | behaviour |
|-------|-----------|
| "usAAPL" / "UsAAPL" / "uSAAPL" / "usBRK" / "UsBRK" / "uSBRK" / "usFD" / "usM" (prefix any case + uppercase base) | explicit "us" prefix, splits to "AAPL" / "BRK" / "FD" / "M" |
| "USAAPL" (all upper, >5) | explicit "us" prefix, splits to "AAPL" |
| "USFD" / "USM" (all upper, ≤5) | bare US ticker, preserved as "USFD" / "USM" |
| "usfd" / "usm" / "usibm" / "usamd" / "usge" / "usbk" / "usaapl" / "usshop" (all-lowercase) | "unsupported" — closed OR-COR-9c3d2c44 + OR-COR-2f0d1a7e |
| "Usfd" / "USibm" / "Usaapl" / "uSfd" / "USaapl" (mixed-case prefix + lowercase base) | "unsupported" — closes OR-COR-7b45f5c1 |
| "aapl" / "shop" (no "us" prefix) | bare lowercase ticker normalised to "AAPL" / "SHOP" (unchanged) |
Implementation:
- Extended the early-return guard in parse_analysis_target() from
`raw.startswith("us") and raw.islower()` (only fully-lowercase
rejection) to `raw[:2].lower() == "us" and not raw[2:].isupper()`
(case-insensitive prefix + base must be entirely uppercase). Any
token whose base contains lowercase letters — whether the prefix is
all-lowercase, mixed-case, or all-uppercase — is now uniformly
rejected up-front as `ParseStatus.UNSUPPORTED` with `exchange="US"`
and a human-readable `unsupported_reason` mentioning the suggested
uppercase base form. This closes OR-COR-9c3d2c44, OR-COR-2f0d1a7e
AND OR-COR-7b45f5c1 under one consistent rule.
- Simplified the elif norm_code and norm_exchange == ""`: branch to
re-split only mixed/upper-case `us`-prefix inputs whose base is fully
uppercase. The condition now requires `raw[2:].isupper()` (base must
be uppercase — already enforced by the early-return guard for the
reject path, so this clause is a defensive confirmation) and
excludes the bare-US-ticker short circuit
`not (raw.isupper() and len(raw) <= 5)` so `USFD` / `USM` ≤5-letter
all-uppercase bare tickers continue to flow through `_split_prefix`
short-circuit rather than to the explicit-prefix split.
Tests:
- Extended `test_lowercase_us_prefix_is_unsupported` with 5 new
parametrized cases (Usfd / USibm / Usaapl / uSfd / USaapl) covering
the OR-COR-7b45f5c1 mixed-case prefix + lowercase base scenario.
All 13 cases assert `asset_type == UNSUPPORTED`,
`exchange == "US"`, `normalized_prefix is None`, `canonical_id == raw`
(verbatim passthrough), and `unsupported_reason` mentioning
"uppercase".
- 254 parser/code-utils/index-loader/yfinance/bse tests pass locally
(249 + 5 new OR-COR-7b45f5c1 cases).
CHANGELOG: updated the existing consolidated bullet to mention the
third closed blocker (OR-COR-7b45f5c1) and the explicit acceptance of
mixed-case prefix + uppercase base forms ("UsBRK" / "uSBRK" / "usFD")
now that the contract is enforced uniformly on the base case.
* fix(#2063): close PR #2129 review blocker OR-COR-us-prefix-nonalpha-guard-gap by extending us-prefix guard to all non-canonical bases
OpenReview Bot 在 PR #2129 head 49e3da6e 上重新复核后给出 1 个未关闭的高置信度 correctness blocker（OR-COR-us-prefix-nonalpha-guard-gap），同时关闭前轮的 4 个 blocker（OR-COR-9c3d2c44 / 2f0d1a7e / 7b45f5c1 三个 round-1/2 blocker 已关闭，本轮只闭合 OR-COR-us-prefix-nonalpha-guard-gap）。本 commit 闭环该剩余 blocker。
== OR-COR-us-prefix-nonalpha-guard-gap root cause ==
src/services/stock_list_parser.py:526-545 的 us-prefix reject guard 用 raw.isalpha() 作为前置条件：
if (
raw.isalpha() # ❌ 前置 isalpha 过滤
and len(raw) > 2
and raw[:2].lower() == "us"
and not raw[2:].isupper()
):
return AnalysisTarget(..., asset_type=UNSUPPORTED, ...)
这意味着含标点或数字的 us-prefix 输入走不到 reject 路径，被 silently rewrite 为不同的 stock：
- parse_analysis_target("usbrk.b") -> stock canonical="BRK.B" (lowecase base+标点)
- parse_analysis_target("usshop.us") -> stock canonical="SHOP.US" (lowercase base+标点)
- parse_analysis_target("us1") -> stock canonical="1" (lowercase prefix+数字 base)
reviewer 指出这些 us-prefixed 输入应被 surfacing 为 unsupported，以免 callers 收到误导性的 canonical stock id（特别是 canonical="1" 不是合法 US symbol shape）。
reviewer 同时给出非阻断建议：补 dotted/numeric us-prefixed inputs 回归测试覆盖。
== 修法 ==
把 guard 从「raw.isalpha() AND base 含 lowercase letter」改为「base 不匹配 canonical US ticker regex」：
_US_TICKER_SHAPE_RE = re.compile(r"^[A-Z]{1,5}(\.[A-Z]{1,2})?$")
if (
len(raw) > 2
and raw[:2].lower() == "us"
and _US_TICKER_SHAPE_RE.match(raw[2:]) is None # ❌ 改为 regex match
):
return AnalysisTarget(..., asset_type=UNSUPPORTED, ...)
新 _US_TICKER_SHAPE_RE 模块级常量与 data_provider/us_index_mapping.py:16-17 和 stock_code_utils._normalize_code_and_exchange 用的同一 regex 一致——canonical US symbol shape：1-5 个大写字母可选跟一个 . + 1-2 个大写字母（covers AAPL/BRK.B/SHOP.US/HKD/USFD 等）。
这个 guard 比 raw.isalpha() + lowercase-letter 检查更严格：
- usbrk.b：base "brk.b" 不 match regex（含 lowercase）→ unsupported ✓
- usshop.us：base "shop.us" 不 match → unsupported ✓
- us1：base "1" 不 match（不是 1-5 大写字母）→ unsupported ✓
- US1：base "1" 不 match → unsupported ✓（含数字的 US-prefix 也被 reject，与 reviewer 期望一致）
- usfd / usibm / Usaapl：base 含 lowercase → 不 match → unsupported ✓（保持原 reject）
- usAAPL / usBRK.B / usSHOP.US：base match → 不 reject → 走原 split-prefix 路径 ✓
- USFD / USBRK.B / AAPL / BRK.B：raw[:2].lower()=="us" false 或 base match → 不 reject → 走原路径 ✓
测试覆盖：
- tests/test_stock_list_parser.py::test_lowercase_us_prefix_is_unsupported parametrize list 加 7 个 new case：
* usbrk.b、usshop.us（lowercase base + punctuation）
* us1、us1a、us12a（lowercase base + digits）
* US1、US12345（all-uppercase but 含数字 invalid US shape）
- 所有 case 断言 asset_type==UNSUPPORTED、exchange=="US"、canonical_id==raw、unsupported_reason 含 "uppercase"
- docstring 与 parametrize 注释同步更新加 OR-COR-us-prefix-nonalpha-guard-gap 解释
== 验证 ==
本地：
- 111 个 stock_list_parser 测试全过（104 已有 + 7 新增 parametrize case）
- PYTHONPATH=src python3 -c "from services.stock_list_parser import parse_analysis_target; for s in [...]: ..." 26 个 case 手动验证全部预期通过
- python3 -m flake8 src/services/stock_list_parser.py tests/test_stock_list_parser.py 我的改动无新增 lint 错误（pre-existing F401 'json'/'Path'/'Optional'/'AnalysisTarget' 不在本 commit 范围）
- backend-gate local run 因 1.8G 内存机 OOM killed（pre-existing limitation，与 PR 2140 解决的 issue #2131 同源）留给 CI web-gate 跑
CI 状态留给 push 后看。
== 真实路径 ==
- PR #2129 review 在 head 49e3da6e 收到 OpenReview Bot OR-COR-us-prefix-nonalpha-guard-gap blocker
- 本 commit 在分支 fix/pr-2122-blockers-r3 上修复并 push
- CI 全绿后请 maintainer 在新 head 复审
* fix(#2063): close PR #2129 round-4 blocker OR-COR-bare-us-suffix-prefix-collision in _split_prefix
OpenReview Bot 在 head 04d86b11 上给出新的 blocker：合法的 bare .US 美股代码如 SHOP.US / HKD.US / BJRI.US / USFD.US 被 _split_prefix() 按 sh/hk/bj/us 前缀错误拆分。
root cause: _split_prefix() 1-5 字母 bare US ticker short-circuit 只覆盖纯字母形态 USFD/SHAK/AAPL；带 .US 后缀的 dotted form 走到已知前缀扫描，前两字符碰巧撞上 sh/hk/bj/us 就被错误拆分。
fix: 用同一 regex _US_TICKER_SHAPE_RE = /^[A-Z]{1,5}(\.[A-Z]{1,2})?$/ 同时覆盖 bare 和 dotted form 作为 short-circuit gate。
回归测试新增 6 个 parametrize case：SHOP.US / HKD.US / BJRI.US / USFD.US / AAPL.US / BRK.B。
本地 117 测试全过（含 6 个新 case），flask8 我的改动无新增 error。CI 交给 push 后看。
* fix(#2129 round-5): extend us-prefix recovery to dotted uppercase US base
OpenReview Bot round-5 review of PR #2129 flagged a remaining
correctness blocker (OR-COR-0e285b84): mixed-case us-prefixed
inputs with a dotted uppercase US base such as usBRK.B /
usABC.US were silently rewritten to bare dotted US tickers
(USBRK.B / USABC.US) after _normalize_code_and_exchange
upper-cased them. _split_prefix then short-circuited on
_US_TICKER_SHAPE_RE and swallowed the user-supplied us prefix,
producing a different canonical id — the same bug pattern that
round-4 already fixed for bare form USFD.US / SHOP.US.
Root cause: the us-prefix recovery branch at
src/services/stock_list_parser.py:~825 guarded its eligible form
with raw.isalpha() (only all-alpha inputs). A dotted base
(e.g. BRK.B) is not all-alpha, so it fell through to raw =
norm_code (USBRK.B) and was then consumed by _split_prefix'
s bare-dotted short-circuit.
Fix: replace the raw.isalpha() + raw[2:].isupper() gate with a
single _US_TICKER_SHAPE_RE.match(raw[2:]) gate (same shape the
upfront guard at lines 566-585 already accepts), while preserving
the bare all-uppercase short-circuit exclusion (not raw[:2].isupper())
so that genuine bare dotted US tickers like SHOP.US / HKD.US /
USFD.US still skip the recovery path and flow through
_split_prefix'
s short-circuit as before.
Add 4 regression test cases covering:
- usAAPL / usBRK — bare US base, explicit prefix preserved
- usBRK.B / usABC.US — dotted uppercase US base, the new
accepted path, normalized_prefix='us' and canonical id is the
bare dotted form (e.g. BRK.B), not the mistaken USBRK.B.
Validation:
python -m pytest tests/test_stock_list_parser.py
-> 121 passed (was 117, +4 new regression cases)
No other tests touched — diff is restricted to the parser
recovery branch and the new regression test suite.
Signed-off-by: xxiaoxiong <2482929840@qq.com>
---------
Signed-off-by: xxiaoxiong <2482929840@qq.com>
Co-authored-by: xxiaoxiong <xxiaoxiong@nicholasxiong.cn>
Co-authored-by: xxiaoxiong <xxiaoxiong@users.noreply.github.com>
* fix: redact short credentials in CLI diagnostics
* fix(review-feedback-2111): Redact indented values under empty sensitive YAML fields and preserve
* fix: close structured diagnostic redaction gaps
* fix(review-feedback-2111): Treat comment-only YAML values as empty blocks and Redact indentless
* fix: redact YAML node property blocks
* fix: redact normalized proxy authorization fields
* fix: close multiline diagnostic redaction gaps
* fix: redact spaced credential labels
* fix(review-feedback-2111): Consume YAML blocks with node properties
* fix: include registered spaced credential labels
* fix: close remaining structured redaction gaps
* fix: redact shell words and explicit YAML mappings
* fix: redact single-quoted structured keys
* fix: redact shell append assignments
* fix: redact quoted YAML explicit keys
* fix(review-feedback-2111): add suffix text or drop segments from the env name, such as DeepSeek
* fix(review-feedback-2111): trimming the new docs/CHANGELOG
* fix(review-feedback-2111): update the PR description's verification counts/ranges to match the
* fix(review-feedback-2111): 评审结论 - 代码检查 ：当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要：基于
* fix(review-feedback-2111): 补上 helper 级和 non-zero-exit preview 级回归用例，避免文档与运行时行为再次漂移
* fix(review-feedback-2111): add focused helper-level and non-zero-exit preview regressions for
* fix(review-feedback-2111): 评审结论 - 代码检查 ：当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要：On the current
* fix(review-feedback-2111): update the PR description's reported current head from 27a013fbf to
* fix: redact sensitive env names embedded inside command substitutions across multi-segment diagnostics
- 覆盖 OPENAI_API_KEY=sk-12345 这种首段为非敏感赋值、值里又嵌敏感名的情况
- 当 $(...) 的前置赋值是敏感名时跳过尾扫避免双重改写，非敏感名仍需进入尾扫
- 新增测试覆盖多段 + 同函数敏感+非敏感赋值的复合诊断文本
- 使用 [A-Z][A-Z0-9_]* token 扫描找到 ALL 中的敏感 env 名引用
* fix(review-2111): redact export SENSITIVE=$(...) without dropping trailing fields
Round-3 review blocker closure for PR #2118 (issue #1784):
OR-COR-7c0a5d41 — the form
export SENSITIVE_ENV=$(printenv OTHER_SECRET) session_id=dup1 token_budget=1000
previously lost ``session_id=dup1`` (case 1) and ALL trailing fields
(case 2 with ``echo OPENAI_API_KEY=sk-12345``) because the second-pass
``$(...)`` scan re-added the same span that the first pass had already
replaced, and ``_replace_spans`` silently dropped the duplicated
region's width worth of trailing characters.
Two fixes:
1. Track first-pass sensitive-assignment replacement spans
(``first_pass_spans``) and skip any ``$(...)`` whose start lies
inside one of those spans. This is the principled guard against
the overlap regardless of where the leading assignment sits.
Previously the second-pass computed a "prior prefix" via three
independent regex branches (semicolon-separated / newline-
separated / head-of-string), each of which only matched bare
``NAME=`` — so ``export NAME=`` slipped through and the second
pass double-rewrote the same span.
2. Add ``(?:export[ \t]+)?`` to all three prior-prefix regexes so
that even if the first-pass span guard were ever evaded, the
leading ``export SENSITIVE=`` would still be recognised and the
second pass would skip the inner ``$(...)``.
Regression tests cover the two exact reproductions from the review
(``session_id=dup1 token_budget=1000`` and ``session_id=dup3``), plus
a non-``export`` control case to lock in the existing behaviour. Full
``tests/test_local_cli_backend.py`` passes 363/363; the ``tests/test_stock_*``
subset is unaffected (460 passed across the CLI + stock subset).
---------
Co-authored-by: zhulinsen <zhuls97@163.com>
Co-authored-by: xxiaoxiong <xxiaoxiong@nicholasxiong.cn>
