* fix(issue-2201): [bug]-分析后无报告生成
* fix(review-feedback-2211): Keep the TTM cutoff anchored to the as-of date and Propagate report
* fix(review-feedback-2211): Move the empty-watchlist check before the trading-day return
* fix(review-feedback-2211): preserve the failure status, but defer returning until after the and
* fix(review-feedback-2211): preserve the failure result, but defer returning until after the
* fix(review-feedback-2211): Handle failures in the market-review-only CLI branch
* feat: support custom Tushare Pro endpoint via TUSHARE_HTTP_URL
Add TUSHARE_HTTP_URL so the Tushare data source can point at a self-hosted
or third-party compatible endpoint when the official api.tushare.pro is not
reachable. Defaults to the official host when unset, so behavior is
unchanged for existing users.
- data_provider/tushare_fetcher.py: add _resolve_tushare_http_url() helper
(strip + http(s):// schema validation) and forward the resolved URL into
_TushareHttpClient, with an info log when a custom endpoint is in use
- .env.example + .github/workflows/00-daily-analysis.yml: document and map
TUSHARE_HTTP_URL so the new option is wired into the daily job without
leaving a half-configured state
- tests: cover env parsing (empty/whitespace/http/https/missing schema),
fetcher fall-through to the official host, and end-to-end POST target
- docs/CHANGELOG.md: flat [Unreleased] entries
Fixes#1985
* refactor: drop unnecessary string-literal type hint in _build_api_client
TushareHttpClient is already defined above TushareFetcher in the module
scope, so a string-literal type hint is not needed for forward reference.
Restore the bare type to match the surrounding code style and reduce the
diff against main.
* docs(tushare): 补 TUSHARE_HTTP_URL 在 full-guide 中英版本的用途/默认行为/workflow 映射说明
按 PR #2048 review 反馈补齐：
- 表格内新增 TUSHARE_HTTP_URL 行（中英文版本同步），明确默认 https://api.tushare.pro 与 http(s):// 前缀要求
- 在 GitHub Actions 段落后补充 TUSHARE_HTTP_URL 的 vars/Secrets 优先级与每日 workflow 0映射说明，与现有非敏感配置（TICKFLOW_PRIORITY）一致
- 完整环境变量列表（中英文版本）补 TUSHARE_HTTP_URL 行，默认值列填 https://api.tushare.pro
- 与代码实现一致：00-daily-analysis.yml 已用 vars.TUSHARE_HTTP_URL || secrets.TUSHARE_HTTP_URL 映射
* docs(tushare): align TUSHARE_HTTP_URL default with runtime and clarify vars/secrets precedence
Per review feedback on PR #2048: the per-repo config contract must stay
consistent across runtime, .env.example, workflow priority, tests and
both zh/en guides. Two fixes applied as a single contract update:
1. Default endpoint alignment. data_provider/tushare_fetcher.py:100,
.env.example, and tests/test_tushare_fetcher_http_client.py all keep
the existing official endpoint http://api.tushare.pro, but the zh/en
full-guide rows had drifted to https://api.tushare.pro. Switching the
documented default to HTTPS would silently change the runtime contract
that the feat commit explicitly preserved. Revert both zh and en
guide rows to http://api.tushare.pro so docs match runtime, .env.example
and the test assertions.
2. vars/secrets precedence wording. The workflow uses
'vars.TUSHARE_HTTP_URL || secrets.TUSHARE_HTTP_URL', which means a
non-empty vars entry always wins and Secrets cannot override it. The
zh/en notes previously suggested 'Secrets as a tamper fallback' which
is incorrect under this precedence and can mislead users into thinking
Secrets has override power. Replace with explicit description of the
real semantics: vars wins when non-empty; Secrets is only selected
when the Variable is empty; for a tamper-resistant deployment put the
value only in Secrets and leave Variables empty.
Both zh and en guides are updated together; the same 6 contract surfaces
(runtime / .env.example / workflow priority / tests / zh guide / en guide)
now describe one consistent contract.
* docs(tushare): remove false Secret-as-tamper-guard claim, document real vars/secrets write-permission model
get_realtime_quote hardcoded pe_ratio and pb_ratio to None for US stocks,
so the valuation block was always empty and every US quote reported
data_quality="partial" with pe_ratio/pb_ratio in missing_fields.
yfinance already returns both via Ticker.info (trailingPE / priceToBook),
and info is fetched a few lines above for the stock name, so this reads
them from the existing dict with no extra request.
The US index path (_get_us_index_realtime_quote) is intentionally left
alone: indices have no PE/PB.
Verified: AAPL now returns pe_ratio=39.67, pb_ratio=45.08 and
missing_fields shrinks from ['amount','pe_ratio','pb_ratio'] to ['amount'].
yfinance 1.2.x returns Ticker.dividends as a single-column DataFrame instead of a
Series. YfinanceFundamentalAdapter iterated it with `.items()`, which for a DataFrame
yields (column_name, Series) — so `_safe_float(Series)` returned None, every dividend
event was dropped, and the TTM cash/count silently fell back to
`info.trailingAnnualDividendRate` (e.g. 24.0 with "0 次") instead of the true TTM sum.
Coerce to a Series before iterating.
Affects every yfinance-backed market (US/HK/JP/KR/TW); surfaced on a live TW report
(2330.TW showed 24.0 / 0 payouts vs the real ~22 across 4). + a regression test that
feeds a single-column DataFrame and asserts the events + TTM sum are parsed.
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
* feat(market): v2.1 — consume tw 三大法人 in report + LLM prompt, fix TWD currency label
The merged 三大法人 (institutional flows) data (#1829/#1841/#1855/#1863) reached
get_fundamental_context but was never surfaced: the report renderer and the LLM
prompt-builder never read the institution block, so a real analysis run showed a tw
report with no 三大法人 figures. This wires the last mile (tw-only, additive,
fail-open) and fixes a TWD currency mislabel found in the same run:
- report render (src/notification.py): _append_institutional_flow renders a 三大法人
net-buy/sell table (外资/投信/自营/合计 in 万股/亿股) when the institution block is
status='ok'; _get_fundamental_blocks now extracts it. + zh/en/ko labels.
- LLM prompt (src/analyzer.py): _format_prompt injects a 三大法人 section (raw net
figures + a qualitative 台股筹码过滤器 note, mirroring the capital_flow section)
when status='ok' and all four core nets are present.
- fetch availability (data_provider/base.py): the tw institution fetch (a ~4-5s
whole-market download) now uses the remaining stage budget instead of the ~3s
per-symbol fetch cap that starved it and made the first/only stock of a run
coin-flip to not_supported. Still bounded by the stage deadline (fail-open).
- currency (src/notification.py): _CURRENCY_SUFFIX gains TWD -> 新台币 so TWD amounts
(revenue/profit/dividend) no longer render as the A-share default 元 (reads as RMB).
Verified end-to-end via a real analysis run: the tw report now shows the 三大法人
table and 新台币-labelled financials, and institution is reliably 'ok' on a
single-stock run. Strictly additive — cn/hk/us/jp/kr byte-identical (gated on
market=='tw' / status=='ok'); no capital_flow_signal / scoring / schema. + 11 tests.
Dual review (code-reviewer APPROVE + silent-failure-hunter): the hunter caught a
prompt-gate asymmetry (only total_net checked vs all four in the render gate);
tightened to require all four core nets, matching the render/base.py gate.
* fix(market): honour FUNDAMENTAL_FETCH_TIMEOUT_SECONDS=0 for the tw institution fetch
Review on #1866: the v2.1 institution-timeout change (use the remaining stage budget
instead of the ~3s per-fetch cap) inadvertently bypassed the existing
FUNDAMENTAL_FETCH_TIMEOUT_SECONDS=0 semantic — with fetch_timeout=0, valuation and
bundle are disabled (they gate on fetch_timeout) but the institution fetch still ran.
Gate the institution fetch on `fetch_timeout > 0` as well, so fetch_timeout=0 disables
it like the other fundamental fetches; when enabled it still uses the remaining stage
budget (the whole-market download needs more than the per-symbol cap). + a regression
test asserting fetch_timeout=0 -> institution not_supported and the fetcher is not called.
---------
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
Two tw-only, strictly-additive, fail-open hardening changes to TwInstitutionalFetcher
(benchmarked against docs/data-source-stability.md):
- C1 circuit breaker: reuse data_provider.realtime_types.CircuitBreaker (the same class
DataFetcherManager uses), keyed per market (twse/tpex), 3-fail / ~5-min cooldown /
half-open probe. When an endpoint is unreachable the fetch is skipped fast and fails
open, instead of paying timeout + throttle on every stock during an outage. The breaker
tracks REACHABILITY: a hard network/HTTP error trips it; an empty / stat!=OK body still
means the endpoint responded, so it counts as success (resets the failure streak and
closes the breaker during recovery -- a no-data day mid-recovery can never strand it open).
- C2 TPEx date guard: TPEx OpenAPI serves only the latest trading day; an explicit date
that does not match the served record now fails open (None) instead of a wrong-day record.
Tests: +6 (breaker opens after 3 failures & skips the 4th fetch; recovers after cooldown;
3 empties don't trip it; TPEx date match / mismatch / no-date). + CHANGELOG.
Dual review (code-reviewer APPROVE + silent-failure-hunter): the hunter flagged a
HALF_OPEN-recovery edge in the shared CircuitBreaker; resolved tw-side by treating an
empty response as success (reachability), without modifying the shared class.
* feat(market): surface tw 三大法人 (institutional flows) in the tw report institution block
Wire the merged TwInstitutionalFetcher (#1829/#1841/#1855) into the tw analysis
report: in _build_offshore_fundamental_context, for market == "tw" only, the
institution block now carries the raw 三大法人 net buy/sell figures (foreign /
trust / dealer / total, unit=shares) from TWSE T86 / TPEx instead of the hardcoded
not_supported. v2 scope confirmed by the maintainer on issue #1777.
- tw-only + strictly additive: cn/hk/us/jp/kr offshore flows are byte-identical
(the not_supported loop just excludes institution; non-tw still gets the same
not_supported block). Pinned by tests for us/hk/jp/kr.
- fail-open + default-on: any error or no-data -> not_supported, never interrupts
the analysis. Wiring (import/construct) failures log at error (visible) but still
fail open; fetch failures log at warning.
- status "ok" only when all four core nets are present (a genuine 0 is kept).
- raw figures only: no capital_flow_signal / scoring / weight / schema / Web.
Tests: tests/test_tw_institution_report_wiring.py (9) covers data->ok, genuine-zero,
None/raise/init-raise fail-open, missing-core-net->not_supported, us/hk/jp/kr
byte-identical + fetcher-unused, and no-derived-key. + CHANGELOG.
Dual review (code-reviewer + silent-failure-hunter): the hunter caught the combined
try swallowing wiring (import/construct) errors silently at info -> split into
loud-but-fail-open (error log + still not_supported, honoring #1777's
never-interrupt rule) and guarded "ok" against a null core net.
* fix(market): address Codex review on tw institution wiring (#1863) — stage timeout, overall status, docs
- P1: run the tw institution fetch under the fundamental stage/fetch budget via
_run_with_retry (like the other offshore blocks), so a slow / rate-limited TWSE/TPEx
call fails open at the deadline instead of pushing the analysis past it.
- P2: count a present tw institution toward the OVERALL fundamental status so a report
with only 三大法人 data still surfaces fundamentals (consumers key off the top-level
status). missing_fields keeps the original three blocks, so offshore markets without
institution data stay byte-identical.
- P2 (docs): sync docs/market-support.md (drop the now-false "data-layer only / not
wired into report" clause) + add a tw institution note to docs/full-guide.md and
docs/full-guide_EN.md (AGENTS.md requires user-visible report changes to update guides).
Tests: + stage-timeout (a 2s fetch is abandoned at the ~0.3s budget, fail-open) +
overall-status surfacing assertion. 10 tw-wiring tests + offshore/tw/jp-kr suites green.
Hardens the merged #1777 TwInstitutionalFetcher. Under concurrent callers for the
same (market, date) the previous check-then-fetch could issue N duplicate upstream
requests -- which for the TWSE T86 RWD endpoint risks tripping its informal
~3 req/5s ban. A per-(market,date) in-flight lock now coalesces same-key callers
into a single fetch (double-checked against the cache); different keys still fetch
in parallel, and the master lock is never held across network I/O.
- data_provider/tw_institutional_fetcher.py: per-key in-flight lock + _read_cache /
_key_lock helpers. Behavior is otherwise unchanged -- still fail-open, still
caches only non-empty results, tw-only, no data_provider/base.py change.
- tests: + concurrent-same-key-coalesces-to-single-fetch (8 threads -> 1 request),
different-keys-not-coalesced, and HTTP-error (429) fail-open
- docs/CHANGELOG.md: [改进] entry
Refs #1777
Co-authored-by: zhulinsen <42829555+ZhuLinsen@users.noreply.github.com>
Implements the #1777 maintainer-greenlit Phase-2 data layer: a self-contained,
tw-only fetcher for Taiwan per-stock institutional (外資/投信/自營商) net
buy/sell. Strictly additive -- no change to the existing cn/hk/us/jp/kr flows in
data_provider/base.py, and not yet wired into the report/Web/scoring path (a
deliberate follow-up per the maintainer's scope).
- data_provider/tw_institutional_fetcher.py (NEW): TwInstitutionalFetcher
- 上市 .TW -> TWSE T86 legacy rwd JSON endpoint (西元 date, comma values)
- 上櫃 .TWO -> TPEx OpenAPI tpex_3insti_daily_trading (民國 date, plain ints)
- T86 columns are read by NAME (validated against the payload `fields` header),
so a TWSE column rename / reorder fails open instead of silently shipping
misaligned numbers under stale indices
- foreign_net = 外陸資 (ex 外資自營商, T86) / dealer-excluded foreign (TPEx) so
the breakdown matches the official 三大法人 total; total_net is the official
figure; unit = shares, signs preserved
- whole-market single-day cache keyed by (market, date), filtered per stock;
~3 req/5s throttle (own lock) for the T86 endpoint
- fail-open: any network/rate-limit/empty/unknown-stock returns None; a missing
or renamed column drops the row (never a fabricated 0); a row whose trading
date cannot be attributed (TPEx 民國 unconvertible) is dropped; empty/failed
fetches are not cached (no TTL-long blackout)
- tests/test_tw_institutional_fetcher.py (NEW): 20 offline tests with fixtures
trimmed from real T86 (2330) / TPEx (3105) responses; pins the net breakdown +
sign, 民國->西元 conversion, routing, caching, and fail-open -- including
column reorder (parsed by name), column rename / missing header (fail-open),
the missing-column-vs-genuine-zero distinction, and unconvertible TPEx dates
- docs/market-support.md + docs/CHANGELOG.md: data-source capability boundary +
OGDL v1 license note; no new config (.env.example untouched)
Addresses the #1829 review (read T86 by field name; drop TPEx rows with an
unconvertible date; foreign_net excludes foreign-dealers).
Sources are 政府開放資料 under 政府資料開放授權條款第 1 版 (OGDL v1, commercial-safe).
Refs #1777
Per the maintainer review on #1772/#1773, scope this PR to market detection
and data routing only — no Taiwan stock-index / seed / pool-resolution, so a
bare code can never be silently rewritten to a .TW suffix (the index path is a
deferred follow-up with the jp/kr-style honest doc + entry-point tests).
- data_provider/base.py: _is_tw_market / _is_tw_suffix_stock, _market_tag,
normalize_stock_code, get_daily_data + get_realtime_quote routing,
_DAILY_MARKET_FETCHER_SUPPORT (YfinanceFetcher only), offshore fundamental path
- data_provider/yfinance_fetcher.py: _get_tw_main_indices (^TWII / ^TWOII) +
.TW/.TWO suffix passthrough
- src/core/trading_calendar.py: XTAI / Asia/Taipei + get_market_for_stock
- src/market_context.py: detect_market + Taiwan prompt guidelines
- src/{market_phase_summary, services/stock_code_utils, core/pipeline}: tw plumbing
(stock_code_utils only recognizes .TW/.TWO codes; no bare->suffix rewrite)
- src/services/decision_signal_extractor.py: gracefully skip tw (recognized by
the data layer but not yet supported by the signal service) — no swallowed
ValueError + traceback
- tests: test_tw_market_support.py + test_decision_signal_extractor.py
(test_build_payload_skips_tw_market_gracefully — real tw skip regression)
- docs/market-support.md + CHANGELOG
Deferred follow-ups: stock index/seed + autocomplete + frontend, API market
enums + api_spec, and the service-layer market enums.
Bare numeric codes keep A-share semantics; only an explicit .TW/.TWO suffix
opts into Taiwan.
Refs #1772
* feat: harden alphasift data source routing
* fix(review-feedback-1680): Keep the sample AlphaSift pin in sync and Avoid circuit-breaking
* fix(review-feedback-1680): Defer half-open health checks until a source is attempted and Treat
* fix(review-feedback-1680): fix only bypasses this branch for fetchers that opt into allow empty
* fix(review-feedback-1680): Avoid accepting truncated Tencent histories
* fix(review-feedback-1680): 修正 .env.example 的 AlphaSift 快照源默认说明，并避免在 docs/CHANGELOG.md
* fix(review-feedback-1680): src/config.py、src/services/alphasift service.py
* fix(review-feedback-1680): requirements.txt、src/config.py、.env.example、src/services/alphasift
* fix(review-feedback-1680): 修复 TencentFetcher 对显式历史窗口的处理问题
* fix(review-feedback-1680): Check the cap was hit before rejecting Tencent history
* fix(review-feedback-1680): Use Tencent's hyphenated date format and Convert Tencent daily volume
* fix(review-feedback-1680): Accept the executor timeout argument and preserve capped windows with
* fix: limit empty daily data opt-in behavior
* docs: restore AlphaSift 3.22 changelog entry
* fix(issue-1464): [bug]-scheduled-analysis-hangs-when-stoc
* fix(review-feedback-1465): Avoid leaking daemon threads on timed-out Akshare calls
* fix(review-feedback-1465): fix can reintroduce hangs/crashes in the exact scheduled pipeline it
* fix(review-feedback-1465): Use a safe start method for timeout subprocesses
* fix(review-feedback-1465): Call freeze support before spawning timeout workers
* feat: enrich email report with financial / dividend / related-boards blocks for CN/HK/US
Render three structured sections in single-stock and dashboard markdown reports
(used by email and other static channels) sourced from fundamental_context:
- 财务摘要: report date, revenue, parent net profit, operating cash flow, ROE,
YoY growth, gross margin
- 股东回报: TTM cash dividend per share (pre-tax), TTM event count, TTM yield,
latest ex-dividend date
- 关联板块: belong_boards with sector-leading/lagging signal & change_pct when
available; auto-collapses to a 2-col (name / type) layout when no sector
ranking data exists (typical for HK/US)
To unlock the offshore (HK/US) data path:
- Add YfinanceFundamentalAdapter producing the same bundle shape as the
AkShare adapter, with growth, financial_report (incl. currency),
TTM dividend events, and sector+industry as belong_boards
- Route get_fundamental_context for hk/us through a new
_build_offshore_fundamental_context helper instead of short-circuiting
to not_supported; keep capital_flow / dragon_tiger / boards as
not_supported (no equivalent feed)
- Stop overwriting belong_boards in the pipeline for non-CN markets so the
adapter-provided sector/industry data survives
Notification renderer:
- Attach fundamental_context onto AnalysisResult so the notification layer
can reach it without an extra DB round-trip
- Currency-aware formatting: USD -> 亿/万美元, HKD -> 亿/万港元,
CNY/other -> 亿/万元; per-share amounts get the same suffix
- Right-align numeric columns and center the report-date column in the
financial summary and shareholder return tables for cleaner rendering
Tests:
- Offline coverage for the new yfinance adapter (symbol conversion, bundle
shape, refusal to use QoQ as YoY, graceful degradation when yfinance is
unavailable)
- Offshore market path through DataFetcherManager.get_fundamental_context
- Adaptive related-boards rendering (2-col when no sector signal,
4-col when at least one row carries data)
- Currency-aware amount formatting for AAPL (USD) and HK09988 (CNY)
* fix: split financial vs dividend currency and recompute TTM yield for HK/US
HK ADRs typically report financialCurrency=CNY but pay dividends in HKD
(live yfinance confirms: 9988.HK / 0700.HK both have financialCurrency=CNY
and currency=HKD; BABA US ADR has financialCurrency=CNY and currency=USD).
Previously the adapter wrote a single currency field shared by both the
financial report and the dividend block, which rendered HK dividends as
yuan in the email report.
This change writes earnings.financial_report.currency from
info.financialCurrency and earnings.dividend.currency from info.currency
independently, and the notification renderer reads each block's own
currency (falling back to financial_report.currency when dividend.currency
is absent, which keeps AkShare A-share rendering unchanged).
TTM dividend yield was sourced from info.dividendYield, mixing the
numerator (TTM cash window) with a denominator-defined yield from a
different yfinance computation. The yield is now recomputed as
ttm_cash_dividend_per_share / latest_price * 100 with both sides in the
trading currency (latest_price falls through info.currentPrice ->
regularMarketPrice -> previousClose). info.dividendYield is now a
last-resort passthrough (no *100, since current yfinance already returns
percent), behind info.trailingAnnualDividendYield (decimal).
Docs (full-guide / full-guide_EN) updated: HK/US now returns
valuation/growth/earnings/belong_boards rather than not_supported, with
the dual-currency split and the TTM yield formula explicitly documented.
CHANGELOG entries rewritten to reflect the corrected behavior.
Verified live against 9988.HK / 0700.HK / AAPL / 600519: the rendered
email shows financial summary in 亿元 for HK ADRs (CNY statements) while
the shareholder return block shows 港元 (HKD dividends), and the
recomputed yields match ttm_cash / latest_price in the same currency.
---------
Co-authored-by: suan baby <suanai@tianle.me>
* feat: add Finnhub and AlphaVantage API key config fields
* feat: add FinnhubFetcher for US market OHLCV and realtime quotes
* feat: add AlphaVantageFetcher for US market OHLCV and realtime quotes
* feat: register FinnhubFetcher and AlphaVantageFetcher in DataFetcherManager
* fix: address PR review feedback on US fetcher routing and AlphaVantage pct_chg
- Fix AlphaVantage pct_chg calculation: sort by date ascending before
computing percentage change to handle newest-first API response
- Extend US daily routing source_order to include FinnhubFetcher and
AlphaVantageFetcher in the failover chain
- Add newest-first pct_chg regression test for AlphaVantage
- Add US routing fallback order test
- Update CHANGELOG.md with new feature and fix entries
- Add docs/specs/ to .gitignore to prevent accidental commits
Verified: 1956 tests passed, ci_gate.sh clean
* fix: address third round PR review - routing gaps, index isolation, CHANGELOG cleanup
- Isolate US index routing: YFinance-first, skip Finnhub/AlphaVantage for index codes
- Add Finnhub/AlphaVantage to US realtime_quote and get_stock_name routing
- Clean CHANGELOG.md: only our 3 Finnhub/AlphaVantage entries in [Unreleased]
* fix(data-provider): skip unavailable optional fetchers and back off longbridge reconnects
* fix(data-provider): back off unavailable longbridge routes
* fix(data-provider): back off noisy pytdx name lookups
* docs: align PR 1249 changelog and compatibility notes
* fix(config): preserve legacy schedule override precedence
* fix: update report title from 'A股智能分析报告' to '股票智能分析报告'
* feat: add support for Hong Kong market in market review and analysis features
* fix: update daily_analysis workflow name and adjust region check in market analyzer
* fix: enhance market region handling in run_market_review and compute_effective_region functions
* test: add Hong Kong market support in MarketReviewLocalizationTestCase
* fix: update market review region handling to include Hong Kong support in documentation and logic
* fix: update daily_analysis workflow name to Chinese and adjust Hong Kong index symbols in yfinance_fetcher
* fix: refactor market review execution to use run_market_review for improved handling
* fix: implement trading day filtering in MarketCommand to skip reviews on market holidays
* fix: update changelog and README to reflect market command enhancements and support for multiple regions
* fix: update YfinanceFetcher to include HK index symbol mappings for improved data retrieval
Co-authored-by: Jonty <jonty.wang@hk.com.sg>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
Problem Statement:
On non-trade dats, get_trade_time() returned the second-to-last trade instead of the latest one.
Root Cause:
_pick_trade_date(trade_dates,use_today=False) skips trade_dates[0], but on non-trade days trade_dates[0] is already the most recent trade date
Solution:
for non-trade days, put use_today=Ture
