* fix: prefetch portfolio snapshot quotes
* fix: keep portfolio quote prefetch concurrent
* fix: stop sharing fetcher manager across portfolio quote workers
Previously _prefetch_realtime_position_prices reused a single
DataFetcherManager for all workers when prefetch_realtime_quotes
returned a full count. That count only signals a successful bulk
fetch on the underlying fetcher's module-level cache; it does not
guarantee every per-symbol get_realtime_quote() is a cache hit
(mixed-market portfolios, cache misses, or providers that report
counts loosely). With a shared manager, manager-owned per-fetcher
locks (_call_fetcher_method) re-serialize the cache-miss reads
and re-trigger the #1803 30s timeout the prefetch was meant to
avoid.
Discard the prefetch manager after warming the module-level
fetcher cache and let each worker own its DataFetcherManager.
Independent locks restore true concurrency, while cache hits
still work because the realtime cache is fetcher-module-level.
Add a regression test that exercises len>=5, full-count bulk
prefetch, and a per-symbol fetch path that still blocks; it
verifies a fresh manager per worker and max_active>=2.
Follow-up to the #1773 data-layer MVP (Taiwan suffix-only detection + routing,
merged in 2086e3c). That MVP deferred the service/API/frontend layers, leaving a
live defect: tw was absent from the DecisionSignal/Portfolio service VALID_MARKETS,
so _normalize_market("tw") raised ValueError on the decision-signal write path.
The analysis pipeline auto-extracts a DecisionSignal after history save
(_extract_decision_signal_after_history_save), so every tw analysis silently
failed to persist a signal while jp/kr succeeded -- tw was the only
yfinance-supported market that could be analyzed but never produced a signal.
Converge the tw market contract for DecisionSignal + Portfolio + Intelligence in
one pass (mirroring jp/kr #1720), per the human review on #1801 asking not to
land it piecemeal:
Backend service + API:
- src/services/{portfolio,intelligence}_service.py: VALID_MARKETS /
_ALLOWED_MARKETS + _normalize_market error strings accept tw
- src/services/decision_signal_service.py: _normalize_market error string
(VALID_MARKETS is imported from portfolio_service, so the set change propagates)
- src/services/decision_signal_extractor.py: drop the now-stale "(e.g. tw)" guard
comment (tw is supported; the guard still protects genuinely-unsupported markets)
- api/v1/schemas/{decision_signals,intelligence,portfolio}.py: Pydantic Literals + tw
- api/v1/endpoints/decision_signals.py + docs/architecture/api_spec.json: market
filter description + DecisionSignalMarket enum gain tw; test_api_schema_pydantic
exact-match vs create_app().openapi() passes (api_spec kept CRLF)
Frontend (DecisionSignal + Portfolio typed consumers only; tsc + vitest pass):
- apps/dsa-web/src/types/{decisionSignals,portfolio}.ts + pages/{DecisionSignalsPage,
PortfolioPage}.tsx + utils/{decisionSignalLabels,stockCode}.ts + i18n/uiText.ts:
add tw to the DecisionSignalMarket / portfolio market unions, the market filter
options, the tw display label, and .TW/.TWO stock-code normalization
- the alert Market-Light surface (types/alerts.ts MarketRegion, featureText
ALERT_MARKET_REGION_*) is intentionally LEFT OUT: the backend market_light_service
is cn/hk/us only, so exposing tw there would be a front/back mismatch
Tests:
- flip the two #1773 graceful-skip regressions to first-class assertions and add
test_extract_and_persist_writes_tw_signal (end-to-end persist guard)
- frontend: PortfolioPage + stockCode vitest gain tw cases
Docs (reconcile the tw contract so changelog/topic docs/code state one fact):
- docs/CHANGELOG.md: rewrite the #1772 [Unreleased] entries so they no longer say
"service/API deferred" + "tw gracefully skipped" alongside "tw now supported"
- docs/market-support.md, docs/decision-signals.md, docs/intelligence-sources.md:
sync the tw market enum / filter / examples; keep the boundary note
Still deferred (separate follow-ups): the Taiwan stock-index/seed + Web autocomplete,
and the alert (大盘红绿灯) Market-Light tw support (needs a market_light backend change).
Refs #1772
Per the maintainer review on #1772/#1773, scope this PR to market detection
and data routing only — no Taiwan stock-index / seed / pool-resolution, so a
bare code can never be silently rewritten to a .TW suffix (the index path is a
deferred follow-up with the jp/kr-style honest doc + entry-point tests).
- data_provider/base.py: _is_tw_market / _is_tw_suffix_stock, _market_tag,
normalize_stock_code, get_daily_data + get_realtime_quote routing,
_DAILY_MARKET_FETCHER_SUPPORT (YfinanceFetcher only), offshore fundamental path
- data_provider/yfinance_fetcher.py: _get_tw_main_indices (^TWII / ^TWOII) +
.TW/.TWO suffix passthrough
- src/core/trading_calendar.py: XTAI / Asia/Taipei + get_market_for_stock
- src/market_context.py: detect_market + Taiwan prompt guidelines
- src/{market_phase_summary, services/stock_code_utils, core/pipeline}: tw plumbing
(stock_code_utils only recognizes .TW/.TWO codes; no bare->suffix rewrite)
- src/services/decision_signal_extractor.py: gracefully skip tw (recognized by
the data layer but not yet supported by the signal service) — no swallowed
ValueError + traceback
- tests: test_tw_market_support.py + test_decision_signal_extractor.py
(test_build_payload_skips_tw_market_gracefully — real tw skip regression)
- docs/market-support.md + CHANGELOG
Deferred follow-ups: stock index/seed + autocomplete + frontend, API market
enums + api_spec, and the service-layer market enums.
Bare numeric codes keep A-share semantics; only an explicit .TW/.TWO suffix
opts into Taiwan.
Refs #1772
main.py --serve-only passes the app to uvicorn as the import string
"api.app:app", so uvicorn imports the app lazily inside the server
thread. That import (litellm + the full app tree, ~10s+ on constrained
hosts) runs inside the 3.0s startup self-check window, so the probe
times out and --serve-only returns 1, causing a container restart loop
on slower machines (e.g. a 2 vCPU VPS). Faster hosts finish the import
under 3s and pass, which is why the issue was spec-dependent and
long unreported.
Import the ASGI app object in the calling thread before the probe and
hand the object to uvicorn, so the heavy import stays out of the probe
window. Genuine import failures still surface immediately. Update the
start_api_server unit tests to stub api.app the same way they already
stub uvicorn.
Fixes#1759
Co-authored-by: gang.wu@ximalaya.com <gang.wu@ximalaya.com>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
efinance writes its search cache (search-cache.json) into its own
package directory (site-packages/efinance/data), which is root-owned
in the image. Since the container runs as the non-root dsa user
(uid 1000), every A-share fetch failed with PermissionError, forcing
EfinanceFetcher (the default priority-0 source) to fail and fall back.
Pre-create and chown the efinance data dir to dsa at build time. The
path is resolved dynamically to avoid hardcoding the Python version.
This is image-internal static content, so the fix belongs at build
time rather than the runtime mount-repair in the entrypoint (#1263),
which only covers user-mounted volumes.
Fixes#1748
Co-authored-by: gang.wu@ximalaya.com <gang.wu@ximalaya.com>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
* fix(issue-1739): [bug]-docker重新部署时，.env设置里的通知变量$content_j
* fix(review-feedback-1741): escape/unescape the braced forms too, or reject them before persisting
* fix(review-feedback-1741): preserve braced templates during env import
* fix(review-feedback-1741): 澄清外部模型/API 配置风险提示是否命中真实 diff
* fix(review-feedback-1741): limit raw parsing to CUSTOM WEBHOOK BODY TEMPLATE or re-apply the
* fix: limit decision signals to AI page
* fix: preserve report signal lookup
* docs: sync decision signal guide
* fix(review-feedback-1737): add a report-id filter or deep link on /decision-signals before
* fix: backfill report decision signals
* fix(review-feedback-1719): 确认并修复“不明确建议”可能被默认回填成 hold 信号的正确性风险 and persist the report date in
* fix(review-feedback-1719): 修复 legacy decision type 被当作明确 action 导致误回填的问题
* fix(review-feedback-1719): Prevent stale backfills from staying active
* fix(review-feedback-1719): anchor the same metadata-based TTL that default expires at uses to the
* fix(review-feedback-1719): 修复或明确处理懒回填时间锚点的时区/时间域不一致风险
* feat: add intelligence source ingestion baseline
* feat: feed local intelligence into analysis contexts
* fix(review-feedback-1708): Reject DNS names that resolve privately and Validate redirect targets
* fix(review-feedback-1709): 处理大盘复盘本地资讯可能被搜索结果截断掉的问题，并澄清结构化检测到的外部模型/API 或运行时配置风险是否为真实变更
* fix(review-feedback-1708): Pin DNS resolution before fetching and Stream feeds before enforcing
* fix(review-feedback-1709): filter by published at for analysis evidence, or keep missing publish
* fix(review-feedback-1708): Sanitize fetch errors before returning them and Normalize nullable
* fix(review-feedback-1709): Normalize symbol scope before lookup
* fix(review-feedback-1708): Avoid rolling back prior item inserts on duplicate races and Reject
* fix(review-feedback-1709): Use the effective news window for local evidence and add the plain
* fix(review-feedback-1708): 落地可配置 RSS/Atom 情报源、存储、查询、retention 和基础安全边界
* fix(review-feedback-1709): 补充“revert 本 PR 或移除本地资讯接入入口/清退本地资讯源配置数据”级别说明即可
* fix(review-feedback-1708): 落地 RSS/Atom 情报源的存储、拉取、查询、retention 和基础安全边界
* fix(review-feedback-1709): 补 Refs 1707
* fix(review-feedback-1709): 解决冲突后再合入
* fix(review-feedback-1709): src/services/intelligence service.py 回退了资讯源 URL 安全防护：移除了 hostname DNS
* fix: harden intelligence source ingestion
* fix(review-feedback-1709): Sanitize upstream fetch errors before returning them
* feat: add NewsNow intelligence sources
* fix(review-feedback-1709): 确认并修复
* fix(review-feedback-1709): 补一个未命中敏感规则的异常回归测试
* fix(review-feedback-1709): 处理
* docs: enhance NEWSNOW_BASE_URL compatibility guidance with official links
- Add official NewsNow GitHub deployment guide link to .env.example
- Include curl-based API contract verification example for production validation
- Update CHANGELOG.md with official repository reference and deployment recommendation
- Clarify risk of public example instance and necessity for self-hosted in production
- All HTTP 500 responses already use sanitize_diagnostic_text for privacy
Ref #1707
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix(review-feedback-1709): 补齐官方 NewsNow 实例链接文档或 API 契约确认，明确指出公开实例风险（需结合当前部署指南文档，更新 .env.example
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat: add intelligence source ingestion baseline
* fix(review-feedback-1708): Reject DNS names that resolve privately and Validate redirect targets
* fix(review-feedback-1708): Pin DNS resolution before fetching and Stream feeds before enforcing
* fix(review-feedback-1708): Sanitize fetch errors before returning them and Normalize nullable
* fix(review-feedback-1708): Avoid rolling back prior item inserts on duplicate races and Reject
* fix(review-feedback-1708): 落地可配置 RSS/Atom 情报源、存储、查询、retention 和基础安全边界
* fix(review-feedback-1708): 落地 RSS/Atom 情报源的存储、拉取、查询、retention 和基础安全边界
