mirror of
https://github.com/ZhuLinsen/daily_stock_analysis
synced 2026-09-20 10:53:33 +08:00
* fix(search): 外股代码映射到中文显示名时英文新闻相关性漏判 (issue #2026) 问题：STOCK_NAME_MAP 把外股 ticker 映射为中文显示名（AAPL→苹果、00700→腾讯控股、 BABA→阿里巴巴集团）后，search 层在三个维度上误判英文新闻： 1. search_stock_news：prefer_chinese 看见 stock_name 含中文就在 is_foreign 之前 return True，导致 AAPL 走中文查询「苹果 AAPL 股票 最新消息」， 英文 provider 收不到 Apple 公司名。 2. search_comprehensive_intel / search_stock_events：英文查询分支直接拼 stock_name，把「苹果」喂给英文 provider。 3. _score_news_relevance：company_identity_terms 仅来自中文 stock_name， 英文标题「Apple reports earnings beat」既无中文匹配也无英文 alias 匹配， 被判 macro_market_news 而非 direct_company_news。 方案 A（替代 PR #2047 的 SearchService 私有 29 项静态表）： 1. src/data/stock_mapping.py 新增同源 sibling 英文表 STOCK_ENGLISH_NAME_MAP， 与 STOCK_NAME_MAP 同文件维护；模块加载时 _assert_foreign_english_map_invariant 静态校验 keys ⊆ STOCK_NAME_MAP 外股子集、且外股子集 ⊆ keys（双向不漂移）。 2. canonicalize_foreign_stock_code 提供 bare / .US / .N / .O / .A / .HK suffix / HK00700 prefix 全形式归一化为 STOCK_ENGLISH_NAME_MAP 的 key 形。 3. foreign_stock_english_aliases 在 stock_name 已含 CJK 时返回 canonical 英文 alias 元组；已是英文名或非外股则返回 ()，调用方各自回退到 stock_name。 4. SearchService._is_foreign_stock / _foreign_english_query_terms / search_stock_news / search_comprehensive_intel / search_stock_events / _score_news_relevance 全部改为读这一个真源；并修复 prefer_chinese 在 is_foreign 且有 alias 时强制为 False，让 foreign 分支真正可达。 5. 测试：tests/test_search_news_freshness.py 新增 8 个用例覆盖 _market_english_aliases 全场景、STOCK_ENGLISH_NAME_MAP ⊆ STOCK_NAME_MAP 双向 invariant、AAPL/AMZN/MSFT/00700/BABA 五 ticker 在三入口的 英文查询构建、_score_news_relevance 英文媒体短名标题归类、A 股对照 不污染。 Fixes #2026. * fix(search): alias 展开 term 去重 + 死分支清理 + CHANGELOG 标签格式 (PR #2049 review) ZhuLinsen 在 PR #2049 复核给出 1 个高置信度 blocker、1 个非阻断建议；massif-01 AI bot 给出 1 个非阻断死代码建议。本次提交关闭三处： 1. OR-COR-3ff1e068 [correctness] 高 confidence blocker 因：STOCK_ENGLISH_NAME_MAP['AAPL'] = ('Apple Inc.', 'Apple')，legal alias Apple Inc. 经 _company_identity_terms 展开后即包含 Apple，循环里 拿 Apple Inc. 命中 snippet Apple 加 16 分；再到第二个 alias Apple 又对同一 snippet 同一 Apple 命中再加 16 分；再叠 12 分事件词 得 44 分越 direct_signal >= 38 阈值，把摘要英文名歧义命中抬升为 direct_company_news。 果（修复前 massif-01 复现）： title='US stocks mixed after Fed comments' snippet='Apple reports earnings beat and revenue grows.' stock_code='AAPL' stock_name='苹果' → relevance_score=44, classification=direct_company_news（错误） 修：在 _score_news_relevance alias 路径引入 seen_identity_terms 集合， 按 term 而非 alias 累加，同一 Apple 在任一 alias 展开中已被计分后， 后续 alias 路径不再重复加分；保留每个 term 在标题/摘要 single-pass 命中 即 break 的语义不变。 果（修复后）： 同输入 → relevance_score=28, classification=sector_related_news （正确降级为背景新闻，因未命中股票代码或公司全称触发第三条原因 '未命中股票代码或公司全称，降级为背景新闻'） 锁回归测试 test_score_news_relevance_english_alias_dedup_prevents_double_count 覆盖：snippet-only + 英文短名歧义命中场景，断言 classification 必为 sector_related_news、relevance_score < 38、reasons 含且仅含一次摘要命中。 2. ZhuLinsen non-blocking：docs/CHANGELOG.md [Unreleased] 条目未用 AGENTS.md §1.4 要求的 - [类型] 描述 扁平标签格式。 修：把 - 修复 #2026 ... 改为 - [修复] #2026 ...，并在描述末尾 补充去重修复一并告知。 3. massif-01 AI bot non-blocking 死代码：_is_foreign_stock 在 canonicalize_foreign_stock_code 已统一 HK00700 / 00700.HK 全形到 00700 之后，原 lower.startswith('hk') 分支不可达。 修：删除该分支，注释中说明 canonical 之后只剩 5 位纯数字一种 HK 形式。 验证： \b \b \b \b 无回归：标题命中、港股综合、A 股对照、reasons 解析、各 search 入口查询构建均保持。
This commit is contained in:
Nicholas-Xiong
@@ -10,6 +10,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
|
||||
## [Unreleased]
|
||||
<!-- 新条目格式：- [类型] 描述（类型取值：新功能/改进/修复/文档/测试/chore）-->
|
||||
<!-- 每条独立一行追加到本段末尾，无需分类标题，合并时冲突最小 -->
|
||||
- [修复] #2026 外股代码映射到中文显示名时英文新闻相关性判定漏判：新增同源 STOCK_ENGLISH_NAME_MAP 单一真源、canonicalize_foreign_stock_code 规范化入口与 _foreign_english_query_terms 别名解析，使 AAPL/00700/BABA 等 ticker 即使 stock_name 为中文也能在查询构建、相关性打分与多维度情报路径上复用 canonical 英文名，并补齐 .US/.HK suffix / HK 前缀全形式的归类与回归用例；同时在 _score_news_relevance 对 alias 展开 term 做去重，避免 legal alias 展开短名与显式 short alias 重复计分。
|
||||
|
||||
## [3.27.0] - 2026-07-19
|
||||
|
||||
|
||||
@@ -1,6 +1,4 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
from __future__ import annotations
|
||||
|
||||
"""
|
||||
===================================
|
||||
股票代码与名称映射
|
||||
@@ -9,6 +7,11 @@ from __future__ import annotations
|
||||
Shared stock code -> name mapping, used by analyzer, data_provider, and name_to_code_resolver.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Dict, Tuple
|
||||
|
||||
# Stock code -> name mapping (common stocks)
|
||||
STOCK_NAME_MAP = {
|
||||
# === A-shares ===
|
||||
@@ -107,6 +110,162 @@ STOCK_NAME_MAP = {
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Foreign-ticker English identity map (sibling of STOCK_NAME_MAP)
|
||||
# ---------------------------------------------------------------------------
|
||||
# Issue #2026: when STOCK_NAME_MAP maps a US/HK ticker to a Chinese display
|
||||
# name (e.g. AAPL -> 苹果), ``SearchService._company_identity_terms`` cannot
|
||||
# derive an English alias from that Chinese name, so English news headlines
|
||||
# like "Apple reports earnings beat" never score as ``direct_company_news``.
|
||||
#
|
||||
# This map is the single source of truth for the English company identity
|
||||
# (legal name + common media short form) of every foreign ticker currently
|
||||
# mapped to Chinese by STOCK_NAME_MAP. The alias list per ticker is a tuple,
|
||||
# ordered from most-specific (legal name) to least-specific (media short name)
|
||||
# so callers can pick the granularity they need.
|
||||
#
|
||||
# Constraint (enforced by the assert below): the keys of this map are a subset
|
||||
# of the foreign-ticker keys of STOCK_NAME_MAP. Adding an English alias for a
|
||||
# ticker requires the ticker to already exist in STOCK_NAME_MAP, so the two
|
||||
# maps cannot drift — exactly the single-source-of-truth property demanded by
|
||||
# issue #2026 ("alias 来源建议不要再复制一份静态映射").
|
||||
#
|
||||
# Note: tickers map to a *tuple* of English names rather than a single legal
|
||||
# name. ``_company_identity_terms`` only strips one legal suffix layer, so it
|
||||
# cannot synthesise the common media short names ("Apple", "Amazon", "Google",
|
||||
# "Alibaba", "Tencent", "Pinduoduo", "Xiaomi", ...) that journalists actually
|
||||
# use in headlines. Encoding those short names explicitly here is the contract
|
||||
# massif-01 asked for on PR #2047 ("将搜索 identity 建模为真实 alias 集, 不要
|
||||
# 继续给 suffix stripping 堆启发式规则").
|
||||
STOCK_ENGLISH_NAME_MAP: Dict[str, Tuple[str, ...]] = {
|
||||
# === US stocks ===
|
||||
"AAPL": ("Apple Inc.", "Apple"),
|
||||
"TSLA": ("Tesla, Inc.", "Tesla"),
|
||||
"MSFT": ("Microsoft Corporation", "Microsoft"),
|
||||
"GOOGL": ("Alphabet Inc.", "Google"),
|
||||
"GOOG": ("Alphabet Inc.", "Google"),
|
||||
"AMZN": ("Amazon.com, Inc.", "Amazon"),
|
||||
"NVDA": ("NVIDIA Corporation", "NVIDIA"),
|
||||
"META": ("Meta Platforms, Inc.", "Meta"),
|
||||
"AMD": ("Advanced Micro Devices, Inc.", "AMD"),
|
||||
"INTC": ("Intel Corporation", "Intel"),
|
||||
"BABA": ("Alibaba Group Holding Limited", "Alibaba"),
|
||||
"PDD": ("PDD Holdings Inc.", "Pinduoduo"),
|
||||
"JD": ("JD.com, Inc.", "JD.com"),
|
||||
"BIDU": ("Baidu, Inc.", "Baidu"),
|
||||
"NIO": ("NIO Inc.", "NIO"),
|
||||
"XPEV": ("XPeng Inc.", "XPeng"),
|
||||
"LI": ("Li Auto Inc.", "Li Auto"),
|
||||
"COIN": ("Coinbase Global, Inc.", "Coinbase"),
|
||||
"MSTR": ("MicroStrategy Incorporated", "MicroStrategy"),
|
||||
# === HK stocks (5-digit) ===
|
||||
"00700": ("Tencent Holdings", "Tencent"),
|
||||
"03690": ("Meituan",),
|
||||
"01810": ("Xiaomi Corporation", "Xiaomi"),
|
||||
"09988": ("Alibaba Group Holding", "Alibaba"),
|
||||
"09618": ("JD.com",),
|
||||
"09888": ("Baidu Inc.", "Baidu"),
|
||||
"01024": ("Kuaishou Technology", "Kuaishou"),
|
||||
"00981": ("SMIC",),
|
||||
"02015": ("Li Auto Inc.", "Li Auto"),
|
||||
"09868": ("XPeng Inc.", "XPeng"),
|
||||
"00005": ("HSBC Holdings", "HSBC"),
|
||||
"01299": ("AIA Group", "AIA"),
|
||||
"00941": ("China Mobile",),
|
||||
"00883": ("CNOOC",),
|
||||
}
|
||||
|
||||
|
||||
def _assert_foreign_english_map_invariant() -> None:
|
||||
"""Verify STOCK_ENGLISH_NAME_MAP keys ⊆ foreign-ticker keys of STOCK_NAME_MAP."""
|
||||
foreign_keys_in_name_map = {
|
||||
code for code in STOCK_NAME_MAP
|
||||
if not (code.isdigit() and len(code) == 6) # exclude A-shares
|
||||
}
|
||||
english_keys = set(STOCK_ENGLISH_NAME_MAP)
|
||||
extra = english_keys - foreign_keys_in_name_map
|
||||
missing = foreign_keys_in_name_map - english_keys
|
||||
if extra or missing:
|
||||
raise AssertionError(
|
||||
f"STOCK_ENGLISH_NAME_MAP drift detected: "
|
||||
f"extra={sorted(extra)}, missing={sorted(missing)}; "
|
||||
f"every foreign-ticker key in STOCK_NAME_MAP must have an English "
|
||||
f"alias entry here, and no foreign English entry may target a "
|
||||
f"ticker absent from STOCK_NAME_MAP."
|
||||
)
|
||||
|
||||
|
||||
_assert_foreign_english_map_invariant()
|
||||
|
||||
|
||||
def canonicalize_foreign_stock_code(stock_code: str) -> str:
|
||||
"""Canonicalize a foreign ticker to the form used as keys in
|
||||
STOCK_NAME_MAP / STOCK_ENGLISH_NAME_MAP.
|
||||
|
||||
Single canonical boundary for ``bare`` / ``prefix`` / ``suffix`` forms:
|
||||
|
||||
``AAPL`` -> ``AAPL``
|
||||
``AAPL.US`` -> ``AAPL``
|
||||
``AAPL.N`` -> ``AAPL`` (NYSE suffix)
|
||||
``AAPL.O`` -> ``AAPL`` (NASDAQ suffix)
|
||||
``00700`` -> ``00700``
|
||||
``00700.HK`` -> ``00700``
|
||||
``HK00700`` -> ``00700``
|
||||
|
||||
A-share codes (``600519``, ``600519.SH``) and unknown forms return the
|
||||
uppercased stripped input unchanged. Callers that need to detect
|
||||
foreign-ness should call ``SearchService._is_foreign_stock`` (which now
|
||||
honours prefix/suffix forms) or check membership of the returned canonical
|
||||
key against STOCK_ENGLISH_NAME_MAP.
|
||||
"""
|
||||
code = (stock_code or "").strip().upper()
|
||||
if not code:
|
||||
return ""
|
||||
# US exchange suffixes: .US / .N (NYSE) / .O (NASDAQ) / .A (AMEX)
|
||||
for suffix in (".US", ".N", ".O", ".A"):
|
||||
if code.endswith(suffix):
|
||||
return code[: -len(suffix)]
|
||||
# HK prefix: ``HK00700`` (7 chars: HK + 5 digits)
|
||||
if code.startswith("HK") and len(code) == 7 and code[2:].isdigit():
|
||||
return code[2:]
|
||||
# HK suffix: ``00700.HK``
|
||||
if code.endswith(".HK") and code[: -len(".HK")].isdigit():
|
||||
return code[: -len(".HK")]
|
||||
return code
|
||||
|
||||
|
||||
def foreign_stock_english_aliases(stock_code: str, stock_name: str) -> Tuple[str, ...]:
|
||||
"""Return English identity aliases for a foreign ticker whose
|
||||
pipeline-supplied ``stock_name`` (from STOCK_NAME_MAP) is Chinese.
|
||||
|
||||
Returns an empty tuple when:
|
||||
* the stock code is not a foreign ticker covered by STOCK_ENGLISH_NAME_MAP
|
||||
(after canonicalization), or
|
||||
* the supplied ``stock_name`` is already English (no CJK chars), which
|
||||
means the data layer already returned an English display name and the
|
||||
search layer has no alias work to do.
|
||||
|
||||
Keeping the alias source in ``stock_mapping`` (rather than as a private
|
||||
constant in ``SearchService``) means the data layer, the analyzer, and the
|
||||
search layer all share one foreign English-alias contract — the single
|
||||
source of truth requested in issue #2026.
|
||||
"""
|
||||
raw_name = (stock_name or "").strip()
|
||||
if not raw_name:
|
||||
return ()
|
||||
# If the supplied name is already English (no CJK), no alias is needed.
|
||||
if not _CONTAINS_CJK_RE.search(raw_name):
|
||||
return ()
|
||||
canonical = canonicalize_foreign_stock_code(stock_code)
|
||||
if not canonical:
|
||||
return ()
|
||||
aliases = STOCK_ENGLISH_NAME_MAP.get(canonical)
|
||||
return tuple(aliases) if aliases else ()
|
||||
|
||||
|
||||
_CONTAINS_CJK_RE = re.compile(r"[\u4e00-\u9fff]")
|
||||
|
||||
|
||||
def is_meaningful_stock_name(name: str | None, stock_code: str) -> bool:
|
||||
"""Return whether a stock name is useful for display or caching."""
|
||||
if not name:
|
||||
|
||||
@@ -38,6 +38,10 @@ from src.config import (
|
||||
normalize_news_strategy_profile,
|
||||
resolve_news_window_days,
|
||||
)
|
||||
from src.data.stock_mapping import (
|
||||
canonicalize_foreign_stock_code,
|
||||
foreign_stock_english_aliases,
|
||||
)
|
||||
from src.services.run_diagnostics import record_provider_run, record_provider_run_started
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -2363,19 +2367,48 @@ class SearchService:
|
||||
|
||||
@staticmethod
|
||||
def _is_foreign_stock(stock_code: str) -> bool:
|
||||
"""判断是否为港股或美股"""
|
||||
code = stock_code.strip()
|
||||
"""判断是否为港股或美股。
|
||||
|
||||
Honours all canonical input forms — bare ticker (``AAPL`` / ``00700``),
|
||||
suffixed ticker (``AAPL.US`` / ``00700.HK``), and prefixed HK ticker
|
||||
(``HK00700``) — by canonicalizing to the key form used in
|
||||
STOCK_ENGLISH_NAME_MAP before applying the existing structural checks.
|
||||
This is the canonical-boundary unification that massif-01 asked for on
|
||||
PR #2047 (so alias resolution and foreign-ness detection no longer
|
||||
disagree on the same input).
|
||||
"""
|
||||
code = canonicalize_foreign_stock_code(stock_code).strip()
|
||||
if not code:
|
||||
return False
|
||||
# 美股：1-5个大写字母，可能包含点（如 BRK.B）
|
||||
if SearchService._US_STOCK_RE.match(code):
|
||||
return True
|
||||
# 港股：带 hk 前缀或 5位纯数字
|
||||
lower = code.lower()
|
||||
if lower.startswith('hk'):
|
||||
return True
|
||||
# 港股：5位纯数字。canonicalize_foreign_stock_code 已把 HK00700 前缀
|
||||
# 与 00700.HK 后缀全部归一为 00700 形式，原 lower.startswith('hk')
|
||||
# 分支在 canonical 之后为不可达死代码，已删除。
|
||||
if code.isdigit() and len(code) == 5:
|
||||
return True
|
||||
return False
|
||||
|
||||
@staticmethod
|
||||
def _foreign_english_query_terms(stock_code: str, stock_name: str) -> Tuple[str, ...]:
|
||||
"""Return English company name(s) to embed in foreign-stock search
|
||||
queries. issue #2026: When STOCK_NAME_MAP maps a foreign ticker to a
|
||||
Chinese display name, the search layer must not pass that Chinese name
|
||||
to English news providers, otherwise the provider misses English
|
||||
headlines entirely.
|
||||
|
||||
Returns the canonical alias tuple from ``STOCK_ENGLISH_NAME_MAP`` if the
|
||||
supplied ``stock_name`` is Chinese and the canonical ticker has English
|
||||
aliases. Otherwise returns an empty tuple (callers fall back to
|
||||
``stock_name`` itself).
|
||||
|
||||
Kept deliberately small and read-only: this helper never mutates the
|
||||
alias set and never invents aliases outside the single source of truth
|
||||
in ``src/data/stock_mapping.py``.
|
||||
"""
|
||||
return foreign_stock_english_aliases(stock_code, stock_name)
|
||||
|
||||
@classmethod
|
||||
def _contains_chinese_text(cls, value: Optional[str]) -> bool:
|
||||
"""Return True when the input contains CJK characters."""
|
||||
@@ -3025,6 +3058,58 @@ class SearchService:
|
||||
add_reason(f"摘要命中公司名 {term}")
|
||||
break
|
||||
|
||||
# Issue #2026: when STOCK_NAME_MAP maps a foreign ticker to a Chinese
|
||||
# display name (e.g. AAPL -> 苹果), the loop above cannot match English
|
||||
# news headlines ("Apple reports earnings beat"). Resolve English
|
||||
# identity aliases from the single source of truth (STOCK_ENGLISH_NAME_MAP
|
||||
# in src/data/stock_mapping.py — sibling of STOCK_NAME_MAP, asserted to
|
||||
# be a subset of its foreign-ticker keys) and feed both the alias
|
||||
# strings and their legal-suffix-stripped variants into the same
|
||||
# identity-term scoring path.
|
||||
english_aliases = foreign_stock_english_aliases(stock_code, stock_name)
|
||||
if english_aliases:
|
||||
# Issue #2026 / PR #2049 review: dedupe identity terms across all
|
||||
# aliases BEFORE scoring. STOCK_ENGLISH_NAME_MAP legal alias
|
||||
# (``Apple Inc.``) is intentionally designed to also expose its
|
||||
# short alias (``Apple``) so the search-query construction path
|
||||
# always has a concise term to put into English queries. But when
|
||||
# the SAME short form appears both as an explicit alias tuple
|
||||
# member AND as the cleaned output of _company_identity_terms on
|
||||
# the legal alias, naive per-alias accumulation would double-count
|
||||
# a single snippet hit on ``Apple`` (16+16=32) and push ambiguous
|
||||
# snippet-only headlines over the direct_company_news threshold.
|
||||
# Collect terms into a set first; score each unique term once.
|
||||
seen_identity_terms: set = set()
|
||||
for alias in english_aliases:
|
||||
for term in cls._company_identity_terms(alias):
|
||||
if term in seen_identity_terms:
|
||||
continue
|
||||
seen_identity_terms.add(term)
|
||||
ambiguous_en = (
|
||||
not cls._contains_chinese_text(term)
|
||||
and term.lower() in cls._AMBIGUOUS_EN_COMPANY_NAMES
|
||||
)
|
||||
title_score = 26 if ambiguous_en else 45
|
||||
snippet_score = 16 if ambiguous_en else 28
|
||||
if cls._contains_identity_term(title, term):
|
||||
score += title_score
|
||||
direct_signal += title_score
|
||||
if ambiguous_en:
|
||||
has_ambiguous_company_signal = True
|
||||
else:
|
||||
has_unambiguous_company_signal = True
|
||||
add_reason(f"标题命中公司英文别名 {term}")
|
||||
break
|
||||
if cls._contains_identity_term(snippet, term):
|
||||
score += snippet_score
|
||||
direct_signal += snippet_score
|
||||
if ambiguous_en:
|
||||
has_ambiguous_company_signal = True
|
||||
else:
|
||||
has_unambiguous_company_signal = True
|
||||
add_reason(f"摘要命中公司英文别名 {term}")
|
||||
break
|
||||
|
||||
has_company_event = cls._contains_any_news_term(full_text, cls._COMPANY_EVENT_TERMS)
|
||||
if has_company_event and direct_signal > 0:
|
||||
score += 12
|
||||
@@ -3605,14 +3690,32 @@ class SearchService:
|
||||
|
||||
# 构建搜索查询（优化搜索效果）
|
||||
is_foreign = self._is_foreign_stock(stock_code)
|
||||
# Issue #2026: When STOCK_NAME_MAP maps a foreign ticker to a Chinese
|
||||
# display name (e.g. AAPL -> 苹果), the English news search query would
|
||||
# otherwise contain the Chinese name and miss English headlines.
|
||||
# Resolve the canonical English alias (single source of truth:
|
||||
# STOCK_ENGLISH_NAME_MAP in src/data/stock_mapping.py) so the foreign
|
||||
# query path uses a real English company name.
|
||||
english_aliases = self._foreign_english_query_terms(stock_code, stock_name)
|
||||
effective_name = english_aliases[0] if english_aliases else stock_name
|
||||
short_name = english_aliases[-1] if english_aliases else None
|
||||
# Issue #2026: Foreign tickers must bypass prefer_chinese even when the
|
||||
# display name is Chinese (e.g. AAPL -> 苹果), otherwise the foreign
|
||||
# branch below is unreachable and English headlines are missed.
|
||||
prefer_chinese = prefer_chinese and not (is_foreign and english_aliases)
|
||||
if focus_keywords:
|
||||
# 如果提供了关键词，直接使用关键词作为查询
|
||||
query = " ".join(focus_keywords)
|
||||
elif prefer_chinese:
|
||||
query = f"{stock_name} {stock_code} 股票 最新消息"
|
||||
elif is_foreign:
|
||||
# 港股/美股使用英文搜索关键词
|
||||
query = f"{stock_name} {stock_code} stock latest news"
|
||||
# 港股/美股使用英文搜索关键词；优先使用英文公司名（issue #2026）
|
||||
if english_aliases and short_name and short_name != effective_name:
|
||||
query = (
|
||||
f"{effective_name} {short_name} {stock_code} stock latest news"
|
||||
)
|
||||
else:
|
||||
query = f"{effective_name} {stock_code} stock latest news"
|
||||
else:
|
||||
# 默认主查询：股票名称 + 核心关键词
|
||||
query = f"{stock_name} {stock_code} 股票 最新消息"
|
||||
@@ -3895,10 +3998,16 @@ class SearchService:
|
||||
event_types = ["earnings report", "insider selling", "quarterly results"]
|
||||
else:
|
||||
event_types = ["年报预告", "减持公告", "业绩快报"]
|
||||
|
||||
|
||||
# Issue #2026: foreign-ticker Chinese display name needs canonical
|
||||
# English alias for English event query (single source of truth in
|
||||
# src/data/stock_mapping.py).
|
||||
english_aliases = self._foreign_english_query_terms(stock_code, stock_name)
|
||||
effective_name = english_aliases[0] if english_aliases else stock_name
|
||||
|
||||
# 构建针对性查询
|
||||
event_query = " OR ".join(event_types)
|
||||
query = f"{stock_name} ({event_query})"
|
||||
query = f"{effective_name} ({event_query})"
|
||||
|
||||
logger.info(f"搜索股票事件: {stock_name}({stock_code}) - {event_types}")
|
||||
|
||||
@@ -3949,17 +4058,25 @@ class SearchService:
|
||||
is_index_etf = self.is_index_or_etf(stock_code, stock_name)
|
||||
|
||||
if is_foreign:
|
||||
# Issue #2026: Foreign-ticker English alias resolution from the
|
||||
# single source of truth (STOCK_ENGLISH_NAME_MAP in
|
||||
# src/data/stock_mapping.py). When STOCK_NAME_MAP maps the ticker
|
||||
# to a Chinese display name, the English news query path must use
|
||||
# the canonical English company name; otherwise English news
|
||||
# providers receive the Chinese name and miss English headlines.
|
||||
english_aliases = self._foreign_english_query_terms(stock_code, stock_name)
|
||||
effective_name = english_aliases[0] if english_aliases else stock_name
|
||||
search_dimensions = [
|
||||
{
|
||||
'name': 'latest_news',
|
||||
'query': f"{stock_name} {stock_code} latest news events",
|
||||
'query': f"{effective_name} {stock_code} latest news events",
|
||||
'desc': '最新消息',
|
||||
'tavily_topic': 'news',
|
||||
'strict_freshness': True,
|
||||
},
|
||||
{
|
||||
'name': 'market_analysis',
|
||||
'query': f"{stock_name} analyst rating target price report",
|
||||
'query': f"{effective_name} analyst rating target price report",
|
||||
'desc': '机构分析',
|
||||
'tavily_topic': None,
|
||||
'strict_freshness': False,
|
||||
@@ -3967,8 +4084,8 @@ class SearchService:
|
||||
{
|
||||
'name': 'risk_check',
|
||||
'query': (
|
||||
f"{stock_name} {stock_code} index performance outlook tracking error"
|
||||
if is_index_etf else f"{stock_name} risk insider selling lawsuit litigation"
|
||||
f"{effective_name} {stock_code} index performance outlook tracking error"
|
||||
if is_index_etf else f"{effective_name} risk insider selling lawsuit litigation"
|
||||
),
|
||||
'desc': '风险排查',
|
||||
'tavily_topic': None if is_index_etf else 'news',
|
||||
@@ -3977,8 +4094,8 @@ class SearchService:
|
||||
{
|
||||
'name': 'earnings',
|
||||
'query': (
|
||||
f"{stock_name} {stock_code} index performance composition outlook"
|
||||
if is_index_etf else f"{stock_name} earnings revenue profit growth forecast"
|
||||
f"{effective_name} {stock_code} index performance composition outlook"
|
||||
if is_index_etf else f"{effective_name} earnings revenue profit growth forecast"
|
||||
),
|
||||
'desc': '业绩预期',
|
||||
'tavily_topic': None,
|
||||
@@ -3987,8 +4104,8 @@ class SearchService:
|
||||
{
|
||||
'name': 'industry',
|
||||
'query': (
|
||||
f"{stock_name} {stock_code} index sector allocation holdings"
|
||||
if is_index_etf else f"{stock_name} industry competitors market share outlook"
|
||||
f"{effective_name} {stock_code} index sector allocation holdings"
|
||||
if is_index_etf else f"{effective_name} industry competitors market share outlook"
|
||||
),
|
||||
'desc': '行业分析',
|
||||
'tavily_topic': None,
|
||||
|
||||
@@ -2177,6 +2177,208 @@ class SearchNewsFreshnessTestCase(unittest.TestCase):
|
||||
parsed = SearchService._normalize_news_publish_date(rfc_text)
|
||||
self.assertEqual(parsed, expected_local_date)
|
||||
|
||||
# ---- Issue #2026: foreign-ticker Chinese display name -> English alias ----
|
||||
# massif-01 on PR #2047 asked for canonical boundary unification: when
|
||||
# STOCK_NAME_MAP maps a foreign ticker to a Chinese display name, the
|
||||
# search/scoring layers must resolve the canonical English alias from the
|
||||
# single source of truth (STOCK_ENGLISH_NAME_MAP in src/data/stock_mapping.py)
|
||||
# before contacting English news providers. These tests pin the contract.
|
||||
|
||||
def test_is_foreign_stock_accepts_canonical_suffix_forms(self) -> None:
|
||||
"""_is_foreign_stock must accept .US/.HK suffix and HK-prefixed inputs (massif-01 blocker 1)."""
|
||||
for code, expected in (
|
||||
("AAPL", True),
|
||||
("AAPL.US", True),
|
||||
("AAPL.N", True),
|
||||
("00700", True),
|
||||
("00700.HK", True),
|
||||
("HK00700", True),
|
||||
("BRK.B", True),
|
||||
("600519", False),
|
||||
("600519.SH", False),
|
||||
("", False),
|
||||
):
|
||||
with self.subTest(stock_code=code):
|
||||
self.assertEqual(
|
||||
SearchService._is_foreign_stock(code),
|
||||
expected,
|
||||
f"_is_foreign_stock({code!r}) should be {expected}",
|
||||
)
|
||||
|
||||
def test_foreign_english_aliases_returns_canonical_tuple_for_chinese_display_name(self) -> None:
|
||||
"""Stock with Chinese display name must resolve to canonical English alias tuple."""
|
||||
cases = (
|
||||
("AAPL", "苹果", ("Apple Inc.", "Apple")),
|
||||
("00700", "腾讯控股", ("Tencent Holdings", "Tencent")),
|
||||
("BABA", "阿里巴巴", ("Alibaba Group Holding Limited", "Alibaba")),
|
||||
("09988", "阿里巴巴", ("Alibaba Group Holding", "Alibaba")),
|
||||
("PDD", "拼多多", ("PDD Holdings Inc.", "Pinduoduo")),
|
||||
)
|
||||
for code, name, expected in cases:
|
||||
with self.subTest(stock_code=code, stock_name=name):
|
||||
aliases = SearchService._foreign_english_query_terms(code, name)
|
||||
self.assertEqual(aliases, expected)
|
||||
|
||||
def test_foreign_english_aliases_empty_for_english_display_name(self) -> None:
|
||||
"""Stock whose STOCK_NAME_MAP value is already English must not invent aliases."""
|
||||
for code, name in (("AMD", "AMD"), ("META", "Meta"), ("COIN", "Coinbase")):
|
||||
with self.subTest(stock_code=code, stock_name=name):
|
||||
aliases = SearchService._foreign_english_query_terms(code, name)
|
||||
self.assertEqual(aliases, ())
|
||||
|
||||
def test_score_news_relevance_english_alias_for_chinese_display_name(self) -> None:
|
||||
"""massif-01 blocker 2: English alias must enable English-news matching."""
|
||||
fresh = datetime.now().date().isoformat()
|
||||
cases = (
|
||||
("AAPL", "苹果", "Apple reports earnings beat", "Quarterly results surpass estimates."),
|
||||
("AAPL.US", "苹果", "Apple reports earnings beat", "Quarterly results surpass estimates."),
|
||||
("00700", "腾讯控股", "Tencent reports profit rise", "Quarterly profit up 12% YoY"),
|
||||
("00700.HK", "腾讯控股", "Tencent reports profit rise", "Quarterly profit up 12% YoY"),
|
||||
("BABA", "阿里巴巴", "Alibaba reports earnings beat", "Quarterly revenue up 9% YoY"),
|
||||
("AMZN", "亚马逊", "Amazon reports earnings beat", "AWS growth re-accelerates"),
|
||||
("PDD", "拼多多", "Pinduoduo reports quarterly earnings", "Active buyers up 15% QoQ"),
|
||||
)
|
||||
for code, name, title, snippet in cases:
|
||||
with self.subTest(stock_code=code, stock_name=name):
|
||||
scored = SearchService._score_news_relevance(
|
||||
_result(title, fresh, snippet=snippet),
|
||||
stock_code=code,
|
||||
stock_name=name,
|
||||
)
|
||||
self.assertEqual(scored.relevance_category, "direct_company_news")
|
||||
joined_reasons = "；".join(scored.relevance_reasons or [])
|
||||
self.assertIn("英文别名", joined_reasons)
|
||||
|
||||
def test_search_stock_news_query_uses_english_alias_for_chinese_display_name(self) -> None:
|
||||
"""massif-01 blocker 1: foreign query path must use canonical English alias, not Chinese name."""
|
||||
service = SearchService(
|
||||
bocha_keys=["dummy_key"],
|
||||
searxng_public_instances_enabled=False,
|
||||
news_max_age_days=3,
|
||||
news_strategy_profile="short",
|
||||
)
|
||||
captured_query = {}
|
||||
|
||||
def _capture(query, max_results, **kwargs):
|
||||
captured_query["value"] = query
|
||||
return _response([_result("Apple earnings beat", datetime.now().date().isoformat())])
|
||||
|
||||
provider = SimpleNamespace(is_available=True, name="USProvider", search=MagicMock(side_effect=_capture))
|
||||
service._providers = [provider]
|
||||
|
||||
with patch("src.search_service.time.sleep"):
|
||||
service.search_stock_news("AAPL", "苹果", max_results=5)
|
||||
|
||||
q = captured_query.get("value", "")
|
||||
self.assertIn("Apple", q)
|
||||
self.assertNotIn("苹果", q)
|
||||
|
||||
def test_search_comprehensive_intel_query_uses_english_alias_for_chinese_display_name(self) -> None:
|
||||
"""search_comprehensive_intel foreign branch must use canonical English alias."""
|
||||
service, mock_search = self._create_service_with_mock_provider(
|
||||
news_max_age_days=3,
|
||||
news_strategy_profile="short",
|
||||
)
|
||||
captured_queries: list[str] = []
|
||||
|
||||
def _capture(query, max_results, **kwargs):
|
||||
captured_queries.append(query)
|
||||
return _response([_result("latest_news", datetime.now().date().isoformat())])
|
||||
|
||||
mock_search.side_effect = _capture
|
||||
|
||||
with patch("src.search_service.time.sleep"):
|
||||
service.search_comprehensive_intel(
|
||||
stock_code="AMZN",
|
||||
stock_name="亚马逊",
|
||||
max_searches=3,
|
||||
)
|
||||
|
||||
self.assertTrue(captured_queries, "no queries captured")
|
||||
for q in captured_queries:
|
||||
self.assertIn("Amazon", q)
|
||||
self.assertNotIn("亚马逊", q)
|
||||
|
||||
def test_search_stock_events_query_uses_english_alias_for_chinese_display_name(self) -> None:
|
||||
"""search_stock_events foreign branch must use canonical English alias."""
|
||||
service = SearchService(
|
||||
bocha_keys=["dummy_key"],
|
||||
searxng_public_instances_enabled=False,
|
||||
news_max_age_days=3,
|
||||
news_strategy_profile="short",
|
||||
)
|
||||
captured_query = {}
|
||||
|
||||
def _capture(query, max_results, **kwargs):
|
||||
captured_query["value"] = query
|
||||
return _response([_result("MSFT earnings beat", datetime.now().date().isoformat())])
|
||||
|
||||
provider = SimpleNamespace(is_available=True, name="USProvider", search=MagicMock(side_effect=_capture))
|
||||
service._providers = [provider]
|
||||
|
||||
with patch("src.search_service.time.sleep"):
|
||||
service.search_stock_events("MSFT", "微软")
|
||||
|
||||
q = captured_query.get("value", "")
|
||||
self.assertIn("Microsoft", q)
|
||||
self.assertNotIn("微软", q)
|
||||
|
||||
def test_stock_english_name_map_is_subset_of_stock_name_map_foreign_keys(self) -> None:
|
||||
"""Single source of truth invariant (massif-01 blocker 3):
|
||||
STOCK_ENGLISH_NAME_MAP keys must be a subset of STOCK_NAME_MAP's
|
||||
foreign-ticker keys. This pins the canonical-boundary contract so the
|
||||
two static tables cannot drift.
|
||||
"""
|
||||
from src.data.stock_mapping import (
|
||||
STOCK_ENGLISH_NAME_MAP,
|
||||
STOCK_NAME_MAP,
|
||||
canonicalize_foreign_stock_code,
|
||||
)
|
||||
|
||||
stock_name_foreign_keys = {
|
||||
canonicalize_foreign_stock_code(code)
|
||||
for code in STOCK_NAME_MAP
|
||||
if SearchService._is_foreign_stock(code)
|
||||
}
|
||||
english_map_keys = {
|
||||
canonicalize_foreign_stock_code(code)
|
||||
for code in STOCK_ENGLISH_NAME_MAP
|
||||
}
|
||||
self.assertTrue(
|
||||
english_map_keys.issubset(stock_name_foreign_keys),
|
||||
f"STOCK_ENGLISH_NAME_MAP has orphan keys: "
|
||||
f"{sorted(english_map_keys - stock_name_foreign_keys)}",
|
||||
)
|
||||
|
||||
def test_score_news_relevance_english_alias_dedup_prevents_double_count(self) -> None:
|
||||
"""massif-01 blocker follow-up: alias expansion dedup prevents
|
||||
double-counting when legal alias and short alias resolve to same term.
|
||||
For AAPL: STOCK_ENGLISH_NAME_MAP['AAPL'] = ('Apple Inc.', 'Apple')
|
||||
_company_identity_terms('Apple Inc.') -> ['Apple Inc.', 'Apple']
|
||||
_company_identity_terms('Apple') -> ['Apple']
|
||||
Without dedup: snippet 'Apple reports earnings beat' would match
|
||||
'Apple' twice (once from each alias path) → 16+16=32 direct_signal
|
||||
With event term +12 = 44 → direct_company_news (incorrect).
|
||||
With dedup: only one 'Apple' term scored → 16 direct_signal
|
||||
With event term +12 = 28 → sector_related_news (correct).
|
||||
"""
|
||||
# Setup: generic market title, snippet with only English alias hit
|
||||
item = SearchResult(
|
||||
title="US stocks mixed after Fed comments",
|
||||
snippet="Apple reports earnings beat and revenue grows.",
|
||||
url="",
|
||||
source=""
|
||||
)
|
||||
scored = SearchService._score_news_relevance(
|
||||
item, stock_code="AAPL", stock_name="苹果"
|
||||
)
|
||||
# Should NOT be direct (insufficient signal without title hit)
|
||||
self.assertEqual(scored.relevance_category, "sector_related_news")
|
||||
self.assertLess(scored.relevance_score, 38)
|
||||
# Should have exactly one hit on the alias term
|
||||
reasons = "；".join(scored.relevance_reasons or [])
|
||||
self.assertIn("摘要命中公司英文别名 Apple", reasons)
|
||||
self.assertNotIn("标题命中公司英文别名", reasons)
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
|
||||
Reference in New Issue
Block a user