mirror of
https://github.com/ZhuLinsen/daily_stock_analysis
synced 2026-09-20 10:53:33 +08:00
feat: add analysis context pack schema (#1449)
This commit is contained in:
Alfred
@@ -26,6 +26,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
|
||||
- [修复] macOS 桌面端将运行时配置迁移到用户数据目录，并在旧 `.app` 包内文件仍可访问时迁移 `.env`、数据库和日志，避免后续替换升级后重新配置。
|
||||
- [改进] 新增运行态市场阶段上下文构造与降级测试。
|
||||
- [文档] 新增 AnalysisContextPack P0 上下文盘点，明确字段质量状态、现有状态映射和首版 pack 边界。
|
||||
- [新功能] 新增 AnalysisContextPack P1 内部契约与脱敏序列化测试。
|
||||
- [修复] 恢复 Agent/历史兼容快照中的关联板块与板块联动字段提取，修复新版首页报告缺少“板块联动”的回归问题。
|
||||
- [改进] P2-min：LLM Prompt 注入市场阶段上下文。
|
||||
|
||||
|
||||
@@ -42,7 +42,7 @@
|
||||
| [Bot 命令与接入](bot-command.md) | Bot 命令、Webhook、平台接入和回调说明 |
|
||||
| [Bot 平台配置](bot/) | 飞书、钉钉、Discord 等 Bot 配置截图和补充说明 |
|
||||
| [实时告警中心](alerts.md) | EventMonitor 基线、Web 规则管理、通知结果、冷却状态和 Phase 边界 |
|
||||
| [分析上下文包 P0 盘点](analysis-context-pack.md) | AnalysisContextPack 首版范围、上下文字段边界、字段质量状态和源码锚点 |
|
||||
| [分析上下文包 P0/P1 契约](analysis-context-pack.md) | AnalysisContextPack 首版范围、上下文字段边界、字段质量状态、P1 内部 schema 和源码锚点 |
|
||||
| [图片识别 Prompt](image-extract-prompt.md) | 图片识别股票信息的 Prompt 与使用边界 |
|
||||
| [OpenClaw Skill 集成](openclaw-skill-integration.md) | OpenClaw / Skill 外部集成说明 |
|
||||
|
||||
|
||||
@@ -43,7 +43,7 @@ This is the entry point for project documentation. The README covers the project
|
||||
| [Bot Commands (EN)](bot-command_EN.md) | Bot commands, webhooks, platform integration, and callback behavior |
|
||||
| [Bot Platform Docs](bot/) <sub><sub></sub></sub> (Chinese-only) | Feishu, DingTalk, Discord, and related Bot configuration screenshots and notes |
|
||||
| [Real-Time Alert Center](alerts.md) <sub><sub></sub></sub> (Chinese-only) | EventMonitor baseline, Web rule management, notification attempts, cooldown state, and phase boundaries |
|
||||
| [Analysis Context Pack P0 Inventory](analysis-context-pack.md) <sub><sub></sub></sub> (Chinese-only) | AnalysisContextPack first-scope boundaries, field quality states, and source anchors |
|
||||
| [Analysis Context Pack P0/P1 Contract](analysis-context-pack.md) <sub><sub></sub></sub> (Chinese-only) | AnalysisContextPack first-scope boundaries, field quality states, P1 internal schema, and source anchors |
|
||||
| [Image Extraction Prompt](image-extract-prompt.md) <sub><sub></sub></sub> (Chinese-only) | Prompt and boundaries for extracting stock information from images |
|
||||
| [OpenClaw Skill Integration](openclaw-skill-integration.md) <sub><sub></sub></sub> (Chinese-only) | OpenClaw / Skill external integration notes |
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# AnalysisContextPack P0：分析上下文盘点与字段边界
|
||||
# AnalysisContextPack：P0 盘点与 P1 内部契约
|
||||
|
||||
本页是 Issue #1389 的 P0 盘点文档，用于记录当前 DSA 分析上下文的真实来源、消费路径、字段状态边界和后续 `AnalysisContextPack` 的首版范围。P0 只做现状盘点和契约边界，不新增 schema、builder 或 runtime 接入。
|
||||
本页是 Issue #1389 的专题文档，用于记录当前 DSA 分析上下文的真实来源、消费路径、字段状态边界，以及 P1 `AnalysisContextPack` 内部契约。P0 负责现状盘点和契约边界；P1 只新增内部 schema/envelope、block catalog、类型约定和脱敏序列化，不新增 builder 或 runtime 接入。
|
||||
|
||||
## 术语与边界
|
||||
|
||||
@@ -19,13 +19,59 @@
|
||||
P0 的目标是让后续 P1/P2/P3 可以基于真实仓库边界设计 `AnalysisContextPack`，而不是提前改造运行时。
|
||||
|
||||
- P0 覆盖普通分析、Agent、告警、持仓、回测、历史、通知七条路径的上下文盘点。
|
||||
- P0 固定字段质量状态词，但不定义 `AnalysisContextPack` schema。
|
||||
- P0 固定字段质量状态词；P1 已新增 `AnalysisContextPack` 内部 schema，但仍不新增 builder、不接入 runtime、不公开完整 pack。
|
||||
- P0 不新增 builder，不新增配置项，不新增数据库字段，不改变 API、报告、历史或通知 payload。
|
||||
- P0 不接入 runtime，不改 `src/` 分析、Agent、告警、持仓、回测或通知逻辑。
|
||||
- P0 不 pack 化 `market_review`、`market_light` 或大盘红绿灯专题快照；这些只作为历史快照中的其他 `report_kind` / 专题消费边界记录。
|
||||
- P0 不把 `fetch_failed` 加入字段质量状态词；`fetch_failed` 与 `not_supported` 的细分留到 P5 数据质量评分与模型提示阶段。
|
||||
- P0 不在 README 扩写实现细节；本页作为专题文档，由 `docs/INDEX.md` / `docs/INDEX_EN.md` 入口发现。
|
||||
|
||||
## P1 内部契约
|
||||
|
||||
P1 落地 `src/schemas/analysis_context_pack.py`，只定义内部 schema/envelope，方便 P2 builder 和 P3 runtime 消费时复用同一结构。P1 不填充运行时数据、不新增 fetcher、不改变 Prompt、不写入 history/task/report metadata，也不把完整 pack 暴露到 API、Web、Bot、Desktop 或通知。
|
||||
|
||||
P1 schema 包含：
|
||||
|
||||
- `PACK_VERSION = "1.0"`，并通过 `AnalysisContextPack.pack_version` 标记契约版本。
|
||||
- `ContextFieldStatus`：只允许 `available`、`missing`、`not_supported`、`fallback`、`stale`、`estimated`、`partial`；`fetch_failed` 仍留到 P5。
|
||||
- `AnalysisSubject`：顶层身份槽，只包含 `code`、`stock_name`、`market`；`exchange`、`currency`、`industry` 留给 P2 builder 从现有 artifacts 填充，不重复新增 `identity` block。
|
||||
- `AnalysisContextItem`：字段级输入项，包含 `status`、`value`、`source`、`timestamp`、`fallback_from`、`missing_reason`、`warnings`、`metadata`。
|
||||
- `AnalysisContextBlock`：数据块级分组，包含 `status`、`items`、`source`、`timestamp`、`warnings`、`metadata`，其中 `items` 是 `Dict[str, AnalysisContextItem]`。
|
||||
- `DataQuality`：P1 只保留 `warnings` 与 `metadata` 容器，不做评分、聚合计数或模型置信度限制。
|
||||
- `AnalysisContextPack`：顶层 envelope，包含 `pack_version`、`subject`、`phase`、`blocks`、`data_quality`、`metadata`、`created_at`。
|
||||
|
||||
时间字段约定：
|
||||
|
||||
- `AnalysisContextPack.created_at` 使用 `datetime`，由 `model_dump(mode="json")` 输出 ISO 8601 字符串。
|
||||
- `AnalysisContextItem.timestamp` 与 `AnalysisContextBlock.timestamp` 使用 `Optional[str]`，约定为 ISO 8601 datetime 字符串；P1 schema 在构造时校验该格式，date-only、自然语言时间或斜杠分隔日期会被拒绝；P2 builder 复用现有 artifact 时间戳时不做强制二次转换。
|
||||
|
||||
状态语义：
|
||||
|
||||
- `block.status` 表示整块可用性。
|
||||
- `item.status` 表示字段级质量。
|
||||
- P1 不实现 `item.status` 到 `block.status` 的自动聚合推导。
|
||||
|
||||
P1 Block Catalog：
|
||||
|
||||
| block key | P1 语义 | P1 边界 |
|
||||
| --- | --- | --- |
|
||||
| `quote` | 实时行情和报价相关输入 | 只定义可表达位置，不抓取或填充数据。 |
|
||||
| `daily_bars` | 完整日线窗口和最近完整日线日期 | P1 不判断 partial bar。 |
|
||||
| `technical` | 技术指标、量价结构和形态 | P1 不生成指标。 |
|
||||
| `fundamentals` | 估值、成长、盈利、财报和股东回报 | P1 不新增基本面 fetcher。 |
|
||||
| `news` | 新闻、公告、舆情和催化事件输入 | P1 不改变新闻搜索。 |
|
||||
| `portfolio` | 是否持仓、账户摘要、成本、数量、仓位和 stale 摘要 | P1 不纳入交易流水、现金流水或完整账户隐私数据。 |
|
||||
| `chip` / `capital_flow` | 筹码、资金流和主力行为 | 后续扩展键，P1 只允许契约表达。 |
|
||||
| `events` / `market_context` | 风险事件、市场宽度、指数、板块和热点环境 | 后续扩展键，不把 `market_review` / `market_light` 作为首版单股 pack。 |
|
||||
|
||||
`phase` 字段只接收 #1386 `MarketPhaseContext.to_dict()` 产物，保持 `Dict[str, Any]`，不重新定义 phase enum 或 phase 子模型。
|
||||
|
||||
脱敏边界：
|
||||
|
||||
- `AnalysisContextPack.to_safe_dict()` 先执行 `model_dump(mode="json")`，再调用 `redact_sensitive_mapping()`。
|
||||
- `redact_sensitive_mapping()` 只做 dict/list 的 key-based 递归脱敏，命中 `api_key`、`access_token`、`refresh_token`、`authorization_header`、`webhook_url`、`password`、`cookie`、`secret`、`token`、`sendkey`、`license_key` 等敏感键或短语时把值替换为 `[REDACTED]`。
|
||||
- P1 不扫描普通字符串值，不做 URL 正则脱敏，不把 `data_api` 或裸 `api` / `key` 当作敏感命中，避免把本契约扩展成通用 secrets engine。
|
||||
|
||||
## 字段质量状态
|
||||
|
||||
未来 pack 的字段质量状态在 P0 只固定下列七词。它们描述字段或数据块的质量，不描述业务流程是否成功。
|
||||
|
||||
@@ -1,12 +1,30 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
===================================
|
||||
Report Engine Schemas
|
||||
DSA Schemas
|
||||
===================================
|
||||
|
||||
Pydantic schemas for LLM report output validation.
|
||||
Pydantic schemas for report output validation and internal contracts.
|
||||
"""
|
||||
|
||||
from src.schemas.analysis_context_pack import (
|
||||
PACK_VERSION,
|
||||
AnalysisContextBlock,
|
||||
AnalysisContextItem,
|
||||
AnalysisContextPack,
|
||||
AnalysisSubject,
|
||||
ContextFieldStatus,
|
||||
DataQuality,
|
||||
)
|
||||
from src.schemas.report_schema import AnalysisReportSchema
|
||||
|
||||
__all__ = ["AnalysisReportSchema"]
|
||||
__all__ = [
|
||||
"AnalysisReportSchema",
|
||||
"PACK_VERSION",
|
||||
"AnalysisContextBlock",
|
||||
"AnalysisContextItem",
|
||||
"AnalysisContextPack",
|
||||
"AnalysisSubject",
|
||||
"ContextFieldStatus",
|
||||
"DataQuality",
|
||||
]
|
||||
|
||||
123
src/schemas/analysis_context_pack.py
Normal file
123
src/schemas/analysis_context_pack.py
Normal file
@@ -0,0 +1,123 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Internal AnalysisContextPack schema for Issue #1389 P1."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from enum import Enum
|
||||
from typing import Any, Dict, List, Literal, Mapping, Optional
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, TypeAdapter, field_validator
|
||||
|
||||
from src.utils.sanitize import redact_sensitive_mapping
|
||||
|
||||
|
||||
PACK_VERSION = "1.0"
|
||||
_PACK_VERSION_ADAPTER = TypeAdapter(Literal["1.0"])
|
||||
|
||||
|
||||
class _AnalysisContextModel(BaseModel):
|
||||
"""Base model for the internal P1 contract."""
|
||||
|
||||
model_config = ConfigDict(validate_assignment=True)
|
||||
|
||||
|
||||
def _validate_iso8601_timestamp(value: Optional[str]) -> Optional[str]:
|
||||
if value is None:
|
||||
return value
|
||||
if "T" not in value:
|
||||
raise ValueError("timestamp must be an ISO 8601 datetime string")
|
||||
normalized = value[:-1] + "+00:00" if value.endswith("Z") else value
|
||||
try:
|
||||
datetime.fromisoformat(normalized)
|
||||
except ValueError as exc:
|
||||
raise ValueError("timestamp must be an ISO 8601 datetime string") from exc
|
||||
return value
|
||||
|
||||
|
||||
class ContextFieldStatus(str, Enum):
|
||||
"""Field or block quality state for the first AnalysisContextPack contract."""
|
||||
|
||||
AVAILABLE = "available"
|
||||
MISSING = "missing"
|
||||
NOT_SUPPORTED = "not_supported"
|
||||
FALLBACK = "fallback"
|
||||
STALE = "stale"
|
||||
ESTIMATED = "estimated"
|
||||
PARTIAL = "partial"
|
||||
|
||||
|
||||
class AnalysisSubject(_AnalysisContextModel):
|
||||
"""Minimal stock identity slot for P1."""
|
||||
|
||||
code: str
|
||||
stock_name: Optional[str] = None
|
||||
market: Optional[str] = None
|
||||
|
||||
|
||||
class AnalysisContextItem(_AnalysisContextModel):
|
||||
"""Field-level input context item."""
|
||||
|
||||
status: ContextFieldStatus
|
||||
value: Optional[Any] = None
|
||||
source: Optional[str] = None
|
||||
timestamp: Optional[str] = None
|
||||
fallback_from: Optional[str] = None
|
||||
missing_reason: Optional[str] = None
|
||||
warnings: List[str] = Field(default_factory=list)
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("timestamp")
|
||||
@classmethod
|
||||
def _timestamp_must_be_iso8601(cls, value: Optional[str]) -> Optional[str]:
|
||||
return _validate_iso8601_timestamp(value)
|
||||
|
||||
|
||||
class AnalysisContextBlock(_AnalysisContextModel):
|
||||
"""Block-level grouping for related context items."""
|
||||
|
||||
status: ContextFieldStatus
|
||||
items: Dict[str, AnalysisContextItem] = Field(default_factory=dict)
|
||||
source: Optional[str] = None
|
||||
timestamp: Optional[str] = None
|
||||
warnings: List[str] = Field(default_factory=list)
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
@field_validator("timestamp")
|
||||
@classmethod
|
||||
def _timestamp_must_be_iso8601(cls, value: Optional[str]) -> Optional[str]:
|
||||
return _validate_iso8601_timestamp(value)
|
||||
|
||||
|
||||
class DataQuality(_AnalysisContextModel):
|
||||
"""Container for future quality summaries without P5 scoring semantics."""
|
||||
|
||||
warnings: List[str] = Field(default_factory=list)
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
|
||||
|
||||
class AnalysisContextPack(_AnalysisContextModel):
|
||||
"""Versioned internal analysis input envelope."""
|
||||
|
||||
subject: AnalysisSubject
|
||||
pack_version: Literal["1.0"] = PACK_VERSION
|
||||
phase: Optional[Dict[str, Any]] = None
|
||||
blocks: Dict[str, AnalysisContextBlock] = Field(default_factory=dict)
|
||||
data_quality: DataQuality = Field(default_factory=DataQuality)
|
||||
metadata: Dict[str, Any] = Field(default_factory=dict)
|
||||
created_at: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
|
||||
|
||||
def to_safe_dict(self) -> Dict[str, Any]:
|
||||
"""Return a JSON-safe dict with sensitive mapping values redacted."""
|
||||
return redact_sensitive_mapping(self.model_dump(mode="json"))
|
||||
|
||||
def model_copy(
|
||||
self,
|
||||
*,
|
||||
update: Optional[Mapping[str, Any]] = None,
|
||||
deep: bool = False,
|
||||
) -> "AnalysisContextPack":
|
||||
"""Copy the pack without bypassing the fixed P1 contract fields."""
|
||||
if update is not None and "pack_version" in update:
|
||||
_PACK_VERSION_ADAPTER.validate_python(update["pack_version"])
|
||||
return super().model_copy(update=update, deep=deep)
|
||||
@@ -7,6 +7,50 @@ import re
|
||||
from typing import Any
|
||||
|
||||
|
||||
_REDACTED = "[REDACTED]"
|
||||
_SENSITIVE_KEY_PARTS = {
|
||||
"authorization",
|
||||
"cookie",
|
||||
"password",
|
||||
"secret",
|
||||
"sendkey",
|
||||
"token",
|
||||
"webhook",
|
||||
}
|
||||
_SENSITIVE_KEY_PHRASES = {
|
||||
"access_token",
|
||||
"accesstoken",
|
||||
"api_key",
|
||||
"apikey",
|
||||
"api_token",
|
||||
"apitoken",
|
||||
"auth_token",
|
||||
"authtoken",
|
||||
"authorization_header",
|
||||
"authorizationheader",
|
||||
"license_key",
|
||||
"licensekey",
|
||||
"private_key",
|
||||
"privatekey",
|
||||
"refresh_token",
|
||||
"refreshtoken",
|
||||
"secret_key",
|
||||
"secretkey",
|
||||
"session_token",
|
||||
"sessiontoken",
|
||||
"send_key",
|
||||
"sendkey",
|
||||
"webhook_url",
|
||||
"webhookurl",
|
||||
}
|
||||
_SENSITIVE_COMPACT_KEY_PHRASES = {
|
||||
phrase.replace("_", "") for phrase in _SENSITIVE_KEY_PHRASES
|
||||
}
|
||||
_SENSITIVE_COMPACT_KEY_PATTERN = re.compile(
|
||||
r"authorization|cookie|password|secret|sendkey|token(?!s)|webhook"
|
||||
)
|
||||
|
||||
|
||||
def sanitize_diagnostic_text(text: Any, *, max_length: int = 300) -> str:
|
||||
"""Redact common secrets and URLs from diagnostic text."""
|
||||
sanitized = str(text or "").strip()
|
||||
@@ -16,3 +60,51 @@ def sanitize_diagnostic_text(text: Any, *, max_length: int = 300) -> str:
|
||||
sanitized = re.sub(r"(?i)(token|secret|password|sendkey)([=:]\s*)[^\s,;&]+", r"\1\2[REDACTED]", sanitized)
|
||||
sanitized = re.sub(r"https?://[^\s]+", "[REDACTED_URL]", sanitized)
|
||||
return " ".join(sanitized.split())[:max_length]
|
||||
|
||||
|
||||
def redact_sensitive_mapping(obj: Any) -> Any:
|
||||
"""Recursively redact sensitive values from mappings by key name only.
|
||||
|
||||
This helper intentionally does not inspect arbitrary string values. P1 only
|
||||
needs a deterministic serializer for AnalysisContextPack dictionaries.
|
||||
"""
|
||||
if isinstance(obj, dict):
|
||||
redacted = {}
|
||||
for key, value in obj.items():
|
||||
if _is_sensitive_mapping_key(key):
|
||||
redacted[key] = _REDACTED
|
||||
else:
|
||||
redacted[key] = redact_sensitive_mapping(value)
|
||||
return redacted
|
||||
if isinstance(obj, list):
|
||||
return [redact_sensitive_mapping(item) for item in obj]
|
||||
return obj
|
||||
|
||||
|
||||
def _is_sensitive_mapping_key(key: Any) -> bool:
|
||||
key_text = str(key or "").strip()
|
||||
if not key_text:
|
||||
return False
|
||||
parts = _mapping_key_parts(key_text)
|
||||
if _has_sensitive_phrase("_".join(parts)):
|
||||
return True
|
||||
return bool(set(parts) & _SENSITIVE_KEY_PARTS)
|
||||
|
||||
|
||||
def _has_sensitive_phrase(normalized_key: str) -> bool:
|
||||
padded_key = f"_{normalized_key}_"
|
||||
if any(f"_{phrase}_" in padded_key for phrase in _SENSITIVE_KEY_PHRASES):
|
||||
return True
|
||||
compact_key = normalized_key.replace("_", "")
|
||||
if any(phrase in compact_key for phrase in _SENSITIVE_COMPACT_KEY_PHRASES):
|
||||
return True
|
||||
return bool(_SENSITIVE_COMPACT_KEY_PATTERN.search(compact_key))
|
||||
|
||||
|
||||
def _mapping_key_parts(key_text: str) -> list[str]:
|
||||
split_camel = re.sub(r"([a-z0-9])([A-Z])", r"\1_\2", key_text)
|
||||
return [
|
||||
part.lower()
|
||||
for part in re.split(r"[^A-Za-z0-9]+", split_camel)
|
||||
if part
|
||||
]
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Contract checks for the AnalysisContextPack P0 inventory doc."""
|
||||
"""Contract checks for the AnalysisContextPack P0/P1 contract doc."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
@@ -24,6 +24,7 @@ def test_analysis_context_pack_doc_has_required_sections() -> None:
|
||||
for heading in (
|
||||
"## 术语与边界",
|
||||
"## P0 范围与非目标",
|
||||
"## P1 内部契约",
|
||||
"## 字段质量状态",
|
||||
"## 现有状态映射",
|
||||
"## 七路径盘点",
|
||||
@@ -96,9 +97,10 @@ def test_analysis_context_pack_doc_records_non_goals_and_safety_boundaries() ->
|
||||
doc = _read_doc()
|
||||
|
||||
for token in (
|
||||
"不定义 `AnalysisContextPack` schema",
|
||||
"P1 已新增 `AnalysisContextPack` 内部 schema",
|
||||
"不新增 builder",
|
||||
"不接入 runtime",
|
||||
"不公开完整 pack",
|
||||
"不 pack 化 `market_review`",
|
||||
"`market_light`",
|
||||
"`fetch_failed` 与 `not_supported` 的细分留到 P5",
|
||||
@@ -113,6 +115,93 @@ def test_analysis_context_pack_doc_records_non_goals_and_safety_boundaries() ->
|
||||
assert token in doc
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_defines_p1_schema_contract() -> None:
|
||||
section = _section(_read_doc(), "P1 内部契约")
|
||||
|
||||
for token in (
|
||||
"`src/schemas/analysis_context_pack.py`",
|
||||
"`PACK_VERSION = \"1.0\"`",
|
||||
"`ContextFieldStatus`",
|
||||
"`AnalysisSubject`",
|
||||
"`AnalysisContextItem`",
|
||||
"`AnalysisContextBlock`",
|
||||
"`DataQuality`",
|
||||
"`AnalysisContextPack`",
|
||||
"`MarketPhaseContext.to_dict()`",
|
||||
):
|
||||
assert token in section
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_records_p1_block_catalog() -> None:
|
||||
section = _section(_read_doc(), "P1 内部契约")
|
||||
|
||||
for token in (
|
||||
"P1 Block Catalog",
|
||||
"`quote`",
|
||||
"`daily_bars`",
|
||||
"`technical`",
|
||||
"`fundamentals`",
|
||||
"`news`",
|
||||
"`portfolio`",
|
||||
"`chip` / `capital_flow`",
|
||||
"`events` / `market_context`",
|
||||
"不重复新增 `identity` block",
|
||||
):
|
||||
assert token in section
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_records_p1_time_and_status_semantics() -> None:
|
||||
section = _section(_read_doc(), "P1 内部契约")
|
||||
|
||||
for token in (
|
||||
"`AnalysisContextPack.created_at` 使用 `datetime`",
|
||||
"`model_dump(mode=\"json\")` 输出 ISO 8601",
|
||||
"`AnalysisContextItem.timestamp`",
|
||||
"`AnalysisContextBlock.timestamp`",
|
||||
"Optional[str]",
|
||||
"构造时校验",
|
||||
"date-only",
|
||||
"`block.status` 表示整块可用性",
|
||||
"`item.status` 表示字段级质量",
|
||||
"不实现 `item.status` 到 `block.status` 的自动聚合推导",
|
||||
):
|
||||
assert token in section
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_records_p1_redaction_contract() -> None:
|
||||
section = _section(_read_doc(), "P1 内部契约")
|
||||
|
||||
for token in (
|
||||
"`AnalysisContextPack.to_safe_dict()`",
|
||||
"`redact_sensitive_mapping()`",
|
||||
"`api_key`",
|
||||
"`access_token`",
|
||||
"`authorization_header`",
|
||||
"`webhook_url`",
|
||||
"`license_key`",
|
||||
"[REDACTED]",
|
||||
"`data_api`",
|
||||
"不扫描普通字符串值",
|
||||
"不做 URL 正则脱敏",
|
||||
):
|
||||
assert token in section
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_keeps_later_phases_out_of_p1() -> None:
|
||||
section = _section(_read_doc(), "P1 内部契约")
|
||||
|
||||
for token in (
|
||||
"不填充运行时数据",
|
||||
"不新增 fetcher",
|
||||
"不改变 Prompt",
|
||||
"不写入 history/task/report metadata",
|
||||
"不把完整 pack 暴露到 API、Web、Bot、Desktop 或通知",
|
||||
"P2 builder",
|
||||
"P3 runtime",
|
||||
):
|
||||
assert token in section
|
||||
|
||||
|
||||
def test_analysis_context_pack_doc_maps_existing_status_terms() -> None:
|
||||
section = _section(_read_doc(), "现有状态映射")
|
||||
|
||||
@@ -159,10 +248,11 @@ def test_analysis_context_pack_doc_updates_indexes_and_changelog() -> None:
|
||||
index_en = (PROJECT_ROOT / "docs" / "INDEX_EN.md").read_text(encoding="utf-8")
|
||||
changelog = (PROJECT_ROOT / "docs" / "CHANGELOG.md").read_text(encoding="utf-8")
|
||||
|
||||
assert "[分析上下文包 P0 盘点](analysis-context-pack.md)" in index
|
||||
assert "[分析上下文包 P0/P1 契约](analysis-context-pack.md)" in index
|
||||
assert (
|
||||
"[Analysis Context Pack P0 Inventory](analysis-context-pack.md) "
|
||||
"<sub><sub></sub></sub> "
|
||||
"[Analysis Context Pack P0/P1 Contract](analysis-context-pack.md) "
|
||||
"<sub><sub></sub></sub> "
|
||||
"(Chinese-only)"
|
||||
) in index_en
|
||||
assert "[文档] 新增 AnalysisContextPack P0 上下文盘点" in changelog
|
||||
assert "[新功能] 新增 AnalysisContextPack P1 内部契约与脱敏序列化测试" in changelog
|
||||
|
||||
316
tests/test_analysis_context_pack_schema.py
Normal file
316
tests/test_analysis_context_pack_schema.py
Normal file
@@ -0,0 +1,316 @@
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Tests for the Issue #1389 P1 AnalysisContextPack schema."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import pytest
|
||||
from pydantic import ValidationError
|
||||
|
||||
from src.core.trading_calendar import build_market_phase_context
|
||||
from src.schemas.analysis_context_pack import (
|
||||
PACK_VERSION,
|
||||
AnalysisContextBlock,
|
||||
AnalysisContextItem,
|
||||
AnalysisContextPack,
|
||||
AnalysisSubject,
|
||||
ContextFieldStatus,
|
||||
DataQuality,
|
||||
)
|
||||
from src.utils.sanitize import redact_sensitive_mapping
|
||||
|
||||
|
||||
def _subject() -> AnalysisSubject:
|
||||
return AnalysisSubject(code="600519", stock_name="贵州茅台", market="cn")
|
||||
|
||||
|
||||
def test_pack_defaults_and_json_serialization_are_stable() -> None:
|
||||
pack = AnalysisContextPack(
|
||||
subject=_subject(),
|
||||
created_at=datetime(2026, 5, 24, 9, 30, tzinfo=timezone.utc),
|
||||
)
|
||||
|
||||
dumped = pack.model_dump(mode="json")
|
||||
json.dumps(dumped, ensure_ascii=False)
|
||||
|
||||
assert dumped["pack_version"] == PACK_VERSION
|
||||
assert dumped["subject"] == {
|
||||
"code": "600519",
|
||||
"stock_name": "贵州茅台",
|
||||
"market": "cn",
|
||||
}
|
||||
assert dumped["blocks"] == {}
|
||||
assert dumped["data_quality"] == {"warnings": [], "metadata": {}}
|
||||
assert dumped["metadata"] == {}
|
||||
assert dumped["created_at"] == "2026-05-24T09:30:00Z"
|
||||
|
||||
|
||||
def test_pack_version_is_fixed_to_p1_contract() -> None:
|
||||
pack = AnalysisContextPack(subject=_subject())
|
||||
|
||||
assert pack.pack_version == PACK_VERSION
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
AnalysisContextPack(subject=_subject(), pack_version="2.0")
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
pack.pack_version = "2.0"
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
pack.model_copy(update={"pack_version": "2.0"})
|
||||
|
||||
copied = pack.model_copy(update={"metadata": {"trace_id": "q-1"}})
|
||||
|
||||
assert copied.pack_version == PACK_VERSION
|
||||
assert copied.metadata == {"trace_id": "q-1"}
|
||||
assert pack.to_safe_dict()["pack_version"] == PACK_VERSION
|
||||
|
||||
|
||||
def test_pack_model_copy_preserves_shallow_copy_semantics() -> None:
|
||||
block = AnalysisContextBlock(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
items={
|
||||
"price": AnalysisContextItem(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
value=1880.0,
|
||||
)
|
||||
},
|
||||
)
|
||||
pack = AnalysisContextPack(subject=_subject(), blocks={"quote": block})
|
||||
|
||||
copied = pack.model_copy(update={"metadata": {"trace_id": "q-1"}})
|
||||
|
||||
assert copied.blocks is pack.blocks
|
||||
assert copied.blocks["quote"] is block
|
||||
assert copied.metadata == {"trace_id": "q-1"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("item_ts", "block_ts"),
|
||||
(
|
||||
("2026-05-24T09:30:00+08:00", "2026-05-24T09:30:01+08:00"),
|
||||
("2026-05-24T01:30:00Z", "2026-05-24T01:30:01Z"),
|
||||
),
|
||||
)
|
||||
def test_item_and_block_timestamp_use_iso_strings(
|
||||
item_ts: str,
|
||||
block_ts: str,
|
||||
) -> None:
|
||||
item = AnalysisContextItem(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
value=1880.0,
|
||||
timestamp=item_ts,
|
||||
)
|
||||
block = AnalysisContextBlock(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
items={"price": item},
|
||||
timestamp=block_ts,
|
||||
)
|
||||
|
||||
dumped = block.model_dump(mode="json")
|
||||
|
||||
assert dumped["timestamp"] == block_ts
|
||||
assert dumped["items"]["price"]["timestamp"] == item_ts
|
||||
|
||||
|
||||
@pytest.mark.parametrize("timestamp", ("yesterday", "2026/05/24", "2026-05-24"))
|
||||
def test_item_and_block_timestamp_reject_non_iso_datetime_strings(
|
||||
timestamp: str,
|
||||
) -> None:
|
||||
with pytest.raises(ValidationError):
|
||||
AnalysisContextItem(status=ContextFieldStatus.AVAILABLE, timestamp=timestamp)
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
AnalysisContextBlock(status=ContextFieldStatus.AVAILABLE, timestamp=timestamp)
|
||||
|
||||
|
||||
def test_item_and_block_reject_invalid_assignment_updates() -> None:
|
||||
item = AnalysisContextItem(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
timestamp="2026-05-24T09:30:00+08:00",
|
||||
)
|
||||
block = AnalysisContextBlock(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
items={"price": item},
|
||||
timestamp="2026-05-24T09:30:01+08:00",
|
||||
)
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
item.timestamp = "yesterday"
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
item.status = "fetch_failed"
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
block.timestamp = "2026/05/24"
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
block.status = "fetch_failed"
|
||||
|
||||
assert item.timestamp == "2026-05-24T09:30:00+08:00"
|
||||
assert item.status == ContextFieldStatus.AVAILABLE
|
||||
assert block.timestamp == "2026-05-24T09:30:01+08:00"
|
||||
assert block.status == ContextFieldStatus.AVAILABLE
|
||||
|
||||
|
||||
def test_context_field_status_allows_only_p0_quality_states() -> None:
|
||||
for state in (
|
||||
"available",
|
||||
"missing",
|
||||
"not_supported",
|
||||
"fallback",
|
||||
"stale",
|
||||
"estimated",
|
||||
"partial",
|
||||
):
|
||||
assert ContextFieldStatus(state).value == state
|
||||
|
||||
with pytest.raises(ValueError):
|
||||
ContextFieldStatus("fetch_failed")
|
||||
|
||||
with pytest.raises(ValidationError):
|
||||
AnalysisContextItem(status="fetch_failed")
|
||||
|
||||
|
||||
def test_market_phase_context_dict_can_be_used_as_phase_slot() -> None:
|
||||
phase = build_market_phase_context(
|
||||
market="cn",
|
||||
current_time=datetime(2026, 5, 24, 9, 0, tzinfo=timezone.utc),
|
||||
trigger_source="system",
|
||||
analysis_intent="auto",
|
||||
).to_dict()
|
||||
pack = AnalysisContextPack(subject=_subject(), phase=phase)
|
||||
|
||||
assert pack.phase == phase
|
||||
assert isinstance(pack.model_dump(mode="json")["phase"], dict)
|
||||
|
||||
|
||||
def test_block_and_item_status_are_independent_contract_fields() -> None:
|
||||
block = AnalysisContextBlock(
|
||||
status=ContextFieldStatus.PARTIAL,
|
||||
items={
|
||||
"price": AnalysisContextItem(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
value=1880.0,
|
||||
),
|
||||
"turnover_rate": AnalysisContextItem(
|
||||
status=ContextFieldStatus.MISSING,
|
||||
missing_reason="provider_empty",
|
||||
),
|
||||
},
|
||||
)
|
||||
|
||||
dumped = block.model_dump(mode="json")
|
||||
|
||||
assert dumped["status"] == "partial"
|
||||
assert dumped["items"]["price"]["status"] == "available"
|
||||
assert dumped["items"]["turnover_rate"]["status"] == "missing"
|
||||
|
||||
|
||||
def test_data_quality_is_container_only() -> None:
|
||||
data_quality = DataQuality(
|
||||
warnings=["quote_stale"],
|
||||
metadata={"note": "P1 does not define scoring"},
|
||||
)
|
||||
|
||||
assert data_quality.model_dump(mode="json") == {
|
||||
"warnings": ["quote_stale"],
|
||||
"metadata": {"note": "P1 does not define scoring"},
|
||||
}
|
||||
|
||||
|
||||
def test_redact_sensitive_mapping_recurses_dicts_and_lists_by_key() -> None:
|
||||
payload = {
|
||||
"API_KEY": "ak-secret",
|
||||
"OPENAI_API_KEY": "openai-secret",
|
||||
"GEMINI_API_KEY": "gemini-secret",
|
||||
"openai_api_key_value": "openai-secret-value",
|
||||
"vendorsecretkey": "vendor-secret-key",
|
||||
"apitoken": "api-token-secret",
|
||||
"secretvalue": "secret-value",
|
||||
"passwordvalue": "password-value",
|
||||
"tokenvalue": "token-value",
|
||||
"data_api": "akshare",
|
||||
"dataApi": "akshare-camel",
|
||||
"api_url": "https://example.test/data",
|
||||
"prompt_tokens": 42,
|
||||
"input_tokens": 11,
|
||||
"output_tokens": 12,
|
||||
"total_tokens": 23,
|
||||
"nested": [
|
||||
{
|
||||
"authorization_header": "Bearer token",
|
||||
"authorizationHeader": "Bearer camel-token",
|
||||
"license_key": "license-secret",
|
||||
"vendor_license_key": "vendor-license-secret",
|
||||
"source": "provider",
|
||||
},
|
||||
{
|
||||
"webhook_url": "https://hooks.example.test/abc",
|
||||
"send_key": "send-key-secret",
|
||||
"sessionToken": "session-token-secret",
|
||||
"normal": "kept",
|
||||
},
|
||||
],
|
||||
"metadata": {"Cookie": "session=abc", "count": 1},
|
||||
}
|
||||
|
||||
redacted = redact_sensitive_mapping(payload)
|
||||
|
||||
assert redacted["API_KEY"] == "[REDACTED]"
|
||||
assert redacted["OPENAI_API_KEY"] == "[REDACTED]"
|
||||
assert redacted["GEMINI_API_KEY"] == "[REDACTED]"
|
||||
assert redacted["openai_api_key_value"] == "[REDACTED]"
|
||||
assert redacted["vendorsecretkey"] == "[REDACTED]"
|
||||
assert redacted["apitoken"] == "[REDACTED]"
|
||||
assert redacted["secretvalue"] == "[REDACTED]"
|
||||
assert redacted["passwordvalue"] == "[REDACTED]"
|
||||
assert redacted["tokenvalue"] == "[REDACTED]"
|
||||
assert redacted["data_api"] == "akshare"
|
||||
assert redacted["dataApi"] == "akshare-camel"
|
||||
assert redacted["api_url"] == "https://example.test/data"
|
||||
assert redacted["prompt_tokens"] == 42
|
||||
assert redacted["input_tokens"] == 11
|
||||
assert redacted["output_tokens"] == 12
|
||||
assert redacted["total_tokens"] == 23
|
||||
assert redacted["nested"][0]["authorization_header"] == "[REDACTED]"
|
||||
assert redacted["nested"][0]["authorizationHeader"] == "[REDACTED]"
|
||||
assert redacted["nested"][0]["license_key"] == "[REDACTED]"
|
||||
assert redacted["nested"][0]["vendor_license_key"] == "[REDACTED]"
|
||||
assert redacted["nested"][0]["source"] == "provider"
|
||||
assert redacted["nested"][1]["webhook_url"] == "[REDACTED]"
|
||||
assert redacted["nested"][1]["send_key"] == "[REDACTED]"
|
||||
assert redacted["nested"][1]["sessionToken"] == "[REDACTED]"
|
||||
assert redacted["nested"][1]["normal"] == "kept"
|
||||
assert redacted["metadata"]["Cookie"] == "[REDACTED]"
|
||||
assert redacted["metadata"]["count"] == 1
|
||||
|
||||
|
||||
def test_pack_safe_dict_redacts_sensitive_metadata_but_keeps_business_fields() -> None:
|
||||
pack = AnalysisContextPack(
|
||||
subject=_subject(),
|
||||
blocks={
|
||||
"quote": AnalysisContextBlock(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
items={
|
||||
"price": AnalysisContextItem(
|
||||
status=ContextFieldStatus.AVAILABLE,
|
||||
value=1880.0,
|
||||
source="akshare",
|
||||
metadata={"access_token": "secret", "data_api": "kept"},
|
||||
)
|
||||
},
|
||||
)
|
||||
},
|
||||
metadata={"webhook_url": "https://hooks.example.test/abc", "trace_id": "q-1"},
|
||||
)
|
||||
|
||||
safe = pack.to_safe_dict()
|
||||
|
||||
assert safe["metadata"]["webhook_url"] == "[REDACTED]"
|
||||
assert safe["metadata"]["trace_id"] == "q-1"
|
||||
price_metadata = safe["blocks"]["quote"]["items"]["price"]["metadata"]
|
||||
assert price_metadata["access_token"] == "[REDACTED]"
|
||||
assert price_metadata["data_api"] == "kept"
|
||||
Reference in New Issue
Block a user