fix: redact short credentials in CLI diagnostics (#2118)

* fix: redact short credentials in CLI diagnostics

* fix(review-feedback-2111): Redact indented values under empty sensitive YAML fields and preserve

* fix: close structured diagnostic redaction gaps

* fix(review-feedback-2111): Treat comment-only YAML values as empty blocks and Redact indentless

* fix: redact YAML node property blocks

* fix: redact normalized proxy authorization fields

* fix: close multiline diagnostic redaction gaps

* fix: redact spaced credential labels

* fix(review-feedback-2111): Consume YAML blocks with node properties

* fix: include registered spaced credential labels

* fix: close remaining structured redaction gaps

* fix: redact shell words and explicit YAML mappings

* fix: redact single-quoted structured keys

* fix: redact shell append assignments

* fix: redact quoted YAML explicit keys

* fix(review-feedback-2111): add suffix text or drop segments from the env name, such as DeepSeek

* fix(review-feedback-2111): trimming the new docs/CHANGELOG

* fix(review-feedback-2111): update the PR description's verification counts/ranges to match the

* fix(review-feedback-2111): 评审结论 - 代码检查 :当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要:基于

* fix(review-feedback-2111): 补上 helper 级和 non-zero-exit preview 级回归用例,避免文档与运行时行为再次漂移

* fix(review-feedback-2111): add focused helper-level and non-zero-exit preview regressions for

* fix(review-feedback-2111): 评审结论 - 代码检查 :当前整个 PR 仍有 1 个未关闭的高置信度代码 blocker。最新复核摘要:On the current

* fix(review-feedback-2111): update the PR description's reported current head from 27a013fbf to

* fix: redact sensitive env names embedded inside command substitutions across multi-segment diagnostics

- 覆盖 OPENAI_API_KEY=sk-12345 这种首段为非敏感赋值、值里又嵌敏感名的情况
- 当 $(...) 的前置赋值是敏感名时跳过尾扫避免双重改写,非敏感名仍需进入尾扫
- 新增测试覆盖多段 + 同函数敏感+非敏感赋值的复合诊断文本
- 使用 [A-Z][A-Z0-9_]* token 扫描找到 ALL  中的敏感 env 名引用

* fix(review-2111): redact export SENSITIVE=$(...) without dropping trailing fields

Round-3 review blocker closure for PR #2118 (issue #1784):

OR-COR-7c0a5d41 — the form

    export SENSITIVE_ENV=$(printenv OTHER_SECRET) session_id=dup1 token_budget=1000

previously lost ``session_id=dup1`` (case 1) and ALL trailing fields
(case 2 with ``echo OPENAI_API_KEY=sk-12345``) because the second-pass
``$(...)`` scan re-added the same span that the first pass had already
replaced, and ``_replace_spans`` silently dropped the duplicated
region's width worth of trailing characters.

Two fixes:

1. Track first-pass sensitive-assignment replacement spans
   (``first_pass_spans``) and skip any ``$(...)`` whose start lies
   inside one of those spans. This is the principled guard against
   the overlap regardless of where the leading assignment sits.
   Previously the second-pass computed a "prior prefix" via three
   independent regex branches (semicolon-separated / newline-
   separated / head-of-string), each of which only matched bare
   ``NAME=`` — so ``export NAME=`` slipped through and the second
   pass double-rewrote the same span.

2. Add ``(?:export[ \t]+)?`` to all three prior-prefix regexes so
   that even if the first-pass span guard were ever evaded, the
   leading ``export SENSITIVE=`` would still be recognised and the
   second pass would skip the inner ``$(...)``.

Regression tests cover the two exact reproductions from the review
(``session_id=dup1 token_budget=1000`` and ``session_id=dup3``), plus
a non-``export`` control case to lock in the existing behaviour. Full
``tests/test_local_cli_backend.py`` passes 363/363; the ``tests/test_stock_*``
subset is unaffected (460 passed across the CLI + stock subset).

---------

Co-authored-by: zhulinsen <zhuls97@163.com>
Co-authored-by: xxiaoxiong <xxiaoxiong@nicholasxiong.cn>
This commit is contained in:
Nicholas-Xiong
2026-07-30 23:49:54 +09:00
committed by GitHub
parent 03bae035a6
commit ee3d3da1c5
5 changed files with 2828 additions and 7 deletions

View File

@@ -15,6 +15,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/).
- [新功能] 新增按 individual SkillAgent 自身 signal、版本化 engine 与本地已存同源日线窗口计算并持久化 `skill_opinion_outcomes` 的核心服务;本阶段不提供管理员 API、表现统计、样本充足度或权重调整。
<!-- 新条目格式:- [类型] 描述(类型取值:新功能/改进/修复/文档/测试/chore-->
<!-- 每条独立一行追加到本段末尾,无需分类标题,合并时冲突最小 -->
- [修复] 本地 CLI 的 `stdout_preview` / `stderr_preview` 按环境变量、JSON、YAML/日志标量与 URL 的独立契约脱敏短凭证,避免小于 32 字符的 API key、secret 或 token 进入诊断;普通字段仅按敏感名称判定,未加引号的 YAML 敏感标量则 fail-closed 脱敏至行尾refs #1784)。
- [修复] `redact_diagnostic_text()``export SENSITIVE_ENV=$(printenv OTHER_SECRET) session_id=...` 形态下不再因第二遍 `$(...)` 扫描与第一遍敏感赋值替换区重叠而吞掉 `session_id` 等尾随非敏感诊断字段;第二遍扫描现以 first-pass 已替换 span 列表为可信跳过表,并对 prior-head / prior-semicolon 分支的 leading regex 加上 `(?:export[ \t]+)?` 前缀,使 `export FOO=$(...)``FOO=$(...)` 在所有分支行为对齐(关闭 PR #2118 review blocker OR-COR-7c0a5d41
## [3.28.0] - 2026-07-26

View File

@@ -52,6 +52,7 @@ AGENT_GENERATION_BACKEND=auto
- 本地 CLI backend 不支持 streaming。请求 stream 时会自动降级为 non-stream不会因此返回 `capability_unsupported`
- 本地 CLI usage 通常不可用,系统不会写入 fake 0 token、fake cost 或 fake cache telemetry。
- 本地 CLI 执行上限有硬边界:`GENERATION_BACKEND_TIMEOUT_SECONDS` 最大 `3600``GENERATION_BACKEND_MAX_OUTPUT_BYTES` 最大 `33554432``GENERATION_BACKEND_MAX_CONCURRENCY` 最大 `16``LOCAL_CLI_BACKEND_MAX_CONCURRENCY` 最大 `4`。诊断 stdout/stderr 与最终响应合计超过输出上限时会返回结构化 `output_too_large`;对 `--output-last-message` presetstdout 中重复打印的最终响应不会重复计入,也不会作为 `stdout_preview` 暴露。
- 本地 CLI 的 `stdout_preview` / `stderr_preview` 在写入结构化 diagnostics 前会脱敏短凭证赋值,不依赖值长度:大写环境变量赋值沿用 child-env 的 fail-closed 敏感名称判定JSON 与 YAML / 普通日志中的标量赋值使用更窄的凭证字段 allowlistURL 继续使用独立的 userinfo / webhook / 敏感参数规则。`token_budget``session_id``sort_key` 等普通排障字段不会仅因包含 `token``session``key` 子串而被删除;但未加引号的 YAML 敏感标量没有可靠的同行边界,因此从该值起到行尾会 fail-closed 脱敏,同行后续字段也不会保留。
- 本地 CLI 默认并发为 1有效并发为 `min(LOCAL_CLI_BACKEND_MAX_CONCURRENCY, GENERATION_BACKEND_MAX_CONCURRENCY)`,不继承 `MAX_WORKERS`
- `AGENT_GENERATION_BACKEND=auto` 不会继承 `GENERATION_BACKEND` 的 local CLI 值Agent 工具调用继续使用 LiteLLM。Web 设置页仅暴露 `auto|litellm`;手写 `AGENT_GENERATION_BACKEND=codex_cli|claude_code_cli|opencode_cli` 不实现 text-only Agent mode会返回明确 unsupported tool-calling 诊断。
- Phase 6a 的 DSA Tool Surface 仍是唯一工具 schema、权限元数据、scope guard、结构化错误和审计/脱敏边界Phase 6 的 Codex AgentBackend 只能通过该 Tool Surface 执行工具。`codex_cli` / `claude_code_cli` / `opencode_cli` 仍是 generation-only不能作为 Agent tool fallback。

View File

@@ -45,6 +45,7 @@ AGENT_GENERATION_BACKEND=auto
- Local CLI backends do not support streaming. Stream requests degrade to non-stream and do not return `capability_unsupported`.
- Local CLI usage is normally unavailable. DSA does not persist fake 0-token, fake cost, or fake cache telemetry.
- Local CLI execution has hard caps: `GENERATION_BACKEND_TIMEOUT_SECONDS` max `3600`, `GENERATION_BACKEND_MAX_OUTPUT_BYTES` max `33554432`, `GENERATION_BACKEND_MAX_CONCURRENCY` max `16`, and `LOCAL_CLI_BACKEND_MAX_CONCURRENCY` max `4`. Diagnostic stdout/stderr plus the final response are counted together; for `--output-last-message` presets, the final response duplicated to stdout is not counted twice and is not exposed in `stdout_preview`.
- Local CLI `stdout_preview` / `stderr_preview` redact short credential assignments before diagnostics are emitted, regardless of value length. Uppercase environment assignments reuse the child-env fail-closed sensitive-name contract; scalar assignments in JSON, YAML, and ordinary logs use a narrower credential-field allowlist; URLs keep separate userinfo, webhook, and sensitive-parameter rules. Ordinary troubleshooting fields such as `token_budget`, `session_id`, and `sort_key` are not removed merely because their names contain `token`, `session`, or `key`; however, an unquoted sensitive YAML scalar has no reliable same-line boundary, so redaction fails closed from that value through the end of the line and does not preserve later fields on that line.
- Local CLI default concurrency is 1. Effective local CLI concurrency is `min(LOCAL_CLI_BACKEND_MAX_CONCURRENCY, GENERATION_BACKEND_MAX_CONCURRENCY)` and does not inherit `MAX_WORKERS`.
- `AGENT_GENERATION_BACKEND=auto` does not inherit local CLI values from `GENERATION_BACKEND`; Agent tool calling remains on LiteLLM. The Web settings page only exposes `auto|litellm`; a hand-written `AGENT_GENERATION_BACKEND=codex_cli|claude_code_cli|opencode_cli` does not enable Agent text-only mode and returns an explicit unsupported tool-calling diagnostic.
- The Phase 6a DSA Tool Surface remains the only tool-schema, permission, scope-guard, structured-error, audit, and redaction boundary. The Phase 6 Codex AgentBackend can execute tools only through this surface. `codex_cli` / `claude_code_cli` / `opencode_cli` remain generation-only and are never Agent tool fallbacks.

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff