mirror of
https://hubproxy.babadafafafafa.cn/https://github.com/telegramdesktop/tdesktop
synced 2026-09-20 08:03:45 +08:00
Make test recovery more direct
Escalate repeated setup failures through direct production seams and preserve disposable submodule probes.
This commit is contained in:
@@ -33,8 +33,9 @@ adapter point; every other rule here still applies.
|
||||
screenshots / graphic resources). Images are optional evidence: read them when present, but their
|
||||
absence is never by itself a planning, implementation, or test blocker. The spec and its cited
|
||||
repository/baseline sources are one side of test design; the implementation diff is the other.
|
||||
- Config: `BUILD` (build command), `EXE` (built binary path), `MAX_ATTEMPTS` (default 4). The test
|
||||
account lives in `out/Debug/` as the portable-data folders described under "Test account" below;
|
||||
- Config: `BUILD` (build command), `EXE` (built binary path), `MAX_ATTEMPTS` (default 4),
|
||||
`MAX_TEST_RUNS` (default 12). The test account lives in `out/Debug/` as the portable-data folders
|
||||
described under "Test account" below;
|
||||
the wrapper has already confirmed the golden one exists (launch gate). All paths are relative to
|
||||
the current checkout — no worktrees are created; the run happens in whatever repository slot it
|
||||
was launched from.
|
||||
@@ -62,10 +63,42 @@ On every TERMINAL exit (APPROVED / BLOCKED / UNRECOVERABLE / cap) "delete the te
|
||||
step in "Leave no test binary behind" below.
|
||||
```
|
||||
|
||||
Early-escalation rule: if two consecutive ASSESS rounds produce the **same failure signature**
|
||||
(same step fails the same way after a fix), stop and return BLOCKED — do not burn the rest of
|
||||
the attempt budget chasing it. Before applying this rule to the macOS cached-language startup
|
||||
signature, perform the one-time clean-rebuild recovery under "Crashes & assertions".
|
||||
Repeated-failure rule: a repeated **failure signature is a demand for a more direct test**, not a
|
||||
terminal result. Never return `BLOCKED` merely because two runs failed at the same setup step.
|
||||
A `TEST_FLAW` rerun must stop repairing the same fixture technique and reduce the distance between
|
||||
the test and the production code this task changed.
|
||||
|
||||
Before authoring each recovery run, append a short `Recovery plan` to `test.md` that states:
|
||||
|
||||
- what the preceding run positively proved;
|
||||
- the exact setup assumption that failed;
|
||||
- the previous technique that is now forbidden;
|
||||
- the next unused directness strategy and why it can reach the changed code even if the failed
|
||||
setup never works.
|
||||
|
||||
Choose the next applicable strategy from this ladder. The order is by task fit, not ceremony, and
|
||||
one recovery may advance several levels:
|
||||
|
||||
1. Add diagnostics that identify the exact production object, key, row, request, or callback and
|
||||
replace guessed predicates with literal state assertions.
|
||||
2. Replace synthetic UI/model setup with an established production data-layer insertion API or a
|
||||
real disposable `live-mutate` fixture in the prepared test account.
|
||||
3. Bypass setup behavior outside this task's diff through a narrow inventoried `_DEBUG` in-situ
|
||||
seam immediately before the changed production function; construct the object by hand or call
|
||||
the real production collector/handler directly, while keeping an independent oracle.
|
||||
4. For network and retry behavior, inject or mock the exact request result / server error at the
|
||||
narrowest transport or callback seam that still executes the changed retry code. Do not wait on
|
||||
a live server when the response is not itself the subject.
|
||||
5. When physical interaction is the subject, drive the exact visible target with real Qt events or
|
||||
the safe hybrid driver. On locked macOS, make this direct interaction in-binary; the lock screen
|
||||
never prevents a more manual overlay.
|
||||
|
||||
After the same signature repeats, use a fresh test-recovery leaf and explicitly forbid the failed
|
||||
approach in its prompt. Early `BLOCKED(test)` is allowed only when a fresh recovery assessment
|
||||
records why every applicable unused strategy above is unsafe, unavailable, or would bypass the
|
||||
changed code, and the performer confirms that record. Otherwise continue until approval,
|
||||
implementation diagnosis, or `MAX_TEST_RUNS`. The macOS cached-language startup signature still
|
||||
gets the one-time clean-rebuild recovery under "Crashes & assertions" before entering this ladder.
|
||||
|
||||
UNRECOVERABLE conditions: the app reaches a login screen / `AUTH_KEY_DUPLICATED` and re-copying the
|
||||
test account does not recover it, or a crash has no usable diagnostic after one retry and the
|
||||
@@ -365,14 +398,11 @@ void SetupScenario(not_null<Runner*> runner) {
|
||||
}
|
||||
```
|
||||
|
||||
**The overlay = replacing `test_scenario.cpp` with the task's scenario.** Author the scenario
|
||||
fresh against the CURRENT implementation from the task's check design; never re-implement
|
||||
**The overlay starts by replacing `test_scenario.cpp` with the task's scenario.** Author the
|
||||
scenario fresh against the CURRENT implementation from the task's check design; never re-implement
|
||||
logging, finding, capturing, watchdogs, or quit handling that the harness already provides —
|
||||
re-derived scaffolding is where capture flaws come from. Only two kinds of edits may touch
|
||||
other files, and both stay in the inventory: a one-line `Test::Fire("task_waitpoint")` in code
|
||||
this task already owns, and a true in-situ injection at the highest level that still exercises
|
||||
the change (often a direct data-layer call like `item->applyEdition(...)` rather than a faked
|
||||
MTP response). The scenario is the complete runtime driver of first resort: prefer
|
||||
re-derived scaffolding is where capture flaws come from. The scenario is the complete runtime
|
||||
driver of first resort: prefer
|
||||
programmatically triggering every required action and judging the saved logs and captures
|
||||
afterwards over any external desktop driver, whether or not one is available. Drive the whole
|
||||
task-specific flow inside the Debug binary on the event loop, waiting for observable state,
|
||||
@@ -381,6 +411,13 @@ session does not reduce required coverage and is never a testing blocker. The sc
|
||||
only when `-testagent` was passed AND the live portable folder carries the `testing` marker,
|
||||
so it can never run against real account data. The overlay must:
|
||||
|
||||
- Prefer keeping the first run centralized in `test_scenario.cpp`, but never treat that module as
|
||||
a sandbox boundary. After a setup or reachability failure, inject probes, fixtures, callbacks,
|
||||
waitpoints, or direct test entry points at any relevant tracked production location, including
|
||||
initialized submodules. Choose the location closest to the changed code that preserves a real
|
||||
execution of that code. Scattered test injections are acceptable when they remove fixture
|
||||
assumptions; all must remain disposable, inventoried, and excluded from implementation commits.
|
||||
|
||||
- Keep any code added outside `test/` inside `#ifdef _DEBUG` blocks only when it would
|
||||
change release behavior; harness calls like `Test::Fire` are runtime no-ops and need no
|
||||
guard.
|
||||
@@ -442,13 +479,14 @@ bypass it with hand-built relative paths.
|
||||
|
||||
### Git mechanics for the overlay (no stash)
|
||||
|
||||
- The inventory in `<WORK_DIR>/test-overlay.paths` is normally exactly
|
||||
`Telegram/SourceFiles/test/test_scenario.cpp`, plus any in-situ injection or `Test::Fire`
|
||||
paths; no unrelated or untracked source path may be used. After building, save the overlay
|
||||
with the workspace helper's `overlay-save` command: it verifies every dirty path against the
|
||||
inventory, writes a nonempty verified `<WORK_DIR>/test-overlay.patch`, and restores only the
|
||||
inventoried overlay paths to the wrapper's restore ref; never hard-reset the repository. The
|
||||
overlay never enters an impl commit.
|
||||
- The inventory in `<WORK_DIR>/test-overlay.paths` normally starts with
|
||||
`Telegram/SourceFiles/test/test_scenario.cpp` and then lists every in-situ injection or
|
||||
`Test::Fire` path. It may name any tracked file in the source checkout or an initialized
|
||||
submodule; no unrelated or untracked path may be used. After building, save the overlay with the
|
||||
workspace helper's `overlay-save` command: it verifies every dirty path against the inventory,
|
||||
writes the top-level patch plus a per-submodule patch bundle when needed, and restores only the
|
||||
inventoried overlay paths to their repository baselines; never hard-reset the repository. The
|
||||
overlay never enters an impl or submodule commit.
|
||||
- Next round, re-apply on top of the new implementation with `overlay-apply` (a `--3way`
|
||||
application that reports conflicted paths). This succeeds ~90% of the time when the tail change
|
||||
was small.
|
||||
@@ -543,7 +581,8 @@ implementation or overlay verdict:
|
||||
`Local::readLangPack()`;
|
||||
- the same signature occurs on two launches.
|
||||
|
||||
Before early escalation, preserve the current overlay and account, stop only the exact-path app,
|
||||
Before changing recovery strategies, preserve the current overlay and account,
|
||||
stop only the exact-path app,
|
||||
follow the portable-folder safety-copy procedure in `AGENTS.md`, then run one full Xcode Debug
|
||||
clean followed by `BUILD` (the configured-tree clean is normally
|
||||
`cmake --build out --config Debug --target clean`). Restore only portable folders missing after
|
||||
@@ -551,7 +590,7 @@ the clean, never overwrite survivors, and retain the external backup through one
|
||||
post-build launch. Rerun the same scenario once. Record the preceding runs as `TEST_FLAW` caused
|
||||
by stale generated-language objects; do not spend an implementation attempt or re-author the
|
||||
overlay. If the identical signature remains after the single clean rebuild, resume normal crash
|
||||
classification and early escalation. Never loop clean rebuilds.
|
||||
classification and the directness ladder. Never loop clean rebuilds.
|
||||
|
||||
### Hangs & freezes (two layers, because they have two causes)
|
||||
|
||||
@@ -574,8 +613,8 @@ A run that never reaches `TEST_COMPLETE` and never dies is a hang. Two independe
|
||||
Classify by which guard tripped: a DeadlockDetector crash with a real main-thread stack in app code
|
||||
is an **IMPL_BUG**; the external cap firing is almost always a **TEST_FLAW** (the overlay didn't
|
||||
drive to `TEST_COMPLETE`/quit) — re-author the overlay — unless the captured stack/log shows the
|
||||
implementation itself wedged, in which case it is an IMPL_BUG. Two external-cap kills in a row with
|
||||
the same signature → BLOCKED (early-escalation rule).
|
||||
implementation itself wedged, in which case it is an IMPL_BUG. Repeated external-cap kills enter
|
||||
the directness ladder above; the same timeout signature alone never blocks the task.
|
||||
|
||||
### Leave no test binary behind
|
||||
|
||||
@@ -644,7 +683,9 @@ starts the next Attempt. Never overwrite history.
|
||||
#### Verdict reasoning
|
||||
<1-3 lines tying the checks to the verdict>
|
||||
#### Root cause / Fix hint (only if IMPL_BUG — the impl-fix agent reads this)
|
||||
#### Failure signature (one line, for early-escalation comparison)
|
||||
#### Failure signature (one line, for recovery comparison)
|
||||
#### Recovery plan (TEST_FLAW reruns only: prior proof, failed assumption,
|
||||
forbidden technique, next directness strategy)
|
||||
```
|
||||
|
||||
## Compact summary the task-runner returns up
|
||||
|
||||
@@ -111,6 +111,12 @@ canonical `Block <full-task-id>` commit. Agent interruption, tool loss, and
|
||||
global environment stops leave the task `in-progress` with its task-scoped
|
||||
local state intact for the next invocation.
|
||||
|
||||
A repeated test setup failure is not exhausted verification by itself. Follow
|
||||
the shared directness ladder: forbid the failed fixture technique and make the
|
||||
next run more manual and closer to the changed production seam. The configured
|
||||
test-run cap is the safety boundary; the former two-identical-signature shortcut
|
||||
must not be used.
|
||||
|
||||
A locked macOS session is not an environment stop or verification blocker.
|
||||
Skip interactive Computer Use and complete the same coverage through the
|
||||
in-binary overlay: drive the flow, log/assert, capture widgets or windows,
|
||||
|
||||
@@ -10,6 +10,7 @@
|
||||
- [Assessment](#phase-3-plan-assessment)
|
||||
- [Implementation and build](#phase-4-implementation)
|
||||
- [Review](#phase-6-code-review-loop)
|
||||
- [Test-flaw recovery](#test-flaw-recovery-and-directness)
|
||||
- [Windows normalization](#phase-7-native-windows-text-normalization)
|
||||
- [Prompt delivery](#prompt-delivery-and-logs)
|
||||
|
||||
@@ -858,6 +859,9 @@ Write <WORK_DIR>/test-design.md:
|
||||
observed
|
||||
- a run plan compressed to the fewest possible runs — normally exactly one —
|
||||
splitting only for checks that cannot share one process lifetime
|
||||
- a fixture fallback plan naming at least two progressively more direct ways
|
||||
to reach the changed production seam if the preferred fixture does not
|
||||
materialize; do not make unrelated UI setup a single point of failure
|
||||
- a `## Reconcile` line reminding the test author to re-verify every check
|
||||
against the final retained diff after the review loop
|
||||
|
||||
@@ -865,6 +869,61 @@ Every check needs an oracle that can come out FAIL. "The screen opened" is
|
||||
not a check. Do not reuse a generic navigate-and-screenshot scenario.
|
||||
```
|
||||
|
||||
## Test-Flaw Recovery And Directness
|
||||
|
||||
Use this prompt for every `TEST_FLAW` recovery. On a repeated signature, use a
|
||||
fresh leaf and include every prior recovery plan. Do not ask it to "try again"
|
||||
or merely repair the same fixture.
|
||||
|
||||
```text
|
||||
You are a test-recovery agent for one Telegram Desktop task. The product
|
||||
implementation is retained; repair only the disposable test overlay and its
|
||||
test report. You are a leaf and must not delegate.
|
||||
|
||||
Read:
|
||||
- the task spec and final retained task diff
|
||||
- <WORK_DIR>/test-design.md
|
||||
- <WORK_DIR>/test.md, including every prior Run and Recovery plan
|
||||
- <WORK_DIR>/test-overlay.paths and the current saved overlay
|
||||
- .agents/shared/test-loop.md, especially the repeated-failure directness ladder
|
||||
|
||||
The latest failure signature is:
|
||||
<FAILURE_SIGNATURE>
|
||||
|
||||
The previous fixture/recovery techniques are forbidden:
|
||||
<FORBIDDEN_TECHNIQUES>
|
||||
|
||||
Before editing, append a Recovery plan to the pending Run in test.md:
|
||||
- Prior proof: the positive facts already established by earlier runs
|
||||
- Failed assumption: the exact setup predicate or fixture behavior that failed
|
||||
- Forbidden technique: the approach you will not repeat
|
||||
- New directness strategy: the next unused ladder strategy
|
||||
- Reachability: why this strategy reaches the task's changed production code
|
||||
even if the failed setup never succeeds
|
||||
|
||||
Then implement the recovery. Prefer fewer assumptions and more manual control:
|
||||
- insert the production object by hand through an established data API;
|
||||
- add a narrow inventoried _DEBUG seam and call the real changed
|
||||
collector/handler directly when unrelated UI setup is flaky;
|
||||
- place that seam in any relevant tracked production file or initialized
|
||||
submodule when centralizing it in test_scenario.cpp adds indirection;
|
||||
- inject the exact server response/error at the callback or transport seam for
|
||||
retry behavior;
|
||||
- use a real disposable account fixture or exact Qt input only when those are
|
||||
part of the behavior under test.
|
||||
|
||||
Keep an independent falsifiable oracle. A direct seam may bypass setup outside
|
||||
the task diff, but it must not reimplement or bypass the changed code itself.
|
||||
Fix every test flaw visible in the latest run together, build Debug, and return
|
||||
the compact phase block.
|
||||
|
||||
If no unused strategy can safely reach the changed code, do not cite the
|
||||
repeated signature as the reason. Add `## Recovery exhaustion` to test.md with
|
||||
one row per ladder strategy: attempted evidence, or the concrete reason it is
|
||||
unsafe, unavailable, or would bypass the task diff. Return `BLOCKED` for the
|
||||
performer to confirm independently.
|
||||
```
|
||||
|
||||
### Step 6s: Review synthesis
|
||||
|
||||
```text
|
||||
|
||||
@@ -416,13 +416,14 @@ rules, with these external-task safety adaptations:
|
||||
`source-commit`).
|
||||
- The overlay is authored against the permanent harness in
|
||||
`Telegram/SourceFiles/test/` and normally consists of replacing
|
||||
`Telegram/SourceFiles/test/test_scenario.cpp` alone — that slot file is
|
||||
always a permitted overlay path. Beyond it, overlay code may modify only
|
||||
tracked task-owned source paths (one-line `Test::Fire` waitpoints or true
|
||||
in-situ injections). Inventory every overlay path in
|
||||
`work/test-overlay.paths`; never introduce an untracked source file, and
|
||||
never re-implement logging, widget-finding, capture, watchdog, or quit
|
||||
mechanics the harness already provides.
|
||||
`Telegram/SourceFiles/test/test_scenario.cpp` alone on the first run. That is
|
||||
a preference, not a boundary: after a setup or reachability failure, overlay
|
||||
code may modify any relevant tracked source path, including initialized
|
||||
submodules, to place a disposable probe or direct entry point beside the
|
||||
changed production code. Inventory every overlay path in
|
||||
`work/test-overlay.paths`; never introduce an untracked source file, commit
|
||||
an overlay or submodule injection, or re-implement logging, widget-finding,
|
||||
capture, watchdog, or quit mechanics the harness already provides.
|
||||
- Save and restore the overlay with the scripted helper instead of manual git
|
||||
mechanics:
|
||||
|
||||
@@ -432,8 +433,9 @@ rules, with these external-task safety adaptations:
|
||||
```
|
||||
|
||||
It verifies every dirty path against the inventory, refuses untracked
|
||||
files, writes a nonempty verified `work/test-overlay.patch`, and restores
|
||||
only inventoried paths to `RUN_REF` — never a repository-wide hard reset.
|
||||
files, writes a verified top-level patch plus per-submodule patch bundle
|
||||
when needed, and restores only inventoried paths to `RUN_REF` or the
|
||||
submodule's current baseline — never a repository-wide hard reset.
|
||||
After an implementation-fix commit (`source-commit --mark-green` moves both
|
||||
`GREEN_REF` and `RUN_REF`), reapply with:
|
||||
|
||||
@@ -505,6 +507,11 @@ rules, with these external-task safety adaptations:
|
||||
re-copies golden — then retry once.
|
||||
- Enforce the in-app watchdog too. Count test runs independently from
|
||||
implementation attempts and stop at `MAX_TEST_RUNS`.
|
||||
- A repeated setup failure is not a reason to stop below that cap. Apply the
|
||||
shared test loop's directness ladder: preserve what the run proved, forbid
|
||||
the failed fixture technique, and make the next overlay more manual and
|
||||
closer to the changed production seam. Once a setup outside the task's diff
|
||||
fails repeatedly, bypass that setup rather than continuing to test it.
|
||||
- Plan the fewest possible runs: one complete programmed scenario per attempt
|
||||
that proves every check in a single execution, splitting only for checks
|
||||
that cannot share one process lifetime. `MAX_TEST_RUNS` is a safety cap,
|
||||
@@ -559,12 +566,14 @@ settled, never the reason to leave a reachable surface unmeasured. Where an
|
||||
acceptance criterion ranges over a parameter — every value of an enum, both
|
||||
halves of a branch, more than one interface scale — the author iterates the
|
||||
range rather than sampling it, because a hand-picked subset is exactly the shape
|
||||
of gap that comes back later as its own task. Missing
|
||||
or ambiguous evidence is `TEST_FLAW`; no expected task delta is `IMPL_BUG`. Two
|
||||
identical consecutive failure signatures block early, except that the macOS
|
||||
cached-language signature first gets the shared test loop's one-time Xcode
|
||||
clean-rebuild recovery. A known implementation bug at the attempt cap is
|
||||
implementation-blocked, not a successful retained commit.
|
||||
of gap that comes back later as its own task. Missing or ambiguous evidence is
|
||||
`TEST_FLAW`; no expected task delta is `IMPL_BUG`. Repeated failure signatures
|
||||
trigger the shared directness ladder, not an automatic block. Block before
|
||||
`MAX_TEST_RUNS` only after a fresh recovery assessment proves that every
|
||||
applicable more-direct strategy is unsafe, unavailable, or would bypass the
|
||||
changed code. The macOS cached-language signature first gets the shared test
|
||||
loop's one-time Xcode clean-rebuild recovery. A known implementation bug at the
|
||||
attempt cap is implementation-blocked, not a successful retained commit.
|
||||
|
||||
Skip runtime testing only for a task with no runnable behavior. Record
|
||||
`NOT_APPLICABLE` and exact file-level validation. Configuration alone is not a
|
||||
|
||||
@@ -48,6 +48,8 @@ PORTABLE_REAL = "real_TelegramForcePortable"
|
||||
PORTABLE_MARKER = "testing"
|
||||
OVERLAY_PATHS_FILE = "test-overlay.paths"
|
||||
OVERLAY_PATCH_FILE = "test-overlay.patch"
|
||||
OVERLAY_SUBMODULES_FILE = "test-overlay-submodules.json"
|
||||
OVERLAY_SUBMODULES_DIR = "test-overlay-submodules"
|
||||
TEST_LOG_FILE = "test_log.txt"
|
||||
TEST_COMPLETE_MARKER = "TEST_COMPLETE"
|
||||
STALE_CRASH_DIR = "stale-crash"
|
||||
@@ -1819,65 +1821,250 @@ def read_overlay_paths(work):
|
||||
return paths
|
||||
|
||||
|
||||
def initialized_submodule_paths(source):
|
||||
lines = run_git(
|
||||
source, "submodule", "status", "--recursive"
|
||||
).stdout.splitlines()
|
||||
result = []
|
||||
for line in lines:
|
||||
if not line or line[0] == "-":
|
||||
continue
|
||||
parts = line[1:].split()
|
||||
if len(parts) >= 2:
|
||||
result.append(parts[1])
|
||||
return sorted(result, key=lambda path: (-path.count("/"), path))
|
||||
|
||||
|
||||
def overlay_inventory_groups(source, inventory):
|
||||
submodules = initialized_submodule_paths(source)
|
||||
groups = {"": []}
|
||||
for value in inventory:
|
||||
path = PurePosixPath(value)
|
||||
if path.is_absolute() or not path.parts or ".." in path.parts:
|
||||
raise WorkspaceError(f"Invalid overlay inventory path: {value!r}")
|
||||
value = path.as_posix()
|
||||
if value in submodules:
|
||||
raise WorkspaceError(
|
||||
"Overlay inventory must name a tracked file inside the "
|
||||
"submodule, not its gitlink: " + value
|
||||
)
|
||||
owner = next(
|
||||
(
|
||||
submodule for submodule in submodules
|
||||
if value.startswith(submodule + "/")
|
||||
),
|
||||
"",
|
||||
)
|
||||
local = value[len(owner) + 1:] if owner else value
|
||||
repository = source / owner if owner else source
|
||||
tracked = run_git(
|
||||
repository,
|
||||
"ls-files",
|
||||
"--error-unmatch",
|
||||
"--",
|
||||
local,
|
||||
check=False,
|
||||
)
|
||||
if tracked.returncode:
|
||||
raise WorkspaceError(
|
||||
"Overlay inventory paths must be tracked files: " + value
|
||||
)
|
||||
groups.setdefault(owner, []).append(local)
|
||||
return groups, submodules
|
||||
|
||||
|
||||
def overlay_coverage(inventory, repository_path):
|
||||
if not repository_path:
|
||||
return inventory
|
||||
prefix = repository_path + "/"
|
||||
return [
|
||||
path[len(prefix):]
|
||||
for path in inventory
|
||||
if path.startswith(prefix)
|
||||
]
|
||||
|
||||
|
||||
def overlay_outside_inventory(source, inventory, submodules):
|
||||
outside = []
|
||||
for repository_path in [""] + submodules:
|
||||
repository = source / repository_path if repository_path else source
|
||||
coverage = overlay_coverage(inventory, repository_path)
|
||||
dirty = changed_paths(repository)
|
||||
gitlinks = set(gitlink_paths(repository, dirty))
|
||||
for path in dirty:
|
||||
covered = path_is_covered(path, coverage)
|
||||
covered_gitlink = (
|
||||
path in gitlinks
|
||||
and any(value.startswith(path + "/") for value in coverage)
|
||||
)
|
||||
if covered or covered_gitlink:
|
||||
continue
|
||||
outside.append(
|
||||
f"{repository_path}/{path}" if repository_path else path
|
||||
)
|
||||
return outside
|
||||
|
||||
|
||||
def clear_overlay_submodule_bundle(work):
|
||||
manifest = work / OVERLAY_SUBMODULES_FILE
|
||||
patches = work / OVERLAY_SUBMODULES_DIR
|
||||
if manifest.is_file():
|
||||
manifest.unlink()
|
||||
if patches.is_dir():
|
||||
shutil.rmtree(patches)
|
||||
|
||||
|
||||
def read_overlay_submodule_bundle(work):
|
||||
manifest = work / OVERLAY_SUBMODULES_FILE
|
||||
if not manifest.is_file():
|
||||
return []
|
||||
data = json.loads(manifest.read_text(encoding="utf-8"))
|
||||
if data.get("version") != 1 or not isinstance(data.get("submodules"), list):
|
||||
raise WorkspaceError(f"Invalid overlay submodule manifest: {manifest}")
|
||||
result = []
|
||||
seen = set()
|
||||
for entry in data["submodules"]:
|
||||
if not isinstance(entry, dict):
|
||||
raise WorkspaceError(f"Invalid overlay submodule entry: {entry!r}")
|
||||
repository = PurePosixPath(str(entry.get("path", "")))
|
||||
patch = PurePosixPath(str(entry.get("patch", "")))
|
||||
if (
|
||||
repository.is_absolute()
|
||||
or not repository.parts
|
||||
or ".." in repository.parts
|
||||
or patch.is_absolute()
|
||||
or not patch.parts
|
||||
or ".." in patch.parts
|
||||
or patch.parts[0] != OVERLAY_SUBMODULES_DIR
|
||||
):
|
||||
raise WorkspaceError(f"Invalid overlay submodule entry: {entry!r}")
|
||||
repository_value = repository.as_posix()
|
||||
if repository_value in seen:
|
||||
raise WorkspaceError(
|
||||
"Duplicate overlay submodule entry: " + repository_value
|
||||
)
|
||||
seen.add(repository_value)
|
||||
result.append({
|
||||
"patch": patch.as_posix(),
|
||||
"path": repository_value,
|
||||
})
|
||||
return result
|
||||
|
||||
|
||||
def command_overlay_save(args):
|
||||
config, slot = task_action_config(args)
|
||||
source = Path(config["source_root"])
|
||||
work = overlay_work_dir(config, slot, args.task)
|
||||
inventory = read_overlay_paths(work)
|
||||
untracked = run_git(
|
||||
source, "ls-files", "--others", "--exclude-standard"
|
||||
).stdout.splitlines()
|
||||
if untracked:
|
||||
raise WorkspaceError(
|
||||
"The overlay may not use untracked source files: "
|
||||
+ ", ".join(untracked)
|
||||
)
|
||||
dirty = changed_paths(source)
|
||||
outside = [
|
||||
path for path in dirty
|
||||
if not path_is_covered(path, inventory)
|
||||
]
|
||||
groups, submodules = overlay_inventory_groups(source, inventory)
|
||||
outside = overlay_outside_inventory(source, inventory, submodules)
|
||||
if outside:
|
||||
raise WorkspaceError(
|
||||
"Dirty source paths are outside the overlay inventory: "
|
||||
+ ", ".join(outside)
|
||||
)
|
||||
patch = run_git_binary(source, "diff", "--binary", "HEAD")
|
||||
if not patch.strip():
|
||||
raise WorkspaceError("The overlay diff is empty; nothing to save")
|
||||
patch_path = work / OVERLAY_PATCH_FILE
|
||||
patch_path.write_bytes(patch)
|
||||
check = subprocess.run(
|
||||
["git", "-C", str(source), "apply", "--check", "--reverse", str(patch_path)],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
clear_overlay_submodule_bundle(work)
|
||||
root_paths = groups.get("", [])
|
||||
patch = (
|
||||
run_git_binary(source, "diff", "--binary", "HEAD", "--", *root_paths)
|
||||
if root_paths
|
||||
else b""
|
||||
)
|
||||
if check.returncode:
|
||||
raise WorkspaceError(
|
||||
"The saved overlay patch does not verify: "
|
||||
+ check.stderr.strip()
|
||||
patch_path = work / OVERLAY_PATCH_FILE
|
||||
if patch.strip():
|
||||
patch_path.write_bytes(patch)
|
||||
else:
|
||||
patch_path.unlink(missing_ok=True)
|
||||
submodule_entries = []
|
||||
patches_dir = work / OVERLAY_SUBMODULES_DIR
|
||||
for repository_path, paths in groups.items():
|
||||
if not repository_path:
|
||||
continue
|
||||
repository = source / repository_path
|
||||
repository_patch = run_git_binary(
|
||||
repository, "diff", "--binary", "HEAD", "--", *paths
|
||||
)
|
||||
if not repository_patch.strip():
|
||||
continue
|
||||
patches_dir.mkdir(parents=True, exist_ok=True)
|
||||
name = hashlib.sha256(repository_path.encode("utf-8")).hexdigest()[:16]
|
||||
relative_patch = f"{OVERLAY_SUBMODULES_DIR}/{name}.patch"
|
||||
submodule_patch_path = work / relative_patch
|
||||
submodule_patch_path.write_bytes(repository_patch)
|
||||
submodule_entries.append({
|
||||
"patch": relative_patch,
|
||||
"path": repository_path,
|
||||
})
|
||||
if submodule_entries:
|
||||
(work / OVERLAY_SUBMODULES_FILE).write_text(
|
||||
json.dumps({
|
||||
"submodules": submodule_entries,
|
||||
"version": 1,
|
||||
}, indent=2, sort_keys=True) + "\n",
|
||||
encoding="utf-8",
|
||||
)
|
||||
if not patch.strip() and not submodule_entries:
|
||||
raise WorkspaceError("The overlay diff is empty; nothing to save")
|
||||
checks = []
|
||||
if patch.strip():
|
||||
checks.append((source, patch_path))
|
||||
checks.extend(
|
||||
(source / entry["path"], work / entry["patch"])
|
||||
for entry in submodule_entries
|
||||
)
|
||||
for repository, saved_patch in checks:
|
||||
check = subprocess.run(
|
||||
[
|
||||
"git", "-C", str(repository), "apply", "--check",
|
||||
"--reverse", str(saved_patch),
|
||||
],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
if check.returncode:
|
||||
raise WorkspaceError(
|
||||
"The saved overlay patch does not verify: "
|
||||
+ check.stderr.strip()
|
||||
)
|
||||
restored = []
|
||||
if args.restore != "none":
|
||||
ref = source_task_ref(args.task, args.restore)
|
||||
if resolved_ref(source, ref) is None:
|
||||
raise WorkspaceError(f"Missing task ref for restore: {ref}")
|
||||
run_git(source, "checkout", ref, "--", *inventory)
|
||||
for repository_path, paths in sorted(
|
||||
groups.items(), key=lambda item: -item[0].count("/")
|
||||
):
|
||||
if not repository_path:
|
||||
continue
|
||||
run_git(source / repository_path, "checkout", "HEAD", "--", *paths)
|
||||
if root_paths:
|
||||
run_git(source, "checkout", ref, "--", *root_paths)
|
||||
restored = inventory
|
||||
remaining = [
|
||||
path for path in changed_paths(source)
|
||||
if path_is_covered(path, inventory)
|
||||
]
|
||||
remaining = []
|
||||
for repository_path, paths in groups.items():
|
||||
repository = source / repository_path if repository_path else source
|
||||
remaining.extend(
|
||||
(
|
||||
f"{repository_path}/{path}"
|
||||
if repository_path else path
|
||||
)
|
||||
for path in changed_paths(repository)
|
||||
if path_is_covered(path, paths)
|
||||
)
|
||||
if remaining:
|
||||
raise WorkspaceError(
|
||||
"Overlay paths remain dirty after restore: "
|
||||
+ ", ".join(remaining)
|
||||
)
|
||||
print(json.dumps({
|
||||
"patch": str(patch_path),
|
||||
"patch_bytes": len(patch),
|
||||
"patch": str(patch_path) if patch.strip() else None,
|
||||
"patch_bytes": len(patch) + sum(
|
||||
(work / entry["patch"]).stat().st_size
|
||||
for entry in submodule_entries
|
||||
),
|
||||
"restored": restored,
|
||||
"submodules": [entry["path"] for entry in submodule_entries],
|
||||
"task": args.task,
|
||||
}, indent=2, sort_keys=True))
|
||||
|
||||
@@ -1887,28 +2074,59 @@ def command_overlay_apply(args):
|
||||
source = Path(config["source_root"])
|
||||
work = overlay_work_dir(config, slot, args.task)
|
||||
patch_path = work / OVERLAY_PATCH_FILE
|
||||
if not patch_path.is_file() or not patch_path.stat().st_size:
|
||||
submodule_entries = read_overlay_submodule_bundle(work)
|
||||
root_patch = patch_path.is_file() and patch_path.stat().st_size
|
||||
if not root_patch and not submodule_entries:
|
||||
raise WorkspaceError(f"Missing overlay patch: {patch_path}")
|
||||
inventory = read_overlay_paths(work)
|
||||
result = subprocess.run(
|
||||
["git", "-C", str(source), "apply", "--3way", str(patch_path)],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
groups, submodules = overlay_inventory_groups(source, inventory)
|
||||
for entry in submodule_entries:
|
||||
if entry["path"] not in groups or entry["path"] not in submodules:
|
||||
raise WorkspaceError(
|
||||
"Overlay submodule manifest is outside the inventory: "
|
||||
+ entry["path"]
|
||||
)
|
||||
if not (work / entry["patch"]).is_file():
|
||||
raise WorkspaceError(
|
||||
"Missing overlay submodule patch: " + entry["patch"]
|
||||
)
|
||||
applications = []
|
||||
if root_patch:
|
||||
applications.append(("", source, patch_path))
|
||||
applications.extend(
|
||||
(entry["path"], source / entry["path"], work / entry["patch"])
|
||||
for entry in submodule_entries
|
||||
)
|
||||
conflicts = run_git(
|
||||
source, "diff", "--name-only", "--diff-filter=U"
|
||||
).stdout.splitlines()
|
||||
applied = not result.returncode and not conflicts
|
||||
outside = [
|
||||
path for path in changed_paths(source)
|
||||
if not path_is_covered(path, inventory)
|
||||
]
|
||||
conflicts = []
|
||||
errors = []
|
||||
for repository_path, repository, saved_patch in applications:
|
||||
result = subprocess.run(
|
||||
[
|
||||
"git", "-C", str(repository), "apply", "--3way",
|
||||
str(saved_patch),
|
||||
],
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
)
|
||||
if result.returncode:
|
||||
errors.append(
|
||||
f"{repository_path or '.'}: {result.stderr.strip()}"
|
||||
)
|
||||
for path in run_git(
|
||||
repository, "diff", "--name-only", "--diff-filter=U"
|
||||
).stdout.splitlines():
|
||||
conflicts.append(
|
||||
f"{repository_path}/{path}" if repository_path else path
|
||||
)
|
||||
outside = overlay_outside_inventory(source, inventory, submodules)
|
||||
applied = not errors and not conflicts and not outside
|
||||
print(json.dumps({
|
||||
"applied": applied,
|
||||
"conflicts": conflicts,
|
||||
"error": result.stderr.strip() if result.returncode else None,
|
||||
"error": "\n".join(errors) if errors else None,
|
||||
"outside_inventory": outside,
|
||||
"submodules": [entry["path"] for entry in submodule_entries],
|
||||
"task": args.task,
|
||||
}, indent=2, sort_keys=True))
|
||||
|
||||
|
||||
Reference in New Issue
Block a user