Commit Graph
2 Commits
Author SHA1 Message Date
inliver233 dd8057cfcb Make the command gate machine-independent, and calibrate the budget fallback
CI went red on `1dc1d9b` in both jobs while the identical command passed locally, and the reason is
worth keeping: the checks depended on state that only exists on a developer's machine.

check_commands.py -- the gate's verdict depended on a gitignored directory
  The previous commit's fix recognised a workbench script by matching its name against a walk of
  `tools/`. That tree is gitignored, so a clean checkout has none of it: `dcc.py` matched locally and
  skipped, and on CI nothing matched, it fell through to `missing-script`, and the gate reported
  drift for three quoted lines. A check whose verdict changes with a directory CI does not have is
  not a check. Two changes remove the dependency:

    * `qualified_basenames()` collects every `.py` the *document itself* writes with a path, so a
      bare name the document has already located elsewhere is a skip, computed from the text.
    * `./dcc.py` is a declared upstream entry point, checked before the workbench-name test.
    * a bare name after a `cd` that resolves nowhere is `after-cd-unresolved` -- unverifiable, which
      is not the same claim as broken, and is now reported as such instead of as drift.

  Verified by running the whole CI sequence against a tree with `tools/` hidden: five gates exit 0,
  the suite passes. That experiment is the one that should have been run before the first push.

check_budget.py -- the measurement was installation-dependent
  The gate measures the always-loaded body in tokens, preferring `tiktoken` and falling back to
  `characters / N`. `N` had been guessed at 3.6 rather than calibrated, so a machine with tiktoken
  saw ~11.1k tokens and passed while a machine without it saw ~14.2k and warned -- and CI's 3.11 job
  is the one without it. `N` is now the measured 4.285 characters per token (47,629 chars -> 11,114
  tokens on SKILL.md), both paths agree within one token, and `tests/unit/test_budget_measurement.py`
  forces both paths and asserts they do not straddle the budget.

  CI now installs `tiktoken` so the comparison exercises the real tokenizer instead of skipping.

Also: two `.py` files are formatted for consistency, and the reconstruction record documents the
CI-only failure, since "it passed on my machine" is the lesson rather than an aside.

Gates: check_repo 0 problem, check_refs 0 dangling / 0 warning, check_routing coherent,
check_budget 0 failure / 0 warning (both measurement paths), check_commands 154 checked / 0 drift,
pytest 617 passed on 3.14 and 609 on 3.11.
2026-09-22 17:09:22 +08:00
inliver233 1dc1d9b707 Fix the command gate's blind spot, and make --json actually be JSON
A reviewer reported seven drifting commands in the evidence record. Reproduced, the report split
three ways, and the middle category was the useful one: the gate had been scoring them clean.

Not reproducible, with the measurement:

  tests/run_tests.py -q   runs. The wrapper forwards its arguments to pytest
                          (`args = list(argv[1:]) or ["-q"]`), and documents that in its own usage.
  dcc.py                  already written as `cd tools/_work/bench/repos/dcc` followed by
                          `python dcc.py ...` -- an external checkout, not a shipped script.

Real, and fixed here: seven commands in two evidence records named a **workbench** script with no
path at all (`analyze_pair.py`, `make_skeleton.py`, `ttd_attach.py`, `b2_measure.py`). The files
exist under `tools/`, which .gitignore excludes, so a reader copying one from the repository root
gets "No such file" -- and two of those documents wrote the correct path elsewhere, so they
disagreed with themselves.

**The actual defect was in `check_commands.py`, not in the seven lines.** `resolve_script()` tried
the document's own directory first, so a bare `analyze_pair.py` inside `docs/tool-verification/`
resolved to `tools/_work/bench/unpack/b3/analyze_pair.py` -- a real file -- and passed. Resolution
now follows the order a reader would try (repository root, then the skill's scripts, then the
document's own directory, which is what `../x.py` in a nested README legitimately means); a bare
name that only resolves under `tools/` is `unqualified-workbench`; and that class counts as **drift,
not as a skip**, because it is a one-token fix that a reader cannot run. The first attempt classified
it as a low-confidence warning, which would have left all seven in place.

One exception is honoured and pinned: a code block opening with `cd tools/_work/bench/repos/dcc` is
telling the reader where to stand, so a bare `python dcc.py` after it is correct.
`documented_commands()` now reports whether a `cd` preceded the command in the same block, and that
case is skipped as `workbench-after-cd`.

Two defects of this pass's own, found while fixing it:

  --json emitted the RESULT= token after the JSON document, so `--json | jq` failed with "Extra
  data". The token is gone from that path; the status is a field in the document, and the exit code
  carries the same meaning.
  The fix's own substitution left four literal backslashes before `$` in the two files it edited --
  a character a reader would copy by mistake. Removed, and verified absent.

`tests/integration/test_check_commands_gate.py` pins the resolution order, the drift classification,
the `cd` exception, and a clean pass over the committed tree, so this cannot come back silently.

Gates: check_commands 153 checked / 0 drift, check_repo 0 problem, check_refs 0 dangling / 0 warning,
check_routing coherent, check_budget 0 failure / 0 warning, pytest 613 passed / 158 skipped / 25
xfailed.
2026-09-22 02:18:40 +08:00