Reconstruction pass: make the gates tell the truth, make the package self-contained, keep the one-hop symptom index

An external audit of this repository landed on four structural findings, and this pass is the
response to them. Everything below was re-measured on this tree before being acted on; two of the
audit's claims needed a correction of my own, and both corrections are noted in place.

**The gates were optimistic, and one of them did not read content at all.**

  doctor.py    the capability table was hard-coded (`repack + sign = bool(java) and python`), so a
               machine with a JDK and no zipalign/apksigner was told it could re-sign; the script
               inventory was a hand-written list of 28 against 51 files on disk, omitting the nine
               most important scripts. It now scans the directory (`registered/checked = 52 / 52`,
               unjudgeable entries counted as `unknown`, never skipped) and resolves capability as a
               real requires-closure, reporting BLOCKED with the missing piece and an installable
               next step. Measured: the old doctor reports `[OK] repack + sign` on a JDK-only PATH
               where keytool, jarsigner, zipalign, apksigner and all eight smali jars are absent.
               The report is allowed to look bad; that is the point.
  check_commands.py (new)
               anchor checks prove a section exists, never that a command runs. This parses each
               script's argparse table with `ast` and checks every documented invocation against it.
               146 commands / 41 interfaces, 0 drift. It caught `sig_probe.py --live --pkg`, which
               does not exist -- the reference had been wrong since it was written.
  check_routing.py (new)
               the symptom index now has one home (`SKILL.md`, because symptom -> file must stay one
               hop) and the inventory has another (`references/routing.md`). This gate proves the
               mirror agrees, that every reference file is named, and that every script is named.

**so_constpatch.py was corrupting APKs it rebuilt.** The writer copied filename and timestamp onto a
fresh ZipInfo and recompressed the whole archive: 435-entry real package, `extra` lost on 205 entries,
`external_attr` rewritten on all 435, 132 untouched entries recompressed, and `resources.arsc` pushed
to an unaligned offset. Android's own `zipalign -c -v 4` fails the old output and passes the new one;
a 26-case fixture suite pins it, and `extractNativeLibs="false"` is now refused by default with a
pointer at `repack.py`. Fixing it exposed the same defect in `repack.py`, where `^lib/[^/]+\.so$`
cannot match `lib/arm64-v8a/*.so` -- so its alignment handling had never once fired on a real APK.

**The package was not self-contained.** 59 references pointed at the repository root, which `npx
skills add` does not install. They now converge on `references/evidence-summary.md` (capability,
conclusion, strength, evidence that ships) plus `evidence/` with the machine-readable companions: 8
references remain and each is explicitly labelled "repository root, not shipped". A drill in which
the new files were copied to an isolated directory and asked three questions answered all three.

**SKILL.md: 572 lines / 15,733 tokens -> 398 lines / ~10,900 tokens**, without losing the two things
that had to stay. The symptom index remains inline; the reference and script indexes moved to the
on-demand routing file. The freed budget went into making the rules *stronger*, not shorter: the
constraints section is now a table of thirteen non-negotiable constraints each paired with the check
that catches a violation, and the tooling obligations are instructions whose violation is a defect
(hedging language in SKILL.md: 1 line, down from a paragraph of hedged advice). Long references gained
a navigation head so a reader can decide from the top whether to load one.

**New: an executable test layer.** `tests/` (unit, CLI contract, no-device integration) plus
`pytest.ini`, `tests/run_tests.py`, a minimal CI job, and `evals/evals.json` written but never run --
stated as such in the file. 608 passed / 158 skipped / 25 xfailed. The 25 xfails are a work order, not
a pass: thirteen scripts report a missing input by exiting 1 instead of 2, seven let a traceback
escape, four treat `--help` as an error, and `preflight --json` lacks a `status` field. A regression
row for `dex_patch_bytes.py` reproduces a previous artifact byte-for-byte
(`67ae1e58e7a44777e5ac8f9f823ab21e…`) and additionally pins the four bytes that may move.

**Two real defects the new tests found in the tooling itself.** `check_repo.py` decoded subprocess
output with the locale encoding, so on a GBK console a script printing an em dash in its usage line
was reported as crashed; it now decodes UTF-8 explicitly. And a CLI contract test drove a *real
attached device* through `coldstart.py` -- logcat, screenshots, five orphan `adb logcat` processes --
which is fixed twice over: child processes run in a private temporary cwd, and the fifteen
device-bound scripts are never executed beyond `--help`/argument parsing.

**Quoting on the device side is now one implementation.** `devsh.su` escaped only double quotes, so
`;`, `$( )` and backticks in a caller-supplied value still reached the device shell while
`coldstart.Device.shell` quoted correctly. Both use `scripts/device_shell.py` now: POSIX single-quote
escaping, argv arrays by default, and identifier validation that refuses a malformed package, path or
component with exit 2 instead of a traceback. The integration test that used to pin the *defect* (it
asserted `su -c "…"` with `\"` escapes) now asserts the property, across five metacharacter payloads.

Gates: check_repo 0 problems (including the content-reading identity scan over git ls-files),
check_refs 0 dangling / 0 warning, check_routing coherent (53 symptom / 45 reference / 54 script rows),
check_budget 0 failure / 0 warning with the always-loaded body now measured in tokens rather than in
"narrative lines with the index excluded", check_commands 146 checked / 0 drift, pytest 608 passed.

Not done, and stated rather than implied: CI has never run on a runner; the evals have never been
executed; the 25 xfails are open; no device-side install/launch was performed in this pass.

tests/cli/contract.py

@@ -0,0 +1,181 @@
+"""Helpers for the CLI contract tests.
+
+Everything here is derived at run time from the scripts' own `--help` output
+rather than from a hard-coded list of argument names, so a new script is covered
+automatically and a renamed flag does not silently stop being tested.
+
+The contract under test is the repository-wide one recorded in
+tools/_work/REBUILD-BRIEF.md section 4:
+
+    0 = success
+    1 = the target result failed
+    2 = bad arguments or input
+    3 = the environment lacks a capability
+    4 = internal tool error
+
+A script that does not follow it yet is not a red test here: the test calls
+``pytest.xfail`` with the measured code in the reason, so the summary line stays
+green and the xfail list is exactly the work order for unifying exit codes.
+"""
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+# A path that does not exist anywhere. Used to drive the "bad input" scenario.
+NONEXISTENT = str(Path("__apkrev_missing__") / "does-not-exist.bin")
+
+# The four contract codes, plus None for "did not return".
+CONTRACT_EXIT_CODES = {0, 1, 2, 3, 4}
+
+
+def parse_leading_json(text):
+    """Parse the first JSON document in `text`, ignoring anything after it.
+
+    Several scripts print a `RESULT=<token>` line after their JSON document (the
+    repository's long-task discipline requires that token last), so a plain
+    json.loads on the whole stream fails. Only the first document is returned.
+    """
+    starts = [i for i in (text.find("{"), text.find("[")) if i >= 0]
+    if not starts:
+        return None
+    try:
+        doc, _end = json.JSONDecoder().raw_decode(text[min(starts):])
+    except ValueError:
+        return None
+    return doc
+
+
+def usage_block(help_text):
+    """Return the joined `usage:` block of an argparse-rendered help text."""
+    lines = help_text.splitlines()
+    start = None
+    for i, line in enumerate(lines):
+        if line.lower().startswith("usage:"):
+            start = i
+            break
+    if start is None:
+        return None
+    block = [lines[start][len("usage:"):].strip()]
+    for line in lines[start + 1:]:
+        if not line.strip() or not line.startswith((" ", "\t")):
+            break
+        block.append(line.strip())
+    return " ".join(block)
+
+
+def _strip_bracketed(block):
+    """Split a usage line into tokens, dropping every bracketed (optional) group."""
+    tokens, depth, cur = [], 0, ""
+    for ch in block:
+        if ch in "[(":
+            depth += 1
+            continue
+        if ch in ")]":
+            depth -= 1
+            continue
+        if depth:
+            continue
+        if ch.isspace():
+            if cur:
+                tokens.append(cur)
+                cur = ""
+        else:
+            cur += ch
+    if cur:
+        tokens.append(cur)
+    return tokens
+
+
+def usage_state(help_text):
+    """Classify a script's usage line.
+
+    Returns one of:
+      "unreadable"    - no argparse-style `usage:` block at all (hand-written usage)
+      "none-required" - a usage block exists but names no positional and no required option
+      "required"      - at least one positional or required option is named
+    """
+    block = usage_block(help_text)
+    if block is None:
+        return "unreadable"
+    return "required" if _strip_bracketed(block)[1:] else "none-required"
+
+
+def missing_input_args(help_text):
+    """Build an argv that points every positional and required option at a missing file.
+
+    Returns None when there is nothing to point at: either the usage block cannot
+    be recovered (the four scripts with a hand-written parser print their own
+    usage, not argparse's) or the script takes no required argument at all.
+    """
+    if usage_state(help_text) != "required":
+        return None
+    tokens = _strip_bracketed(usage_block(help_text))
+    args, i = [], 1          # tokens[0] is the program name
+    while i < len(tokens):
+        token = tokens[i]
+        if token.startswith("-"):
+            args.append(token)
+            if i + 1 < len(tokens) and not tokens[i + 1].startswith("-"):
+                args.append(NONEXISTENT)
+                i += 1
+        else:
+            args.append(NONEXISTENT)
+        i += 1
+    return args or None
+
+
+def has_option(help_text, option):
+    return option in help_text
+
+
+# Scripts that reach a device, a live frida server or a network peer once their
+# arguments parse. They are listed explicitly for one reason: this suite must not
+# touch a device, and a development host may well have one attached. Measured on
+# the machine this harness was written on, `adb devices` reported one online
+# device, and running `coldstart.py --pkg <name>` really did start
+# `adb logcat -v threadtime` and capture frames. So the exclusion is not
+# theoretical: it is the difference between a unit test and a device test.
+#
+# `--help` and an unknown flag are still exercised for these scripts: argparse
+# answers both before any device call is reached, which was verified for every
+# one of them (`--help` returned in well under a second, an unknown flag returned
+# 2). It is the argument-complete scenarios that are refused.
+DEVICE_OR_NETWORK_BOUND = {
+    "coldstart.py", "datastore_inject.py", "devsh.py", "frida_rpc_serve.py",
+    "grab_crash.py", "install_test.py", "lib_map.py", "mt_mcp_probe.py",
+    "probe_api.py", "run_probe.py", "sig_probe.py", "snap.py",
+    "spawn_patch_detach.py", "tls_check.py", "usb_net_proxy.py",
+}
+
+
+def execution_refusal(name, scenario, help_text):
+    """Why this script must not be executed for `scenario`, or None if it may be.
+
+    Returns a skip reason rather than a verdict: not executing a script is a
+    coverage limit, not a contract violation, and the two must not be reported
+    with the same word.
+    """
+    if name in DEVICE_OR_NETWORK_BOUND:
+        return ("%s reaches a device or the network once its arguments parse; %s is not "
+                "executed because this suite does not touch a device" % (name, scenario))
+    state = usage_state(help_text)
+    if state == "unreadable":
+        return ("%s prints its own usage instead of argparse's, so %s cannot be derived "
+                "automatically" % (name, scenario))
+    if state == "none-required":
+        return ("%s requires no argument, so %s is legal and the contract does not apply"
+                % (name, scenario))
+    return None
+
+# The scripts whose --json path is reachable on a bare checkout. Everything else
+# that advertises --json needs a sample or a device to get that far; those are
+# skipped with that reason rather than asserted against a guess.
+JSON_RUNNABLE = {
+    "doctor.py": ["--json"],
+    "preflight.py": ["--json"],
+}
+
+# Keys the unified JSON contract requires at minimum.
+JSON_REQUIRED_KEYS = ("status",)
+JSON_CONTEXT_KEYS = ("exit_code", "capability", "evidence", "warnings", "next_action")

tests/cli/test_cli_contract.py

@@ -0,0 +1,262 @@
+"""CLI contract tests: every shipped script, four invocation shapes each.
+
+What is asserted, per the repository-wide contract in REBUILD-BRIEF section 4:
+
+  1. `--help` succeeds and never prints an uncaught traceback.
+  2. An unknown flag is a usage error: exit 2.
+  3. An invocation with required arguments missing, or with every input pointed
+     at a file that does not exist, is a usage error: exit 2.
+  4. A script that advertises `--json` emits a parseable JSON document that
+     states a `status`.
+  5. Every observed exit code is one of 0/1/2/3/4.
+
+Two clean outcomes are distinguished, and the distinction is the whole point:
+
+  * `xfail` means the script is on the tree and does **not** meet the contract.
+    The reason carries the measured value. The xfail list this file produces is
+    the work order for exit-code unification -- it is not a defect to be fixed
+    here, because every one of those scripts has an owner.
+  * `skip` means the scenario is not applicable or not reachable on a machine
+    with no device, no network and no frida. Those are coverage gaps, and they
+    are reported as such in docs/tool-verification/EXTENSION-test-harness.md.
+
+Each scenario runs once per script and is cached for the session, so the suite
+spends its time on assertions rather than on re-spawning interpreters.
+"""
+from __future__ import annotations
+
+import pytest
+
+from conftest import all_scripts, run_script
+from contract import (
+    CONTRACT_EXIT_CODES, DEVICE_OR_NETWORK_BOUND, JSON_REQUIRED_KEYS,
+    JSON_RUNNABLE, execution_refusal, has_option, missing_input_args,
+    parse_leading_json,
+)
+
+pytestmark = pytest.mark.cli
+
+SCRIPTS = all_scripts()
+BAD_FLAG = "--zzz-not-a-real-flag"
+
+HELP_TIMEOUT = 60
+SCENARIO_TIMEOUT = 12
+
+
+def why(name):
+    if name in DEVICE_OR_NETWORK_BOUND:
+        return " (device/network bound: it may legitimately block)"
+    return ""
+
+
+def require_returned(result, what):
+    if result.timed_out:
+        pytest.xfail("%s did not return within %ss%s; there is no exit code to check"
+                     % (what, SCENARIO_TIMEOUT, why(result.argv[1])))
+    if result.returncode not in CONTRACT_EXIT_CODES:
+        pytest.fail("%s returned %r, outside the 0/1/2/3/4 contract vocabulary"
+                    % (what, result.returncode))
+
+
+def expect_exit(result, expected, what):
+    """Pass when the measured code matches; otherwise record it as an xfail."""
+    require_returned(result, what)
+    if result.returncode != expected:
+        pytest.xfail("%s: contract says exit %d, measured %d%s"
+                     % (what, expected, result.returncode, why(result.argv[1])))
+
+
+# --------------------------------------------------------------------------- #
+# one cached run per script per scenario
+# --------------------------------------------------------------------------- #
+@pytest.fixture(scope="session")
+def help_results():
+    return {n: run_script(n, ["--help"], timeout=HELP_TIMEOUT) for n in SCRIPTS}
+
+
+@pytest.fixture(scope="session")
+def badflag_results():
+    return {n: run_script(n, [BAD_FLAG], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS}
+
+
+@pytest.fixture(scope="session")
+def empty_results():
+    """Bare invocations, for the scripts that may be run at all.
+
+    Device-bound scripts are excluded here too: their no-argument path is exactly
+    where several of them start doing device work (`usb_net_proxy.py` did not
+    return within 20 s when probed by hand), and this suite does not touch a device.
+    """
+    return {n: run_script(n, [], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS
+            if n not in DEVICE_OR_NETWORK_BOUND}
+
+
+@pytest.fixture(scope="session")
+def missing_results(help_results):
+    """Argument-complete runs against a missing file, for the scripts that may be run at all.
+
+    Device-bound scripts are skipped here rather than executed and timed out: on
+    the host this harness was written on, `adb devices` reported an online device,
+    so running them would have touched real hardware.
+    """
+    results = {}
+    for name in SCRIPTS:
+        if name in DEVICE_OR_NETWORK_BOUND:
+            continue
+        args = missing_input_args(help_results[name].output)
+        if args is None:
+            continue
+        results[name] = run_script(name, args, timeout=SCENARIO_TIMEOUT)
+    return results
+
+
+# --------------------------------------------------------------------------- #
+# 1. --help
+# --------------------------------------------------------------------------- #
+class TestHelp:
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_help_never_crashes(self, name, help_results):
+        """Hard assertion: an uncaught traceback from --help is a defect, not a drift."""
+        result = help_results[name]
+        assert not result.timed_out, "--help did not return for %s" % name
+        assert not result.has_traceback, result.output[-2000:]
+
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_help_prints_something(self, name, help_results):
+        assert help_results[name].output.strip(), "--help printed nothing"
+
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_help_exits_zero(self, name, help_results):
+        expect_exit(help_results[name], 0, "%s --help" % name)
+
+
+# --------------------------------------------------------------------------- #
+# 2. unknown flag
+# --------------------------------------------------------------------------- #
+class TestUnknownFlag:
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_unknown_flag_is_never_success(self, name, badflag_results):
+        result = badflag_results[name]
+        require_returned(result, "%s %s" % (name, BAD_FLAG))
+        assert result.returncode != 0, (
+            "an unknown flag was reported as success:\n%s" % result.output[-2000:])
+
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_unknown_flag_exits_two(self, name, badflag_results):
+        expect_exit(badflag_results[name], 2, "%s %s" % (name, BAD_FLAG))
+
+
+# --------------------------------------------------------------------------- #
+# 3a. empty invocation
+# --------------------------------------------------------------------------- #
+class TestEmptyInvocation:
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_no_arguments_is_a_usage_error(self, name, help_results, empty_results):
+        """Only applies when the script actually requires arguments.
+
+        A script whose usage block has no positional and no required option may
+        legitimately succeed with no arguments; demanding exit 2 from those would
+        invent a rule the contract does not state. Device-bound scripts are not run
+        at all -- see `contract.execution_refusal`.
+        """
+        refusal = execution_refusal(name, "a bare invocation", help_results[name].output)
+        if refusal:
+            pytest.skip(refusal)
+        expect_exit(empty_results[name], 2, "%s with no arguments" % name)
+
+
+# --------------------------------------------------------------------------- #
+# 3b. bad input
+# --------------------------------------------------------------------------- #
+class TestMissingInput:
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_missing_input_file_is_a_usage_error(self, name, help_results, missing_results):
+        if name not in missing_results:
+            pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
+                        or "not executed for %s" % name)
+        expect_exit(missing_results[name], 2, "%s with a missing input" % name)
+
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_missing_input_is_never_success(self, name, help_results, missing_results):
+        if name not in missing_results:
+            pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
+                        or "not executed for %s" % name)
+        result = missing_results[name]
+        require_returned(result, "%s with a missing input" % name)
+        assert result.returncode != 0, (
+            "a missing input was reported as success:\n%s" % result.output[-2000:])
+
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_bad_input_reports_cleanly_not_as_an_exception(self, name, help_results,
+                                                           missing_results):
+        """A missing file is an expected failure mode; it must not escape as an exception.
+
+        Several scripts currently let `FileNotFoundError` propagate. Each is
+        recorded here as an xfail naming the exception; the fix belongs to that
+        script's owner, not to this suite.
+        """
+        if name not in missing_results:
+            pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
+                        or "not executed for %s" % name)
+        result = missing_results[name]
+        require_returned(result, "%s with a missing input" % name)
+        if result.has_traceback:
+            tail = [l for l in result.output.splitlines() if l.strip()]
+            pytest.xfail("%s reports a missing input by letting an exception escape "
+                         "instead of exiting 2 cleanly: %s"
+                         % (name, (tail[-1] if tail else "uncaught exception")[:140]))
+
+
+# --------------------------------------------------------------------------- #
+# 4. the --json contract
+# --------------------------------------------------------------------------- #
+def json_capable(help_results):
+    return sorted(n for n, r in help_results.items() if has_option(r.output, "--json"))
+
+
+def test_json_capable_scripts_are_discoverable(help_results):
+    found = json_capable(help_results)
+    assert found, "no script advertises --json; the discovery method is broken"
+
+
+@pytest.mark.parametrize("name,args", sorted(JSON_RUNNABLE.items()))
+def test_json_output_is_parseable_and_states_a_status(name, args):
+    result = run_script(name, args, timeout=120)
+    require_returned(result, "%s %s" % (name, " ".join(args)))
+    doc = parse_leading_json(result.stdout)
+    assert doc is not None, (
+        "%s %s printed no parseable JSON document:\n%s"
+        % (name, " ".join(args), result.output[-2000:]))
+    missing = [k for k in JSON_REQUIRED_KEYS if not isinstance(doc, dict) or k not in doc]
+    if missing:
+        present = sorted(doc) if isinstance(doc, dict) else type(doc).__name__
+        pytest.xfail("%s --json does not carry %s; top-level keys are %s"
+                     % (name, missing, present))
+
+
+@pytest.mark.parametrize("name", SCRIPTS)
+def test_json_capable_but_unreachable_scripts_are_named(name, help_results):
+    """Coverage honesty: a --json script is either exercised here or explicitly not."""
+    if not has_option(help_results[name].output, "--json"):
+        pytest.skip("%s does not advertise --json" % name)
+    if name in JSON_RUNNABLE:
+        return
+    pytest.skip("%s advertises --json, but reaching that path needs a device or a "
+                "sample; the gap is listed in EXTENSION-test-harness.md" % name)
+
+
+# --------------------------------------------------------------------------- #
+# 5. the exit-code vocabulary
+# --------------------------------------------------------------------------- #
+class TestExitCodeVocabulary:
+    @pytest.mark.parametrize("name", SCRIPTS)
+    def test_observed_codes_stay_inside_the_vocabulary(self, name, help_results,
+                                                       badflag_results):
+        for label, result in (("--help", help_results[name]),
+                              (BAD_FLAG, badflag_results[name])):
+            if result.returncode is not None:
+                assert result.returncode in CONTRACT_EXIT_CODES, (
+                    "%s %s returned %r" % (name, label, result.returncode))
+            assert result.returncode is None or result.returncode >= 0, (
+                "a negative code is a crash signal, not an exit status: %s %s"
+                % (name, label))