Four-state labels, and the negatives that used to look alike

Every claim carried one of three labels, and the third was doing two jobs. `unverified` meant
both "someone asserted it and nobody here paid for the counters" and "no evidence was gathered
at all" -- different statements with different next actions. This adds `unknown` for the second.

The distinction that pays for itself lives inside a *negative* result, because an empty probe has
three causes and only one of them is a finding:

  absent                  -- coverage reached the question, nothing is there
  present but unreadable  -- the thing is there and this route cannot read it
  unknown                 -- nothing covered the question

The middle one is the dangerous one: it produces byte-identical output to the first, and it is the
normal case on the input this skill is aimed at. A packed APK returning four classes is not an APK
with four classes, and "the string was not in the dex" is a statement about a probe. The rule now
says to name the probe's coverage and run a control that must hit before writing "not present", and
to record `unknown` about the probe rather than `absent` about the target.

Also in this pass:

  * P47 -- two tools disagree about the same artifact, and the disagreement is your parser. One
    extraction rule reused across outputs that do not share a format: an unescaped `$` dropped every
    descriptor containing one, and dotted names were compared against DEX descriptors so one side
    parsed to nothing. The comfortable failure is the one that confirms what you already expected,
    which is why it survived the sanity check it should not have.
  * dexsql: a cache is second-hand evidence and it ages silently. A stale cache answers for the old
    artifact with no error, and `--trusted-input` skips revalidating the recorded source inputs --
    so a cached run supports "true of the artifact the cache was built from", not "true of the
    artifact in front of me".
  * ete-diff-tracing: a divergence is a location, not a cause. The differential method produces a
    positional claim, and three readings it does not authorise are written down beside the step that
    produces it.
  * CONTRIBUTING.md -- the gates, what a green run does and does not establish, the sync rules a
    change has to keep (a new reference must be named in the inventory; renaming a heading breaks
    every section reference that pointed at it), and the two failure modes to watch when editing the
    gates: a limit that cannot fail, and a gate whose answer varies with the runner. README links it
    from both language editions.
This commit is contained in:
inliver233
2026-10-16 19:17:31 +08:00
parent b8aa0a529b
commit 6ab905b2c8
12 changed files with 274 additions and 6 deletions
+107
View File
@@ -0,0 +1,107 @@
# Contributing
This repository is a skill plus the evidence behind it. Two properties keep it maintainable by more
than one person, and both are **enforced by scripts rather than requested in prose**:
- **every claim carries a strength label**, and
- **every structural relationship is checked** — paths resolve, the on-demand inventory matches the
entry point, documented commands exist, the always-loaded part has not crept.
The checks exist because both properties decay silently. A stale index still reads as complete; a
claim written from expectation reads like a measurement. If you change something and a gate goes red,
the gate is usually right.
## Run the gates before you push
```bash
python check_repo.py && python check_refs.py && python check_routing.py \
&& python check_commands.py && python check_budget.py && python -m pytest -q
```
Nothing above needs a device, a sample APK, or the network. CI runs the same set on Python 3.11–3.14
plus `mypy` and `flake8`, so a green local run is what a green CI run means.
If `flake8`/`mypy` are not installed, that is the one part CI checks that you cannot reproduce
locally; everything else is plain Python.
## What each gate owns
| Gate | Owns | It cannot see |
|---|---|---|
| `check_repo.py` | frontmatter parses, scripts answer `--help`, every referenced path resolves, every reference is reachable, README paths exist, no target identity on the tracked surface | whether a claim is true |
| `check_refs.py` | every section reference reaches a real heading in the file it names | whether the section still says what the link implies |
| `check_routing.py` | `references/routing.md` and `SKILL.md` agree: symptom mirror identical, every reference file named, every script named | semantics — you can name a file in the wrong row |
| `check_commands.py` | every `python scripts/x.py --flag` a document tells a reader to run matches the script's argparse table | commands inside fenced blocks it cannot attribute to a script |
| `check_budget.py` | `SKILL.md` size in lines and tokens, index-row length, long files with a navigable head | whether a new file is load-bearing or merely adjacent |
**A green run proves consistency, not correctness.** It says paths resolve and the inventory is
complete. It does not say a mechanism works, a tool behaves as described on your device, or a fixture
stands in for a real target. Only the strength labels and the evidence record answer those.
## What a change has to keep in step
This is the list that costs a contributor a red CI run if they miss it:
| If you add or change | Also do | Caught by |
|---|---|---|
| a reference file | name it in `references/routing.md`; make it reachable from `SKILL.md` or the README; give it a "load this when" head if it passes 100 lines | `check_routing.py`, `check_repo.py`, `check_budget.py` |
| a script | add a row to the script table in `references/routing.md`; declare its capability in `scripts/capabilities.py` | `check_routing.py`, `check_repo.py` |
| a symptom-index row | mirror it in `routing.md` (the mirror is compared, not generated) | `check_routing.py` |
| a section heading | update every section reference that pointed at the old name | `check_refs.py` |
| a command in a document | keep the flags real — copy them from `--help`, do not retype them | `check_commands.py` |
| `SKILL.md` | watch the budget: it is loaded in full on every activation, index rows included | `check_budget.py` |
| a capability claim | add or update the row in `evidence/capability-matrix.json` **and** the line in `references/evidence-summary.md` | review — no gate compares them |
| a new pitfall | next number, same four headings (`Symptom` / `Root cause` / `Why it is hard to see` / `The rule`) | review |
The last two rows are deliberately unenforced. A gate can check that a file is named; it cannot check
that a claim is honest, which is why those two are the ones a reviewer has to read.
## The label discipline
Every claim in this repository carries one of four labels (`SKILL.md` §Coverage defines them and
`references/coverage-and-limits.md` carries the long form): `observed`, `inferred`, `unverified`,
`unknown`. Two rules matter more than the rest:
- **`observed` requires a command and its output.** "I reasoned it out" is `inferred`; "someone said
so" is `unverified`; "nobody looked" is `unknown`.
- **A negative result is `absent`, `unknown` or `present but unreadable`, and only the first is a
finding.** A probe that came back empty because it could not read the input looks identical to one
that came back empty because the thing is not there. That confusion is *normal* on hardened targets,
so name the probe's coverage and run a control that must hit before writing anything down as
"not there".
If you measured something on one version, one device or one ABI, say which. A number without its
scope is the most common way a correct measurement becomes a wrong claim.
## Pull requests
- **Say what you ran.** Paste the command and the output that matters, not a summary of your
confidence. Report only checks that actually executed; "should pass" is not a result.
- **Label your own claims** in the same four labels. A large change whose assumptions are labelled is
easier to review than a small one that presents them as findings.
- **Keep the gate output in the PR body** when a gate was the reason for a change. It is the cheapest
evidence a reviewer can check.
- **Do not treat a green CI run as approval.** It establishes that code runs and paths resolve — not
that a claim is true, not that a limit is honest, and not that the change belongs here.
- **Prefer a narrow change with a stated boundary** over a broad one with an implied guarantee. Where
something could not be exercised here, ship the measured half plus an explicitly labelled template
for the rest, and say what would have to exist for the other half to become `observed`.
- **One variable at a time.** If a result changed, it should be attributable to the one thing you
changed (`references/long-task-discipline.md` §single-variable discipline).
## Changing the gates themselves
The gates are code and they can be wrong in two directions: blind (a defect they should catch passes)
and noisy (they fail on a clean tree). Both are testable, and this repository has been bitten by both:
- **A limit that cannot fail is decorative.** `check_budget.py` once computed its hard index-row limit
and reported `warn` on both branches, so an index row could grow without bound while CI stayed
green. Watch for the shape where a threshold is defined, evaluated and never escalates.
- **A gate that only varies with the runner is not a gate.** `check_budget.py`'s token estimate had a
guessed constant, so CI (no `tiktoken`) and a developer machine reached different verdicts.
- **The way to find the first class is to inject the defect and require the failure.** Add the case to
`tests/unit/` so the next person cannot re-break it: `test_budget_measurement.py` pins both
behaviours above.
When you fix a gate, add the test that would have caught it, and say in the PR which defect the test
was written against.
+40
View File
@@ -598,6 +598,11 @@ Four qualifications that the Coverage section states in full and that belong her
## Repository maintenance
**Changing something here? Start with [CONTRIBUTING.md](CONTRIBUTING.md).** It lists the gates, what a
green run does and does not establish, the sync rules a change has to keep (a new reference must be
named in the inventory, renaming a heading breaks every section reference that pointed at it), and the
two failure modes to watch for when editing the gates themselves.
Four tools live at the root and are not part of the installed skill:
```
@@ -626,6 +631,41 @@ Consistency has a natural counter-pressure -- a broken path fails loudly, and so
Bloat has none, which is why the third tool exists: every pass adds a reference, an index row and
a coverage claim, and without a measurement nothing in the repository notices.
### What a green gate establishes, and what it does not
The checks above prove **consistency**, not truth. `check_repo.py`, `check_refs.py`,
`check_routing.py`, `check_commands.py` and `check_budget.py` passing means paths resolve, the
inventory is complete, documented flags exist, the entry point has not crept, and no target
identity reached the tracked surface. They say nothing about whether a *claim* is right: a
reference can cite a heading that exists and still describe a mechanism nobody ran, and a budget
gate cannot tell a load-bearing file from an adjacent one. Those are the two questions a reader
actually has, and the only things that answer them are the strength labels and the evidence record.
### Integrating an outside contribution
**Fail closed.** If the contribution's fit with what this repository claims to cover, its
side effects, or the verification needed to establish either one is **materially unresolved**, do
not merge it and do not ship it: leave it pending, narrow it, or decline it. Recording the open
question is a valid outcome; converting it into "none" or "probably fine" is not.
Four things that are not approval, however good they look: **a green CI run, a finished
implementation, upstream popularity, and the absence of an objection.** They establish that code
runs and that nobody has complained, which is a different statement from *this belongs here and its
limits are known*.
Two consequences that decide most of these cases:
- **Labels over volume.** A large contribution whose every claim carries a correct `observed` /
`inferred` / `unverified` / `unknown` label is easier to accept than a small one that presents
assumptions as findings, because the first one can be checked. Ask what was actually run, and
treat "no claim was upgraded to `observed` by reading about it" as a good sign rather than a
weakness.
- **Unresolved is `unknown`, never absent.** A capability that has not been exercised here is
`unknown` -- not excluded, not disqualified, and also not merged on the assumption that it
probably works. Where a contribution ships a device-bound or toolchain-bound half, the mergeable
part is the half that was measured plus an explicitly labelled template for the rest, with the
gate that says what would have to exist for the other half to become `observed`.
`tests/benchmark.md` holds the **regression matrix**: dimension -> public target -> the scripts the
row exercises -> measured result -> strength label. It is the checklist to re-run before trusting
any claim under `docs/tool-verification/`. Samples are downloaded into `tools/_work/` and are never
+17
View File
@@ -461,6 +461,8 @@ Coverage 一节完整陈述、同时也应当放在这里的四条限定:
## 仓库维护
**要改动这里的东西?先读 [CONTRIBUTING.md](CONTRIBUTING.md)。** 它列出各道门禁、一次全绿意味着什么又不意味着什么、改动必须同步的规则(新增一个 reference 必须在清单里被点名;重命名一个标题会让所有指向它的章节引用断掉),以及修改门禁脚本自身时要留意的两类失效模式。
根目录下住着四个工具,它们不属于已安装的 skill:
```
@@ -485,6 +487,21 @@ build_scripts.py 审计机器特定残留物(绝对路径、凭据)
一致性有一种天然的反向压力 —— 一条断掉的路径会大声失败,然后有人修它。膨胀没有,这正是第三个工具存在的原因:每次迭代都会加一个 reference、一行索引和一条覆盖主张,而没有测量的话,仓库里没有任何东西会注意到。
### 全绿的门禁证明了什么,又不证明什么
上面的检查证明的是**一致性**,不是真相。`check_repo.py`、`check_refs.py`、`check_routing.py`、`check_commands.py`、`check_budget.py` 全绿,意味着路径可解析、清单完整、文档里的 flag 真实存在、入口点没有膨胀、受跟踪表面上没有目标身份。它们对**一个主张是否正确**什么也没说:一个引用可以指向真实存在的标题,却描述一个没人跑过的机制;预算门禁也分不清一个文件是承重还是只是相邻。而这两件事恰恰是读者真正要问的,能回答它们的只有强度标注和证据记录。
### 接纳外部贡献
**失败关闭。** 如果一项贡献与仓库宣称的覆盖范围的契合度、它的副作用、或确立这两者所需的验证**仍有实质未决**,就不要合并、不要发布:让它挂起、缩小它,或者拒绝它。把未决问题记录下来是有效的结果;把它变成「无」或「大概没问题」则不是。
有四样东西不是批准,无论它们看起来多好:**全绿的 CI、完成的实现、上游的流行度、以及没有人提出异议。** 它们证明的是代码能跑、还没人抱怨,这与「这属于这里、而且它的边界已知」是两回事。
两个决定多数此类判断的推论:
- **标注优先于体量。** 一份每一项主张都带着正确 `observed` / `inferred` / `unverified` / `unknown` 标注的大改动,比一份把假设当成结论的小改动更容易被接受,因为前者可以被核对。要问的是实际跑了什么;把「没有哪个主张是靠读资料升级成 `observed` 的」看作好迹象,而不是弱点。
- **未解决就是 `unknown`,绝不是不存在。** 一个在这里没有被演练过的能力是 `unknown`:不是被排除,不是被取消资格,也不是因为「大概能行」就合并。当一项贡献含有一半绑定设备或工具链的内容,可合并的是**被测量过的那一半**,加上为其余部分明确标注的模板,以及说明「另一半要变成 `observed` 需要什么存在」的那道门。
`tests/benchmark.md` 保存**回归矩阵**:维度 -> 公开目标 -> 该行演练的脚本 -> 实测结果 -> 强度标注。它是你在信任 `docs/tool-verification/` 下任何主张之前要重跑的检查表。样本会下载进 `tools/_work/` 且从不提交,因此每一行都点名它的公开来源并记录它所针对的哈希。
`docs/tool-verification/` 也不属于已安装的 skill。它是针对一个真实目标、一次测量迭代的证据记录:每个脚本实际做了什么、哪个独立方法确认了它、发现了哪些缺陷,以及该目标未能演练哪些场景。它存在的意义,是让 `SKILL.md` 里的 **Coverage** 主张可以被对照运行来检查,而不是被信任;也让缺口被写在下一个人能找到它们的地方。
+21 -3
View File
@@ -8,7 +8,7 @@ metadata:
capability_registry: skills/apk-reverse/scripts/capabilities.py
evidence_summary: skills/apk-reverse/references/evidence-summary.md
last_reconstruction_pass: "2026-09-22"
strength_labels: "observed | inferred | unverified"
strength_labels: "observed | inferred | unverified | unknown"
---
# APK Reverse Engineering & Patching
@@ -102,13 +102,31 @@ that almost fits is more dangerous than no method at all, because it arrives wit
vocabulary and a set of reassuring numbers.
**How strong these claims are — this is load-bearing, not cosmetic.** Every claim in this skill carries
one of three labels, and you must label your own results the same way:
one of four labels, and you must label your own results the same way:
| Label | Means | May it justify a decision? |
|---|---|---|
| `observed` | a command was run and its output exists | yes |
| `inferred` | follows from an observation, but the step is reasoned | provisionally |
| `unverified` | assumed, or reported elsewhere and never reproduced here | only as something to test |
| `unknown` | nothing here covers it: no probe ran, or the probe's coverage did not reach it | no — and never as absence |
**A negative result has three shapes, and folding them together is the failure this section exists
for.** `unverified` means somebody asserted something and nobody here paid for the counters; `unknown`
means no evidence was gathered at all. But a probe that *did* run and came back empty means one of two
different things, and only one of them is a finding:
- **absent** — the probe's coverage reached the question and the thing genuinely is not there;
- **present but unreadable** — the thing is there and *this route* cannot read it: an encrypted payload
inside a shell, a library the running ABI never loads, a check that fires before your hooks exist, a
field that exists under a type your parser rejects.
The second is the dangerous one, because it produces byte-identical output to the first and it is the
**normal** case on the hardened targets this skill is aimed at. A packed APK returning four classes is
not an APK with four classes. So before "not present" goes into the record, name the probe that covered
the question and run a control that must hit. If the control is silent too, the honest statement is
`unknown` about **your probe**, not `absent` about the target — and the next move is a different probe,
not a stronger conclusion.
**The inventory that replaces a guess:** the per-route list of what is covered and what is not — by
verified mechanism versus by documented-but-mostly-inferred route — is in
@@ -390,7 +408,7 @@ These are moments where continuing to push forward is the wrong move. Each has c
| **Neutralise a native terminate path by returning, never by making it not return** | a spin stub freezes the caller **holding its lock**; unrelated threads wedge and the symptom (hang, external kill, restart loop) looks nothing like the cause, with no crash record | return a benign value, prefer success (`0`) over failure (`-1`), never touch `pthread_exit`/`exit`/`abort`/`snprintf`/`closedir`. `native-tamper-and-suicide.md` |
| **Look before you conclude, and look while you wait** | a screen that is actually looked at answers in one step what coordinate-guessing cannot answer in five | capture **and inspect**; byte-identical samples mean nothing will change. `scripts/snap.py`, `environment.md` §look at the screen |
| **Bound every command, and calibrate the bound from a measurement** | an unbounded call turns a stall into "still working", which is indistinguishable from progress | time the operation once, record it, derive the deadline from it. `long-task-discipline.md` §Bound every wait |
| **Every claim carries a label — `observed`, `inferred` or `unverified`** | "probably", "should be" and "in theory" are not findings, and an unlabelled guess propagates as a fact | `observed` requires a command **and its output**; nothing else may be written as established |
| **Every claim carries a label — `observed`, `inferred`, `unverified` or `unknown`** | "probably", "should be" and "in theory" are not findings, and an unlabelled guess propagates as a fact | `observed` requires a command **and its output**; nothing else may be written as established. A probe that did not reach the question is `unknown`, never `absent` |
| **"Done" means the user-visible outcome** | a blocking dialog still on screen means the task is not done, however clean the log is | §What "done" means, all six items; absence of a log line is never evidence of success |
| **Never discard a route on compound evidence** | if a failure followed two simultaneous changes, the attribution is a hypothesis — mis-attributed failures have removed viable routes for hours | re-run the abandonment single-variable before writing it down |
| **Prove the device changed before measuring anything** | install success describes the *request*, not the app on disk | hash the on-device artifact against your build; otherwise every later observation describes the previous build |
@@ -11,7 +11,8 @@
"strength_labels": {
"observed": "A command was run and its output exists behind the claim.",
"inferred": "Follows from a documented mechanism or a neighbouring measurement; the step itself was not executed.",
"unverified": "Assumed, or reported by someone else, and not reproduced in this record."
"unverified": "Assumed, or reported by someone else, and not reproduced in this record.",
"unknown": "Nothing here covers it: not probed, or probed without the coverage to answer. Missing evidence, never a negative result -- do not read an `unknown` row as 'absent' or 'safe'."
},
"evidence_rules": [
"Every path in `evidence` is relative to the installed skill root and resolves in an installed copy.",
@@ -15,6 +15,8 @@ take. Silence in this list is not support for a route; it means nobody here paid
- **inferred** — follows from a documented mechanism or a neighbouring measurement; the step itself
was not executed.
- **unverified** — assumed, or reported by someone else, and not reproduced in this record.
- **unknown** — nothing here covers it: not probed, or probed without the coverage to answer. Missing
evidence, not a negative result.
A row whose strength is `unverified` is a statement about *this skill's evidence*, not about the
mechanism's truth. It means nobody has yet put a command and its output behind it here.
@@ -18,11 +18,25 @@ what was and was not exercised. Load it when a claim's strength decides whether
- **inferred** — follows from documented mechanism or from a neighbouring measurement; the step
itself was not executed.
- **unverified** — assumed, or reported by someone else, and not reproduced in this repository.
- **unknown** — nothing here covers it: the question was never probed, or the probe's coverage did
not reach it. This is missing coverage, not a negative result.
Reserve `observed` strictly (`references/long-task-discipline.md` does). "I reasoned it out" is
`inferred`. The distinction is load-bearing, not cosmetic: an `inferred` route may still be right,
but nobody here has paid for the counters yet.
**Read a negative result as `unknown` unless the probe's coverage is established, and check for the
third state first.** An empty result has three possible causes and only one of them is a finding:
**absent** (coverage reached the question, nothing is there), **present but unreadable** (the thing is
there, and this route cannot read it), or **`unknown`** (nothing covered it). The second is the one
that costs hours, because it is indistinguishable from the first and it is the *normal* case on
hardened input: "the string was not in the dex", "the hook never fired", "no request reached the
endpoint" each have a failure mode that produces the same empty output as a true negative — a query
whose argument never arrived intact, a hook installed after the check that matters, a request refused
before it was built. A small class count on a packed target is the payload being unreadable, not
absent. Name the probe's coverage, run a control that must hit, and write `unknown` about the probe
rather than `absent` about the target.
## Covered, by verified mechanisms — the evidence
- **Client-side ads, promos and splash/popup/tab configuration**, including the server-issued UI
@@ -32,6 +32,8 @@ what is *exempt*, in code, that can be read and argued with.
- **inferred** — follows from an observed fact or from documented mechanism; the step itself
was not executed.
- **unverified** — assumed, or reported elsewhere, and not reproduced in this repository.
- **unknown** — nothing here covers it: not probed, or probed without the coverage to answer. Missing
evidence, not a negative result.
Claim strength in this file means *how well the statement about desensitization was measured*,
not how confident the writing sounds.
@@ -127,6 +127,22 @@ parse-and-build path is already 0.12 s on this archive. Treat the cache as a way
across many queries in a long session, not as a speed switch; on small archives it is a net loss
(cache read 0.071 s against a 0.042 s direct query on the 38.7 MB sample).
**A cache is second-hand evidence, and it ages silently.** The cache keys on the artifact it recorded,
and a stale cache answers for the **old** file with no error: query a rebuilt APK through yesterday's
`cache.dxc2` and every count, string and caller list describes the previous build. Two consequences
worth acting on:
- **The cache file is a separate object from the source.** "I queried this APK" is a claim you have to
keep true yourself, the same way `repack-and-sign.md` requires the on-device hash before any later
measurement means anything. Record the source path **and its hash** next to the cache, and rebuild
after the artifact changes.
- **`--trusted-input` skips revalidating the cache's recorded source inputs.** It exists for a
controlled build pipeline, and reaching for it to make a suspicious artifact load converts a stale
answer into an apparently fresh one. Do not use it on a target you are still investigating.
Scope the claim accordingly: a cached run supports "this was true of the artifact the cache was built
from", not "this is true of the artifact in front of me".
**Read-only is enforced, and the error surface is uniform.** Missing class → `Error: not_found:
selected file, method, or class not found`, exit 1. Missing input, a non-dex input, an unknown column,
an unknown relation and a `DELETE` all exit 1 with a named cause; the write is refused.
@@ -122,6 +122,24 @@ pair was built to ask:
same window, one-bit difference. A diff across two builds is noise with a confident
rendering.
**A divergence is a location, not a cause.** Everything §4 produces is a positional claim: *these
blocks execute only when input A is present*. That is already valuable, and it is routinely read as
more than it is. Three upgrades it does **not** authorise on its own:
- **"this block validates the credential"** — it may be a downstream consequence of a decision taken
earlier; the first divergence is the honest candidate for the decision, and even that is a
candidate.
- **"patching this block flips the outcome"** — untested until the patch exists and both inputs have
been run through it (R2, single variable, with the unpatched run as control).
- **"the blocks that matched are irrelevant"** — untested until a control run shows they stay matched
when the outcome changes.
What promotes a candidate to a cause: step 5's swapped-input repeat, or a patch that moves the
outcome and a control that does not. Until one of those has run, the finding to write down is "the
first divergence is at X" — a place, with the evidence that it is input-dependent — and not "X is
what decides this". The distinction is the same one `verification.md` draws between *the mechanism
is proven* and *the behaviour changed*.
## 5. What the obfuscator actually built — read from two open implementations
The differential reads better when you know what the two families actually emit. The facts
+32
View File
@@ -1405,3 +1405,35 @@ data is still plaintext** — a JNI boundary beats a TLS interior every time. Ea
the next layer: TLS-blind says "own crypto", dex-empty says "JS/native bundle", and the
RegisterNative inventory turns "somewhere in libfoo" into a function address. *(Absorbed from a
public write-up, 2026-09.)*
## P47. Two tools disagree about the same artifact, and the disagreement is your parser
**Symptom**
Two indexers report different class counts for one APK: one says 647, the other says 38. Both exit
cleanly, both print plausible output, and the smaller number has an innocent explanation ready to
hand — a hardened target really does expose few classes — so the discrepancy is filed as "the tools
disagree on packed input" and the investigation moves on with a wrong number in the record.
**Root cause**
One extraction rule was reused across outputs that do not share a format. Two defects in one pass:
an unescaped metacharacter in the pattern (`$` inside a class descriptor is an end-of-line anchor, so
every name containing `$` or a non-ASCII character was dropped), and two different representations
compared as if they were one (a tool printing dotted Java names against a tool printing DEX
descriptors, so one side parsed to nothing). Neither is a property of the tools. The measurement was
reporting the parser's blind spot and attributing it to the artifact.
**Why it is hard to see**
Cross-checking tools is what this skill tells you to do, and the comfortable failure is the one that
*confirms* something you already expect: a packed APK returning few classes is true, so the wrong
number passes the one sanity check you were most likely to run. Both sides also produced output of
the right *shape*, which reads as agreement.
**The rule**
**A count disagreement between two tools is a claim about your parser until you have proved
otherwise.** Parse each tool's own output with a parser written for that format, normalise both to
one representation, and only then compare — and let the comparison carry its own control: a query
whose answer you already know, pushed through the same path, before any "the tools disagree" verdict
is recorded. Two traps recur: regex metacharacters that are legal characters in the data (`$` in
descriptors, `[` in array types, `.` in package names), and representation mismatches that hide
behind a matching line count. The encoding cousins of this failure are in `framework-runtimes.md`
and the line-ending trap in `patch-audit.md`.
+3 -2
View File
@@ -18,9 +18,10 @@ are nothing of the kind:
`scripts/repack.py` handles both routes and **refuses to guess**: it inventories the set, decides
which route is legal from what the members actually carry, prints the reason, and only then acts.
**Strength labels** (same three this repository uses everywhere): **observed** = reproduced with an
**Strength labels** (the same set this repository uses everywhere): **observed** = reproduced with an
exact command and output; **inferred** = follows from observed facts, step not executed;
**unverified** = reported or assumed, not independently confirmed. The measurements behind this
**unverified** = reported or assumed, not independently confirmed; **unknown** = not covered here at
all, which is missing evidence rather than a negative result. The measurements behind this
file are recorded in `references/evidence-summary.md` §The capability matrix.