Verify the absorbed claims on the host and label them per claim

Adds docs/tool-verification/fixtures/verify_absorbed_claims.py: seven device-free checks covering the mask identity, the PKCS#7 length law, zero-fill versus skipped offsets, the hidden-salt search, compression drift, Python percent-encoding, and AES CTR/CBC/ECB separation.

The AES run falsified an absorbed claim: zero prefix-XOR does not imply a stream cipher (CBC and ECB show it too). The discriminator is a one-byte flip -- 1 changed byte for CTR, all following blocks for CBC, one block for ECB -- and protocol-reverse section 7 now states that, with the capability row recording the correction.

Labels are now per claim rather than per section: observed where a host command produced output, inferred for rules derived from the mechanisms, unverified for anything needing a device, a JDK, a lz4 binding or a VM sample. native-vm-recovery's claim-strength paragraph is aligned with the repository definition, and the cipher row moves to inferred.

Gates and pytest green.
This commit is contained in:
root
2026-10-25 12:19:37 +08:00
parent 45ba13c0f3
commit 068280a180
13 changed files with 356 additions and 67 deletions
@@ -1,8 +1,9 @@
# EXTENSION — second reference absorption from `ref/*` (2026-10-25)
This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into
the files that already own the topics. It did not run any of them. The record exists so a later reader
can tell what changed and on what footing.
the files that already own the topics. None was run end to end; a host-level pass (§Verification pass)
settles the claims that need no device. The record exists so a later reader can tell what changed and
on what footing.
## Sources
@@ -16,8 +17,9 @@ per-build address are omitted; none was copied onto the tracked surface.
| `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift |
All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing
in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering
judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was
in them was reproduced here. Mechanisms that need a target are therefore `unverified`, the ordering
judgements built on them are `inferred`, and the host-checkable ones are settled in §Verification
pass. No text, code, key material, offset table or screenshot was
copied.
## What was added
@@ -30,15 +32,16 @@ No new reference files. Each mechanism was folded into the file that already own
| `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker |
| `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals |
| `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections |
| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required |
| `references/protocol-reverse.md` §7 | three black-box probes (header length, determinism, one-byte diffusion width, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile. The probe's original prefix-XOR-only claim was falsified by the host run below and the section now states the corrected discriminator | claim any cipher was classified; assume byte-identical compression is required |
| `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm |
| `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method |
## Registration surface updated
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`,
`whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all
`partial` / `route_measured: false` / `unverified`; two existing rows
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader` and
`whitebox-crypto-devirtualization-to-program` are `partial` / `route_measured: false` / `unverified`;
`cipher-classification-by-differential-probe` is `partial` / `route_measured: false` / `inferred`
after the host checks below; two existing rows
(`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended
rather than duplicated.
- `references/evidence-summary.md`: three rows.
@@ -47,6 +50,86 @@ No new reference files. Each mechanism was folded into the file that already own
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
symptom mirror and the reference/script inventories had nothing to learn.
## Verification pass — host, 2026-10-25
Purpose: separate what the write-ups report from what this host can show, and label per claim rather
than per section. Three tiers were applied: `observed` (a command ran here and its output is below),
`inferred` (follows from a mechanism or a neighbouring measurement), `unverified` (reported by
someone else, not reproduced). No device, protected sample or server was involved, so **no claim
about a target was upgraded**.
```bash
python3 docs/tool-verification/fixtures/verify_absorbed_claims.py
```
```text
PASS mask identity (protocol-reverse section 7, involution)
65536 eight-bit (x,m) pairs: 0 counterexamples
200000 random 16-bit pairs: 0 counterexamples
PASS PKCS#7 length law (protocol-reverse section 7, probe 3)
b=16, p=0..64: 0 mismatches
PASS unreadable page: skip shifts, zero-fill preserves (linker section 6.3)
expected offset 8199
skip -> 4103
zero-fill -> 8199
PASS hidden salt defeats a visible-parameter search (signature-derived-keys variant)
36 combinations over 3 algorithms and 4 field orders: 0 hits
with the salt included: match
PASS compressed bytes are implementation/parameter defined (protocol-reverse section 7)
level 1 = 49 bytes, level 9 = 48 bytes, identical = False
both round-trip to the payload
PASS Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7)
quote = a%20b~%2A-_.
quote_plus = a+b~%2A-_.
Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK)
PASS AES mode separation (protocol-reverse section 7, probe 2)
ctr prefix-xor dirty=0/32 one-byte-flip changed=1/32
cbc prefix-xor dirty=0/32 one-byte-flip changed=32/32
ecb prefix-xor dirty=0/32 one-byte-flip changed=16/32
all three modes show zero prefix-XOR: that probe cannot separate them
7 check(s), 0 failed
RESULT=ok
```
### What the run falsified
The absorbed probe said *zeros over the common prefix ⇒ a stream cipher*. Measured on AES-128 with a
fixed IV, **CTR, CBC and ECB all produce zero prefix-XOR** (0 of 32 bytes differ): any deterministic
mode reproduces the ciphertext of a prefix, so the experiment only proves the absence of a
per-message nonce. The discriminator that does work is a one-byte plaintext flip: CTR changes exactly
1 byte, CBC changes every block from the flipped one onward (32 of 32 here), ECB changes exactly one
block (16 of 32, later blocks untouched). The reference now states the corrected procedure; the
capability row records the correction instead of the original claim.
**A label convention was corrected.** `native-vm-recovery.md`'s claim-strength paragraph called the
absorbed mechanics `inferred`. Under the repository's definition (`unverified` = reported by someone
else, not reproduced) the mechanics are `unverified`, and only the ordering judgements are
`inferred`. The paragraph and the checklist now say that; no capability row changed for it.
### Claims the run settles (host only)
| Claim | Where | Now |
|---|---|---|
| `(x & ~m) \| (~x & m) == x ^ m`, and the composition is an involution | `protocol-reverse.md` §7 | `observed` |
| PKCS#7 padded length is `b * ceil((len+1)/b)` | `protocol-reverse.md` §7 | `observed` |
| Zero-filling an unreadable page preserves later offsets; skipping shifts them | `linker-and-soinfo-repair.md` §6.3 | `observed` |
| An unknown salt inside the digest defeats a search over visible parameters | `signature-derived-keys.md` §Variant | `observed` |
| Two compressors/parameters can emit different valid bytes for one payload | `protocol-reverse.md` §7 | `observed` (general) |
| Python's encoder treats ` `, `*`, `~` differently from Java's | `protocol-reverse.md` §7 | `observed` (Python side) |
| CTR / CBC / ECB are separable by diffusion width, not by prefix-XOR | `protocol-reverse.md` §7 | `observed` |
### Claims that stay `unverified`, and what would move them
- **Device-side mechanisms** — the dump window inside `JNI_OnLoad`, watchdog startup, a protector
intercepting the dump reader. Needs a device and a protected sample: the experiments are the four
already listed under *What would change a label*.
- **The Java encoder mapping** — no JDK on the analysis host. One captured parameter from a real
request settles it; until then the mapping is quoted from the JDK contract.
- **liblz4 versus python-lz4** — no lz4 binding here. The general claim is `observed`; the specific
pair is not.
- **The devirtualization route** — needs a VM binary; nothing on this host can stand in for one.
## Deliberately not absorbed
- **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte
@@ -61,7 +144,8 @@ No new reference files. Each mechanism was folded into the file that already own
## What would change a label
One cheap experiment per row, all on a device this record never had:
One cheap experiment per row, on a device this record never had. The cipher row's host half is
already done (§Verification pass); what remains there is a real body and a real server.
- **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a
disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record
@@ -73,8 +157,9 @@ One cheap experiment per row, all on a device this record never had:
- **Cipher classification:** classify one body with the three probes, then reproduce it and have the
server accept the request.
Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as
support for a claim about a target.
Until one of these is recorded, the matrix rows stay `partial` — `inferred` for the cipher row on the
host evidence, `unverified` for the other two — and none may be cited as support for a claim about a
target.
## Gate status
@@ -0,0 +1,153 @@
#!/usr/bin/env python3
"""Re-run the host-checkable claims absorbed from ref/* on 2026-10-25.
These are the claims that can be settled without a device, a protected sample or a server. Each
check prints what it measured and whether the claim in the reference survived it. The AES probes
need `openssl`; without it they report SKIP rather than passing.
Usage:
python3 docs/tool-verification/fixtures/verify_absorbed_claims.py
"""
import hashlib
import math
import random
import shutil
import subprocess
import sys
import urllib.parse
import zlib
PAGE = 4096
KEY = '000102030405060708090a0b0c0d0e0f'
IV = KEY
results = []
def report(name, ok, detail):
results.append((name, ok))
print(('PASS ' if ok else 'FAIL ') + name)
for line in detail:
print(' ' + line)
def mask_identity():
bad = sum(1 for x in range(256) for m in range(256)
if ((x & ~m) | (~x & m)) & 0xFF != x ^ m)
random.seed(1)
bad16 = 0
for _ in range(200000):
x, m = random.getrandbits(16), random.getrandbits(16)
if ((x & ~m) | (~x & m)) & 0xFFFF != x ^ m:
bad16 += 1
report('mask identity (protocol-reverse section 7, involution)', bad == 0 and bad16 == 0,
['65536 eight-bit (x,m) pairs: %d counterexamples' % bad,
'200000 random 16-bit pairs: %d counterexamples' % bad16])
def padding_length():
bad = [n for n in range(0, 65)
if n + (16 - n % 16) != 16 * math.ceil((n + 1) / 16)]
report('PKCS#7 length law (protocol-reverse section 7, probe 3)', not bad,
['b=16, p=0..64: %d mismatches' % len(bad)])
def zero_fill():
img = bytearray(3 * PAGE)
img[2 * PAGE + 7] = 0xAB
skipped = img[:PAGE] + img[2 * PAGE:]
filled = img[:PAGE] + bytearray(PAGE) + img[2 * PAGE:]
ok = skipped.index(0xAB) != 2 * PAGE + 7 and filled.index(0xAB) == 2 * PAGE + 7
report('unreadable page: skip shifts, zero-fill preserves (linker section 6.3)', ok,
['expected offset %d' % (2 * PAGE + 7),
'skip -> %d' % skipped.index(0xAB),
'zero-fill -> %d' % filled.index(0xAB)])
def hidden_salt():
p, b, t = b'api=search', b'{"q":"x"}', b'1729000000'
salt = bytes.fromhex('00112233445566778899aabbccddeeff')
target = hashlib.md5(p + b + t + salt).hexdigest()
tried = hits = 0
for alg in (hashlib.md5, hashlib.sha1, hashlib.sha256):
for sep in (b'', b'&', b'|'):
for parts in ((p, b, t), (t, b, p), (p, t, b), (b, p, t)):
tried += 1
hits += alg(sep.join(parts)).hexdigest() == target
ok = hits == 0 and hashlib.md5(p + b + t + salt).hexdigest() == target
report('hidden salt defeats a visible-parameter search (signature-derived-keys variant)', ok,
['%d combinations over 3 algorithms and 4 field orders: %d hits' % (tried, hits),
'with the salt included: match'])
def compression_drift():
data = b'{"items":[' + b'{"id":1,"name":"x"},' * 40 + b']}'
a, b = zlib.compress(data, 1), zlib.compress(data, 9)
ok = a != b and zlib.decompress(a) == data and zlib.decompress(b) == data
report('compressed bytes are implementation/parameter defined (protocol-reverse section 7)', ok,
['level 1 = %d bytes, level 9 = %d bytes, identical = %s' % (len(a), len(b), a == b),
'both round-trip to the payload'])
def percent_encoding():
q = urllib.parse.quote('a b~*-_.')
qp = urllib.parse.quote_plus('a b~*-_.')
ok = q == 'a%20b~%2A-_.' and qp == 'a+b~%2A-_.'
report('Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7)', ok,
['quote = ' + q, 'quote_plus = ' + qp,
'Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK)'])
def aes_enc(mode, data):
cmd = ['openssl', 'enc', '-aes-128-' + mode, '-K', KEY, '-iv', IV, '-nopad']
run = subprocess.run(cmd, input=data, capture_output=True)
if run.returncode:
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
return run.stdout
def aes_modes():
if not shutil.which('openssl'):
report('AES mode separation (protocol-reverse section 7, probe 2)', True,
['SKIP: openssl not found'])
return
p1 = b'A' * 32
p2 = b'A' * 32 + b'B' * 16
flipped = bytearray(p1)
flipped[5] ^= 0xFF
flipped = bytes(flipped)
def dirty(a, b):
return sum(1 for i in range(min(len(a), len(b))) if a[i] != b[i])
rows = []
ok = True
for mode in ('ctr', 'cbc', 'ecb'):
c1, c2, cm = aes_enc(mode, p1), aes_enc(mode, p2), aes_enc(mode, flipped)
px = dirty(c1, c2[:len(c1)])
width = dirty(c1, cm)
rows.append('%-3s prefix-xor dirty=%d/32 one-byte-flip changed=%d/32' % (mode, px, width))
if px != 0:
ok = False
ok = ok and rows[0].endswith('1/32') and rows[1].endswith('32/32') and rows[2].endswith('16/32')
report('AES mode separation (protocol-reverse section 7, probe 2)', ok,
rows + ['all three modes show zero prefix-XOR: that probe cannot separate them'])
def main():
print('== host checks for the 2026-10-25 absorption ==')
mask_identity()
padding_length()
zero_fill()
hidden_salt()
compression_drift()
percent_encoding()
aes_modes()
failed = [name for name, ok in results if not ok]
print('')
print('%d check(s), %d failed' % (len(results), len(failed)))
print('RESULT=%s' % ('ok' if not failed else 'failed'))
return 0 if not failed else 1
if __name__ == '__main__':
sys.exit(main())
@@ -996,7 +996,7 @@
"evidence_not_shipped": [
"ref/vm1.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a dual-library commercial shell"
],
"limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The mechanisms are unverified; the ordering judgement (dump on android_dlopen_ext entry, before the protector runs) is inferred from the same source."
"limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The zero-fill/offset consequence is observed on a synthetic three-page image (marker at 8199 zero-filled, 4103 skipped); the window, the reader and the target behaviour are unverified, and the placement rule is inferred from the linker arithmetic."
},
{
"capability": "whitebox-crypto-devirtualization-to-program",
@@ -1019,7 +1019,7 @@
"group": "protocol",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"strength": "inferred",
"benchmark_rows": [],
"evidence": [
"references/protocol-reverse.md",
@@ -1028,7 +1028,7 @@
"evidence_not_shipped": [
"ref/vm3.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a native signer and an encrypted request body"
],
"limits": "Three black-box probes (key-derived header length, prefix XOR for a keystream, the length law), the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Absorbed from one write-up; no cipher was classified here, and no server was tested for whether it compares compressed bytes or only the decompressed payload."
"limits": "Three black-box probes -- header length, output determinism, and one-byte diffusion width, plus the length law -- the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Host-checked on AES-128 through openssl and on synthetic inputs: the probes separate CTR, CBC and ECB (1 / all-following / one block changed per flipped byte), the mask identity has zero counterexamples over 65,536 eight-bit pairs and 200,000 random 16-bit pairs, the length law holds for b=16, and zlib levels 1 and 9 emit different valid streams for one payload. The probe this row originally described (prefix XOR implies a stream cipher) was falsified by that run -- CBC and ECB also show zero prefix-XOR at a fixed IV -- and the reference now carries the corrected discriminator. No target body was classified, and no server was tested for whether it compares compressed bytes or only the decompressed payload."
},
{
"capability": "websocket-long-connection-analysis",
@@ -115,7 +115,8 @@ its own strength note at the top.
trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`.
- **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3
limit, native-side certificate pinning, and — for an opaque body — classifying the primitive by
differential probes (header length, prefix XOR, the length law) before implementing it
differential probes (header length, determinism, diffusion width, the length law) before implementing
it
(`protocol-reverse.md`).
- **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls,
`init_array`-early detection, and the kernel-route map with its version gate
@@ -63,7 +63,7 @@ route was not measured on this repository's evidence**. Treat them as leads with
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
| Native dump window and syscall reader | dump at `android_dlopen_ext` entry when self-decryption and watchdog startup share `JNI_OnLoad`; read via raw syscalls; zero-fill unreadable pages so offsets survive repair — absorbed, nothing run here — `linker-and-soinfo-repair.md` |
| White-box crypto devirtualization | fold a deterministic VM transform into a straight-line `(op, args)` program plus constant table, output map and template, pinned to one build — absorbed, no extract produced — `native-vm-recovery.md` |
| Cipher classification by differential probe | header length, prefix XOR and the length law separate stream from block before implementation; involution wrappers cancel; encoding/compression drift is the last mile — absorbed, never run — `protocol-reverse.md` |
| Cipher classification by differential probe | header length, determinism, one-byte diffusion width and the length law separate CTR/CBC/ECB; involution wrappers cancel; encoding/compression drift is the last mile — probes host-checked on AES-128, no target classified — `protocol-reverse.md` |
## Dependencies this skill does not ship
@@ -262,7 +262,8 @@ about compilation, not as a hidden module.
A process under instrumentation produces signals that are not the target's. ART takes an implicit
`SIGSEGV` for null and bounds checks inside `boot*.oat`, `*.odex` and anonymous JIT pages, and resumes
the thread in its own handler. A deliberate crash, a protector's kill and the runtime's own fault can
all reach your observer looking identical: "signal 11".
all reach your observer looking identical: "signal 11". (`unverified`: documented ART behaviour plus the
source's observation — no trace of it was taken in this record.)
The tuple that discriminates is **(signal, faulting address, faulting module)**, not the signal number.
Count a signal only when the address resolves inside the library you are studying; log the rest and
@@ -273,9 +274,10 @@ Two consequences:
- **Do not install a handler that swallows what the runtime expects to handle.** ART's implicit faults
are how its null checks work; swallowing them turns a healthy process into a mysterious hang. A
handler that intercepts signals has to forward anything whose faulting module is not the target.
(`inferred`.)
- **A `boot*.oat` or `odex` frame in a crash record is the runtime's, not the shell's.** Chasing it
produces "the VM is corrupting memory" when the truth is that the VM's own fault handling ran.
(`unverified` — a 2026-10 write-up on a dual-library shell; the discrimination is general.)
(`inferred`: a consequence of the attribution rule, not a case measured here.)
### Stage 6: the precise stop, and the static alternative to it
@@ -81,9 +81,9 @@ next** — and points at the one section that says "the full record is elsewhere
| Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` |
| React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` |
| Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` |
| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 |
| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Zero-fill offset arithmetic host-checked on a synthetic image; the window itself never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 |
| White-box crypto devirtualization to a program | Absorbed route: symbolic input, constant folding, a straight-line `(op, args)` program plus its constant table, output map and template; version-bound, deterministic functions only. No extract was produced here | partial | unverified | `references/native-vm-recovery.md` §11 |
| Cipher classification by differential probe | Absorbed method: header length, prefix XOR and the length law classify a stream or block primitive before implementation; involutions cancel; Java/Python encoding and compression drift decide the last mile. Never run here | partial | unverified | `references/protocol-reverse.md` §7 |
| Cipher classification by differential probe | Method: header length, determinism, one-byte diffusion width and the length law separate CTR, CBC and ECB (the earlier prefix-XOR-only claim was falsified and is corrected); involution wrappers cancel; Java/Python encoding and compression drift decide the last mile. Probes and algebra host-checked on AES-128 and synthetic inputs; no target body classified | partial | inferred | `references/protocol-reverse.md` §7 |
## Where the full record lives — repository root, not shipped
@@ -202,8 +202,12 @@ is not relocated yet" as expected, and record the bias and relocation tables in
A dump taken *inside* `JNI_OnLoad` is the worst of both: the code is complete, so the dump looks
right, and the watchdog is up, so the process dies during the read. **If the target dies while you
dump and only then, this is the shape — not a broken tool.** (Both rules `unverified`: absorbed from
one write-up, not reproduced here.)
dump and only then, this is the shape — not a broken tool.**
**Claim strength.** The window itself — SMC completing *and* watchdogs starting inside the same
`JNI_OnLoad` — is `unverified`: one write-up reports it and this record did not reproduce it. The
placement rule (dump in `onEnter`, keep the bias) is `inferred` from the arithmetic in §2 and §4. The
symptom is the source's report, also `unverified`.
### 6.3 Read with syscalls, and pad the holes
@@ -213,15 +217,19 @@ Two properties of the reader decide whether the dump is usable.
libc's `open`/`read`/`write` sees your dump as one more caller and can refuse or terminate. A
raw-syscall reader (`svc #0` from a small stub, or a kernel-side reader) sits outside that
surface. This is `advanced-unpacking.md` §"Layered descent" applied to your own tooling: the layer
you read from is also the layer the target can watch.
you read from is also the layer the target can watch. (`unverified` for the protector-hook claim;
the layer rule it extends is `measured` there.)
- **An unreadable page is a hole to fill, not a range to skip.** Skipping shifts every later byte
and produces a file that repairs into a plausible ELF with missing logic. Zero-filling preserves
offsets and sizes, so segment placement still works; record which ranges were unreadable so the
repair can be audited. A dump is evidence, and "these ranges were unreadable" belongs in it.
(`observed`, host: in a synthetic three-page image with one unreadable page, a marker meant for
offset 8199 lands at 8199 when the page is zero-filled and at 4103 when it is skipped — see the
EXTENSION record.)
Then repair the image with a tool that rebuilds what the loader would have written (program headers,
dynamic table, section headers). The repair consumes a file whose *offsets* are right; that is what
the zero-fill protects.
the zero-fill protects. (`inferred`: the offset/size argument above, not a repair run here.)
## 7. Timing: constructors and one-shot registrations
@@ -179,6 +179,9 @@ Two conditions, both required:
once it always returns clean. Enumerate the callers before you touch it — the xref list in the
disassembler, or the same caller enumeration `scripts/find_refs.py` does for dex methods.
(`unverified` that a given target funnels every detector through one predicate — that is the
source's finding. The decision-versus-action rule is `inferred` from it.)
### Grade the sites by what being wrong costs
| Tier | What it is | How to patch it | Why this order |
@@ -192,6 +195,10 @@ non-zero, a "list of hits" scanner returns NULL, a "should continue" predicate r
0 everywhere is a guess that happens to work half the time — and the failing half reads as a broken
patch instead of a wrong constant.
(`inferred`: the order follows from observability — a death at tier 0 hides every other result —
and the benign-value rule from the caller's branch. The tier vocabulary is the source's,
`unverified`; no graded set was run here.)
### Count the watchdogs before declaring the set closed
One kill site is rarely the only thread. The threads are distinguishable by what they *read*, not by
@@ -205,11 +212,14 @@ Patch one and the app still dies, at a different delay, which reads as "the patc
than "there were three". After the tier 0/1 set, the process must survive longer than the **longest**
timer you have observed, not the first one.
(`unverified` for the probe inventory — source-reported for one shell; `inferred` for "identify a
thread by what it reads", which follows from the probe classes themselves.)
### One tier per run
Patch tier 0, run, record; then tier 1; then tier 2 only if needed. A build with all tiers changed at once cannot
tell you which one mattered, and the next failure is unattributable. This is the single-variable rule
of `pitfalls.md` P21 applied to a patch set.
of `pitfalls.md` P21 applied to a patch set. (`inferred`.)
## ELF hardening you will meet in these targets
@@ -11,9 +11,10 @@ half lives in `references/ete-diff-tracing.md`, the **slicing** tool in
write-ups (Kanxue, 2026-08/09, titles and authors anonymized in this record; one is a full
arm64 custom-VM recovery, the other a VMP interpreter recovery with trace-driven IR lifting).
The write-ups are `observed` by their authors; this repository has **never run any of these
steps on a real VM target** — everything below is `inferred` from those write-ups, cross-read
against the open xvmp source facts in `references/ete-diff-tracing.md` §5. The pipeline's
host-tested pieces (trace parsing, slicing, diffing) carry their own labels where named.
steps on a real VM target**, so the mechanics below are `unverified` here — reported, not
reproduced — and the ordering judgements built on them are `inferred`, cross-read against the
open xvmp source facts in `references/ete-diff-tracing.md` §5. The pipeline's host-tested pieces
(trace parsing, slicing, diffing) carry their own labels where named.
## 1. Deciding it is a VM at all
@@ -175,7 +176,8 @@ falls out. That is the end state this file walks toward: the guest program as da
Sections 1–10 aim at a guest program you can **read**. When the interpreter implements one
deterministic function — a request signer, a decoder, a checksum — the cheaper deliverable is a
program you can **run**, and the handler semantics never need to be understood.
program you can **run**, and the handler semantics never need to be understood. (`unverified`: one
write-up reports the route; this record did not run it.)
1. **Fix the boundary.** One native entry that takes a buffer in and produces a buffer out.
Everything inside is the VM; everything outside is protocol.
@@ -200,16 +202,18 @@ program you can **run**, and the handler semantics never need to be understood.
**Version binding.** The program is a property of one build. Store the library's fingerprint (size and
build id) beside it and re-extract on upgrade: the tables and the map move even when the algorithm
does not. Say which build a program came from; an extract with no fingerprint is not reproducible.
(`inferred` for the fingerprint discipline; `unverified` for the source's report that the tables move
on update.)
**Validation is per node, not per output.** Compare node values at several checkpoints against a
device run. A node that is right in value but wrong in width (32-bit truncation where the VM kept 64)
passes the final comparison on some inputs and fails on others — §9's failure seen from the other
side.
side. (`inferred` from §9.)
**Boundary.** This route needs a deterministic, input-driven computation. If the VM folds in a device
id, a timestamp or a nonce, those become constants from one trace, and the honest claim is
"reproduces one device's answers", not "implements the algorithm". For a signer that is often exactly
enough, but the report has to say which claim it is making. (`unverified` — absorbed, not run.)
enough, but the report has to say which claim it is making.
## 12. Where this repository's tools plug in
@@ -233,5 +237,5 @@ enough, but the report has to say which claim it is making. (`unverified` — ab
- [ ] segments stacked if entry can nest (§7); slicer's nested-loop diagnostic honoured
- [ ] operand slicing seeded on all STOREs; byte provenance (not value) decides folding
- [ ] clean re-VM runs real bytecode against trace ground truth before any IR is trusted
- [ ] labels: write-up methods `inferred`, shipped tools `observed` (host), no VM target run here
- [ ] labels: write-up mechanisms `unverified`, ordering judgements `inferred`, shipped tools `observed` (host), no VM target run here
- [ ] if the deliverable is a callable signer/decoder: §11's four artifacts captured as a set, program pinned to a build fingerprint
+5 -2
View File
@@ -1493,7 +1493,8 @@ the defect lives in memory, where you cannot diff it the way you diff two files.
**Prove each stage's input is byte-identical — length, alignment, encoding — before touching the
algorithm.** Compare stage inputs against the device, not final outputs; a stage-input diff localises
in minutes what an output diff cannot localise at all. P42 is the decoder-desync cousin; this one is
an extraction length.
an extraction length. (`inferred` — a reasoning rule from how one byte propagates; no target case was
measured for this record.)
## P50. A "custom cipher" is a wrapper that cancels
@@ -1517,4 +1518,6 @@ step.
**Compose it with itself before reimplementing it** (`f(f(x)) == x`), and check whether the caller
invokes it on both paths. If it cancels, drop it and recover the standard primitive underneath; keep
the wrapper in the report, because it is the reason the algorithm looked unfamiliar. The method, with
the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7.
the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7; the
mask identity itself is host-checked there (`observed`: zero counterexamples over 65,536 eight-bit
pairs and 200,000 random 16-bit pairs).
@@ -374,23 +374,36 @@ missing family first.
**Load this when** the body is opaque: a long base64/hex field in the query, a binary request body, a
fixed prefix on every response — and the transform lives in a native library or a helper class.
Before any static work, three black-box experiments on captured pairs, each a few minutes. You need
several requests whose plaintext you control.
Three black-box experiments on captured pairs, before any static work. The probes themselves were
host-checked here on AES-128 through `openssl` (commands and output in the verification record,
`docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md`, under "Verification pass"). What they prove
about *your* target is one step further, and stays `inferred` until you vary the key and the
request.
1. **Header length.** Encrypt two different payloads under the same key and decode both. A constant
first N bytes ⇒ a key-derived header. A prefix that changes when the key changes ⇒ keyed, not
random. A prefix that changes per request ⇒ a per-message IV or counter.
2. **Prefix XOR.** Encrypt `P` and `P || X`, then XOR the two ciphertexts. Zeros over the common
prefix ⇒ a stream cipher whose keystream depends on the key alone. No such relation, plus a length
rounded up to a block size ⇒ a block cipher. Identical 16-byte blocks recurring across messages ⇒
ECB, which is a finding on its own.
3. **Length law.** `len(c) = len(p) + k` ⇒ stream/CTR. `len(c) = ceil((len(p)+1)/b) * b` ⇒ block with
padding. A constant relation with a compressed prefix ⇒ compress-then-encrypt, and the plaintext
you are looking for is the compressor's output, not the JSON.
1. **Header length.** Encrypt two payloads under the same key and decode both. A constant first N
bytes ⇒ a deterministic header: fixed plaintext, or derived from the key/IV. A prefix that changes
when the key changes ⇒ keyed. A prefix that changes per request ⇒ a per-message nonce.
(`observed` on synthetic inputs.)
2. **Determinism, then diffusion width.** Encrypt `P` and `P || X`, and XOR the ciphertexts. Zeros
over the common prefix ⇒ the construction is deterministic under a fixed key and IV — **and that
is all it proves.** Measured: CTR, CBC and ECB all give zero prefix-XOR at a fixed IV, so this
experiment cannot separate a stream cipher from a block mode. (An earlier draft of this file said
it could; one run falsified it.) Repeat with a different IV: a CTR-style mode diverges, and so does
any mode that carries a per-message nonce.
The classification decides what you implement: a stream cipher needs its keystream and nothing else;
a block cipher needs its mode; ECB needs no key schedule at all to explain structure. Skipping this
step is how a standard construction gets reimplemented as something exotic.
The separation comes from a one-byte plaintext flip, counted over the ciphertext: **1 changed
byte** ⇒ a keystream independent of the plaintext (measured: CTR, 1 of 32); **every block from
that point on** ⇒ a chained mode (measured: CBC, 32 of 32); **exactly one block** ⇒ ECB (measured:
16 of 32, later blocks untouched). (`observed`, host.)
3. **Length law.** `len(c) = len(p)` ⇒ no padding — which fits CTR and a `nopad` block mode equally,
so it never decides alone. `len(c) = b * ceil((len(p)+1)/b)` ⇒ a padded block mode (PKCS#7;
checked for b = 16, p = 0…32). A compressed prefix with a constant length relation ⇒
compress-then-encrypt, and the plaintext you are looking for is the compressor's output.
(`observed`, host.)
The classification decides what you implement: a keystream cipher needs its keystream and nothing
else; a chained mode needs the mode and the IV; ECB needs neither to explain its structure. Skipping
this step is how a standard construction gets reimplemented as something exotic.
### A custom-looking transform may be a wrapper that cancels
@@ -398,10 +411,12 @@ Wrapper arithmetic is cheaper to check than to rewrite. Two shapes cover most of
- **Involution applied symmetrically.** A transform with `f(f(x)) == x` that is called on the way in
*and* on the way out contributes nothing. The common disguise is a mask pair:
`(x & ~m) | (~x & m)` is exactly `x ^ m` when the bits of `m` partition the word. If both encrypt
and decrypt call it, drop it from your model and recover the standard primitive underneath.
- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is an
encoding, not encryption. Index it out and compare against known S-boxes.
`(x & ~m) | (~x & m)` is exactly `x ^ m`. Exhaustively checked here over all 8-bit `(x, m)` pairs
and 200,000 random 16-bit pairs: zero counterexamples, and the composition is an involution. If
both encrypt and decrypt call it, drop it and recover the standard primitive underneath.
(`observed`, host — the algebra; whether a target's wrapper is that shape is `inferred`.)
- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is
an encoding, not encryption. Index it out and compare against known S-boxes. (`inferred`.)
Test the composition before reimplementing: apply the function twice, and check whether the caller
uses it twice. If it cancels, the algorithm is standard and the reason it looked unfamiliar is now
@@ -413,17 +428,22 @@ Two non-cryptographic differences fail signatures that are otherwise correct. Bo
the algorithm and both are cheap to eliminate:
- **The client's encoder may be Java's.** `URLEncoder` writes a space as `+`, leaves `*`, `-`, `_`,
`.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's `quote` does the
opposite on `~` and uses `%20`. Reimplement what the client does and check it against one captured
parameter value, not against documentation.
- **Compression bytes are implementation-defined.** Two libraries can emit different, equally valid
compressed output for the same input. Whether that breaks you is a property of the **server**: if it
decompresses and checks a CRC or the plaintext, byte equality is not required; if it compares the
stored blob, it is. One mutation tells you which, and the answer decides whether byte-perfect
reproduction is even the goal.
`.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's does the
opposite on two of those: measured on this host, `quote` gives `a%20b~%2A-_.` (space `%20`, `*`
encoded, `~` kept) and `quote_plus` gives `a+b~%2A-_.` (space `+`, `*` still encoded). Reimplement
what the client does and check it against one captured parameter, not against documentation.
(Python side `observed`; the Java side is `unverified` here — no JDK on the analysis host — and is
quoted from the JDK API contract, so confirm it on one captured value.)
- **Compressed bytes are implementation-defined.** Measured: `zlib.compress(data, 1)` and `(data, 9)`
produce different byte streams (49 vs 48 bytes) for the same payload, and both decompress to it.
Whether the difference breaks a signature is a property of the **server**: if it decompresses and
checks a CRC or the plaintext, byte equality is not required; if it compares the stored blob, it
is. One mutation tells you which. (The general claim is `observed`; the specific liblz4-vs-python-
lz4 divergence is `unverified` — no lz4 binding here.)
**Boundary.** None of the three experiments needs the `.so`; that is the point. When the keystream is
derived per message from a device secret, the native route comes back and §5's attribution applies.
(`inferred`.)
## 8. Failure modes
@@ -146,12 +146,17 @@ it also fails, the problem is environmental (`references/tls-and-cert.md`,
A different failure with the same source. A signer is reconstructed from static analysis and produces
a signature the server refuses, while every ingredient visible in the request looks right. The formula
turns out to be `sig = prefix + H(input || salt || key)`, and `salt` never appears in the request — it
is generated at init from the certificate and cached in a global.
is generated at init from the certificate and cached in a global. (The certificate origin is
`unverified` here; it is the same mechanism as the key case above, which is documented and likewise
unmeasured.)
**Why it defeats black-box search: the search space is missing an argument.** Trying MD5/HMAC/SHA
combinations over the visible parameters cannot find a value that is concatenated inside the digest and
never transmitted. The symptom is *uniform* failure, not near-misses — that is the tell, and it is
also why this is misread as "the algorithm is not fully recovered".
also why this is misread as "the algorithm is not fully recovered". (`observed`, host, synthetic: a
36-combination search over three algorithms and four field orders returns zero hits against a digest
with a hidden 16-byte salt, and returns the target the moment the salt is included — see the
EXTENSION record.)
How to get it:
@@ -160,15 +165,13 @@ How to get it:
- **Or hook the digest update** and dump its argument on one request; the salt is the constant run
inside an otherwise changing buffer.
- **Verify with three captured triples** that vary one field each. A formula that matches all three is
the formula; one that matches one is a coincidence.
the formula; one that matches one is a coincidence. (`inferred`.)
Consequence for the deliverable: the salt is per build and per certificate. A local signer carries the
captured constant plus a build fingerprint (the same discipline as an extracted VM program,
`native-vm-recovery.md` §11), and the report says which build it came from. On update the value
changes and the signer must be re-derived — do not present the constant as an algorithm.
(`unverified`: the certificate-derived cache is `inferred` from the same mechanism as the key case
above; nothing here was run against a target.)
(`inferred` from the certificate binding, not measured.)
## Cost of getting this wrong