From 068280a18014db25e34177fded73f6bc0bac0bf0 Mon Sep 17 00:00:00 2001 From: root Date: Sun, 25 Oct 2026 12:19:37 +0800 Subject: [PATCH] Verify the absorbed claims on the host and label them per claim Adds docs/tool-verification/fixtures/verify_absorbed_claims.py: seven device-free checks covering the mask identity, the PKCS#7 length law, zero-fill versus skipped offsets, the hidden-salt search, compression drift, Python percent-encoding, and AES CTR/CBC/ECB separation. The AES run falsified an absorbed claim: zero prefix-XOR does not imply a stream cipher (CBC and ECB show it too). The discriminator is a one-byte flip -- 1 changed byte for CTR, all following blocks for CBC, one block for ECB -- and protocol-reverse section 7 now states that, with the capability row recording the correction. Labels are now per claim rather than per section: observed where a host command produced output, inferred for rules derived from the mechanisms, unverified for anything needing a device, a JDK, a lz4 binding or a VM sample. native-vm-recovery's claim-strength paragraph is aligned with the repository definition, and the cipher row moves to inferred. Gates and pytest green. --- .../EXTENSION-ref-absorption-2026-10b.md | 107 ++++++++++-- .../fixtures/verify_absorbed_claims.py | 153 ++++++++++++++++++ .../evidence/capability-matrix.json | 6 +- .../references/coverage-and-limits.md | 3 +- .../references/coverage/route-inventory.md | 2 +- .../references/detection-and-anti-analysis.md | 6 +- .../references/evidence-summary.md | 4 +- .../references/linker-and-soinfo-repair.md | 16 +- .../references/native-tamper-and-suicide.md | 12 +- .../references/native-vm-recovery.md | 18 ++- skills/apk-reverse/references/pitfalls.md | 7 +- .../references/protocol-reverse.md | 74 +++++---- .../references/signature-derived-keys.md | 15 +- 13 files changed, 356 insertions(+), 67 deletions(-) create mode 100644 docs/tool-verification/fixtures/verify_absorbed_claims.py diff --git a/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md b/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md index 5c459b6..2ed1e79 100644 --- a/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md +++ b/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md @@ -1,8 +1,9 @@ # EXTENSION — second reference absorption from `ref/*` (2026-10-25) This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into -the files that already own the topics. It did not run any of them. The record exists so a later reader -can tell what changed and on what footing. +the files that already own the topics. None was run end to end; a host-level pass (§Verification pass) +settles the claims that need no device. The record exists so a later reader can tell what changed and +on what footing. ## Sources @@ -16,8 +17,9 @@ per-build address are omitted; none was copied onto the tracked surface. | `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift | All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing -in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering -judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was +in them was reproduced here. Mechanisms that need a target are therefore `unverified`, the ordering +judgements built on them are `inferred`, and the host-checkable ones are settled in §Verification +pass. No text, code, key material, offset table or screenshot was copied. ## What was added @@ -30,15 +32,16 @@ No new reference files. Each mechanism was folded into the file that already own | `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker | | `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals | | `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections | -| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required | +| `references/protocol-reverse.md` §7 | three black-box probes (header length, determinism, one-byte diffusion width, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile. The probe's original prefix-XOR-only claim was falsified by the host run below and the section now states the corrected discriminator | claim any cipher was classified; assume byte-identical compression is required | | `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm | | `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method | ## Registration surface updated -- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`, - `whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all - `partial` / `route_measured: false` / `unverified`; two existing rows +- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader` and + `whitebox-crypto-devirtualization-to-program` are `partial` / `route_measured: false` / `unverified`; + `cipher-classification-by-differential-probe` is `partial` / `route_measured: false` / `inferred` + after the host checks below; two existing rows (`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended rather than duplicated. - `references/evidence-summary.md`: three rows. @@ -47,6 +50,86 @@ No new reference files. Each mechanism was folded into the file that already own - `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the symptom mirror and the reference/script inventories had nothing to learn. +## Verification pass — host, 2026-10-25 + +Purpose: separate what the write-ups report from what this host can show, and label per claim rather +than per section. Three tiers were applied: `observed` (a command ran here and its output is below), +`inferred` (follows from a mechanism or a neighbouring measurement), `unverified` (reported by +someone else, not reproduced). No device, protected sample or server was involved, so **no claim +about a target was upgraded**. + +```bash +python3 docs/tool-verification/fixtures/verify_absorbed_claims.py +``` + +```text +PASS mask identity (protocol-reverse section 7, involution) + 65536 eight-bit (x,m) pairs: 0 counterexamples + 200000 random 16-bit pairs: 0 counterexamples +PASS PKCS#7 length law (protocol-reverse section 7, probe 3) + b=16, p=0..64: 0 mismatches +PASS unreadable page: skip shifts, zero-fill preserves (linker section 6.3) + expected offset 8199 + skip -> 4103 + zero-fill -> 8199 +PASS hidden salt defeats a visible-parameter search (signature-derived-keys variant) + 36 combinations over 3 algorithms and 4 field orders: 0 hits + with the salt included: match +PASS compressed bytes are implementation/parameter defined (protocol-reverse section 7) + level 1 = 49 bytes, level 9 = 48 bytes, identical = False + both round-trip to the payload +PASS Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7) + quote = a%20b~%2A-_. + quote_plus = a+b~%2A-_. + Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK) +PASS AES mode separation (protocol-reverse section 7, probe 2) + ctr prefix-xor dirty=0/32 one-byte-flip changed=1/32 + cbc prefix-xor dirty=0/32 one-byte-flip changed=32/32 + ecb prefix-xor dirty=0/32 one-byte-flip changed=16/32 + all three modes show zero prefix-XOR: that probe cannot separate them + +7 check(s), 0 failed +RESULT=ok +``` + +### What the run falsified + +The absorbed probe said *zeros over the common prefix ⇒ a stream cipher*. Measured on AES-128 with a +fixed IV, **CTR, CBC and ECB all produce zero prefix-XOR** (0 of 32 bytes differ): any deterministic +mode reproduces the ciphertext of a prefix, so the experiment only proves the absence of a +per-message nonce. The discriminator that does work is a one-byte plaintext flip: CTR changes exactly +1 byte, CBC changes every block from the flipped one onward (32 of 32 here), ECB changes exactly one +block (16 of 32, later blocks untouched). The reference now states the corrected procedure; the +capability row records the correction instead of the original claim. + +**A label convention was corrected.** `native-vm-recovery.md`'s claim-strength paragraph called the +absorbed mechanics `inferred`. Under the repository's definition (`unverified` = reported by someone +else, not reproduced) the mechanics are `unverified`, and only the ordering judgements are +`inferred`. The paragraph and the checklist now say that; no capability row changed for it. + +### Claims the run settles (host only) + +| Claim | Where | Now | +|---|---|---| +| `(x & ~m) \| (~x & m) == x ^ m`, and the composition is an involution | `protocol-reverse.md` §7 | `observed` | +| PKCS#7 padded length is `b * ceil((len+1)/b)` | `protocol-reverse.md` §7 | `observed` | +| Zero-filling an unreadable page preserves later offsets; skipping shifts them | `linker-and-soinfo-repair.md` §6.3 | `observed` | +| An unknown salt inside the digest defeats a search over visible parameters | `signature-derived-keys.md` §Variant | `observed` | +| Two compressors/parameters can emit different valid bytes for one payload | `protocol-reverse.md` §7 | `observed` (general) | +| Python's encoder treats ` `, `*`, `~` differently from Java's | `protocol-reverse.md` §7 | `observed` (Python side) | +| CTR / CBC / ECB are separable by diffusion width, not by prefix-XOR | `protocol-reverse.md` §7 | `observed` | + +### Claims that stay `unverified`, and what would move them + +- **Device-side mechanisms** — the dump window inside `JNI_OnLoad`, watchdog startup, a protector + intercepting the dump reader. Needs a device and a protected sample: the experiments are the four + already listed under *What would change a label*. +- **The Java encoder mapping** — no JDK on the analysis host. One captured parameter from a real + request settles it; until then the mapping is quoted from the JDK contract. +- **liblz4 versus python-lz4** — no lz4 binding here. The general claim is `observed`; the specific + pair is not. +- **The devirtualization route** — needs a VM binary; nothing on this host can stand in for one. + ## Deliberately not absorbed - **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte @@ -61,7 +144,8 @@ No new reference files. Each mechanism was folded into the file that already own ## What would change a label -One cheap experiment per row, all on a device this record never had: +One cheap experiment per row, on a device this record never had. The cipher row's host half is +already done (§Verification pass); what remains there is a real body and a real server. - **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record @@ -73,8 +157,9 @@ One cheap experiment per row, all on a device this record never had: - **Cipher classification:** classify one body with the three probes, then reproduce it and have the server accept the request. -Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as -support for a claim about a target. +Until one of these is recorded, the matrix rows stay `partial` — `inferred` for the cipher row on the +host evidence, `unverified` for the other two — and none may be cited as support for a claim about a +target. ## Gate status diff --git a/docs/tool-verification/fixtures/verify_absorbed_claims.py b/docs/tool-verification/fixtures/verify_absorbed_claims.py new file mode 100644 index 0000000..d91b840 --- /dev/null +++ b/docs/tool-verification/fixtures/verify_absorbed_claims.py @@ -0,0 +1,153 @@ +#!/usr/bin/env python3 +"""Re-run the host-checkable claims absorbed from ref/* on 2026-10-25. + +These are the claims that can be settled without a device, a protected sample or a server. Each +check prints what it measured and whether the claim in the reference survived it. The AES probes +need `openssl`; without it they report SKIP rather than passing. + +Usage: + python3 docs/tool-verification/fixtures/verify_absorbed_claims.py +""" +import hashlib +import math +import random +import shutil +import subprocess +import sys +import urllib.parse +import zlib + +PAGE = 4096 +KEY = '000102030405060708090a0b0c0d0e0f' +IV = KEY +results = [] + + +def report(name, ok, detail): + results.append((name, ok)) + print(('PASS ' if ok else 'FAIL ') + name) + for line in detail: + print(' ' + line) + + +def mask_identity(): + bad = sum(1 for x in range(256) for m in range(256) + if ((x & ~m) | (~x & m)) & 0xFF != x ^ m) + random.seed(1) + bad16 = 0 + for _ in range(200000): + x, m = random.getrandbits(16), random.getrandbits(16) + if ((x & ~m) | (~x & m)) & 0xFFFF != x ^ m: + bad16 += 1 + report('mask identity (protocol-reverse section 7, involution)', bad == 0 and bad16 == 0, + ['65536 eight-bit (x,m) pairs: %d counterexamples' % bad, + '200000 random 16-bit pairs: %d counterexamples' % bad16]) + + +def padding_length(): + bad = [n for n in range(0, 65) + if n + (16 - n % 16) != 16 * math.ceil((n + 1) / 16)] + report('PKCS#7 length law (protocol-reverse section 7, probe 3)', not bad, + ['b=16, p=0..64: %d mismatches' % len(bad)]) + + +def zero_fill(): + img = bytearray(3 * PAGE) + img[2 * PAGE + 7] = 0xAB + skipped = img[:PAGE] + img[2 * PAGE:] + filled = img[:PAGE] + bytearray(PAGE) + img[2 * PAGE:] + ok = skipped.index(0xAB) != 2 * PAGE + 7 and filled.index(0xAB) == 2 * PAGE + 7 + report('unreadable page: skip shifts, zero-fill preserves (linker section 6.3)', ok, + ['expected offset %d' % (2 * PAGE + 7), + 'skip -> %d' % skipped.index(0xAB), + 'zero-fill -> %d' % filled.index(0xAB)]) + + +def hidden_salt(): + p, b, t = b'api=search', b'{"q":"x"}', b'1729000000' + salt = bytes.fromhex('00112233445566778899aabbccddeeff') + target = hashlib.md5(p + b + t + salt).hexdigest() + tried = hits = 0 + for alg in (hashlib.md5, hashlib.sha1, hashlib.sha256): + for sep in (b'', b'&', b'|'): + for parts in ((p, b, t), (t, b, p), (p, t, b), (b, p, t)): + tried += 1 + hits += alg(sep.join(parts)).hexdigest() == target + ok = hits == 0 and hashlib.md5(p + b + t + salt).hexdigest() == target + report('hidden salt defeats a visible-parameter search (signature-derived-keys variant)', ok, + ['%d combinations over 3 algorithms and 4 field orders: %d hits' % (tried, hits), + 'with the salt included: match']) + + +def compression_drift(): + data = b'{"items":[' + b'{"id":1,"name":"x"},' * 40 + b']}' + a, b = zlib.compress(data, 1), zlib.compress(data, 9) + ok = a != b and zlib.decompress(a) == data and zlib.decompress(b) == data + report('compressed bytes are implementation/parameter defined (protocol-reverse section 7)', ok, + ['level 1 = %d bytes, level 9 = %d bytes, identical = %s' % (len(a), len(b), a == b), + 'both round-trip to the payload']) + + +def percent_encoding(): + q = urllib.parse.quote('a b~*-_.') + qp = urllib.parse.quote_plus('a b~*-_.') + ok = q == 'a%20b~%2A-_.' and qp == 'a+b~%2A-_.' + report('Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7)', ok, + ['quote = ' + q, 'quote_plus = ' + qp, + 'Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK)']) + + +def aes_enc(mode, data): + cmd = ['openssl', 'enc', '-aes-128-' + mode, '-K', KEY, '-iv', IV, '-nopad'] + run = subprocess.run(cmd, input=data, capture_output=True) + if run.returncode: + raise RuntimeError(run.stderr.decode('utf-8', 'replace')) + return run.stdout + + +def aes_modes(): + if not shutil.which('openssl'): + report('AES mode separation (protocol-reverse section 7, probe 2)', True, + ['SKIP: openssl not found']) + return + p1 = b'A' * 32 + p2 = b'A' * 32 + b'B' * 16 + flipped = bytearray(p1) + flipped[5] ^= 0xFF + flipped = bytes(flipped) + + def dirty(a, b): + return sum(1 for i in range(min(len(a), len(b))) if a[i] != b[i]) + + rows = [] + ok = True + for mode in ('ctr', 'cbc', 'ecb'): + c1, c2, cm = aes_enc(mode, p1), aes_enc(mode, p2), aes_enc(mode, flipped) + px = dirty(c1, c2[:len(c1)]) + width = dirty(c1, cm) + rows.append('%-3s prefix-xor dirty=%d/32 one-byte-flip changed=%d/32' % (mode, px, width)) + if px != 0: + ok = False + ok = ok and rows[0].endswith('1/32') and rows[1].endswith('32/32') and rows[2].endswith('16/32') + report('AES mode separation (protocol-reverse section 7, probe 2)', ok, + rows + ['all three modes show zero prefix-XOR: that probe cannot separate them']) + + +def main(): + print('== host checks for the 2026-10-25 absorption ==') + mask_identity() + padding_length() + zero_fill() + hidden_salt() + compression_drift() + percent_encoding() + aes_modes() + failed = [name for name, ok in results if not ok] + print('') + print('%d check(s), %d failed' % (len(results), len(failed))) + print('RESULT=%s' % ('ok' if not failed else 'failed')) + return 0 if not failed else 1 + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/skills/apk-reverse/evidence/capability-matrix.json b/skills/apk-reverse/evidence/capability-matrix.json index c596869..b5a6005 100644 --- a/skills/apk-reverse/evidence/capability-matrix.json +++ b/skills/apk-reverse/evidence/capability-matrix.json @@ -996,7 +996,7 @@ "evidence_not_shipped": [ "ref/vm1.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a dual-library commercial shell" ], - "limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The mechanisms are unverified; the ordering judgement (dump on android_dlopen_ext entry, before the protector runs) is inferred from the same source." + "limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The zero-fill/offset consequence is observed on a synthetic three-page image (marker at 8199 zero-filled, 4103 skipped); the window, the reader and the target behaviour are unverified, and the placement rule is inferred from the linker arithmetic." }, { "capability": "whitebox-crypto-devirtualization-to-program", @@ -1019,7 +1019,7 @@ "group": "protocol", "status": "partial", "route_measured": false, - "strength": "unverified", + "strength": "inferred", "benchmark_rows": [], "evidence": [ "references/protocol-reverse.md", @@ -1028,7 +1028,7 @@ "evidence_not_shipped": [ "ref/vm3.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a native signer and an encrypted request body" ], - "limits": "Three black-box probes (key-derived header length, prefix XOR for a keystream, the length law), the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Absorbed from one write-up; no cipher was classified here, and no server was tested for whether it compares compressed bytes or only the decompressed payload." + "limits": "Three black-box probes -- header length, output determinism, and one-byte diffusion width, plus the length law -- the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Host-checked on AES-128 through openssl and on synthetic inputs: the probes separate CTR, CBC and ECB (1 / all-following / one block changed per flipped byte), the mask identity has zero counterexamples over 65,536 eight-bit pairs and 200,000 random 16-bit pairs, the length law holds for b=16, and zlib levels 1 and 9 emit different valid streams for one payload. The probe this row originally described (prefix XOR implies a stream cipher) was falsified by that run -- CBC and ECB also show zero prefix-XOR at a fixed IV -- and the reference now carries the corrected discriminator. No target body was classified, and no server was tested for whether it compares compressed bytes or only the decompressed payload." }, { "capability": "websocket-long-connection-analysis", diff --git a/skills/apk-reverse/references/coverage-and-limits.md b/skills/apk-reverse/references/coverage-and-limits.md index c7716db..e4a189b 100644 --- a/skills/apk-reverse/references/coverage-and-limits.md +++ b/skills/apk-reverse/references/coverage-and-limits.md @@ -115,7 +115,8 @@ its own strength note at the top. trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`. - **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3 limit, native-side certificate pinning, and — for an opaque body — classifying the primitive by - differential probes (header length, prefix XOR, the length law) before implementing it + differential probes (header length, determinism, diffusion width, the length law) before implementing + it (`protocol-reverse.md`). - **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls, `init_array`-early detection, and the kernel-route map with its version gate diff --git a/skills/apk-reverse/references/coverage/route-inventory.md b/skills/apk-reverse/references/coverage/route-inventory.md index 2343eab..38ff2c2 100644 --- a/skills/apk-reverse/references/coverage/route-inventory.md +++ b/skills/apk-reverse/references/coverage/route-inventory.md @@ -63,7 +63,7 @@ route was not measured on this repository's evidence**. Treat them as leads with | Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` | | Native dump window and syscall reader | dump at `android_dlopen_ext` entry when self-decryption and watchdog startup share `JNI_OnLoad`; read via raw syscalls; zero-fill unreadable pages so offsets survive repair — absorbed, nothing run here — `linker-and-soinfo-repair.md` | | White-box crypto devirtualization | fold a deterministic VM transform into a straight-line `(op, args)` program plus constant table, output map and template, pinned to one build — absorbed, no extract produced — `native-vm-recovery.md` | -| Cipher classification by differential probe | header length, prefix XOR and the length law separate stream from block before implementation; involution wrappers cancel; encoding/compression drift is the last mile — absorbed, never run — `protocol-reverse.md` | +| Cipher classification by differential probe | header length, determinism, one-byte diffusion width and the length law separate CTR/CBC/ECB; involution wrappers cancel; encoding/compression drift is the last mile — probes host-checked on AES-128, no target classified — `protocol-reverse.md` | ## Dependencies this skill does not ship diff --git a/skills/apk-reverse/references/detection-and-anti-analysis.md b/skills/apk-reverse/references/detection-and-anti-analysis.md index a69e994..c2e8e11 100644 --- a/skills/apk-reverse/references/detection-and-anti-analysis.md +++ b/skills/apk-reverse/references/detection-and-anti-analysis.md @@ -262,7 +262,8 @@ about compilation, not as a hidden module. A process under instrumentation produces signals that are not the target's. ART takes an implicit `SIGSEGV` for null and bounds checks inside `boot*.oat`, `*.odex` and anonymous JIT pages, and resumes the thread in its own handler. A deliberate crash, a protector's kill and the runtime's own fault can -all reach your observer looking identical: "signal 11". +all reach your observer looking identical: "signal 11". (`unverified`: documented ART behaviour plus the +source's observation — no trace of it was taken in this record.) The tuple that discriminates is **(signal, faulting address, faulting module)**, not the signal number. Count a signal only when the address resolves inside the library you are studying; log the rest and @@ -273,9 +274,10 @@ Two consequences: - **Do not install a handler that swallows what the runtime expects to handle.** ART's implicit faults are how its null checks work; swallowing them turns a healthy process into a mysterious hang. A handler that intercepts signals has to forward anything whose faulting module is not the target. + (`inferred`.) - **A `boot*.oat` or `odex` frame in a crash record is the runtime's, not the shell's.** Chasing it produces "the VM is corrupting memory" when the truth is that the VM's own fault handling ran. - (`unverified` — a 2026-10 write-up on a dual-library shell; the discrimination is general.) + (`inferred`: a consequence of the attribution rule, not a case measured here.) ### Stage 6: the precise stop, and the static alternative to it diff --git a/skills/apk-reverse/references/evidence-summary.md b/skills/apk-reverse/references/evidence-summary.md index 23ef3c9..c187568 100644 --- a/skills/apk-reverse/references/evidence-summary.md +++ b/skills/apk-reverse/references/evidence-summary.md @@ -81,9 +81,9 @@ next** — and points at the one section that says "the full record is elsewhere | Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` | | React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` | | Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` | -| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 | +| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Zero-fill offset arithmetic host-checked on a synthetic image; the window itself never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 | | White-box crypto devirtualization to a program | Absorbed route: symbolic input, constant folding, a straight-line `(op, args)` program plus its constant table, output map and template; version-bound, deterministic functions only. No extract was produced here | partial | unverified | `references/native-vm-recovery.md` §11 | -| Cipher classification by differential probe | Absorbed method: header length, prefix XOR and the length law classify a stream or block primitive before implementation; involutions cancel; Java/Python encoding and compression drift decide the last mile. Never run here | partial | unverified | `references/protocol-reverse.md` §7 | +| Cipher classification by differential probe | Method: header length, determinism, one-byte diffusion width and the length law separate CTR, CBC and ECB (the earlier prefix-XOR-only claim was falsified and is corrected); involution wrappers cancel; Java/Python encoding and compression drift decide the last mile. Probes and algebra host-checked on AES-128 and synthetic inputs; no target body classified | partial | inferred | `references/protocol-reverse.md` §7 | ## Where the full record lives — repository root, not shipped diff --git a/skills/apk-reverse/references/linker-and-soinfo-repair.md b/skills/apk-reverse/references/linker-and-soinfo-repair.md index fe3d12d..23083cd 100644 --- a/skills/apk-reverse/references/linker-and-soinfo-repair.md +++ b/skills/apk-reverse/references/linker-and-soinfo-repair.md @@ -202,8 +202,12 @@ is not relocated yet" as expected, and record the bias and relocation tables in A dump taken *inside* `JNI_OnLoad` is the worst of both: the code is complete, so the dump looks right, and the watchdog is up, so the process dies during the read. **If the target dies while you -dump and only then, this is the shape — not a broken tool.** (Both rules `unverified`: absorbed from -one write-up, not reproduced here.) +dump and only then, this is the shape — not a broken tool.** + +**Claim strength.** The window itself — SMC completing *and* watchdogs starting inside the same +`JNI_OnLoad` — is `unverified`: one write-up reports it and this record did not reproduce it. The +placement rule (dump in `onEnter`, keep the bias) is `inferred` from the arithmetic in §2 and §4. The +symptom is the source's report, also `unverified`. ### 6.3 Read with syscalls, and pad the holes @@ -213,15 +217,19 @@ Two properties of the reader decide whether the dump is usable. libc's `open`/`read`/`write` sees your dump as one more caller and can refuse or terminate. A raw-syscall reader (`svc #0` from a small stub, or a kernel-side reader) sits outside that surface. This is `advanced-unpacking.md` §"Layered descent" applied to your own tooling: the layer - you read from is also the layer the target can watch. + you read from is also the layer the target can watch. (`unverified` for the protector-hook claim; + the layer rule it extends is `measured` there.) - **An unreadable page is a hole to fill, not a range to skip.** Skipping shifts every later byte and produces a file that repairs into a plausible ELF with missing logic. Zero-filling preserves offsets and sizes, so segment placement still works; record which ranges were unreadable so the repair can be audited. A dump is evidence, and "these ranges were unreadable" belongs in it. + (`observed`, host: in a synthetic three-page image with one unreadable page, a marker meant for + offset 8199 lands at 8199 when the page is zero-filled and at 4103 when it is skipped — see the + EXTENSION record.) Then repair the image with a tool that rebuilds what the loader would have written (program headers, dynamic table, section headers). The repair consumes a file whose *offsets* are right; that is what -the zero-fill protects. +the zero-fill protects. (`inferred`: the offset/size argument above, not a repair run here.) ## 7. Timing: constructors and one-shot registrations diff --git a/skills/apk-reverse/references/native-tamper-and-suicide.md b/skills/apk-reverse/references/native-tamper-and-suicide.md index 04302c3..4003413 100644 --- a/skills/apk-reverse/references/native-tamper-and-suicide.md +++ b/skills/apk-reverse/references/native-tamper-and-suicide.md @@ -179,6 +179,9 @@ Two conditions, both required: once it always returns clean. Enumerate the callers before you touch it — the xref list in the disassembler, or the same caller enumeration `scripts/find_refs.py` does for dex methods. +(`unverified` that a given target funnels every detector through one predicate — that is the +source's finding. The decision-versus-action rule is `inferred` from it.) + ### Grade the sites by what being wrong costs | Tier | What it is | How to patch it | Why this order | @@ -192,6 +195,10 @@ non-zero, a "list of hits" scanner returns NULL, a "should continue" predicate r 0 everywhere is a guess that happens to work half the time — and the failing half reads as a broken patch instead of a wrong constant. +(`inferred`: the order follows from observability — a death at tier 0 hides every other result — +and the benign-value rule from the caller's branch. The tier vocabulary is the source's, +`unverified`; no graded set was run here.) + ### Count the watchdogs before declaring the set closed One kill site is rarely the only thread. The threads are distinguishable by what they *read*, not by @@ -205,11 +212,14 @@ Patch one and the app still dies, at a different delay, which reads as "the patc than "there were three". After the tier 0/1 set, the process must survive longer than the **longest** timer you have observed, not the first one. +(`unverified` for the probe inventory — source-reported for one shell; `inferred` for "identify a +thread by what it reads", which follows from the probe classes themselves.) + ### One tier per run Patch tier 0, run, record; then tier 1; then tier 2 only if needed. A build with all tiers changed at once cannot tell you which one mattered, and the next failure is unattributable. This is the single-variable rule -of `pitfalls.md` P21 applied to a patch set. +of `pitfalls.md` P21 applied to a patch set. (`inferred`.) ## ELF hardening you will meet in these targets diff --git a/skills/apk-reverse/references/native-vm-recovery.md b/skills/apk-reverse/references/native-vm-recovery.md index 4ce7eb2..8bc34d7 100644 --- a/skills/apk-reverse/references/native-vm-recovery.md +++ b/skills/apk-reverse/references/native-vm-recovery.md @@ -11,9 +11,10 @@ half lives in `references/ete-diff-tracing.md`, the **slicing** tool in write-ups (Kanxue, 2026-08/09, titles and authors anonymized in this record; one is a full arm64 custom-VM recovery, the other a VMP interpreter recovery with trace-driven IR lifting). The write-ups are `observed` by their authors; this repository has **never run any of these -steps on a real VM target** — everything below is `inferred` from those write-ups, cross-read -against the open xvmp source facts in `references/ete-diff-tracing.md` §5. The pipeline's -host-tested pieces (trace parsing, slicing, diffing) carry their own labels where named. +steps on a real VM target**, so the mechanics below are `unverified` here — reported, not +reproduced — and the ordering judgements built on them are `inferred`, cross-read against the +open xvmp source facts in `references/ete-diff-tracing.md` §5. The pipeline's host-tested pieces +(trace parsing, slicing, diffing) carry their own labels where named. ## 1. Deciding it is a VM at all @@ -175,7 +176,8 @@ falls out. That is the end state this file walks toward: the guest program as da Sections 1–10 aim at a guest program you can **read**. When the interpreter implements one deterministic function — a request signer, a decoder, a checksum — the cheaper deliverable is a -program you can **run**, and the handler semantics never need to be understood. +program you can **run**, and the handler semantics never need to be understood. (`unverified`: one +write-up reports the route; this record did not run it.) 1. **Fix the boundary.** One native entry that takes a buffer in and produces a buffer out. Everything inside is the VM; everything outside is protocol. @@ -200,16 +202,18 @@ program you can **run**, and the handler semantics never need to be understood. **Version binding.** The program is a property of one build. Store the library's fingerprint (size and build id) beside it and re-extract on upgrade: the tables and the map move even when the algorithm does not. Say which build a program came from; an extract with no fingerprint is not reproducible. +(`inferred` for the fingerprint discipline; `unverified` for the source's report that the tables move +on update.) **Validation is per node, not per output.** Compare node values at several checkpoints against a device run. A node that is right in value but wrong in width (32-bit truncation where the VM kept 64) passes the final comparison on some inputs and fails on others — §9's failure seen from the other -side. +side. (`inferred` from §9.) **Boundary.** This route needs a deterministic, input-driven computation. If the VM folds in a device id, a timestamp or a nonce, those become constants from one trace, and the honest claim is "reproduces one device's answers", not "implements the algorithm". For a signer that is often exactly -enough, but the report has to say which claim it is making. (`unverified` — absorbed, not run.) +enough, but the report has to say which claim it is making. ## 12. Where this repository's tools plug in @@ -233,5 +237,5 @@ enough, but the report has to say which claim it is making. (`unverified` — ab - [ ] segments stacked if entry can nest (§7); slicer's nested-loop diagnostic honoured - [ ] operand slicing seeded on all STOREs; byte provenance (not value) decides folding - [ ] clean re-VM runs real bytecode against trace ground truth before any IR is trusted -- [ ] labels: write-up methods `inferred`, shipped tools `observed` (host), no VM target run here +- [ ] labels: write-up mechanisms `unverified`, ordering judgements `inferred`, shipped tools `observed` (host), no VM target run here - [ ] if the deliverable is a callable signer/decoder: §11's four artifacts captured as a set, program pinned to a build fingerprint diff --git a/skills/apk-reverse/references/pitfalls.md b/skills/apk-reverse/references/pitfalls.md index d0a17a6..f783eec 100644 --- a/skills/apk-reverse/references/pitfalls.md +++ b/skills/apk-reverse/references/pitfalls.md @@ -1493,7 +1493,8 @@ the defect lives in memory, where you cannot diff it the way you diff two files. **Prove each stage's input is byte-identical — length, alignment, encoding — before touching the algorithm.** Compare stage inputs against the device, not final outputs; a stage-input diff localises in minutes what an output diff cannot localise at all. P42 is the decoder-desync cousin; this one is -an extraction length. +an extraction length. (`inferred` — a reasoning rule from how one byte propagates; no target case was +measured for this record.) ## P50. A "custom cipher" is a wrapper that cancels @@ -1517,4 +1518,6 @@ step. **Compose it with itself before reimplementing it** (`f(f(x)) == x`), and check whether the caller invokes it on both paths. If it cancels, drop it and recover the standard primitive underneath; keep the wrapper in the report, because it is the reason the algorithm looked unfamiliar. The method, with -the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7. +the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7; the +mask identity itself is host-checked there (`observed`: zero counterexamples over 65,536 eight-bit +pairs and 200,000 random 16-bit pairs). diff --git a/skills/apk-reverse/references/protocol-reverse.md b/skills/apk-reverse/references/protocol-reverse.md index c030842..abb9b5d 100644 --- a/skills/apk-reverse/references/protocol-reverse.md +++ b/skills/apk-reverse/references/protocol-reverse.md @@ -374,23 +374,36 @@ missing family first. **Load this when** the body is opaque: a long base64/hex field in the query, a binary request body, a fixed prefix on every response — and the transform lives in a native library or a helper class. -Before any static work, three black-box experiments on captured pairs, each a few minutes. You need -several requests whose plaintext you control. +Three black-box experiments on captured pairs, before any static work. The probes themselves were +host-checked here on AES-128 through `openssl` (commands and output in the verification record, +`docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md`, under "Verification pass"). What they prove +about *your* target is one step further, and stays `inferred` until you vary the key and the +request. -1. **Header length.** Encrypt two different payloads under the same key and decode both. A constant - first N bytes ⇒ a key-derived header. A prefix that changes when the key changes ⇒ keyed, not - random. A prefix that changes per request ⇒ a per-message IV or counter. -2. **Prefix XOR.** Encrypt `P` and `P || X`, then XOR the two ciphertexts. Zeros over the common - prefix ⇒ a stream cipher whose keystream depends on the key alone. No such relation, plus a length - rounded up to a block size ⇒ a block cipher. Identical 16-byte blocks recurring across messages ⇒ - ECB, which is a finding on its own. -3. **Length law.** `len(c) = len(p) + k` ⇒ stream/CTR. `len(c) = ceil((len(p)+1)/b) * b` ⇒ block with - padding. A constant relation with a compressed prefix ⇒ compress-then-encrypt, and the plaintext - you are looking for is the compressor's output, not the JSON. +1. **Header length.** Encrypt two payloads under the same key and decode both. A constant first N + bytes ⇒ a deterministic header: fixed plaintext, or derived from the key/IV. A prefix that changes + when the key changes ⇒ keyed. A prefix that changes per request ⇒ a per-message nonce. + (`observed` on synthetic inputs.) +2. **Determinism, then diffusion width.** Encrypt `P` and `P || X`, and XOR the ciphertexts. Zeros + over the common prefix ⇒ the construction is deterministic under a fixed key and IV — **and that + is all it proves.** Measured: CTR, CBC and ECB all give zero prefix-XOR at a fixed IV, so this + experiment cannot separate a stream cipher from a block mode. (An earlier draft of this file said + it could; one run falsified it.) Repeat with a different IV: a CTR-style mode diverges, and so does + any mode that carries a per-message nonce. -The classification decides what you implement: a stream cipher needs its keystream and nothing else; -a block cipher needs its mode; ECB needs no key schedule at all to explain structure. Skipping this -step is how a standard construction gets reimplemented as something exotic. + The separation comes from a one-byte plaintext flip, counted over the ciphertext: **1 changed + byte** ⇒ a keystream independent of the plaintext (measured: CTR, 1 of 32); **every block from + that point on** ⇒ a chained mode (measured: CBC, 32 of 32); **exactly one block** ⇒ ECB (measured: + 16 of 32, later blocks untouched). (`observed`, host.) +3. **Length law.** `len(c) = len(p)` ⇒ no padding — which fits CTR and a `nopad` block mode equally, + so it never decides alone. `len(c) = b * ceil((len(p)+1)/b)` ⇒ a padded block mode (PKCS#7; + checked for b = 16, p = 0…32). A compressed prefix with a constant length relation ⇒ + compress-then-encrypt, and the plaintext you are looking for is the compressor's output. + (`observed`, host.) + +The classification decides what you implement: a keystream cipher needs its keystream and nothing +else; a chained mode needs the mode and the IV; ECB needs neither to explain its structure. Skipping +this step is how a standard construction gets reimplemented as something exotic. ### A custom-looking transform may be a wrapper that cancels @@ -398,10 +411,12 @@ Wrapper arithmetic is cheaper to check than to rewrite. Two shapes cover most of - **Involution applied symmetrically.** A transform with `f(f(x)) == x` that is called on the way in *and* on the way out contributes nothing. The common disguise is a mask pair: - `(x & ~m) | (~x & m)` is exactly `x ^ m` when the bits of `m` partition the word. If both encrypt - and decrypt call it, drop it from your model and recover the standard primitive underneath. -- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is an - encoding, not encryption. Index it out and compare against known S-boxes. + `(x & ~m) | (~x & m)` is exactly `x ^ m`. Exhaustively checked here over all 8-bit `(x, m)` pairs + and 200,000 random 16-bit pairs: zero counterexamples, and the composition is an involution. If + both encrypt and decrypt call it, drop it and recover the standard primitive underneath. + (`observed`, host — the algebra; whether a target's wrapper is that shape is `inferred`.) +- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is + an encoding, not encryption. Index it out and compare against known S-boxes. (`inferred`.) Test the composition before reimplementing: apply the function twice, and check whether the caller uses it twice. If it cancels, the algorithm is standard and the reason it looked unfamiliar is now @@ -413,17 +428,22 @@ Two non-cryptographic differences fail signatures that are otherwise correct. Bo the algorithm and both are cheap to eliminate: - **The client's encoder may be Java's.** `URLEncoder` writes a space as `+`, leaves `*`, `-`, `_`, - `.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's `quote` does the - opposite on `~` and uses `%20`. Reimplement what the client does and check it against one captured - parameter value, not against documentation. -- **Compression bytes are implementation-defined.** Two libraries can emit different, equally valid - compressed output for the same input. Whether that breaks you is a property of the **server**: if it - decompresses and checks a CRC or the plaintext, byte equality is not required; if it compares the - stored blob, it is. One mutation tells you which, and the answer decides whether byte-perfect - reproduction is even the goal. + `.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's does the + opposite on two of those: measured on this host, `quote` gives `a%20b~%2A-_.` (space `%20`, `*` + encoded, `~` kept) and `quote_plus` gives `a+b~%2A-_.` (space `+`, `*` still encoded). Reimplement + what the client does and check it against one captured parameter, not against documentation. + (Python side `observed`; the Java side is `unverified` here — no JDK on the analysis host — and is + quoted from the JDK API contract, so confirm it on one captured value.) +- **Compressed bytes are implementation-defined.** Measured: `zlib.compress(data, 1)` and `(data, 9)` + produce different byte streams (49 vs 48 bytes) for the same payload, and both decompress to it. + Whether the difference breaks a signature is a property of the **server**: if it decompresses and + checks a CRC or the plaintext, byte equality is not required; if it compares the stored blob, it + is. One mutation tells you which. (The general claim is `observed`; the specific liblz4-vs-python- + lz4 divergence is `unverified` — no lz4 binding here.) **Boundary.** None of the three experiments needs the `.so`; that is the point. When the keystream is derived per message from a device secret, the native route comes back and §5's attribution applies. +(`inferred`.) ## 8. Failure modes diff --git a/skills/apk-reverse/references/signature-derived-keys.md b/skills/apk-reverse/references/signature-derived-keys.md index 00f9501..c8b4742 100644 --- a/skills/apk-reverse/references/signature-derived-keys.md +++ b/skills/apk-reverse/references/signature-derived-keys.md @@ -146,12 +146,17 @@ it also fails, the problem is environmental (`references/tls-and-cert.md`, A different failure with the same source. A signer is reconstructed from static analysis and produces a signature the server refuses, while every ingredient visible in the request looks right. The formula turns out to be `sig = prefix + H(input || salt || key)`, and `salt` never appears in the request — it -is generated at init from the certificate and cached in a global. +is generated at init from the certificate and cached in a global. (The certificate origin is +`unverified` here; it is the same mechanism as the key case above, which is documented and likewise +unmeasured.) **Why it defeats black-box search: the search space is missing an argument.** Trying MD5/HMAC/SHA combinations over the visible parameters cannot find a value that is concatenated inside the digest and never transmitted. The symptom is *uniform* failure, not near-misses — that is the tell, and it is -also why this is misread as "the algorithm is not fully recovered". +also why this is misread as "the algorithm is not fully recovered". (`observed`, host, synthetic: a +36-combination search over three algorithms and four field orders returns zero hits against a digest +with a hidden 16-byte salt, and returns the target the moment the salt is included — see the +EXTENSION record.) How to get it: @@ -160,15 +165,13 @@ How to get it: - **Or hook the digest update** and dump its argument on one request; the salt is the constant run inside an otherwise changing buffer. - **Verify with three captured triples** that vary one field each. A formula that matches all three is - the formula; one that matches one is a coincidence. + the formula; one that matches one is a coincidence. (`inferred`.) Consequence for the deliverable: the salt is per build and per certificate. A local signer carries the captured constant plus a build fingerprint (the same discipline as an extracted VM program, `native-vm-recovery.md` §11), and the report says which build it came from. On update the value changes and the signer must be re-derived — do not present the constant as an algorithm. - -(`unverified`: the certificate-derived cache is `inferred` from the same mechanism as the key case -above; nothing here was run against a target.) +(`inferred` from the certificate binding, not measured.) ## Cost of getting this wrong