Absorb three write-ups on dump timing, native VM devirtualization and encrypted bodies

vm1: dump window at android_dlopen_ext entry, raw-syscall reader, zero-filled holes; predicate-first patching with tiered ordering; module-filtered signal attribution.

vm2: devirtualization to a straight-line (op, args) program with constant table, output map and template; version binding; per-node validation.

vm3: three black-box cipher probes, involution wrappers, encoding/compression drift; a certificate-derived salt inside the signature digest.

Also adds pitfalls P49-P50, three capability rows (all partial/unverified), the evidence/route records and EXTENSION-ref-absorption-2026-10b.md. Renumbers native-vm-recovery section 11 to 12. Gates and pytest green.
This commit is contained in:
root
2026-10-25 12:10:41 +08:00
parent f497180e35
commit 45ba13c0f3
13 changed files with 440 additions and 12 deletions
@@ -21,7 +21,7 @@ postings; no sample, tool output or target fact from them entered the repository
| W1: full arm64 custom-VM recovery (long-form) | 0x00–0x1A method narrative | `references/native-vm-recovery.md` (existence signals, dispatcher separation, partial deflattening, relocation tables, PC proofs, descriptor/ISA cautions, clean-re-VM loop) |
| W2: VMP interpreter recovery with trace-driven IR lifting | anchor/trace/lifting narrative | `references/native-vm-recovery.md` (register anchoring + triple verification, nested segment stacks, STORE-backward slicing, byte provenance, IR shape), `vm_handler_slice.py` nested-loop diagnostic |
| W3: login-capture war story (packed + MADP/Weex + gomobile) | route-by-route复盘 | `references/pitfalls.md` P43–P46; the stack-ABI note in `references/hardware-watchpoints.md` |
| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §11 — no new file |
| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §12 — no new file |
## 2. The one code change, and why — observed
@@ -54,7 +54,7 @@ spin-loop note on the real code path. Both regressions are now pinned by tests.
plaintext boundary; a RegisterNative inventory is one run for a full method table).
- W4's full IDA script: the technique is named (state-register MOV/MOVK constant propagation,
CMP-site collection, predecessor walk past blocks with no CMP on the state register) and
pointed at from `native-vm-recovery.md` §11; writing an IDA-independent implementation is a
pointed at from `native-vm-recovery.md` §12; writing an IDA-independent implementation is a
future pass, not an absorbed claim.
- Every sample identity in all four sources: anonymized by the authors, kept anonymized here;
`.ref/` (the local copies) is untracked and git-ignored in effect (never added).
@@ -0,0 +1,83 @@
# EXTENSION — second reference absorption from `ref/*` (2026-10-25)
This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into
the files that already own the topics. It did not run any of them. The record exists so a later reader
can tell what changed and on what footing.
## Sources
Three files, described by shape. Titles, authors, product names, package identifiers and every
per-build address are omitted; none was copied onto the tracked surface.
| File | What it is |
|---|---|
| `vm1.md` | A dual-library commercial shell on ARM64: two SOs, one owning Java/ART checks, the other owning an `init_proc` scheduler with inline-shellcode `exit_group`, three watchdog threads and SMC. Covers dump timing, a syscall-level reader, single-predicate patching and a tiered patch order |
| `vm2.md` | A white-box VM protecting one signing routine: threaded-code interpreter, tens of thousands of obfuscated nodes, and a devirtualization to a straight-line `(op, args)` program with constant table, output map and fixed template |
| `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift |
All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing
in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering
judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was
copied.
## What was added
No new reference files. Each mechanism was folded into the file that already owns its topic:
| File | Absorbs | Deliberately does not |
|---|---|---|
| `references/linker-and-soinfo-repair.md` §6.2–6.3 | the dump window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share it; the raw-syscall reader outside a self-hooked libc; zero-filling unreadable pages so segment offsets survive repair | claim that a dump taken at `android_dlopen_ext` entry is complete; name a specific reader |
| `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker |
| `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals |
| `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections |
| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required |
| `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm |
| `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method |
## Registration surface updated
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`,
`whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all
`partial` / `route_measured: false` / `unverified`; two existing rows
(`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended
rather than duplicated.
- `references/evidence-summary.md`: three rows.
- `references/coverage/route-inventory.md`: three rows.
- `references/coverage-and-limits.md`: three absorbed-route bullets extended.
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
symptom mirror and the reference/script inventories had nothing to learn.
## Deliberately not absorbed
- **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte
salt value, the signing prefix and template string, the package identifier, the shell's own
`sub_*` names. They are only valid for one build and they carry identity.
- **Performance and effort claims** ("1 ms", "days to hours") — no measurement exists here to anchor
them.
- **The AI-assisted-analysis narrative** in one source: it is advice about tool use, not a reversing
mechanism, and it is not reproducible as written.
- **Thread and watchdog tables as data.** The *method* (enumerate by probe, then prove the set closed)
was absorbed; the table itself would invite applying one build's layout to another.
## What would change a label
One cheap experiment per row, all on a device this record never had:
- **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a
disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record
whether the process survives.
- **Patch grading:** run the same target twice — kill sites only, then the predicate only — and show
which one survives longer than the observed timer.
- **Devirtualization:** extract one program, run it against a device call on ten inputs, and show
per-node equality at three checkpoints.
- **Cipher classification:** classify one body with the three probes, then reproduce it and have the
server accept the request.
Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as
support for a claim about a target.
## Gate status
`check_repo.py`, `check_refs.py`, `check_routing.py`, `check_commands.py`, `check_budget.py` and
`pytest -q` were run after the change; the result is in the session record, not in this file. A green
run proves the inventory resolves; it says nothing about whether the mechanisms work.
@@ -250,7 +250,7 @@
"docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 3 (repository root, not shipped)",
"tests/benchmark.md row B2 (repository root, not shipped)"
],
"limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed."
"limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed. Site selection is a separate claim: the predicate-first rule and the tiered grading (added 2026-10-25) are absorbed from a write-up and unverified here -- no graded patch set was run, and the ordering judgement is inferred."
},
{
"capability": "native-instruction-tracing-stalker",
@@ -654,7 +654,7 @@
"evidence_not_shipped": [
"docs/tool-verification/README.md (repository root, not shipped)"
],
"limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one."
"limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one. A second variant -- a certificate-derived salt concatenated inside the digest, invisible to any black-box search -- is documented from the same 2026-10-25 write-up and likewise unmeasured."
},
{
"capability": "blocked-react-native-ios",
@@ -982,6 +982,54 @@
],
"limits": "Parsing of all three recovery sources (@DebugMetadata c=, @Metadata d2 first non-stdlib descriptor, jadx 'renamed from' comments), the third-party skip list and the map-file round trip are observed on embedded fixtures only. The upstream recovery-rate claims (30-50% of classes overall, ~100% of *Repository/*ViewModel/*Impl, ~80% of data-class DTOs) are unverified here: no real obfuscated Kotlin app was run through the script. Method and field names are never recovered; heavily inlined shapes can surface under a wrong primary class, so every mapping row is a hint. The script reads decompiled sources, not the dex: a decompiler that drops annotations removes the source entirely."
},
{
"capability": "native-dump-window-and-syscall-reader",
"group": "native",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": [
"references/linker-and-soinfo-repair.md",
"references/advanced-unpacking.md"
],
"evidence_not_shipped": [
"ref/vm1.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a dual-library commercial shell"
],
"limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The mechanisms are unverified; the ordering judgement (dump on android_dlopen_ext entry, before the protector runs) is inferred from the same source."
},
{
"capability": "whitebox-crypto-devirtualization-to-program",
"group": "native-dynamic",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": [
"references/native-vm-recovery.md",
"references/vmp-differential-analysis.md"
],
"evidence_not_shipped": [
"ref/vm2.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a white-box VM protecting one signing routine"
],
"limits": "The route -- symbolic input, constant folding, loop and branch elimination, an (op, args) straight-line program plus its constant table, output map and output template -- is absorbed from one write-up and never run. It applies only to a deterministic, input-driven function: device id, time or nonce inputs become captured constants, and the claim then degrades to reproducing one device. The program is version-bound; no extract was produced here."
},
{
"capability": "cipher-classification-by-differential-probe",
"group": "protocol",
"status": "partial",
"route_measured": false,
"strength": "unverified",
"benchmark_rows": [],
"evidence": [
"references/protocol-reverse.md",
"references/signature-derived-keys.md"
],
"evidence_not_shipped": [
"ref/vm3.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a native signer and an encrypted request body"
],
"limits": "Three black-box probes (key-derived header length, prefix XOR for a keystream, the length law), the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Absorbed from one write-up; no cipher was classified here, and no server was tested for whether it compares compressed bytes or only the decompressed payload."
},
{
"capability": "websocket-long-connection-analysis",
"group": "protocol",
@@ -114,7 +114,9 @@ its own strength note at the top.
- **Instruction-level tracing and de-obfuscation** — the OLLVM shapes, a Stalker trace, the
trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`.
- **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3
limit, native-side certificate pinning (`protocol-reverse.md`).
limit, native-side certificate pinning, and — for an opaque body — classifying the primitive by
differential probes (header length, prefix XOR, the length law) before implementing it
(`protocol-reverse.md`).
- **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls,
`init_array`-early detection, and the kernel-route map with its version gate
(`kernel-and-environment-hardening.md`).
@@ -136,8 +138,9 @@ its own strength note at the top.
(`scripts/vm_handler_slice.py`).
- **Native VM recovery procedure** — the full method (dispatcher-layer separation, register
anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM
validation) absorbed from two public write-ups; every step `inferred`, none run here
(`native-vm-recovery.md`).
validation) plus a devirtualization end state that emits a runnable straight-line program with its
constant table, output map and template; absorbed from public write-ups, every step `inferred` and
none run here (`native-vm-recovery.md`).
- **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition
signals, the call-it-first route (AST restore + environment fill), stack- and register-machine
reading procedures, AST instruction extraction with register/immediate typing, the four
@@ -180,8 +183,10 @@ its own strength note at the top.
- **Linker-level SO unpacking and `soinfo` repair** — the `dlopen` pipeline as a loader
specification, `load_bias` and segment arithmetic including `seg_page_offset`, `DT_*` parsing,
relocation targets and the defining-library-versus-caller bias, the minimal `soinfo` repair set,
anti-dump header XOR detected by ELF invariants, and the dump window at `LoadSegments`. Source-
reading synthesis plus a Frida cookbook; no loader was reimplemented here
anti-dump header XOR detected by ELF invariants, the dump window at `LoadSegments`, the narrower
window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share that
function, a raw-syscall reader outside a self-hooked libc, and zero-filled unreadable pages.
Source-reading synthesis plus two write-ups; no loader was reimplemented here
(`linker-and-soinfo-repair.md`).
- **VM protector analysis** — envelope triage, VM_DATA/VM_CONTEXT/vEIP/vStack anatomy, data-anchored
breakpoints, forged-return recognition, trace-then-search-by-constant, virtual eflags as writable
@@ -61,6 +61,9 @@ route was not measured on this repository's evidence**. Treat them as leads with
| Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` |
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
| Native dump window and syscall reader | dump at `android_dlopen_ext` entry when self-decryption and watchdog startup share `JNI_OnLoad`; read via raw syscalls; zero-fill unreadable pages so offsets survive repair — absorbed, nothing run here — `linker-and-soinfo-repair.md` |
| White-box crypto devirtualization | fold a deterministic VM transform into a straight-line `(op, args)` program plus constant table, output map and template, pinned to one build — absorbed, no extract produced — `native-vm-recovery.md` |
| Cipher classification by differential probe | header length, prefix XOR and the length law separate stream from block before implementation; involution wrappers cancel; encoding/compression drift is the last mile — absorbed, never run — `protocol-reverse.md` |
## Dependencies this skill does not ship
@@ -257,6 +257,27 @@ shipped library. A crash frame in JIT output cannot be mapped to a file offset.
API where the framework compiles reflection at runtime, and treat "the frame has no file" as a fact
about compilation, not as a hidden module.
### Filter signals by module before believing them
A process under instrumentation produces signals that are not the target's. ART takes an implicit
`SIGSEGV` for null and bounds checks inside `boot*.oat`, `*.odex` and anonymous JIT pages, and resumes
the thread in its own handler. A deliberate crash, a protector's kill and the runtime's own fault can
all reach your observer looking identical: "signal 11".
The tuple that discriminates is **(signal, faulting address, faulting module)**, not the signal number.
Count a signal only when the address resolves inside the library you are studying; log the rest and
let them through untouched.
Two consequences:
- **Do not install a handler that swallows what the runtime expects to handle.** ART's implicit faults
are how its null checks work; swallowing them turns a healthy process into a mysterious hang. A
handler that intercepts signals has to forward anything whose faulting module is not the target.
- **A `boot*.oat` or `odex` frame in a crash record is the runtime's, not the shell's.** Chasing it
produces "the VM is corrupting memory" when the truth is that the VM's own fault handling ran.
(`unverified` — a 2026-10 write-up on a dual-library shell; the discrimination is general.)
### Stage 6: the precise stop, and the static alternative to it
**Input signal:** a named module + offset from Stage 2, Stage 3 or Stage 5. **Action:** neutralise
@@ -81,6 +81,9 @@ next** — and points at the one section that says "the full record is elsewhere
| Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` |
| React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` |
| Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` |
| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 |
| White-box crypto devirtualization to a program | Absorbed route: symbolic input, constant folding, a straight-line `(op, args)` program plus its constant table, output map and template; version-bound, deterministic functions only. No extract was produced here | partial | unverified | `references/native-vm-recovery.md` §11 |
| Cipher classification by differential probe | Absorbed method: header length, prefix XOR and the length law classify a stream or block primitive before implementation; involutions cancel; Java/Python encoding and compression drift decide the last mile. Never run here | partial | unverified | `references/protocol-reverse.md` §7 |
## Where the full record lives — repository root, not shipped
@@ -13,7 +13,8 @@ source-reading write-ups (Android 5.0 and 8.x `bionic/linker`) and one Frida coo
was reimplemented and no dump was produced here. Source citations are `unverified` against your
tree until you open the matching AOSP tag. The *arithmetic* (bias, page offset, relocation target)
is `inferred` from that source and should be checked with one control load before it is trusted.
The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants.
The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants. The dump-window and reader rules in §6.2–6.3 were added 2026-10 from a write-up
on a dual-library commercial shell; they are `unverified` operating rules, not measurements.
## 1. The pipeline, as a specification for a loader
@@ -186,6 +187,43 @@ Deleting the header instead of XORing it is a weaker defence but a nastier sympt
*cleanly* rejected, so the analyst blames the dump site. If the dump site is right and the header
is missing, the correct conclusion is "the shell removed it", not "the dump was early".
### 6.2 The window can close inside `JNI_OnLoad`
The dump points above are ordered by *how much of the image exists*. A protector that modifies itself
adds a second axis: *how long the process lives*. A shell that decrypts its own code in
`JNI_OnLoad` (SMC) and starts watchdog threads from the same function leaves exactly one interval in
which both hold — segments mapped, SMC not yet run. Dumping after `dlopen` returns is late twice
over: the self-decryption has completed, and the watchdog is already armed by the time any
post-return hook runs.
So: hook `android_dlopen_ext`, and dump in `onEnter` when the target path matches. Treat "the image
is not relocated yet" as expected, and record the bias and relocation tables in the same pass
(§2, §4). An unrelocated dump plus its bias is recoverable; an unrelocated dump without one is not.
A dump taken *inside* `JNI_OnLoad` is the worst of both: the code is complete, so the dump looks
right, and the watchdog is up, so the process dies during the read. **If the target dies while you
dump and only then, this is the shape — not a broken tool.** (Both rules `unverified`: absorbed from
one write-up, not reproduced here.)
### 6.3 Read with syscalls, and pad the holes
Two properties of the reader decide whether the dump is usable.
- **The reader must not travel through the library the protector hooks.** A shell that inline-hooks
libc's `open`/`read`/`write` sees your dump as one more caller and can refuse or terminate. A
raw-syscall reader (`svc #0` from a small stub, or a kernel-side reader) sits outside that
surface. This is `advanced-unpacking.md` §"Layered descent" applied to your own tooling: the layer
you read from is also the layer the target can watch.
- **An unreadable page is a hole to fill, not a range to skip.** Skipping shifts every later byte
and produces a file that repairs into a plausible ELF with missing logic. Zero-filling preserves
offsets and sizes, so segment placement still works; record which ranges were unreadable so the
repair can be audited. A dump is evidence, and "these ranges were unreadable" belongs in it.
Then repair the image with a tool that rebuilds what the loader would have written (program headers,
dynamic table, section headers). The repair consumes a file whose *offsets* are right; that is what
the zero-fill protects.
## 7. Timing: constructors and one-shot registrations
`dlopen` does not return until `CallConstructors` has run the dependencies' constructors, then the
@@ -159,6 +159,59 @@ terminate path is the end of the check; nothing downstream re-reads "did I die".
rather than assuming: after a successful suppression the app must reach its **normal UI and stay
there**, not loop through half-initialised states.
## Which site — and in what order
Patch sites are not interchangeable. Fixing the wrong one costs a day; fixing them in the wrong order
costs the same day twice, because you cannot tell which change mattered. Sort the candidates first.
### One predicate may be deciding for all of them
Group the branches you found by the function whose return value they test. When every detector funnels
into one `is_clean()`-shaped call, rewriting that return is the whole bypass: the kill sites are never
reached, and the branches you did not enumerate are covered too.
Two conditions, both required:
- the function is a **decision** (returns a verdict), not the **action** (performs the kill). Patching
an action changes the response; patching a decision changes the belief. The second is what makes a
bypass stable.
- you have checked its other callers. A predicate that also gates a feature will disable that feature
once it always returns clean. Enumerate the callers before you touch it — the xref list in the
disassembler, or the same caller enumeration `scripts/find_refs.py` does for dex methods.
### Grade the sites by what being wrong costs
| Tier | What it is | How to patch it | Why this order |
|---|---|---|---|
| **Tier 0** | terminate primitives: `exit_group`/`tgkill`/`kill` call sites, the shellcode executor, the unified kill point | replace with a no-op or a returning stub | until they stop, every other result is unobservable — the process dies first |
| **Tier 1** | decision functions: the shared predicate, each scanner (`/proc` walk, `TracerPid` read, module checksum) | return the benign value *for that caller's branch* | they are the belief; a clean verdict also skips the report path |
| **Tier 2** | auxiliary gates: adb/debug flags, telemetry, feature switches | only if observed to gate the thing you care about | unbounded work; most are inert in your environment |
The benign value is not always 0. Read the caller's branch: a "found something" scanner returns
non-zero, a "list of hits" scanner returns NULL, a "should continue" predicate returns 1. Returning
0 everywhere is a guess that happens to work half the time — and the failing half reads as a broken
patch instead of a wrong constant.
### Count the watchdogs before declaring the set closed
One kill site is rarely the only thread. The threads are distinguishable by what they *read*, not by
their names:
- a `/proc/self/task`, `/proc/self/fd` or `/proc/self/maps` walk — looking for your instrumentation;
- `TracerPid`, the parent pid, or a task in state `T` — looking for a debugger or a suspended process;
- a checksum over a loaded module — looking for a modified page.
Patch one and the app still dies, at a different delay, which reads as "the patch did not work" rather
than "there were three". After the tier 0/1 set, the process must survive longer than the **longest**
timer you have observed, not the first one.
### One tier per run
Patch tier 0, run, record; then tier 1; then tier 2 only if needed. A build with all tiers changed at once cannot
tell you which one mattered, and the next failure is unattributable. This is the single-variable rule
of `pitfalls.md` P21 applied to a patch set.
## ELF hardening you will meet in these targets
Hardening libraries are not normal ELF files. Assume the following until measured otherwise.
@@ -171,7 +171,47 @@ pre-IP and post-IP are both recorded (`0x2 -> 0x3`), the 200k-line IR sequence *
execution trace: +1 is fallthrough, jumps are branches, back-edges are loops — the guest CFG
falls out. That is the end state this file walks toward: the guest program as data.
## 11. Where this repository's tools plug in
## 11. The other end state: a program you can run
Sections 1–10 aim at a guest program you can **read**. When the interpreter implements one
deterministic function — a request signer, a decoder, a checksum — the cheaper deliverable is a
program you can **run**, and the handler semantics never need to be understood.
1. **Fix the boundary.** One native entry that takes a buffer in and produces a buffer out.
Everything inside is the VM; everything outside is protocol.
2. **Symbolic input, concrete everything else.** Seed the input bytes as symbols and run the VM (live,
or over the recorded trace). Constant-fold every node whose value does not depend on them; delete
loops and branches whose condition folds. What cannot fold is the part you must reproduce.
3. **What remains is straight-line.** An SSA list of `(op, args)`, where `args` are constants or the
indices of earlier nodes, and `op` is the ALU subset the real semantics needed — read input byte,
load from the constant table, add/sub, and/or/xor, shift, truncate, sign-extend. No loop, no
dispatcher, no handler table.
4. **Capture four artifacts together, from one trace.** They are only valid as a set:
- the program;
- the constant table and its base (program offsets are relative to it);
- the **output map** — which node byte lands at which output offset;
- the **output template** — the fixed characters the transform does not compute.
The map and the template are what people skip. Without them the transform is right and the string
is wrong, which looks like an off-by-one in the algorithm and is not.
5. **Implement it in tens of lines.** A loop over `(op, args)` with a value array. No emulator, no JNI,
no device — a function you can call at unit-test speed.
**Version binding.** The program is a property of one build. Store the library's fingerprint (size and
build id) beside it and re-extract on upgrade: the tables and the map move even when the algorithm
does not. Say which build a program came from; an extract with no fingerprint is not reproducible.
**Validation is per node, not per output.** Compare node values at several checkpoints against a
device run. A node that is right in value but wrong in width (32-bit truncation where the VM kept 64)
passes the final comparison on some inputs and fails on others — §9's failure seen from the other
side.
**Boundary.** This route needs a deterministic, input-driven computation. If the VM folds in a device
id, a timestamp or a nonce, those become constants from one trace, and the honest claim is
"reproduces one device's answers", not "implements the algorithm". For a signer that is often exactly
enough, but the report has to say which claim it is making. (`unverified` — absorbed, not run.)
## 12. Where this repository's tools plug in
| Step | Tool |
|---|---|
@@ -194,3 +234,4 @@ falls out. That is the end state this file walks toward: the guest program as da
- [ ] operand slicing seeded on all STOREs; byte provenance (not value) decides folding
- [ ] clean re-VM runs real bytecode against trace ground truth before any IR is trusted
- [ ] labels: write-up methods `inferred`, shipped tools `observed` (host), no VM target run here
- [ ] if the deliverable is a callable signer/decoder: §11's four artifacts captured as a set, program pinned to a build fingerprint
+47
View File
@@ -1471,3 +1471,50 @@ signature consequences — `signature-derived-keys.md`), or by giving the loader
`.so` extracted from the APK and call it the module: those addresses belong to a copy the process
never mapped, so a breakpoint that does land is worse than one that does not. The full attach sequence
is in `ida-so-debugging.md`.
## P49. A reimplemented signer is byte-wrong, and the defect is in the extraction
**Symptom**
A reproduced signer emits a signature of the right shape and length and the server rejects it. Every
intermediate value looks sane; the difference is at the tail. The working hypothesis becomes "there is
a final MAC", then "a truncated hash", then "a per-version tweak", and each one gets its own day.
**Root cause**
The input to the pipeline is not byte-identical to the device's. The dump read two bytes past the real
buffer, or a length field in the container lies, or the buffer was copied at an unaligned offset. One
wrong byte early makes every later stage wrong, and the result still has the right shape.
**Why it is hard to see**
A wrong algorithm and a wrong input produce the same class of output: plausible, wrong, deterministic.
Byte-level total failure fits every hypothesis, so there is unlimited room for confirmation bias. And
the defect lives in memory, where you cannot diff it the way you diff two files.
**The rule**
**Prove each stage's input is byte-identical — length, alignment, encoding — before touching the
algorithm.** Compare stage inputs against the device, not final outputs; a stage-input diff localises
in minutes what an output diff cannot localise at all. P42 is the decoder-desync cousin; this one is
an extraction length.
## P50. A "custom cipher" is a wrapper that cancels
**Symptom**
Reimplementing an odd transform produces output that is either double-transformed or exactly what the
standard primitive would have produced — the transform appears to do nothing, which reads as a missing
step rather than an extra one.
**Root cause**
An involution applied symmetrically. `(x & ~m) | (~x & m)` is `x ^ m` for a mask whose bits partition
the word; called on both the encrypt and the decrypt path, it cancels. A fixed keyless permutation is
the other member of the family: an encoding dressed as encryption.
**Why it is hard to see**
The function name and the bit arithmetic read as custom crypto, and the wrapper sits between you and
the primitive, so reading either alone misleads. A no-op transform is not something you expect to find
*inside* a protected algorithm, so it gets modelled as key mixing and carried through every later
step.
**The rule**
**Compose it with itself before reimplementing it** (`f(f(x)) == x`), and check whether the caller
invokes it on both paths. If it cancels, drop it and recover the standard primitive underneath; keep
the wrapper in the report, because it is the reason the algorithm looked unfamiliar. The method, with
the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7.
@@ -369,7 +369,63 @@ frame's dependency set, and the envelope decodes stably across several samples.
is still "which state did this frame consume", no amount of body decoding helps — go capture the
missing family first.
## 7. Failure modes
## 7. Encrypted bodies — classify the primitive before you implement it
**Load this when** the body is opaque: a long base64/hex field in the query, a binary request body, a
fixed prefix on every response — and the transform lives in a native library or a helper class.
Before any static work, three black-box experiments on captured pairs, each a few minutes. You need
several requests whose plaintext you control.
1. **Header length.** Encrypt two different payloads under the same key and decode both. A constant
first N bytes ⇒ a key-derived header. A prefix that changes when the key changes ⇒ keyed, not
random. A prefix that changes per request ⇒ a per-message IV or counter.
2. **Prefix XOR.** Encrypt `P` and `P || X`, then XOR the two ciphertexts. Zeros over the common
prefix ⇒ a stream cipher whose keystream depends on the key alone. No such relation, plus a length
rounded up to a block size ⇒ a block cipher. Identical 16-byte blocks recurring across messages ⇒
ECB, which is a finding on its own.
3. **Length law.** `len(c) = len(p) + k` ⇒ stream/CTR. `len(c) = ceil((len(p)+1)/b) * b` ⇒ block with
padding. A constant relation with a compressed prefix ⇒ compress-then-encrypt, and the plaintext
you are looking for is the compressor's output, not the JSON.
The classification decides what you implement: a stream cipher needs its keystream and nothing else;
a block cipher needs its mode; ECB needs no key schedule at all to explain structure. Skipping this
step is how a standard construction gets reimplemented as something exotic.
### A custom-looking transform may be a wrapper that cancels
Wrapper arithmetic is cheaper to check than to rewrite. Two shapes cover most of what looks strange:
- **Involution applied symmetrically.** A transform with `f(f(x)) == x` that is called on the way in
*and* on the way out contributes nothing. The common disguise is a mask pair:
`(x & ~m) | (~x & m)` is exactly `x ^ m` when the bits of `m` partition the word. If both encrypt
and decrypt call it, drop it from your model and recover the standard primitive underneath.
- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is an
encoding, not encryption. Index it out and compare against known S-boxes.
Test the composition before reimplementing: apply the function twice, and check whether the caller
uses it twice. If it cancels, the algorithm is standard and the reason it looked unfamiliar is now
written down.
### The last mile is encoding, not crypto
Two non-cryptographic differences fail signatures that are otherwise correct. Both are invisible in
the algorithm and both are cheap to eliminate:
- **The client's encoder may be Java's.** `URLEncoder` writes a space as `+`, leaves `*`, `-`, `_`,
`.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's `quote` does the
opposite on `~` and uses `%20`. Reimplement what the client does and check it against one captured
parameter value, not against documentation.
- **Compression bytes are implementation-defined.** Two libraries can emit different, equally valid
compressed output for the same input. Whether that breaks you is a property of the **server**: if it
decompresses and checks a CRC or the plaintext, byte equality is not required; if it compares the
stored blob, it is. One mutation tells you which, and the answer decides whether byte-perfect
reproduction is even the goal.
**Boundary.** None of the three experiments needs the `.so`; that is the point. When the keystream is
derived per message from a device secret, the native route comes back and §5's attribution applies.
## 8. Failure modes
**A decode that looks valid but is not.** protobuf is self-synchronising enough that random bytes
often decode into a plausible-looking field list. Guard against it by (a) decoding several samples of
@@ -141,6 +141,36 @@ it also fails, the problem is environmental (`references/tls-and-cert.md`,
- **Certificate pinning to the app's own signature** is a different problem: it is verified,
not used as a key. See `references/tls-and-cert.md`.
## Variant: the value is an input to the digest, not the key
A different failure with the same source. A signer is reconstructed from static analysis and produces
a signature the server refuses, while every ingredient visible in the request looks right. The formula
turns out to be `sig = prefix + H(input || salt || key)`, and `salt` never appears in the request — it
is generated at init from the certificate and cached in a global.
**Why it defeats black-box search: the search space is missing an argument.** Trying MD5/HMAC/SHA
combinations over the visible parameters cannot find a value that is concatenated inside the digest and
never transmitted. The symptom is *uniform* failure, not near-misses — that is the tell, and it is
also why this is misread as "the algorithm is not fully recovered".
How to get it:
- **Find the buffer.** It is written once (during `JNI_OnLoad` or the first signature call), it is a
fixed-size array, and no exported function returns it. A memory read after init is enough.
- **Or hook the digest update** and dump its argument on one request; the salt is the constant run
inside an otherwise changing buffer.
- **Verify with three captured triples** that vary one field each. A formula that matches all three is
the formula; one that matches one is a coincidence.
Consequence for the deliverable: the salt is per build and per certificate. A local signer carries the
captured constant plus a build fingerprint (the same discipline as an extracted VM program,
`native-vm-recovery.md` §11), and the report says which build it came from. On update the value
changes and the signer must be re-derived — do not present the constant as an algorithm.
(`unverified`: the certificate-derived cache is `inferred` from the same mechanism as the key case
above; nothing here was run against a target.)
## Cost of getting this wrong
Without this step the task stalls in the worst possible way: the artifact is correct, the