From 45ba13c0f3497092629a1af919444d192d2533c9 Mon Sep 17 00:00:00 2001 From: root Date: Sun, 25 Oct 2026 12:10:41 +0800 Subject: [PATCH] Absorb three write-ups on dump timing, native VM devirtualization and encrypted bodies vm1: dump window at android_dlopen_ext entry, raw-syscall reader, zero-filled holes; predicate-first patching with tiered ordering; module-filtered signal attribution. vm2: devirtualization to a straight-line (op, args) program with constant table, output map and template; version binding; per-node validation. vm3: three black-box cipher probes, involution wrappers, encoding/compression drift; a certificate-derived salt inside the signature digest. Also adds pitfalls P49-P50, three capability rows (all partial/unverified), the evidence/route records and EXTENSION-ref-absorption-2026-10b.md. Renumbers native-vm-recovery section 11 to 12. Gates and pytest green. --- .../EXTENSION-absorption-kx.md | 4 +- .../EXTENSION-ref-absorption-2026-10b.md | 83 +++++++++++++++++++ .../evidence/capability-matrix.json | 52 +++++++++++- .../references/coverage-and-limits.md | 15 ++-- .../references/coverage/route-inventory.md | 3 + .../references/detection-and-anti-analysis.md | 21 +++++ .../references/evidence-summary.md | 3 + .../references/linker-and-soinfo-repair.md | 40 ++++++++- .../references/native-tamper-and-suicide.md | 53 ++++++++++++ .../references/native-vm-recovery.md | 43 +++++++++- skills/apk-reverse/references/pitfalls.md | 47 +++++++++++ .../references/protocol-reverse.md | 58 ++++++++++++- .../references/signature-derived-keys.md | 30 +++++++ 13 files changed, 440 insertions(+), 12 deletions(-) create mode 100644 docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md diff --git a/docs/tool-verification/EXTENSION-absorption-kx.md b/docs/tool-verification/EXTENSION-absorption-kx.md index 5727c6c..26a0307 100644 --- a/docs/tool-verification/EXTENSION-absorption-kx.md +++ b/docs/tool-verification/EXTENSION-absorption-kx.md @@ -21,7 +21,7 @@ postings; no sample, tool output or target fact from them entered the repository | W1: full arm64 custom-VM recovery (long-form) | 0x00–0x1A method narrative | `references/native-vm-recovery.md` (existence signals, dispatcher separation, partial deflattening, relocation tables, PC proofs, descriptor/ISA cautions, clean-re-VM loop) | | W2: VMP interpreter recovery with trace-driven IR lifting | anchor/trace/lifting narrative | `references/native-vm-recovery.md` (register anchoring + triple verification, nested segment stacks, STORE-backward slicing, byte provenance, IR shape), `vm_handler_slice.py` nested-loop diagnostic | | W3: login-capture war story (packed + MADP/Weex + gomobile) | route-by-route复盘 | `references/pitfalls.md` P43–P46; the stack-ABI note in `references/hardware-watchpoints.md` | -| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §11 — no new file | +| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §12 — no new file | ## 2. The one code change, and why — observed @@ -54,7 +54,7 @@ spin-loop note on the real code path. Both regressions are now pinned by tests. plaintext boundary; a RegisterNative inventory is one run for a full method table). - W4's full IDA script: the technique is named (state-register MOV/MOVK constant propagation, CMP-site collection, predecessor walk past blocks with no CMP on the state register) and - pointed at from `native-vm-recovery.md` §11; writing an IDA-independent implementation is a + pointed at from `native-vm-recovery.md` §12; writing an IDA-independent implementation is a future pass, not an absorbed claim. - Every sample identity in all four sources: anonymized by the authors, kept anonymized here; `.ref/` (the local copies) is untracked and git-ignored in effect (never added). diff --git a/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md b/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md new file mode 100644 index 0000000..5c459b6 --- /dev/null +++ b/docs/tool-verification/EXTENSION-ref-absorption-2026-10b.md @@ -0,0 +1,83 @@ +# EXTENSION — second reference absorption from `ref/*` (2026-10-25) + +This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into +the files that already own the topics. It did not run any of them. The record exists so a later reader +can tell what changed and on what footing. + +## Sources + +Three files, described by shape. Titles, authors, product names, package identifiers and every +per-build address are omitted; none was copied onto the tracked surface. + +| File | What it is | +|---|---| +| `vm1.md` | A dual-library commercial shell on ARM64: two SOs, one owning Java/ART checks, the other owning an `init_proc` scheduler with inline-shellcode `exit_group`, three watchdog threads and SMC. Covers dump timing, a syscall-level reader, single-predicate patching and a tiered patch order | +| `vm2.md` | A white-box VM protecting one signing routine: threaded-code interpreter, tens of thousands of obfuscated nodes, and a devirtualization to a straight-line `(op, args)` program with constant table, output map and fixed template | +| `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift | + +All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing +in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering +judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was +copied. + +## What was added + +No new reference files. Each mechanism was folded into the file that already owns its topic: + +| File | Absorbs | Deliberately does not | +|---|---|---| +| `references/linker-and-soinfo-repair.md` §6.2–6.3 | the dump window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share it; the raw-syscall reader outside a self-hooked libc; zero-filling unreadable pages so segment offsets survive repair | claim that a dump taken at `android_dlopen_ext` entry is complete; name a specific reader | +| `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker | +| `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals | +| `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections | +| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required | +| `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm | +| `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method | + +## Registration surface updated + +- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`, + `whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all + `partial` / `route_measured: false` / `unverified`; two existing rows + (`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended + rather than duplicated. +- `references/evidence-summary.md`: three rows. +- `references/coverage/route-inventory.md`: three rows. +- `references/coverage-and-limits.md`: three absorbed-route bullets extended. +- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the + symptom mirror and the reference/script inventories had nothing to learn. + +## Deliberately not absorbed + +- **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte + salt value, the signing prefix and template string, the package identifier, the shell's own + `sub_*` names. They are only valid for one build and they carry identity. +- **Performance and effort claims** ("1 ms", "days to hours") — no measurement exists here to anchor + them. +- **The AI-assisted-analysis narrative** in one source: it is advice about tool use, not a reversing + mechanism, and it is not reproducible as written. +- **Thread and watchdog tables as data.** The *method* (enumerate by probe, then prove the set closed) + was absorbed; the table itself would invite applying one build's layout to another. + +## What would change a label + +One cheap experiment per row, all on a device this record never had: + +- **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a + disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record + whether the process survives. +- **Patch grading:** run the same target twice — kill sites only, then the predicate only — and show + which one survives longer than the observed timer. +- **Devirtualization:** extract one program, run it against a device call on ten inputs, and show + per-node equality at three checkpoints. +- **Cipher classification:** classify one body with the three probes, then reproduce it and have the + server accept the request. + +Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as +support for a claim about a target. + +## Gate status + +`check_repo.py`, `check_refs.py`, `check_routing.py`, `check_commands.py`, `check_budget.py` and +`pytest -q` were run after the change; the result is in the session record, not in this file. A green +run proves the inventory resolves; it says nothing about whether the mechanisms work. diff --git a/skills/apk-reverse/evidence/capability-matrix.json b/skills/apk-reverse/evidence/capability-matrix.json index 5d51e5b..c596869 100644 --- a/skills/apk-reverse/evidence/capability-matrix.json +++ b/skills/apk-reverse/evidence/capability-matrix.json @@ -250,7 +250,7 @@ "docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 3 (repository root, not shipped)", "tests/benchmark.md row B2 (repository root, not shipped)" ], - "limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed." + "limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed. Site selection is a separate claim: the predicate-first rule and the tiered grading (added 2026-10-25) are absorbed from a write-up and unverified here -- no graded patch set was run, and the ordering judgement is inferred." }, { "capability": "native-instruction-tracing-stalker", @@ -654,7 +654,7 @@ "evidence_not_shipped": [ "docs/tool-verification/README.md (repository root, not shipped)" ], - "limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one." + "limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one. A second variant -- a certificate-derived salt concatenated inside the digest, invisible to any black-box search -- is documented from the same 2026-10-25 write-up and likewise unmeasured." }, { "capability": "blocked-react-native-ios", @@ -982,6 +982,54 @@ ], "limits": "Parsing of all three recovery sources (@DebugMetadata c=, @Metadata d2 first non-stdlib descriptor, jadx 'renamed from' comments), the third-party skip list and the map-file round trip are observed on embedded fixtures only. The upstream recovery-rate claims (30-50% of classes overall, ~100% of *Repository/*ViewModel/*Impl, ~80% of data-class DTOs) are unverified here: no real obfuscated Kotlin app was run through the script. Method and field names are never recovered; heavily inlined shapes can surface under a wrong primary class, so every mapping row is a hint. The script reads decompiled sources, not the dex: a decompiler that drops annotations removes the source entirely." }, + { + "capability": "native-dump-window-and-syscall-reader", + "group": "native", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/linker-and-soinfo-repair.md", + "references/advanced-unpacking.md" + ], + "evidence_not_shipped": [ + "ref/vm1.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a dual-library commercial shell" + ], + "limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The mechanisms are unverified; the ordering judgement (dump on android_dlopen_ext entry, before the protector runs) is inferred from the same source." + }, + { + "capability": "whitebox-crypto-devirtualization-to-program", + "group": "native-dynamic", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/native-vm-recovery.md", + "references/vmp-differential-analysis.md" + ], + "evidence_not_shipped": [ + "ref/vm2.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a white-box VM protecting one signing routine" + ], + "limits": "The route -- symbolic input, constant folding, loop and branch elimination, an (op, args) straight-line program plus its constant table, output map and output template -- is absorbed from one write-up and never run. It applies only to a deterministic, input-driven function: device id, time or nonce inputs become captured constants, and the claim then degrades to reproducing one device. The program is version-bound; no extract was produced here." + }, + { + "capability": "cipher-classification-by-differential-probe", + "group": "protocol", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/protocol-reverse.md", + "references/signature-derived-keys.md" + ], + "evidence_not_shipped": [ + "ref/vm3.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a native signer and an encrypted request body" + ], + "limits": "Three black-box probes (key-derived header length, prefix XOR for a keystream, the length law), the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Absorbed from one write-up; no cipher was classified here, and no server was tested for whether it compares compressed bytes or only the decompressed payload." + }, { "capability": "websocket-long-connection-analysis", "group": "protocol", diff --git a/skills/apk-reverse/references/coverage-and-limits.md b/skills/apk-reverse/references/coverage-and-limits.md index e55412e..c7716db 100644 --- a/skills/apk-reverse/references/coverage-and-limits.md +++ b/skills/apk-reverse/references/coverage-and-limits.md @@ -114,7 +114,9 @@ its own strength note at the top. - **Instruction-level tracing and de-obfuscation** — the OLLVM shapes, a Stalker trace, the trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`. - **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3 - limit, native-side certificate pinning (`protocol-reverse.md`). + limit, native-side certificate pinning, and — for an opaque body — classifying the primitive by + differential probes (header length, prefix XOR, the length law) before implementing it + (`protocol-reverse.md`). - **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls, `init_array`-early detection, and the kernel-route map with its version gate (`kernel-and-environment-hardening.md`). @@ -136,8 +138,9 @@ its own strength note at the top. (`scripts/vm_handler_slice.py`). - **Native VM recovery procedure** — the full method (dispatcher-layer separation, register anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM - validation) absorbed from two public write-ups; every step `inferred`, none run here - (`native-vm-recovery.md`). + validation) plus a devirtualization end state that emits a runnable straight-line program with its + constant table, output map and template; absorbed from public write-ups, every step `inferred` and + none run here (`native-vm-recovery.md`). - **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition signals, the call-it-first route (AST restore + environment fill), stack- and register-machine reading procedures, AST instruction extraction with register/immediate typing, the four @@ -180,8 +183,10 @@ its own strength note at the top. - **Linker-level SO unpacking and `soinfo` repair** — the `dlopen` pipeline as a loader specification, `load_bias` and segment arithmetic including `seg_page_offset`, `DT_*` parsing, relocation targets and the defining-library-versus-caller bias, the minimal `soinfo` repair set, - anti-dump header XOR detected by ELF invariants, and the dump window at `LoadSegments`. Source- - reading synthesis plus a Frida cookbook; no loader was reimplemented here + anti-dump header XOR detected by ELF invariants, the dump window at `LoadSegments`, the narrower + window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share that + function, a raw-syscall reader outside a self-hooked libc, and zero-filled unreadable pages. + Source-reading synthesis plus two write-ups; no loader was reimplemented here (`linker-and-soinfo-repair.md`). - **VM protector analysis** — envelope triage, VM_DATA/VM_CONTEXT/vEIP/vStack anatomy, data-anchored breakpoints, forged-return recognition, trace-then-search-by-constant, virtual eflags as writable diff --git a/skills/apk-reverse/references/coverage/route-inventory.md b/skills/apk-reverse/references/coverage/route-inventory.md index baf04f3..2343eab 100644 --- a/skills/apk-reverse/references/coverage/route-inventory.md +++ b/skills/apk-reverse/references/coverage/route-inventory.md @@ -61,6 +61,9 @@ route was not measured on this repository's evidence**. Treat them as leads with | Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` | | Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` | | Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` | +| Native dump window and syscall reader | dump at `android_dlopen_ext` entry when self-decryption and watchdog startup share `JNI_OnLoad`; read via raw syscalls; zero-fill unreadable pages so offsets survive repair — absorbed, nothing run here — `linker-and-soinfo-repair.md` | +| White-box crypto devirtualization | fold a deterministic VM transform into a straight-line `(op, args)` program plus constant table, output map and template, pinned to one build — absorbed, no extract produced — `native-vm-recovery.md` | +| Cipher classification by differential probe | header length, prefix XOR and the length law separate stream from block before implementation; involution wrappers cancel; encoding/compression drift is the last mile — absorbed, never run — `protocol-reverse.md` | ## Dependencies this skill does not ship diff --git a/skills/apk-reverse/references/detection-and-anti-analysis.md b/skills/apk-reverse/references/detection-and-anti-analysis.md index 397dab1..a69e994 100644 --- a/skills/apk-reverse/references/detection-and-anti-analysis.md +++ b/skills/apk-reverse/references/detection-and-anti-analysis.md @@ -257,6 +257,27 @@ shipped library. A crash frame in JIT output cannot be mapped to a file offset. API where the framework compiles reflection at runtime, and treat "the frame has no file" as a fact about compilation, not as a hidden module. +### Filter signals by module before believing them + +A process under instrumentation produces signals that are not the target's. ART takes an implicit +`SIGSEGV` for null and bounds checks inside `boot*.oat`, `*.odex` and anonymous JIT pages, and resumes +the thread in its own handler. A deliberate crash, a protector's kill and the runtime's own fault can +all reach your observer looking identical: "signal 11". + +The tuple that discriminates is **(signal, faulting address, faulting module)**, not the signal number. +Count a signal only when the address resolves inside the library you are studying; log the rest and +let them through untouched. + +Two consequences: + +- **Do not install a handler that swallows what the runtime expects to handle.** ART's implicit faults + are how its null checks work; swallowing them turns a healthy process into a mysterious hang. A + handler that intercepts signals has to forward anything whose faulting module is not the target. +- **A `boot*.oat` or `odex` frame in a crash record is the runtime's, not the shell's.** Chasing it + produces "the VM is corrupting memory" when the truth is that the VM's own fault handling ran. + (`unverified` — a 2026-10 write-up on a dual-library shell; the discrimination is general.) + + ### Stage 6: the precise stop, and the static alternative to it **Input signal:** a named module + offset from Stage 2, Stage 3 or Stage 5. **Action:** neutralise diff --git a/skills/apk-reverse/references/evidence-summary.md b/skills/apk-reverse/references/evidence-summary.md index 235ef91..23ef3c9 100644 --- a/skills/apk-reverse/references/evidence-summary.md +++ b/skills/apk-reverse/references/evidence-summary.md @@ -81,6 +81,9 @@ next** — and points at the one section that says "the full record is elsewhere | Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` | | React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` | | Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` | +| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 | +| White-box crypto devirtualization to a program | Absorbed route: symbolic input, constant folding, a straight-line `(op, args)` program plus its constant table, output map and template; version-bound, deterministic functions only. No extract was produced here | partial | unverified | `references/native-vm-recovery.md` §11 | +| Cipher classification by differential probe | Absorbed method: header length, prefix XOR and the length law classify a stream or block primitive before implementation; involutions cancel; Java/Python encoding and compression drift decide the last mile. Never run here | partial | unverified | `references/protocol-reverse.md` §7 | ## Where the full record lives — repository root, not shipped diff --git a/skills/apk-reverse/references/linker-and-soinfo-repair.md b/skills/apk-reverse/references/linker-and-soinfo-repair.md index 4d1369d..fe3d12d 100644 --- a/skills/apk-reverse/references/linker-and-soinfo-repair.md +++ b/skills/apk-reverse/references/linker-and-soinfo-repair.md @@ -13,7 +13,8 @@ source-reading write-ups (Android 5.0 and 8.x `bionic/linker`) and one Frida coo was reimplemented and no dump was produced here. Source citations are `unverified` against your tree until you open the matching AOSP tag. The *arithmetic* (bias, page offset, relocation target) is `inferred` from that source and should be checked with one control load before it is trusted. -The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants. +The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants. The dump-window and reader rules in §6.2–6.3 were added 2026-10 from a write-up +on a dual-library commercial shell; they are `unverified` operating rules, not measurements. ## 1. The pipeline, as a specification for a loader @@ -186,6 +187,43 @@ Deleting the header instead of XORing it is a weaker defence but a nastier sympt *cleanly* rejected, so the analyst blames the dump site. If the dump site is right and the header is missing, the correct conclusion is "the shell removed it", not "the dump was early". +### 6.2 The window can close inside `JNI_OnLoad` + +The dump points above are ordered by *how much of the image exists*. A protector that modifies itself +adds a second axis: *how long the process lives*. A shell that decrypts its own code in +`JNI_OnLoad` (SMC) and starts watchdog threads from the same function leaves exactly one interval in +which both hold — segments mapped, SMC not yet run. Dumping after `dlopen` returns is late twice +over: the self-decryption has completed, and the watchdog is already armed by the time any +post-return hook runs. + +So: hook `android_dlopen_ext`, and dump in `onEnter` when the target path matches. Treat "the image +is not relocated yet" as expected, and record the bias and relocation tables in the same pass +(§2, §4). An unrelocated dump plus its bias is recoverable; an unrelocated dump without one is not. + +A dump taken *inside* `JNI_OnLoad` is the worst of both: the code is complete, so the dump looks +right, and the watchdog is up, so the process dies during the read. **If the target dies while you +dump and only then, this is the shape — not a broken tool.** (Both rules `unverified`: absorbed from +one write-up, not reproduced here.) + +### 6.3 Read with syscalls, and pad the holes + +Two properties of the reader decide whether the dump is usable. + +- **The reader must not travel through the library the protector hooks.** A shell that inline-hooks + libc's `open`/`read`/`write` sees your dump as one more caller and can refuse or terminate. A + raw-syscall reader (`svc #0` from a small stub, or a kernel-side reader) sits outside that + surface. This is `advanced-unpacking.md` §"Layered descent" applied to your own tooling: the layer + you read from is also the layer the target can watch. +- **An unreadable page is a hole to fill, not a range to skip.** Skipping shifts every later byte + and produces a file that repairs into a plausible ELF with missing logic. Zero-filling preserves + offsets and sizes, so segment placement still works; record which ranges were unreadable so the + repair can be audited. A dump is evidence, and "these ranges were unreadable" belongs in it. + +Then repair the image with a tool that rebuilds what the loader would have written (program headers, +dynamic table, section headers). The repair consumes a file whose *offsets* are right; that is what +the zero-fill protects. + + ## 7. Timing: constructors and one-shot registrations `dlopen` does not return until `CallConstructors` has run the dependencies' constructors, then the diff --git a/skills/apk-reverse/references/native-tamper-and-suicide.md b/skills/apk-reverse/references/native-tamper-and-suicide.md index 48df345..04302c3 100644 --- a/skills/apk-reverse/references/native-tamper-and-suicide.md +++ b/skills/apk-reverse/references/native-tamper-and-suicide.md @@ -159,6 +159,59 @@ terminate path is the end of the check; nothing downstream re-reads "did I die". rather than assuming: after a successful suppression the app must reach its **normal UI and stay there**, not loop through half-initialised states. +## Which site — and in what order + +Patch sites are not interchangeable. Fixing the wrong one costs a day; fixing them in the wrong order +costs the same day twice, because you cannot tell which change mattered. Sort the candidates first. + +### One predicate may be deciding for all of them + +Group the branches you found by the function whose return value they test. When every detector funnels +into one `is_clean()`-shaped call, rewriting that return is the whole bypass: the kill sites are never +reached, and the branches you did not enumerate are covered too. + +Two conditions, both required: + +- the function is a **decision** (returns a verdict), not the **action** (performs the kill). Patching + an action changes the response; patching a decision changes the belief. The second is what makes a + bypass stable. +- you have checked its other callers. A predicate that also gates a feature will disable that feature + once it always returns clean. Enumerate the callers before you touch it — the xref list in the + disassembler, or the same caller enumeration `scripts/find_refs.py` does for dex methods. + +### Grade the sites by what being wrong costs + +| Tier | What it is | How to patch it | Why this order | +|---|---|---|---| +| **Tier 0** | terminate primitives: `exit_group`/`tgkill`/`kill` call sites, the shellcode executor, the unified kill point | replace with a no-op or a returning stub | until they stop, every other result is unobservable — the process dies first | +| **Tier 1** | decision functions: the shared predicate, each scanner (`/proc` walk, `TracerPid` read, module checksum) | return the benign value *for that caller's branch* | they are the belief; a clean verdict also skips the report path | +| **Tier 2** | auxiliary gates: adb/debug flags, telemetry, feature switches | only if observed to gate the thing you care about | unbounded work; most are inert in your environment | + +The benign value is not always 0. Read the caller's branch: a "found something" scanner returns +non-zero, a "list of hits" scanner returns NULL, a "should continue" predicate returns 1. Returning +0 everywhere is a guess that happens to work half the time — and the failing half reads as a broken +patch instead of a wrong constant. + +### Count the watchdogs before declaring the set closed + +One kill site is rarely the only thread. The threads are distinguishable by what they *read*, not by +their names: + +- a `/proc/self/task`, `/proc/self/fd` or `/proc/self/maps` walk — looking for your instrumentation; +- `TracerPid`, the parent pid, or a task in state `T` — looking for a debugger or a suspended process; +- a checksum over a loaded module — looking for a modified page. + +Patch one and the app still dies, at a different delay, which reads as "the patch did not work" rather +than "there were three". After the tier 0/1 set, the process must survive longer than the **longest** +timer you have observed, not the first one. + +### One tier per run + +Patch tier 0, run, record; then tier 1; then tier 2 only if needed. A build with all tiers changed at once cannot +tell you which one mattered, and the next failure is unattributable. This is the single-variable rule +of `pitfalls.md` P21 applied to a patch set. + + ## ELF hardening you will meet in these targets Hardening libraries are not normal ELF files. Assume the following until measured otherwise. diff --git a/skills/apk-reverse/references/native-vm-recovery.md b/skills/apk-reverse/references/native-vm-recovery.md index baf3d19..4ce7eb2 100644 --- a/skills/apk-reverse/references/native-vm-recovery.md +++ b/skills/apk-reverse/references/native-vm-recovery.md @@ -171,7 +171,47 @@ pre-IP and post-IP are both recorded (`0x2 -> 0x3`), the 200k-line IR sequence * execution trace: +1 is fallthrough, jumps are branches, back-edges are loops — the guest CFG falls out. That is the end state this file walks toward: the guest program as data. -## 11. Where this repository's tools plug in +## 11. The other end state: a program you can run + +Sections 1–10 aim at a guest program you can **read**. When the interpreter implements one +deterministic function — a request signer, a decoder, a checksum — the cheaper deliverable is a +program you can **run**, and the handler semantics never need to be understood. + +1. **Fix the boundary.** One native entry that takes a buffer in and produces a buffer out. + Everything inside is the VM; everything outside is protocol. +2. **Symbolic input, concrete everything else.** Seed the input bytes as symbols and run the VM (live, + or over the recorded trace). Constant-fold every node whose value does not depend on them; delete + loops and branches whose condition folds. What cannot fold is the part you must reproduce. +3. **What remains is straight-line.** An SSA list of `(op, args)`, where `args` are constants or the + indices of earlier nodes, and `op` is the ALU subset the real semantics needed — read input byte, + load from the constant table, add/sub, and/or/xor, shift, truncate, sign-extend. No loop, no + dispatcher, no handler table. +4. **Capture four artifacts together, from one trace.** They are only valid as a set: + - the program; + - the constant table and its base (program offsets are relative to it); + - the **output map** — which node byte lands at which output offset; + - the **output template** — the fixed characters the transform does not compute. + + The map and the template are what people skip. Without them the transform is right and the string + is wrong, which looks like an off-by-one in the algorithm and is not. +5. **Implement it in tens of lines.** A loop over `(op, args)` with a value array. No emulator, no JNI, + no device — a function you can call at unit-test speed. + +**Version binding.** The program is a property of one build. Store the library's fingerprint (size and +build id) beside it and re-extract on upgrade: the tables and the map move even when the algorithm +does not. Say which build a program came from; an extract with no fingerprint is not reproducible. + +**Validation is per node, not per output.** Compare node values at several checkpoints against a +device run. A node that is right in value but wrong in width (32-bit truncation where the VM kept 64) +passes the final comparison on some inputs and fails on others — §9's failure seen from the other +side. + +**Boundary.** This route needs a deterministic, input-driven computation. If the VM folds in a device +id, a timestamp or a nonce, those become constants from one trace, and the honest claim is +"reproduces one device's answers", not "implements the algorithm". For a signer that is often exactly +enough, but the report has to say which claim it is making. (`unverified` — absorbed, not run.) + +## 12. Where this repository's tools plug in | Step | Tool | |---|---| @@ -194,3 +234,4 @@ falls out. That is the end state this file walks toward: the guest program as da - [ ] operand slicing seeded on all STOREs; byte provenance (not value) decides folding - [ ] clean re-VM runs real bytecode against trace ground truth before any IR is trusted - [ ] labels: write-up methods `inferred`, shipped tools `observed` (host), no VM target run here +- [ ] if the deliverable is a callable signer/decoder: §11's four artifacts captured as a set, program pinned to a build fingerprint diff --git a/skills/apk-reverse/references/pitfalls.md b/skills/apk-reverse/references/pitfalls.md index 5a39650..d0a17a6 100644 --- a/skills/apk-reverse/references/pitfalls.md +++ b/skills/apk-reverse/references/pitfalls.md @@ -1471,3 +1471,50 @@ signature consequences — `signature-derived-keys.md`), or by giving the loader `.so` extracted from the APK and call it the module: those addresses belong to a copy the process never mapped, so a breakpoint that does land is worse than one that does not. The full attach sequence is in `ida-so-debugging.md`. + +## P49. A reimplemented signer is byte-wrong, and the defect is in the extraction + +**Symptom** +A reproduced signer emits a signature of the right shape and length and the server rejects it. Every +intermediate value looks sane; the difference is at the tail. The working hypothesis becomes "there is +a final MAC", then "a truncated hash", then "a per-version tweak", and each one gets its own day. + +**Root cause** +The input to the pipeline is not byte-identical to the device's. The dump read two bytes past the real +buffer, or a length field in the container lies, or the buffer was copied at an unaligned offset. One +wrong byte early makes every later stage wrong, and the result still has the right shape. + +**Why it is hard to see** +A wrong algorithm and a wrong input produce the same class of output: plausible, wrong, deterministic. +Byte-level total failure fits every hypothesis, so there is unlimited room for confirmation bias. And +the defect lives in memory, where you cannot diff it the way you diff two files. + +**The rule** +**Prove each stage's input is byte-identical — length, alignment, encoding — before touching the +algorithm.** Compare stage inputs against the device, not final outputs; a stage-input diff localises +in minutes what an output diff cannot localise at all. P42 is the decoder-desync cousin; this one is +an extraction length. + +## P50. A "custom cipher" is a wrapper that cancels + +**Symptom** +Reimplementing an odd transform produces output that is either double-transformed or exactly what the +standard primitive would have produced — the transform appears to do nothing, which reads as a missing +step rather than an extra one. + +**Root cause** +An involution applied symmetrically. `(x & ~m) | (~x & m)` is `x ^ m` for a mask whose bits partition +the word; called on both the encrypt and the decrypt path, it cancels. A fixed keyless permutation is +the other member of the family: an encoding dressed as encryption. + +**Why it is hard to see** +The function name and the bit arithmetic read as custom crypto, and the wrapper sits between you and +the primitive, so reading either alone misleads. A no-op transform is not something you expect to find +*inside* a protected algorithm, so it gets modelled as key mixing and carried through every later +step. + +**The rule** +**Compose it with itself before reimplementing it** (`f(f(x)) == x`), and check whether the caller +invokes it on both paths. If it cancels, drop it and recover the standard primitive underneath; keep +the wrapper in the report, because it is the reason the algorithm looked unfamiliar. The method, with +the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7. diff --git a/skills/apk-reverse/references/protocol-reverse.md b/skills/apk-reverse/references/protocol-reverse.md index 61a004f..c030842 100644 --- a/skills/apk-reverse/references/protocol-reverse.md +++ b/skills/apk-reverse/references/protocol-reverse.md @@ -369,7 +369,63 @@ frame's dependency set, and the envelope decodes stably across several samples. is still "which state did this frame consume", no amount of body decoding helps — go capture the missing family first. -## 7. Failure modes +## 7. Encrypted bodies — classify the primitive before you implement it + +**Load this when** the body is opaque: a long base64/hex field in the query, a binary request body, a +fixed prefix on every response — and the transform lives in a native library or a helper class. + +Before any static work, three black-box experiments on captured pairs, each a few minutes. You need +several requests whose plaintext you control. + +1. **Header length.** Encrypt two different payloads under the same key and decode both. A constant + first N bytes ⇒ a key-derived header. A prefix that changes when the key changes ⇒ keyed, not + random. A prefix that changes per request ⇒ a per-message IV or counter. +2. **Prefix XOR.** Encrypt `P` and `P || X`, then XOR the two ciphertexts. Zeros over the common + prefix ⇒ a stream cipher whose keystream depends on the key alone. No such relation, plus a length + rounded up to a block size ⇒ a block cipher. Identical 16-byte blocks recurring across messages ⇒ + ECB, which is a finding on its own. +3. **Length law.** `len(c) = len(p) + k` ⇒ stream/CTR. `len(c) = ceil((len(p)+1)/b) * b` ⇒ block with + padding. A constant relation with a compressed prefix ⇒ compress-then-encrypt, and the plaintext + you are looking for is the compressor's output, not the JSON. + +The classification decides what you implement: a stream cipher needs its keystream and nothing else; +a block cipher needs its mode; ECB needs no key schedule at all to explain structure. Skipping this +step is how a standard construction gets reimplemented as something exotic. + +### A custom-looking transform may be a wrapper that cancels + +Wrapper arithmetic is cheaper to check than to rewrite. Two shapes cover most of what looks strange: + +- **Involution applied symmetrically.** A transform with `f(f(x)) == x` that is called on the way in + *and* on the way out contributes nothing. The common disguise is a mask pair: + `(x & ~m) | (~x & m)` is exactly `x ^ m` when the bits of `m` partition the word. If both encrypt + and decrypt call it, drop it from your model and recover the standard primitive underneath. +- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is an + encoding, not encryption. Index it out and compare against known S-boxes. + +Test the composition before reimplementing: apply the function twice, and check whether the caller +uses it twice. If it cancels, the algorithm is standard and the reason it looked unfamiliar is now +written down. + +### The last mile is encoding, not crypto + +Two non-cryptographic differences fail signatures that are otherwise correct. Both are invisible in +the algorithm and both are cheap to eliminate: + +- **The client's encoder may be Java's.** `URLEncoder` writes a space as `+`, leaves `*`, `-`, `_`, + `.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's `quote` does the + opposite on `~` and uses `%20`. Reimplement what the client does and check it against one captured + parameter value, not against documentation. +- **Compression bytes are implementation-defined.** Two libraries can emit different, equally valid + compressed output for the same input. Whether that breaks you is a property of the **server**: if it + decompresses and checks a CRC or the plaintext, byte equality is not required; if it compares the + stored blob, it is. One mutation tells you which, and the answer decides whether byte-perfect + reproduction is even the goal. + +**Boundary.** None of the three experiments needs the `.so`; that is the point. When the keystream is +derived per message from a device secret, the native route comes back and §5's attribution applies. + +## 8. Failure modes **A decode that looks valid but is not.** protobuf is self-synchronising enough that random bytes often decode into a plausible-looking field list. Guard against it by (a) decoding several samples of diff --git a/skills/apk-reverse/references/signature-derived-keys.md b/skills/apk-reverse/references/signature-derived-keys.md index f292a34..00f9501 100644 --- a/skills/apk-reverse/references/signature-derived-keys.md +++ b/skills/apk-reverse/references/signature-derived-keys.md @@ -141,6 +141,36 @@ it also fails, the problem is environmental (`references/tls-and-cert.md`, - **Certificate pinning to the app's own signature** is a different problem: it is verified, not used as a key. See `references/tls-and-cert.md`. +## Variant: the value is an input to the digest, not the key + +A different failure with the same source. A signer is reconstructed from static analysis and produces +a signature the server refuses, while every ingredient visible in the request looks right. The formula +turns out to be `sig = prefix + H(input || salt || key)`, and `salt` never appears in the request — it +is generated at init from the certificate and cached in a global. + +**Why it defeats black-box search: the search space is missing an argument.** Trying MD5/HMAC/SHA +combinations over the visible parameters cannot find a value that is concatenated inside the digest and +never transmitted. The symptom is *uniform* failure, not near-misses — that is the tell, and it is +also why this is misread as "the algorithm is not fully recovered". + +How to get it: + +- **Find the buffer.** It is written once (during `JNI_OnLoad` or the first signature call), it is a + fixed-size array, and no exported function returns it. A memory read after init is enough. +- **Or hook the digest update** and dump its argument on one request; the salt is the constant run + inside an otherwise changing buffer. +- **Verify with three captured triples** that vary one field each. A formula that matches all three is + the formula; one that matches one is a coincidence. + +Consequence for the deliverable: the salt is per build and per certificate. A local signer carries the +captured constant plus a build fingerprint (the same discipline as an extracted VM program, +`native-vm-recovery.md` §11), and the report says which build it came from. On update the value +changes and the signer must be re-derived — do not present the constant as an algorithm. + +(`unverified`: the certificate-derived cache is `inferred` from the same mechanism as the key case +above; nothing here was run against a target.) + + ## Cost of getting this wrong Without this step the task stalls in the worst possible way: the artifact is correct, the