mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-17 15:14:24 +08:00
Absorb three write-ups on dump timing, native VM devirtualization and encrypted bodies
vm1: dump window at android_dlopen_ext entry, raw-syscall reader, zero-filled holes; predicate-first patching with tiered ordering; module-filtered signal attribution. vm2: devirtualization to a straight-line (op, args) program with constant table, output map and template; version binding; per-node validation. vm3: three black-box cipher probes, involution wrappers, encoding/compression drift; a certificate-derived salt inside the signature digest. Also adds pitfalls P49-P50, three capability rows (all partial/unverified), the evidence/route records and EXTENSION-ref-absorption-2026-10b.md. Renumbers native-vm-recovery section 11 to 12. Gates and pytest green.
This commit is contained in:
@@ -21,7 +21,7 @@ postings; no sample, tool output or target fact from them entered the repository
|
||||
| W1: full arm64 custom-VM recovery (long-form) | 0x00–0x1A method narrative | `references/native-vm-recovery.md` (existence signals, dispatcher separation, partial deflattening, relocation tables, PC proofs, descriptor/ISA cautions, clean-re-VM loop) |
|
||||
| W2: VMP interpreter recovery with trace-driven IR lifting | anchor/trace/lifting narrative | `references/native-vm-recovery.md` (register anchoring + triple verification, nested segment stacks, STORE-backward slicing, byte provenance, IR shape), `vm_handler_slice.py` nested-loop diagnostic |
|
||||
| W3: login-capture war story (packed + MADP/Weex + gomobile) | route-by-route复盘 | `references/pitfalls.md` P43–P46; the stack-ABI note in `references/hardware-watchpoints.md` |
|
||||
| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §11 — no new file |
|
||||
| W4: manual OLLVM deflattening | predicate algebra + IDA scripting | cross-checked against existing content; its MOV/MOVK constant-propagation state mapping is referenced as the static counterpart in `native-vm-recovery.md` §12 — no new file |
|
||||
|
||||
## 2. The one code change, and why — observed
|
||||
|
||||
@@ -54,7 +54,7 @@ spin-loop note on the real code path. Both regressions are now pinned by tests.
|
||||
plaintext boundary; a RegisterNative inventory is one run for a full method table).
|
||||
- W4's full IDA script: the technique is named (state-register MOV/MOVK constant propagation,
|
||||
CMP-site collection, predecessor walk past blocks with no CMP on the state register) and
|
||||
pointed at from `native-vm-recovery.md` §11; writing an IDA-independent implementation is a
|
||||
pointed at from `native-vm-recovery.md` §12; writing an IDA-independent implementation is a
|
||||
future pass, not an absorbed claim.
|
||||
- Every sample identity in all four sources: anonymized by the authors, kept anonymized here;
|
||||
`.ref/` (the local copies) is untracked and git-ignored in effect (never added).
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
# EXTENSION — second reference absorption from `ref/*` (2026-10-25)
|
||||
|
||||
This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into
|
||||
the files that already own the topics. It did not run any of them. The record exists so a later reader
|
||||
can tell what changed and on what footing.
|
||||
|
||||
## Sources
|
||||
|
||||
Three files, described by shape. Titles, authors, product names, package identifiers and every
|
||||
per-build address are omitted; none was copied onto the tracked surface.
|
||||
|
||||
| File | What it is |
|
||||
|---|---|
|
||||
| `vm1.md` | A dual-library commercial shell on ARM64: two SOs, one owning Java/ART checks, the other owning an `init_proc` scheduler with inline-shellcode `exit_group`, three watchdog threads and SMC. Covers dump timing, a syscall-level reader, single-predicate patching and a tiered patch order |
|
||||
| `vm2.md` | A white-box VM protecting one signing routine: threaded-code interpreter, tens of thousands of obfuscated nodes, and a devirtualization to a straight-line `(op, args)` program with constant table, output map and fixed template |
|
||||
| `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift |
|
||||
|
||||
All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing
|
||||
in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering
|
||||
judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was
|
||||
copied.
|
||||
|
||||
## What was added
|
||||
|
||||
No new reference files. Each mechanism was folded into the file that already owns its topic:
|
||||
|
||||
| File | Absorbs | Deliberately does not |
|
||||
|---|---|---|
|
||||
| `references/linker-and-soinfo-repair.md` §6.2–6.3 | the dump window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share it; the raw-syscall reader outside a self-hooked libc; zero-filling unreadable pages so segment offsets survive repair | claim that a dump taken at `android_dlopen_ext` entry is complete; name a specific reader |
|
||||
| `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker |
|
||||
| `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals |
|
||||
| `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections |
|
||||
| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required |
|
||||
| `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm |
|
||||
| `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method |
|
||||
|
||||
## Registration surface updated
|
||||
|
||||
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`,
|
||||
`whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all
|
||||
`partial` / `route_measured: false` / `unverified`; two existing rows
|
||||
(`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended
|
||||
rather than duplicated.
|
||||
- `references/evidence-summary.md`: three rows.
|
||||
- `references/coverage/route-inventory.md`: three rows.
|
||||
- `references/coverage-and-limits.md`: three absorbed-route bullets extended.
|
||||
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
|
||||
symptom mirror and the reference/script inventories had nothing to learn.
|
||||
|
||||
## Deliberately not absorbed
|
||||
|
||||
- **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte
|
||||
salt value, the signing prefix and template string, the package identifier, the shell's own
|
||||
`sub_*` names. They are only valid for one build and they carry identity.
|
||||
- **Performance and effort claims** ("1 ms", "days to hours") — no measurement exists here to anchor
|
||||
them.
|
||||
- **The AI-assisted-analysis narrative** in one source: it is advice about tool use, not a reversing
|
||||
mechanism, and it is not reproducible as written.
|
||||
- **Thread and watchdog tables as data.** The *method* (enumerate by probe, then prove the set closed)
|
||||
was absorbed; the table itself would invite applying one build's layout to another.
|
||||
|
||||
## What would change a label
|
||||
|
||||
One cheap experiment per row, all on a device this record never had:
|
||||
|
||||
- **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a
|
||||
disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record
|
||||
whether the process survives.
|
||||
- **Patch grading:** run the same target twice — kill sites only, then the predicate only — and show
|
||||
which one survives longer than the observed timer.
|
||||
- **Devirtualization:** extract one program, run it against a device call on ten inputs, and show
|
||||
per-node equality at three checkpoints.
|
||||
- **Cipher classification:** classify one body with the three probes, then reproduce it and have the
|
||||
server accept the request.
|
||||
|
||||
Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as
|
||||
support for a claim about a target.
|
||||
|
||||
## Gate status
|
||||
|
||||
`check_repo.py`, `check_refs.py`, `check_routing.py`, `check_commands.py`, `check_budget.py` and
|
||||
`pytest -q` were run after the change; the result is in the session record, not in this file. A green
|
||||
run proves the inventory resolves; it says nothing about whether the mechanisms work.
|
||||
@@ -250,7 +250,7 @@
|
||||
"docs/tool-verification/EXTENSION-benchmark-l1-l3.md section 3 (repository root, not shipped)",
|
||||
"tests/benchmark.md row B2 (repository root, not shipped)"
|
||||
],
|
||||
"limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed."
|
||||
"limits": "The tooling works; the patch does not. Writes landed 0.04-0.05 s after resume and survived detach, and the patched run and a read-and-write-back control both still died with SIGABRT at the same site: neutralising the terminate routine plus both of its branch inputs was insufficient, and the region is a dense cluster of mov w0,#imm ; ret stubs. Holding the process paused until the patch lands structurally cannot work, because a frozen process has no libraries mapped. On that target the pristine original already reports tamper, so no clean baseline exists there. No surviving-target result is claimed. Site selection is a separate claim: the predicate-first rule and the tiered grading (added 2026-10-25) are absorbed from a write-up and unverified here -- no graded patch set was run, and the ordering judgement is inferred."
|
||||
},
|
||||
{
|
||||
"capability": "native-instruction-tracing-stalker",
|
||||
@@ -654,7 +654,7 @@
|
||||
"evidence_not_shipped": [
|
||||
"docs/tool-verification/README.md (repository root, not shipped)"
|
||||
],
|
||||
"limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one."
|
||||
"limits": "The offline candidate path from an APK and the authoritative live read are both documented in the reference; neither has a public-target measurement here. The failure shape is a rebuilt APK that installs, launches, and then fails every signed request, which is easy to report as a server problem and is not one. A second variant -- a certificate-derived salt concatenated inside the digest, invisible to any black-box search -- is documented from the same 2026-10-25 write-up and likewise unmeasured."
|
||||
},
|
||||
{
|
||||
"capability": "blocked-react-native-ios",
|
||||
@@ -982,6 +982,54 @@
|
||||
],
|
||||
"limits": "Parsing of all three recovery sources (@DebugMetadata c=, @Metadata d2 first non-stdlib descriptor, jadx 'renamed from' comments), the third-party skip list and the map-file round trip are observed on embedded fixtures only. The upstream recovery-rate claims (30-50% of classes overall, ~100% of *Repository/*ViewModel/*Impl, ~80% of data-class DTOs) are unverified here: no real obfuscated Kotlin app was run through the script. Method and field names are never recovered; heavily inlined shapes can surface under a wrong primary class, so every mapping row is a hint. The script reads decompiled sources, not the dex: a decompiler that drops annotations removes the source entirely."
|
||||
},
|
||||
{
|
||||
"capability": "native-dump-window-and-syscall-reader",
|
||||
"group": "native",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "unverified",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/linker-and-soinfo-repair.md",
|
||||
"references/advanced-unpacking.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"ref/vm1.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a dual-library commercial shell"
|
||||
],
|
||||
"limits": "Operating rules only, absorbed 2026-10-25 and never exercised here: the dump window that closes inside JNI_OnLoad when self-decryption and watchdog startup share that function, the raw-syscall reader that stays outside a self-hooked libc, and zero-filling unreadable pages so segment offsets survive repair. The mechanisms are unverified; the ordering judgement (dump on android_dlopen_ext entry, before the protector runs) is inferred from the same source."
|
||||
},
|
||||
{
|
||||
"capability": "whitebox-crypto-devirtualization-to-program",
|
||||
"group": "native-dynamic",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "unverified",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/native-vm-recovery.md",
|
||||
"references/vmp-differential-analysis.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"ref/vm2.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a white-box VM protecting one signing routine"
|
||||
],
|
||||
"limits": "The route -- symbolic input, constant folding, loop and branch elimination, an (op, args) straight-line program plus its constant table, output map and output template -- is absorbed from one write-up and never run. It applies only to a deterministic, input-driven function: device id, time or nonce inputs become captured constants, and the claim then degrades to reproducing one device. The program is version-bound; no extract was produced here."
|
||||
},
|
||||
{
|
||||
"capability": "cipher-classification-by-differential-probe",
|
||||
"group": "protocol",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "unverified",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/protocol-reverse.md",
|
||||
"references/signature-derived-keys.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"ref/vm3.md (repository root, git-ignored, not shipped): 2026-10-25 write-up on a native signer and an encrypted request body"
|
||||
],
|
||||
"limits": "Three black-box probes (key-derived header length, prefix XOR for a keystream, the length law), the involution-wrapper algebra, and Java-versus-Python encoding plus compression drift. Absorbed from one write-up; no cipher was classified here, and no server was tested for whether it compares compressed bytes or only the decompressed payload."
|
||||
},
|
||||
{
|
||||
"capability": "websocket-long-connection-analysis",
|
||||
"group": "protocol",
|
||||
|
||||
@@ -114,7 +114,9 @@ its own strength note at the top.
|
||||
- **Instruction-level tracing and de-obfuscation** — the OLLVM shapes, a Stalker trace, the
|
||||
trace-to-CFG route, and the measured boundaries in `native-dbi-and-deobfuscation.md`.
|
||||
- **Protocol reversing beyond REST** — protobuf without a schema, gRPC frame capture, the QUIC/HTTP3
|
||||
limit, native-side certificate pinning (`protocol-reverse.md`).
|
||||
limit, native-side certificate pinning, and — for an opaque body — classifying the primitive by
|
||||
differential probes (header length, prefix XOR, the length law) before implementing it
|
||||
(`protocol-reverse.md`).
|
||||
- **What to do when userspace hooking provably cannot reach the check** — raw `svc` syscalls,
|
||||
`init_array`-early detection, and the kernel-route map with its version gate
|
||||
(`kernel-and-environment-hardening.md`).
|
||||
@@ -136,8 +138,9 @@ its own strength note at the top.
|
||||
(`scripts/vm_handler_slice.py`).
|
||||
- **Native VM recovery procedure** — the full method (dispatcher-layer separation, register
|
||||
anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM
|
||||
validation) absorbed from two public write-ups; every step `inferred`, none run here
|
||||
(`native-vm-recovery.md`).
|
||||
validation) plus a devirtualization end state that emits a runnable straight-line program with its
|
||||
constant table, output map and template; absorbed from public write-ups, every step `inferred` and
|
||||
none run here (`native-vm-recovery.md`).
|
||||
- **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition
|
||||
signals, the call-it-first route (AST restore + environment fill), stack- and register-machine
|
||||
reading procedures, AST instruction extraction with register/immediate typing, the four
|
||||
@@ -180,8 +183,10 @@ its own strength note at the top.
|
||||
- **Linker-level SO unpacking and `soinfo` repair** — the `dlopen` pipeline as a loader
|
||||
specification, `load_bias` and segment arithmetic including `seg_page_offset`, `DT_*` parsing,
|
||||
relocation targets and the defining-library-versus-caller bias, the minimal `soinfo` repair set,
|
||||
anti-dump header XOR detected by ELF invariants, and the dump window at `LoadSegments`. Source-
|
||||
reading synthesis plus a Frida cookbook; no loader was reimplemented here
|
||||
anti-dump header XOR detected by ELF invariants, the dump window at `LoadSegments`, the narrower
|
||||
window that closes inside `JNI_OnLoad` when self-decryption and watchdog startup share that
|
||||
function, a raw-syscall reader outside a self-hooked libc, and zero-filled unreadable pages.
|
||||
Source-reading synthesis plus two write-ups; no loader was reimplemented here
|
||||
(`linker-and-soinfo-repair.md`).
|
||||
- **VM protector analysis** — envelope triage, VM_DATA/VM_CONTEXT/vEIP/vStack anatomy, data-anchored
|
||||
breakpoints, forged-return recognition, trace-then-search-by-constant, virtual eflags as writable
|
||||
|
||||
@@ -61,6 +61,9 @@ route was not measured on this repository's evidence**. Treat them as leads with
|
||||
| Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` |
|
||||
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
|
||||
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
|
||||
| Native dump window and syscall reader | dump at `android_dlopen_ext` entry when self-decryption and watchdog startup share `JNI_OnLoad`; read via raw syscalls; zero-fill unreadable pages so offsets survive repair — absorbed, nothing run here — `linker-and-soinfo-repair.md` |
|
||||
| White-box crypto devirtualization | fold a deterministic VM transform into a straight-line `(op, args)` program plus constant table, output map and template, pinned to one build — absorbed, no extract produced — `native-vm-recovery.md` |
|
||||
| Cipher classification by differential probe | header length, prefix XOR and the length law separate stream from block before implementation; involution wrappers cancel; encoding/compression drift is the last mile — absorbed, never run — `protocol-reverse.md` |
|
||||
|
||||
## Dependencies this skill does not ship
|
||||
|
||||
|
||||
@@ -257,6 +257,27 @@ shipped library. A crash frame in JIT output cannot be mapped to a file offset.
|
||||
API where the framework compiles reflection at runtime, and treat "the frame has no file" as a fact
|
||||
about compilation, not as a hidden module.
|
||||
|
||||
### Filter signals by module before believing them
|
||||
|
||||
A process under instrumentation produces signals that are not the target's. ART takes an implicit
|
||||
`SIGSEGV` for null and bounds checks inside `boot*.oat`, `*.odex` and anonymous JIT pages, and resumes
|
||||
the thread in its own handler. A deliberate crash, a protector's kill and the runtime's own fault can
|
||||
all reach your observer looking identical: "signal 11".
|
||||
|
||||
The tuple that discriminates is **(signal, faulting address, faulting module)**, not the signal number.
|
||||
Count a signal only when the address resolves inside the library you are studying; log the rest and
|
||||
let them through untouched.
|
||||
|
||||
Two consequences:
|
||||
|
||||
- **Do not install a handler that swallows what the runtime expects to handle.** ART's implicit faults
|
||||
are how its null checks work; swallowing them turns a healthy process into a mysterious hang. A
|
||||
handler that intercepts signals has to forward anything whose faulting module is not the target.
|
||||
- **A `boot*.oat` or `odex` frame in a crash record is the runtime's, not the shell's.** Chasing it
|
||||
produces "the VM is corrupting memory" when the truth is that the VM's own fault handling ran.
|
||||
(`unverified` — a 2026-10 write-up on a dual-library shell; the discrimination is general.)
|
||||
|
||||
|
||||
### Stage 6: the precise stop, and the static alternative to it
|
||||
|
||||
**Input signal:** a named module + offset from Stage 2, Stage 3 or Stage 5. **Action:** neutralise
|
||||
|
||||
@@ -81,6 +81,9 @@ next** — and points at the one section that says "the full record is elsewhere
|
||||
| Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` |
|
||||
| React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` |
|
||||
| Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` |
|
||||
| Native dump window and syscall reader | Absorbed operating rules: dump at `android_dlopen_ext` entry because the window closes inside `JNI_OnLoad`, a raw-syscall reader outside a self-hooked libc, and zero-filled holes so repair keeps offsets. Never exercised here | partial | unverified | `references/linker-and-soinfo-repair.md` §6.2 |
|
||||
| White-box crypto devirtualization to a program | Absorbed route: symbolic input, constant folding, a straight-line `(op, args)` program plus its constant table, output map and template; version-bound, deterministic functions only. No extract was produced here | partial | unverified | `references/native-vm-recovery.md` §11 |
|
||||
| Cipher classification by differential probe | Absorbed method: header length, prefix XOR and the length law classify a stream or block primitive before implementation; involutions cancel; Java/Python encoding and compression drift decide the last mile. Never run here | partial | unverified | `references/protocol-reverse.md` §7 |
|
||||
|
||||
## Where the full record lives — repository root, not shipped
|
||||
|
||||
|
||||
@@ -13,7 +13,8 @@ source-reading write-ups (Android 5.0 and 8.x `bionic/linker`) and one Frida coo
|
||||
was reimplemented and no dump was produced here. Source citations are `unverified` against your
|
||||
tree until you open the matching AOSP tag. The *arithmetic* (bias, page offset, relocation target)
|
||||
is `inferred` from that source and should be checked with one control load before it is trusted.
|
||||
The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants.
|
||||
The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants. The dump-window and reader rules in §6.2–6.3 were added 2026-10 from a write-up
|
||||
on a dual-library commercial shell; they are `unverified` operating rules, not measurements.
|
||||
|
||||
## 1. The pipeline, as a specification for a loader
|
||||
|
||||
@@ -186,6 +187,43 @@ Deleting the header instead of XORing it is a weaker defence but a nastier sympt
|
||||
*cleanly* rejected, so the analyst blames the dump site. If the dump site is right and the header
|
||||
is missing, the correct conclusion is "the shell removed it", not "the dump was early".
|
||||
|
||||
### 6.2 The window can close inside `JNI_OnLoad`
|
||||
|
||||
The dump points above are ordered by *how much of the image exists*. A protector that modifies itself
|
||||
adds a second axis: *how long the process lives*. A shell that decrypts its own code in
|
||||
`JNI_OnLoad` (SMC) and starts watchdog threads from the same function leaves exactly one interval in
|
||||
which both hold — segments mapped, SMC not yet run. Dumping after `dlopen` returns is late twice
|
||||
over: the self-decryption has completed, and the watchdog is already armed by the time any
|
||||
post-return hook runs.
|
||||
|
||||
So: hook `android_dlopen_ext`, and dump in `onEnter` when the target path matches. Treat "the image
|
||||
is not relocated yet" as expected, and record the bias and relocation tables in the same pass
|
||||
(§2, §4). An unrelocated dump plus its bias is recoverable; an unrelocated dump without one is not.
|
||||
|
||||
A dump taken *inside* `JNI_OnLoad` is the worst of both: the code is complete, so the dump looks
|
||||
right, and the watchdog is up, so the process dies during the read. **If the target dies while you
|
||||
dump and only then, this is the shape — not a broken tool.** (Both rules `unverified`: absorbed from
|
||||
one write-up, not reproduced here.)
|
||||
|
||||
### 6.3 Read with syscalls, and pad the holes
|
||||
|
||||
Two properties of the reader decide whether the dump is usable.
|
||||
|
||||
- **The reader must not travel through the library the protector hooks.** A shell that inline-hooks
|
||||
libc's `open`/`read`/`write` sees your dump as one more caller and can refuse or terminate. A
|
||||
raw-syscall reader (`svc #0` from a small stub, or a kernel-side reader) sits outside that
|
||||
surface. This is `advanced-unpacking.md` §"Layered descent" applied to your own tooling: the layer
|
||||
you read from is also the layer the target can watch.
|
||||
- **An unreadable page is a hole to fill, not a range to skip.** Skipping shifts every later byte
|
||||
and produces a file that repairs into a plausible ELF with missing logic. Zero-filling preserves
|
||||
offsets and sizes, so segment placement still works; record which ranges were unreadable so the
|
||||
repair can be audited. A dump is evidence, and "these ranges were unreadable" belongs in it.
|
||||
|
||||
Then repair the image with a tool that rebuilds what the loader would have written (program headers,
|
||||
dynamic table, section headers). The repair consumes a file whose *offsets* are right; that is what
|
||||
the zero-fill protects.
|
||||
|
||||
|
||||
## 7. Timing: constructors and one-shot registrations
|
||||
|
||||
`dlopen` does not return until `CallConstructors` has run the dependencies' constructors, then the
|
||||
|
||||
@@ -159,6 +159,59 @@ terminate path is the end of the check; nothing downstream re-reads "did I die".
|
||||
rather than assuming: after a successful suppression the app must reach its **normal UI and stay
|
||||
there**, not loop through half-initialised states.
|
||||
|
||||
## Which site — and in what order
|
||||
|
||||
Patch sites are not interchangeable. Fixing the wrong one costs a day; fixing them in the wrong order
|
||||
costs the same day twice, because you cannot tell which change mattered. Sort the candidates first.
|
||||
|
||||
### One predicate may be deciding for all of them
|
||||
|
||||
Group the branches you found by the function whose return value they test. When every detector funnels
|
||||
into one `is_clean()`-shaped call, rewriting that return is the whole bypass: the kill sites are never
|
||||
reached, and the branches you did not enumerate are covered too.
|
||||
|
||||
Two conditions, both required:
|
||||
|
||||
- the function is a **decision** (returns a verdict), not the **action** (performs the kill). Patching
|
||||
an action changes the response; patching a decision changes the belief. The second is what makes a
|
||||
bypass stable.
|
||||
- you have checked its other callers. A predicate that also gates a feature will disable that feature
|
||||
once it always returns clean. Enumerate the callers before you touch it — the xref list in the
|
||||
disassembler, or the same caller enumeration `scripts/find_refs.py` does for dex methods.
|
||||
|
||||
### Grade the sites by what being wrong costs
|
||||
|
||||
| Tier | What it is | How to patch it | Why this order |
|
||||
|---|---|---|---|
|
||||
| **Tier 0** | terminate primitives: `exit_group`/`tgkill`/`kill` call sites, the shellcode executor, the unified kill point | replace with a no-op or a returning stub | until they stop, every other result is unobservable — the process dies first |
|
||||
| **Tier 1** | decision functions: the shared predicate, each scanner (`/proc` walk, `TracerPid` read, module checksum) | return the benign value *for that caller's branch* | they are the belief; a clean verdict also skips the report path |
|
||||
| **Tier 2** | auxiliary gates: adb/debug flags, telemetry, feature switches | only if observed to gate the thing you care about | unbounded work; most are inert in your environment |
|
||||
|
||||
The benign value is not always 0. Read the caller's branch: a "found something" scanner returns
|
||||
non-zero, a "list of hits" scanner returns NULL, a "should continue" predicate returns 1. Returning
|
||||
0 everywhere is a guess that happens to work half the time — and the failing half reads as a broken
|
||||
patch instead of a wrong constant.
|
||||
|
||||
### Count the watchdogs before declaring the set closed
|
||||
|
||||
One kill site is rarely the only thread. The threads are distinguishable by what they *read*, not by
|
||||
their names:
|
||||
|
||||
- a `/proc/self/task`, `/proc/self/fd` or `/proc/self/maps` walk — looking for your instrumentation;
|
||||
- `TracerPid`, the parent pid, or a task in state `T` — looking for a debugger or a suspended process;
|
||||
- a checksum over a loaded module — looking for a modified page.
|
||||
|
||||
Patch one and the app still dies, at a different delay, which reads as "the patch did not work" rather
|
||||
than "there were three". After the tier 0/1 set, the process must survive longer than the **longest**
|
||||
timer you have observed, not the first one.
|
||||
|
||||
### One tier per run
|
||||
|
||||
Patch tier 0, run, record; then tier 1; then tier 2 only if needed. A build with all tiers changed at once cannot
|
||||
tell you which one mattered, and the next failure is unattributable. This is the single-variable rule
|
||||
of `pitfalls.md` P21 applied to a patch set.
|
||||
|
||||
|
||||
## ELF hardening you will meet in these targets
|
||||
|
||||
Hardening libraries are not normal ELF files. Assume the following until measured otherwise.
|
||||
|
||||
@@ -171,7 +171,47 @@ pre-IP and post-IP are both recorded (`0x2 -> 0x3`), the 200k-line IR sequence *
|
||||
execution trace: +1 is fallthrough, jumps are branches, back-edges are loops — the guest CFG
|
||||
falls out. That is the end state this file walks toward: the guest program as data.
|
||||
|
||||
## 11. Where this repository's tools plug in
|
||||
## 11. The other end state: a program you can run
|
||||
|
||||
Sections 1–10 aim at a guest program you can **read**. When the interpreter implements one
|
||||
deterministic function — a request signer, a decoder, a checksum — the cheaper deliverable is a
|
||||
program you can **run**, and the handler semantics never need to be understood.
|
||||
|
||||
1. **Fix the boundary.** One native entry that takes a buffer in and produces a buffer out.
|
||||
Everything inside is the VM; everything outside is protocol.
|
||||
2. **Symbolic input, concrete everything else.** Seed the input bytes as symbols and run the VM (live,
|
||||
or over the recorded trace). Constant-fold every node whose value does not depend on them; delete
|
||||
loops and branches whose condition folds. What cannot fold is the part you must reproduce.
|
||||
3. **What remains is straight-line.** An SSA list of `(op, args)`, where `args` are constants or the
|
||||
indices of earlier nodes, and `op` is the ALU subset the real semantics needed — read input byte,
|
||||
load from the constant table, add/sub, and/or/xor, shift, truncate, sign-extend. No loop, no
|
||||
dispatcher, no handler table.
|
||||
4. **Capture four artifacts together, from one trace.** They are only valid as a set:
|
||||
- the program;
|
||||
- the constant table and its base (program offsets are relative to it);
|
||||
- the **output map** — which node byte lands at which output offset;
|
||||
- the **output template** — the fixed characters the transform does not compute.
|
||||
|
||||
The map and the template are what people skip. Without them the transform is right and the string
|
||||
is wrong, which looks like an off-by-one in the algorithm and is not.
|
||||
5. **Implement it in tens of lines.** A loop over `(op, args)` with a value array. No emulator, no JNI,
|
||||
no device — a function you can call at unit-test speed.
|
||||
|
||||
**Version binding.** The program is a property of one build. Store the library's fingerprint (size and
|
||||
build id) beside it and re-extract on upgrade: the tables and the map move even when the algorithm
|
||||
does not. Say which build a program came from; an extract with no fingerprint is not reproducible.
|
||||
|
||||
**Validation is per node, not per output.** Compare node values at several checkpoints against a
|
||||
device run. A node that is right in value but wrong in width (32-bit truncation where the VM kept 64)
|
||||
passes the final comparison on some inputs and fails on others — §9's failure seen from the other
|
||||
side.
|
||||
|
||||
**Boundary.** This route needs a deterministic, input-driven computation. If the VM folds in a device
|
||||
id, a timestamp or a nonce, those become constants from one trace, and the honest claim is
|
||||
"reproduces one device's answers", not "implements the algorithm". For a signer that is often exactly
|
||||
enough, but the report has to say which claim it is making. (`unverified` — absorbed, not run.)
|
||||
|
||||
## 12. Where this repository's tools plug in
|
||||
|
||||
| Step | Tool |
|
||||
|---|---|
|
||||
@@ -194,3 +234,4 @@ falls out. That is the end state this file walks toward: the guest program as da
|
||||
- [ ] operand slicing seeded on all STOREs; byte provenance (not value) decides folding
|
||||
- [ ] clean re-VM runs real bytecode against trace ground truth before any IR is trusted
|
||||
- [ ] labels: write-up methods `inferred`, shipped tools `observed` (host), no VM target run here
|
||||
- [ ] if the deliverable is a callable signer/decoder: §11's four artifacts captured as a set, program pinned to a build fingerprint
|
||||
|
||||
@@ -1471,3 +1471,50 @@ signature consequences — `signature-derived-keys.md`), or by giving the loader
|
||||
`.so` extracted from the APK and call it the module: those addresses belong to a copy the process
|
||||
never mapped, so a breakpoint that does land is worse than one that does not. The full attach sequence
|
||||
is in `ida-so-debugging.md`.
|
||||
|
||||
## P49. A reimplemented signer is byte-wrong, and the defect is in the extraction
|
||||
|
||||
**Symptom**
|
||||
A reproduced signer emits a signature of the right shape and length and the server rejects it. Every
|
||||
intermediate value looks sane; the difference is at the tail. The working hypothesis becomes "there is
|
||||
a final MAC", then "a truncated hash", then "a per-version tweak", and each one gets its own day.
|
||||
|
||||
**Root cause**
|
||||
The input to the pipeline is not byte-identical to the device's. The dump read two bytes past the real
|
||||
buffer, or a length field in the container lies, or the buffer was copied at an unaligned offset. One
|
||||
wrong byte early makes every later stage wrong, and the result still has the right shape.
|
||||
|
||||
**Why it is hard to see**
|
||||
A wrong algorithm and a wrong input produce the same class of output: plausible, wrong, deterministic.
|
||||
Byte-level total failure fits every hypothesis, so there is unlimited room for confirmation bias. And
|
||||
the defect lives in memory, where you cannot diff it the way you diff two files.
|
||||
|
||||
**The rule**
|
||||
**Prove each stage's input is byte-identical — length, alignment, encoding — before touching the
|
||||
algorithm.** Compare stage inputs against the device, not final outputs; a stage-input diff localises
|
||||
in minutes what an output diff cannot localise at all. P42 is the decoder-desync cousin; this one is
|
||||
an extraction length.
|
||||
|
||||
## P50. A "custom cipher" is a wrapper that cancels
|
||||
|
||||
**Symptom**
|
||||
Reimplementing an odd transform produces output that is either double-transformed or exactly what the
|
||||
standard primitive would have produced — the transform appears to do nothing, which reads as a missing
|
||||
step rather than an extra one.
|
||||
|
||||
**Root cause**
|
||||
An involution applied symmetrically. `(x & ~m) | (~x & m)` is `x ^ m` for a mask whose bits partition
|
||||
the word; called on both the encrypt and the decrypt path, it cancels. A fixed keyless permutation is
|
||||
the other member of the family: an encoding dressed as encryption.
|
||||
|
||||
**Why it is hard to see**
|
||||
The function name and the bit arithmetic read as custom crypto, and the wrapper sits between you and
|
||||
the primitive, so reading either alone misleads. A no-op transform is not something you expect to find
|
||||
*inside* a protected algorithm, so it gets modelled as key mixing and carried through every later
|
||||
step.
|
||||
|
||||
**The rule**
|
||||
**Compose it with itself before reimplementing it** (`f(f(x)) == x`), and check whether the caller
|
||||
invokes it on both paths. If it cancels, drop it and recover the standard primitive underneath; keep
|
||||
the wrapper in the report, because it is the reason the algorithm looked unfamiliar. The method, with
|
||||
the differential probes that identify the primitive underneath, is `protocol-reverse.md` §7.
|
||||
|
||||
@@ -369,7 +369,63 @@ frame's dependency set, and the envelope decodes stably across several samples.
|
||||
is still "which state did this frame consume", no amount of body decoding helps — go capture the
|
||||
missing family first.
|
||||
|
||||
## 7. Failure modes
|
||||
## 7. Encrypted bodies — classify the primitive before you implement it
|
||||
|
||||
**Load this when** the body is opaque: a long base64/hex field in the query, a binary request body, a
|
||||
fixed prefix on every response — and the transform lives in a native library or a helper class.
|
||||
|
||||
Before any static work, three black-box experiments on captured pairs, each a few minutes. You need
|
||||
several requests whose plaintext you control.
|
||||
|
||||
1. **Header length.** Encrypt two different payloads under the same key and decode both. A constant
|
||||
first N bytes ⇒ a key-derived header. A prefix that changes when the key changes ⇒ keyed, not
|
||||
random. A prefix that changes per request ⇒ a per-message IV or counter.
|
||||
2. **Prefix XOR.** Encrypt `P` and `P || X`, then XOR the two ciphertexts. Zeros over the common
|
||||
prefix ⇒ a stream cipher whose keystream depends on the key alone. No such relation, plus a length
|
||||
rounded up to a block size ⇒ a block cipher. Identical 16-byte blocks recurring across messages ⇒
|
||||
ECB, which is a finding on its own.
|
||||
3. **Length law.** `len(c) = len(p) + k` ⇒ stream/CTR. `len(c) = ceil((len(p)+1)/b) * b` ⇒ block with
|
||||
padding. A constant relation with a compressed prefix ⇒ compress-then-encrypt, and the plaintext
|
||||
you are looking for is the compressor's output, not the JSON.
|
||||
|
||||
The classification decides what you implement: a stream cipher needs its keystream and nothing else;
|
||||
a block cipher needs its mode; ECB needs no key schedule at all to explain structure. Skipping this
|
||||
step is how a standard construction gets reimplemented as something exotic.
|
||||
|
||||
### A custom-looking transform may be a wrapper that cancels
|
||||
|
||||
Wrapper arithmetic is cheaper to check than to rewrite. Two shapes cover most of what looks strange:
|
||||
|
||||
- **Involution applied symmetrically.** A transform with `f(f(x)) == x` that is called on the way in
|
||||
*and* on the way out contributes nothing. The common disguise is a mask pair:
|
||||
`(x & ~m) | (~x & m)` is exactly `x ^ m` when the bits of `m` partition the word. If both encrypt
|
||||
and decrypt call it, drop it from your model and recover the standard primitive underneath.
|
||||
- **A fixed permutation.** The same 256-byte table applied to every input with no key dependence is an
|
||||
encoding, not encryption. Index it out and compare against known S-boxes.
|
||||
|
||||
Test the composition before reimplementing: apply the function twice, and check whether the caller
|
||||
uses it twice. If it cancels, the algorithm is standard and the reason it looked unfamiliar is now
|
||||
written down.
|
||||
|
||||
### The last mile is encoding, not crypto
|
||||
|
||||
Two non-cryptographic differences fail signatures that are otherwise correct. Both are invisible in
|
||||
the algorithm and both are cheap to eliminate:
|
||||
|
||||
- **The client's encoder may be Java's.** `URLEncoder` writes a space as `+`, leaves `*`, `-`, `_`,
|
||||
`.` alone, and percent-encodes `~` as `%7E` with uppercase hex over UTF-8. Python's `quote` does the
|
||||
opposite on `~` and uses `%20`. Reimplement what the client does and check it against one captured
|
||||
parameter value, not against documentation.
|
||||
- **Compression bytes are implementation-defined.** Two libraries can emit different, equally valid
|
||||
compressed output for the same input. Whether that breaks you is a property of the **server**: if it
|
||||
decompresses and checks a CRC or the plaintext, byte equality is not required; if it compares the
|
||||
stored blob, it is. One mutation tells you which, and the answer decides whether byte-perfect
|
||||
reproduction is even the goal.
|
||||
|
||||
**Boundary.** None of the three experiments needs the `.so`; that is the point. When the keystream is
|
||||
derived per message from a device secret, the native route comes back and §5's attribution applies.
|
||||
|
||||
## 8. Failure modes
|
||||
|
||||
**A decode that looks valid but is not.** protobuf is self-synchronising enough that random bytes
|
||||
often decode into a plausible-looking field list. Guard against it by (a) decoding several samples of
|
||||
|
||||
@@ -141,6 +141,36 @@ it also fails, the problem is environmental (`references/tls-and-cert.md`,
|
||||
- **Certificate pinning to the app's own signature** is a different problem: it is verified,
|
||||
not used as a key. See `references/tls-and-cert.md`.
|
||||
|
||||
## Variant: the value is an input to the digest, not the key
|
||||
|
||||
A different failure with the same source. A signer is reconstructed from static analysis and produces
|
||||
a signature the server refuses, while every ingredient visible in the request looks right. The formula
|
||||
turns out to be `sig = prefix + H(input || salt || key)`, and `salt` never appears in the request — it
|
||||
is generated at init from the certificate and cached in a global.
|
||||
|
||||
**Why it defeats black-box search: the search space is missing an argument.** Trying MD5/HMAC/SHA
|
||||
combinations over the visible parameters cannot find a value that is concatenated inside the digest and
|
||||
never transmitted. The symptom is *uniform* failure, not near-misses — that is the tell, and it is
|
||||
also why this is misread as "the algorithm is not fully recovered".
|
||||
|
||||
How to get it:
|
||||
|
||||
- **Find the buffer.** It is written once (during `JNI_OnLoad` or the first signature call), it is a
|
||||
fixed-size array, and no exported function returns it. A memory read after init is enough.
|
||||
- **Or hook the digest update** and dump its argument on one request; the salt is the constant run
|
||||
inside an otherwise changing buffer.
|
||||
- **Verify with three captured triples** that vary one field each. A formula that matches all three is
|
||||
the formula; one that matches one is a coincidence.
|
||||
|
||||
Consequence for the deliverable: the salt is per build and per certificate. A local signer carries the
|
||||
captured constant plus a build fingerprint (the same discipline as an extracted VM program,
|
||||
`native-vm-recovery.md` §11), and the report says which build it came from. On update the value
|
||||
changes and the signer must be re-derived — do not present the constant as an algorithm.
|
||||
|
||||
(`unverified`: the certificate-derived cache is `inferred` from the same mechanism as the key case
|
||||
above; nothing here was run against a target.)
|
||||
|
||||
|
||||
## Cost of getting this wrong
|
||||
|
||||
Without this step the task stalls in the worst possible way: the artifact is correct, the
|
||||
|
||||
Reference in New Issue
Block a user