apk-reverse/docs/tool-verification/EXTENSION-sampling-crash.md

# Extension verification — sampling, crash forensics, symbolization, de-virtualization

Covers the second hardware-layer pass: `references/hardware-sampling-and-crash.md`,
`scripts/trace_addr2sym.py`, `scripts/vm_handler_slice.py`, and the source-reading
corrections to the flattening recipes. Strength labels as this directory's README defines
them.

Reference environment: Linux container host, Python 3.13, cross `gcc 16.2.0` targeting
`aarch64-linux-gnu` (the toolchain that makes the symbolizer's "real ELF" half testable).
Network access was used once, read-only, to fetch the two public obfuscator implementations
the calibration cites (below). No Android device, no coresight/SPE/BRBE hardware.

**Headline: the address→symbol loop is measured against a real cross-compiled library; the
de-virtualization slicer is measured on synthetic traces AND end to end on a real
IR-virtualized + flattened binary (built by the completed toolchain, host-traced, sliced,
and fully opcode-named); the SPE/BRBE producer side remains documented-and-gated. The
flattening-recipe corrections are read from public source, and that source is named.**

---

## 1. `trace_addr2sym.py` against a real library — observed

```
$ gcc -shared -fPIC -O2 -o libtlib.so tlib.c      # cross gcc -> aarch64 ELF64
$ readelf --syms -W libtlib.so
... 00000000000005e0  132 FUNC GLOBAL DEFAULT 9 encrypt_block
... 0000000000000664    4 FUNC GLOBAL DEFAULT 9 decrypt_block
$ python skills/apk-reverse/scripts/trace_addr2sym.py --elf libtlib.so --offs 0x610
  0x610              -> encrypt_block+0x30  [vaddr 0x610 file 0x610 .text+0x130]
```

The resolution agrees with `readelf` exactly (0x610 − 0x5e0 = 0x30, inside the sized symbol);
a point outside any function falls to the nearest-below symbol with its section shown
(`0x700 -> _fini+0x98 [.eh_frame+0x40]`), which is GNU-addr2line-shaped behaviour and is
pinned by unit test together with the PT_LOAD `vaddr≠offset` conversion on a hand-built
minimal ELF. That minimal-ELF fixture also covers the stripped-library fallback
(section+offset, no symbols).

## 2. `vm_handler_slice.py` on the calibrated shape — observed

A synthetic interpreter trace in the shape the public IR virtualizer emits
(dispatcher D, handler blocks, one const-materialization segment, an unclosed tail):

```
$ python skills/apk-reverse/scripts/vm_handler_slice.py --log vm.log --opcode-map map.json
dispatcher: libvm.so+0x1000  (41.7% of 24 blocks, 10 hits)
segments: 10 total, 7 unique, avg len 2.00   [last segment UNCLOSED: exit path or truncated window]
opcode trajectory (first 20): OP_LOAD OP_LOAD OP_STORE OP_BR OP_LOAD OP_LOAD OP_BR OP_CALL OP_LOAD OP_RET
```

What the output pins, and why each part is the part a de-virtualization needs: the
**dispatcher share** (with the diagnostic that separates a real dispatch loop from a spin
loop), the **segment table** (one row per handler chain, first block = handler entry =
the `--opcode-map` key), the **trajectory** (the guest program, one signature per executed
guest opcode), the **OP_CALL materialization segment** appearing as its own signature (the
virtualizer's const dispatch — the second observation point the calibration names), and the
**UNCLOSED tail** flagged rather than silently dropped. The auto-detection rule is
top-of-histogram **plus predecessor diversity** (≥3 distinct predecessors), not a naive
count margin: in a stack-shaped VM the hottest handler executes on every guest instruction,
exactly as often as the dispatcher. That refusal-to-accept is the load-bearing part of the
auto rule. Section 6 measures all of this against a real hardened binary.

## 3. The flattening-recipe corrections — observed source, inferred runtime

Fetched read-only from the public implementations the user supplied
(github.com/dpkg123/ollvm-aosp-toolchain: branch `ollvm-r614150-clang23-rebase+omp`
[Hikari family] and `ollvm-r614150-clang23+omp` [Pluto family]; commits 63fec41f and
9b7cf75e for the IR virtualizer; 2026-10):

- **Hikari `Flattening.cpp`**: `switchVar` is an entry-block **alloca** written through a
  `switchVarAddr` double pointer (indirect store per transition); case values are
  `scramble32`-permuted; EH-bearing functions are skipped.
- **Pluto `Flattening.cpp`**: two allocas (`switchVar`, `switchXor`) with a **rolling XOR**
  (`enc^=delta; key^=delta; cond=enc^key` per dispatch); case values fully random; the true
  case exists only in the `switchCond` register for one instruction; EH/`callbr`/`indirectbr`
  functions skipped.
- **Hikari `AntiDebugging.cpp`**: on Darwin/AArch64 it injects **inline-assembly raw `svc`**
  with a randomized register-setup order and a **randomized svc immediate**; elsewhere it
  links precompiled anti-debug IR. Consequence recorded in the reference: byte signatures
  cannot survive the randomization; semantic site-location (`svc_scan.py`) and hardware
  observation do not care.
- **The IR virtualizer**: `vm_exec(code, size, vm_args, num)` stubs, boxed-argument array,
  11 one-byte opcodes + 19 arithmetic sub-ops, const/call materialization dispatch,
  switch-to-if lowering.

These corrected `hardware-watchpoints.md`'s state-variable recipe (the old text assumed a
global and a readable value — wrong for both families as shipped) and added the
family-difference table to `ete-diff-tracing.md` §5. The fetches live under
`tools/_work/refs/` (git-ignored); **runtime behaviour of these passes is `inferred` from
source** — except the IR virtualizer's runtime, which this pass built, executed and
measured (§5–§6).

## 4. What this pass deliberately did not do

- No SPE capture, no BRBE record read, no `perf -j any` on any device: those routes ship as
  documented-and-gated, and the reference's §5 failure modes say exactly how each gate
  fails silently.
- No hardened *closed-source* VM was sliced: the fixture in §6 is an open, self-built
  virtualizer; against a closed VMP the same slicing yields the trajectory, and the
  opcode map becomes a manual labelling task over the (usually few dozen) unique segments —
  which is exactly the workflow the script documents.

## 5. The xvmp toolchain, completed and verified — observed (workbench clone)

The public tree ships the compiler half of its IR virtualizer only, and that half was
never compiled: no TU includes the virtualizer headers, and on dev trees they do not build.
Three facts to keep when working with that tree:

- `getOrCreateVMExec` emits a **weak stub with no implementation** when `vm_exec` is absent
  — a build from that tree as-is links and then falls into the empty stub at runtime.
- Unsupported IR is **skipped with an `errs()` line and no failure** — "compiled clean" is
  not "every function was rewritten": read stderr for `[irvm] skip` lines, or a silently
  skipped function stays native and quietly skews a trace pair.
- The companion write-up (vmp.md) describes a **different, earlier prototype** — 32-bit
  fixed-width ISA, thunk bridges, an LVMP container — none of which is in the tree. Trust
  the code, not the article.

This repository completed the missing half. On the workbench clone of that tree (SSH
environment, 64-core), all four `omp` branches now carry:

- **Pass wiring**: `IRVM.cpp/.h` (new-PM pass beside the obfuscation manager, enabled by
  `-mllvm -irobf-irvm`, functions opt in via `section(".irvm")`) — the first TU that ever
  compiled the virtualizer headers, which required fixing per-tree upstream breakage (the
  revived `ADT/Optional.h` lacks `NoneType`; `DataLayout(Module*)` removed; `Br` split into
  `UncondBr`/`CondBr` on newer trees; `CastOps` renumbered — `FPExt` moved 17→47; the
  Hikari `compat/CallSite.h` called `DenseMapInfo` members dev-main deleted). One branch
  additionally had an **ungated** `IRVMModuleTool` call at the top of
  `Obfuscation::runOnModule` — plain native builds rewrote `.irvm` functions into thunks
  over the weak stub (and native -O1/-O2 escaped only because small functions get inlined
  away); removing it made IRVM strictly flag-gated.
- **FORMAT v3 bytecode**: 8-byte aligned zero-extended slot words; explicit
  `OP_LOAD/OP_STORE` access widths; src/dst bit widths plus a **stable CC_ kind enum** for
  `OP_CAST` (raw LLVM opcode numbers break across trees); operand width for `OP_CMP`;
  **per-index byte scales** for `OP_GEP` (a single element size mis-addresses the
  multi-index GEPs `-O0` emits verbatim). The branch condition is operand 0, never the
  last operand (that is the false-successor label).
- **Runtime**: `vm_exec.c` strong-symbol interpreter (23 hand-packed bytecode unit checks
  on host) and `VMP.h` user header (`IRVM_SECTION`, weak `handle_call` fallback,
  `VMP_TRACE=1` per-step interpreter trace).
- **Verification** (`test_irvm.c`: loops, globals, arrays via GEP, fpu, casts, external
  printf): virtualized vs native output **identical at -O0/-O1/-O2 on all four `omp`
  branches** (upstream commits `9b7cf75..ee23d66`, `cb1e2c1..63f100d`, `8a85cce..ee23d66`,
  `b6b6578..7929a03`).

## 6. The fixture, built, traced, sliced, and named — measured (host)

End to end on the workbench clone with the §5 toolchain:

- **Sample** (`tools/_work/irvm_fixture/fixture_sample.c`): three `.irvm` functions —
  `checksum` (fnv1a loop over 32 bytes), `stats` (fpu + branches), `mix` (byte-shuffle
  casts) — each called 200× from main. `stats` stays native: at -O1 the optimizer's
  if-conversion produces a `select` before the IRVM pass sees it (the documented skip
  hazard; a `volatile` interlock does not survive either). The fixture ships 2/3
  virtualized — as-is, since that is the real workflow.
- **Hardened build**: `-O1 -mllvm -irobf-irvm -mllvm -enable-fco -mllvm -enable-bcfobf
  -mllvm -enable-subobf` plus `runtime/vm_exec.c`; output **identical to native**.
  This tree's Hikari flag is `-enable-fco` (not `-enable-fla`). `handle_call` needs no
  user-side bridge — the pass emits the real const-materialization dispatch into the
  module — but only when at least one function was rewritten: with all candidates inlined
  away (-O1 + single call site) the link fails on the weak `vm_exec` reference chain.
  `noinline` on the `.irvm` functions keeps candidates (and the emitted `handle_call`)
  alive.
- **Guest ground truth**: `VMP_TRACE=1` → 17,961 `[vm] step=` lines (this build traces
  CAST/ARITH/CMP/RET steps only).
- **Host capture**: frida stalker does not work in these containers (measured on both a
  Docker host and an Android-kernel sandbox, frida 16.7.19 and 17.23.3): execution
  primitives pass (anonymous RWX mmap, W^X flip, memfd), the agent loads and runs the
  script, a ptrace syscall-error tracer around the injection tree shows **zero** denied
  syscalls — the failure is inside the injected agent's hook/trampoline machinery, past
  the syscall layer. Two operational notes: an outer ptrace tracer conflicts with frida
  spawn (frida ptrace-attaches the child it spawns → NotSupportedError — do not nest this
  repo's ptrace tooling and frida against the same tree); and the working host capture is
  a 33-line **ptrace single-stepper** (`stepper.c`): 24,582,699 `0x<rip>` lines in 36 s,
  filtered to the main image (23.97 M lines). On Android devices the stalker route keeps
  its own verification status per the skill's evidence rules.
- **Slicing**: `vm_handler_slice.py --dispatcher 0x55b98249c470` (manual — with an
  instruction-per-line stream the dispatcher block's instructions tie in count, so `auto`
  correctly refuses) → **95,184 segments, 18 unique signatures**, `RESULT=ok`. The
  diagnostics fire as designed: the tail stays open (exit path), and the nested-dispatch
  warning marks the outer fla state machine — one guest step crosses the chosen dispatcher
  ~5.3× (95,184 / 17,961), and the len-20 segment (83,286 hits) is the flat state-machine
  shuttle, not a handler.
- **Per-level separation and naming**: the 18 signatures split into the shuttle class and
  16 handler segments (~59 per round). No single host block fires once per guest step
  (fla scattered the switch into per-state entries), so the 200-count round-boundary
  segments split the trajectory into per-call rounds, and each round's signature sequence
  aligns positionally against the VMP_TRACE guest sequence. Result (round 2, all 59
  handler segments named):
  `loop1 x8: [ARITH_OR_SHL, OP_CAST, ARITH_ADD_CMP_latch]` →
  `LOOP_BOUNDARY_BR LOOP2_ENTRY_BR LOOP2_PROLOGUE` →
  `loop2 x8: [ARITH_OR, ARITH_SUB_INDEX, OP_CAST, ARITH11_ADD_CMP_latch]` → `OP_RET` —
  positionally isomorphic to the guest ops (`[or, shl, CAST, add, cmp]` / `[or, sub,
  CAST(zext), arith11, add, cmp]`, many ops per segment under fla), with the latch firing
  7/8 times (the last iteration exits via the boundary segments) and the segment counts
  matching the guest histograms exactly (3232/1600/1400/200). `--opcode-map` yields a
  fully named trajectory, zero unmapped (map at `tools/_work/irvm_fixture/opcode_map.json`).
- **Two findings that carry beyond this fixture**:
  1. `-O1` LICM **hoisted the pure virtualized call** out of the 200× driver loop —
     `checksum` executed once (`c == h*200 mod 2^32`). A virtualized function is not
     protected against being treated as pure by the optimizer that virtualized it; defeat
     hoisting with per-round argument changes when designing fixtures.
  2. `apply_map` originally keyed handler names by the segment's *first block* — under an
     instruction stream (or an outer flattening pass) every segment shares the
     dispatch-successor block, and only the signature distinguishes handlers. Signature
     keys are tried first; first-block keys remain supported.

Artifacts under `tools/_work/irvm_fixture/` (git-ignored): sample, stepper, the frida
attempt, guest trace, host excerpt, opcode map. The full host trace stays on the workbench
clone.