Files
root ab22b225ff Hardware pass: hardware watchpoints, ETE differential tracing, kernel-side event injection
Add the layer under the software tooling, in the repository's shape: what runs
on a host is measured, and every device-bound or kernel-bound piece is a gated,
labelled template.

- scripts/hw_watch.py: arm64 hardware watchpoints/execution breakpoints. WCR/
  BCR encoders with the 8-byte-window and alignment rules (unit-tested to the
  canonical 0x1FFD/0x3F05), a self-contained on-device ptrace helper that
  compiles clean as a static aarch64 ELF (observed: cross gcc 16.2), adb
  orchestration, and hit decoding to module+offset. The stealth ledger is
  stated three times on purpose: the ptrace route leaves TracerPid visible.
- scripts/ete_capture.py: CoreSight ETE capture with the five gates (kernel
  era, coresight sysfs, cs_etm PMU, perf-with-CORESIGHT, permission), the
  perf/OpenCSD build recipe, and the per-thread record wrapper whose
  --control one-instruction run must pass before any real capture.
- scripts/trace_diff.py: differential reduction of two execution traces into
  first-divergence / divergent regions / blocks-unique-to-each-side.
  Producer-agnostic: Stalker BLK logs (raw or run_probe-wrapped) and decoded
  cs_etm address streams alike, host-tested on synthetic flattened-dispatcher
  fixtures.
- scripts/kernel_instrument_scaffold.py: KPM/LKM watchpoint + register-rewrite
  templates plus an honest userspace carrier; every kernel file carries a
  NOT COMPILED AND NOT LOADED banner that verify enforces, and the
  register-rewrite action is fenced as experimental in the template itself.
- references/hardware-watchpoints.md, references/ete-diff-tracing.md,
  references/kernel-event-injection.md: the mechanism, the gates, the traps,
  and the composition recipes (watch the key buffer; diff two runs; flip the
  decision at the trap), with hand-offs instead of duplication.
- Registration: 3 symptom rows (SKILL.md + routing mirror), 3 reference rows,
  4 script rows, coverage/route-inventory/evidence rows, capability-matrix
  entries, known-limitations entry, eval case 4, README trees and coverage
  paragraphs in both languages, and three EXTENSION evidence records stating
  exactly what was measured on this host (encoders, compile, fixtures,
  generate/verify) and what was not (no watchpoint armed, no cs_etm stream,
  nothing kernel-side compiled or loaded anywhere).

Hardware pass 2: SPE/BRBE crash forensics, address symbolization, VM de-virtualization

Calibrated against the two public obfuscator implementations the user supplied
(dpkg123/ollvm-aosp-toolchain: Hikari-family and Pluto-family flattening, the
xvmp IR virtualizer at 63fec41f/9b7cf75e; fetched read-only into
tools/_work/refs, git-ignored). Source facts observed, runtime inferred.

- scripts/trace_addr2sym.py: runtime addresses -> symbol + static offset,
  zero-dependency ELF64 (PT_LOAD vaddr<->file-off conversion, .dynsym/.symtab,
  stripped section+off fallback). Host-verified against a real cross-compiled
  aarch64 .so with readelf as the independent cross-check, plus a hand-built
  minimal ELF covering vaddr!=offset; accepts hw_watch hits, trace mod+off
  keys, or raw address lists. Fixes on the way: --base int coercion, and
  nearest-below symbol semantics for PCs in inter-symbol gaps.
- scripts/vm_handler_slice.py: slice an interpreter trace into handler
  segments (dispatcher detection = histogram top + predecessor diversity; the
  first 4x-margin rule refused the calibrated fixture itself and was replaced),
  signature clustering, open-tail handling, and the guest-opcode trajectory via
  an optional handler->opcode map. Calibrated against xvmp's documented shape:
  vm_exec(code,size,vm_args,num) stubs, boxed arguments observable before
  interpretation, 11 opcodes + 19 arithmetic sub-ops, const-materialization
  dispatch as a distinct segment.
- references/hardware-sampling-and-crash.md: SPE (data flow without a known
  address; PC + data address per sample), BRBE (branch records when a crash
  stack cannot be trusted), the crash loop (tombstone -> branch records ->
  symbolize), and what each adds to the OLLVM/VMP routes. All
  documented-and-gated: no SPE sample or BRBE record was captured here.
- Flattening-recipe corrections from source: Hikari keeps a frame-resident
  switchVar written through an indirect pointer with scramble32-permuted cases;
  Pluto stores switchVar/switchXor rolling-XOR so memory only ever holds noise
  -- hardware-watchpoints.md's state-variable recipe now distinguishes the
  families, and ete-diff-tracing.md carries the family table plus the xvmp
  shape. AntiDebugging injects raw svc inline asm with randomized fields, so
  byte signatures cannot survive it; semantic site-location can.
- Registration: 3 symptom rows (untrustworthy crash stack; runtime addresses
  needing symbols; interpreter loop reading as noise), reference/script rows,
  coverage/evidence/capability-matrix entries, README trees in both languages,
  and EXTENSION-sampling-crash.md recording exactly what was measured (real .so
  symbolization, slicer fixtures, the slicer's self-found dispatcher-rule bug)
  and what was not (no SPE/BRBE capture, no hardened binary ever traced).

Fix all PR #1 review findings by severity; un-entrench the tests that pinned them

Major (device-breaking / wrong results):
- hw_watch: match the arm64 user_hwdebug_state slot ABI (address register
  first, control second -- the swap armed nothing correctly)
- hw_watch: every post-seize error path now clears slots and detaches;
  PTRACE_O_EXITKILL turned an analysis failure into a SIGKILL of the target
- hw_watch: forward genuine target SIGTRAPs (raise/tgkill/int3); only ptrace
  event stops are consumed -- the reference's signal-preservation promise
- hw_watch: run checks push/build/chmod and the helper's exit status; a dead
  capture is RESULT=blocked, never ok-no-hits
- hw_watch: decode offsets from the module base, not each mapping's start
- ete_capture: --control now requires the control record to DECODE
  (perf script event lines), the documented gate, not just a zero exit
- ete_capture: -o before the -- workload separator in --cmd mode; all
  values shell-quoted into su -c; stale remote perf.data removed first;
  perf exit status checked (timeout's 124 is the normal window end)
- kernel scaffold: --set-reg restricted to x0..x30; the rewrite is gated by
  its own breakpoint event so a watchpoint hit cannot run it; LKM pins the
  task (get/put_task_struct) across registration and calls the four-argument
  register_user_hw_breakpoint; --break rows initialize .addr/.len/.bp_type
  (there is no .pc member); --id validated against the module-id grammar
- trace_addr2sym: exact module matching (path or basename, never substring);
  load bias aligned against the PT_LOAD (bias = start - (foff - p_offset)
  - p_vaddr); PCs outside the ELF span are refused instead of landing on the
  last symbol; --json stdout is a pure document (RESULT= in text mode only)
- trace_diff: block keys need 0x or >=8 hex digits (PIDs/counts no longer
  parse); vm_handler_slice: dispatcher needs predecessor diversity, no
  frequency-margin fallback; both JSON outputs pure (check_commands' rule)

Minor/nitpick:
- capability-matrix + evidence-summary: trace reduction status ok->partial
  (no real-target producer ran it; the file's own definition of ok)
- hardware-sampling-and-crash.md and the script examples carry --elf
- kernel-event-injection.md no longer claims the callback's first statement
  is a filter (it prints unfiltered; the filter is a required change)
- Elf64 caches symbol addresses for per-row bisect
- docstring coverage of the touched surface: 16.67% -> 100%

Tests: every json.loads workaround stripping RESULT= is gone; new regression
tests cover the slot ABI, release-on-error, SIGTRAP forwarding, non-first
mapping offsets, -o placement and su -c quoting, the control decode gate,
x31/--id refusals, the guarded rewrite, outside-ELF PCs and bias math,
short-token refusal and the dispatcher margin rule. The minimal-ELF fixture
now packs a full 8-field Elf64_Phdr (its 7-field pack silently shifted
p_memsz onto p_align). 739 passed; flake8/mypy and all five gates clean.

Address the follow-up review on 0bd8dfe (6 findings)

- ete_capture: two-layer quoting for the control-record, decode and cleanup
  commands -- shlex.quote spliced between literal single quotes broke out of
  su -c for any spaced output name; --seconds must be positive (timeout 0
  disables the timer), refused before any adb check
- kernel scaffold: --set-reg now refuses sp/pc as well (the template has no
  rewrite path for them; a generate that silently drops the action is the
  misleading kind of success); both templates arm with a sampling period
  (LKM via hw_breakpoint_init, KPM spells out sample_period=1) -- a zero
  period is a non-sampling event whose callback never fires: armed, silent
- trace_addr2sym: contains_vaddr tests each PT_LOAD's loaded span instead
  of the whole-file extent; a PC in the unmapped gap between segments is
  refused rather than named after the nearest symbol below
- trace_diff: a bare all-digit token is indistinguishable from a date or
  count, so block keys need 0x or a non-decimal hex digit; a date-only log
  stays refused as a capture failure

Tests: two-layer quoting with a spaced control name, --seconds refusal,
sp/pc in the rejection list, sampling-period assertions for both templates,
a two-segment gap fixture, and a date-only parser case. 743 passed; flake8,
mypy and all five gates clean.
2026-10-15 15:33:28 +08:00

4.3 KiB

Extension verification — ETE differential tracing (gates, reducer, differ)

Covers the hardware-layer pass that added scripts/ete_capture.py, scripts/trace_diff.py and references/ete-diff-tracing.md. Strength labels as this directory's README defines them.

Reference environment: Linux container host, Python 3.13, adb present with no device. No coresight-capable hardware exists in this environment at all; the reference device of the earlier passes (kernel 4.14.186) has the ETE gate closed permanently (observed there).

Headline: the gate table, the recipe and the capture wrapper are host-measured; the differential reducer is host-measured on synthetic flattened-dispatcher fixtures. No ETE trace was captured or decoded anywhere in this pass — the producer side is unverified and the --control run exists precisely so the next device says so in one command.


1. The differ, measured on a synthetic OLLVM shape — observed

Two fixture logs built to the flattened-dispatcher rhythm D R1 D R2 … — run A taking R5/R7 (wrong password), run B taking R3/R4 (right password), one log in raw stalker_trace.js grammar, one wrapped by run_probe.py ([ts] [ts] TRACE BLK …) to prove the parser survives both:

$ python skills/apk-reverse/scripts/trace_diff.py diff --a a.log --b b.log
run A: 8 events, 5 distinct blocks
run B: 8 events, 5 distinct blocks
shared block executions: 6   divergent regions: 2
first divergence: after run #5 (A) / #5 (B), 5/5 events in, last common block liba.so+0x1000
region 1 (replace)
  A: liba.so+0x6000        B: liba.so+0x4000
blocks unique to A ...: 0x6000, 0x8000      blocks unique to B ...: 0x4000, 0x5000

What the output pins, and why each number is the one the method needs:

  • the identical prefix (D R1 D R2, 5 events) is the shared machinery — exactly where parsing ends and the input-dependent path begins;
  • first divergence names the dispatcher entry whose state constant differed;
  • unique-to-each-side lists are the input-dependent block set, which is the deliverable the reference file's §4 interprets per obfuscation.

reduce on one side prints the histogram with the dispatcher as the wide-margin top (4 hits vs 1 for every real block) — the same shape stalker_report.py documents for Stalker logs, reached from a different producer.

2. Gate and capture plumbing — observed (to the no-device boundary)

gates prints the five-gate table; gates --device with adb present but no device degrades to five OPEN rows and RESULT=partial rather than a traceback — measured. capture --pid 1 without a device exits through the pull failure with RESULT=blocked and exit code 3 (no-device capability refusal, per the CLI contract) — measured. recipe prints the Termux and cross build paths for perf-with-CORESIGHT — inferred from the kernel's tools/perf CoreSight documentation and the OpenCSD project, not from a build performed here.

3. What this pass deliberately did not do

  • No perf-with-CORESIGHT was built (host or device): the recipe is a documented path, and a reader who builds it owes the control record (capture --control) before any real capture.
  • No decoded cs_etm stream was fed to trace_diff.py; the plain parser accepts the address-bearing lines perf script emits, but that exact pairing is unverified.
  • No claim about ETE's runtime overhead or its visibility to targets was measured. The reference states the mechanism (no code modification, no ptrace) and refuses to go further.

4. Facts this pass pinned for the next one

  • The differ is producer-agnostic by design: the same command answers Stalker logs today and cs_etm decode output tomorrow. That is why the diff subcommand ships now, not after the first ETE capture — it is useful the day a Stalker pair exists.
  • The five gates each fail silently downstream; the reference's table says which silence each gate produces. The one-instruction control record is the only gate that exercises the whole pipe, which is why capture refuses to continue when it fails.
  • Run design is part of the tool contract, not user care: trace_diff.py refuses empty logs with a message that points at the zero-event diagnostics, because an empty trace is a capture failure wearing an analysis result's clothes.