Hardware pass: hardware watchpoints, ETE differential tracing, kernel-side event injection

Add the layer under the software tooling, in the repository's shape: what runs
on a host is measured, and every device-bound or kernel-bound piece is a gated,
labelled template.

- scripts/hw_watch.py: arm64 hardware watchpoints/execution breakpoints. WCR/
  BCR encoders with the 8-byte-window and alignment rules (unit-tested to the
  canonical 0x1FFD/0x3F05), a self-contained on-device ptrace helper that
  compiles clean as a static aarch64 ELF (observed: cross gcc 16.2), adb
  orchestration, and hit decoding to module+offset. The stealth ledger is
  stated three times on purpose: the ptrace route leaves TracerPid visible.
- scripts/ete_capture.py: CoreSight ETE capture with the five gates (kernel
  era, coresight sysfs, cs_etm PMU, perf-with-CORESIGHT, permission), the
  perf/OpenCSD build recipe, and the per-thread record wrapper whose
  --control one-instruction run must pass before any real capture.
- scripts/trace_diff.py: differential reduction of two execution traces into
  first-divergence / divergent regions / blocks-unique-to-each-side.
  Producer-agnostic: Stalker BLK logs (raw or run_probe-wrapped) and decoded
  cs_etm address streams alike, host-tested on synthetic flattened-dispatcher
  fixtures.
- scripts/kernel_instrument_scaffold.py: KPM/LKM watchpoint + register-rewrite
  templates plus an honest userspace carrier; every kernel file carries a
  NOT COMPILED AND NOT LOADED banner that verify enforces, and the
  register-rewrite action is fenced as experimental in the template itself.
- references/hardware-watchpoints.md, references/ete-diff-tracing.md,
  references/kernel-event-injection.md: the mechanism, the gates, the traps,
  and the composition recipes (watch the key buffer; diff two runs; flip the
  decision at the trap), with hand-offs instead of duplication.
- Registration: 3 symptom rows (SKILL.md + routing mirror), 3 reference rows,
  4 script rows, coverage/route-inventory/evidence rows, capability-matrix
  entries, known-limitations entry, eval case 4, README trees and coverage
  paragraphs in both languages, and three EXTENSION evidence records stating
  exactly what was measured on this host (encoders, compile, fixtures,
  generate/verify) and what was not (no watchpoint armed, no cs_etm stream,
  nothing kernel-side compiled or loaded anywhere).

Hardware pass 2: SPE/BRBE crash forensics, address symbolization, VM de-virtualization

Calibrated against the two public obfuscator implementations the user supplied
(dpkg123/ollvm-aosp-toolchain: Hikari-family and Pluto-family flattening, the
xvmp IR virtualizer at 63fec41f/9b7cf75e; fetched read-only into
tools/_work/refs, git-ignored). Source facts observed, runtime inferred.

- scripts/trace_addr2sym.py: runtime addresses -> symbol + static offset,
  zero-dependency ELF64 (PT_LOAD vaddr<->file-off conversion, .dynsym/.symtab,
  stripped section+off fallback). Host-verified against a real cross-compiled
  aarch64 .so with readelf as the independent cross-check, plus a hand-built
  minimal ELF covering vaddr!=offset; accepts hw_watch hits, trace mod+off
  keys, or raw address lists. Fixes on the way: --base int coercion, and
  nearest-below symbol semantics for PCs in inter-symbol gaps.
- scripts/vm_handler_slice.py: slice an interpreter trace into handler
  segments (dispatcher detection = histogram top + predecessor diversity; the
  first 4x-margin rule refused the calibrated fixture itself and was replaced),
  signature clustering, open-tail handling, and the guest-opcode trajectory via
  an optional handler->opcode map. Calibrated against xvmp's documented shape:
  vm_exec(code,size,vm_args,num) stubs, boxed arguments observable before
  interpretation, 11 opcodes + 19 arithmetic sub-ops, const-materialization
  dispatch as a distinct segment.
- references/hardware-sampling-and-crash.md: SPE (data flow without a known
  address; PC + data address per sample), BRBE (branch records when a crash
  stack cannot be trusted), the crash loop (tombstone -> branch records ->
  symbolize), and what each adds to the OLLVM/VMP routes. All
  documented-and-gated: no SPE sample or BRBE record was captured here.
- Flattening-recipe corrections from source: Hikari keeps a frame-resident
  switchVar written through an indirect pointer with scramble32-permuted cases;
  Pluto stores switchVar/switchXor rolling-XOR so memory only ever holds noise
  -- hardware-watchpoints.md's state-variable recipe now distinguishes the
  families, and ete-diff-tracing.md carries the family table plus the xvmp
  shape. AntiDebugging injects raw svc inline asm with randomized fields, so
  byte signatures cannot survive it; semantic site-location can.
- Registration: 3 symptom rows (untrustworthy crash stack; runtime addresses
  needing symbols; interpreter loop reading as noise), reference/script rows,
  coverage/evidence/capability-matrix entries, README trees in both languages,
  and EXTENSION-sampling-crash.md recording exactly what was measured (real .so
  symbolization, slicer fixtures, the slicer's self-found dispatcher-rule bug)
  and what was not (no SPE/BRBE capture, no hardened binary ever traced).

Fix all PR #1 review findings by severity; un-entrench the tests that pinned them

Major (device-breaking / wrong results):
- hw_watch: match the arm64 user_hwdebug_state slot ABI (address register
  first, control second -- the swap armed nothing correctly)
- hw_watch: every post-seize error path now clears slots and detaches;
  PTRACE_O_EXITKILL turned an analysis failure into a SIGKILL of the target
- hw_watch: forward genuine target SIGTRAPs (raise/tgkill/int3); only ptrace
  event stops are consumed -- the reference's signal-preservation promise
- hw_watch: run checks push/build/chmod and the helper's exit status; a dead
  capture is RESULT=blocked, never ok-no-hits
- hw_watch: decode offsets from the module base, not each mapping's start
- ete_capture: --control now requires the control record to DECODE
  (perf script event lines), the documented gate, not just a zero exit
- ete_capture: -o before the -- workload separator in --cmd mode; all
  values shell-quoted into su -c; stale remote perf.data removed first;
  perf exit status checked (timeout's 124 is the normal window end)
- kernel scaffold: --set-reg restricted to x0..x30; the rewrite is gated by
  its own breakpoint event so a watchpoint hit cannot run it; LKM pins the
  task (get/put_task_struct) across registration and calls the four-argument
  register_user_hw_breakpoint; --break rows initialize .addr/.len/.bp_type
  (there is no .pc member); --id validated against the module-id grammar
- trace_addr2sym: exact module matching (path or basename, never substring);
  load bias aligned against the PT_LOAD (bias = start - (foff - p_offset)
  - p_vaddr); PCs outside the ELF span are refused instead of landing on the
  last symbol; --json stdout is a pure document (RESULT= in text mode only)
- trace_diff: block keys need 0x or >=8 hex digits (PIDs/counts no longer
  parse); vm_handler_slice: dispatcher needs predecessor diversity, no
  frequency-margin fallback; both JSON outputs pure (check_commands' rule)

Minor/nitpick:
- capability-matrix + evidence-summary: trace reduction status ok->partial
  (no real-target producer ran it; the file's own definition of ok)
- hardware-sampling-and-crash.md and the script examples carry --elf
- kernel-event-injection.md no longer claims the callback's first statement
  is a filter (it prints unfiltered; the filter is a required change)
- Elf64 caches symbol addresses for per-row bisect
- docstring coverage of the touched surface: 16.67% -> 100%

Tests: every json.loads workaround stripping RESULT= is gone; new regression
tests cover the slot ABI, release-on-error, SIGTRAP forwarding, non-first
mapping offsets, -o placement and su -c quoting, the control decode gate,
x31/--id refusals, the guarded rewrite, outside-ELF PCs and bias math,
short-token refusal and the dispatcher margin rule. The minimal-ELF fixture
now packs a full 8-field Elf64_Phdr (its 7-field pack silently shifted
p_memsz onto p_align). 739 passed; flake8/mypy and all five gates clean.

Address the follow-up review on 0bd8dfe (6 findings)

- ete_capture: two-layer quoting for the control-record, decode and cleanup
  commands -- shlex.quote spliced between literal single quotes broke out of
  su -c for any spaced output name; --seconds must be positive (timeout 0
  disables the timer), refused before any adb check
- kernel scaffold: --set-reg now refuses sp/pc as well (the template has no
  rewrite path for them; a generate that silently drops the action is the
  misleading kind of success); both templates arm with a sampling period
  (LKM via hw_breakpoint_init, KPM spells out sample_period=1) -- a zero
  period is a non-sampling event whose callback never fires: armed, silent
- trace_addr2sym: contains_vaddr tests each PT_LOAD's loaded span instead
  of the whole-file extent; a PC in the unmapped gap between segments is
  refused rather than named after the nearest symbol below
- trace_diff: a bare all-digit token is indistinguishable from a date or
  count, so block keys need 0x or a non-decimal hex digit; a date-only log
  stays refused as a capture failure

Tests: two-layer quoting with a spaced control name, --seconds refusal,
sp/pc in the rejection list, sampling-period assertions for both templates,
a two-segment gap fixture, and a date-only parser case. 743 passed; flake8,
mypy and all five gates clean.
This commit is contained in:
root
2026-10-15 15:33:28 +08:00
parent 7b6c6932a9
commit ab22b225ff
33 changed files with 5358 additions and 6 deletions
+113
View File
@@ -0,0 +1,113 @@
"""Unit tests for ete_capture.py: the command construction the review broke.
The capture path needs a device, so what is host-testable is exactly what the
review found wrong before: where `-o` sits relative to the `--` workload
separator, how values are quoted on their way into one `su -c '...'` string,
and that the control record is held to a decode check, not just an exit code
(the decode gate itself is asserted by construction: the control command and
the decode command are what `capture --control` runs, in that order).
"""
import shlex
from conftest import load_script
ec = load_script("ete_capture.py")
class Args:
"""The argparse namespace shape cmd_capture/perf_record_command read."""
def __init__(self, pid=None, cmd=None, seconds=10, event="cs_etm//u",
out="perf.data", control=False, serial=None):
"""The argparse namespace shape cmd_capture reads."""
self.pid = pid
self.cmd = cmd
self.seconds = seconds
self.event = event
self.out = out
self.control = control
self.serial = serial
def split_device_shell(perf_cmd):
"""What the device shell sees: strip the trailing 2>&1 and split su args."""
assert perf_cmd.endswith(" 2>&1"), perf_cmd
return shlex.split(perf_cmd[:-len(" 2>&1")])
def test_output_option_precedes_the_workload_separator():
"""`perf record` hands everything after `--` to the target command, so a
trailing `-o` is argv for the workload and perf writes to its default
path (the pull then fetches a stale file)."""
cmd = ec.perf_record_command(Args(cmd="app --flag"))
toks = split_device_shell(cmd)
assert toks[:2] == ["su", "-c"]
inner = toks[2]
sep = inner.index(" -- ") # the workload separator, not --per-thread
out_at = inner.index("-o ")
assert out_at < sep, "-o must come before the workload separator"
assert inner[sep:].startswith(" -- app --flag")
def test_pid_mode_quotes_the_pid_and_keeps_o_before_target():
"""Pid mode quotes the pid and keeps o before target."""
cmd = ec.perf_record_command(Args(pid="1234"))
inner = " ".join(split_device_shell(cmd)[2:])
assert "-p 1234" in inner
assert inner.index("-o ") < inner.index("-p 1234")
def test_values_with_quotes_cannot_break_out_of_su_c():
"""Every interpolated value travels inside one su -c string: an unquoted
single quote would close it and run the rest in the outer device shell."""
evil = "x'; rm -rf /data/local/tmp/*; '"
cmd = ec.perf_record_command(Args(cmd='sh -c "%s"' % evil,
event="cs_etm/@tmc_etr0/u"))
toks = split_device_shell(cmd)
assert toks[:2] == ["su", "-c"]
# the su argument is ONE token: the device shell never sees the payload
# as separate commands, whatever it contains
assert len(toks) == 3
inner = toks[2]
assert "perf record" in inner and "-o /data/local/tmp/perf.data" in inner
def test_out_filename_and_event_are_quoted():
"""Out filename and event are quoted."""
cmd = ec.perf_record_command(Args(pid="7", event="my event",
out="dir/my perf.data"))
toks = split_device_shell(cmd)
inner = toks[2]
assert "perf record -e 'my event'" in inner
assert "-o '/data/local/tmp/my perf.data'" in inner
def test_control_commands_exist_and_decode_runs_the_recorded_file():
"""The --control guarantee is that the record DECODES: the decode command
must point perf script at the control recording's file."""
rec = ec.control_record_command("a.ctl.data")
dec = ec.control_decode_command("a.ctl.data")
assert "perf record" in rec and "-- true" in rec and "-o a.ctl.data" in rec
assert "perf script" in dec and "-i a.ctl.data" in dec
def test_control_commands_quote_across_both_device_shell_layers():
"""A spaced output name travels inside one su -c string: quote it inside
the inner command, then quote the whole command for su -c -- a splice of
literal single quotes would run in the outer device shell."""
for cmd in (ec.control_record_command("my ctl.data"),
ec.control_decode_command("my ctl.data")):
assert cmd.endswith(" 2>&1")
toks = shlex.split(cmd[:-len(" 2>&1")])
assert toks[:2] == ["su", "-c"] and len(toks) == 3 # one su argument
assert "my ctl.data" in toks[2]
def test_seconds_must_be_a_positive_window():
"""`timeout 0` disables the timer (or kills instantly): the CLI would
wait on a record that never ends. Refused before any adb check."""
from conftest import run_script
for bad in ("0", "-5"):
r = run_script("ete_capture.py", ["capture", "--pid", "1", "--seconds", bad])
assert r.returncode == 2, (bad, r.output)
assert "--seconds" in r.output and "RESULT=refused" in r.output
+251
View File
@@ -0,0 +1,251 @@
"""Unit tests for hw_watch.py: encoders, refusals, emit, hit decoding.
What is asserted here is the **host-measured half** of the watchpoint route: the
WCR/BCR bit encodings against the ARM DDI 0487 field map, the alignment rules
that refuse impossible specs locally instead of on the device, the emitted
helper's shape, and the hit-log decoder. Device behaviour (seize/arm/trap) is
deliberately not simulated -- see docs/tool-verification/EXTENSION-hw-watchpoints.md.
"""
import json
import shutil
import subprocess
import pytest
from conftest import load_script, run_script
hw = load_script("hw_watch.py")
# ---------------------------------------------------------------------------
# encoders -- the two canonical values plus the variations
# ---------------------------------------------------------------------------
class TestWatchEncoders:
def test_canonical_8byte_rw(self):
# BAS=0xff, LSC=both(0b11), PAC=EL0(0b10), E=1
"""Canonical 8byte rw."""
assert hw.encode_watch(0x7b2000000000, 8, "rw") == (0x7b2000000000, 0x1FFD)
def test_load_only_and_store_only(self):
"""Load only and store only."""
wvr, wcr_r = hw.encode_watch(0x1000, 8, "r")
wvr, wcr_w = hw.encode_watch(0x1000, 8, "w")
assert wcr_r == 0x1FED # LSC=01 (load)
assert wcr_w == 0x1FF5 # LSC=10 (store)
def test_unaligned_start_shifts_bas(self):
# bytes 4..7 of the window: BAS = 0xf0 << 5 = 0x1E00
"""Unaligned start shifts bas."""
wvr, wcr = hw.encode_watch(0x1004, 4, "w")
assert wvr == 0x1000 # WVR canonicalised to the aligned window
assert wcr & 0x1FE0 == 0x1E00
assert wcr == 0x1E15 # | store | PAC | E
def test_len_1_at_offset_7(self):
"""Len 1 at offset 7."""
wvr, wcr = hw.encode_watch(0x1007, 1, "rw")
assert wcr & 0x1FE0 == (0x80 << 5)
def test_window_crossing_refused(self):
"""Window crossing refused."""
with pytest.raises(hw.PlanError, match="crosses"):
hw.encode_watch(0x1006, 8, "rw")
with pytest.raises(hw.PlanError, match="crosses"):
hw.encode_watch(0x1004, 8, "r")
def test_length_bounds(self):
"""Length bounds."""
with pytest.raises(hw.PlanError):
hw.encode_watch(0x1000, 0, "rw")
with pytest.raises(hw.PlanError):
hw.encode_watch(0x1000, 9, "rw")
def test_access_vocabulary(self):
"""Access vocabulary."""
with pytest.raises(hw.PlanError):
hw.encode_watch(0x1000, 8, "x")
class TestBreakEncoders:
def test_canonical_address_match(self):
# BAS all-ones (0x3F << 8), PMC=EL0 (0b10 << 1), BT=0, E=1
"""Canonical address match."""
assert hw.encode_break(0x7b20001234c0) == (0x7b20001234c0, 0x3F05)
def test_misaligned_pc_refused(self):
"""Misaligned pc refused."""
with pytest.raises(hw.PlanError, match="4-byte aligned"):
hw.encode_break(0x1002)
with pytest.raises(hw.PlanError):
hw.encode_break(0x1001)
def test_kernel_abi_constants_are_the_uapi_values():
"""Pinned because a drifted constant compiles fine and arms nothing."""
assert hw.PTRACE_GETREGSET == 0x4204
assert hw.PTRACE_SETREGSET == 0x4205
assert hw.PTRACE_SEIZE == 0x4206
assert hw.NT_ARM_HW_BREAK == 0x402
assert hw.NT_ARM_HW_WATCH == 0x403
assert hw.TRAP_HWBKPT == 4
assert hw.USER_PT_REGS_BYTES == 272
assert hw.REG_OFFSETS["pc"] == 256 and hw.REG_OFFSETS["sp"] == 248
# ---------------------------------------------------------------------------
# emit -- the helper's shape and buildability
# ---------------------------------------------------------------------------
def test_emit_writes_helper_and_readme(tmp_path):
"""Emit writes helper and readme."""
out = tmp_path / "hww"
r = run_script("hw_watch.py", ["emit", "--out", str(out)])
assert r.returncode == 0, r.output
c = (out / "hwwatch_helper.c").read_text(encoding="utf-8")
readme = (out / "README.md").read_text(encoding="utf-8")
# the ABI surface the encoders promised
for marker in ("NT_ARM_HW_WATCH", "NT_ARM_HW_BREAK", "TRAP_HWBKPT",
"PTRACE_SEIZE", "PTRACE_O_TRACECLONE", "PTRACE_SINGLESTEP",
"PTRACE_LISTEN", "0x3Fu << 8"):
assert marker in c, marker
# the honesty devices
assert "TracerPid" in c
assert "TEMPLATE" in c
assert "aarch64-linux-android24-clang" in readme
def test_emitted_helper_compiles_when_a_cross_toolchain_exists(tmp_path):
"""When gcc targets aarch64 here (it does on this CI image), the emitted
helper must build as a static ELF. Skipped where no compiler is present:
the compile is evidence, not a portability promise."""
cc = shutil.which("gcc")
if cc is None:
pytest.skip("no C compiler on this host")
probe = subprocess.run([cc, "-dumpmachine"], capture_output=True, text=True)
if "aarch64" not in (probe.stdout or ""):
pytest.skip("host gcc does not target aarch64 (%r)" % probe.stdout.strip())
out = tmp_path / "hww"
run_script("hw_watch.py", ["emit", "--out", str(out)])
exe = tmp_path / "hwwatch"
built = subprocess.run([cc, "-static", "-O2", "-o", str(exe),
str(out / "hwwatch_helper.c")],
capture_output=True, text=True)
assert built.returncode == 0, built.stderr
assert exe.stat().st_size > 10000
# ---------------------------------------------------------------------------
# decode -- hits in, module+offset out
# ---------------------------------------------------------------------------
HIT_LOG = """READY pid=1234 watch_slots=4 break_slots=6
ARM tid=1234 watch=1 break=0
ARM tid=1240 watch=1 break=0
HIT tid=1240 kind=watch pc=0x7b20001234c0 addr=0x7b2000feedd0 sp=0x7fc000123000 lr=0x7b2000120000 x0=0x1 x1=0x2 x2=0x3 x3=0x4 x4=0x5 x5=0x6 x6=0x7 x7=0x8
ARMFAIL tid=1250 slot=watch1 capacity=0
HIT tid=1234 kind=break pc=0x7b2000129000 addr=0x7b2000129000 sp=0x7fc000124000 lr=0x0 x0=0x0 x1=0x0 x2=0x0 x3=0x0 x4=0x0 x5=0x0 x6=0x0 x7=0x0
DONE hits=2 reason=seconds
"""
MAPS = """7b2000000000-7b2000200000 r--p 00000000 fd:01 100 /data/app/liba.so
7b2000200000-7b2000400000 r-xp 00200000 fd:01 100 /data/app/liba.so
7b2000400000-7b2000420000 rw-p 00400000 fd:01 100 /data/app/liba.so
"""
def test_parse_hits_tolerates_interleaved_lines(tmp_path):
"""Parse hits tolerates interleaved lines."""
log = tmp_path / "hits.log"
log.write_text(HIT_LOG, encoding="utf-8")
rows = hw.parse_hits(str(log))
assert len(rows) == 2
assert rows[0]["kind"] == "watch" and rows[0]["pc"] == "0x7b20001234c0"
assert rows[1]["kind"] == "break"
def test_decode_resolves_module_offset(tmp_path):
"""Decode resolves module offset."""
log = tmp_path / "hits.log"
log.write_text(HIT_LOG, encoding="utf-8")
maps = tmp_path / "maps.txt"
maps.write_text(MAPS, encoding="utf-8")
r = run_script("hw_watch.py", ["decode", "--hits", str(log), "--maps", str(maps)])
assert r.returncode == 0, r.output
assert "+0x1234c0" in r.output # pc lands in the first r--p range
assert "unresolved_pc=0" in r.output
rj = run_script("hw_watch.py", ["decode", "--hits", str(log), "--maps", str(maps),
"--json"])
doc = json.loads(rj.stdout) # --json stdout is one pure document
assert doc["status"] == "ok" and doc["hits"] == 2
assert doc["rows"][0]["site"].endswith("liba.so+0x1234c0")
def test_decode_offsets_come_from_the_module_base_not_the_mapping(tmp_path):
"""A PC in a library's r-xp mapping (its second mapping) must report the
offset from the MODULE base -- the number IDA/Ghidra expects -- not from
that mapping's own start, which resets the offset at each boundary."""
log = tmp_path / "hits.log"
log.write_text("HIT tid=1 kind=break pc=0x7b2000212345 addr=0x7b2000212345 "
"sp=0x0 lr=0x0 x0=0x0\n", encoding="utf-8")
maps = tmp_path / "maps.txt"
maps.write_text(MAPS, encoding="utf-8")
r = run_script("hw_watch.py", ["decode", "--hits", str(log), "--maps", str(maps)])
assert r.returncode == 0, r.output
assert "liba.so+0x212345" in r.output # not +0x12345 (mapping-relative)
def test_decode_without_maps_leaves_pcs_raw(tmp_path):
"""Decode without maps leaves pcs raw."""
log = tmp_path / "hits.log"
log.write_text(HIT_LOG, encoding="utf-8")
r = run_script("hw_watch.py", ["decode", "--hits", str(log)])
assert r.returncode == 0
assert "unresolved" in r.output and "no --maps" in r.output
def test_plan_json_and_refusal_exit_codes():
"""Plan json and refusal exit codes."""
r = run_script("hw_watch.py", ["plan", "--addr", "0x1000", "--len", "8",
"--access", "rw", "--json"])
assert r.returncode == 0
doc = json.loads(r.stdout)
assert doc["specs"][0]["cr"] == 0x1FFD
r = run_script("hw_watch.py", ["plan", "--addr", "0x1004", "--len", "8"])
assert r.returncode == 2
assert "crosses" in r.output
# ---------------------------------------------------------------------------
# the emitted helper: kernel-ABI regressions pinned in the C source
# ---------------------------------------------------------------------------
def emitted_helper(tmp_path):
"""Emit the on-device helper; return its C source."""
out = tmp_path / "hww"
run_script("hw_watch.py", ["emit", "--out", str(out)])
return (out / "hwwatch_helper.c").read_text(encoding="utf-8")
def test_helper_slot_layout_matches_the_kernel_abi(tmp_path):
"""user_hwdebug_state carries the 64-bit address register FIRST and the
32-bit control register SECOND (arch/arm64 uapi); swapping them arms
every slot with the ctrl bits as the address."""
c = emitted_helper(tmp_path)
assert "struct hwslot { uint64_t wvr; uint32_t wcr; uint32_t pad; };" in c
def test_helper_error_paths_release_seized_threads(tmp_path):
"""PTRACE_O_EXITKILL turns an exit-without-detach into a SIGKILL of the
target: every post-seize failure must route through release_all()."""
c = emitted_helper(tmp_path)
assert "static void release_all(void)" in c
# capacity probe failure and arming failure both detach instead of return
assert c.count("release_all();") >= 3 # cap fail, arm fail, clean exit
# partial seize failure detaches the threads seized so far
assert "EXITKILLed when we exit" in c
def test_helper_forwards_genuine_sigtrap(tmp_path):
"""Only our TRAP_HWBKPT hits and ptrace event stops are consumed; a
target-generated SIGTRAP (raise/tgkill/int3) must be forwarded or the
app's handler never runs."""
c = emitted_helper(tmp_path)
assert "if (event != 0) {" in c # event stops: consumed
assert "genuine" in c and "pt(PTRACE_CONT, tid, 0, sig);" in c
@@ -0,0 +1,189 @@
"""Unit tests for kernel_instrument_scaffold.py: spec validation, generate,
verify, and the honesty devices (the unverified banner, the carrier that
cannot arm anything, the experimental action fence)."""
import json
import pytest
from conftest import load_script, run_script
ks = load_script("kernel_instrument_scaffold.py")
class TestSpecValidation:
def test_watch_spec_roundtrip(self):
"""Watch spec roundtrip."""
row = ks.parse_spec_watch("w:0x7b2000123450:8:w")
assert row == {"addr": "0x7b2000123450", "len": 8, "access": "w"}
def test_window_rule_shared_with_hw_watch(self):
"""Window rule shared with hw watch."""
with pytest.raises(ValueError, match="crosses"):
ks.parse_spec_watch("w:0x7b2000123456:8:w")
def test_break_alignment(self):
"""Break alignment."""
assert ks.parse_spec_break("b:0x1234c0")["pc"] == "0x1234c0"
with pytest.raises(ValueError, match="4-byte aligned"):
ks.parse_spec_break("b:0x1234c2")
def test_set_reg_shape(self):
"""Set reg shape."""
row = ks.parse_spec_break("b:0x1234c0", set_reg="x0=0x1")
assert row["set_reg"] == "x0" and row["set_value"] == "0x1"
with pytest.raises(ValueError):
ks.parse_spec_break("b:0x1234c0", set_reg="x0")
def test_set_reg_rejects_registers_the_template_cannot_write(self):
"""x31 is sp's slot in user_pt_regs and the array ends at x30: an
index past 30 writes sp/pc/pstate or off the frame."""
for bad in ("x31=0x1", "x99=0x1", "ax5=0x1", "x0", "y0=0x1"):
with pytest.raises(ValueError):
ks.parse_spec_break("b:0x1234c0", set_reg=bad)
for good in ("x0=0x1", "x30=0x2", "x30=5"):
ks.parse_spec_break("b:0x1234c0", set_reg=good)
def generate(tmp_path, extra=()):
"""Generate the standard scaffold; return its output directory."""
out = tmp_path / "kwatch"
argv = ["generate", "--out", str(out), "--pid", "12345",
"--watch", "w:0x7b2000123450:8:w", "--break", "b:0x7b20001234c0",
"--set-reg", "x0=0x1"] + list(extra)
r = run_script("kernel_instrument_scaffold.py", argv)
assert r.returncode == 0, r.output
return out
class TestGenerate:
def test_layout_and_config(self, tmp_path):
"""Layout and config."""
out = generate(tmp_path)
expected = ["config/instrument.json", "kpm/src/kwatch.c", "kpm/Makefile",
"lkm/kwatch.c", "lkm/Makefile", "module.prop", "README.md",
"post-fs-data.sh"]
for rel in expected:
assert (out / rel).is_file(), rel
cfg = json.loads((out / "config" / "instrument.json").read_text(encoding="utf-8"))
assert cfg["pid"] == 12345
assert cfg["watches"][0]["addr"] == "0x7b2000123450"
assert cfg["breaks"][0]["set_reg"] == "x0"
def test_kernel_files_carry_the_banner_and_the_kpm_lessons(self, tmp_path):
"""Kernel files carry the banner and the kpm lessons."""
out = generate(tmp_path)
kpm = (out / "kpm" / "src" / "kwatch.c").read_text(encoding="utf-8")
lkm = (out / "lkm" / "kwatch.c").read_text(encoding="utf-8")
for body in (kpm, lkm):
assert "NOT COMPILED AND NOT LOADED" in body
assert "EXPERIMENTAL" in body # the set-reg fence
assert "kfunc_def" in kpm
# the banner text mentions the generator, so a stripped banner is traceable
assert "kernel_instrument_scaffold.py" in kpm
# the user-task breakpoint API: four arguments, task last (the per-cpu
# register_wide_* family cannot arm on one target task)
assert "register_user_hw_breakpoint" in lkm
assert "&attr, wp_hit, NULL, task)" in lkm
# the task pointer is pinned past the RCU section and released after
# registration: registration can sleep, an unpinned task can be freed
assert "get_task_struct(task)" in lkm and "put_task_struct(task)" in lkm
# the execution breakpoint row initializes fields wp_cfg declares
# (.addr/.len/.bp_type) -- there is no .pc member to initialize
for body in (kpm, lkm):
assert ".pc = " not in body
assert ".addr = 0x7b20001234c0, .len = 8, .bp_type = 4" in kpm
def test_breakpoint_rows_initialize_declared_fields(self, tmp_path):
"""Breakpoint rows initialize declared fields."""
out = generate(tmp_path)
for body in ((out / "kpm" / "src" / "kwatch.c").read_text(encoding="utf-8"),
(out / "lkm" / "kwatch.c").read_text(encoding="utf-8")):
assert ".pc = " not in body # no such member
assert ".addr = 0x7b20001234c0, .len = 8, .bp_type = 4" in body
def test_events_carry_a_sampling_period(self, tmp_path):
"""A zero sample_period is a non-sampling event: perf arms it, the
overflow callback (wp_hit) never runs, and the module reports armed
events that stay silent forever."""
out = generate(tmp_path)
kpm = (out / "kpm" / "src" / "kwatch.c").read_text(encoding="utf-8")
lkm = (out / "lkm" / "kwatch.c").read_text(encoding="utf-8")
assert "attr.sample_period = 1;" in kpm # hw_breakpoint_init, spelled out
assert "hw_breakpoint_init(&attr);" in lkm
def test_set_reg_rewrite_is_gated_to_its_breakpoint_event(self, tmp_path):
"""wp_hit fires for every registered event (watch and break alike), so
the rewrite must check it landed on its own breakpoint."""
out = generate(tmp_path)
for body in ((out / "kpm" / "src" / "kwatch.c").read_text(encoding="utf-8"),
(out / "lkm" / "kwatch.c").read_text(encoding="utf-8")):
assert "event->attr.bp_addr == 0x7b20001234c0" in body
assert body.count("regs->regs[0] = 0x1;") == 1
# the write sits inside the guard, not at callback top level
guarded = body.split("regs->regs[0] = 0x1;")[0]
assert "event->attr.bp_type & 4" in guarded
def test_id_is_restricted_to_a_module_identifier(self, tmp_path):
"""--id becomes paths, Makefile targets and module.prop entries: a
path separator or leading dot would write outside --out."""
for bad in ("../evil", "a/b", ".hidden", "", "a b"):
argv = ["generate", "--out", str(tmp_path / "x"), "--pid", "1",
"--watch", "w:0x1000:8:w", "--id", bad]
r = run_script("kernel_instrument_scaffold.py", argv)
assert r.returncode == 2, bad
argv = ["generate", "--out", str(tmp_path / "ok"), "--pid", "1",
"--watch", "w:0x1000:8:w", "--id", "kwatch.v2-x"]
r = run_script("kernel_instrument_scaffold.py", argv)
assert r.returncode == 0, r.output
def test_carrier_says_what_it_cannot_do(self, tmp_path):
"""Carrier says what it cannot do."""
out = generate(tmp_path)
readme = (out / "README.md").read_text(encoding="utf-8")
assert "cannot arm a watchpoint" in readme
def test_no_specs_is_refused(self, tmp_path):
"""No specs is refused."""
r = run_script("kernel_instrument_scaffold.py",
["generate", "--out", str(tmp_path / "x"), "--pid", "1"])
assert r.returncode == 2
class TestVerify:
def test_clean_generation_verifies(self, tmp_path):
"""Clean generation verifies."""
out = generate(tmp_path)
r = run_script("kernel_instrument_scaffold.py", ["verify", "--dir", str(out)])
assert r.returncode == 0, r.output
assert "RESULT=ok" in r.output
def test_stripped_banner_is_a_problem(self, tmp_path):
"""Stripped banner is a problem."""
out = generate(tmp_path)
target = out / "kpm" / "src" / "kwatch.c"
body = target.read_text(encoding="utf-8").replace(
"NOT COMPILED AND NOT LOADED", "COMPILED SOMEWHERE, TRUST ME")
target.write_text(body, encoding="utf-8")
r = run_script("kernel_instrument_scaffold.py", ["verify", "--dir", str(out)])
assert r.returncode == 1
assert "banner" in r.output
def test_missing_config_is_a_problem(self, tmp_path):
"""Missing config is a problem."""
out = generate(tmp_path)
(out / "config" / "instrument.json").unlink()
r = run_script("kernel_instrument_scaffold.py", ["verify", "--dir", str(out)])
assert r.returncode == 1
assert "missing" in r.output
class TestGates:
def test_gate_table_includes_the_perf_breakpoint_layer(self):
"""Gate table includes the perf breakpoint layer."""
routes = {g["route"] for g in ks.GATES}
assert "perf hw-breakpoint slots" in routes
assert "KernelPatch / APatch KPM" in routes
r = run_script("kernel_instrument_scaffold.py", ["gates", "--json"])
assert r.returncode == 0
doc = json.loads(r.stdout) # --json stdout is one pure document
assert doc["status"] == "ok" and len(doc["gates"]) == len(ks.GATES)
+260
View File
@@ -0,0 +1,260 @@
"""Unit tests for trace_addr2sym.py: the zero-dependency ELF64 walker, both offset
spaces, and the input adapters.
Two evidence layers, matching docs/tool-verification/EXTENSION-sampling-crash.md:
a hand-built minimal ELF (always runs, covers vaddr!=offset and the stripped
fallback) and a real cross-compiled aarch64 .so (runs where the toolchain exists,
cross-checked against readelf).
"""
import json
import shutil
import struct
import subprocess
import pytest
from conftest import load_script, run_script
tas = load_script("trace_addr2sym.py")
# ---------------------------------------------------------------------------
# a hand-built minimal ELF64: one PT_LOAD with vaddr != offset, one dynsym
# ---------------------------------------------------------------------------
def build_minimal_elf(path, vaddr_delta=0x1000, with_gap=False):
"""Two-symbol dynsym; one PT_LOAD, or two with an unmapped gap.
Symbols: sym_a @ 0x2000 (size 0x100), sym_b @ 0x2400 (size 0x10).
The load segment maps file [0x0,0x4000) at vaddr 0x1000..0x5000, so
file offset F <-> vaddr F + vaddr_delta for the segment, while the symbols
live at vaddr 0x2000/0x2400 regardless (they sit inside the segment).
with_gap=True shrinks segment 1 to vaddr 0x1000..0x3000 and adds a second
PT_LOAD at vaddr 0x4000..0x5000: the span [0x3000,0x4000) is loaded by
NEITHER segment, the shape real shared objects have between PT_LOADs.
"""
assert vaddr_delta == 0x1000
data = bytearray(b"\x00" * 0x4000)
# ELF header
data[0:4] = b"\x7fELF"
struct.pack_into("<B", data, 4, 2) # ELFCLASS64
struct.pack_into("<B", data, 5, 1) # little-endian
struct.pack_into("<H", data, 16, 3) # e_type = ET_DYN
struct.pack_into("<H", data, 18, 0xB7) # e_machine = AArch64
e_phoff, e_shoff = 0x40, 0x3000
struct.pack_into("<Q", data, 0x20, e_phoff)
struct.pack_into("<Q", data, 0x28, e_shoff)
struct.pack_into("<H", data, 0x36, 56) # e_phentsize
struct.pack_into("<H", data, 0x38, 1) # e_phnum
struct.pack_into("<H", data, 0x3A, 64) # e_shentsize
struct.pack_into("<H", data, 0x3C, 3) # e_shnum: null, dynsym, dynstr
struct.pack_into("<H", data, 0x3E, 2) # e_shstrndx (unused: names empty)
if with_gap:
struct.pack_into("<H", data, 0x38, 2) # e_phnum: two PT_LOADs
struct.pack_into("<IIQQQQQQ", data, e_phoff,
1, 5, 0, 0x1000, 0, 0x2000, 0x2000, 0x1000)
struct.pack_into("<IIQQQQQQ", data, e_phoff + 56,
1, 5, 0x2500, 0x4000, 0, 0x1000, 0x1000, 0x1000)
else:
# PT_LOAD file[0,0x4000) @ vaddr 0x1000 (8 fields, Elf64_Phdr:
# p_paddr included -- a 7-field pack silently shifts p_memsz onto p_align)
struct.pack_into("<IIQQQQQQ", data, e_phoff,
1, 5, 0, 0x1000, 0, 0x4000, 0x4000, 0x1000)
# dynstr @ 0x200, dynsym @ 0x400
strtab = b"\x00sym_a\x00sym_b\x00"
data[0x200:0x200 + len(strtab)] = strtab
# Elf64_Sym is 24B: st_name, st_info, st_other/shndx, st_value, st_size
def sym(name_off, info, value, size):
"""Sym."""
return struct.pack("<IBBHQQ", name_off, info, 0, 1, value, size)
syms = sym(1, (1 << 4) | 2, 0x2000, 0x100) + sym(7, (1 << 4) | 2, 0x2400, 0x10)
data[0x400:0x400 + len(syms)] = syms
# section headers @ 0x3000: null, dynsym(link=dynstr idx 2), dynstr
def shdr(sh_type, sh_offset, sh_size, sh_link=0, sh_entsize=24):
"""Shdr."""
return struct.pack("<IIQQQQIIQQ", 0, sh_type, 0, 0, sh_offset, sh_size,
sh_link, 0, 0, sh_entsize)
sh = (shdr(0, 0, 0)
+ shdr(11, 0x400, len(syms), sh_link=2)
+ shdr(3, 0x200, len(strtab), 0, 0))
data[e_shoff:e_shoff + len(sh)] = sh
with open(path, "wb") as fh:
fh.write(bytes(data))
def test_minimal_elf_roundtrip(tmp_path):
"""Minimal elf roundtrip."""
p = tmp_path / "min.so"
build_minimal_elf(p)
elf = tas.Elf64(str(p))
assert [s["name"] for s in elf.symbols] == ["sym_a", "sym_b"]
# vaddr -> file: the segment adds 0x1000 going file->vaddr
assert elf.vaddr_to_file(0x2000) == 0x1000 # 0x2000 - 0x1000
assert elf.file_to_vaddr(0x1000) == 0x2000
sym, off = elf.resolve(0x2042)
assert (sym, off) == ("sym_a", 0x42)
sym, off = elf.resolve(0x2405)
assert (sym, off) == ("sym_b", 5)
# past every symbol but still inside the load segment: nearest-below
# fallback (addr2line-style, gaps included) -- a PC OUTSIDE the segment is
# refused by describe(), not folded onto the last symbol (see below)
sym, off = elf.resolve(0x3f00)
assert (sym, off) == ("sym_b", 0x1b00)
# the span gate: a PC from another module must not be attributed here
assert elf.contains_vaddr(0x3f00) and not elf.contains_vaddr(0xdead0000)
assert "outside-elf" in tas.describe(elf, 0xdead0000)
def test_offs_are_file_flag_and_vaddr_space(tmp_path):
"""file offset input and vaddr input must not be conflated."""
p = tmp_path / "min.so"
build_minimal_elf(p)
r1 = run_script("trace_addr2sym.py", ["--elf", str(p), "--offs", "0x2042"])
assert r1.returncode == 0 and "sym_a+0x42" in r1.output
r2 = run_script("trace_addr2sym.py", ["--elf", str(p), "--offs", "0x2042",
"--offs-are-file"])
assert r2.returncode == 0
# file 0x2042 -> vaddr 0x3042: past sym_a's 0x100-size, nearest-below is sym_b
assert "sym_b+0xc42" in r2.output
def test_absolute_addrs_need_a_base(tmp_path):
"""Absolute addrs need a base."""
p = tmp_path / "min.so"
build_minimal_elf(p)
r = run_script("trace_addr2sym.py", ["--elf", str(p), "--addrs", "0x7b2000002042"])
assert r.returncode == 2
assert "load base" in r.output # ASLR: refuse rather than guess
r = run_script("trace_addr2sym.py", ["--elf", str(p), "--addrs", "0x7b2000002042",
"--base", "0x7b2000000000"])
assert r.returncode == 0 and "sym_a+0x42" in r.output
# ---------------------------------------------------------------------------
# real cross-compiled library (skipped where no aarch64 gcc exists)
# ---------------------------------------------------------------------------
@pytest.fixture(scope="module")
def real_so(tmp_path_factory):
"""Real so."""
cc = shutil.which("gcc")
if cc is None:
pytest.skip("no C compiler on this host")
probe = subprocess.run([cc, "-dumpmachine"], capture_output=True, text=True)
if "aarch64" not in (probe.stdout or ""):
pytest.skip("host gcc does not target aarch64 (%r)" % probe.stdout.strip())
d = tmp_path_factory.mktemp("so")
src = d / "tlib.c"
src.write_text("int encrypt_block(unsigned char *b,int n){for(int i=0;i<n;i++)"
"b[i]^=0x5A;return n;}\n"
"int decrypt_block(unsigned char *b,int n){return encrypt_block(b,n);}\n",
encoding="utf-8")
so = d / "libtlib.so"
built = subprocess.run([cc, "-shared", "-fPIC", "-O2", "-o", str(so), str(src)],
capture_output=True, text=True)
assert built.returncode == 0, built.stderr
return so
def test_real_so_symbols_agree_with_readelf(real_so):
"""Real so symbols agree with readelf."""
syms = subprocess.run(["readelf", "--syms", "-W", str(real_so)],
capture_output=True, text=True).stdout
addr = None
for line in syms.splitlines():
parts = line.split()
if len(parts) > 7 and parts[7] == "encrypt_block" and parts[3] == "FUNC":
addr = int(parts[1], 16)
break
assert addr is not None
r = run_script("trace_addr2sym.py", ["--elf", str(real_so), "--offs",
"0x%x" % (addr + 0x30), "--json"])
assert r.returncode == 0
doc = json.loads(r.stdout) # --json stdout is a pure JSON document
assert doc["resolved"][0]["where"].startswith("encrypt_block+0x30")
def test_real_so_stripped_fallback(real_so, tmp_path):
"""Real so stripped fallback."""
stripped = tmp_path / "stripped.so"
shutil.copy(real_so, stripped)
if shutil.which("strip"):
subprocess.run(["strip", str(stripped)], capture_output=True)
r = run_script("trace_addr2sym.py", ["--elf", str(stripped), "--offs", "0x610"])
assert r.returncode == 0
assert ".text+" in r.output # section+offset fallback, no crash
def test_hits_and_trace_adapters(tmp_path):
"""Hits and trace adapters."""
p = tmp_path / "min.so"
build_minimal_elf(p)
hits = tmp_path / "hits.log"
hits.write_text("HIT tid=1 kind=watch pc=0x7b2000002042 addr=0x7b200000feedd0 "
"sp=0x0 lr=0x0 x0=0x0 x1=0x0 x2=0x0 x3=0x0 x4=0x0 x5=0x0 x6=0x0 x7=0x0\n",
encoding="utf-8")
maps = tmp_path / "maps.txt"
# physically correct for this ELF: the segment (p_offset 0, p_vaddr 0x1000)
# is mapped at 0x7b2000001000, so the load bias is 0x7b2000001000 - 0x1000
maps.write_text("7b2000001000-7b2000005000 r--p 00000000 fd:01 1 "
"/data/app/min.so\n", encoding="utf-8")
r = run_script("trace_addr2sym.py", ["--hits", str(hits), "--maps", str(maps),
"--elf", str(p)])
assert r.returncode == 0, r.output
assert "sym_a+0x42" in r.output
# a neighbour whose name merely CONTAINS the hint must not be matched
maps.write_text("7b3000001000-7b3000005000 r--p 00000000 fd:01 1 "
"/data/app/min.so.v2\n", encoding="utf-8")
r = run_script("trace_addr2sym.py", ["--hits", str(hits), "--maps", str(maps),
"--elf", str(p)])
assert r.returncode == 2 and "no load bias" in r.output
trace = tmp_path / "t.log"
trace.write_text("BLK 0 min.so+0x2042\nBLK 1 xmin.so+0x2405\n", encoding="utf-8")
r = run_script("trace_addr2sym.py", ["--trace", str(trace), "--mod", "min.so",
"--elf", str(p)])
assert r.returncode == 0
assert "sym_a+0x42" in r.output and "sym_b+0x5" not in r.output
def test_load_bias_subtracts_segment_vaddr(tmp_path):
"""p_vaddr != 0: the bias is map_start - (foff - p_offset) - p_vaddr,
not the mapping start."""
p = tmp_path / "min.so"
build_minimal_elf(p)
elf = tas.Elf64(str(p))
maps = tmp_path / "maps.txt"
maps.write_text("7b2000001000-7b2000005000 r--p 00000000 fd:01 1 /data/app/min.so\n",
encoding="utf-8")
assert tas.load_bias_from_maps(str(maps), "min.so", elf) == 0x7b2000000000
# full-path hint matches too; a mere-substring neighbour does not
maps.write_text("7b2000001000-7b2000005000 r--p 00000000 fd:01 1 /data/app/min.so\n"
"7b3000000000-7b3000004000 r--p 00000000 fd:01 1 /data/app/min.so.v2\n",
encoding="utf-8")
assert tas.load_bias_from_maps(str(maps), "min.so", elf) == 0x7b2000000000
assert tas.load_bias_from_maps(str(maps), "/data/app/min.so", elf) == 0x7b2000000000
def test_pcs_in_the_gap_between_segments_are_refused(tmp_path):
"""Real shared objects leave unmapped gaps between PT_LOADs: a PC in a
gap is outside every segment and must be refused, not named after the
nearest symbol below it."""
p = tmp_path / "gap.so"
build_minimal_elf(p, with_gap=True)
elf = tas.Elf64(str(p))
assert elf.contains_vaddr(0x2000) # segment 1
assert elf.contains_vaddr(0x4500) # segment 2
assert not elf.contains_vaddr(0x3800) # the gap
assert "outside-elf" in tas.describe(elf, 0x3800)
def test_json_stdout_is_pure(tmp_path):
"""--json mode must be parseable as one document (no trailing RESULT=)."""
p = tmp_path / "min.so"
build_minimal_elf(p)
r = run_script("trace_addr2sym.py", ["--elf", str(p), "--offs", "0x2042", "--json"])
assert r.returncode == 0
doc = json.loads(r.stdout)
assert doc["status"] == "ok" and "RESULT" not in r.stdout
+183
View File
@@ -0,0 +1,183 @@
"""Unit tests for trace_diff.py: parsers, run-length, and the differential.
The fixtures are synthetic flattened-dispatcher traces (D R1 D R2 ...), because
that is the shape the differ exists to answer: same machinery, different
input-dependent blocks. One fixture is in raw stalker_trace.js grammar, the
other wrapped by run_probe.py, so the parser's tolerance is under test too.
"""
import json
import pytest
from conftest import load_script, run_script
td = load_script("trace_diff.py")
DISP = {"D": 0x1000, "R1": 0x2000, "R2": 0x3000, "R3": 0x4000,
"R4": 0x5000, "R5": 0x6000, "R7": 0x8000}
def stalker_log(seq, wrap=False):
"""Render a synthetic stalker log from a block sequence."""
lines = []
for i, k in enumerate(seq.split()):
body = "BLK %d liba.so+0x%x" % (i, DISP[k])
lines.append(("[123.4] [56.7] TRACE %s" % body) if wrap else body)
lines.append("DONE reason=stop-rpc blocks=8 blk=8 calls=0 truncated=0")
return "\n".join(lines) + "\n"
RUN_A = "D R1 D R2 D R5 D R7" # wrong password: rejects after two extra blocks
RUN_B = "D R1 D R2 D R3 D R4" # right password: accepts
@pytest.fixture
def pair(tmp_path):
"""Two synthetic trace logs: unwrapped and run_probe-wrapped."""
a = tmp_path / "a.log"
b = tmp_path / "b.log"
a.write_text(stalker_log(RUN_A), encoding="utf-8")
b.write_text(stalker_log(RUN_B, wrap=True), encoding="utf-8")
return a, b
class TestParsers:
def test_stalker_unwrapped_and_wrapped(self, pair):
"""Stalker unwrapped and wrapped."""
a, b = pair
blocks = td.read_blocks(str(a))
assert blocks == ["liba.so+0x%x" % DISP[k] for k in RUN_A.split()]
def test_wrapped_log_parses_identically(self, pair):
"""Wrapped log parses identically."""
a, b = pair
assert td.parse_line_stalker("[h] [d] TRACE BLK 7 liba.so+0x8000") == "liba.so+0x8000"
def test_plain_format_keeps_module_keys_and_hex(self, tmp_path):
"""Plain format keeps module keys and hex."""
p = tmp_path / "plain.log"
p.write_text("0x7f0000001000\n7f0000002000 x\nlibb.so+0x9f6c0\nnoise line\n",
encoding="utf-8")
blocks = td.read_blocks(str(p), "plain")
# keys are kept verbatim (lowercased), so 0x-less and 0x-full forms stay distinct
assert blocks == ["0x7f0000001000", "7f0000002000", "libb.so+0x9f6c0"]
def test_plain_format_refuses_ambiguous_decimal_tokens(self, tmp_path):
"""Bare runs under 8 hex digits are PIDs, counts and timestamp
fragments; an all-digit 8+ run is indistinguishable from a date --
taking either as a block key adds noise, and a log with no real
addresses must stay refused (an empty trace is a capture failure,
not a result)."""
p = tmp_path / "pids.log"
p.write_text("app 1234 567\nsample count 99\ndead\n", encoding="utf-8")
with pytest.raises(td.TraceError, match="capture"):
td.read_blocks(str(p), "plain")
d = tmp_path / "date.log"
d.write_text("20261015\n", encoding="utf-8")
with pytest.raises(td.TraceError, match="capture"):
td.read_blocks(str(d), "plain")
p2 = tmp_path / "mixed.log"
p2.write_text("app 1234\n7f0000002000 x\n", encoding="utf-8")
assert td.read_blocks(str(p2), "plain") == ["7f0000002000"]
def test_empty_log_is_a_capture_failure_not_a_result(self, tmp_path):
"""Empty log is a capture failure not a result."""
p = tmp_path / "empty.log"
p.write_text("READY ...\nDONE reason=timeout blocks=0 blk=0 calls=0\n",
encoding="utf-8")
with pytest.raises(td.TraceError, match="capture"):
td.read_blocks(str(p))
def test_auto_detection_prefers_stalker_then_plain(self, tmp_path):
"""Auto detection prefers stalker then plain."""
p = tmp_path / "mixed.log"
p.write_text("BLK 1 liba.so+0x1000\n0x7f0000002000\n", encoding="utf-8")
assert td.read_blocks(str(p)) == ["liba.so+0x1000", "0x7f0000002000"]
class TestReduce:
def test_histogram_names_the_dispatcher(self, pair):
"""Histogram names the dispatcher."""
a, _ = pair
rep = td.reduce_trace(td.read_blocks(str(a)))
assert rep["events"] == 8
assert rep["top"][0] == ("liba.so+0x1000", 4) # D: the wide-margin top
assert rep["runs"] == 8
def test_run_length_collapses_repeats(self):
"""Run length collapses repeats."""
assert td.run_length(["D", "D", "D", "R1", "R1"]) == \
[("D", 3), ("R1", 2)]
class TestDiff:
def test_first_divergence_after_the_shared_prefix(self, pair):
"""First divergence after the shared prefix."""
a, b = pair
d = td.diff_traces(td.read_blocks(str(a)), td.read_blocks(str(b)))
fd = d["first_divergence"]
assert fd is not None
# shared prefix D R1 D R2 = 5 events on both sides
assert fd["event_index_a"] == 5 and fd["event_index_b"] == 5
assert fd["last_common_block"] == "liba.so+0x1000" # the dispatcher
# shared span = prefix D,R1,D,R2,D (5 runs) + trailing D (1) = 6 runs
assert d["identical_run_span"] == 6
def test_unique_sets_are_the_input_dependent_blocks(self, pair):
"""Unique sets are the input dependent blocks."""
a, b = pair
d = td.diff_traces(td.read_blocks(str(a)), td.read_blocks(str(b)))
only_a = {k for k, _ in d["unique_to_a"]}
only_b = {k for k, _ in d["unique_to_b"]}
assert only_a == {"liba.so+0x6000", "liba.so+0x8000"} # R5, R7
assert only_b == {"liba.so+0x4000", "liba.so+0x5000"} # R3, R4
def test_identical_traces_report_no_divergence(self, pair):
"""Identical traces report no divergence."""
a, _ = pair
blocks = td.read_blocks(str(a))
d = td.diff_traces(blocks, blocks)
assert d["first_divergence"] is None
assert d["divergent_regions"] == 0
assert "NO DIVERGENCE" in td.render_diff(d)
def test_render_names_every_part(self, pair):
"""Render names every part."""
a, b = pair
d = td.diff_traces(td.read_blocks(str(a)), td.read_blocks(str(b)))
text = td.render_diff(d, top=5)
for marker in ("first divergence", "region 1", "unique to A", "unique to B"):
assert marker in text
class TestCli:
def test_reduce_cli(self, pair):
"""Reduce cli."""
a, _ = pair
r = run_script("trace_diff.py", ["reduce", "--log", str(a), "--top", "3"])
assert r.returncode == 0 and "RESULT=ok" in r.output
assert "liba.so+0x1000" in r.output
def test_diff_cli_json(self, pair):
"""Diff cli json."""
a, b = pair
r = run_script("trace_diff.py", ["diff", "--a", str(a), "--b", str(b), "--json"])
assert r.returncode == 0, r.output
doc = json.loads(r.stdout) # --json stdout is one pure document
assert doc["status"] == "ok"
assert doc["first_divergence"]["event_index_a"] == 5
assert {k for k, _ in doc["unique_to_b"]} == \
{"liba.so+0x4000", "liba.so+0x5000"}
def test_missing_file_is_usage_error(self):
"""Missing file is usage error."""
r = run_script("trace_diff.py", ["diff", "--a", "nope.log", "--b", "nada.log"])
assert r.returncode == 2
def test_empty_file_refused_with_the_lesson(self, tmp_path):
"""Empty file refused with the lesson."""
p = tmp_path / "e.log"
p.write_text("DONE reason=timeout blocks=0 blk=0\n", encoding="utf-8")
r = run_script("trace_diff.py", ["reduce", "--log", str(p)])
assert r.returncode == 2
assert "capture failure" in r.output or "zero-event" in r.output
+167
View File
@@ -0,0 +1,167 @@
"""Unit tests for vm_handler_slice.py: dispatcher detection, slicing, clustering,
the open tail, and the opcode-map trajectory.
The fixture is a synthetic interpreter trace in the calibrated shape (dispatcher
loop + handler chains + one const-materialization segment + unclosed tail), per
docs/tool-verification/EXTENSION-sampling-crash.md.
"""
import json
import pytest
from conftest import load_script, run_script
vs = load_script("vm_handler_slice.py")
H = {"D": 0x1000, "load": 0x2000, "arith": 0x3000, "store": 0x4000,
"br": 0x5000, "ret": 0x6000, "mat": 0x7000, "spin": 0x8000}
RUN = "D load D load arith D store D br D load D load arith store D br D mat D load arith D ret"
MAP = {"libvm.so+0x2000": "OP_LOAD", "libvm.so+0x3000": "OP_ARITH",
"libvm.so+0x4000": "OP_STORE", "libvm.so+0x5000": "OP_BR",
"libvm.so+0x6000": "OP_RET", "libvm.so+0x7000": "OP_CALL"}
@pytest.fixture
def vm_log(tmp_path):
"""Vm log."""
p = tmp_path / "vm.log"
p.write_text("\n".join("BLK %d libvm.so+0x%x" % (i, H[k])
for i, k in enumerate(RUN.split())) + "\n",
encoding="utf-8")
return p
class TestPickDispatcher:
def test_top_plus_predecessor_diversity(self):
"""Top plus predecessor diversity."""
blocks = ["libvm.so+0x%x" % H[k] for k in RUN.split()]
assert vs.pick_dispatcher(blocks) == "libvm.so+0x1000"
def test_hottest_handler_may_tie_with_dispatcher(self):
"""A stack-shaped VM runs OP_LOAD on every guest instruction: D and the
hottest handler occur equally often, and when that handler is a single
block the pair is a genuine 2-cycle (each has one predecessor -- the
other). That is ambiguity, not a dispatcher: refuse and demand an
explicit --dispatcher."""
seq = "D load D load D load D load D load D load"
blocks = ["libvm.so+0x%x" % H[k] for k in seq.split()]
assert vs.pick_dispatcher(blocks) is None
def test_tie_with_multiblock_handler_resolves(self):
"""Same tie, but the hot handler is two blocks: its tail block is a
predecessor of D that the handler's own head never is, and other
handlers join -- predecessor diversity decides for D."""
seq = ("D load arith D load arith D load arith "
"D store D load arith D br D load arith")
blocks = ["libvm.so+0x%x" % H[k] for k in seq.split()]
assert vs.pick_dispatcher(blocks) == "libvm.so+0x1000"
def test_no_loop_refused(self):
"""No loop refused."""
blocks = ["a.so+0x1", "a.so+0x2", "a.so+0x3"]
assert vs.pick_dispatcher(blocks) is None
def test_frequency_margin_cannot_substitute_for_predecessor_diversity(self):
"""A block 3x hotter than the runner-up but entered from one
predecessor is a plain hot loop, not a dispatch loop: the docstring's
rule is top + many distinct predecessors, so no frequency fallback
may rescue it."""
blocks = ["hot+0x1"] + ["foo+0x2", "hot+0x1"] * 4 + ["bar+0x3", "hot+0x1"]
assert vs.pick_dispatcher(blocks) is None
def test_single_entry_histogram_is_accepted(self):
"""Only one distinct block: nothing to be diverse against."""
assert vs.pick_dispatcher(["a+0x1"] * 8) == "a+0x1"
class TestSlicing:
def test_segments_preamble_tail(self):
"""Segments preamble tail."""
blocks = ["libvm.so+0x%x" % H[k] for k in RUN.split()]
segments, preamble, tail = vs.slice_segments(blocks, "libvm.so+0x1000")
runs = [r for r, _closed in segments]
assert runs[0] == ["libvm.so+0x2000"] # load
assert runs[1] == ["libvm.so+0x2000", "libvm.so+0x3000"] # load arith
assert ["libvm.so+0x7000"] in runs # const materialization
assert segments[-1] == (["libvm.so+0x6000"], False) # ret: unclosed
assert preamble == [] and tail == ["libvm.so+0x6000"]
def test_preamble_kept_separate(self):
"""Preamble kept separate."""
blocks = ["entry+0x1", "D", "h+0x2", "D", "h+0x3"]
segments, preamble, tail = vs.slice_segments(blocks, "D")
assert preamble == ["entry+0x1"]
assert [r for r, _ in segments] == [["h+0x2"], ["h+0x3"]]
class TestCluster:
def test_signatures_and_counts(self):
"""Signatures and counts."""
blocks = ["libvm.so+0x%x" % H[k] for k in RUN.split()]
segments, _, _ = vs.slice_segments(blocks, "libvm.so+0x1000")
trajectory, stats = vs.cluster(segments)
assert len(trajectory) == len(segments)
# the load-only segment occurs twice (positions 0 and 4)
load_sig = vs.signature(["libvm.so+0x2000"])
assert stats[load_sig]["count"] == 2
# every trajectory entry is a known signature
assert set(trajectory) <= set(stats)
def test_spin_loop_diagnostic(self):
"""Spin loop diagnostic."""
blocks = ["a+0x1"] * 20
segments, _, _ = vs.slice_segments(blocks, "a+0x1")
segments = [(["b+0x2"], True)] * 20 if not segments else segments
trajectory, stats = vs.cluster(segments)
diag = vs.diagnostics(blocks, "a+0x1", [r for r, _ in segments], stats)
assert any("spin" in n for n in diag["notes"])
class TestCli:
def test_full_run_with_opcode_map(self, vm_log, tmp_path):
"""Full run with opcode map."""
mp = tmp_path / "map.json"
mp.write_text(json.dumps(MAP), encoding="utf-8")
r = run_script("vm_handler_slice.py", ["--log", str(vm_log),
"--opcode-map", str(mp)])
assert r.returncode == 0, r.output
assert "dispatcher: libvm.so+0x1000" in r.output
assert "UNCLOSED" in r.output
assert "OP_CALL" in r.output # const materialization named
assert r.output.rstrip().endswith("RESULT=ok")
def test_json_shape(self, vm_log, tmp_path):
"""Json shape."""
mp = tmp_path / "map.json"
mp.write_text(json.dumps(MAP), encoding="utf-8")
r = run_script("vm_handler_slice.py", ["--log", str(vm_log),
"--opcode-map", str(mp), "--json"])
doc = json.loads(r.stdout) # --json stdout is one pure document
assert doc["status"] == "ok"
assert doc["opcode_trajectory"][-1] == "OP_RET"
assert all("sig" in s for s in doc["segments"])
def test_explicit_dispatcher(self, vm_log):
"""Explicit dispatcher."""
r = run_script("vm_handler_slice.py", ["--log", str(vm_log),
"--dispatcher", "libvm.so+0x1000"])
assert r.returncode == 0 and "segments: 10 total" in r.output
def test_non_vm_trace_refused(self, tmp_path):
"""Non vm trace refused."""
p = tmp_path / "flat.log"
p.write_text("BLK 0 a+0x1\nBLK 1 a+0x2\nBLK 2 a+0x3\n", encoding="utf-8")
r = run_script("vm_handler_slice.py", ["--log", str(p)])
assert r.returncode == 2
assert "interpreter" in r.output or "dispatcher" in r.output
def test_unmapped_handler_blocks_reported(self, vm_log, tmp_path):
"""Unmapped handler blocks reported."""
mp = tmp_path / "map.json"
mp.write_text(json.dumps({"libvm.so+0x2000": "OP_LOAD"}), encoding="utf-8")
r = run_script("vm_handler_slice.py", ["--log", str(vm_log),
"--opcode-map", str(mp)])
assert r.returncode == 0
assert "unmapped handler blocks" in r.output