antior/apk-reverse
1
0
Fork 0
mirror of https://codeberg.org/reverse-skill-lab/apk-reverse synced 2026-11-15 21:34:28 +08:00
Code Issues Packages Projects Releases Wiki Activity
52 Commits 1 Branch 0 Tags
e151d7d6056d14aa25d96e2f63fde1363d3049fc
Commit Graph
19 Commits
Author SHA1 Message Date
inliver233 e151d7d605 Merge PR #6 "Hardware pass (stage 1)" (dabao1955/feat/ete) 
Merges the hardware-layer pass onto the dexsql commit: arm64 hardware
watchpoints, ETE differential tracing, kernel-side event injection, SPE/BRBE
sampling and crash forensics, address symbolization and interpreter-trace
de-slicing. 33 files, 6 new scripts, 6 new test modules, 4 new reference
documents.

The only conflict was in evidence/capability-matrix.json: both sides appended
to the same array tail. Resolved by keeping both -- the PR's seven entries
(44 rows total now) -- and re-validated as JSON before staging.

Verified on the merged tree, not on the PR branch alone:

  * check_repo.py, check_refs.py, check_commands.py, check_routing.py and
    check_budget.py all pass with zero problems;
  * pytest: 766 passed, 172 skipped, 25 xfailed -- the 141 tests this PR adds
    all pass alongside the 625 that were already there;
  * the PR's host-side claims were reproduced by hand, not taken on trust:
    `hw_watch.py plan --addr 0x1004 --len 4 --access w` returns
    `cr: 7701` (WCR 0x1E15) exactly as the record says, both refusals exit 2
    with the documented reason strings, and the kernel scaffold's
    generate/verify round trip returns RESULT=ok.

One documentation defect was found and fixed in the merge: EXTENSION-kernel-instrument.md
recorded `verified 3 kernel-side file(s)` while the same page's own generate
output lists four (kpm/src/kwatch.c, kpm/Makefile, lkm/kwatch.c, lkm/Makefile)
and the tool reports 4. The page now says 4.

Everything the PR ships that needs a device or a kernel toolchain stays
labelled unverified in the files themselves, which is the property that made
it mergeable.
 2026-10-16 02:05:59 +08:00
inliver233 cbbeb58343 Adopt dexsql as a third indexer, parity-checked against rasc, droidasc and ddc 
LSPosed/dexsql exposes the whole dex as SQL tables (callers, field accesses,
numeric constants, debug lines, annotations, hiddenapi flags) plus the APK
manifest and resources.arsc, plus an MCP server. It is a third indexer at the
same stage as ASC, so it is registered the same way:

  * scripts/capabilities.py gains tool:dexsql and the dex_query_sql capability,
    so doctor.py reports it instead of guessing. INSTALL_HINTS replaces the
    rasc-only hardcoded path list, so a release download or a from-source build
    that landed outside PATH is not reported BLOCKED.
  * references/dexsql-sql-query-layer.md carries the table model, the recipes,
    the MCP tool list and the boundaries; references/toolchain.md gains the
    Tier-1 row and a section; routing.md, SKILL.md, both READMEs and the
    evidence record are updated in step.

What was measured, on four archives (1.4 MB / 1,396 classes, 4.3 MB / 647,
38.7 MB / 4 hardened, 34.8 MB / 30,768):

  * class-definition sets identical to rasc, droidasc and ddc on all four,
    zero differences in either direction;
  * string cross-reference agrees with ASC site for site -- and the match
    semantics differ, which is recorded as a trap (`=` is not `findrefs`);
  * decode emits the same instruction sequence as ddc disasm, line for line;
  * the persistent cache is NOT a speed switch: 0.121 s -> 0.098 s on a 34.8 MB
    archive, a net loss on a small one. Recorded as measured, not as sold.

Full class listing, best of 3: dexsql 0.136 s against droidasc 0.200 s and
rasc 0.055 s on the 34.8 MB archive, so it does not replace rasc for a class
list -- it answers the joins and aggregates the other two cannot express.
 2026-10-16 02:01:36 +08:00
root ab22b225ff Hardware pass: hardware watchpoints, ETE differential tracing, kernel-side event injection 
Add the layer under the software tooling, in the repository's shape: what runs
on a host is measured, and every device-bound or kernel-bound piece is a gated,
labelled template.

- scripts/hw_watch.py: arm64 hardware watchpoints/execution breakpoints. WCR/
  BCR encoders with the 8-byte-window and alignment rules (unit-tested to the
  canonical 0x1FFD/0x3F05), a self-contained on-device ptrace helper that
  compiles clean as a static aarch64 ELF (observed: cross gcc 16.2), adb
  orchestration, and hit decoding to module+offset. The stealth ledger is
  stated three times on purpose: the ptrace route leaves TracerPid visible.
- scripts/ete_capture.py: CoreSight ETE capture with the five gates (kernel
  era, coresight sysfs, cs_etm PMU, perf-with-CORESIGHT, permission), the
  perf/OpenCSD build recipe, and the per-thread record wrapper whose
  --control one-instruction run must pass before any real capture.
- scripts/trace_diff.py: differential reduction of two execution traces into
  first-divergence / divergent regions / blocks-unique-to-each-side.
  Producer-agnostic: Stalker BLK logs (raw or run_probe-wrapped) and decoded
  cs_etm address streams alike, host-tested on synthetic flattened-dispatcher
  fixtures.
- scripts/kernel_instrument_scaffold.py: KPM/LKM watchpoint + register-rewrite
  templates plus an honest userspace carrier; every kernel file carries a
  NOT COMPILED AND NOT LOADED banner that verify enforces, and the
  register-rewrite action is fenced as experimental in the template itself.
- references/hardware-watchpoints.md, references/ete-diff-tracing.md,
  references/kernel-event-injection.md: the mechanism, the gates, the traps,
  and the composition recipes (watch the key buffer; diff two runs; flip the
  decision at the trap), with hand-offs instead of duplication.
- Registration: 3 symptom rows (SKILL.md + routing mirror), 3 reference rows,
  4 script rows, coverage/route-inventory/evidence rows, capability-matrix
  entries, known-limitations entry, eval case 4, README trees and coverage
  paragraphs in both languages, and three EXTENSION evidence records stating
  exactly what was measured on this host (encoders, compile, fixtures,
  generate/verify) and what was not (no watchpoint armed, no cs_etm stream,
  nothing kernel-side compiled or loaded anywhere).

Hardware pass 2: SPE/BRBE crash forensics, address symbolization, VM de-virtualization

Calibrated against the two public obfuscator implementations the user supplied
(dpkg123/ollvm-aosp-toolchain: Hikari-family and Pluto-family flattening, the
xvmp IR virtualizer at 63fec41f/9b7cf75e; fetched read-only into
tools/_work/refs, git-ignored). Source facts observed, runtime inferred.

- scripts/trace_addr2sym.py: runtime addresses -> symbol + static offset,
  zero-dependency ELF64 (PT_LOAD vaddr<->file-off conversion, .dynsym/.symtab,
  stripped section+off fallback). Host-verified against a real cross-compiled
  aarch64 .so with readelf as the independent cross-check, plus a hand-built
  minimal ELF covering vaddr!=offset; accepts hw_watch hits, trace mod+off
  keys, or raw address lists. Fixes on the way: --base int coercion, and
  nearest-below symbol semantics for PCs in inter-symbol gaps.
- scripts/vm_handler_slice.py: slice an interpreter trace into handler
  segments (dispatcher detection = histogram top + predecessor diversity; the
  first 4x-margin rule refused the calibrated fixture itself and was replaced),
  signature clustering, open-tail handling, and the guest-opcode trajectory via
  an optional handler->opcode map. Calibrated against xvmp's documented shape:
  vm_exec(code,size,vm_args,num) stubs, boxed arguments observable before
  interpretation, 11 opcodes + 19 arithmetic sub-ops, const-materialization
  dispatch as a distinct segment.
- references/hardware-sampling-and-crash.md: SPE (data flow without a known
  address; PC + data address per sample), BRBE (branch records when a crash
  stack cannot be trusted), the crash loop (tombstone -> branch records ->
  symbolize), and what each adds to the OLLVM/VMP routes. All
  documented-and-gated: no SPE sample or BRBE record was captured here.
- Flattening-recipe corrections from source: Hikari keeps a frame-resident
  switchVar written through an indirect pointer with scramble32-permuted cases;
  Pluto stores switchVar/switchXor rolling-XOR so memory only ever holds noise
  -- hardware-watchpoints.md's state-variable recipe now distinguishes the
  families, and ete-diff-tracing.md carries the family table plus the xvmp
  shape. AntiDebugging injects raw svc inline asm with randomized fields, so
  byte signatures cannot survive it; semantic site-location can.
- Registration: 3 symptom rows (untrustworthy crash stack; runtime addresses
  needing symbols; interpreter loop reading as noise), reference/script rows,
  coverage/evidence/capability-matrix entries, README trees in both languages,
  and EXTENSION-sampling-crash.md recording exactly what was measured (real .so
  symbolization, slicer fixtures, the slicer's self-found dispatcher-rule bug)
  and what was not (no SPE/BRBE capture, no hardened binary ever traced).

Fix all PR #1 review findings by severity; un-entrench the tests that pinned them

Major (device-breaking / wrong results):
- hw_watch: match the arm64 user_hwdebug_state slot ABI (address register
  first, control second -- the swap armed nothing correctly)
- hw_watch: every post-seize error path now clears slots and detaches;
  PTRACE_O_EXITKILL turned an analysis failure into a SIGKILL of the target
- hw_watch: forward genuine target SIGTRAPs (raise/tgkill/int3); only ptrace
  event stops are consumed -- the reference's signal-preservation promise
- hw_watch: run checks push/build/chmod and the helper's exit status; a dead
  capture is RESULT=blocked, never ok-no-hits
- hw_watch: decode offsets from the module base, not each mapping's start
- ete_capture: --control now requires the control record to DECODE
  (perf script event lines), the documented gate, not just a zero exit
- ete_capture: -o before the -- workload separator in --cmd mode; all
  values shell-quoted into su -c; stale remote perf.data removed first;
  perf exit status checked (timeout's 124 is the normal window end)
- kernel scaffold: --set-reg restricted to x0..x30; the rewrite is gated by
  its own breakpoint event so a watchpoint hit cannot run it; LKM pins the
  task (get/put_task_struct) across registration and calls the four-argument
  register_user_hw_breakpoint; --break rows initialize .addr/.len/.bp_type
  (there is no .pc member); --id validated against the module-id grammar
- trace_addr2sym: exact module matching (path or basename, never substring);
  load bias aligned against the PT_LOAD (bias = start - (foff - p_offset)
  - p_vaddr); PCs outside the ELF span are refused instead of landing on the
  last symbol; --json stdout is a pure document (RESULT= in text mode only)
- trace_diff: block keys need 0x or >=8 hex digits (PIDs/counts no longer
  parse); vm_handler_slice: dispatcher needs predecessor diversity, no
  frequency-margin fallback; both JSON outputs pure (check_commands' rule)

Minor/nitpick:
- capability-matrix + evidence-summary: trace reduction status ok->partial
  (no real-target producer ran it; the file's own definition of ok)
- hardware-sampling-and-crash.md and the script examples carry --elf
- kernel-event-injection.md no longer claims the callback's first statement
  is a filter (it prints unfiltered; the filter is a required change)
- Elf64 caches symbol addresses for per-row bisect
- docstring coverage of the touched surface: 16.67% -> 100%

Tests: every json.loads workaround stripping RESULT= is gone; new regression
tests cover the slot ABI, release-on-error, SIGTRAP forwarding, non-first
mapping offsets, -o placement and su -c quoting, the control decode gate,
x31/--id refusals, the guarded rewrite, outside-ELF PCs and bias math,
short-token refusal and the dispatcher margin rule. The minimal-ELF fixture
now packs a full 8-field Elf64_Phdr (its 7-field pack silently shifted
p_memsz onto p_align). 739 passed; flake8/mypy and all five gates clean.

Address the follow-up review on 0bd8dfe (6 findings)

- ete_capture: two-layer quoting for the control-record, decode and cleanup
  commands -- shlex.quote spliced between literal single quotes broke out of
  su -c for any spaced output name; --seconds must be positive (timeout 0
  disables the timer), refused before any adb check
- kernel scaffold: --set-reg now refuses sp/pc as well (the template has no
  rewrite path for them; a generate that silently drops the action is the
  misleading kind of success); both templates arm with a sampling period
  (LKM via hw_breakpoint_init, KPM spells out sample_period=1) -- a zero
  period is a non-sampling event whose callback never fires: armed, silent
- trace_addr2sym: contains_vaddr tests each PT_LOAD's loaded span instead
  of the whole-file extent; a PC in the unmapped gap between segments is
  refused rather than named after the nearest symbol below
- trace_diff: a bare all-digit token is indistinguishable from a date or
  count, so block keys need 0x or a non-decimal hex digit; a date-only log
  stays refused as a capture failure

Tests: two-layer quoting with a spaced control name, --seconds refusal,
sp/pc in the rejection list, sampling-period assertions for both templates,
a two-segment gap fixture, and a date-only parser case. 743 passed; flake8,
mypy and all five gates clean.
 2026-10-15 15:33:28 +08:00
inliver233 971a961217 Register the ASC indexers as capabilities so G2 reports them instead of guessing 
The kit documented rasc and told the reader to reach for it first, but nothing knew whether it was
present: doctor.py had no entry for either indexer, so an environment gate could not answer "is the
fastest indexer available here", and the reference file had no way to say where a built binary lives.

  capabilities.py  two new capabilities, dex_index_rust and dex_index_python, plus the two tool atoms
                   they depend on. rasc's install line is its build command, because that is what
                   acquiring it means; its cost is marked measured (a clean clone built in 117 s here)
                   rather than estimated. The probe also accepts the work-area path that
                   rasc_build.py builds into, so a machine that followed the kit's own instructions
                   is not reported as BLOCKED.
  doctor.py        reports both, with the next action on the blocked one. Verified: OK with the built
                   binary present, BLOCKED with the build command when it is moved away.
  toolchain.md     says the capability is registered, so G2 does not have to guess.

Also fixed, found while verifying this: `doctor.py --json` printed the RESULT= token after the JSON
document, so its output was not parseable by the tool the flag exists for. Same defect that was fixed
in check_commands.py; the status stays a field in the document and the exit code carries the meaning.

Gates: check_repo 0, check_refs 0 dangling, check_routing coherent, check_budget 0, check_commands 0
drift, pytest 625 passed, flake8 0, mypy clean over 55 files.
 2026-09-22 20:57:10 +08:00
inliver233 f928b5889f Correct the rasc enum finding: the constant bodies are one query away, not lost 
The first write-up said 
asc getclass loses the method bodies of an enum whose constants override an
abstract method. Querying the classes JADX itself names in its comment settles it: they exist as
PasswordConverter/2/3 and both tools decompile them fully (404-407 B, containing convert/getType).

What is real, and narrower: 
asc's outer-class view does not inline them and says nothing about it,
and neither does the Python tool -- so this is a missing convenience step relative to JADX, not a
regression against droidasc, and not unrecoverable code. droidasc's outer listing is richer
(2,170 B vs 314 B, with \, \() and the constructors explicit).

The episode is kept in the record because of how it went wrong: the harness counts literals and pointed
at a real difference while misdescribing it, and the hand check turned that into an overstated claim by
stopping at the first comparison instead of querying the subclass the other tool had named.
 2026-09-22 20:50:01 +08:00
inliver233 a1ada96cc3 Lint clean: adopt the PR's type/style gate, apply it to the current tree, and drop its path filter 
PR #3 proposed adding mypy and flake8 to CI, with configurations for both. The idea is right and the
configurations are kept. Three things about the PR itself needed correcting, and all three were found
by running the gate rather than reading the diff:

  1. It was written against an older tree. Its own edits reference code that no longer exists there
     (`unknown, positions, warnings = ...`, a `struct` import at line 857 that is used 600 lines
     later), so merging it as-is would have removed a live import. The fixes were re-derived against
     the current tree instead, and the PR is superseded.
  2. It added a `paths:` filter to the workflow, watching only `*.py` plus the two lint configs. That
     would stop `check_refs.py` and `check_routing.py` -- documentation gates -- from running on a
     documentation-only commit, which is exactly the change they exist to catch. The workflow keeps
     the PR's `actions/*@v7` bump and gains no filter; the reason is written next to the trigger.
  3. `pip install --upgrade pip pytest` was dropped to just the packages, because upgrading pip in CI
     is not a goal and can fail for reasons unrelated to this repository.

What the gate found on the current tree, and what was done about it -- 70 flake8 findings and 3 mypy
errors, none of which were cosmetic-only:

  F401 (23)  unused imports, removed. These were safe to delete because the import statement itself
             cannot have a side effect that matters here.
  F841 (12)  unused bindings, handled three ways rather than one. Dead code was deleted
             (`mt_mcp_probe.print_waiting_help`'s `host`, `repack.py`'s `signer_jar`); a computed
             value that was then forgotten was **wired up** (`rasc_build.py` now names the linker
             target in its build-failure hint instead of discarding it); and a binding that is
             intentional kept its place with `# noqa: F841` and the reason -- `java2c_probe` reads
             dex header fields one by one against the format spec, and deleting three of them to
             please a linter would have made the parser stop lining up with the format it implements.
  E741 (14)  the ambiguous loop name `l`, renamed with an AST pass that resolves each binding to its
             own scope. A regex rename would have been wrong: several scopes bind their own `l`.
             That pass had a defect of its own -- it treated a comprehension target as living in the
             comprehension scope while leaving references to it in the enclosing one -- which
             produced seven `undefined name 'l'`. Found by running flake8, fixed, and the counting
             bug is documented in `check_routing.py`, whose own `tables()` had the same class of
             error in the other direction.
  style      continuation indents, blank lines between top-level definitions, and lines over length.
             The line limit is 160, not 120, and the number is a **measurement**: this repository's
             secret-scanner rule table holds raw regex source of 148 and 156 characters, which cannot
             be split without changing the pattern.
  mypy (3)   two missing annotations on module-level containers, and one optional-dependency rebind
             (`dexutil = None` after `except ImportError`) that is the repository's own idiom and
             now says so with a `type: ignore[assignment]` and the reason. No code was reordered to
             satisfy the checker.

Verified on a venv containing only `mypy` and `flake8` -- the poorest environment the configuration
must work in, and the one CI uses: **flake8 reports 0, mypy reports "Success: no issues found in 55
source files"**. Behaviour is unchanged: 625 passed / 163 skipped / 25 xfailed, the same five gates
green (check_repo 0 problem, check_refs 0 dangling, check_routing coherent, check_budget 0 failure,
check_commands 0 drift), and every key script still prints help.

  .flake8        new, with the line-length and complexity numbers and their reasons
  mypy.ini       new; optional imports declared, nothing excluded from the analysis
  ci.yml         `actions/*@v7`; new `lint` job on one interpreter; no path filter
  30 files       the findings above
 2026-09-22 19:56:34 +08:00
inliver233 9b159253c6 Adopt rasc (the Rust ASC) as the indexer, with its measured blind spot written down 
The Python `droidasc` indexer works and is one `pip install` away, but the same project has a Rust
re-implementation on its `rust` branch, and this pass answers whether it can be used instead of the
Python one. It can, with one documented caveat. Everything below was measured here; nothing is taken
from the upstream README's numbers.

**What was built.** `rasc` publishes no artifact -- no release asset, and `cargo install rasc`
fetches an unrelated maths parser -- so adopting it means building it. rustup (gnu host, rustc 1.98.1)
plus a 64-bit MinGW-w64 gcc 16.1.0 as the linker, `cargo build --release` in **117 s**, producing a
2.13 MB binary from `MG1937/ASC@e809c7b` (rust branch, sha256 `1becdb86…c0264b`). The build traps are
recorded because they cost time here: the host's existing MinGW is **32-bit** and fails with
`sorry, unimplemented: 64-bit mode not compiled in`, E:\vs2022 has no VC tools, and the registered WSL2
distribution cannot start (`HCS_E_SERVICE_NOT_AVAILABLE`) -- so neither the MSVC nor the Linux path
was available.

**What it answers.** Same CLI shape as the Python tool (`classes` / `manifest` / `getclass` /
`findrefs {string,type,method,field}`), and on this repository's archives the answers match:

  MASTG UnCrackable-Level3 (1.4 MB)   classes 5.6x   findrefs string 15.6x   findrefs type 16.2x
  a real 34.8 MB app (30,768 classes) classes 4.0x   -- class sets IDENTICAL, 0 differences either way

`scripts/rasc_build.py` wraps the whole path -- `--check`, `--build`, `--verify <apk>` -- so the
verification travels with the kit instead of living in a note: `--verify` compares the class-definition
set against `droidasc` and fails on any difference. It resolves the repository root by walking up
rather than assuming a layout, prepends a non-default `CARGO_HOME` to PATH, and reports a missing
toolchain as exit 3 with the install step.

**The blind spot, and why the rule is a rule.** The rasc project ships its own JADX-parity harness.
Run here on a fair sample (app-like classes; the harness refuses to judge a class JADX cannot
decompile), 17 of 20 were judged and 16 matched literal-for-literal with 0 reorderings. The one
difference is not cosmetic: `org.bouncycastle.crypto.PasswordConverter` is an enum whose constants
override a method, JADX emits the three anonymous subclasses with their `convert` / `getType` bodies,
and **rasc emits a bare constant list -- the bodies are gone and no warning is emitted**. Clean,
plausible, incomplete output is precisely the failure shape this repository catalogues, so:

  use rasc to locate, and to read ordinary classes; confirm an enum that shows no bodies with a
  second reader; keep droidasc and JADX as the cross-checks the skill already requires.

The automated verdict said "literals rasc lost: 1", which understates it -- counting literals cannot
see a dropped body. That is recorded in the evidence file rather than smoothed over.

**What changed in the kit.** `references/rasc-and-droidsaw.md` (the measurements, the failing class,
the build traps, provenance); `scripts/rasc_build.py`; `toolchain.md` Tier 1 now lists `rasc` first
with `droidasc` as the fallback and the cross-check; README structure and script tables; the evidence
record `docs/tool-verification/EXTENSION-rasc.md`.

**One inconsistency this exposed and fixed.** `check_repo.py` looked for a reference or script name in
`SKILL.md` + the README only, while this repository's reference inventory lives in
`references/routing.md` -- so a correctly registered file was reported as orphaned, and the script
half of that check was not actually running. The registration surface is now SKILL.md + README +
`references/routing.md`, nested reference directories are read too, and a planted unregistered script
was used to confirm the check still fires.

Gates: check_repo 0 problem, check_refs 0 dangling / 0 warning, check_routing coherent, check_budget
0 failure / 0 warning, check_commands 154 checked / 0 drift, pytest 625 passed / 163 skipped /
25 xfailed.
 2026-09-22 19:38:50 +08:00
inliver233 2beade67f2 Reconstruction pass: make the gates tell the truth, make the package self-contained, keep the one-hop symptom index 
An external audit of this repository landed on four structural findings, and this pass is the
response to them. Everything below was re-measured on this tree before being acted on; two of the
audit's claims needed a correction of my own, and both corrections are noted in place.

**The gates were optimistic, and one of them did not read content at all.**

  doctor.py    the capability table was hard-coded (`repack + sign = bool(java) and python`), so a
               machine with a JDK and no zipalign/apksigner was told it could re-sign; the script
               inventory was a hand-written list of 28 against 51 files on disk, omitting the nine
               most important scripts. It now scans the directory (`registered/checked = 52 / 52`,
               unjudgeable entries counted as `unknown`, never skipped) and resolves capability as a
               real requires-closure, reporting BLOCKED with the missing piece and an installable
               next step. Measured: the old doctor reports `[OK] repack + sign` on a JDK-only PATH
               where keytool, jarsigner, zipalign, apksigner and all eight smali jars are absent.
               The report is allowed to look bad; that is the point.
  check_commands.py (new)
               anchor checks prove a section exists, never that a command runs. This parses each
               script's argparse table with `ast` and checks every documented invocation against it.
               146 commands / 41 interfaces, 0 drift. It caught `sig_probe.py --live --pkg`, which
               does not exist -- the reference had been wrong since it was written.
  check_routing.py (new)
               the symptom index now has one home (`SKILL.md`, because symptom -> file must stay one
               hop) and the inventory has another (`references/routing.md`). This gate proves the
               mirror agrees, that every reference file is named, and that every script is named.

**so_constpatch.py was corrupting APKs it rebuilt.** The writer copied filename and timestamp onto a
fresh ZipInfo and recompressed the whole archive: 435-entry real package, `extra` lost on 205 entries,
`external_attr` rewritten on all 435, 132 untouched entries recompressed, and `resources.arsc` pushed
to an unaligned offset. Android's own `zipalign -c -v 4` fails the old output and passes the new one;
a 26-case fixture suite pins it, and `extractNativeLibs="false"` is now refused by default with a
pointer at `repack.py`. Fixing it exposed the same defect in `repack.py`, where `^lib/[^/]+\.so$`
cannot match `lib/arm64-v8a/*.so` -- so its alignment handling had never once fired on a real APK.

**The package was not self-contained.** 59 references pointed at the repository root, which `npx
skills add` does not install. They now converge on `references/evidence-summary.md` (capability,
conclusion, strength, evidence that ships) plus `evidence/` with the machine-readable companions: 8
references remain and each is explicitly labelled "repository root, not shipped". A drill in which
the new files were copied to an isolated directory and asked three questions answered all three.

**SKILL.md: 572 lines / 15,733 tokens -> 398 lines / ~10,900 tokens**, without losing the two things
that had to stay. The symptom index remains inline; the reference and script indexes moved to the
on-demand routing file. The freed budget went into making the rules *stronger*, not shorter: the
constraints section is now a table of thirteen non-negotiable constraints each paired with the check
that catches a violation, and the tooling obligations are instructions whose violation is a defect
(hedging language in SKILL.md: 1 line, down from a paragraph of hedged advice). Long references gained
a navigation head so a reader can decide from the top whether to load one.

**New: an executable test layer.** `tests/` (unit, CLI contract, no-device integration) plus
`pytest.ini`, `tests/run_tests.py`, a minimal CI job, and `evals/evals.json` written but never run --
stated as such in the file. 608 passed / 158 skipped / 25 xfailed. The 25 xfails are a work order, not
a pass: thirteen scripts report a missing input by exiting 1 instead of 2, seven let a traceback
escape, four treat `--help` as an error, and `preflight --json` lacks a `status` field. A regression
row for `dex_patch_bytes.py` reproduces a previous artifact byte-for-byte
(`67ae1e58e7a44777e5ac8f9f823ab21e…`) and additionally pins the four bytes that may move.

**Two real defects the new tests found in the tooling itself.** `check_repo.py` decoded subprocess
output with the locale encoding, so on a GBK console a script printing an em dash in its usage line
was reported as crashed; it now decodes UTF-8 explicitly. And a CLI contract test drove a *real
attached device* through `coldstart.py` -- logcat, screenshots, five orphan `adb logcat` processes --
which is fixed twice over: child processes run in a private temporary cwd, and the fifteen
device-bound scripts are never executed beyond `--help`/argument parsing.

**Quoting on the device side is now one implementation.** `devsh.su` escaped only double quotes, so
`;`, `$( )` and backticks in a caller-supplied value still reached the device shell while
`coldstart.Device.shell` quoted correctly. Both use `scripts/device_shell.py` now: POSIX single-quote
escaping, argv arrays by default, and identifier validation that refuses a malformed package, path or
component with exit 2 instead of a traceback. The integration test that used to pin the *defect* (it
asserted `su -c "…"` with `\"` escapes) now asserts the property, across five metacharacter payloads.

Gates: check_repo 0 problems (including the content-reading identity scan over git ls-files),
check_refs 0 dangling / 0 warning, check_routing coherent (53 symptom / 45 reference / 54 script rows),
check_budget 0 failure / 0 warning with the always-loaded body now measured in tokens rather than in
"narrative lines with the index excluded", check_commands 146 checked / 0 drift, pytest 608 passed.

Not done, and stated rather than implied: CI has never run on a runner; the evals have never been
executed; the 25 xfails are open; no device-side install/launch was performed in this pass.
 2026-09-22 01:37:51 +08:00
inliver233 6cffc3d2ac Absorption pass: take the mechanics from four external skill repositories, keep our discipline 
Four projects were cloned for comparison (reverse-skill, awesome-game-security,
android-reverse-engineering-claude-skill, MobileRE-Skill) and read for what they do that this
repository does not. None is vendored and none becomes a runtime dependency: what crosses over is a
phase order, a checklist or a command pattern, with the source URL and the retrieval date recorded in
the file that absorbed it.

The rule applied to every candidate: does this make a judgement easier or a verification harder? The
repository is a decision-and-verification layer, and the four things kept are all of that kind.

  new  scripts/scan_leaks.py            target identity scanned before publishing, with an explicit
                                        do-not-anonymize list and four RESULT=/exit states
  new  references/precedents/           the positive half of the record: route including dead ends, a
                                        grade per assertion, measured pits, and a write-back checklist
  new  scripts/svc_scan.py              does a libc-level hook structurally see this call at all
  new  scripts/anti_detect_probe.js     observer-only: name the check, and report the environment it reads
  new  references/desensitization-and-leak-scans.md
  new  references/dart-aot.md sections   the 32-bit snapshot's real string-record format

  check_repo.py now delegates the identity scan to scripts/scan_leaks.py and runs it over
  git ls-files, so the gate reads content instead of layout for the first time. An explicit file list
  cannot be narrowed by the default work-area exemptions -- that defect was caught by the gate itself.

Two claims that this repository had already written down turned out to be wrong, and both are
recorded as refuted rather than quietly rewritten:

  dart-aot.md   the armv7 string table was documented as [len*2 u32le][UTF-16LE]. It is not. The same
                extractor keeps 4,980 chained entries on arm64 and zero on armv7, and the byte layer
                shows the 32-bit record is [header u32][byte-count u32le][UTF-8 payload]: two adjacent
                literals carry 0x46 (70) and 0x48 (72), exactly their byte lengths. The zero was a
                format mismatch, not an empty table. The engine-version banner on the same target is
                66 bytes of "0.0.1" with a blank build id, so the documented regex returns a
                version-shaped lie.
  leak scan     the pass that added the scanner found 26 strong hits inside its own evidence file on
                the day it was written -- the fixture's literal match values, quoted to prove the
                rules fire. The corpus may contain values; the published surface may not.

Measured, not asserted. The check that fires was named on a public MASTG target: three arms with one
variable, the control alive at the end and the probe arm dead in about 300 ms, with strstr("frida") at
t=303 ms and no tombstone. Reproducibility is bounded honestly -- one full sequence in three arms, on
a target whose whole window is sub-second. svc_scan.py's two independent decoders agree on an
identical 214-site set, and ptrace-free dumping gained a negative boundary: a target whose dex is
deflated inside its APK has no whole-image dex in memory to export, for reasons that say nothing about
protection.

Also in this pass: the detector-locating pipeline (detection-and-anti-analysis.md Step 3) with its
branch conditions; the layered Java-to-svc descent and the dump's honesty boundary in
advanced-unpacking.md; a sourced capability-gap list in toolchain.md where every row is graded
URL-verified or measured; the RESULT=-token convention; and SKILL.md opening with an immediate-action
block, because the symptom index and the gates were sitting in the part of a long file nobody reads.

Gates: check_repo.py 0 problem, check_refs.py 0 dangling / 0 warning, check_budget.py 0 failure /
0 warning / 1 pre-existing note. The scan reports RESULT=leaks_found_strong_only, exit 0: four
endpoint/weak hits, all of them addresses from the RFC 5737 documentation range in usage examples.
 2026-09-21 22:51:09 +08:00
inliver233 fd2a1150be Structure: sink the evidence record out of SKILL.md, and gate the budget 
SKILL.md was 658 lines against the Anthropic Agent Skills guidance of keeping the body under 500, and the growth sat where it costs most: the Coverage section had accumulated the evidence for every claim plus a per-pass verification history (153 lines / 11.8 KB), and Hand-off had grown two knowledge tables. Both now load on demand, with SKILL.md keeping the decision-level statement and a pointer.

  Coverage   153 -> 68 lines   new references/coverage-and-limits.md
  Hand-off    51 -> 19 lines   new references/handoff-boundaries.md
  SKILL.md   658 -> 543 lines  (404 narrative + 139 index)
  index rows 24.0 KB -> ~19 KB  descriptions now say when to load, not what is inside

check_budget.py is the substance of this pass. check_repo.py proves consistency, and consistency has a natural counter-pressure: a broken path fails loudly and someone fixes it. Bloat has none -- every pass adds a reference, an index row and a coverage claim, and nothing in the repository notices. The new gate measures narrative lines separately from index lines, because 139 of SKILL.md's lines are the discoverability tax for 86 bundled files and counting them against a prose budget would push the skill to drop files it needs. Conclusions restated beyond the benchmark matrix, the evidence record and the coverage statement are reported as notes.

No measured content changed in this pass: every claim, strength label and negative result is where it was, and the two new references carry the moved text verbatim plus the corrections the benchmark pass produced.
 2026-09-21 22:12:58 +08:00
inliver233 fa2f62107f docs: stop citing workbench paths that are not shipped 
Eight citations in the reference documents pointed at tools/ paths, and .gitignore excludes tools/ -- so neither an installed skill nor a clone of this repository can open them. Each is now either an inline statement of the fact or a pointer to the evidence file under docs/tool-verification/ that actually holds it. One of the eight was introduced by the benchmark pass itself (protocol-reverse.md cited a workbench script).

The evidence records keep quoting tools/_work/... deliberately: there they are operation records of where a run happened. docs/tool-verification/README.md now states that distinction and the rule that follows from it -- a reference document never cites a tools/ path as its evidence, even when the work happened there.
 2026-09-21 22:01:28 +08:00
inliver233 6221a072ca Benchmark pass: measure the documented routes, close five tool gaps, correct four claims 
Public targets under the existing routes, recorded per row in tests/benchmark.md with each row's own strength label: the MASTG crackmes (equal-length patch -> repack -> re-sign with a zero-change control, and the anti-debug race with its time-to-death baseline), DexPatcher's ezAndroid sample, two real split APK sets, and a self-built extraction-skeleton set. Hardening-shape discrimination now separates Java2C from an extraction shell, a VMP and JNI sinking.

New: java2c_probe.py and java2c-and-jni-sinking.md; split-apk.md with repack.py growing set inventory, unified re-signing and merge; protobuf_decode_raw.py cross-checked against the official runtime; vmp_diff_harness.py and its cost table; kernelsu_syscall_mask.py with three kernel-side templates; stalker_trace.js gains Stalker.exclude() and a zero-event warning.

Corrections, each because a measurement contradicted a recorded conclusion: the trivial-body ratio is bimodal rather than a threshold and the old threshold claim was deleted; a Dex-VMP verdict was withdrawn after dexdump contradicted a hand-written opcode table; the assumption that Stalker.exclude() fixes zero-event delivery is disproved (it keeps the process alive, nothing more); and a repack.py signing-path claim expired mid-pass when a second writer added one.

Emulated and kernel-side routes are labelled unverified where no toolchain or device could run them; each evidence file states what was not run.
 2026-09-21 21:57:23 +08:00
inliver233 ce2e4f98e1 Extension pass: module route, extraction shells, emulation/RPC, DBI, kernel layer, protocols 
Closes the six coverage gaps named in an external review of this skill, and moves the
deliverable-form decision out of the single assumption it rested on.

New references (7):
- lsposed-and-modules.md      deliver a hook module when a repack is refused
- advanced-unpacking.md       extraction shells, FART-style recovery, the root-side dump
- emulation-and-rpc.md        call the routine instead of reading it
- native-dbi-and-deobfuscation.md  OLLVM, Stalker traces, two measured boundaries
- protocol-reverse.md         protobuf/gRPC/QUIC and native-side pinning
- kernel-and-environment-hardening.md  when userspace hooking cannot reach the check
- on-device-tooling.md        MT Manager, its APK MCP, working from the phone

New scripts (8): dex_dump_validate.py (+ --trim for page-aligned captures),
dex_mem_scan.py, lsposed_scaffold.py, frida_rpc_serve.py + rpc_template.js,
stalker_trace.js + stalker_report.py, mt_mcp_probe.py

SKILL.md:
- G1 carries a four-form deliverable table (rebuilt APK / module / RPC service / report)
  with the evidence that says when to leave the repack column, plus a matching hand-off
  section on how "verified" moves with the form
- Coverage gains a documented-routes tier carrying its own strength note; Not-covered
  loses the now-inaccurate "general unpacker" line and gains the limits that do apply
- 13 new symptom rows, 7 reference rows, 8 script rows, a fifth tooling obligation
- the pid-drift symptom is stated with its competing causes, not a single attribution

toolchain.md gains measured ASC/ddc cards; README gains the structure, capability and
scope entries for everything above; .gitignore stops tracking the local workspace.

Evidence: docs/tool-verification/EXTENSION-*.md (7 files), same three labels as the rest
of the record. Measured: the gradle-free module build chain (5.27 s end to end), a
successful injection confirmed through the LSPosed file channel (9 rounds) after three
logcat queries came back empty on a ROM with a broken logd, 17 dex images recovered by
root-side /proc/<pid>/mem dumping, the trivial-body metric separating a shell skeleton
from real bodies, ASC/ddc on a hardened APK, and Stalker's zero-event and crash
boundaries. The pid-drift attribution was corrected twice against better measurements.
 2026-09-21 20:57:41 +08:00
inliver233 5fa80a5bea find_refs: read .dex directly, and stop reporting unreadable input as zero refs 
The docstring promised "a smali tree or a directory of dex files" and delivered neither: iter_files() yielded only .smali, a direct .dex was read as UTF-8 text and regex-matched against smali syntax, and a dex-only directory yielded nothing at all. A real dex, a dex directory and a typo path all printed [total refs] 0 and a note blaming the needle, so the one tool the skill recommends for judging blast radius before patching returned a confident false zero.

A .dex is now decoded through dexutil (no baksmali needed), a directory is walked for .smali and .dex alike, an archive is opened for its classes*.dex, input that cannot be read is refused with exit 2 and a reason, and every run prints [scanned] N file(s) so the two cases can never print identically again. Also corrects the recorded exit code for this defect: it was 1, not 0, which moves it from "no signal at all" to "the signal was there and the message contradicted it".

Verified against a fixture whose call graph is known by construction (docs/tool-verification/fixtures/make_fixture_dex.py, committed as the evidence): the .dex and the equivalent smali tree both return 3/2/2, matching what was written. And against the real target: Ljava/lang/String;->length gives 356 refs in 2.3 s where it previously gave 0; a five-dex directory gives 9,472; a typo path exits 2.

Docs updated with it: F9 records the opcode-table finding the fix exposed, F2/F3 and TOOL-VERDICTS.md are corrected, and dex-patching.md now tells the reader to look at the [scanned] line before the count.
 2026-09-21 18:19:23 +08:00
inliver233 03485faa2a dexutil: take opcode names and widths from the format specification 
The opcode table was shifted by one from 0x16 to 0x2C, and nine widths were wrong with it: 0x1C was named monitor-enter and given one unit where the specification has const-class at two; 0x17 was const-wide/16 at 2 where the spec has const-wide/32 at 3; 0x18 was 3 where the spec has const-wide at 5; 0x26 was throw at 1 where the spec has fill-array-data at 3. Each wrong entry shifts every instruction after it in the same method, and the decode keeps producing plausible instructions, so nothing looks broken until an operand index walks off its table.

Measured on a real 8.9 MB dex before the fix: 65 of 50,791 method bodies did not decode to their declared boundary, and 32 carried an operand index outside its table. After: 50,791 of 50,791 land exactly on insns_off + insns_size*2, with 0 illegal indices. Cross-checked against androguard 4.1.4, which shares no code with this project: identical per-instruction width sequences for all 50,791 bodies.

Names and widths now come from the format specification table, and width is a lookup (OP_UNITS) instead of a chain of range tests -- the shape that let a one-observation-at-a-time repair look complete. The goto family moves with it in branch_target() and describe(), and in dex_find_insn.py kind map, where a search for a branch was matching throw and goto/32.
 2026-09-21 18:19:16 +08:00
inliver233 ab0a5ae0e2 dexutil: fix instruction widths that desynchronised the decode 
The decoder could lose sync mid-method and keep producing plausible instructions,
only shifted -- so every offset derived after the fault point was silently wrong.
Found by differential testing against two independent decoders (dexlib2 and
androguard), not by reading the table and trusting it.

Fixed, each verified by asserting the walk ends exactly on
insns_off + insns_size*2 across three real dex images:

- Payload widths. The field after `00 <ident>` is a different quantity per payload
  kind: packed-switch has size(uint16)+first_key(int32), sparse-switch has
  size(uint16)+size*(key,target), fill-array-data has element_width(uint16)+
  size(uint32). All three were read as `1 + <one uint16>`, so a 30-element
  four-byte-wide array payload counted as 5 units instead of 64.
- 0x22 is 22c (2 units); 0x23/0x24 are 35c/3rc (3 units). They sat in each other's
  groups -- the inline comment already said `22c new-array` on a group returning 3.
- 0x20 was grouped as 2 units.

Result: methods that fail to land on the exact end went from 137/113/70 to 44/51/30
across the three images, a 55-68% reduction.

Two further slot-wise edits (shifting the goto family by one width) were made during
the investigation and REVERTED: measurement showed alignment got worse, because the
name table's misalignment had made the wrong slot look authoritative.

Remaining defect, recorded in-file rather than guessed at: OP_NAMES is misaligned with
the opcode slots from roughly 0x1a onward (the slot named `array-length` decodes as
`instance-of`, `goto/16` decodes as `goto`). Widths key on the opcode value, not the
name, so this does not corrupt a decode by itself -- but the names are unsafe as a
lookup key and they misled a reader here. Fixing it means rebuilding the table from an
independent decoder in one pass, not editing it slot by slot.

check_repo.py: 0 problems. check_refs.py: 182 reachable, 0 dangling.
 2026-09-20 19:37:19 +08:00
inliver233 206a2f5465 Toolchain guidance, observation ordering, and target-neutral records 
Capability-first tooling guidance, plus removal of target identity from the repo.

Toolchain (references/toolchain.md):
- New ASC (droidasc) section + README Requirements row. An agent should reach for an
  indexer before any full decompile. Covers install, query subcommands, the division
  of labour with ddc (ASC locates, ddc reads), and its measured limits -- including the
  unverified-negative trap when a non-ASCII needle goes through a shell, and the
  mangled-class case where ASC is the only route.
- ddc: two measured limits -- a renamed class is unreachable via -c, and a decompiled
  body can be semantically wrong rather than merely ugly.
- New "Closing a capability gap": installing the missing tool IS the task, with measured
  install costs set against the cost of the workaround.
- New "Using the kit's scripts instead of writing your own", with two-tools-disagreeing
  as the bug detector it is.
- Ghidra section for the aarch64 case: headless invocation, the flattening caveat, and
  the "a decompiler cannot go where the data is not" boundary.
- HaE placed in Tier 3 as traffic triage, naming the three situations where it is the
  wrong next step.

Observation ordering:
- SKILL.md: new Tooling section with four obligations; G2 now requires a written
  capability inventory; symptom index gains four rows (silent feature loss with no
  request at all, telling your patch apart from the target's own behaviour, spawn keeps
  the Activity stack down, the device server being killed by the ROM).
- dynamic-frida.md: the patch-then-detach ordering, and device-server survival notes.
- New scripts/spawn_patch_detach.py and scripts/hook_patch_only.js.

Neutrality:
- docs/verification-jiongnew/ -> docs/tool-verification/. Target-specific transcripts
  dropped; all target identity removed. The record keeps the measurements and loses
  the specimen.
- No target identity remains anywhere in the tree.

check_repo.py: 0 problems. check_refs.py: 182 reachable, 0 dangling, 0 warnings.
 2026-09-20 19:14:36 +08:00
inliver233 e193f425ba Measure the toolkit on a real Flutter target and correct what it claimed 
Ran the scripts against a real Flutter AOT / multi-dex / 36-.so APK (Dart 3.6.0,
com.tingfeng.tool) and recorded what each one actually did. Two of them were wrong
in the direction that reads as a target finding, one Coverage claim overstated what
ships, and four documentation statements were false.

Fixed:

  dart_pprefs.py decoded only the general-purpose register file, so `ldr dN,[x27,#imm]`
  (0xFD400000) was never matched and every double constant read from the object pool
  was dropped -- 302 pool offsets and 1802 sites on this sample, one of them with 150
  references that would have been reported as zero. The documented advice is to count
  references before editing a shared constant, so a one-sided quiet gap here produces
  exactly the wrong decision. Now covers both register files; a purpose-written
  arbiter agrees with it to 0/0 across .text, against 0/302 before.

Documented but deliberately not fixed, because a fix would not have been run:

  elf_plt.py misses the last PLT stub slot (`off < fsz - 16`), which on libloader.so
  turned a real, called import (dl_iterate_phdr) into "not a resolved stub".
  find_refs.py advertises dex-directory input but iter_files() accepts only .smali,
  so it answers "no references" with exit 0 for input it never read.
  so_constpatch.py's rebuild drops the zip `extra` field and destroys 4 KB alignment.
  Each has its root cause and fix sketch recorded.

Documentation corrections in dart-aot.md:

  blutter's asm/ does contain full instruction listings with pool annotations -- the
  file said to ignore it, and it is the richest artifact in the chain.
  The first build costs ~78 s, not "tens of minutes"; MSVC 19.34 suffices, not VS 2022.
  The file-PP offset mapping is not recoverable from the binary (4241 shared strings,
  4237 distinct deltas), so a snapshot-decoding front end is a dependency, now stated.
  Documented aotopsy as the no-toolchain route, and the trap that its version label is
  a structural profile name (it says 3.6.2 for a 3.6.0 snapshot).

Coverage now separates what was measured from what is inferred, names blutter/aotopsy
as a prerequisite instead of implying the pool decodes itself, lists the scripts this
pass did not run, and says plainly that the packer / VMP / integrity-check scenarios
were not exercised at all.

Added hand-off points for JNI, dex-observed hardening, and the existing native boundary
documents, plus docs/verification-jiongnew/ -- the evidence record, including the four
gate outputs verbatim. No second skill: the gap is one front-end dependency, not a
subject area, and relocating Flutter docs would not have supplied it.
 2026-09-20 04:59:38 +08:00
inliver233 7176b5e1c9 Restructure to the skills/<name>/ layout the skills CLI resolves 
The skill lived at the repository root, which the CLI reads as a single-skill repository and which makes a second skill impossible to add. SKILL.md, references/ and scripts/ move under skills/apk-reverse/ with git mv, so history follows and no in-skill reference needs changing (they are relative to SKILL.md).

check_repo.py now discovers every skills/*/SKILL.md, validates frontmatter by parsing it instead of pattern-matching it, and requires the README to name skills through explicit skills/<name>/... paths.

check_refs.py is new: it audits file.md section cross-references against the real headings in the target file. It found six dangling ones, all fixed here.

SKILL.md: the description is now quoted. Unquoted, it contained a colon-space, which is invalid YAML, so the CLI skipped the skill and reported No skills found. Adds a Coverage section stating what is out of scope and what to do when nothing matches. toolchain.md: IDA is not GUI-only (idalib-mcp is a headless MCP server), plus a section on MCP servers as external dependencies and the deobfuscate-before-LLM boundary. README.md: install via npx skills add, the new layout, coverage and maintenance tools.

Verified: check_repo.py and check_refs.py both clean, build_scripts.py audit 0 findings, npx skills add . --list reports apk-reverse, and an install into a scratch directory lands SKILL.md plus 28 references and 38 scripts.
 2026-09-20 03:37:46 +08:00