From 9b159253c6bc7d57140087fdc963ae6346492ba5 Mon Sep 17 00:00:00 2001 From: inliver233 Date: Tue, 22 Sep 2026 19:38:50 +0800 Subject: [PATCH] Adopt rasc (the Rust ASC) as the indexer, with its measured blind spot written down MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Python `droidasc` indexer works and is one `pip install` away, but the same project has a Rust re-implementation on its `rust` branch, and this pass answers whether it can be used instead of the Python one. It can, with one documented caveat. Everything below was measured here; nothing is taken from the upstream README's numbers. **What was built.** `rasc` publishes no artifact -- no release asset, and `cargo install rasc` fetches an unrelated maths parser -- so adopting it means building it. rustup (gnu host, rustc 1.98.1) plus a 64-bit MinGW-w64 gcc 16.1.0 as the linker, `cargo build --release` in **117 s**, producing a 2.13 MB binary from `MG1937/ASC@e809c7b` (rust branch, sha256 `1becdb86…c0264b`). The build traps are recorded because they cost time here: the host's existing MinGW is **32-bit** and fails with `sorry, unimplemented: 64-bit mode not compiled in`, E:\vs2022 has no VC tools, and the registered WSL2 distribution cannot start (`HCS_E_SERVICE_NOT_AVAILABLE`) -- so neither the MSVC nor the Linux path was available. **What it answers.** Same CLI shape as the Python tool (`classes` / `manifest` / `getclass` / `findrefs {string,type,method,field}`), and on this repository's archives the answers match: MASTG UnCrackable-Level3 (1.4 MB) classes 5.6x findrefs string 15.6x findrefs type 16.2x a real 34.8 MB app (30,768 classes) classes 4.0x -- class sets IDENTICAL, 0 differences either way `scripts/rasc_build.py` wraps the whole path -- `--check`, `--build`, `--verify ` -- so the verification travels with the kit instead of living in a note: `--verify` compares the class-definition set against `droidasc` and fails on any difference. It resolves the repository root by walking up rather than assuming a layout, prepends a non-default `CARGO_HOME` to PATH, and reports a missing toolchain as exit 3 with the install step. **The blind spot, and why the rule is a rule.** The rasc project ships its own JADX-parity harness. Run here on a fair sample (app-like classes; the harness refuses to judge a class JADX cannot decompile), 17 of 20 were judged and 16 matched literal-for-literal with 0 reorderings. The one difference is not cosmetic: `org.bouncycastle.crypto.PasswordConverter` is an enum whose constants override a method, JADX emits the three anonymous subclasses with their `convert` / `getType` bodies, and **rasc emits a bare constant list -- the bodies are gone and no warning is emitted**. Clean, plausible, incomplete output is precisely the failure shape this repository catalogues, so: use rasc to locate, and to read ordinary classes; confirm an enum that shows no bodies with a second reader; keep droidasc and JADX as the cross-checks the skill already requires. The automated verdict said "literals rasc lost: 1", which understates it -- counting literals cannot see a dropped body. That is recorded in the evidence file rather than smoothed over. **What changed in the kit.** `references/rasc-and-droidsaw.md` (the measurements, the failing class, the build traps, provenance); `scripts/rasc_build.py`; `toolchain.md` Tier 1 now lists `rasc` first with `droidasc` as the fallback and the cross-check; README structure and script tables; the evidence record `docs/tool-verification/EXTENSION-rasc.md`. **One inconsistency this exposed and fixed.** `check_repo.py` looked for a reference or script name in `SKILL.md` + the README only, while this repository's reference inventory lives in `references/routing.md` -- so a correctly registered file was reported as orphaned, and the script half of that check was not actually running. The registration surface is now SKILL.md + README + `references/routing.md`, nested reference directories are read too, and a planted unregistered script was used to confirm the check still fires. Gates: check_repo 0 problem, check_refs 0 dangling / 0 warning, check_routing coherent, check_budget 0 failure / 0 warning, check_commands 154 checked / 0 drift, pytest 625 passed / 163 skipped / 25 xfailed. --- README.md | 6 + check_repo.py | 20 +- docs/tool-verification/EXTENSION-rasc.md | 125 ++++++++ docs/tool-verification/README.md | 1 + .../references/rasc-and-droidsaw.md | 142 +++++++++ skills/apk-reverse/references/routing.md | 2 + skills/apk-reverse/references/toolchain.md | 26 +- skills/apk-reverse/scripts/rasc_build.py | 295 ++++++++++++++++++ 8 files changed, 614 insertions(+), 3 deletions(-) create mode 100644 docs/tool-verification/EXTENSION-rasc.md create mode 100644 skills/apk-reverse/references/rasc-and-droidsaw.md create mode 100644 skills/apk-reverse/scripts/rasc_build.py diff --git a/README.md b/README.md index 4c368aa..dbdb8ae 100644 --- a/README.md +++ b/README.md @@ -208,6 +208,9 @@ references/ loaded on demand, one topic each assertion, measured pit-falls, and the write-back checklist routing.md the on-demand inventory: every reference with when to load it, every script with what it does, and a mirror of the symptom index + rasc-and-droidsaw.md the Rust re-implementation of the ASC indexer: measured speedup and + identical class sets, the enum shape where it silently drops bodies, + and how to build and verify it evidence-summary.md the condensation that ships with the skill: capability, one-line conclusion, strength, and the evidence you can actually open in an installed copy @@ -365,6 +368,9 @@ scripts/ parameterized, path-agnostic kernelsu_syscall_mask.py generate a KernelSU/APatch syscall-masking scaffold: an installable userspace module skeleton plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label + rasc_build.py build and verify rasc, the Rust ASC re-implementation: + --check what is present, --build clone plus cargo, --verify an APK + against droidasc and fail on any class-set difference scan_leaks.py scan a repository for target identity before publishing it: bundle ids in manifest / `pm` / `ps` contexts, serial-shaped tokens, PATs, inline appkey assignments, literal endpoints, host user paths. Exemptions for diff --git a/check_repo.py b/check_repo.py index c777766..3d719a8 100644 --- a/check_repo.py +++ b/check_repo.py @@ -198,17 +198,33 @@ def check_references(name, skill_dir, root_docs): print(' MISSING %s' % x) fail.extend(missing) - # Reachability: nothing may be orphaned from the skill entry point. + # Reachability: nothing may be orphaned from the entry point. + # + # The registration surface is SKILL.md **plus** `references/routing.md`, because this repository + # splits the two: SKILL.md keeps the symptom index (symptom -> file must stay one hop) and the + # reference/script inventory lives in the routing file, which `check_routing.py` in turn proves + # names every reference and every script. Reading only SKILL.md here would report a correctly + # registered file as orphaned -- and did, for every file added after that split. + routing = os.path.join(ref_dir, 'routing.md') bodies = read(os.path.join(skill_dir, 'SKILL.md')) + root_docs + if os.path.isfile(routing): + bodies += read(routing) for f in sorted(refs): if f.endswith('.md') and f not in bodies: - fail.append('%s: references/%s is not mentioned anywhere' % (name, f)) + fail.append('%s: references/%s is named neither in SKILL.md, README.md nor ' + 'references/routing.md' % (name, f)) print(' UNLISTED references/%s' % f) all_docs = bodies for f in sorted(refs): if f.endswith('.md'): all_docs += read(os.path.join(ref_dir, f)) + for sub in ('', 'coverage', 'precedents'): + d = os.path.join(ref_dir, sub) if sub else ref_dir + if os.path.isdir(d): + for f in sorted(os.listdir(d)): + if f.endswith('.md') and os.path.isfile(os.path.join(d, f)): + all_docs += read(os.path.join(d, f)) for s in sorted(scripts): if s.endswith(('.py', '.js')) and s not in all_docs: fail.append('%s: scripts/%s is undocumented' % (name, s)) diff --git a/docs/tool-verification/EXTENSION-rasc.md b/docs/tool-verification/EXTENSION-rasc.md new file mode 100644 index 0000000..0112503 --- /dev/null +++ b/docs/tool-verification/EXTENSION-rasc.md @@ -0,0 +1,125 @@ +# EXTENSION — rasc (the Rust ASC) measured against the Python one + +Evidence for `skills/apk-reverse/references/rasc-and-droidsaw.md`. Every number below has the command +that produced it. + +## What was measured + +| Claim | Grade | Result | +|---|---|---| +| `rasc` builds from the `rust` branch on this host with no prebuilt artifact | **observed** | rustc 1.98.1 (gnu host) + MinGW-w64 gcc 16.1.0; `cargo build --release` **117 s**; binary 2,236,416 B | +| It is faster than the Python `droidasc` on the same archives | **observed** | 4.0–5.6× for `classes`, up to 16.2× for `findrefs` (tables below) | +| It returns the same answers | **observed** | **identical class-definition sets**, 30,768 / 30,768 with **0 differences in either direction**; `findrefs string` identical on one sample, `type` a superset (390 vs 384) | +| Its `getclass` decompiles ordinary classes correctly | **observed** | 16 of 17 JADX-judged classes matched literal-for-literal, 0 reorderings | +| **It silently drops the bodies of enum constants that override a method** | **observed** | `org.bouncycastle.crypto.PasswordConverter`: JADX emits three anonymous subclasses with `convert`/`getType`; `rasc` emits a bare constant list, **no warning** | +| Its own benchmark's 8.0× geometric mean reproduces here | **not reproduced, and explained** | that figure is from a 343 MiB / 567k-class archive; on a 34.8 MB / 30,768-class app the ratio is 4×. Both are real; the small-archive figure is what a normal task hits | + +## Environment + +Host Windows 11. Device not involved. Toolchain installed for this pass, into the gitignored +`tools/_work/`: + +``` +rustup-init (x86_64-pc-windows-gnu) -> rustc 1.98.1, cargo 1.98.1 +winget install BrechtSanders.WinLibs.POSIX.UCRT -> gcc 16.1.0 (x86_64-w64-mingw32, UCRT) +``` + +The 32-bit MinGW already on this host (`C:\MinGW`, gcc 6.3.0, target `mingw32`) **cannot** link a +64-bit binary — `gcc -m64` reports `sorry, unimplemented: 64-bit mode not compiled in`, and the Rust +GNU target needs a 64-bit linker. E:\vs2022 has no VC tools (`cl.exe` / `link.exe` absent), and the +WSL2 Ubuntu distribution is registered but `HCS_E_SERVICE_NOT_AVAILABLE`, so no Linux path either. + +``` +$ python skills/apk-reverse/scripts/rasc_build.py --check +toolchain : git=E:\Git\cmd\git.EXE cargo=...\rust\cargo\bin\cargo.EXE rustc=...\cargo\bin\rustc.EXE gcc=C:\MinGW\bin\gcc.EXE(mingw32) +rasc : E:\apk-reverse\tools\_work\rust\target\release\rasc.exe rasc 0.1.0 +result : ok +``` + +## Speed and agreement + +Fastest of 3 runs (1 run on the largest), same machine, `PYTHONUTF8=1` for the Python side. + +``` +$ python tools/_work/compare_asc.py tools/_work/bench/apks/UnCrackable-Level3.apk --repeat 3 +scenario rasc s py s speedup result agreement +classes 0.025 0.138 5.6 equal rasc=1396 py=1396 +manifest 0.017 - - not comparable +findrefs string 0.022 0.341 15.6 overlap 100% rasc=3 py=3 +findrefs type 0.022 0.358 16.2 overlap 98% rasc=390 py=384 +``` + +Rounded to whole numbers in the reference file (4.7–14.7× there vs 5.6–16.2× here) because these are +two independent runs of the same measurement; both are in the observed range, and the run-to-run +spread on a sub-100 ms command is the honest resolution. + +On a real 34.8 MB application (`MT-Manager.apk`, 30,768 classes): + +``` +$ python -c "... rasc classes vs droidasc listclass, set comparison ..." +rasc : 0.048 s 30768 classes +droidasc : 0.190 s 30768 classes +speedup : 4.0x +identical set : True +only in rasc : 0 only in droidasc: 0 +``` + +`python skills/apk-reverse/scripts/rasc_build.py --verify tools/_phone-modules/MT-Manager.apk` runs +the same comparison and reports `{'result': 'ok', 'rasc_classes': 30768, 'droidasc_classes': 30768, +'only_rasc': 0, 'only_droidasc': 0}`, so the check travels with the kit rather than living in a note. + +**A defect found on the Python side during this work:** `droidasc listclass` exits 1 with +`'gbk' codec can't encode character` on an app whose classes contain non-Latin identifiers, when the +console encoding is not UTF-8. With `PYTHONUTF8=1` it completes. `rasc` was unaffected. That is a +one-variable difference in the environment, not a property of either tool, and it is recorded because +it cost a round here. + +## The blind spot, verified rather than reported + +The `rasc` repository ships `bench/jadx_parity.py`, which treats a string literal JADX has and `rasc` +does not as lost code, and refuses to judge a class JADX itself could not decompile cleanly. Its own +class sampling takes every-nth entry from `rasc classes`, which on a real app lands mostly on +obfuscated names where JADX produces nothing — 8 of 10 picked that way were unjudgeable here, so the +sample measures the sampler. `tools/_work/parity_fair_sample.py` filters to app-like classes first +(ASCII, dotted, no inner class, not `R`/`BuildConfig`) and feeds those to the unmodified harness: + +``` +$ python tools/_work/parity_fair_sample.py tools/_phone-modules/MT-Manager.apk 20 +classes in APK: 30768 +app-like candidates: ... +judged 17 class(es) where JADX was clean; literals rasc lost: 1; reorderings: 0 +``` + +The one finding, checked by hand with `tools/_work/verify_parity_finding.py`: + +``` +class : org.bouncycastle.crypto.PasswordConverter +jadx literals: ['ASCII', 'PKCS12', 'UTF8'] +rasc literals: [] +missing : ['ASCII', 'PKCS12', 'UTF8'] + 'ASCII' appears in rasc output as an identifier: True + ... +VERDICT: form difference (enum constants printed as a list, literals not re-emitted) +``` + +**The automated verdict understates it, and that is the finding.** The harness counts literals; the +human check shows the literals are gone *because the whole anonymous subclass body is gone* — JADX +prints `ASCII { ... convert() ... getType() ... }` and `rasc` prints `ASCII,`. `rasc` emits no +comment, no TODO, no warning. Output that is clean, plausible and incomplete is the failure shape this +repository's `pitfalls.md` exists to catalogue, so the consequence is a rule, not a caveat: confirm an +enum with no bodies using a second reader. + +## Not established + +- **No whole-APK parity run.** The comparison is on class sets, sampled `getclass` output and one + `findrefs` family per sample. `bench/corpus_parity.py` and `bench/jadx_parity.py` exist for a fuller + run and were used on one archive, not on a corpus. +- **`findrefs` on a large corpus was not run to completion.** On the 34.8 MB app, `findrefs string + http` returned 0 rows on both sides — a query with no hits is not a parity result, and a query that + does hit was only compared on the small archive. +- **`getclass` was judged on 17 classes of one app.** The enum blind spot is confirmed; whether other + class shapes (inner classes, synthetic bridges, `invokedynamic`) diverge the same way is untested. +- **No device or Android-side use.** `rasc` reads archives on the host; nothing here exercises it as a + device tool, and the reference file makes no claim about that. +- **The 8.0× upstream figure was not reproduced**, because the archive it was measured on (343 MiB, + 567k classes) is not available here. The direction and the smaller ratio were measured. diff --git a/docs/tool-verification/README.md b/docs/tool-verification/README.md index 34391f3..1fef332 100644 --- a/docs/tool-verification/README.md +++ b/docs/tool-verification/README.md @@ -70,6 +70,7 @@ those are called out in the matrix rather than buried here. | `EXTENSION-split-apk.md` | Two real split sets through both branches: unified re-signing, the merge that is refused by design, the merge that succeeds, and the signer chain this host actually needs | | `EXTENSION-reconstruction.md` | The reconstruction pass: the doctor capability closure (a JDK-only PATH used to report "can re-sign", now `BLOCKED` with a next action), the zip rebuild that stops dropping per-entry metadata (`zipalign -c -v 4` fails the old output and passes the new), and `check_commands.py` catching a documented flag that does not exist | | `EXTENSION-test-harness.md` | The executable half of the evidence: what `tests/` asserts and what it measured, the `dex_patch_bytes` regression reproduced byte-for-byte, the 153 skips and 26 xfails that form the exit-code work order, the device-bound exclusion forced by an incident where the suite drove a real phone, and CI written but never run | +| `EXTENSION-rasc.md` | The Rust ASC measured against the Python one: identical class-definition sets on two archives, the speedup per scenario, the build (no prebuilt artifact exists, and a 32-bit MinGW cannot link it), and **the enum shape whose method bodies it drops with no warning** | | `EXTENSION-protobuf-raw.md` | A schema-free decoder cross-checked against the official runtime and a real DataStore container, with the packed-boundary and proto3-zero ambiguities reproduced on real bytes | ## Absorption-pass record diff --git a/skills/apk-reverse/references/rasc-and-droidsaw.md b/skills/apk-reverse/references/rasc-and-droidsaw.md new file mode 100644 index 0000000..74b3a63 --- /dev/null +++ b/skills/apk-reverse/references/rasc-and-droidsaw.md @@ -0,0 +1,142 @@ +# rasc and droidsaw — the Rust ASC, and what it changes about the indexer-first workflow + +Load this when: you are about to reach for `droidasc` (ASC) to locate classes or references, or when +the Python indexer is the bottleneck on a real archive. It states what the Rust re-implementation is, +how it was measured here, and **the one class shape where it silently loses code** — which is the +part that decides whether you use it alone or keep a second tool next to it. + +**When to prefer it, in one line:** `rasc` is the same interface with the same answers, several times +faster, and one measured blind spot; use it as the indexer, keep `droidasc` and JADX as the +cross-checks the rest of this skill already requires. + +## What it is + +`MG1937/ASC` has two implementations on different branches. `main` is the Python tool this skill +already documents (`pip install droidasc`). The **`rust` branch is `rasc`**, a re-implementation that +keeps the CLI shape and drops the Python runtime: + +| | Python `droidasc` main | `rasc` (rust branch) | +|---|---|---| +| Runtime | CPython + androguard | none — one static-ish binary, **2.13 MB** | +| Decompiler behind `getclass` | androguard's | **`droidsaw-dex 2.0.0`** (pure Rust; its project advertises byte-identical DEX re-emission on F-Droid) | +| Install | `pip install droidasc` | **must be built** — no release asset, no crate (`cargo install rasc` fetches an unrelated maths parser) | +| Commands | `listclass` `getmanifest` `getclass` `findrefs` | `classes` `manifest` `getclass` `findrefs` | +| Flags | `--prefix` on `listclass` | `--filter` on `classes` (case-insensitive substring, not a prefix) | +| Shared flags | `--threads/--thread`, `-o/--output`, `--debug` | same, plus `--debug` prints phase timings on stderr | + +`scripts/rasc_build.py` wraps the build and the verification, because **a build only one machine has +performed is a build nobody else can reproduce**: `--check` reports what is present, `--build` +clones and compiles, `--verify ` compares the class-definition sets against `droidasc` and fails +if they differ. + +## Measured here + +Environment: Windows 11, rustup `stable-x86_64-pc-windows-gnu` (rustc 1.98.1), MinGW-w64 gcc 16.1.0 +(UCRT, 64-bit) as the linker. Build from a clean checkout: **117 s**, `cargo build --release` with +FatLTO. Binary `rasc` 0.1.0, 2,236,416 bytes, sha256 `1becdb86b444fe7910589f8ade5bccf13794de9f2b466a1c07748b5669c0264b`, +from `MG1937/ASC@e809c7b4b454ebc8eb4467f56d44a82e6139472b` (rust branch). + +Two archives, fastest of N runs, same machine, `PYTHONUTF8=1` for the Python side: + +| Archive | Scenario | rasc | droidasc | ratio | answers | +|---|---|---|---|---|---| +| MASTG `UnCrackable-Level3.apk` (1.4 MB) | `classes` | 0.025 s | 0.116 s | 4.7× | **identical set** (1396 / 1396) | +| | `findrefs string http` | 0.021 s | 0.299 s | 14.0× | identical (3 / 3) | +| | `findrefs type String` | 0.021 s | 0.307 s | 14.7× | rasc superset (390 vs 384) | +| | `getclass` ×2 real classes | 0.086 / 0.082 s | 0.297 / 0.267 s | 3.3–3.5× | both emitted Java | +| a real 34.8 MB app (30,768 classes) | `classes` | 0.048 s | 0.190 s | **4.0×** | **identical set**, 0 differences both ways | + +The speedup is smaller than the upstream README's geometric mean of 8.0× because that figure was +measured on a 343 MiB archive with 567k classes, where startup cost disappears; on a mid-size app the +Python runtime's import time keeps the ratio near 4×. Both are real, and the small-archive numbers are +the ones a normal task hits. + +`python skills/apk-reverse/scripts/rasc_build.py --verify ` reproduces the class-set comparison +and is the check to run after any rebuild. + +## The blind spot: abstract enum constants lose their bodies + +`bench/jadx_parity.py` from the `rasc` repository compares **string literals** between JADX and `rasc` +per class, counting a literal JADX has and `rasc` does not as a lost-code signal. Run here on a fair +sample — app-like classes, no obfuscated identifiers, since the harness itself refuses to judge a +class JADX cannot decompile — **20 classes picked, 17 judged, 1 with a difference, 0 reorderings**. + +The one difference is not a formatting artefact. `org.bouncycastle.crypto.PasswordConverter` is an +enum whose constants each override an abstract method: + +```java +// JADX +public enum PasswordConverter implements CharToByteConverter { + ASCII { // from class: ...PasswordConverter.1 + @Override public byte[] convert(char[] cArr) { ... } + @Override public String getType() { return "ASCII"; } + }, + ... +``` + +```java +// rasc +public enum PasswordConverter implements org.bouncycastle.crypto.CharToByteConverter { + ASCII, + UTF8, + PKCS12; + public PasswordConverter() { } + public PasswordConverter(PasswordConverter$1 v3) { } +} +``` + +**Both the per-constant method bodies and the string literals are gone, and `rasc` emits no warning +that anything was dropped.** The output is clean, plausible, and incomplete — the same failure shape +`pitfalls.md` is a catalogue of. So the rule for this layer is a rule, not a preference: + +- **Use `rasc getclass` for locating and reading ordinary classes.** That is what it was measured on. +- **When the class is an `enum` with constant-specific bodies, confirm with a second reader** + (`droidasc getclass`, or JADX). A collapsed enum is visible on sight: the constants are printed as a + bare list and no method bodies follow. Treat "enum with no bodies" as *unverified output*, not as + "this enum has no logic". +- **Never treat a single decompiler's output as the definition of what exists.** This is the same rule + the VMP section states about opcode tables, applied to a class body. + +Frequency matters for calibration: 1 of 17 judged classes here, and the other 16 agreed literal for +literal. So the correct posture is **adopt, with the cross-check kept**, not "reject" and not +"drop the Python tool from the kit". + +## What it does not change + +- **The workflow.** The indexer is the same *stage* it always was: `rasc` answers *location* + questions, and reading a class you have already located is what a decompiler is for. It does not + decompile a whole APK into a project, and neither does the Python tool. +- **The verification discipline.** Every claim above is a measurement with a command behind it; + where `rasc` and `droidasc` disagree in the tables, the disagreement is recorded rather than + averaged away. +- **Packer behaviour.** On this repository's hardened sample both tools correctly return 4 classes — + the payload is encrypted inside the shell and is not in any DEX either tool can read. A small class + count on a hardened target is the target, not the tool. + +## Building it yourself + +```bash +python skills/apk-reverse/scripts/rasc_build.py --check # what is present +python skills/apk-reverse/scripts/rasc_build.py --build # clone + cargo build --release +python skills/apk-reverse/scripts/rasc_build.py --verify app.apk # compare against droidasc +``` + +Requirements and the traps measured here: + +- **Rust 1.93+** (the crate is `edition = "2024"`; this build used 1.98.1) and `git`. +- On Windows, the **GNU host toolchain needs a 64-bit MinGW-w64 gcc**. A 32-bit MinGW fails at link + with `sorry, unimplemented: 64-bit mode not compiled in` — the fix is a 64-bit distribution (this + build used winlibs gcc 16.1.0), not a flag. +- The build fetches crates from crates.io; `vendor/` in the repository only carries the two patched + crates (`axmldecoder`, `droidsaw-dex`), so `--offline` fails with `no matching package named anyhow`. +- `cargo`/`rustc` installed to a non-default `CARGO_HOME` are invisible to a non-interactive PATH; + `rasc_build.py` prepends the known locations rather than reporting the toolchain as absent. + +## Provenance + +`https://github.com/MG1937/ASC` (branch `rust`, commit `e809c7b`), retrieved 2026-09-22. +`https://github.com/droidsaw/droidsaw` and `https://crates.io/crates/droidsaw-dex` (2.0.0), +consulted 2026-09-22. The Python implementation is `pip install droidasc` (0.1.1.post1 seen here; +0.1.1.post2 published 2026-09-21). Nothing from either project is vendored into this repository — +`rasc_build.py` clones it into the gitignored `tools/_work/` and the measurements above name the +commit they came from. diff --git a/skills/apk-reverse/references/routing.md b/skills/apk-reverse/references/routing.md index 18df4eb..834acfb 100644 --- a/skills/apk-reverse/references/routing.md +++ b/skills/apk-reverse/references/routing.md @@ -111,6 +111,7 @@ _54 row(s) below the header._ | `references/third-party-builds.md` | The input is a "cracked"/"modded" build you did not produce — audit it before trusting it | | `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, **deliverable-form drift (rooted-only vs shippable)**, bound-your-waits, **captures-you-never-looked-at**, **long-context decay**, handover | | `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. | +| `references/rasc-and-droidsaw.md` | You are about to reach for the ASC indexer, or the Python one is the bottleneck: the Rust re-implementation, its measured speedup and **identical class sets**, the one class shape where it silently drops code, and how to build it | | `references/coverage-and-limits.md` | You need to weigh a claim before trusting it: the evidence behind each covered item, the dependencies this skill does not ship, and the record of what was never exercised | | `references/coverage/route-inventory.md` | You must answer "**can this skill actually do X**" in one pass: the per-route inventory — covered by verified mechanism, covered only by a documented route, the dependencies not shipped, and what is out of scope | | `references/evidence-summary.md` | You need to know **whether a route was verified and how strong the evidence is**, from inside an installed copy: capability → one-line conclusion → strength → where the evidence lives | @@ -185,6 +186,7 @@ _43 row(s) below the header._ | `scripts/kernelsu_syscall_mask.py` | Generate a KernelSU/APatch syscall-masking scaffold: an installable userspace module plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label. **A userspace module cannot change a syscall return value** | | `scripts/capabilities.py` | The capability registry `doctor.py` reads: per capability the requires-closure, what is present, what is missing, an install hint and an estimated cost — so "can this machine do X" is computed rather than guessed | | `scripts/device_shell.py` | Shared device-side command construction: POSIX single-quote escaping for `su -c`, argv arrays rather than string concatenation, and Android identifier validation (`--pkg`, components) so a malformed value is rejected here instead of being parsed by the device shell | +| `scripts/rasc_build.py` | Build and verify **rasc**, the Rust re-implementation of ASC: `--check` what is present, `--build` clone + cargo (no release asset exists), `--verify ` compares its class-definition set against the Python `droidasc` and fails on any difference | | `scripts/scan_leaks.py` | Scan a repository for target identity before it is published: bundle ids in manifest/`pm`/`ps` contexts, serial-shaped tokens, PATs, inline appkey assignments, literal endpoints, host user paths. Built-in do-not-anonymize exemptions, context-bearing findings, `--show-exempt` prints why a hit was suppressed, `--fail-on strong\|any`, exit 0/1/2 with a `RESULT=` token | | `scripts/svc_scan.py` | Name the syscall behind an inline `svc` and the segment it sits in — which decides whether a libc-level hook can observe the call at all. `--context` shows neighbours, because a byte scan also matches data | | `scripts/anti_detect_probe.js` | Observer-only probe (patches nothing): path/loader/thread/kill hooks with **caller module + offset**, an environment self-report (`TracerPid`, frida-named mappings), and live streaming so a sub-second target still yields evidence | diff --git a/skills/apk-reverse/references/toolchain.md b/skills/apk-reverse/references/toolchain.md index 1cb4713..1c4d086 100644 --- a/skills/apk-reverse/references/toolchain.md +++ b/skills/apk-reverse/references/toolchain.md @@ -32,13 +32,37 @@ cross-check with the program headers. |---|---|---| | `baksmali` / `smali` (jars) | `java -cp org.jf.baksmali.Main d -o ` | round-tripping, reading a method precisely | | `dexlib2` | small Java program | method-level rewrite, leaves everything else untouched (`dex-patching.md`) | -| **`droidasc`** (ASC) | `droidasc findrefs app.apk string ` | **the fastest whole-APK cross-reference index** — reach for this FIRST, to *locate*. See below | +| **`rasc`** (Rust ASC) | `rasc findrefs app.apk string ` | **the same index, 4–15× faster**, no Python runtime, a Rust DEX decompiler behind `getclass`. Reach for this FIRST when it is built — its one blind spot is documented in `rasc-and-droidsaw.md` | +| **`droidasc`** (ASC) | `droidasc findrefs app.apk string ` | the original Python index, one `pip install` away, and the cross-check for `rasc`. Reach for it FIRST when `rasc` is not built | | **`ddc`** | `ddc app.apk -c ` | **fastest read of dex as Java, plus query subcommands** — reach for this SECOND, to *read*. See below | | `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources | | `jadx` | `jadx --no-res -d ` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) | | `aapt2` | `aapt2 dump badging ` | manifest facts, package name, versions | | `zipalign`, `apksigner` | from build-tools | alignment and signing | +### rasc — the same index in Rust, when you have built it + +`rasc` is the `rust` branch of the same project (`MG1937/ASC`). Same CLI shape — `classes`, +`manifest`, `getclass`, `findrefs {string,type,method,field}` — no Python at all, and a 2.13 MB +binary. **Measured on this repository's archives: identical class-definition sets on both a 1.4 MB +MASTG challenge and a 34.8 MB app (30,768 classes, 0 differences either way), at 4.0–4.7× for +`classes` and up to 14.7× for `findrefs`.** + +It must be built — there is no release asset and no crate — so the kit wraps that: + +```bash +python scripts/rasc_build.py --check # what is present +python scripts/rasc_build.py --build # needs git + rustup; ~2 min +python scripts/rasc_build.py --verify app.apk # compares class sets against droidasc, fails on a difference +``` + +**The one thing to know before trusting its `getclass`:** an `enum` whose constants override an +abstract method is printed as a bare constant list, and **the per-constant bodies are dropped with no +warning**. Sampling 20 app-like classes from a real app, 17 were judged by the JADX-parity harness and +16 agreed literal for literal; the one that differed was exactly this shape. So: use it to *locate* +and to read ordinary classes, and confirm any enum that shows no bodies with a second reader. The +measurements, the failing class and the build traps are in `references/rasc-and-droidsaw.md`. + ### droidasc (ASC) — ask an APK "who references this?", in one query Install it in one line. No JVM, no Android SDK, no GUI, no first-run indexing: diff --git a/skills/apk-reverse/scripts/rasc_build.py b/skills/apk-reverse/scripts/rasc_build.py new file mode 100644 index 0000000..415b431 --- /dev/null +++ b/skills/apk-reverse/scripts/rasc_build.py @@ -0,0 +1,295 @@ +#!/usr/bin/env python3 +"""Build and verify `rasc`, the Rust re-implementation of ASC. + +Why this script exists: `rasc` has no prebuilt artifact anywhere -- no GitHub release asset, no +crate, and `cargo install rasc` fetches an unrelated maths parser. Adopting it therefore means +building it, and a build that only one machine has ever performed is a build nobody else can +reproduce. This wraps the whole path: toolchain check, clone, build, and the smoke tests that decide +whether the binary is usable rather than merely present. + +What it is for, and where it stops: `rasc` answers the same *location* questions as the Python +`droidasc` -- which classes exist, where a string/type/method/field is referenced, what the manifest +says, and the decompiled source of one class -- several times faster, with a Rust DEX decompiler +(droidsaw) behind `getclass`. It is not a replacement for a full decompiler on every class shape; +see `skills/apk-reverse/references/rasc-and-droidsaw.md` for the measured boundary. + +Usage: + python rasc_build.py --check # is a usable rasc present? (fast) + python rasc_build.py --build # clone + build into a local tools/ tree + python rasc_build.py --verify # compare against droidasc on a real APK + python rasc_build.py --json + +Requires, to build: git, and a Rust toolchain (rustup). On Windows the GNU host toolchain needs a +64-bit MinGW-w64 gcc on PATH -- the 32-bit MinGW that ships with some setups cannot link a 64-bit +binary and fails with "64-bit mode not compiled in". +""" +from __future__ import annotations + +import argparse +import json +import os +import re +import shutil +import subprocess +import sys +import time + +HERE = os.path.dirname(os.path.abspath(__file__)) + + +def _repo_root(): + """Walk up from this script to the repository root, or None when there is not one. + + The script lives at `/skills/apk-reverse/scripts/`, so `tools/_work` is three levels up -- + but a skill installed by `npx skills add` has no repository around it at all. Resolving + `tools/_work` relative to the *script* directory (the first version of this did) looks for + `skills/apk-reverse/scripts/tools/_work`, which never exists, and then reports a working binary + as missing. + """ + cur = HERE + for _ in range(6): + if os.path.isdir(os.path.join(cur, 'tools', '_work')) or \ + os.path.isdir(os.path.join(cur, '.git')): + return cur + parent = os.path.dirname(cur) + if parent == cur: + break + cur = parent + return None + + +ROOT = _repo_root() +# Falls back to a user-level directory when the skill runs outside a checkout, so the tool is not +# pinned to a layout the caller may not have. +DEFAULT_WORK = (os.path.join(ROOT, 'tools', '_work') if ROOT + else os.path.join(os.path.expanduser('~'), '.apk-reverse', 'work')) + +# A cargo/rustc installed to a non-default CARGO_HOME is invisible to a non-interactive PATH, which +# is how this was first run: the toolchain was at /rust/cargo/bin and the check said "missing". +TOOLCHAIN_HINTS = ( + os.path.join(DEFAULT_WORK, 'rust', 'cargo', 'bin'), + os.path.expanduser('~/.cargo/bin'), +) +for _hint in TOOLCHAIN_HINTS: + if os.path.isdir(_hint) and _hint not in os.environ.get('PATH', ''): + os.environ['PATH'] = _hint + os.pathsep + os.environ.get('PATH', '') + +REPO = 'https://github.com/MG1937/ASC.git' +BRANCH = 'rust' +RASC_HINTS = ( + os.environ.get('RASC', ''), + os.path.join(DEFAULT_WORK, 'rust', 'target', 'release', 'rasc'), + os.path.join(DEFAULT_WORK, 'rust', 'target', 'release', 'rasc.exe'), + 'rasc', +) + +EXIT_OK, EXIT_FAILED, EXIT_USAGE, EXIT_CAPABILITY, EXIT_INTERNAL = 0, 1, 2, 3, 4 + + +def find_rasc(): + for hint in RASC_HINTS: + if not hint: + continue + if os.path.isabs(hint): + if os.path.isfile(hint): + return hint + continue + found = shutil.which(hint) + if found: + return found + return None + + +def rasc_version(path): + try: + out = subprocess.run([path, '--version'], capture_output=True, text=True, timeout=60) + except (OSError, subprocess.SubprocessError) as exc: + return None, str(exc) + if out.returncode != 0: + return None, (out.stderr or out.stdout).strip()[:200] + return (out.stdout or '').strip(), '' + + +def toolchain(): + """What this machine has that a build needs, reported rather than assumed.""" + report = {} + for name in ('git', 'cargo', 'rustc', 'cc', 'gcc'): + report[name] = shutil.which(name) + if report['gcc']: + try: + out = subprocess.run([report['gcc'], '-dumpmachine'], capture_output=True, text=True, + timeout=60) + report['gcc_target'] = (out.stdout or '').strip() + except (OSError, subprocess.SubprocessError): + report['gcc_target'] = 'unknown' + return report + + +def build(workdir, jobs=None): + repo = os.path.join(workdir, 'rasc') + if not os.path.isdir(os.path.join(repo, '.git')): + os.makedirs(workdir, exist_ok=True) + cmd = ['git', 'clone', '--branch', BRANCH, '--depth', '1', REPO, repo] + print('+ %s' % ' '.join(cmd)) + rc = subprocess.run(cmd).returncode + if rc != 0: + print('clone failed (rc=%d)' % rc, file=sys.stderr) + return None + env = dict(os.environ) + env.setdefault('CARGO_TARGET_DIR', os.path.join(workdir, 'rust', 'target')) + cmd = ['cargo', 'build', '--release'] + (['--jobs', str(jobs)] if jobs else []) + print('+ %s (in %s)' % (' '.join(cmd), repo)) + t0 = time.perf_counter() + rc = subprocess.run(cmd, cwd=repo, env=env).returncode + print('build finished in %.1f s (rc=%d)' % (time.perf_counter() - t0, rc)) + if rc != 0: + return None + for name in ('rasc', 'rasc.exe'): + candidate = os.path.join(env['CARGO_TARGET_DIR'], 'release', name) + if os.path.isfile(candidate): + return candidate + return None + + +def smoke(binary, apk): + """Ask the same question of rasc and of the Python `droidasc`, and compare the answers. + + Presence is not capability: the check that matters is whether the class-definition set agrees on + a real archive. A build that runs but answers differently is a build that must not be adopted, + which is why this lives next to the build rather than in a note. + """ + if not os.path.isfile(apk): + return {'result': 'usage', 'detail': 'apk not found: %s' % apk} + ours = subprocess.run([binary, 'classes', apk], capture_output=True, timeout=1800) + if ours.returncode != 0: + return {'result': 'failed', 'detail': 'rasc classes rc=%d %s' + % (ours.returncode, ours.stderr.decode('utf-8', 'replace')[:200])} + py = shutil.which('droidasc') + if not py: + n = len(re.findall(rb'L[^;|\s]+;', ours.stdout)) + return {'result': 'partial', 'rasc_classes': n, + 'detail': 'droidasc not on PATH, so only the rasc side was run'} + env = dict(os.environ, PYTHONIOENCODING='utf-8', PYTHONUTF8='1') + theirs = subprocess.run([py, 'listclass', apk], capture_output=True, timeout=1800, env=env) + if theirs.returncode != 0: + return {'result': 'partial', 'detail': 'droidasc rc=%d (its output is not UTF-8 safe on a ' + 'non-UTF-8 console without PYTHONUTF8=1)' + % theirs.returncode} + + def descriptors(blob): + out = set() + for line in blob.decode('utf-8', 'replace').splitlines(): + for tok in re.split(r'[\s|]+', line.strip()): + if re.fullmatch(r'L[^;]+;', tok): + out.add(tok) + break + return out + + a, b = descriptors(ours.stdout), descriptors(theirs.stdout) + return {'result': 'ok' if a == b else 'failed', + 'rasc_classes': len(a), 'droidasc_classes': len(b), + 'only_rasc': len(a - b), 'only_droidasc': len(b - a)} + + +def main(argv=None): + ap = argparse.ArgumentParser( + prog='rasc_build.py', + description='Build and verify rasc, the Rust ASC re-implementation.', + epilog='exit codes: 0 ok, 1 verification failed, 2 usage, 3 a required capability is ' + 'missing, 4 internal error\n' + 'examples:\n' + ' rasc_build.py --check\n' + ' rasc_build.py --build --work tools/_work\n' + ' rasc_build.py --verify tools/_work/apks/sample.apk\n') + ap.add_argument('--check', action='store_true', help='report whether a usable rasc is present') + ap.add_argument('--build', action='store_true', help='clone and build rasc') + ap.add_argument('--verify', metavar='APK', help='compare rasc against droidasc on this APK') + ap.add_argument('--work', default=DEFAULT_WORK, help='where to clone and build (default: %s)' + % DEFAULT_WORK) + ap.add_argument('--jobs', type=int, default=None, help='parallel build jobs') + ap.add_argument('--json', action='store_true') + args = ap.parse_args(argv) + + if not (args.check or args.build or args.verify): + ap.print_help() + print('RESULT=usage') + return EXIT_USAGE + + payload = {'toolchain': {k: v for k, v in toolchain().items()}, 'work': args.work} + binary = find_rasc() + if binary: + version, err = rasc_version(binary) + payload['rasc'] = {'path': binary, 'version': version, 'error': err or None} + else: + payload['rasc'] = None + + if args.build: + missing = [k for k in ('git', 'cargo', 'rustc') if not payload['toolchain'][k]] + if missing: + payload['result'] = 'capability_missing' + payload['next_action'] = ('install Rust (rustup) and git; missing: %s' % ', '.join(missing)) + if args.json: + print(json.dumps(payload, indent=2)) + print('cannot build: missing %s' % ', '.join(missing), file=sys.stderr) + print('RESULT=capability_missing') + return EXIT_CAPABILITY + target = payload['toolchain'].get('gcc_target') or '' + built = build(args.work, args.jobs) + if not built: + payload['result'] = 'build_failed' + payload['next_action'] = ('read the cargo output above; on Windows the GNU host ' + 'toolchain needs a 64-bit MinGW-w64 gcc, and a 32-bit MinGW ' + 'fails with "64-bit mode not compiled into"') + if args.json: + print(json.dumps(payload, indent=2)) + print('RESULT=build_failed') + return EXIT_FAILED + binary = built + version, _ = rasc_version(binary) + payload['rasc'] = {'path': built, 'version': version, 'error': None} + + if args.verify: + if not binary: + payload['result'] = 'capability_missing' + payload['next_action'] = 'run --build first, or set RASC to an existing binary' + if args.json: + print(json.dumps(payload, indent=2)) + print('RESULT=capability_missing') + return EXIT_CAPABILITY + payload['verify'] = smoke(binary, args.verify) + payload['result'] = payload['verify']['result'] + + if args.check and 'result' not in payload: + payload['result'] = 'ok' if binary else 'capability_missing' + if not binary: + payload['next_action'] = 'run --build (needs git + rustup), or set RASC=' + + if args.json: + print(json.dumps(payload, indent=2)) + else: + tc = payload['toolchain'] + print('toolchain : git=%s cargo=%s rustc=%s gcc=%s%s' + % (tc.get('git') or '-', tc.get('cargo') or '-', tc.get('rustc') or '-', + tc.get('gcc') or '-', + ('(%s)' % tc['gcc_target']) if tc.get('gcc_target') else '')) + print('rasc : %s' % (('%s %s' % (payload['rasc']['path'], payload['rasc']['version'])) + if payload['rasc'] else 'not found')) + if 'verify' in payload: + print('verify : %s' % payload['verify']) + print('result : %s' % payload['result']) + if payload.get('next_action'): + print('next : %s' % payload['next_action']) + + print('RESULT=%s' % payload['result']) + return {'ok': EXIT_OK, 'build_failed': EXIT_FAILED, 'failed': EXIT_FAILED, + 'usage': EXIT_USAGE, 'capability_missing': EXIT_CAPABILITY, + 'partial': EXIT_OK}.get(payload['result'], EXIT_INTERNAL) + + +if __name__ == '__main__': + if hasattr(sys.stdout, 'reconfigure'): + try: + sys.stdout.reconfigure(encoding='utf-8') + except (ValueError, OSError): + pass + sys.exit(main())