antior/apk-reverse
1
0
Fork 0
mirror of https://codeberg.org/reverse-skill-lab/apk-reverse synced 2026-11-15 20:04:22 +08:00
Code Issues Packages Projects Releases Wiki Activity

Benchmark pass: measure the documented routes, close five tool gaps, correct four claims

Browse Source 
Public targets under the existing routes, recorded per row in tests/benchmark.md with each row's own strength label: the MASTG crackmes (equal-length patch -> repack -> re-sign with a zero-change control, and the anti-debug race with its time-to-death baseline), DexPatcher's ezAndroid sample, two real split APK sets, and a self-built extraction-skeleton set. Hardening-shape discrimination now separates Java2C from an extraction shell, a VMP and JNI sinking.

New: java2c_probe.py and java2c-and-jni-sinking.md; split-apk.md with repack.py growing set inventory, unified re-signing and merge; protobuf_decode_raw.py cross-checked against the official runtime; vmp_diff_harness.py and its cost table; kernelsu_syscall_mask.py with three kernel-side templates; stalker_trace.js gains Stalker.exclude() and a zero-event warning.

Corrections, each because a measurement contradicted a recorded conclusion: the trivial-body ratio is bimodal rather than a threshold and the old threshold claim was deleted; a Dex-VMP verdict was withdrawn after dexdump contradicted a hand-written opcode table; the assumption that Stalker.exclude() fixes zero-event delivery is disproved (it keeps the process alive, nothing more); and a repack.py signing-path claim expired mid-pass when a second writer added one.

Emulated and kernel-side routes are labelled unverified where no toolchain or device could run them; each evidence file states what was not run.
This commit is contained in:
inliver233
2026-09-21 21:57:23 +08:00
parent bea9955eb6
commit 6221a072ca
29 changed files with 10130 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+58 -4
View File
@@ -77,8 +77,10 @@ measurements.
- Deciding **what the deliverable should be when an APK is not an option** — a repack refused by
several independent checks is *blocked*, not expensive, and the fallback ladder is a system-level
module, a local RPC service, or an honest report with a stated boundary.
- Telling a **real memory dump from an extraction-shell skeleton**, by measuring the trivial-body
ratio instead of eyeballing it — and knowing which recovery route applies, including the root-side
- Telling a **real memory dump from an extraction-shell skeleton** — and knowing that the
trivial-body ratio is *bimodal rather than a threshold*: it reads ~100% for a bare `return-void`
skeleton, but is blind to nop-cleared bodies, throw stubs and partial extraction, which are the
shapes real shells actually use. Then knowing which recovery route applies, including the root-side
dump for when `frida` itself is refused by the target.
- **Calling a routine instead of reversing it** when reversing costs more than invoking: emulated
execution on the host, or a live function service-ified over Frida RPC.
@@ -91,6 +93,15 @@ measurements.
native-side certificate pinning that ignores the system trust store.
- Working **from the phone itself**: MT Manager's edit/repack/sign flow and its APK MCP surface,
LSPosed Manager, and on-device data inspection, alongside the PC toolchain rather than instead of it.
- Telling **Java2C apart from an extraction shell** before spending hours hunting a decrypted DEX that
does not exist at any point in the process lifetime — the code was compiled into a `.so`, and the
shapes are separated by a measured native-density difference of roughly 2000x, not by intuition.
- Handling a build that arrives as a **split APK / App Bundle set**: reading the set off a device,
signing every member with one keystore for `pm install-multiple`, or merging code/native members
into a standalone APK when that is legal.
- Working a **real Dex VMP** with the known-plaintext differential — which links can be automated and
which cannot, what a compiled fixture can and cannot reach, and how to *prove* a derived
private-opcode table instead of asserting one.
## Structure
@@ -207,6 +218,19 @@ references/ loaded on demand, one topic each
gate, and when to stop escalating
on-device-tooling.md working from the phone itself: MT Manager edit/repack/sign and its APK MCP,
LSPosed Manager, Termux+frida, on-device data inspection
java2c-and-jni-sinking.md Java2C and JNI sinking, the two hardening shapes most easily confused
with an extraction shell: the table that separates landing shell /
extraction shell / VMP / Java2C / JNI sinking, why the code is in the
`.so` and *never* in a dumped dex, and why a `Java_*` symbol search
comes back empty (dynamic registration, `-fvisibility=hidden`)
split-apk.md App Bundle / split APK sets: what the set is, pulling it off a device,
merging into one APK vs signing the set as a unit, the install refusals
and what each means, and making an installable fixture from a pulled set
vmp-differential-analysis.md
the known-plaintext differential for a real Dex VMP: which links can be
automated and which cannot (the upload is the bottleneck), the coverage a
compiled fixture can reach, how to *prove* a derived private-opcode
table, smali generation, and when the route is closed
scripts/ parameterized, path-agnostic
doctor.py run this first: capability report + per-script runnability, finds
tools installed off-PATH or as runnable jars, and surfaces the
@@ -242,7 +266,9 @@ scripts/ parameterized, path-agnostic
dart_disasm.py annotated windowed disassembly of Dart AOT code + B/BL caller index
find_refs.py count callers of a method before patching it
repack.py rebuild APK, strip only signatures, keep META-INF/services/, write a
4-byte-aligned archive (resources.arsc STORED+aligned), sign, verify
4-byte-aligned archive (resources.arsc STORED+aligned), sign, verify;
also split APK / App Bundle sets: inventory, sign every member with one
keystore, or merge code/native members into a standalone APK
devsh.py quoting-safe ADB shell helper
usb_net_proxy.py give an offline device network over USB
datastore_inject.py encode/inject AndroidX DataStore preferences safely
@@ -294,6 +320,19 @@ scripts/ parameterized, path-agnostic
with an explicit diagnostic for the measured zero-event case
mt_mcp_probe.py probe MT Manager's on-device APK MCP (Streamable HTTP, port 8787):
JSON-RPC handshake plus the grouped tool inventory
java2c_probe.py collect the evidence that separates Java2C from an extraction shell, a
VMP and ordinary JNI sinking: native density and stub ratio from the dex,
JNI_OnLoad / dynamic registration / toolchain strings from the `.so`,
each item labelled strong/medium/weak
protobuf_decode_raw.py schema-free protobuf decode: hex / file / stdin to a JSON tree, every
length-delimited field kept as a candidate set with ties labelled rather
than guessed, plus a byte-exact re-encode to check a round trip
vmp_diff_harness.py build a labelled opcode-coverage fixture, derive a candidate private-
opcode map from an original/hardened dex pair, verify the comparison in a
closed loop, and render a restored stream as a smali skeleton
kernelsu_syscall_mask.py generate a KernelSU/APatch syscall-masking scaffold: an installable
userspace module skeleton plus KPM/LKM/eBPF kernel-side templates, each
with its version gate and an explicit unverified label
```
## Install
@@ -383,7 +422,13 @@ tier of documented routes: **module-side delivery** when a repack is blocked,
**extraction-shell recovery** and its VMP boundary, **emulation and live RPC** for
calling rather than reading, **instruction-level tracing** against OLLVM, **protocol
reversing** beyond REST, the **kernel-side route** map for when userspace hooking is
provably out of reach, and **on-device tooling**. Unity/IL2CPP logic recovery,
provably out of reach, and **on-device tooling**. A **benchmark pass** then put public
targets under those routes (`tests/benchmark.md`): it added **Java2C discrimination** (the
misdiagnosis that sends an agent hunting a decrypted DEX that never exists), **split APK /
App Bundle handling**, **schema-free protobuf decoding**, a **Dex-VMP differential** harness,
and **kernel-module templates with their version gates** — and it corrected two earlier
claims, replacing the trivial-body *threshold* with a measured bimodal result and reopening
a VMP verdict that a hand-written opcode table had got wrong. Unity/IL2CPP logic recovery,
React Native/Hermes bytecode internals, and defeating a server-side authority are **not**
covered, and the skill is written to say so and stop rather than apply the nearest
documented procedure to a target it was not written for.
@@ -402,6 +447,10 @@ Three qualifications that the Coverage section states in full and that belong he
`docs/tool-verification/EXTENSION-*.md`, one file per topic, with its own strength note. The
common shape there is *the tool was measured, the route was not* — so read those files before
treating any of the newer documents as a verified path.
- **The benchmark pass is recorded per row, with that row's own strength.** `tests/benchmark.md`
names each public target, the scripts the row exercises, what actually happened (including the
rows that failed and the rows nobody ran), and how strong the evidence is. Rows marked
`unverified` are statements about the evidence in this repository, not about the mechanism.
## Repository maintenance
@@ -415,6 +464,11 @@ check_refs.py every cross-reference that names a section of another
build_scripts.py audit for machine-specific leftovers (absolute paths, credentials)
```
`tests/benchmark.md` holds the **regression matrix**: dimension -> public target -> the scripts the
row exercises -> measured result -> strength label. It is the checklist to re-run before trusting
any claim under `docs/tool-verification/`. Samples are downloaded into `tools/_work/` and are never
committed, so each row names its public source and records the hash it was run against.
`docs/tool-verification/` is not part of the installed skill either. It is the evidence record
for one measurement pass against a real target: what each script actually did, which independent
method confirmed it, which defects were found, and which scenarios the target could not exercise.
docs/tool-verification/EXTENSION-benchmark-l1-l3.md
+638
View File
@@ -0,0 +1,638 @@
# Extension verification — benchmark B1/B2 against OWASP MASTG Crackmes
Covers the pass that turned the `tests/benchmark.md` rows **B1** (equal-length dex patch + repack +
re-sign installability, gates G2/G4) and **B2** (anti-debug / self-destruct race) from `pending`
into measured results.
Strength labels follow `docs/tool-verification/README.md`: **observed** (exact command and its
output reproduced here), **inferred** (follows from observed facts, the step itself not executed),
**unverified** (assumed or reported and not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0, host frida 16.7.19 (frida-tools 13.7.1),
JDK 17.0.4.1 at `C:\Program Files\Java\jdk-17.0.4.1` (not on `PATH`), `apksigner.bat` /
`zipalign.exe` at `E:\tools\bin`. Device: **`<DEVICE>`**, Android 11 / API 30, arm64-v8a, root via
`su`, Magisk denylist `enforced` with an **empty** list (no app is root-hidden). Package under test:
**`<PKG>`**.
All device operations ran under the shared device lock (`tools/_work/devlock.py`).
---
## 0. Headline
- **B1 is `observed` and positive.** An equal-length dex edit (2 bytes of a conditional branch
replaced by `nop`s), through `dex_patch_bytes.py` → `repack.py --no-sign` → `zipalign` →
`apksigner` → root `pm install`, installs and **demonstrably changes the app's behaviour on
screen**: the original and a zero-change control both show a blocking `Root detected!` dialog, the
patched build reaches `MainActivity` with the dialog gone.
- **`repack.py`'s default signing path did not work when this pass started, and the blocker was not
`apksigner`.** `apksigner`/`zipalign` are both present and both work; at that moment `repack.py`
signed only through an `uber-apk-signer` jar, which is not installed. **The repository changed
mid-pass** — another writer landed a `sign_with_apksigner()` route — and re-running the same
command then produced a fully signed, `[result] OK` APK in one invocation with byte-identical
output to the manual route (`sha256 ad6a51ce…7e56`). Both states are recorded below, because the
stale state is what `tools/_work/ENV.md` predicted and the new state is what a reader will meet.
This is also a reminder that a "the tool cannot do X" finding on a shared worktree has an expiry
date.
- **B2 is a split result.** The two scripts run and the probe writes its bytes (0.04–0.05 s after
resume, gated on `PATCHED`), so the *tooling* is verified. But the **patch does not prevent the
target's termination**: `libfoo.so` aborts through `goodbye()+12` at **3.22 s** after resume, and
three patched sites including `goodbyev` itself do not change that. Neutralising the obvious death
site is **not sufficient** on this target.
- **A finding that changes the whole B2 setup:** on this device the **unmodified, original APK
already reports `Rooting or tampering detected.`** and its tamper path runs before anything is
attached. L3's native CRC self-check fails against the pristine community APK on Android 11 /
arm64-v8a, so the "pristine baseline" the row assumed does not exist here.
---
## 1. Gate G2 and G4 — environment and control (`observed`)
`scripts/doctor.py`:
```
platform : Windows 11 / AMD64
python : 3.14.0
[OK ] static (dex/zip/strings) [OK ] repack + sign
[OK ] native ELF patching [-- ] jadx decompile
[-- ] apktool unpack [OK ] device work
[OK ] root on device [OK ] frida dynamic
[OK ] java 17.0.4.1 [OK ] adb 1.0.41 [OK ] frida 16.7.19
[-- ] apktool not on PATH
[-- ] unzip not on PATH
java jars found near this skill : (none)
device : <DEVICE> arm64-v8a Android 11 (sdk 30) root: yes
clock skew: 3.06 s
```
The `(none)` on the jar line matters for B1: no `baksmali`/`smali`/`uber-apk-signer` jar exists, so
every signing route that goes through a jar is unavailable before a single command is run.
**Keystore and tool locations** (recorded because `repack.py` resolves both off-`PATH`):
```
keytool : C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe
jarsigner : C:\Program Files\Java\jdk-17.0.4.1\bin\jarsigner.exe
apksigner : E:\tools\bin\apksigner.bat
zipalign : E:\tools\bin\zipalign.exe
```
`repack.py`'s `sibling_tool()` cannot find `keytool` from `C:\Program Files\Common Files\Oracle\Java\javapath\java.exe`,
because that directory holds only forwarders. Passing `--keytool` explicitly is required.
---
## 2. B1 — L1 equal-length dex patch, full chain (`observed`)
### 2.1 Source facts
```
$ python -c "... load_dex('UnCrackable-Level1.apk') ..."
classes.dex 5528 bytes
magic dex\n035\0
checksum field 7fb7d9fa
adler32(d[12:]) 7fb7d9fa -> matches
signature field b7fafe72cb521450c4470043caa332da61d1bec7
sha1(d[32:]) 5253237b04c2d002967bae8ef81daae08550c90c
-> MISMATCH
```
Two facts that a reader needs before trusting the tool's output:
- The **shipped dex's `signature` field is stale** while its `checksum` is correct. `dex_patch_bytes.py`
prints `header before: checksum_ok=True signature_ok=False` and explains it. If the tool had
written the two fields in the wrong order, that starting state would have hidden the bug.
- The class that owns the behaviour is
`Lsg/vantagepoint/uncrackable1/MainActivity;.onCreate(Landroid/os/Bundle;)V`, and the gate is the
first arm of a three-way root `OR`:
```
0x9c8 invoke-static Lsg/vantagepoint/a/c;.a()Z
0x9ce move-result
0x9d0 39000e00 if-nez v0 -> 0x9ec <-- the edit site
0x9d4 invoke-static Lsg/vantagepoint/a/c;.b()Z
```
`0x9ec` is `const-string "Root detected!"` followed by the private `a(String)` that builds the
blocking dialog. **The branch structure, not the method name, was read before editing** — the edit
target is the arm whose *fall-through* continues into the next check.
### 2.2 The patch
```json
[{
"name": "l1-first-root-check-fallthrough",
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
"method": "onCreate", "desc": "(Landroid/os/Bundle;)V",
"match": {"kind": "if-testz", "bytes": "39000e00"},
"expect_next": {"kind": "invoke", "target_class": "Lsg/vantagepoint/a/c;",
"target_method": "b"},
"replace": {"kind": "nops"}
}]
```
```
$ python skills/apk-reverse/scripts/dex_patch_bytes.py tools/_work/bench/l1l3/dex/classes.dex \
--spec tools/_work/bench/l1l3/l1_root.spec.json --dry-run
-- [l1-first-root-check-fallthrough] Lsg/vantagepoint/uncrackable1/MainActivity;.onCreate(...)V
site 0x9d0 0x9d0: 39000e00 if-nez v0 -> 0x9ec
(before) 0x9c8: 710027000000 invoke-static Lsg/vantagepoint/a/c;.a()Z
(before) 0x9ce: 0a00 move-result
-> 0x9d0: 00000000 if-nez
(after) 0x9d4: 710028000000 invoke-static Lsg/vantagepoint/a/c;.b()Z
(after) 0x9da: 0a00 move-result
polarity: expect_next satisfied
== dry run: nothing written
```
`expect_next` is what pins the polarity: the instruction after the branch had to be the *next* root
check, so the fall-through is the "continue" side. Applied:
```
== header checksum 0x7fb7d9fa -> 0x86e1d928
signature b7fafe72cb521450 -> eb632328fd1ba03c
self-verify: checksum_ok=True signature_ok=True
== wrote classes.patched.dex (5528 bytes, delta 0)
```
**The header order question, answered by byte comparison rather than by the tool's own report**
(`observed`): the diff of the whole file against the source is exactly the two fields plus the patch
site.
```
sizes 5528 5528
diff offsets raw: ['0x8','0xa','0xb','0xc'...'0x1f', '0x9d0','0x9d2']
site 0x9d0 orig 39000e00 patched 00000000
8..12 fad9b77f -> 28d9e186 (checksum, bytes 8..12)
12..32 b7fafe72cb521450c4470043caa332da61d1bec7
-> eb632328fd1ba03c7ebc6df181e0708a761aa2e3 (signature, bytes 12..32)
selfverify checksum True signature True
```
Recomputing `zlib.adler32(d[12:])` and `hashlib.sha1(d[32:])` **on the written file** reproduces both
header fields, which is only possible if the signature was computed before the checksum. The patch
owns 4 bytes and the header owns 24; nothing else moved.
### 2.3 Repack — where the documented pipeline actually fails
`repack.py`'s default signing route:
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dex classes.dex=classes.patched.dex --out out/l1_patched.apk \
--workdir out --ks out/bench.keystore \
--keytool ".../jdk-17.0.4.1/bin/keytool.exe" \
--jarsigner ".../jdk-17.0.4.1/bin/jarsigner.exe" \
--apksigner E:\tools\bin\apksigner.bat --zipalign E:\tools\bin\zipalign.exe
[zip] - META-INF/CERT.RSA (signature artifact)
[zip] - META-INF/CERT.SF (signature artifact)
[zip] - META-INF/MANIFEST.MF (signature artifact)
[zip] + classes.dex (5528 bytes <- .../classes.patched.dex)
[zip] unsigned written: 60967 bytes
[ks] generated E:\...\out\bench.keystore (alias=apkreverse)
error: signer jar not found: uber-apk-signer.jar
pass --signer-jar <path>, or set APK_SIGNER_JAR.
Any zipalign+apksigner based signer works here.
[exit code: 1]
```
Two separate facts, and the second is the one worth recording:
1. `--signer-jar` defaults to `uber-apk-signer.jar` and no such jar exists here, so the default run
cannot sign. (`ENV.md` predicted this.)
2. **The error message's second line is misleading in this environment.** "Any zipalign+apksigner
based signer works here" is true, but `repack.py` has no code path that *uses* a plain
`zipalign`+`apksigner` pair for signing — `sign_apk()` only ever invokes
`java -jar <signer_jar>`. `--apksigner`/`--zipalign` are used for **verification only**. So the
documented Route B works, but only if the caller performs the sign step outside `repack.py`.
The `--no-sign` flag is the supported seam.
Working route, both builds through the **same** pipeline:
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dexdir tools/_work/bench/l1l3/dex_orig \
--out out/l1_control_unsigned.apk --workdir out/ctrl --no-sign
[zip] unsigned written: 60967 bytes
[zip] resources.arsc STORED offset=0xd4d4 aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
[result] UNSIGNED (--no-sign): good for inspection, not installable as-is
$ (same, with --dex classes.dex=classes.patched.dex) -> out/l1_patched_unsigned.apk
[zip] resources.arsc STORED offset=0xd4d4 aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
```
`AndroidManifest.xml` is reported `STORED ... NOT ALIGNED`, which is correct and not a defect: the
4-byte alignment requirement is `resources.arsc` (and uncompressed `lib/*.so`), not the manifest.
Signing, explicitly, both builds, same key:
```
$ E:\tools\bin\zipalign.exe -p -f 4 l1_control_unsigned.apk l1_control_aligned.apk ; rc=0
$ E:\tools\bin\apksigner.bat sign --ks bench.keystore --ks-pass pass:<pw> --key-pass pass:<pw> \
--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true \
--out l1_control.apk l1_control_aligned.apk ; rc=0
$ E:\tools\bin\apksigner.bat verify --print-certs --verbose \
--min-sdk-version 21 --max-sdk-version 34 l1_control.apk
Verifies
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
Signer #1 certificate SHA-256 digest: ad13912308e0b6ded5d8be7da2868356348655033d1dde402d8f70d452fbf6db
```
Identical result for `l1_patched.apk`, with the same certificate digest, so the two builds differ
only in the dex bytes.
### 2.4 On-device result — the behaviour change, seen on the screen (`observed`)
Two builds were produced and both were checked, because "the pipeline works" and "the patch changed
the app" are different claims and only the second one is the deliverable.
**The one-command pipeline, re-run after the repository changed** (this supersedes §2.3's manual
route as the recommended invocation):
```
$ python skills/apk-reverse/scripts/repack.py --apk UnCrackable-Level1.apk \
--dex classes.dex=classes.patched.dex --out out/l1_patched_final.apk \
--workdir out/fin --ks out/bench.keystore \
--keytool "C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe" \
--apksigner E:\tools\bin\apksigner.bat --zipalign E:\tools\bin\zipalign.exe
[sign] route: apksigner
[out] ...\out\l1_patched_final.apk (70647 bytes)
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
[result] OK
local sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56
```
The manual `zipalign`+`apksigner` route earlier in this section produced **the same sha256**, so the
two paths agree byte for byte and the result does not depend on which one a reader follows.
Install through the **root** path (`pm install -r -d`, which bypasses this ROM's streamed-install
interception), and the installed `base.apk` is hashed and compared to the local file so "installed"
cannot be confused with "the previous build is still there":
```
control device sha256 e5a9f335155e3e20219d89bc859ae89cc0a82f7135123397c526d94bfca4b3b4
local sha256 e5a9f335155e3e20219d89bc859ae89cc0a82f7135123397c526d94bfca4b3b4 identical
patched device sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56
local sha256 ad6a51cef4699b73ff052d4e049ca9efd64e9570900d19509f0b7a7c20f87e56 identical
```
`am start -W` for both builds:
```
LaunchState: COLD TotalTime: 241 ms WaitTime: 249 ms Status: ok (control)
LaunchState: COLD TotalTime: 232 ms WaitTime: 234 ms Status: ok (patched)
```
Window focus, sampled four times at 2 s intervals:
```
original mCurrentFocus=Window{a5d2ba7 u0 Root detected!} (all four samples)
control mCurrentFocus=Window{c4047e2 u0 Root detected!} (all four samples)
patched mCurrentFocus=Window{8c94fc1 u0 <PKG>/...MainActivity} (all four samples)
```
And the images, which are the evidence that actually counts —
`tools/_work/bench/l1l3/evidence/00_original/00_original_03.png` shows the modal
`Root detected! / This is unacceptable. The app is now going to exit. / OK` over a greyed-out input
screen. The same shot on the control build (`evidence/01_control/*.png`, and again on
`evidence/12_control_final/*.png` from the final one-command build) is byte-comparable to the
original's. `evidence/13_patched_final/13_patched_final.png` — the final patched build — shows the
screen **without any dialog**: input field live with the caret in it, `VERIFY` enabled, and
`mCurrentFocus` on `<PKG>/...MainActivity`.
**G4 is therefore cleared in the strong form**: the control is not just "the same APK" — it went
through the identical repack-and-sign pipeline with a byte-identical dex and still shows the old
behaviour. The behaviour change is attributable to the four patched bytes, and it reproduced on a
second, independently built APK from the final pipeline.
### 2.5 What the app's own check actually is (`observed`, and it contradicts the obvious reading)
The functional check was probed with a Frida hook rather than inferred from the crypto:
```
[a.b] input="8d127684cbc37c17616d806cf50473cc" -> 16 bytes: 8d127684cbc37c17616d806cf50473cc
[lib a.a] key=8d127684cbc37c17616d806cf50473cc
data=e5426215cb5b9a06c3a0b5e6a4bd769a49e8f074f82eff1d95ab7c17147618e7
-> 17 bytes 492077616e7420746f2062656c69657665 ascii="I want to believe"
[a.a] REAL input="" -> false
```
So: a **fixed 16-byte AES key** (`8d12…73cc`, a `const-string` in the dex) decrypts a fixed
base64 blob to `I want to believe`, and the user's input is compared against that plaintext. My first
reading — that the hex string was itself the secret — is **wrong**, and it was falsified by running
it: entering `8d127684cbc37c17616d806cf50473cc` produces `Nope...`.
`TextUtils.equals` is **not** the comparison site: hooking it captured only dialog-internal
comparisons (`("Nope...", "<null>")`). The comparison lives in the decrypted-bytes path of
`a.a(String)`.
Two device facts that cost time and are worth inheriting:
- **The input field cannot be driven on this ROM.** `adb shell input text "I%swant%sto%sbelieve"`
and a per-character `input keyevent` loop both landed `iwantto`: 7 of 17 characters survived and
every space was dropped. Three independent quoting routes gave the same result, so the field
itself rejects what the injection path sends. Reading the field back with `uiautomator dump` after
each attempt is what distinguished "the app said Nope" from "my input never arrived" — the first
two attempts looked like a wrong secret and were not.
- `EditText.setText(String)` is not reachable from outside (only
`setText(CharSequence, TextView$BufferType)` is declared, and `TextView.BufferType` is not exposed
as a property on the class object); an **empty** `EditText` also returns a **null** `Editable`, so
`getText().length()` fails with "not a function" and looks like a bridge limitation rather than an
empty field.
The success branch was therefore reached through the app's own method with the boolean forced, which
is honest as long as it is labelled — and it is:
`evidence/09_success_branch/09_success_branch.png` shows the app's `Success! / This is the correct
secret.` dialog. That image proves **the patched build reaches and renders the app's own success
branch**; it does not prove the secret was typed and accepted. The check itself is evidenced by the
libfoo/`a.b`/`a.a` log above.
### 2.6 B1 verdict
`observed`: the equal-length edit, the header recomputation in the correct order, the STORED+aligned
repack, v1+v2+v3 signing, the root-path install, the device-side hash match, and the on-screen
behaviour change all reproduce. G2 and G4 both clear.
---
## 3. B2 — L3 self-destruct race (`observed` for the tooling, `unverified`/negative for the patch)
### 3.1 What L3 actually does (read from the dex and the tombstone, not assumed)
`MainActivity.onCreate` calls `verifyLibs()` **first**, then `init("pizzapizzapizzapizzapizz".getBytes())`,
then an `AsyncTask`, then three root checks, then `isDebuggable()`, then `setContentView` — and
`setContentView` sits **after** the gate:
```
0x11a3ce invoke-static Lsg/vantagepoint/util/RootDetection;.checkRoot1()Z
...
0x11a406 sget Lsg/vantagepoint/uncrackable3/MainActivity;.tampered:I
0x11a40a if-eqz v0 -> 0x11a418 ; tampered == 0 -> skip the dialog
0x11a40e const-string "Rooting or tampering detected."
0x11a412 invoke-direct MainActivity;.showDialog(Ljava/lang/String;)V
0x11a418 ... setContentView
```
So when the tamper bit is set the app **never renders its UI**; it sits on a blocking dialog whose OK
button is wired to an `onClick` that exits. `verifyLibs()` is the source of that bit:
- it reads four ABI-keyed expected CRCs out of `resources.arsc` string resources
(`R.string` ids `0x7f0b002b`, `0x7f0b0029`, `0x7f0b0034`, `0x7f0b0035`),
- compares them to the zip entry CRCs of `lib/<abi>/libfoo.so`, and
- compares `classes.dex`'s zip CRC to the value the native `baz()` returns.
The native library exports exactly three JNI entries and one terminate routine:
```
Java_sg_vantagepoint_uncrackable3_CodeCheck_bar vaddr=0x33a4 size=256
Java_sg_vantagepoint_uncrackable3_MainActivity_init vaddr=0x3310 size=136
Java_sg_vantagepoint_uncrackable3_MainActivity_baz vaddr=0x3398 size=12
_Z7goodbyev vaddr=0x3080 size=24
_Z4randv vaddr=0xe4c size=36
```
and its strings name the detection inputs: `/proc/self/maps`, `frida`, `xposed`, `ptrace`,
`waitpid`, `getppid`, `pthread_create`, plus two log strings:
`"Tampering detected! Terminating..."` and `"Error opening /proc/self/maps! Terminating..."`.
`libfoo.so`'s single RX `PT_LOAD` is `off=0x0 vaddr=0x0 filesz=0x4700`, so **file offset == virtual
address == runtime offset from the module base** for everything in `.text`. That is what makes
`hook_patch_only.js`'s `FILE_OFFSET` meaningful here, and it is not true in general — a library whose
text segment has a non-zero `vaddr` would need `p_vaddr + delta`.
### 3.2 The baseline, and the finding that invalidates the assumed baseline
`observed`: the **unmodified, pristine APK** on this device shows the tamper dialog:
```
$ am start -n <PKG>/sg.vantagepoint.uncrackable3.MainActivity
833 ms pid=5377 5413 mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
1638 ms ... mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
...
10712 ms ... mCurrentFocus=Window{1d9ee56 u0 Rooting or tampering detected.}
```
and `logcat` shows the CRCs it compared (`V/UnCrackable3`), with `tampered` set. The process stays
alive in this state for at least 25 s: the tamper dialog alone does **not** kill the process, it waits
for the OK button.
`observed` — the **time-to-death baseline with a frida probe attached**, which is the number the row
needs:
```
$ python ttd_attach.py --package <PKG> --activity sg.vantagepoint.uncrackable3.MainActivity \
--js l3_read_crc.js --launch-delay 1.5 --seconds 20
[i] pre-attach main pid=7020 at 1.88s
[i] attached at +0.00s (probe loaded)
attach+ 0.75s main_pid=7020
... (alive)
attach+ 4.90s main_pid=7020
attach+ 5.34s main_pid=-
=== logcat ===
F/libc ( 7020): Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 7052 (tg.uncrackable3)
F/DEBUG ( 7203): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
F/DEBUG ( 7203): #02 pc 00000000000031ac .../lib/arm64/libfoo.so
I/Zygote( 1007): Process 7020 exited due to signal 6 (Aborted)
```
**`TTD(frida attached) = 5.34 s`, and the death is a native `SIGABRT` originating in `goodbye()+12`.**
Note which process dies: `7020` is the main `<PKG>` process, `7052`/`tg.uncrackable3` is the failing
*thread*. That distinction matters — see §3.4.
### 3.3 The two scripts, run as shipped (`observed`)
Neither had ever been executed before this pass.
```
$ python skills/apk-reverse/scripts/spawn_patch_detach.py \
--package <PKG> --js skills/apk-reverse/scripts/hook_patch_only.js \
--captures 2 --interval 3 --no-screencap
[i] spawned pid=7442
[!] PATCH_BYTES is empty -- configure this probe before using it
[MSG] PATCHFAIL
[i] patched=False
[i] detached (memory writes persist, hooks are gone)
=== capture 0 ===
7567 00:01 sh -c ps -A -o PID,ETIME,ARGS | grep -i uncrackable3
7570 00:01 grep -i uncrackable3
mCurrentFocus=Window{7c35f28 u0 com.oppo.launcher/com.oppo.launcher.Launcher}
```
Three defects, all confirmed by later runs:
1. **`hook_patch_only.js` is a template, and its guard is correct but the guard is the whole
result.** `PATCH_BYTES = []` correctly refuses and reports `PATCHFAIL`, so an unconfigured run
cannot silently claim success. That is good behaviour; it just means the shipped file tests
nothing.
2. **`spawn_patch_detach.py` resumed the target before waiting for `PATCHED`.** The original order
was `resume(pid)` → then wait. That makes the wait meaningless: by the time the probe's
`Process.findModuleByName` poll sees the library, the target has been running.
3. **`--activity` defaults to `None` and the code then did nothing.** After `am force-stop` and a
spawn, omitting `--activity` left the app un-launched, so the capture set was of the launcher and
read exactly like "the app died". Fixed by resolving the launcher through
`cmd package resolve-activity --brief <pkg>` when `--activity` is absent.
**Correction to my own first fix.** I changed the order so the driver held the target **paused**
until `PATCHED`. That cannot work with a module-polling probe, and the run says so:
```
[i] probe loaded; target still PAUSED, waiting up to 15.0s for the patch
[5.00s][alive] [10.00s][alive] [15.00s][alive]
[!] the probe did not report PATCHED in 15.0s
[i] resumed pid=7666
[15.11s][i] libfoo.so base=0x764c0aa000 bytes@0x3080 = fd 7b bf a9 ...
[15.12s][PATCH] now = 00 00 80 52 d6 03 5f c0 ...
```
**While the process is frozen its libraries are not mapped**, so the poll can never succeed: 15 s
paused produced nothing, and `libfoo.so base=0x764c0aa000` appeared 0.11 s after resume. The shipped
order was right about *that*; what was wrong was detaching before the write landed. The corrected
driver resumes, then waits for `PATCHED` (reported loudly when it never arrives), then detaches.
### 3.4 The experiment, and its result
Sites were chosen from the tombstone and the ELF, not guessed. `goodbyev` at `0x3080`:
```
0x3080 a9bf7bfd stp x29, x30, [sp, #-16]!
0x3084 910003fd mov x29, sp
0x3088 321f07e0
0x308c 97fff725 bl 0x2ff0 <-- the call the tombstone names as goodbye()+12
0x3090 2a1f03e0
0x3094 97fff71f bl 0x2ff8
```
and the two helpers it tail-calls are trivial return-value stubs:
```
0x2ff0 52801a40 mov w0, #0xd2 0x2ffc d65f03c0 ret
0x2ff8 52800cc0 mov w0, #0x66 0x3000 321f0fe0 ...
```
So probe v2 patches the **return values the branch reads** (`0x2ff0`, `0x2ff8`) as well as
`goodbyev`'s entry, each rewritten to `mov w0, #0 ; ret` (+ nops, equal length), with an assertion on
the expected original bytes before each write.
Patched condition:
```
$ python b2_measure.py --js l3_hook_patch_v2.js --tag PATCHED --probe-seconds 25
[i] spawned pid=10000 (paused)
[i] resumed pid=10000
[0.04s][i] libfoo.so base=0x764b89d000 size=90112
[0.04s][i] goodbye entry = fd 7b bf a9 fd 03 00 91 e0 07 1f 32 25 f7 ff 97
[0.04s][i] goodbye+12 = 25 f7 ff 97
[0.04s][PATCH] 0x2ff0 (helper A -> return 0) 40 1a 80 52 c0 03 5f d6 -> 00 00 80 52 d6 03 5f c0
[0.05s][PATCH] 0x2ff8 (helper B -> return 0) c0 0c 80 52 c0 03 5f d6 -> 00 00 80 52 d6 03 5f c0
[0.05s][PATCH] 0x3080 (goodbyev entry -> no-op) fd 7b bf a9 ... -> 00 00 80 52 d6 03 5f c0 1f 20 03 d5 1f 20 03 d5
[0.05s][i] sites written=3 skipped=0
[i] patched=True
3.22s pid=10000 alive=False
V/UnCrackable3(10000): Tampering detected! Terminating...
F/libc (10000): Fatal signal 6 (SIGABRT), code -6 (SI_TKILL) in tid 10032 (tg.uncrackable3), pid 10000
F/DEBUG (10036): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
I/Zygote( 1007): Process 10000 exited due to signal 6 (Aborted)
[RESULT] tag=PATCHED spawned=10000 ttd_after_resume=3.22 survived=False
```
Control condition (identical reads and writes, original bytes written back):
```
$ python b2_measure.py --js l3_hook_control_v2.js --tag control_v2 --probe-seconds 22
[0.04s][CONTROL] 0x2ff0 unchanged (wrote back 40 1a 80 52 c0 03 5f d6)
[0.04s][CONTROL] 0x2ff8 unchanged (wrote back c0 0c 80 52 c0 03 5f d6)
[0.04s][CONTROL] 0x3080 unchanged (wrote back fd 7b bf a9 fd 03 00 91)
[0.04s][i] control sites ok=3
V/UnCrackable3(9312): Tampering detected! Terminating...
F/DEBUG ( 9348): #01 pc 000000000000308c .../lib/arm64/libfoo.so (goodbye()+12)
I/Zygote( 1007): Process 9312 exited due to signal 6 (Aborted)
```
**Both conditions terminate at `goodbye()+12` with `SIGABRT`.** The probes apply the patch
successfully, inside the race window (0.04–0.05 s after resume against a ~3.2 s deadline), and the
target still dies.
### 3.5 What this means, and what to do instead
`observed`:
- `spawn_patch_detach.py` **can** deliver an early memory patch: the write landed 0.05 s after resume
and survived the detach. The mechanism the row tests is real.
- `hook_patch_only.js` is a **template with correct guard rails**, not a working probe.
- The patch as configured is **insufficient**: three sites including the terminate routine's entry
and both of its branch inputs do not stop the abort.
`inferred` (labelled, not executed): the terminate path is not confined to `goodbyev` and its two
stubs. `goodbyev` is 24 bytes and its only two calls are the two stubs; the `bl` at `+12` is the
observed faulting frame, and yet patching both the callee and the caller changed nothing. The most
consistent explanation is that the tampering thread reaches the *same abort* through a second site
that this static pass did not map — the region around `0x2f80–0x3080` is a dense run of
`mov w0, #imm ; ret` pairs (values 0x11, 0x1b, 0x43, 0x5b, 0x67, 0x7b, 0x92, 0xa2, 0xa3, 0xae, 0xcc,
0xd2, 0xe3, 0xe7, 0xe8, 0xed, 0xf6, 0xfd, …), i.e. a jump-table-shaped cluster of tiny accessors
that a proper disassembly would resolve into a call graph, and `rand` (`_Z4randv`, 0xe4c, 36 B) is
part of the same obfuscated scheme. **A real fix needs a full AArch64 disassembly of `.text`
(~18 KB) rather than four hand-picked offsets.**
`observed` — the route that should be tried first next time: the abort is reachable from a **Java**
layer that is already fully mapped. `verifyLibs()` computes `tampered` and `onCreate` branches on it,
and the dialog's OK button is a plain `DialogInterface.OnClickListener` in `MainActivity$1`. Patching
the dex in that class (`if-eqz v0 -> 0x11a418` → nops, forcing the non-tamper side) removes the
dialog and lets `setContentView` run **without touching native code at all** — and unlike the native
patch it is a route this repository already has measured tooling for (see B1). This was not executed
in this pass; it is the recommended next experiment, not a result.
`observed` — a limitation the next pass must budget for: on this device the L3 **original** APK
already fails its own CRC self-check, so the tamper state is on before any instrumentation. Whatever
the follow-up tests must therefore be measured against *that* baseline, not against a clean one.
---
## 4. Task C — L2 (not run)
Not attempted: A and B consumed the pass. `UnCrackable-Level2.apk` is on disk
(`tools/_work/bench/apks/`, 901022 B, sha256 prefix `4c7980ef1f6cc295`) and the `dynamic-frida.md`
spawn route is untested for it. Recorded as **not run**, not as a negative result.
---
## 5. Defects found in the repository's own tooling
| Where | What | Evidence | Status |
|---|---|---|---|
| `scripts/repack.py` (`sign_apk`) | **Stale finding, re-opened and closed.** At the start of this pass it signed **only** via `java -jar <signer_jar>`; `--apksigner`/`--zipalign` were verification-only, so the "any zipalign+apksigner based signer works here" hint had no code path behind it | §2.3 exit 1 with `signer jar not found` | **fixed by another writer mid-pass**; re-measured in §2.4 — `[sign] route: apksigner`, `[result] OK`, byte-identical output |
| `scripts/repack.py` | The `signer jar not found` message is still emitted by the jar-only `sign_apk()` and still claims a zipalign+apksigner route works. When no `--apksigner` was supplied (the first run in §2.3) the message is the *only* guidance a reader gets, and it points at a function that does not exist | §2.3 | documented; the message should name the apksigner route it now has |
| `scripts/spawn_patch_detach.py` | Detached before the probe's write landed, so the write could be missed entirely while the run still reported `patched=False` and looked merely unlucky | §3.3 | **fixed** — resume, wait for `PATCHED`, report loudly, then detach |
| `scripts/spawn_patch_detach.py` | `--activity` absent left the target un-launched after a spawn, so the captures were of the launcher | §3.3 | **fixed** — resolve the launcher via `cmd package resolve-activity` |
| `scripts/spawn_patch_detach.py` | Unbounded `adb`/frida calls; `adb_run()`'s rc is never inspected for the launcher step | §3.3 | partially fixed (`--launch-timeout`, rc surfaced) |
| `scripts/hook_patch_only.js` | Guard rails are correct (`PATCH_BYTES` empty → `PATCHFAIL`); it is a template, and the docs imply it is a runnable probe | §3.3 | documented |
| `tools/_work/ENV.md` | Attributes the B1 signing problem to the apksigner/zipalign route; the blocker was the missing uber-apk-signer jar, and both apksigner and zipalign work | §2.3 | corrected here |
| `tests/benchmark.md` B2 | Describes L3 as a self-destruct race against a pristine baseline; on this device the original APK is already in its tamper state | §3.2 | corrected here |
---
## 6. Files produced by this pass
Workbench (not part of the skill; `tools/` is git-ignored):
- `tools/_work/bench/l1l3/l1_root.spec.json`, `dex/classes.patched.dex`, `dex_orig/classes.dex`
- `tools/_work/bench/l1l3/out/l1_control.apk`, `l1_patched.apk`, `l1_control_now.apk`,
`l1_patched_final.apk` (+ unsigned/aligned intermediates)
- `tools/_work/bench/l1l3/l1_probe.py`, `l1_type.py`, `l1_verify_field.py`, `l1_try_secret.py`,
`l1_drive.js`, `l1_probe_full.js`, `attach_probe.py`
- `tools/_work/bench/l1l3/l3_hook_patch_v2.js`, `l3_hook_control_v2.js`, `l3_read_crc.js`,
`elf_l3.py`, `dis_l3.py`
- `tools/_work/bench/l1l3/b2_measure.py`, `ttd.py`, `ttd_attach.py`
- `tools/_work/bench/l1l3/evidence/**` (PNG captures: `00_original`, `01_control`,
`02_patched_nosecret`, `09_success_branch`, `12_control_final`, `13_patched_final`)
A note on making §2.4 reproducible: `l1_probe.py` reuses the tag as the remote filename, so
`--tag` must be unique per run. Reusing a tag after a failed `adb push` installs the *previous*
remote file, which is exactly how the first attempt at the final patched verification showed
`Root detected!` — the control build's hash, from the control run's leftover remote path. The hash
comparison in the same script is what caught it; a screenshot alone would not have.
docs/tool-verification/EXTENSION-extraction-shell-bench.md
+538
View File
@@ -0,0 +1,538 @@
# Extension verification — extraction shells and the Dex-VMP boundary (bench rows B3 / B4)
Covers benchmark rows **B3** (extraction-shell skeleton detection) and **B4** (Dex-VMP
boundary: refuse-to-repack and downgrade the deliverable). Strength labels are this
directory's own three tiers: **observed** (an exact command and its output are quoted
here), **inferred** (follows from observed facts; the step was not executed),
**unverified** (assumed or externally reported, not reproduced here).
Everything below was produced on this repository's Windows 11 host with the rootless
PC toolchain in `tools/_work/ENV.md`; the device results in §4 name a
`<DEVICE>` and use `<PKG>` placeholders for the target package.
## 0. Headline
| Question | Answer | Strength |
|---|---|---|
| Does `stub%` separate a skeleton copy from a genuine dex? | **Only when the shell writes bare `return-void`.** Three other measured emptying shapes score 0.0 %–2.3 %, at or below the *untouched* control (1.9 %) | observed |
| Was the pre-existing `trivial_ratio` detector correct? | **No — two defects**, one of which inverted the ranking and named a fully-nop-wiped skeleton the "most likely original" | observed |
| Is `repos/CyReverse`'s shipped "extraction shell" asset usable as a shell fixture? | **No.** The pair differs by six bytes and one body; neither is a shell | observed |
| Is `DexPatcher/assets/ezAndroid.apk` a Dex VMP? | **No.** It is a JNI-sinking target: 7 `native` declarations, an exported `Java_*` for each, 34,566 clean dalvik instruction decodes, zero structural errors | observed |
| Can a static dex probe decide VMP vs not? | **It can decide "not dex-VMP"** when the bodies decode cleanly and each native declaration has an export. It **cannot** decide whether the native island performs runtime hardening | observed (negative half) / unverified (positive half) |
| Does `dex_mem_scan.py` find and cut a dex out of a blob? | **Yes** — exact offset, no false hits in 2 MiB of noise, image accepted by the judge | observed |
| Does it reach a dex in a live process's anonymous mapping? | **Not measured here.** The fixture is built and the commands staged; the device was held by another row throughout | unverified |
One claim in this pass was **retracted**: §3.4 records a Dex-VMP verdict drawn from a
desynchronised decoder and the measurement that killed it.
---
## 1. B3 — the shipped CyReverse "extraction shell" assets are a 6-byte difference, not a shell
Source: `repos/CyReverse/app/src/main/assets/{classes3.dex, classes3_extracted.dex}`
(12,536 B each).
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3
== dex dump validation: 2 file(s), 2 parse, 0 rejected ==
name size sha256[:12] ver cksum sig class noco code stub%
cyrev-extracted.dex 12536 c2d7345e955a 037 ok BAD 4 0 20 10.0%
cyrev-skeleton.dex 12536 ba9d5b120a3a 037 ok ok 4 0 20 10.0%
```
A byte diff of the pair (`tools/_work/bench/unpack/b3/analyze_pair.py`) locates every
difference:
```
$ python analyze_pair.py
len A=12536 len B=12536
first diff offset = 8 (0x8)
diff byte count = 6
diff ranges:
0x8 .. 0x8 (1 B) <- dex checksum (header offset 8)
0xa .. 0xb (2 B) <- dex signature, first half (offset 12 is 4 B wide)
0x121c .. 0x121c (1 B) <- INSNS of PluginClass->getString, units 0 and 2
0x121e .. 0x121e (1 B)
0x1220 .. 0x1220 (1 B)
```
Per-method census of the pair shows the "extracted" copy is the *degraded* one:
```
== cyrev-skeleton.dex == Lcom/cyrus/example/plugin/PluginClass;->getString insns=3 units=001a 0087 0011
== cyrev-extracted.dex == Lcom/cyrus/example/plugin/PluginClass;->getString insns=3 units=0000 0000 0000
```
`const-string v0, "…"; return-object v0` was overwritten with three `0x0000` units in
exactly one of the twenty bodies, and the header was re-fixed. The shipped
`classes3_extracted.dex` is **not** an extraction shell and **not** a recovered dex:
it is a single-body demonstration of body clearing, with 19 of 20 bodies identical to
the skeleton. Neither file can test extraction-shell routing — 10.0 % `stub%` on both
sides is a coincidence of the same body set, not a signal.
**Consequence for the benchmark row:** B3 cannot be run against this asset pair. The
skeleton measurement below had to be built from a real application dex instead.
## 2. B3 — measured `stub%` across every emptying shape, with a zero-change control
Fixture builder: `tools/_work/bench/unpack/b3/make_skeleton.py` derives each variant
from a real 982,480-byte application dex (`src-r2pay.dex`, extracted from
`tools/_work/bench/apks/r2pay-v1.0.apk`, sha256 `c89ff2364879…`). Every variant is
**byte-length preserving**: only the `insns` array of a `code_item` is rewritten and
the tail is nop-filled, so offsets, class table and string table are untouched and the
header is re-fixed with `dexutil.fix_dex_header` (signature first, checksum last).
```
$ python make_skeleton.py src-r2pay.dex skel
source: src-r2pay.dex
ctrl.dex fraction=0.00 mode=return emptied=0/5061
skel-25.dex fraction=0.25 mode=return emptied=1265/5061
skel-50.dex fraction=0.50 mode=return emptied=2531/5061
skel-75.dex fraction=0.75 mode=return emptied=3796/5061
skel-all.dex fraction=1.00 mode=return emptied=5061/5061
pruned.dex fraction=1.00 mode=pruned emptied=5061/5061
nopfilled.dex fraction=1.00 mode=nop emptied=5061/5061
allstub.dex fraction=1.00 mode=allstub emptied=5061/5061
throwstub.dex fraction=1.00 mode=throw emptied=4134/5061
```
Verdicts from `dex_dump_validate.py` over the whole set — the decisive matrix:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skelall
name size sha256[:12] ver cksum sig class noco code stub% erased% emptied%
allstub.dex 982480 c8d7ca8fb163 035 ok ok 647 250 5061 100.0% 0.0% 100.0%
ctrl.dex 982480 1b3a97896b40 035 ok ok 647 250 5061 1.9% 0.0% 1.9%
nopfilled.dex 982480 de92dcea4eaf 035 ok ok 647 250 5061 0.0% 100.0% 100.0%
pruned.dex 982480 58d7fa3dce48 035 ok ok 647 250 5061 93.2% 0.0% 93.2%
skel-25.dex 982480 63346d9b41f5 035 ok ok 647 250 5061 1.7% 0.0% 1.7%
skel-50.dex 982480 d84601b7aa47 035 ok ok 647 250 5061 1.7% 0.0% 1.7%
skel-75.dex 982480 64114285656d 035 ok ok 647 250 5061 2.1% 0.0% 2.1%
skel-all.dex 982480 6345daef0a3f 035 ok ok 647 250 5061 2.3% 0.0% 2.3%
throwstub.dex 982480 cab8fd51a743 035 ok ok 647 250 5061 1.9% 0.0% 1.9%
```
Read the `mode=` column against `stub%`:
| Emptying shape (what the shell leaves behind) | `stub%` | Separated from `ctrl` (1.9 %)? |
|---|---|---|
| bare `return-void` in every body (`allstub`) | **100.0 %** | yes, by 98 points |
| body truncated to `const/4 v0,#0; return v0` + nop fill (`pruned`) | 93.2 % | yes |
| nop fill, no return at all (`nopfilled`) | **0.0 %** | **no — scores *below* the untouched control** |
| `new RuntimeException; throw` stub (`throwstub`) | 1.9 % | **no — indistinguishable from control** |
| 25 % / 50 % / 75 % of bodies emptied with a legal minimal body | 1.7 / 1.7 / 2.1 % | **no** |
Three conclusions, all observed:
1. **The threshold is not a number, it is a shape.** `stub%` is bimodal — it is either
≈ control (blind) or ≈ 100 % (sighted). There is no intermediate band where a
threshold could sit, so the "tens of percent" calibration the reference file
describes has no measured support.
2. **Partial extraction is invisible.** `skel-75.dex` — three quarters of all bodies
emptied — reports 2.1 % against a 1.9 % control, a 0.2-point difference inside the
sampling noise of what a real app's own trivial bodies contribute.
3. **The two shapes that defeat it are the two that real shells prefer.**
`throw new RuntimeException` stubs are the classic 360/legu family shape, and a
nop-wiped body is what a shell writes when it clears the slot before decrypting.
The false-negative on `nopfilled` was a **defect, not a boundary**, and is fixed in §3.
The `throwstub` blind spot is a **boundary** and is documented as one.
## 3. B3 — defects found in `dex_dump_validate.py`, and the fix
### 3.1 Defect 1: a fully nop-filled body was classified `real`
`classify_body` skipped every `0x0000` unit and then fell through to `return "real"`
when nothing else was found. Measured consequence: `nopfilled.dex` — **all 5,061
bodies wiped** — reported `stub% = 0.0`, and the ranking named it the most likely
original:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/skel/{nopfilled,ctrl}.dex
ranking (most likely original first):
1. nopfilled.dex stub%=0.0 cksum=ok sig=ok classes=647
…
verdict: nopfilled.dex is the most likely original (1 copy(ies) in this set)
```
An agent routing on that line patches a wiped skeleton. That is the failure the file
exists to prevent, so the defect is on the wrong side of "the tool still produces
output".
**Fix.** `classify_body` gained two classes and a wider body scan:
- `erased` — `insns_size > 0` with every unit `0x0000`. A real compiler never emits a
whole body of nops, so this is skeleton evidence.
- `minimal` — a body truncated to `const/4 vR,#0; return vR` (legal, and the shape the
detector is blind to by design). Reported separately so the blind spot is visible
rather than hidden inside `real`.
- The early-exit `meaningful > 1 → real` became `meaningful > 2`, so the two-unit
minimal body reaches its own verdict instead of being mislabelled.
`trivial_ratio` keeps its original definition so previously recorded numbers stay
comparable; `emptied_ratio = (stub + erased) / with_code` is the new skeleton signal,
and `rank_key` sorts on it **low-first** with `bodies_erased` as a tiebreak.
### 3.2 Defect 2 (retracted in-flight): `0xNN0E` is not malformed
An intermediate version of the fix classified a lone `0x020E` unit as `malformed`,
reading a non-zero high byte on a `0x0E` opcode as an illegal `return-void`. The
control dump proved otherwise — 82 of the untouched dex's own bodies are exactly that
shape — and `const/4 v0, #0; return v0` is a legal minimal non-void body. The class was
renamed `minimal`, its comment corrected, and the fixture extended with `pruned.dex`
to measure it deliberately (93.2 %, `stub%` sighted). Recorded here because the
mislabel would have taught the next reader that ordinary short methods are damage.
### 3.3 Post-fix behaviour, and no regression on the recorded fixture
`nopfilled.dex` is now ranked last and flagged:
```
8. nopfilled.dex stub%=0.0 erased%=100.0 cksum=ok sig=ok classes=647 <-- SKELETON: emptied%=100, extraction-shell shape
9. …
verdict: skel-25.dex is the most likely original (1 copy(ies) in this set)
```
The pre-existing acceptance fixture from `EXTENSION-unpacking.md` §2 still produces
its recorded verdict, with only the two new columns appended:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'
== dex dump validation: 7 file(s), 4 parse, 3 rejected ==
01_original_shell.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4% 0.0% 3.4%
02_duplicate.dex 8923752 bcd1c0488e86 035 ok BAD 4 160 29 3.4% 0.0% 3.4%
03_broken_magic.dex 8923752 33eee53b1317 ? REJECTED: magic=b'XXXX'
04_tampered_body.dex 8923752 b89bc8cc73d9 035 BAD BAD 4 160 29 3.4% 0.0% 3.4%
05_truncated.dex 65536 e6687390a663 035 REJECTED: file_size=8923752 actual=65536
06_extracted_skeleton.dex 8923752 85e56fed1d56 035 ok ok 4 160 29 100.0% 0.0% 100.0%
07_tiny_garbage.dex 16 be45cb2605bf ? REJECTED: too small (16 B) to carry a dex header
verdict: 01_original_shell.dex is the most likely original (2 copy(ies) in this set)
```
Same winner, same three rejections, same dedupe group. The change is additive on that
fixture and corrective on the new one.
### 3.3b The ranking was still pointing at a modified image (found by independent review)
The first fix above added the `emptied_ratio` signal — and then **kept ranking on a
metric this same pass had just proved insensitive to skeletons**. Reproduced by
leading the review on the variant set:
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skel
ranking (most likely original first):
1. skel-25.dex stub%=1.7 erased%=0.0 cksum=ok sig=ok classes=647
2. skel-50.dex stub%=1.7 erased%=0.0 cksum=ok sig=ok classes=647
3. ctrl.dex stub%=1.9 erased%=0.0 cksum=ok sig=ok classes=647
…
verdict: skel-25.dex is the most likely original (1 copy(ies) in this set)
```
`skel-25.dex` — a dex with **25 % of its bodies removed** — was named the most likely
original, and the genuine untouched control ranked third. The cause is the bimodality
of §2: inside the low band the ratio is not a signal, and 1.7 % versus 1.9 % is noise
(the extracted bodies are replaced by `const/4 + return`, which *lowers* the
`return-void` count relative to the control). An agent routing on that line adopts a
modified image as its baseline, and every later diff is then built on the wrong
artifact — the same class of failure as the defect in §3.1, reached by a different
path.
**Fix.** `rank_key` became three-level, with an explicit eviction level so that either
signal can disqualify an image instead of the two merely tying:
1. `emptied_ratio >= 0.5` sorts last. Nothing mostly emptied is a candidate however
else it scores. This level exists because the count below cannot see the
`return-void`-in-every-body skeleton: that image has `minimal = erased = 0`, so a
pure-count key ranked it **first** — measured, and it is why this level was added
rather than the count being promoted alone.
2. `bodies_minimal + bodies_erased` low-first. This orders the low band, where the
count is monotone: 82 untouched, then 1299 / 2524 / 3736 / 4946 at 25 / 50 / 75 /
100 %.
3. `emptied_ratio` low-first, then header verdicts, then name.
Result on the same command:
```
ranking (most likely original first):
1. ctrl.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
2. only.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
3. throwstub.dex stub%=1.9 erased%=0.0 min/erased=82 cksum=ok sig=ok
4. skel-25.dex stub%=1.7 erased%=0.0 min/erased=1299 cksum=ok sig=ok
5. skel-50.dex stub%=1.7 erased%=0.0 min/erased=2524 cksum=ok sig=ok
6. skel-75.dex stub%=2.1 erased%=0.0 min/erased=3736 cksum=ok sig=ok
7. skel-all.dex stub%=2.3 erased%=0.0 min/erased=4946 cksum=ok sig=ok
8. allstub.dex stub%=100.0 erased%=0.0 min/erased=0 cksum=ok sig=ok <-- SKELETON
9. pruned.dex stub%=93.2 erased%=0.0 min/erased=344 cksum=ok sig=ok <-- SKELETON
10. nopfilled.dex stub%=0.0 erased%=100.0 min/erased=5061 cksum=ok sig=ok <-- SKELETON
verdict: ctrl.dex is the best-supported candidate for the original (1 copy(ies) in this set)
```
The control wins and every skeleton sorts behind it. The verdict line now states the
key and its limits instead of declaring a winner outright, and phrases the result as
"best-supported candidate" with an instruction to confirm before patching. `--trim`,
`--json` and `--find` are unchanged (verified: the recorded fixture still returns
`01_original_shell.dex` first, `--json` only gained keys). Note the residual limit the
report prints rather than hides: **`throwstub.dex` still ranks third**, because no body
statistic here sees a `throw`-stub skeleton. That is a boundary, not a bug — it is
listed in §7 with the reproduction.
### 3.4 A retracted Dex-VMP verdict — the measurement that killed it
An intermediate workbench script of mine
(`tools/_work/bench/unpack/b4/vmp_probe.py`, `vmp_methods.py`) decoded instruction
widths from a hand-written table and reported:
```
code_items=22424 desync=1393 (6.21%) bad_op=0xf8 / 0xef / 0xea / …
```
I read that as "bodies present but not decodable → private opcodes → Dex VMP". **The
reading was wrong, and the error was my table's.** The official decoder settles it:
```
$ E:\tools\android-14\dexdump.exe -d tools/_work/bench/unpack/b4/orig/classes.dex
Processing 'orig\classes.dex'...
Opened 'orig\classes.dex', DEX version '035'
… 943,223 lines …
instruction lines (hex:...) : 348566
code items resolved ('insns size :') : 22424
code '(none)' : 1179
```
One of the methods my script called a desync decodes cleanly under `dexdump`:
```
--- androidx.appcompat.app.AlertController$ButtonHandler.<init>:(Landroid/content/DialogInterface;)V
insns size : 11 16-bit code units
1fd564: 7010 9b05 0100 |0000: invoke-direct {v1}, Landroid/os/Handler;.<init>:()V
1fd56a: 2200 040e |0003: new-instance v0, Ljava/lang/ref/WeakReference;
1fd56e: 7020 fb76 2000 |0005: invoke-direct {v0, v2}, Ljava/lang/ref/WeakReference;.<init>:(Ljava/lang/Object;)V
1fd574: 5b10 ed08 |0008: iput-object v0, v1, …$ButtonHandler;.mDialog:…;
1fd578: 0e00 |000a: return-void
```
My script consumed 9 of 11 units and stopped on `0xed`; `0xed` is a legal
`invoke-*/range` opcode. **The lesson is the one the reference file already states and
this pass then failed to apply: a hand-rolled opcode table is not a VMP detector, and
"my decoder desynchronised" is a statement about my decoder.** Any future
"rare-opcode density" metric must be validated against `dexdump` on the same file
before it is allowed to inform a verdict.
The corrected count of bodies that fail to decode as dalvik in this dex is **0**.
Correction confirmed by an independent path: `dexdump` emits 34,566 instruction lines
and reports no structural error for any of the 22,424 code items.
## 4. B4 — the Dex-VMP judgement chain, re-opened
### 4.1 Static findings (`observed`)
Target: `repos/DexPatcher/assets/ezAndroid.apk`, 2,984,841 B, sha256
`76e3fa8119cf8b7c…`, package `<PKG>` (the `ezAndroid` CTF sample shipped in DexPatcher).
```
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/b4/orig/classes.dex
classes=2905 methods_with_code=22424 stub%=1.9 cksum=ok sig=BAD
```
- **Native declarations: 7**, all in the app's own obfuscated classes, each with no
code (`code : (none)`), enumerated from `dexdump`:
`Lcom/example/ezandroid/C;->cf(Ljava/lang/String;I)Z`,
`Lcom/example/ezandroid/MU;->GMS(Ljava/lang/String;)Ljava/lang/String;`,
`Lcom/example/ezandroid/S;->Stringfjni()Ljava/lang/String;`,
`Lcom/example/ezandroid/S;->onCreate`, `Lcom/example/ezandroid/S2;->onCreate`,
`Lcom/example/ezandroid/S$1;->onClick`, `Lcom/example/ezandroid/S2$1;->onClick`.
- **Two JNI exports** in `lib/arm64-v8a/libezandroid.so` (206,904 B, aarch64):
`Java_com_example_ezandroid_S_Stringfjni` @ `0xea6c` (184 B) and
`Java_com_example_ezandroid_MainActivity_stringFromJNI` @ `0xe91c` (184 B).
Both are thin JNI wrappers that construct a `std::string` and dispatch through a
virtual call — the body of the check is behind the vtable, not in the wrapper.
- **No `JNI_OnLoad` and no `RegisterNatives` symbol** anywhere in the library; the
string `JNI_OnLoad` does not occur in the file at all.
- **`assets/` holds six small opaque files** (`0OooOO` 118 B, `O0ooOO` 126 B,
`oo0Ooo` 86 B, `ooO0oo` 132 B, `OOoo0O` 398 B, `OOooO0` 260 B). Each begins with a
three-unit header (`05 00 38 00 02 00 …`) followed by 2-byte-aligned units.
- **No dex string references those six filenames.** Grep over the full 943,223-line
`dexdump` output for `0OooOO|O0ooOO|oo0Ooo|ooO0oo|OOoo0O|OOooO0|assets/` returns
**0 hits**.
### 4.2 Verdict: JNI sinking, not Dex VMP (`observed`)
The reference file's Dex-VMP test is "the dump's method bodies are intact but decode
as private opcodes — rare-opcode density off the charts, unknown opcode slots". On this
target **zero method bodies fail to decode**, and every `native` declaration has a
matching exported `Java_*`. The dex is not carrying an encoded program; the code left
the dex for the `.so`, which is `java2c-and-jni-sinking.md`'s territory, not this
file's. The two must not be conflated: an agent that reads "bodies decode as nonsense"
here will go hunting a private disassembler for code that is ordinary ARM64 in a
206 KB library.
The six `assets/` files are a separate, **unresolved** observation: they are not
referenced by any dex string, so they cannot be load-bearing at the dex layer. Whether
the native island reads them by constructing the path at runtime, and what they
contain, is **unverified** here — it needs a dynamic run (§4.3), not more static
searching.
### 4.3 What static analysis structurally cannot see (the B4 boundary)
`java2c`'s independent pass reached the same classification on this sample and recorded
that the hardening takes effect **at runtime only**. That is the general shape, and it
is the half of B4 that no dex probe can settle:
| Question | Answerable statically? | Why |
|---|---|---|
| Are the bodies real dalvik or private opcodes? | yes | `dexdump` decodes them or does not; measured 0 failures here |
| Did the code leave the dex for native? | yes | `native` access flags + `Java_*` exports / `JNI_OnLoad` |
| Does the native island check the signature / root / debugger? | **no** | the logic is ARM64 behind a vtable; the check may not exist in any dex |
| Do those `assets/` blobs get decrypted and used? | **no** | the path may be assembled at runtime; grep for the literal names finds nothing |
| Does a repack actually fail, and why? | **no** | needs the repack installed and run against a control (G4) |
**Rule this pass proposes for the reference file:** a static dex probe may return
"this is *not* dex-VMP" when the bodies decode cleanly and every native declaration has
an export. It may **not** return "this is not hardened" from the same evidence —
those are different claims, and the second one is the misdiagnosis
`java2c-and-jni-sinking.md` was written to stop.
### 4.4 Repack control — `observed` up to the install, `unverified` beyond
G4 requires a zero-change control through the same pipeline. Built with the
repository's own repacker against an explicit toolchain (this host has no uber-apk-signer;
`keytool` lives in the JDK that `ENV.md` does not list):
```
$ python skills/apk-reverse/scripts/repack.py \
--apk tools/_work/bench/unpack/b4/ezAndroid.apk \
--out tools/_work/bench/unpack/b4/ez-ctrl.apk \
--ks tools/_work/bench/unpack/b4/ctrl.jks --signer apksigner \
--apksigner E:\tools\android-14\apksigner.bat \
--zipalign E:\tools\android-14\zipalign.exe \
--keytool "C:\Program Files\Java\jdk-17.0.4.1\bin\keytool.exe"
[in ] E:\apk-reverse\tools\_work\bench\unpack\b4\ezAndroid.apk (2984841 bytes)
[zip] - META-INF/1N0R.SF / 1N0R.RSA / MANIFEST.MF (signature artifact)
[zip] unsigned written: 2828919 bytes
[zip] AndroidManifest.xml STORED offset=0x31
[zip] classes.dex 3960320 bytes method=8 crc=037e9cb3
[zip] resources.arsc STORED offset=0x202f14 aligned
[sign] route: apksigner
[verify:zipalign] rc=0
[verify:v2block] present
[cert] SHA1 C9:89:C5:75:A4:83:AE:E0:A6:4B:F4:F6:0D:E3:D2:73:C2:8B:65:69
[result] OK
```
Zero-change repack, v2 block present, alignment verified, `classes.dex` carried through
at its original CRC (`037e9cb3`). **Install and launch on `<DEVICE>` are `unverified`
in this record**: the shared device was held by a parallel benchmark row (`B1 L1 repack
control + patched install`, `B2 L3 frida`) for the whole window available to this pass.
No claim is made here about whether the control launches; the artifact and the command
above are the reproducible half, and the device half is the open item.
Consequently **"repacking is truly refused on this target" is `inferred`, not
`observed`** — it follows from the native island being the only home of the app's own
logic (`Stringfjni`, `cf`, `GMS` are all native declarations), which makes a
dex-level patch unable to reach the behaviour, but no device run confirms a refusal.
The honest deliverable form for this target is the G1 row **annotated analysis report
with a stated boundary**, with `lsposed_scaffold.py` / `frida_rpc_serve.py` named as
the routes *if* a device run shows the build is refused.
## 5. `dex_mem_scan.py` — the cut half is measured, the in-process half is not
The script's job splits cleanly into *find-and-cut* (locate `dex\n03x` in a blob and
write exactly the declared `file_size`) and *reach the blob at all* (do it against a
live process's anonymous mappings through `/proc/<pid>/mem`). Only the first half was
measurable in this pass, and it was measured with a synthetic high-entropy capture
built to have the shape a heap buffer presents:
```
$ python … # 2 MiB of random bytes with a 4,124-byte dex written at offset 0x9a000
fixture dex: 4124 bytes, magic=b'dex\n039\x00'
capture written: …/memscan/anonregion.bin 2097152 bytes dex at 0x9a000
$ python skills/apk-reverse/scripts/dex_mem_scan.py …/anonregion.bin --dump …/scanout
== dex memory scan: 1 capture(s), 1 hit(s) ==
capture offset ver file_size classes state
anonregion.bin 0x9a000 039 4124 1 ok
wrote …/scanout/anonregion+0x0009a000.dex
$ python skills/apk-reverse/scripts/dex_dump_validate.py …/scanout
name size sha256[:12] ver cksum sig class noco code stub%
anonregion+0x0009a000.dex 4124 d3dedcd2b75e 039 ok ok 1 0 5 0.0%
```
Exact offset, zero false hits in 2 MiB of noise, the extracted image byte-identical
by construction and accepted by the judge including its checksum and signature. That
is the **whole find-and-cut path, observed** — the halves compose correctly.
**What is not measured.** The live-process half: the fixture that would exercise it is
built and staged (§6), but the shared device was held by a parallel benchmark row for
the entire window this pass had, so `dex_mem_scan.py` has still never been run against
a real anonymous mapping here. The one related number that exists is from the earlier
root-dump pass, and it is a **zero**: 142 anonymous regions scanned, no hits, against
17 dex images found in *named* ART mappings with no search at all. A zero-hit scan is
weak evidence for the reasons listed in the script's own header, and the honest
summary is "the finder works on a shape that looks right; its in-process reach is
unverified".
To close it (all commands staged, none run in this pass):
```
adb push tools/_work/bench/unpack/memscan/out2/memfixture.jar /data/local/tmp/
adb push tools/_work/bench/unpack/b3/src-r2pay.dex /data/local/tmp/payload.dex
adb shell 'su -c "CLASSPATH=/data/local/tmp/memfixture.jar app_process /system/bin \
com.revprobe.memfixture.Main /data/local/tmp/payload.dex"' &
# the fixture prints its own pid, truncates the on-disk payload so the heap copy is the
# only remaining source, and writes /data/local/tmp/memfixture.maps
adb shell 'su -c "grep -c . /proc/<pid>/maps"' # confirm cheap maps first
adb shell 'su -c "grep -v \"/\" /proc/<pid>/maps"' # the anonymous set to export
```
Note the fixture deliberately truncates its source file: a scan that finds the dex
after that point cannot have found a file copy, which is what makes the result
attributable to memory rather than to disk.
## 6. Not done — and why
- **No device run of the B4 control build.** The shared device was locked by another
benchmark row for the whole window; `devlock.py acquire unpack` timed out twice
(900 s and 1,800 s waits). Install, launch, time-to-death and refusal/success are
therefore `unverified`.
- **No in-process run of `dex_mem_scan.py`.** The fixture is built and staged
(`tools/_work/bench/unpack/memscan/`, §5), the arm64 `app_process` route is chosen
because this host has **no `android.jar`** and therefore cannot compile a real APK
at all, and the closing commands are listed — but the device was never available, so
the anonymous-mapping hit rate remains unmeasured. The only related measurement is
the earlier pass's **0 of 142** anonymous regions.
- **Whether the `assets/` blobs are live.** Needs a dynamic run (read the files out of
the process after the native island runs, or hook the path construction).
- **No end-to-end FART splice.** The reference file's FART loop needs a live
extraction-shell target; nothing in `repos/` is one (§1), and building an actively
resisting shell is out of scope for this pass.
- **A real APK build on this host.** `E:\tools\android-14` ships `d8`, `aapt2`,
`apksigner` and `zipalign` but **no `android.jar`**, and no `android.jar` exists
anywhere on the machine (searched `E:\tools`, `C:\Program Files`, `E:\Android`). Any
`javac` against `android.*` fails with "package android.os does not exist". APKs can
still be *repacked* here (`repack.py` does not recompile), which is why §4.4 works
and the self-built-APK route in the task brief does not.
## 7. Reproduction
```
python tools/_work/bench/unpack/b3/make_skeleton.py tools/_work/bench/unpack/b3/src-r2pay.dex tools/_work/bench/unpack/b3/skel
python tools/_work/bench/unpack/b3/analyze_pair.py
python tools/_work/bench/unpack/b3/crosscheck.py tools/_work/bench/unpack/b3/skel/*.dex
python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/bench/unpack/b3/skelall
python skills/apk-reverse/scripts/dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'
python tools/_work/bench/unpack/b4/vmp_boundary.py tools/_work/bench/unpack/b4/orig/classes.dex
E:\tools\android-14\dexdump.exe -d tools/_work/bench/unpack/b4/orig/classes.dex
python skills/apk-reverse/scripts/dex_mem_scan.py tools/_work/bench/unpack/memscan/anonregion.bin --dump tools/_work/bench/unpack/memscan/scanout
```
Raw captures live beside the fixtures: `evidence/prefix-validate.txt` (before the fix,
including the inverted ranking) and `evidence/postfix-validate.txt` (after). The
`memscan/` tree carries the staged (unrun) in-process fixture: `plain/com/revprobe/memfixture/Main.java`
(no-`android.jar` form), `build.ps1` (the APK build that cannot complete here, kept as
the record of *why*), and `out2/{classes.dex,memfixture.jar}`.
To re-verify the fix from scratch, the two commands that matter are
`dex_dump_validate.py …/skelall` (expect `nopfilled.dex` flagged, not ranked first) and
`dex_dump_validate.py tools/_work/t2-unpack --find 'Lcom/stub/'` (expect
`01_original_shell.dex` still the winner).
docs/tool-verification/EXTENSION-java2c.md
+316
View File
@@ -0,0 +1,316 @@
# Extension verification — Java2C identification and routing
Covers the 2026-09 extension pass that added `references/java2c-and-jni-sinking.md` and
`scripts/java2c_probe.py`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, JDK 17,
build-tools (`d8`/`aapt2`) from a local Android 14 SDK, `gcc` from MinGW only.
**No Android NDK, no host `clang`, no `wsl` (HCS unavailable), and no device-side
`clang`/Termux.** A rooted Android 11 / API 30 / arm64-v8a `<DEVICE>` was attached and
reachable but **was not used at all** in this pass: every result below is static. No
package name, project name or device serial appears in this record.
Samples are identified by hash rather than by name. They are public MASTG training apps
(a no-native baseline, and two carrying a JNI sink), one VMP-labelled app taken from
`tools/_work/bench/repos/`, one real third-party aarch64 ELF already on the host, and one
locally synthesised Java2C-shaped dex.
---
## 1. The compile chain could not be closed — observed
The intended route was to build a real Java2C `.so` with `amimo/dcc` and probe it. `dcc`
is present at `tools/_work/bench/repos/dcc/` and its Python dependencies are satisfied.
The build stops at the C compiler, and every alternative was tried exactly once before
being written off (two-strike rule — no third variant was attempted):
```
$ which gcc clang
gcc.exe C:\MinGW\bin\gcc.exe # MinGW targets PE/COFF, not ELF
gcc/clang for Android: none
$ wsl -l -q ; wsl -e bash -lc "which gcc"
Ubuntu-E
(empty) -> exit 1 with: Wsl/Service/CreateInstance/CreateVm/HCS/HCS_E_SERVICE_NOT_AVAILABLE
$ adb shell "which clang clang++ gcc ; ls /data/data/com.termux"
(no output for which)
ls: /data/data/com.termux: No such file or directory
```
Related **observed** fact: the `dcc` checkout ships **no prebuilt binary at all** — a
recursive search for `*.so`, `*.apk`, `*.a` over the whole repo returns zero files. There
is no ready-made Java2C library to fall back on.
Consequence: **no ELF file anywhere in this pass was produced by a Dex-to-C compiler.**
Every Java2C-specific criterion in the reference document is therefore **inferred**.
## 2. Degenerate route A — `dcc` generates the C without an NDK — observed
`dcc.py` takes `--no-build`, and `dcc_main(..., do_compile=False)` writes the generated
sources and the JNI project *before* `build_project(project_dir)` would be called. With
`--no-build` and no `-o`, the apktool repack path is skipped entirely (it is guarded by
`if is_apk(apkfile) and outapk`). So the translation stage runs to completion on a machine
with no NDK.
```
$ cd tools/_work/bench/repos/dcc
$ python dcc.py <SAMPLE_DIR>/baseline.apk --no-build --dynamic-register \
--source-dir <WORK>/dcc-out
# (emits only Python 3.14 SyntaxWarnings from the bundled androguard)
$ find <WORK>/dcc-out -type f
jni/Android.mk
jni/Application.mk
jni/nc/Dex2C.cpp jni/nc/Dex2C.h
jni/nc/DynamicRegister.cpp jni/nc/DynamicRegister.h
jni/nc/ScopedLocalRef.h jni/nc/ScopedPthreadMutexLock.h
jni/nc/well_known_classes.cpp jni/nc/well_known_classes.h
jni/nc/compiled_methods.txt
jni/nc/Java_<PKG>_<Class>_<method>__<mangled-proto>.cpp
```
This is the mapping evidence the pass needed, and it is exact rather than imagined:
`compiled_methods.txt` is a one-line dex signature, and it produced exactly one C file
whose name is the same identifier in JNI long form:
```
L<PKG>/MainActivity;onCreate(Landroid/os/Bundle;)V
-> Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2.cpp
```
The generated function opens with the parameter shape the reference document cites as the
`.so`-side signature of a translated method:
```c
/* L<PKG>/MainActivity;->onCreate(Landroid/os/Bundle;)V */
void Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2(JNIEnv *env, jobject thiz, jobject p2){
jobject v0 = NULL;
...
jclass cls0 = NULL,cls1 = NULL,cls2 = NULL,cls3 = NULL;
```
And the registration half is generated as real text, which is the strongest single piece
of evidence produced in this pass that the "no `Java_*` symbol to search for" path is not
hypothetical:
```c
// DynamicRegister.cpp
const char *dynamic_register_compile_methods(JNIEnv *env) {
jclass clazz;
clazz = env->FindClass("<PKG>/MainActivity");
if (clazz == nullptr)
return "Class not found: <PKG>/MainActivity";
const JNINativeMethod export_method_0[] = {
{"onCreate", "(Landroid/os/Bundle;)V", (void *)Java_<PKG>_MainActivity_onCreate__Landroid_os_Bundle_2}
};
env->RegisterNatives(clazz, export_method_0, 1);
env->DeleteLocalRef(clazz);
return nullptr;
}
```
Two further facts read out of the generated `jni/Application.mk`, both **observed** and
both load-bearing for the JNI-boundary section of the reference document:
```
APP_STL := c++_static
APP_CPPFLAGS += -fvisibility=hidden
APP_ABI := armeabi-v7a arm64-v8a
```
`-fvisibility=hidden` means the shipped library can present as "exports no `Java_*`
symbol" **even when the translation used static naming** — the second, independent reason a
symbol search returns nothing. `c++_static` is the origin of the `libc++` string criterion,
and is also why that criterion is weak: it is an ordinary NDK setting.
## 3. The probe against real JNI-sinking targets — observed
This is the discriminating measurement the reference document is built on. The tool was
run unmodified:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/jni-a.apk
classes.dex: 720084 B, 625 classes, 5081 methods, native 2 (0.04%),
native-dominated classes 0, fully native 0
libfoo.so: 14176 B, aarch64, dynsym 20 entries, Java_* 2, JNI_OnLoad False
libfoo.so: 13948 B, arm, dynsym 26 entries, Java_* 2, ...
libfoo.so: 13788 B, x86, dynsym 16 entries, Java_* 2, ...
libfoo.so: 14440 B, x86_64, dynsym 16 entries, Java_* 2, ...
verdict : jni-sinking
confidence : medium
```
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/jni-b.apk
classes.dex: 1934860 B, 1396 classes, 11182 methods, native 3 (0.03%),
native-dominated classes 0, fully native 0
libfoo.so (arm/arm64: 3 Java_* symbols, x86/x86_64: 3 Java_* symbols)
verdict : jni-sinking
confidence : medium
```
Two independent real targets, and both give the same two results: **native density three
orders of magnitude below the Java2C shape**, and **an exact 1:1 between the dex native
declarations and the exported `Java_*` symbols** (2:2 and 3:3, across four ABIs each).
That 1:1 is the point. It is *not* a Java2C signal on its own — both shapes produce it —
which is why the density measurement and the native layer must be read together. A tool
that reported on either alone would misroute one of these two samples.
Baseline control, same command, no native code at all:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --apk <SAMPLE_DIR>/baseline.apk
classes.dex: 5528 B, 7 classes, 15 methods, native 0 (0.00%), ...
verdict : no-hardening-signal
confidence : medium
```
## 4. The probe against a Java2C-shaped dex — observed
No Java2C library could be built, so the dex half of the shape was synthesised. A Java
source declaring two classes whose methods are almost entirely `native` (plus one mixed
class, standing in for an ordinary JNI boundary) was compiled with `javac --release 8` and
assembled with `d8 --min-api 19`:
```
$ javac --release 8 -d classes src/<PKG>/Calc.java # exit 0
$ d8 --min-api 19 --output out classes/<PKG>/*.class # exit 0
out/classes.dex 1752 B sha256 59e0b1d34f00578e1343ee5f344d0e58829f482b643a7843c8ad280311ce9457
$ python skills/apk-reverse/scripts/java2c_probe.py --dex out/classes.dex
classes.dex: 1752 B, 3 classes, 29 methods, native 24 (82.76%),
native-dominated classes 2, fully native 1
verdict : java2c-suspect-needs-so
confidence : low
```
The low confidence is the correct answer, not a shortcoming: the tool refuses to claim
Java2C from the dex alone, and says so in its route block. That refusal is the guard
against the misdiagnosis this pass exists to prevent.
Combining that dex with a real library that registers dynamically makes the `java2c` branch
reachable, which is what proves the branch is not dead code — the input is composite and is
labelled as such:
```
$ python skills/apk-reverse/scripts/java2c_probe.py --dex out/classes.dex \
--so <WORK>/libandroid.so
verdict : java2c
confidence : medium
evidence: (strong) native layer: dynamic registration in libandroid.so
(JNI_OnLoad exported, zero Java_* symbols)
```
## 5. The JNI boundary — why `RegisterNatives` cannot be found — observed
The reference document claims a symbol search fails silently and that looking for a
`RegisterNatives` symbol is structurally unable to work. Both are measured here.
A real 273,120 B aarch64 system library pulled from one of the samples exports a JNI entry
point and no `Java_*` symbols, and the probe flags it as dynamic registration — the
affirmative check that does work:
```
libandroid.so: 273120 B, aarch64, dynsym 504 entries, Java_* 0,
JNI_OnLoad True, RegisterNatives-import False
(strong) native layer: dynamic registration in libandroid.so
(JNI_OnLoad exported, zero Java_* symbols)
```
Independent confirmation that the absence is real and not a parser gap — the same check run
outside the script:
```
$ python -c "d=open('gadget.so','rb').read(); [print(k, d.count(k.encode())) for k in
['JNI_OnLoad','RegisterNatives','Java_','Dex2C','libc++_shared.so','frida']]"
JNI_OnLoad 0
RegisterNatives 0
Java_ 0
Dex2C 0
libc++_shared.so 0
frida 311
```
A 25.8 MB real third-party aarch64 ELF with 311 occurrences of its own name contains zero
`JNI_OnLoad` and zero `Java_`. The probe reported the same (0 `Java_*`, `JNI_OnLoad False`),
so the two readings agree and the probe is not reporting a parse failure as an absence.
**Why the `RegisterNatives` search cannot fire** `[observed]`: the library above registers
dynamically and contains **zero literal `RegisterNatives` symbols**. The cause is a
documented property of the NDK C++ headers `[inferred]`: `jni.h` declares `RegisterNatives`
as an inline member of `JNIEnv` that dispatches through the function table, so a call site
emits no external symbol. The observed half is the missing symbol; the explanation is
inferred from the mechanism.
## 6. Coverage of the probe's code paths — observed
| Path | Exercised by | Result |
|---|---|---|
| dex header + class_defs + class_data (ULEB128) | 5 real dex images + 1 synthetic | Parsed; header size match reported |
| `direct_methods` / `virtual_methods` decoding | the synthetic fixture (has both) | Correct after the defect in §7 |
| native density, native-dominated classes | all six | 0.03–0.04 % real vs 82.76 % synthetic |
| no-`code_item` and trivial-body ratio | all six | 0 on the fixture, 4–5 % on the real apps |
| ELF header, arch, section table | 10 real ELFs (4 ABI variants of two `libfoo.so`, 2 app libs, 1 gadget) | aarch64/arm/x86/x86_64 all identified |
| `.dynsym` parse, `Java_*` extraction | real `libfoo.so` ×8 | 2 and 3 symbols, matching the dex exactly |
| `JNI_OnLoad` export detection | real system library | True |
| dynamic-registration verdict | same library | Fired |
| string-signature scan | 25.8 MB ELF, all samples | Weak hits only (`dcc` ×6 on the gadget); no false strong hit |
| `--json` | synthetic fixture | Valid JSON, metrics block parses |
| `--dir`, `--apk`, `--dex`, `--so` | all four modes | OK |
| `--help` | direct | OK |
The string scan produced exactly one class of hit across every sample: the weak `dcc`
substring (6 occurrences inside a 25.8 MB binary, i.e. coincidental). **No strong marker was
ever produced by any real sample**, which is why the Java2C verdict rests on structure
rather than on strings.
## 7. Not exercised — and why the Java2C verdict stays inferred
- **A real Java2C `.so`.** Never produced; the compile chain is dead (§1). Everything about
how a *compiled* Java2C library presents — whether the generated functions keep their
names under `-fvisibility=hidden`, whether `Dex2C` runtime strings survive into `.rodata`
— is **inferred** from the generated sources and the build flags, not observed.
- **The runtime claim that no dex bytecode exists in memory.** This is the mechanism the
whole document turns on and it is **inferred from the translation model**. No live Java2C
app was run. It is falsifiable and cheap to test on a live sample; that test is named in
the reference document.
- **VMP identification by static dex metrics.** One VMP-labelled sample (sha256
`76e3fa8119cf8b7c4b4768be557d71863433a8ae4d2ad40901eae6bb829349bc`, a 2,984,841 B APK
from the local bench repos, identified by hash only)
was measured and reported **`jni-sinking`**: its static dex has 23,603 methods of which 7
are native, with a 4 % stub ratio, and the archive contains no additional encrypted
payload. **This is a recorded boundary, not a success**: whatever hardening that sample
carries is applied at runtime, and a static dex probe is structurally unable to see it.
Do not read this row as "the sample is unhardened".
- **Anything on a device.** The `<DEVICE>` was not used; there is no dynamic evidence in
this record.
## 8. Two defects this pass found in its own tool — observed
Both were found by running the tool against the synthetic fixture and checking the numbers
by hand, and both are fixed in the shipped script.
1. **`direct_methods` and `virtual_methods` were decoded as one continuous run.** They are
two independent encoded lists; each one's first `method_idx_diff` is relative to 0.
Walking them as one run mis-attributed every virtual method to the following class. The
fixture exposed it as `Codec: 5 methods` where the source declares 4. Fix: decode them as
two passes. After the fix the per-class counts match the source exactly (Calc 20/21
native, Codec 3/3, Helper 1/2).
2. **The "whole class is native" criterion was unreachable.** `javac` — and, per the
generated output, a translation pass — always leaves `<init>`/`<clinit>` behind, so no
class is ever literally all-native unless a constructor is counted as native. The
fixture showed `fully native 0` while two of its classes were plainly native-shaped. Fix:
constructors are excluded and the test is now ">= 70 % of >= 3 named methods native",
reported alongside the literal count. After the fix: `native-dominated classes 2,
fully native 1`.
Both defects produced *plausible* output — a wrong count and a zero that reads as "nothing
found" — which is the failure shape this whole repository is organised against. Recording
them here because a reader relying on either number would have drawn a wrong conclusion.
docs/tool-verification/EXTENSION-kernel-weapons.md
+236
View File
@@ -0,0 +1,236 @@
# Extension verification — kernel-level weapons (templates, gates, and a closed door)
Covers the extension pass that added `scripts/kernelsu_syscall_mask.py` and the `§4a` section of
`references/kernel-and-environment-hardening.md`. Strength labels as defined in this directory's
README: **observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0; `<DEVICE>` = rooted Android 11 (API 30)
arm64-v8a, Magisk alpha, no APatch/KernelPatch.
**Headline: the generator, the gate table and the userspace module skeleton are measured. No
kernel-side artifact was compiled or loaded anywhere, and every route is closed on the reference
device. That is the deliverable — a template plus a measured statement of the door.**
---
## 1. Kernel gate, measured on the device — observed
```
$ adb shell 'uname -r; cat /proc/version'
4.14.186+
Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902) clang version
11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79),
LLD 11.0.1 (/buildbot/tmp/tmp6_m7QH ...)) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022
```
4.14.186 is below the 5.10 GKI gate for the eBPF route, and the device carries no
KernelPatch-patched image. Read-only query; no device lock was needed and no state was changed.
## 2. Host toolchain probe — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py gates
== kernel-side routes and their gates ==
eBPF tracepoint/kprobe GKI kernel 5.10+ (Android 12+)
why: Android's BPF/tracing support as bpftrace-class tooling and
stackplz expect it is a GKI-era feature; BTF is required for
CO-RE style probes.
check: uname -r -> expect 5.10.x or newer; ls /sys/kernel/btf/vmlinux
KernelPatch / APatch KPM KernelPatch-patched boot image + kpm toolchain
why: KPMs load through kpimg injected into the kernel image's
payload segment; building one needs a bare-metal ARM64
compiler (aarch64-none-elf-gcc), not the NDK.
check: ls /data/adb/ | grep -i apatch ; which aarch64-none-elf-gcc
Out-of-tree LKM Kernel source matching the device's exact version + vermagic
why: A module built against a different kernel revision is refused
at load time on a mismatched vermagic, and the device's kernel
source is frequently not published at all.
check: uname -r ; look for a matching kernel source tree for the device
seccomp-BPF (for contrast) any modern kernel, but it is not this route
why: seccomp can make a syscall FAIL (SECCOMP_RET_ERRNO/TRAP); it
cannot rewrite the CONTENT a successful read returns. It closes
doors, it does not paint them -- which is the whole requirement
for a spoofed /proc read.
check: n/a -- this is a capability ceiling, not a version check
== host probe (what can be checked from here) ==
bare-metal ARM64 compiler: NOT FOUND
the KPM route cannot be built from this host as configured
ndk-build: NOT FOUND
make: NOT FOUND
A kernel source tree, a patched boot image and the device's own kernel headers cannot be
checked from the host; they are device facts.
```
The three `NOT FOUND` lines are the measured reason the KPM route cannot be built from here, and
they are also consistent with `ENV.md`'s record that the machine has no Android NDK. The LKM route
additionally needs kernel source that does not exist for this device.
## 3. `generate` — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py generate \
--out tools/_work/t5-vmp/sysmask --package '<PKG>'
generated module scaffold in tools/_work/t5-vmp/sysmask
module.prop 165 B
customize.sh 166 B
post-fs-data.sh 830 B
service.sh 467 B
uninstall.sh 298 B
config/syscall_mask.json 1079 B
kpm/syscall-mask.c 8096 B
kpm/Makefile 880 B
lkm/syscall-mask.c 2434 B
ebpf/syscall_mask.bpf.c 4338 B
README.md 3752 B
STATUS: the userspace module is installable; every kernel-side file is an UNVERIFIED template.
No kernel code was compiled or loaded.
```
`module.prop`, as generated (the package identity is the `<PKG>` placeholder by design — this file
carries no real package name):
```
id=syscall-mask
name=Syscall Mask
version=0.1.0
versionCode=1
author=apk-reverse
description=config-driven syscall return masking (openat/read/stat on /proc/self/*)
```
The configuration table rendered into the KPM template from `config/syscall_mask.json`, so the
rule set is data rather than hand-edited C:
```
static const struct mask_rule rules[] = {
{ "/proc/self/maps", 0, MASK_DENY, -2, 0 },
{ "/proc/self/maps", 0, MASK_FAKE_SIZE, -2, 0 },
{ "/proc/self/status", "TracerPid", MASK_FILTER_LINE, -2, 0 },
{ "/proc/self/status", "TracerPid", MASK_FILTER_LINE, -2, 0 },
};
```
And the KPM lifecycle surface, which is the part an interface drift would break first:
```
int kfunc_def(strncpy_from_user)(char *dst, const char __user *src, long count);
/* fp_hook_syscalln(__NR_openat, before_openat, 0, 0); */
/* fp_hook_syscalln(__NR_newfstatat, before_newfstatat, 0, 0); */
/* fp_hook_syscalln(__NR_read, before_read, 0, 0); */
KPM_INIT(mask_init);
KPM_EXIT(mask_exit);
```
Those names (`KPM_NAME` / `KPM_INIT` / `KPM_EXIT`, `kfunc_def`, `fp_hook_syscalln`,
`hook_fargs4_t`, `syscall_argn`, `args->skip_origin`, `args->ret`) are taken from a public KPM
development write-up rather than invented — blackr0ck, *APatch KPM 开发：绕过 DirtySepolicy /
Duck Detector 的 SELinux Root 检测*, [bbs.kanxue.com/thread-291665.htm](https://bbs.kanxue.com/thread-291665.htm),
fetched 2026-02. **They are still `unverified` here**: nobody compiled against a `kpmodule.h` in
this pass, and the generated template says so in its own header for exactly that reason.
The registration calls are left **commented out** in the template. A generated file that arms a
kernel hook by default is a file that arms a kernel hook the first time somebody runs `make` in a
hurry.
## 4. `verify` — observed
```
$ python skills/apk-reverse/scripts/kernelsu_syscall_mask.py verify --dir tools/_work/t5-vmp/sysmask
== verifying tools/_work/t5-vmp/sysmask ==
module.prop: author=apk-reverse, description=config-driven syscall return masking
(openat/read/stat on /proc/self/*), id=syscall-mask, name=Syscall Mask,
version=0.1.0, versionCode=1
config: 4 rule(s), actions=['deny', 'fake_size', 'filter_line']
kpm target present: Makefile, syscall-mask.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
lkm target present: syscall-mask.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
ebpf target present: syscall_mask.bpf.c
status: UNVERIFIED TEMPLATE -- not compiled, not loaded
kpm/syscall-mask.c: KPM_* lifecycle macros present, kfunc_def not extern
== 0 problem(s): structure is consistent ==
(this says nothing about whether the kernel side compiles or loads)
```
`verify` is deliberately read-only and deliberately narrow: it checks `module.prop` fields, the rule
JSON, and the one static trap in the KPM template that has a documented consequence (an `extern`
`kfunc_def` becomes `*UND*` and the loader refuses the module). It states on its last line what it
does **not** establish, because "0 problems" on a template is the easiest sentence in this file to
over-read.
### 4a. A false positive this check produced, and its fix — observed
The first run reported:
```
== 1 problem(s) ==
- syscall-mask.c declares kfunc_def with `extern`, which the loader rejects as an unknown symbol
verify rc=1
```
The template does not declare it with `extern` — its comment *explains* why `extern` is forbidden,
and the check's own regex ran from that prose across to the real declaration's semicolon. A linter
that reads comments as code produces reports nobody can act on. The fix strips `/* */` and `//`
comments before any token check (`_strip_c_comments`). Both the false positive and the fix are
recorded here rather than quietly repaired, because the failure mode — a check that fires on its
own documentation — is one a reader will meet again.
## 5. The userspace module's ceiling — observed by construction, and worth stating plainly
`post-fs-data.sh` and `service.sh` in the generated skeleton do not load anything. That is not an
omission: a KernelSU/Magisk/APatch-userspace module runs as root in the normal world and cannot
change what a syscall returns. The generated `README.md` says this in its first paragraph, and
`post-fs-data.sh` carries the reasoning in a comment:
```
# The kernel side (if you built and loaded one) owns the syscall table; this
# script's whole job is to make the configuration visible to it and to leave a
# record that the module actually ran. Do not put a syscall hook here: this is
# userspace, and nothing here can change what openat/read/stat return.
```
The uninstall hook likewise refuses to unload a kernel module, because a silent unload during
package removal is how a device ends up in a boot loop.
## 6. eBPF route — gate table and the capability ceiling
| Route | Gate | Check on the device | Status here |
|---|---|---|---|
| eBPF tracepoint/kprobe | GKI 5.10+ with tracing/BTF | `uname -r`; `ls /sys/kernel/btf/vmlinux` | **closed** — device is 4.14.186 |
| seccomp-BPF | any modern kernel | n/a | not this route: it can only fail a syscall, not rewrite a successful read |
| KernelPatch / APatch KPM | patched boot image + `aarch64-none-elf-gcc` | `ls /data/adb/ \| grep -i apatch`; `which aarch64-none-elf-gcc` | **closed** — neither present; patch-the-boot-image risk declined |
| Out-of-tree LKM | kernel source matching vermagic | `uname -r` + a matching source tree | **closed** — no source for this device |
The probe template (`ebpf/syscall_mask.bpf.c`) is a `sys_enter_openat` tracepoint observer with a
uid filter map and a ringbuf event, and it documents two dead ends in the file itself:
`bpf_override_return()` only applies to `ALLOW_ERROR_INJECTION` functions, which raw syscall
entries are not, and a tracepoint cannot rewrite a result at all. **A tracepoint observes; it does
not spoof.** The honest eBPF shape for this job is therefore observation — which is still worth
having, because `§5` of the reference file requires you to establish whether the target uses a raw
`svc` or a libc call *before* designing any spoof.
## 7. What was not done
- **No kernel-side code was compiled.** Not the KPM, not the LKM, not the eBPF program. No
`kpmodule.h`, no kernel headers, no `bpftool`, no bare-metal ARM64 compiler on this host.
- **Nothing was loaded, flashed or activated on `<DEVICE>`.** No boot-image patch, no module load,
no kernel modification. The task forbids it for good reason: a kernel module that faults takes
the device down before it can report why.
- **The KPM API surface is `unverified`.** The macro and helper names follow a public write-up; the
argument arity of `fp_hook_syscalln` in particular is marked `TODO(verify-on-device)` in the
generated source rather than guessed, because a wrong arity in a hook registration is a
load-time failure at best.
- **The struct-stat size offset is a TODO, not a value.** The `newfstatat` handler deliberately does
not carry a magic offset: it is ABI-specific and copying one from a different architecture is how
a spoof corrupts an unrelated field.
- **`verify` establishes structure, not capability.** Its own output says so.
docs/tool-verification/EXTENSION-protobuf-raw.md
+383
View File
@@ -0,0 +1,383 @@
# Extension verification — schema-free protobuf decoding
Covers the pass that added `skills/apk-reverse/scripts/protobuf_decode_raw.py` and the measured half
of `references/protocol-reverse.md` §1 ("The decoder that ships here"). Strength labels as defined in
this directory's README: **observed** (exact command + output recorded here), **inferred** (follows
from observed facts, step not executed), **unverified** (assumed or externally reported, not
independently confirmed here).
Reference environment for this pass: Windows host, Python 3.14.0, `protobuf` **6.33.6**. No device
was used (see §7) — every result below is host-side, offline, and reproducible with the standard
library alone. Workbench artifacts live in `tools/_work/t5-proto/` (not part of the skill, and
`tools/` is gitignored).
The delivered script imports **no** third-party module. `google.protobuf` is used here for one
purpose only: to *generate* reference bytes from a real schema, so the hand-written decoder is
checked against the reference implementation instead of against itself.
---
## 0. Summary of what this pass established
| Claim | Label |
|---|---|
| `protobuf_decode_raw.py` `--help`, `--selftest`, hex/file/stdin inputs, `--json`, `--max-depth`, `--reencode` all run on this host | observed |
| 26/26 built-in known-answer checks pass (field decode **and** byte-exact round trip) | observed |
| The decoder agrees field-by-field with a message serialized by `google.protobuf` 6.33.6, and its re-encode is byte-identical to `SerializeToString()` | observed |
| A real AndroidX DataStore file written by `scripts/datastore_inject.py` decodes to the correct structure and round-trips byte-identically | observed |
| Editing one value in the JSON tree and re-encoding produced a file that `datastore_inject.py` read back as the edited value | observed |
| proto3 zero semantics: an implicit (no-presence) field set to 0 writes **nothing**; a presence-carrying field set to 0 writes its tag plus a zero byte | observed |
| A packed field and a nested message are **both** legal readings of the same length-delimited bytes (measured on real encoded bytes, not argued) | observed |
| `--split varint-length` is protobuf's `writeDelimitedTo` framing; handed a gRPC frame it silently produced 5 frames instead of failing | observed |
| Decoding a **device-side** protobuf file (real app data read over adb) | **not done** (device lock held by another agent; §7) |
| Any protobuf payload recovered from the benchmark APKs | **not done** — the one APK scanned carries no `.pb`/`.proto` resource (§7) |
| `blackboxprotobuf` / `protobuf-inspector` behaviour | unverified (not installed on this host; the reference still says so) |
---
## 1. Input forms, measured
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --help
usage: protobuf_decode_raw.py [-h] [--hex STR] [--json] [--max-depth N]
[--no-expand] [--split {none,varint-length}]
[--reencode JSON] [--out PATH] [--check PATH]
[--selftest]
[input]
... exit = 0
```
Hex text, a binary file and stdin are all accepted, and the two stdin shapes are told apart without
a flag:
```
$ '0a220a176578616d706c655f6578706972795f65706f63685f6d7312072080b08fe6b277' \
| python skills/apk-reverse/scripts/protobuf_decode_raw.py -
input: stdin (hex text), 36 byte(s)
frame 0 bytes 0..36 (36 B) status=end_of_buffer
f1 len-delimited(34) view=nested_message
...
$ cmd /c "python skills\apk-reverse\scripts\protobuf_decode_raw.py - < tools\_work\t5-proto\ds.preferences_pb"
input: stdin (raw bytes), 81 byte(s)
frame 0 bytes 0..81 (81 B) status=end_of_buffer
```
Separators and escapes, in one command (two frames, `0x` prefixes, spaces, a `\xNN` escape):
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --hex "0x08 0x96 0x01 | \x12\x07testing"
frame 0 bytes 0..3 (3 B) status=end_of_buffer
f1 varint 150
frame 1 bytes 3..12 (9 B) status=end_of_buffer
f2 len-delimited(7) view=utf8_string
...
string: 'testing'
```
`08,96,\n01` (commas and a newline) and the single token `0x089601` both decode to the same field.
**This command is the one that found the script's first real defect — see §6.**
---
## 2. Known-answer fixtures: `--selftest`
Twelve fixtures are built byte by byte inside the script (no external file, no schema), covering a
canonical varint, a multi-byte varint, `uint64` maximum, two-level nesting, a packed field, fixed64
and fixed32, an explicit zero, a deprecated group, a non-canonical varint, a 10-byte overflow, two
top-level messages in one stream, and varint-length framing. Each is decoded with an expected value
**and** passed through the round trip.
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --selftest
== known-answer fixtures (bytes produced by hand in this script) ==
canonical varint 150 089601
field 1 varint value PASS 150
unsigned max uint64 08ffffffffffffffffff01
field 1 varint value PASS 18446744073709551615
nested two levels 089601120774657374696e671a060801120210072206038e029ea705...
field 3 nested child count PASS 2
deprecated group 3/4 43080544
field 8 has children PASS 1
non-canonical varint 08968100
field 1 varint value PASS 150
10-byte varint overflow 08ffffffffffffffffff7f
field 1 varint overflow PASS True
...
== round trip: decode -> re-encode -> byte comparison ==
canonical varint 150 MATCH (1 fields, 1 identical)
nested two levels MATCH (7 fields, 7 identical)
non-canonical varint MATCH (1 fields, 0 identical) re-encode emits the minimal
form 089601, original 08968100, and the report states why
varint length framing MATCH (7 fields, 7 identical)
== the two wire-format traps ==
explicit zero bytes 2800 decodes to value 0; absent field decodes to no fields at all: PASS
packed payload 038e029ea705 offers live=['bytes', 'packed_varint'], rejected=['nested_message',
'utf8_string'] -> view=packed_varint : PASS
selftest: 26/26 PASS exit = 0
```
The non-canonical fixture is the one deliberate exception: the re-encode is *expected* to differ
(`08968100` -> `089601`, the minimal form), and the tool reports the difference with its reason
rather than normalising it away silently. That is the honest form of "round trip": a difference that
is explained is evidence; a difference that is not is a bug.
---
## 3. Cross-check against the official protobuf runtime
`tools/_work/t5-proto/official_ref.py` builds a real `FileDescriptorProto` (`Outer` with int64 /
string / message / repeated int32 (packed) / fixed64 / float / enum / proto3-optional / bytes /
fixed32 fields, plus a nested `Inner`), serializes it with `SerializeToString(deterministic=True)`,
and then asks the hand-written decoder to reproduce it.
```
$ python tools\_work\t5-proto\official_ref.py
== official runtime bytes (protobuf 6.33.6) ==
58 B: 089601120774657374696e671a0508011201782206038e029ea705290807060504030201350000c03f380240004a06038e029ea70555efbeadde
== field-by-field agreement with the schema ==
field 1 count == 150 PASS decoded 150
field 2 name == "testing" PASS decoded 'testing'
field 3 inner walked as a nested message PASS view=nested_message children=2
field 4 nums read as packed [3,270,86942] PASS view=packed_varint values=[3, 270, 86942]
field 5 stamp fixed64 hex 0807060504030201 PASS 0807060504030201
field 6 ratio fixed32 hex 0000c03f (1.5f) PASS 0000c03f
field 7 color enum == 2 PASS
field 8 explicit_zero is present and 0 PASS
field 9 blob hex 038e029ea705 PASS
field 10 crc fixed32 hex efbeadde PASS efbeadde
== round trip against the official bytes ==
re-encode is byte-identical to SerializeToString() PASS 10/10 fields identical
no field was rewritten PASS []
official-runtime cross-check: 21/21 PASS exit = 0
```
Field 5 is the useful false-friend: the decoder offers `as_double_repr` for those 8 bytes, and the
official value is a `fixed64` integer. Nothing on the wire distinguishes them, which is why the
candidate list exists instead of a single typed value.
### 3.1 The four-way tie, measured on real bytes
A really-encoded nested message — `Inner{id:1, tag:"x"}` inside field 3, inner payload `0801120178`
— is reported as:
```
nested message payload 0801120178 -> {'nested_message': 0.5, 'packed_varint': 0.5,
'utf8_string': 0.35, 'bytes': 0.2}
```
`0801120178` is simultaneously a valid two-field message **and** a valid five-element packed varint
array. The decoder refuses to choose; the `tie:` line states that the displayed `view` is a display
default. A real packed field (`2206038e029ea705`) yields `packed_varint 0.55` with
`nested_message` **rejected** — the rejection is printed with its reason instead of being omitted.
### 3.2 proto3 zero — the reference sentence, corrected by measurement
The reference previously stated that `28 00` is byte-identical to an absent field. The runtime shows
that the sentence needs splitting into two halves:
```
implicit int64 count = 0 : 120178 (3 B)
count never touched : 120178 (3 B)
implicit (no presence) zero is byte-identical to absent PASS
that byte string contains no tag for field 1 PASS
optional int32 explicit_zero=0: 1201784000 (5 B)
a presence-carrying field DOES put tag+0x00 on the wire PASS 1201784000
the optional zero costs 2 bytes the implicit zero never costs PASS 5 B vs 3 B
the decoder sees that 0 and flags the ambiguity PASS ["proto3_explicit_zero"]
```
Read it as two separate facts:
1. For a field **without** presence, `value = 0` and "never set" are the same bytes (no bytes at
all), so a field **missing** from a decode is not evidence that the value in use was 0.
2. A `0` that **is** on the wire was emitted on purpose — a presence-carrying field (`40 00` here,
field 8) or a hand-rolled writer. The decoder cannot tell which, so it flags the case.
`references/protocol-reverse.md` §1 consequence 3 now carries that correction.
---
## 4. A real AndroidX DataStore file
`scripts/datastore_inject.py` writes the real AndroidX Preferences protobuf container
(`map<string, Value>`); the decoder has never seen that schema.
```
$ python skills/apk-reverse/scripts/datastore_inject.py --file tools\_work\t5-proto\ds.preferences_pb \
--key example_expiry_epoch_ms --type long --value 4102444800000 \
--extra example_flag=true:bool --extra example_name=hello:string
[ok] wrote tools\_work\t5-proto\ds.preferences_pb (81 bytes)
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\ds.preferences_pb
input: tools\_work\t5-proto\ds.preferences_pb, 81 byte(s)
frame 0 bytes 0..81 (81 B) status=end_of_buffer
f1 len-delimited(34) view=nested_message
...
f1 len-delimited(23) view=utf8_string
string: 'example_expiry_epoch_ms'
f2 len-delimited(7) view=nested_message
...
f4 varint 4102444800000
f1 len-delimited(18) view=nested_message <- the second map entry
...
```
The map-entry structure, the key strings and the `long` value all come out correctly with no schema;
every length-delimited field still shows its ties (`nested_message` vs `packed_varint` vs
`utf8_string`) because those ties are real.
### 4.1 Round trip, and an edit confirmed by the other tool
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\ds.preferences_pb --json > tools\_work\t5-proto\ds.tree.json
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --reencode tools\_work\t5-proto\ds.tree.json \
--out tools\_work\t5-proto\ds.reenc.bin --check tools\_work\t5-proto\ds.preferences_pb
re-encoded 81 byte(s) from 3 field(s)
fields re-encoded byte-identically: 3/3
written to tools\_work\t5-proto\ds.reenc.bin
check against tools\_work\t5-proto\ds.preferences_pb (81 B, the whole file): MATCH exit = 0
```
Then one value is edited in the JSON tree and re-encoded:
```
editing field 4 from 4102444800000
re-encoded 80 byte(s) from 3 field(s)
fields re-encoded byte-identically: 2/3
CHANGED field 1: expected 0a22...80b08fe6b277, got 0a21...20ffc7afa025 (the tree was edited)
$ python skills/apk-reverse/scripts/datastore_inject.py --in tools\_work\t5-proto\ds.edited.bin --list
[in] tools\_work\t5-proto\ds.edited.bin (80 bytes)
example_expiry_epoch_ms long=9999999999 (value=6 bytes)
example_flag bool=True (value=2 bytes)
example_name string='hello' (value=7 bytes)
```
Two independent facts in one run: the file shrank 81 -> 80 B because a 6-byte varint became a
5-byte one **and the enclosing length prefix was recomputed** (the re-encode rebuilds lengths, it
does not copy them), and a different tool of this repository read the edited bytes back as the new
value. A round trip that only agrees with itself would not have caught either.
---
## 5. Framing: `writeDelimitedTo` is not a gRPC frame
`tools/_work/t5-proto/framing_check.py`:
```
message (20 B): 089601120774657374696e672206038e029ea705
writeDelimitedTo form (21 B): 14089601120774657374696e672206038e029ea705
varint-length framing: exactly one frame PASS got 1
a second message makes two frames PASS
gRPC frame (25 B): 0000000014089601120774657374696e672206038e029ea705
decoded with --split varint-length: 5 frame(s): [(1, 1, 'end_of_buffer'), (2, 2, ...),
(3, 3, ...), (4, 4, ...), (5, 25, ...)]
gRPC frames are NOT varint-length framed (the split misfires) PASS got 5 frame(s)
stripping the 5-byte gRPC prefix decodes the message correctly PASS field 1=150, field 4=[3, 270, 86942]
'|' separates two independently framed messages PASS frames=2
framing check: 8/8 PASS exit = 0
```
| Framing | Bytes in front of the message | `--split varint-length` |
|---|---|---|
| protobuf `writeDelimitedTo` | varint length | correct — 1 frame per message |
| gRPC | 1 compression-flag byte + **4-byte big-endian** length | **wrong, and it does not fail**: the 4 zero bytes become four empty frames |
The same gRPC bytes decoded with **no** splitting reported `stray_end_group` (exit 1), and a
`writeDelimitedTo` file decoded with no splitting reported the same error — that error is a *refusal*,
whereas the wrongful split is a *silent reinterpretation*. Strip the 5-byte gRPC prefix yourself
(`references/protocol-reverse.md` §3), and check the frame count either way.
A `--split varint-length` round trip compares **frame bodies only**, and says so:
```
check against tools\_work\t5-proto\two_frames.bin (26 B, frame bodies only: 2 of 26 byte(s) are
length prefixes (framing, not message data)): MATCH
```
---
## 6. Defects these checks found (in the code, not in the target)
Four defects, three of them in the delivered script, all found by the checks above rather than by
reading the code.
| # | Defect (observed) | How it surfaced | Fix |
|---|---|---|---|
| 1 | When two candidate readings tied, both were lowered to the same confidence as the `bytes` fallback, and the tie was then broken by **alphabetical order** — `bytes` sorts before `nested_message`, so a real nested message was never expanded and `--reencode` copied it as an opaque blob | `--selftest`: `nested two levels / field 3 nested child count` returned `0` instead of `2` | `bytes` now scores 0.2 and ties are broken by an explicit `VIEW_PRIORITY`, structural candidates first |
| 2 | In the `\xNN` path, whitespace and commas were kept as data bytes, so `"08 96 01 | \x12\x07testing"` decoded the space after the bar as `field 4 varint 18` and then hit `invalid_wire_type` | the input-form check in §1 | whitespace and commas are stripped before unescaping, in that branch too |
| 3 | Two fixtures were mis-designed, not the code: a bare packed stream and a length-prefixed frame were round-tripped as if each were one top-level message, which reported `FAIL` for correct behaviour | `--selftest` round-trip section | the packed fixture is now a field *value*; the framing fixture compares frame bodies |
| 4 | The cross-check's own assertion expected `0800` for the explicit zero, but the field under test is number 8, whose tag is `0x40` — the official bytes were `...4000`, and the assertion, not the decoder, was wrong | `official_ref.py` | assertion compares `4000` and the 2-byte cost |
| 5 | `--no-expand` did nothing: the expansion branch never consulted the flag, so a 7-level payload was still walked to the bottom | the deep-payload check below | the branch honours `--no-expand` and records `expansion: suppressed by --no-expand` on the field |
| 6 | Hitting `--max-depth` was invisible: the depth cut-off only appeared as a `rejected` note inside one candidate, so the report showed a shallow tree with no statement that anything had been left unexamined | the same check | a `depth_limit` ambiguity is emitted, naming the path and the limit |
Defect 1 is worth its own line in any future reader's mind: a *heuristic* tie-break silently changed
which bytes the tool re-emitted, and it was caught only because a fixture asserted a child count.
Defects 5 and 6 are the other direction — the tool silently *did more* than asked, and silently
*stopped* without saying so; both are ways a report can be read as complete when it is not.
### 6.1 The deep-payload check
A 7-level nesting (`08 01` wrapped six times in field 1, 14 bytes) is the fixture:
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py tools\_work\t5-proto\deep.bin
frame 0 bytes 0..14 (14 B) status=end_of_buffer
[depth_limit] frame0.1.1.1.1.1.1 @2: a nested reading of this field was not attempted:
--max-depth 5 reached. The bytes here may well be a message; re-run with a larger --max-depth
before concluding that they are not
$ python ... tools\_work\t5-proto\deep.bin --max-depth 2
[depth_limit] frame0.1.1.1 @2: ... --max-depth 2 reached. ...
$ python ... tools\_work\t5-proto\deep.bin --no-expand
expansion: suppressed by --no-expand
```
The truncation is reported as an ambiguity with the path of the field it stopped at, so "no nested
message here" can never be read off a truncated tree.
---
## 7. What was **not** done
* **No device was used.** The pass intended to pull a real app's `.preferences_pb` from
`<DEVICE>`: `python tools\_work\devlock.py acquire proto --timeout 1800` waited 120 s and timed out
with the lock held by another agent (`held by bench for 297s`, note "L1 repack control+patched
install & screen; L3 frida"). Per the two-strike rule the device source was dropped rather than
contended for, and the real-file evidence comes from the DataStore file this repository's own tool
writes (§4). Reading a **device-side** protobuf file with this decoder is therefore **unverified**.
Device facts used elsewhere in this repository (attach-by-name, the broken `logd` channel) were
not re-tested here.
* **No protobuf payload was recovered from the benchmark APKs.** Scanning the available sample
(`<PKG>`-style de-identified, 4,293,620 B, 435 zip entries) for `.pb` / `.proto` / `protobuf`
resources found none; the MASTG crackmes carry no protobuf at all. So no wild-captured payload was
decoded in this pass — `inferred` for any claim that a real traffic body looks like these fixtures,
though §3's bytes come from the reference implementation and §4's from a real AndroidX container.
* **No live traffic.** Nothing was proxied, so the gRPC section of the reference is exercised only
as *framing* (§5), not as captured HTTP/2.
* `blackboxprotobuf` / `protobuf-inspector` were not installed or compared; the reference still
labels them unverified.
* `--max-depth` is measured up to a 7-level fixture (§6.1); no real payload deeper than that was
decoded, and no payload was decoded whose *only* reading is a deep message.
## 8. Artifacts produced by this pass
`tools/_work/t5-proto/` (not committed): `official_ref.py` (official-runtime cross-check, 21 checks),
`framing_check.py` (8 checks), `ds.preferences_pb` (81 B, real DataStore container),
`ds.tree.json` / `ds.edited.json` (decoded trees), `ds.reenc.bin` (byte-identical round trip),
`ds.edited.bin` (80 B, edited value read back by `datastore_inject.py`), `two_frames.bin`,
`frames.tree.json`, `frames.reenc.bin`.
Files changed in the repository by this pass: `skills/apk-reverse/scripts/protobuf_decode_raw.py`
(new), and `skills/apk-reverse/references/protocol-reverse.md` (a new measured subsection under §1,
the corrected proto3-zero wording in §1 consequence 3, and a new framing failure mode in §6).
`SKILL.md` and `README.md` were deliberately not touched; the index rows for the new script are
handed to the lead to register.
docs/tool-verification/EXTENSION-split-apk.md
+337
View File
@@ -0,0 +1,337 @@
# Extension verification — split APK / App Bundle sets
Covers the pass that added split-set support to `scripts/repack.py` and wrote
`references/split-apk.md`. Strength labels as defined in this directory's README:
**observed** (exact command + output reproduced here), **inferred** (follows from observed
facts, step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment for this pass: Windows 11 host, Python 3.14.0, Java 17.0.4.1
(`keytool` NOT on PATH — it was passed explicitly as
`<jdk>/bin/keytool.exe`), `apksigner` 0.9 run from `lib/apksigner.jar`, `zipalign.exe` from the
same build-tools layout. **No `uber-apk-signer` anywhere on the machine**, which is the condition
that made the signing-route fallback a requirement rather than a nicety. Target: one rooted physical
Android 11 (API 30) arm64-v8a device, written `<DEVICE>` throughout.
All package names, application directory names and file names are replaced with placeholders
(`<PKG>`, `<SET>`, `<SET_A>`, `<SET_B>`). The measurements, byte counts, entry counts and outputs
are unchanged; only the identifiers are.
Fixtures: two **real split sets already installed on the device**, copied out read-only
(`adb pull` of the system directory; nothing on the device was uninstalled, replaced or modified).
They were chosen because one exercises each side of the merge/resign decision:
| | Members | Shape |
|---|---|---|
| Set A | 8 | base + 7 density splits, every density split carrying its own resources |
| Set B | 9 | base + 1 ABI split (native libraries only) + 7 density splits that carry a placeholder table and nothing else |
Neither set is third-party: `pm list packages -3` reported **zero** split sets among ~40 installed
third-party packages, while four preinstalled Google components were split. That distribution is
recorded because "pick a third-party split app off the device" is not reliably available.
## 1. Reading a set: what the members actually contain — observed
`pm path <PKG>` on each package returned one line per member (8 and 9 lines respectively), which is
how the targets were identified in the first place:
```
$ adb shell 'pm path <PKG>'
package:/system/priv-app/<DIR>/<SET>.apk
package:/system/priv-app/<DIR>/<SET>-arm64_v8a.apk
package:/system/priv-app/<DIR>/<SET>-hdpi.apk
... (one line per member)
```
`scripts/repack.py --split-mode analyze` over set A (identifiers replaced):
```
== split set: 8 apk(s), 6629510 bytes total
density <SET_A>-hdpi.apk 49623 B split=config.hdpi res=39 entries arsc=9392
density <SET_A>-ldpi.apk 45501 B split=config.ldpi res=39 entries arsc=9392
density <SET_A>-mdpi.apk 45501 B split=config.mdpi res=39 entries arsc=9392
density <SET_A>-tvdpi.apk 74227 B split=config.tvdpi res=78 entries arsc=15124
density <SET_A>-xhdpi.apk 53754 B split=config.xhdpi res=39 entries arsc=9432
density <SET_A>-xxhdpi.apk 57878 B split=config.xxhdpi res=39 entries arsc=10324
density <SET_A>-xxxhdpi.apk 57876 B split=config.xxxhdpi res=39 entries arsc=11200
base <SET_A>.apk 6245150 B split=- dex=classes.dex res=581 entries arsc=1924488 assets=1
[plan] base = <SET_A>.apk (package=<PKG>)
[plan] MERGE BLOCKED: merging is not legal for 7 of 7 splits: config.hdpi, config.ldpi, config.mdpi,
config.tvdpi, config.xhdpi, config.xxhdpi, config.xxxhdpi carry their own resources, and folding those in
means merging resources.arsc (a resource-compiler job). Use --split-mode resign, or pass
--drop-split-resources to accept losing exactly those resources
[plan] resign = legal: one keystore for all 8 members, install the set with
[result] analyze only: nothing was written. Pass --split-mode merge|resign|auto to act on this set.
```
And over set B:
```
== split set: 9 apk(s), 1411272 bytes total
abi <SET_B>-arm64_v8a.apk 255861 B split=config.arm64_v8a libs=arm64-v8a
density <SET_B>-hdpi.apk 16797 B split=config.hdpi arsc=40
... (6 more density members, all 16797 B, all arsc=40)
base <SET_B>.apk 1037832 B split=- dex=classes.dex res=2 entries arsc=71316 assets=1
[plan] base = <SET_B>.apk (package=<PKG>)
[plan] note: every non-base split carries only code/native libraries, so a merged single APK is legal
[plan] note: config.arm64_v8a provides libs for arm64-v8a -- keep only the ABIs the target device runs,
selected with --abi
[plan] merge = legal: 8 member(s) fold into the base
```
Four facts this pass established with those numbers, and that the reference file now depends on:
- **The ABI member carries no `resources.arsc` at all** (set B: `libs=arm64-v8a`, no arsc column).
Its two `.so` files are the whole payload. This is the member that folds cleanly.
- **Each density member carries its own table and its own `res/` entries** (set A: 39-78 entries,
9.0-15.1 KB tables). `resources.arsc` is per member, not per set — the base's 1.88 MB table does
not contain the density variants.
- **A `resources.arsc` can be a 40-byte placeholder.** All seven density members of set B carry
`arsc=40` and zero `res/` entries, while still being legally required to carry a table. A
"has an arsc → owns resources" test would have blocked a merge that is entirely legal here, which
is why the script's test is *`res/` entries or a table over 1 KB*.
- **Base and split share one package name**, read from the binary AXML `package` attribute of every
member, and the base is identifiable as the member whose manifest carries **no `split` attribute**
(set A: `split=-`).
## 2. Unified re-signing — observed
`--split-mode resign` over set A with one keystore, run through the new `apksigner` route:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_A>/ --split-mode resign \
--split-out-dir signed_set/ --ks key.jks \
--keytool <jdk>/bin/keytool.exe --apksigner <build-tools>/lib/apksigner.jar \
--zipalign <build-tools>/zipalign.exe
[resign] signer route: apksigner
[resign] <SET_A>-hdpi.apk -> signed_set/<SET_A>-hdpi.apk (41298 bytes)
[resign] <SET_A>-ldpi.apk -> signed_set/<SET_A>-ldpi.apk (37202 bytes)
[resign] <SET_A>-mdpi.apk -> signed_set/<SET_A>-mdpi.apk (37202 bytes)
[resign] <SET_A>-tvdpi.apk -> signed_set/<SET_A>-tvdpi.apk (61642 bytes)
[resign] <SET_A>-xhdpi.apk -> signed_set/<SET_A>-xhdpi.apk (41337 bytes)
[resign] <SET_A>-xxhdpi.apk -> signed_set/<SET_A>-xxhdpi.apk (49563 bytes)
[resign] <SET_A>-xxxhdpi.apk -> signed_set/<SET_A>-xxxhdpi.apk (49577 bytes)
[resign] <SET_A>.apk -> signed_set/<SET_A>.apk (4172089 bytes)
[result] OK: 8 members, one certificate (sha256 1eb31d9c…2c44)
```
Every member's independent `apksigner verify --print-certs --verbose --min-sdk-version 21
--max-sdk-version 34` returned `Verifies` with:
```
Verified using v1 scheme (JAR signing): true
Verified using v2 scheme (APK Signature Scheme v2): true
Verified using v3 scheme (APK Signature Scheme v3): true
Signer #1 certificate DN: CN=apkreverse, OU=dev, O=dev, L=NA, ST=NA, C=NA
Signer #1 certificate SHA-256 digest: 1eb31d9c9c81104367a66a0201125aa91d28a6a09eb1dcb5029c36bc0bde2c44
```
All eight printed the **same** SHA-256 — that, not per-file `Verifies`, is the acceptance
criterion for a set, and it is the check the run performs explicitly.
**The signing-route fallback is the load-bearing part of this section.** With no
`uber-apk-signer` on the machine, the pre-change script could not sign anything at all: it exited
with `signer jar not found`, naming a jar the user never had. The new `--signer auto` resolves to
`zipalign -p -f 4` + `apksigner sign --v1 --v2 --v3`, which produced the output above. `--apksigner`
accepts `apksigner.bat` (verified: `rc=0`, `--version` → `0.9`), a `lib/apksigner.jar` value run
through `java -jar` (verified: same), and the extensionless shell wrapper (which is a shell script
and does not run on Windows — that is a fact about the wrapper, not about this script).
### The signing route this pass actually used — and why it had to change
`uber-apk-signer` is `repack.py`'s default signer, and it is **not installed on this machine**
(no `uber-apk-signer.jar`, `APK_SIGNER_JAR` unset). Before this change that made every signed build
impossible here: the script exited with `signer jar not found` and named a jar the user never had.
What actually ran in all four measured routes:
| Item | Value used |
|---|---|
| signer | `zipalign.exe` + `apksigner`, the latter run from `lib/apksigner.jar` through `java -jar` |
| `--apksigner` | `<build-tools>/lib/apksigner.jar`. A `.bat` value was also tested and works (`rc=0`, `--version` → `0.9`); the extensionless wrapper in the same layout is a shell script and does not run on Windows |
| `--zipalign` | `<build-tools>/zipalign.exe` |
| `--keytool` | `<jdk>/bin/keytool.exe` — **not on PATH**, and signing aborts without a way to create the keystore, so this flag is mandatory on such a machine |
| scheme flags | `--v1-signing-enabled true --v2-signing-enabled true --v3-signing-enabled true` |
Alignment is handled **per member, and in this order**:
1. the script writes the archive itself, keeping `AndroidManifest.xml` and `resources.arsc` STORED
and 4-byte-aligning `resources.arsc` plus uncompressed `lib/**`, then re-reads the file and
prints the gate (`== alignment gate: resources.arsc STORED and 4-byte aligned (OK)`);
2. `zipalign -p -f 4` runs over that output (`-p` adds the page alignment the loader wants for
`extractNativeLibs="false"` libraries);
3. `apksigner sign` runs **last** — it adds signature entries without reshuffling the archive,
whereas a `jarsigner`-style rewrite after aligning destroys the alignment and the install is then
refused with `[-124]`.
For a split set this is repeated for every member, which is why `resign_split_set()` de-signs and
re-aligns each member individually before signing it: a single member whose `resources.arsc` is
compressed or unaligned is refused by the installer on its own, and the resulting error names that
member's package rather than the set.
## 3. Merging — observed, including the refusal
Set B, where every non-base member is code or native libraries:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_B>/ --split-mode merge \
--out merged.apk --abi arm64-v8a --ks key.jks --apksigner <...>/lib/apksigner.jar
[merge] + lib/arm64-v8a/lib<name>_compat_libc++.so (18944 bytes <- <SET_B>-arm64_v8a.apk)
[merge] + lib/arm64-v8a/lib<name>utilsjni.so (206680 bytes <- <SET_B>-arm64_v8a.apk)
[merge] folded: dex=0 libs=2 assets=0 conflicts=0
[merge] DROPPED 7 entry/ies. This is a DOWNGRADED build -- say so when you deliver it:
[zip] classes.dex 872160 bytes method=8 crc=<...>
[zip] resources.arsc STORED offset=0x581fc aligned
== alignment gate: resources.arsc STORED and 4-byte aligned (OK)
[sign:zipalign] rc=0
[sign:apksigner] rc=0
[out] merged.apk (603824 bytes)
[verify:apksigner] rc=0 Verifies / v1 true / v2 true / v3 true
[result] OK
```
`dex=0` is correct rather than suspicious: the base owns `classes.dex` and no member of this set
carries a second one, so no renumbering was needed. The `libs=2` fold is the whole point — a
one-file APK that now carries the native libraries the ABI member used to provide separately.
Set A, where merging is **not** legal, refuses rather than producing a broken artifact:
```
$ python skills/apk-reverse/scripts/repack.py --split-dir <SET_A>/ --split-mode merge \
--out merged.apk --no-sign
error: refusing to merge <SET_A>-hdpi.apk -- it carries its own resources (39 res entries,
arsc=9392 bytes).
Folding it in would require merging resources.arsc, which this script does not do.
Either use --split-mode resign (recommended for a resource split), or pass
--drop-split-resources to accept that the resources it owns are lost.
exit code 1
```
This is the enforcement the reference file promises: the refusal happens **before** any output is
written (the `--out` path does not exist afterwards), and the escape hatch is explicit, printed, and
required. The mechanism is **observed**; whether a build produced through that hatch merely loses
its density variants or misrenders is **inferred** — it was not exercised, because the only sets
available were a legal-merge set and a refused set, and manufacturing the third case would have
meant hand-editing a resource table.
## 4. Single-APK behaviour is unchanged — observed
Both pre-existing paths were re-run through the same new signing route, on an unmodified sample
APK from `tools/_work/bench/apks/`:
- **zero-change roundtrip** (`--apk … --out … --ks …`, no `--dexdir`): the run dropped exactly the
three original signature artifacts (`META-INF/CERT.RSA`, `META-INF/CERT.SF`, `META-INF/MANIFEST.MF`),
wrote a 60964-byte unsigned archive, signed to 70647 bytes, reported `== alignment gate:
resources.arsc STORED and 4-byte aligned (OK)` and `[result] OK` with v1/v2/v3 all `true`;
- **explicit dex replacement** (`--dexdir` supplying the same `classes.dex`, 5528 bytes):
`[repl] classes.dex<-…/classes.dex`, the entry re-added at the same size and CRC, alignment gate
OK, `[result] OK`.
Both were run with `--no-sign` as well, which exits 0 and reports `UNSIGNED (--no-sign)`. `--help`
works, exits 0, and carries the split examples and the route-decision text; `--split-mode analyze`
exits 0; a merge refusal exits 1; a failed verification would exit 2.
The output fix worth recording: `AndroidManifest.xml` used to be printed as `NOT ALIGNED` in the zip
layout (it only has to be STORED, unlike `resources.arsc`). It now prints `-` for entries that need
no alignment, so the report cannot be misread as a failing gate.
## 5. Install and launch on the device — **unverified**
**Not executed.** The device was held by a concurrent device experiment for the whole window
available to this pass (`python tools/_work/devlock.py status` reported the lock held by another
teammate for 1417 s, note naming its own install/launch and Frida work), and the lead's decision was
to ship this gap labelled rather than wait for it. Heavy device work must be serialised
(`tools/_work/ENV.md`), so the honest label for everything below is **unverified** — a claim about
installability that was never installed is exactly the failure `references/verification.md` exists
to prevent.
Prepared, and not run — the sequence is the one in `references/split-apk.md` §4, against the
re-signed renamed fixture:
```
adb install-multiple -r <signed_set>/*.apk
adb shell cmd package resolve-activity --brief <PKG>
adb shell am start -n <PKG>/<launcher-activity>
adb shell 'dumpsys activity activities | grep -m2 ResumedActivity'
adb exec-out screencap -p > launch.png
```
What the pass *did* establish without a device, and what each fact does not cover:
- **The artifacts are installable in principle, not in fact.** All eight members of the re-signed set
verify with v1/v2/v3 `true` and one identical SHA-256 (§2), and the merged single APK verifies the
same way (§3, §4). Uniformity of the certificate and a passing `apksigner verify` are necessary
conditions for a set install; the platform installer accepting the set is the step that was not run.
- **The negative control was not run either.** The signature-conflict text quoted in
`references/split-apk.md` §6 (a pulled set reinstalled under its own package name) is the
platform's documented shape and the shape a competing tool reports; no captured output from
`<DEVICE>` backs it.
- Every statement about **what the archives contain** — split roles, arsc ownership, the 40-byte
placeholder tables, the fold set, the refusal — is **observed** and device-independent.
- Every statement about **what the package manager does with them** is **inferred**.
**What closing it needs:** one device window that is not shared, then the sequence above (about three
minutes of device time). The two things most likely to show up there, and already accounted for in
the script: an OEM installer interception on `adb install-multiple` (there is a root
`pm install-multiple` path for it) and a launcher activity that needs resolving rather than guessing.
## 6. Maintenance gate — observed
Run at the repository root with the change in place:
```
$ python check_repo.py
== result: 1 problem(s)
- skills/apk-reverse/scripts/dex_strpatch.py --help crashed:
$ python check_refs.py
== section references: 242 reachable ==
== result: 0 dangling, 0 warning(s)
```
`check_refs.py` is clean, which covers the three `references/repack-and-sign.md` §-references the new
file adds, and `check_repo.py` reports one problem in a **different** file belonging to a different
task; `repack.py` parses and answers `--help` with exit 0.
`check_repo.py` requires every `references/*.md` to be named in `SKILL.md` or the root `README.md`,
and this task was not permitted to edit either, so the reachability step was pre-checked in a **copy**
of the repository (`tools/_work/repo-check/`, workbench only, never in the tree) by running the gate
twice on one snapshot with the index lines as the only variable:
```
without the index lines: == result: 5 problem(s) ... UNLISTED references/split-apk.md
with the index lines: == result: 4 problem(s) ... (the UNLISTED line is gone)
```
The new file therefore contributes nothing to the gate once listed. The listing itself was added to
`README.md` by the lead, and the run quoted above is the result.
## 7. Deliberately not done — and what stays unverified
- **No `.aab` was processed.** A bundle input would need `bundletool` (absent here, and it is the
tool that owns the correct resource merge). This pass only ever handled maps of an *installed* set.
- **No fixture was hand-built with `aapt2`/`d8`.** Those became available on this machine only after
the fixtures were chosen (a build-tools archive was unpacked late in the round), and the two real
sets already exercised both branches of the decision. Building a minimal split set by hand remains
the cheapest way to reproduce this topic on a machine with no installed bundle and no device, and
the route is **inferred** — it was not attempted here. It would not change the resource-merge
boundary either: `aapt2 link` merges resources when it *builds* an APK from sources, not when it is
handed two finished ones.
- **No resource merge was implemented or attempted**, by design
(`references/split-apk.md` §5). The consequence is that a bundle whose resources live in splits
cannot be reduced to one faithful APK by anything in this repository.
- **`apktool` was not exercised, because it is not usable on this machine.** Measured:
`apktool.bat d …` fails with `Error: Unable to access jarfile <missing-path>`, while
`lib/apksigner.jar` and `zipalign.exe` in the same layout work. Any manifest edit that needs
`apktool` is therefore **unverified** here, and AXML attributes are read-only in this pass (the
script parses the string pool and the root element's attributes; nothing writes AXML).
- **Feature splits (`isFeatureSplit`) were not measured.** Neither available set has one. The
handling of `isFeatureSplit` / `configForSplit` in the reference file is **inferred** from the
platform's documented semantics plus the fact that both attributes are readable through the same
AXML reader that was verified against real manifests.
- **Split APKs that are not App-Bundle-derived** (the older per-density `-hdpi.apk` naming style,
or an OEM's own module set) were not available; both sets here use the standard `config.<qualifier>`
split names, and the reader accepts either because it reads the manifest attribute rather than the
file name.
- **A XAPK/APKS archive** (a store bundle of bundles, usually a zip holding the split set plus
`manifest.json`) was not processed. The script's `--split-dir` recursion handles a *directory* of
APKs, so an unpacked XAPK works; the archive itself is not read.
docs/tool-verification/EXTENSION-stalker-exclude.md
+89
View File
@@ -0,0 +1,89 @@
# Stalker `exclude` — what it fixes, and what it does not
**Question.** Community practice says a followed thread that runs through `libc`/`libart` on arm64
gives deadlocks, watchdog `SIGABRT`s and zero-event traces, and that `Stalker.exclude()` is the
remedy. This pass measured which of those two claims survives contact with a device.
**Strength summary:** the *crash* half is `observed` with a clean control; the *zero-event* half is
`observed` as a **negative** — exclusion did not change it. Every command and its output are below.
## Design
Three arms, one device, one package, one module, one follow window. Exactly one variable moves
between `baseline` → `control` → `treatment`.
| Arm | What it does | Why it exists |
|---|---|---|
| `baseline` | spawn, attach, inject the script, **resume, and never follow** | Without this arm a dead process cannot be attributed: "following killed it" and "the app dies under frida anyway" look identical |
| `control` | follow the main thread for 6 s with `excludeModules: []` | The shape that produced the earlier zero-event / `SIGSEGV` results |
| `treatment` | same follow, with the script's built-in list of 20 system modules excluded | The proposed remedy |
The harness also neutralises one trap in the script itself: `CONFIG.autoStart` defaults to `true`, so
loading `stalker_trace.js` starts a follow with *default* options before a runtime configuration can
land. Left alone, that run is neither the control nor the treatment and contaminates both. The
harness patches `autoStart` to `false` in the injected source and asserts the patch count.
For the same reason the harness resumes explicitly: `frida` spawns a process **suspended**, so
without `device.resume(pid)` the followed thread executes nothing and both arms report a zero-event
trace *manufactured by the harness*. That failure mode is indistinguishable from the one under
investigation, which is exactly why it has to be engineered out before the measurement means
anything.
## Result
```
baseline alive=True (no follow installed)
control alive=False err=script has been destroyed
EXCL|excluded=0/0 []
treatment alive=True DONE|reason=timeout blocks=0 blk=0 calls=0 truncated=0
EXCL|excluded=20/20 [libm.so,libdl.so,libart.so,libartbase.so,libnativehelper.so,…] not-loaded=3
WARN|zero events 1500ms after follow (blocks=0 blk=0 calls=0) -- the pipeline is NOT proven
```
## What this settles
1. **A follow with no exclusion killed the target process, and the baseline arm rules out the
alternatives.** The process survives an attach plus resume with no follow, and dies only when the
follow is installed. That is `observed`, not inferred.
2. **Exclusion kept the process alive.** The treatment arm reached its normal `DONE|reason=timeout`
with the process still running. Excluding the system libraries from a follow is not a style
preference.
3. **Exclusion did not restore event delivery.** The treatment arm still reported
`blocks=0 blk=0 calls=0` after a full 6-second window. The zero-event trace is therefore a
**separate defect from the load problem**, and `Stalker.exclude()` must not be presented as its
fix. This partially contradicts the community framing, which lists "no events" among the symptoms
exclusion cures.
4. **The zero-event warning added to `stalker_trace.js` this pass fires correctly** — `treatment`
emitted `WARN|zero events 1500ms after follow`, while `control` could not, because its process was
already gone. A dead pipeline now announces itself instead of being read as "the code did not run".
## What this does not settle
- **Why events do not arrive.** This measurement says it is not the exclusion list. It does not
identify the cause, and it does not distinguish "no events" from "the followed thread genuinely ran
nothing in that window" — the per-arm control that would separate those (following a thread
executing a known loop) was not run.
- **Whether the crash is reachable without a follow.** Only one package and one module were used, and
the harness kills the process at the end of each arm, so nothing is claimed about a long-lived follow.
- **Anything about other ROMs, other frida versions, or armv7.** Single-device result.
- **The multiplier.** Nothing here measures the 20-50x cost figure; this is about survival and
delivery, not throughput.
## Environment
`<DEVICE>` (Android 11 / API 30 / arm64-v8a, Magisk root), frida host 16.7.19, on-device
frida-server 16.7.19 reached over `adb forward tcp:27099`. Target: the MASTG `UnCrackable-Level1`
sample, package `<PKG>`, module `libc.so` followed on the main thread for 6 s per arm. Device
`load average` was ~30 throughout — high for an 8-core phone, and worth recording because the
earlier incident in `native-dbi-and-deobfuscation.md` §6 happened on a loaded device.
## Reproducing
```
python tools/_work/b6/stalker_ab.py --package <PKG> --module libc.so \
--mode baseline|control|treatment --seconds 6 --device 127.0.0.1:27099 \
--out tools/_work/b6/r_<mode>.json
```
The driver is a work artefact under `tools/_work/` (git-ignored), not part of the skill. Its three
arms and the two harness traps it neutralises are the reusable part.
docs/tool-verification/EXTENSION-vmp-diff.md
+293
View File
@@ -0,0 +1,293 @@
# Extension verification — differential hardening for Dex-VMP
Covers the extension pass that added `references/vmp-differential-analysis.md` and
`scripts/vmp_diff_harness.py`. Strength labels as defined in this directory's README:
**observed** (exact command + output here), **inferred** (follows from observed facts,
step not executed), **unverified** (assumed or externally reported, not independently
confirmed).
Reference environment: Windows 11 host, Python 3.14.0, JDK 17.0.4.1
(`C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe`), Android build-tools 2.19 / D8
8.2.2-dev off-PATH. **No device and no network were involved in this pass** — unlike the
unpacking and rootdump passes, this one is deliberately machine-local, because the thing
worth measuring here is the comparison logic, not the platform.
**Headline: the local half of the route is measured and closes exactly; the
platform half was not exercised at all, and the cost table says why it cannot be.**
---
## 1. Environment discovery — observed, and one correction to `tools/_work/ENV.md`
`ENV.md` records `E:\tools\android-14\` as a complete build-tools directory. At pass time
it held **three** entries:
```
$ Get-ChildItem 'E:\tools\android-14\' -Name
apksigner.bat
lib
zipalign.exe
```
`aapt2.exe`, `d8.bat` and `dexdump.exe` were absent, so the `build` subcommand had no
toolchain. The complete build-tools tree was sitting unextracted in `E:\tools\bt.zip`:
```
$ (bt.zip entries) | Where-Object { $_.Name -match '^(aapt2|d8|dexdump|split-select)' }
android-14/aapt2.exe 4566296
android-14/d8.bat 3156
android-14/dexdump.exe 986392
android-14/split-select.exe 1589016
android-14/lib/d8.jar 14929869
```
After `Expand-Archive -Path 'E:\tools\bt.zip' -DestinationPath 'E:\tools\' -Force`:
```
$ E:\tools\android-14\aapt2.exe version
Android Asset Packaging Tool (aapt) 2.19-10229193
$ E:\tools\android-14\d8.bat --version
D8 8.2.2-dev (build facedf41bbd28b563d1e9e09c5f72d7c5ca598d8 from go/r8bot (luci-r8-custom-ci-focal-14-zjz7))
```
Two further tool facts worth recording, both **observed**:
- **`E:\agb_apktool.jar` does not exist**, so `E:\tools\bin\apktool.bat` (which is
`java -jar "E:\agb_apktool.jar" %*`) resolves to nothing. apktool is unavailable on this
machine, and a bare `apktool d` fails with a jar-not-found error rather than a
recognisable "tool missing" message.
- `javac` is on PATH only through the Oracle `javapath` forwarder; the real JDK is at
`C:\Program Files\Java\jdk-17.0.4.1`. The harness takes `--javac` explicitly for this
reason.
## 2. `build` — observed
```
$ python skills/apk-reverse/scripts/vmp_diff_harness.py build \
--build-tools E:\tools\android-14 --out tools/_work/t5-vmp/fixture \
--javac "C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe" --apk
javac: C:\Program Files\Java\jdk-17.0.4.1\bin\javac.exe
d8: E:\tools\android-14\d8.bat
aapt2: E:\tools\android-14\aapt2.exe
generated: ...\src\OpCoverBig.java (37196 B)
$ ... javac -g --release 11 -d ...\classes ...\OpCoverProbe.java ...\OpCoverBig.java
javac ok: 9 class files
$ E:\tools\android-14\d8.bat --min-api 26 --no-desugaring --output ...\dex <9 .class paths>
d8 ok: ...\dex\classes.dex (35028 B)
== ...\classes.dex (35028 bytes) ==
classes=9 methods_with_code=75 instructions=3065 desynced=0
opcode coverage: 218/224 (97.3%)
not emitted by this fixture (6):
0x09 move-object/16 move-object/16 (32x) needs both registers >= 256; the object
frame reaches 256 on the source side only, which d8 encodes as 0x08
0x1b const-string/jumbo needs a string_ids index > 65535, i.e. a fixture carrying
65536+ string constants
0x2a goto/32 needs a >32767-code-unit backward jump; javac refuses the method
first with 'code too large' (its own 64 KB per-method bytecode cap)
0xfd invoke-custom/range d8 emitted the 35c form for every lambda shape tried, including
a six-parameter one
0xfe const-method-handle no Java-language literal; reachable only by hand-written smali
or direct dex construction
0xff const-method-type same as 0xFE
$ E:\tools\android-14\aapt2.exe link --manifest ...\apk\AndroidManifest.xml -o ...\apk\base.apk
aapt2 ok: ...\probe.apk (20482 B, unsigned)
build rc=0
```
Structural facts behind those numbers: **0 desynced methods** (every `decode_all` walk
ended exactly on `insns_off + insns_size*2`, i.e. the format table and the emitted stream
agree), 3,065 instructions across 75 method bodies, 9 classes.
### 2a. The two toolchain traps this measured — observed
Both are encoded in the script so they are not re-derived:
- **`d8` refuses a non-existent `--output` directory.** First run:
`Error: Invalid output: ...\dex / Output must be a .zip or .jar archive or an existing
directory / Compilation failed` and rc=1. Same class of trap as `frida-dexdump -o`.
`build` creates the directory first.
- **`aapt2 link` needs `android.jar` for every `android:` attribute.** A manifest carrying
`android:versionCode` / `android:versionName` / `android:minSdkVersion` produced five
`error: attribute android:... not found` lines and rc=1. A manifest using only the
un-namespaced `package` attribute links cleanly with no `-I` flag at all — which is what
the harness ships, at the cost of an APK with no version attributes. That is acceptable
for a dex carrier and is stated in the output.
### 2b. `goto/32` is unreachable from Java — observed
Driving the generator to 33,000 loop-body statements to push the backward jump past the
20t range:
```
$ python tools/_work/t5-vmp/gen_big_probe.py 33000
generated ...\OpCoverBig.java: 34442 lines, 766468 bytes
javac rc=1
OpCoverBig.java:1427: error: code too large
public static int gotoFar(int x) {
1 error
```
The JVM's own 64 KB per-method bytecode limit fires before the dex branch-distance
threshold can be crossed. `goto/32` therefore cannot be covered by any javac-built
fixture, at any generator size — this is a structural ceiling, not a tuning problem.
## 3. Closed loop — observed, and this is the load-bearing result
`simulate` relabels every opcode through a bijection whose ground truth it writes to disk.
`compare` is then given only the two dex files and must re-derive that table.
```
$ python .../vmp_diff_harness.py simulate .../fixture/dex/classes.dex \
--out tools/_work/t5-vmp/sim.dex --seed 7 --table tools/_work/t5-vmp/truth.json
simulated: ...classes.dex -> ...sim.dex
mode=relabel, 3065 method body(ies)/instruction(s) touched (seed 7)
header recomputed: checksum_ok=True signature_ok=True
ground-truth table written to ...truth.json (compare must re-derive it)
$ python .../vmp_diff_harness.py compare .../fixture/dex/classes.dex tools/_work/t5-vmp/sim.dex
aligned=75 resized=0 stripped=0 missing=0 added=0
aligned slots compared: 3065
verdict: USABLE
(map: 218 rows, every one verdict=high)
compare rc=0
```
Scoring the derivation against the ground truth:
```
ground-truth bijection size : 256
opcodes the fixture emits : 218
compare rows : 218
compare high-confidence rows : 218
compare undetermined : 6
compare non-injective : 0
compare run verdict : usable
derived entries that are WRONG : 0 (none)
derived entries with no truth : 0 []
emitted opcodes NOT derived : 0 []
EXACT RECOVERY: 218/218 emitted opcodes (100.0%)
CLOSED-LOOP VERDICT: IDENTICAL
```
Why the denominator is 218 and not 256: the ground-truth bijection covers all 256 byte
values, but 32 of those are unallocated dex slots and 6 are opcodes this fixture never
emits — and an opcode that never appears leaves no known plaintext. The six are reported
as `undetermined` with their reasons, never guessed. **Every entry the run did produce was
correct, and every opcode that could be read was read.**
## 4. Negative shapes — observed
The decision table's failure branches were driven on purpose, because a detector that has
never been seen to fire is not a detector.
**Stub shape** (`--mode stub`: every body replaced by an equal-length `return-void` fill —
the extraction shape):
```
$ python .../vmp_diff_harness.py compare .../classes.dex tools/_work/t5-vmp/stub.dex --quiet
aligned=75 resized=0 stripped=0 missing=0 added=0
aligned slots compared: 3065
verdict: NOT-USABLE
! 1 private byte(s) are claimed by up to 218 original opcodes each. No injective
relabelling can do that: it is the signature of method bodies replaced by one
common stub, and any substitution read off such a pair describes the stub, not a
private opcode space.
rc=1
```
This is the failure mode worth reading twice. In the per-opcode direction the reading is
**perfect** — 218 rows, all `high`, zero conflicts — because every original opcode maps to
the same private byte. Only the reverse (`private byte → set of originals`) check exposes
it. An implementation without that check would have returned a confident, wholly
meaningless table, and the first version of this one did exactly that: measured before the
fix, the same input returned `verdict: PARTIALLY-USABLE` alongside a note reading
"0 of 218 opcodes saw more than one candidate byte" — a warning that contradicted itself.
**Length-changing shape** (`--mode shrink`: `insns_size` rewritten without moving bytes, a
deliberately inconsistent image that exists only to drive the detector):
```
$ python .../vmp_diff_harness.py compare .../classes.dex tools/_work/t5-vmp/shrink.dex --quiet
aligned=0 resized=75 stripped=0 missing=0 added=0
aligned slots compared: 0
verdict: NOT-APPLICABLE
! 75 method(s) kept a body but changed its length. Boundaries cannot be projected
across a length change; a private opcode stream with its own length table is not
reachable this way.
rc=1
```
Regression after both fixes: the relabel loop still reports `USABLE` and the closed-loop
check still reports `IDENTICAL` — recorded above, re-run after the verdict-logic change.
**Not driven:** the `stripped` shape (`code_off == 0`) needs a class-data rewrite rather
than a byte edit, so its detector path is **inferred** from the shape counters, not
observed firing. The counters themselves (`stripped=0` in every run above) are observed.
## 5. `emit-smali` — observed
```
$ python .../vmp_diff_harness.py emit-smali tools/_work/t5-vmp/sim.dex \
--table tools/_work/t5-vmp/cmp.json --class LOpCoverProbe --method arithInt
; instructions restored: 3065 ; unmapped-opcode stops: 0
; THIS IS A READING SKELETON, NOT ASSEMBLABLE SMALI.
...
.class public LOpCoverProbe;
.super Ljava/lang/Object;
.source "OpCoverProbe.java"
# ---- arithInt(II)I registers=4 insns_size=29
.method public static arithInt(II)I
.registers 4
# 0x0000 90000203 add-int
# 0x0004 d000e803 add-int/lit16
# 0x0008 d8000007 add-int/lit8
# 0x000c b130 sub-int/2addr
...
# 0x0038 0f00 return
.end method
```
Two checks that the restore is real rather than plausible: the stream decodes to the
**original fixture's** `arithInt` body (compare it against `OpCoverProbe.java`'s 17
operations), and the payload-bearing `switchPacked` method also restores cleanly —
`packed-switch` followed by its payload block, stepped over by its own layout rather than
mistaken for opcodes. `unmapped-opcode stops: 0` is the honest coverage measure, and it is
zero only because the table is complete for this fixture.
Three defects were found and fixed by this run, each one a silent-wrong-answer shape:
- `--class LOpCoverProbe;` matched **zero** classes because the filter accepted only one of
the three spellings (`LOpCoverProbe;` / `LOpCoverProbe` / `OpCoverProbe`) — output read
exactly like a class that is not in the file.
- `.source` printed `"<string_idx 34748 out of range>"`: `source_file_idx` sits at
`class_def_item + 16`, not `+ 20` (that is `annotations_off`).
- Methods rendered without access flags, so the skeleton was not readable as smali.
## 6. What was not done
- **No hardening platform was contacted.** The entire submission link of the chain is
`unverified`, and its cost estimate in the reference file is reasoning about how those
services are distributed (accounts, rate limits, download-page delivery, possible refusal
of a synthetic fixture), not a measured upload. This is the pass's largest gap and it is
not closeable from this machine.
- **No real VMP sample was used.** `tools/_work/bench/repos/DexPatcher/assets/ezAndroid.apk`
is recorded in `ENV.md` as a real Dex-VMP challenge sample and was **not** opened in this
pass; the differential route needs a *matching hardened counterpart of the same build*,
which no available artifact provides.
- **The engine-behaviour assumption is inferred**: that a real VMP substitutes per-opcode
at stable instruction length. `simulate` implements the weakest form of that assumption
deliberately — it is the only form this harness can check itself against.
- **Table stability across inputs is unverified.** Whether one fixture's table applies to
another build of the same app, or survives a vendor version change, was not tested and is
the reason the reference file says to re-derive per target.
- **The `stripped` detector path did not fire** (see §4).
- `--mode shrink` writes a structurally inconsistent dex by design; `dexutil.Dex.check()`
will not reject it because it does not range-check `insns` against the file. It is a
detector probe, is labelled as such in `--help`, and must not be treated as a build.
docs/tool-verification/README.md
+18
View File
@@ -46,6 +46,24 @@ evidence here, one file per topic, in the same three labels:
| `EXTENSION-native-dbi.md` | Stalker: the zero-event boundary, the crash from following a hot libc export, and an offline protobuf round-trip that caught a defect in its own decoder |
| `EXTENSION-kernel-ondevice.md` | The MT Manager APK MCP probe, the module-environment facts it could verify, and the kernel routes that this device's 4.14 kernel puts out of reach |
## Benchmark-pass record
The benchmark pass put **public targets** under the documented routes — `tests/benchmark.md` is the
matrix, and these files are the evidence behind its rows. Same three labels. Several of these rows
came back negative, and four of them overturned a conclusion this repository had already recorded;
those are called out in the matrix rather than buried here.
| File | Covers |
|---|---|
| `EXTENSION-benchmark-l1-l3.md` | The first two matrix rows: the L1 equal-length patch -> repack -> re-sign chain with its on-screen behaviour change and its zero-change control, and the L3 time-to-death baseline, the two spawn/patch/detach orderings that cannot work, and the terminate path that patching the obvious death site does not stop |
| `EXTENSION-extraction-shell-bench.md` | `dex_dump_validate.py` against an 11-variant skeleton set: the trivial-body ratio as a bimodal result rather than a threshold, the three script defects the variants exposed, and the ranking that pointed at a modified image until independent review caught it |
| `EXTENSION-stalker-exclude.md` | Stalker exclusion measured across three arms: the no-follow baseline, the follow that kills the process, and the follow with 20 modules excluded that survives but still delivers zero events |
| `EXTENSION-java2c.md` | Java2C against JNI sinking: native-declaration density roughly 2000x apart, why a `Java_*` symbol search fails silently, and the Dex-to-C build chain no toolchain on this host could complete |
| `EXTENSION-vmp-diff.md` | The known-plaintext differential: a 218-of-224-opcode coverage fixture, a closed-loop check that recovered 218/218 mappings with zero fabrications, and the one link that cannot be automated |
| `EXTENSION-kernel-weapons.md` | The generated KernelSU/APatch scaffold and its three kernel-side templates, all shipped unbuilt, plus the structural correction that a userspace module cannot change a syscall return value |
| `EXTENSION-split-apk.md` | Two real split sets through both branches: unified re-signing, the merge that is refused by design, the merge that succeeds, and the signer chain this host actually needs |
| `EXTENSION-protobuf-raw.md` | A schema-free decoder cross-checked against the official runtime and a real DataStore container, with the packed-boundary and proto3-zero ambiguities reproduced on real bytes |
## Adding a run
Record the tools, versions and exact commands, and label each claim. Keep target-specific facts out:
skills/apk-reverse/SKILL.md
+20 -2
View File
@@ -107,6 +107,13 @@ vocabulary and a set of reassuring numbers.
- Packers, custom loaders and code virtualization: identifying them, measuring the validation
boundary, and the routes that survive it (`packers.md`,
`code-virtualization-and-custom-linkers.md`).
- **Java2C** — recognising that the code left the dex entirely, and the evidence that separates it
from an extraction shell, a VMP and ordinary JNI sinking (`java2c-and-jni-sinking.md`,
`scripts/java2c_probe.py`). Measured: `native` declaration density was **0.03-0.04%** in three real
JNI samples against **82.76%** in a Java2C-shaped fixture — roughly 2000x — and the `Java_*` symbol
count matched the dex native-method count 1:1 per ABI. **No library produced by a Dex-to-C compiler
was built here** (no NDK, no host clang, no device clang, WSL unavailable), so every Java2C-specific
criterion is `inferred`; the JNI-sinking side and the discrimination itself are `observed`.
- The native layer: `.so` hosts, tamper-triggered self-termination, forged ELF structure, and
neutralising a terminate path without freezing the process (`native-and-so.md`,
`native-tamper-and-suicide.md`).
@@ -299,7 +306,8 @@ unread in this repository.
| A `FORTIFY: pthread_mutex_lock called on a destroyed mutex` abort in a Flutter app, on the **main** thread, before the first frame completes | `dart-aot.md` — check `libapp.so` is actually being loaded; Flutter's engine bootstrap is the usual place a native lifecycle fault surfaces |
| Log says a **Java-layer** signature/integrity check **passed**, yet the process dies | `code-virtualization-and-custom-linkers.md` §a Java-layer "signature killer" is a decoy |
| Deleting a library fixes validation but yields `UnsatisfiedLinkError: dlopen failed: library "X" not found` | `code-virtualization-and-custom-linkers.md` §the deadlock that eats hours |
| Whole classes appear as bare `native` declarations with no body | `code-virtualization-and-custom-linkers.md` |
| Whole classes appear as bare `native` declarations with no body | `java2c-and-jni-sinking.md` — read it **before** dumping memory: if this is Java2C there is no DEX to find, at any point in the process lifetime. A handful of `native` methods in an otherwise ordinary dex is JNI sinking, not this |
| A `Java_*` search over a hardened library returns nothing at all | `java2c-and-jni-sinking.md` §The JNI boundary — why a symbol search fails silently — dynamic registration, or `-fvisibility=hidden`. The check that works is "exports `JNI_OnLoad` and zero `Java_*`" |
| A library's **SONAME does not match its filename** | `code-virtualization-and-custom-linkers.md`, `native-and-so.md` |
| Your edit had **no effect at all**, with no error | `server-config-and-updates.md` §3 (the value may be server-sent), then `packers.md` §map the validation boundary |
| Process **hangs** with no crash record, or dies to a `uid 0` killer | `native-tamper-and-suicide.md` §the rule (you probably made a terminate path *not return*) |
@@ -336,6 +344,9 @@ unread in this repository.
| The traffic is protobuf/gRPC/QUIC, or a proxy sees TLS but requests still fail on a Flutter app | `references/protocol-reverse.md` — schema-less protobuf, frame capture, and native-side pinning |
| Userspace hooks land and the app still dies: the check reads `/proc/self/status` through a raw `svc`, or runs before `JNI_OnLoad` | `references/kernel-and-environment-hardening.md` — what the next layer up and down can actually do, and when to stop |
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md` §1. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer, and `docs/tool-verification/EXTENSION-protobuf-raw.md` records exactly what a schema-free decode can and cannot settle |
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
## Gates — clear these before you patch, in order
@@ -556,6 +567,8 @@ Load only what the current step needs.
| `references/byte-level-patching.md` | You want to change behaviour by editing a few bytes rather than rebuilding a method — equal-length patches, locating an instruction's exact offset, dex header integrity fields, branch polarity, verifier legality |
| `references/packers.md` | The app is packed/hardened, or an edit makes it die before your code runs. Also load before discarding any route as "blocked by the shell" |
| `references/code-virtualization-and-custom-linkers.md` | **No packer, dex is readable, and a re-signed build still dies** — whole classes turned into `native` declarations, a private loader with a mismatched SONAME, an embedded self-decrypting payload, or a Java-layer "signature killer" that logs success while a native check kills you. Covers the keep-it/drop-it deadlock and the string-redirect escape |
| `references/java2c-and-jni-sinking.md` | **Whole classes are bare `native` declarations and you are about to hunt for a decrypted DEX** — Java2C has none, ever: the code was compiled into a `.so`, so the route is reading that library, not dumping memory. Also the JNI boundary itself — why a `Java_*` symbol search fails silently — and the evidence that separates Java2C from an extraction shell, a VMP and ordinary JNI sinking |
| `references/vmp-differential-analysis.md` | Method bodies are present but decode as **private opcodes** — a real Dex VMP: the known-plaintext differential, which links can and cannot be automated, how to *prove* a derived opcode table, smali generation, and when the route is closed |
| `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works |
| `references/dart-aot.md` | The logic lives in a Dart AOT snapshot (`libapp.so`): pinning the Dart version, building a matching decompiler, the object pool and reference indexing, register/boolean conventions, locating and patching Dart code |
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** |
@@ -586,6 +599,7 @@ Load only what the current step needs.
| `references/protocol-reverse.md` | The traffic is protobuf without a schema, gRPC, or QUIC/HTTP3; or a proxy sees TLS while the app still fails — schema recovery, frame capture, and native-side certificate pinning (Flutter/BoringSSL) with its boundaries |
| `references/kernel-and-environment-hardening.md` | Userspace hooking provably cannot reach the check — raw `svc` syscalls, `init_array`-early detection, a ROM that hunts instrumentation: what each layer up and down can still do, the kernel-route map and its version gate, and when escalating is the wrong answer |
| `references/on-device-tooling.md` | Working **from the phone itself**: MT Manager edit/repack/sign and its built-in APK MCP, LSPosed Manager, Termux+frida, and on-device data inspection |
| `references/split-apk.md` | The target is a **split APK / App Bundle set** (`base.apk` + `split_config.*.apk`), or `pm path <PKG>` returned several files: reading a set off a device, when merging into one APK is legal versus when only unified re-signing works, the install refusal each mistake produces, the manifest attributes a merge breaks, and ABI/density matching |
## Script index
@@ -608,7 +622,7 @@ All scripts are parameterized and path-agnostic; pass paths explicitly. Run `--h
| `scripts/dart_pool_strings.py` | Recover string literals from a Dart AOT snapshot: framed entries, the one-byte vs UTF-16 split, file offsets, and a run-length noise filter |
| `scripts/dart_disasm.py` | Annotated windowed disassembly of Dart AOT code (pool + boolean annotations) plus a B/BL caller index |
| `scripts/find_refs.py` | Count and list callers of a method/field (blast-radius check). Takes a smali tree, a `.dex`, a directory of either, or an `.apk`. Always prints what it scanned, so "the input was unreadable" cannot be mistaken for "nothing references it" |
| `scripts/repack.py` | Rebuild an APK with replaced dex, strip only signatures, keep `META-INF/services/`, **write a 4-byte-aligned archive** (`resources.arsc` STORED+aligned, which Android R+ refuses to install without), sign, and verify |
| `scripts/repack.py` | Rebuild an APK with replaced dex, strip only signatures, keep `META-INF/services/`, **write a 4-byte-aligned archive** (`resources.arsc` STORED+aligned, which Android R+ refuses to install without), sign, and verify. Also handles **split APK / App Bundle sets**: inventory one (`--split-dir`/`--split`), sign every member with a single keystore for `pm install-multiple` (`--split-mode resign`), or fold code/native members into one standalone APK (`--split-mode merge` — refused by design when a member carries its own `resources.arsc`). `--signer auto/jar/apksigner` covers hosts without uber-apk-signer; see `references/split-apk.md` |
| `scripts/dexpatch/` | dexlib2 method-level rewriter (for changes that genuinely need new instructions) + build notes |
| `scripts/devsh.py` | Quoting-safe ADB shell helper for rooted devices |
| `scripts/usb_net_proxy.py` | Give an offline device network over USB (adb reverse + local proxy) |
@@ -638,3 +652,7 @@ All scripts are parameterized and path-agnostic; pass paths explicitly. Run `--h
| `scripts/stalker_trace.js` | Instruction-level tracing with Frida Stalker: configurable module/offset targets, trigger selection, the event stream, output-size rules, and `transform` customisation |
| `scripts/stalker_report.py` | Reduce a `stalker_trace.js` log into block histograms and call sequences, and print an explicit diagnostic for the measured **zero-event** case |
| `scripts/mt_mcp_probe.py` | Probe MT Manager's on-device APK MCP (Streamable HTTP, `127.0.0.1:8787/mcp`): JSON-RPC handshake plus grouped `mt_apk_*` tool inventory; prints start-it-by-hand instructions and exits 2 while the service is down |
| `scripts/java2c_probe.py` | Collect the evidence that separates **Java2C** from an extraction shell, a VMP and ordinary JNI sinking: per-class `native` density, native-dominated classes and stub ratio from the dex, plus `Java_*` / `JNI_OnLoad` / dynamic-registration / toolchain-string evidence from the `.so`. Every item carries a strong/medium/weak label and the weak ones print their own warning — reading a single criterion is how this gets misdiagnosed |
| `scripts/protobuf_decode_raw.py` | **Schema-free protobuf decode**: hex string, binary file or stdin to a JSON tree, with every length-delimited field reported as a candidate set (nested message / UTF-8 / packed array / opaque bytes) and ties labelled rather than guessed. Flags the proto3 explicit-zero and packed-boundary ambiguities; `--max-depth`, `--split`, and `--reencode --check` for a byte-exact round trip. Cross-checked field-for-field against the official runtime |
| `scripts/vmp_diff_harness.py` | Differential hardening for Dex-VMP: build a labelled opcode-coverage fixture (**218 of 224 opcodes, measured**), derive a candidate private-opcode map from an original/hardened dex pair with per-entry confidence and a run-level verdict, forge known-table fixtures to verify the comparison in a closed loop, and render a restored stream as a smali skeleton. The upload-to-a-hardening-service link is **not automatable** — see `references/vmp-differential-analysis.md` §2 |
| `scripts/kernelsu_syscall_mask.py` | Generate a configurable KernelSU/APatch syscall-masking scaffold: an installable userspace module skeleton plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label. **A KernelSU userspace module cannot change a syscall return value** — the kernel-side templates are the part that could, and they are shipped unbuilt |
skills/apk-reverse/references/advanced-unpacking.md
+125 -18
View File
@@ -14,11 +14,13 @@ extraction shell and no VMP, so nothing here was exercised end to end against a
hardened sample by this skill's own verification pass. The FART/Youpk mechanisms and
the Android 12-16 failure analysis are **inferred** from public work (FART/Youpk
release notes and a 2026-08 Kanxue thread, `thread-292312`, on why classic
active-invocation hooks die on modern ART). The one **measured** element is the
detection metric: the trivial-body statistic was exercised by
`scripts/dex_dump_validate.py` against a surgical fixture derived from a real hardened
sample (see `docs/tool-verification/EXTENSION-unpacking.md`). Label your own results
the same way.
active-invocation hooks die on modern ART). The detection metric has since been
**measured against a purpose-built extraction-shell fixture set**, and that pass
invalidated the threshold this file used to state — see the calibration table in
§Measuring extraction instead of guessing and the commands behind it in
`docs/tool-verification/EXTENSION-extraction-shell-bench.md`. The older surgical fixture
that first exercised the script is in `docs/tool-verification/EXTENSION-unpacking.md`.
Label your own results the same way.
Two parts of this file have since been exercised against live hardened targets on a
rooted device. The **root-side memory route** (§Dumping when frida is refused) was run
@@ -28,7 +30,9 @@ traps and the throughput numbers are recorded in
`docs/tool-verification/EXTENSION-rootdump.md`. The `frida-dexdump` refusal that
motivates that section is in `docs/tool-verification/EXTENSION-device-run.md`. The
FART/active-invocation half of the file is still **inferred** — no extraction-shell
target has been spliced end to end here.
target has been spliced end to end here, and the two candidate samples shipped in
`repos/CyReverse` turned out not to be extraction shells at all (their two "shell"
assets differ by six bytes; row B3 in the bench evidence file).
## The four shapes a dump can be
@@ -39,9 +43,18 @@ directory and read the `stub%` column — the four shapes separate on it.
|---|---|---|---|
| Parses, many classes, real bodies | near-baseline | First-generation landing shell; the whole-dex dump **is** the original | Done — filter (`recon.md` §Unpacking a dex-level packer) and proceed to patching |
| Parses, many classes, most bodies are `return-void` stubs or nop fills | high | **Extraction shell skeleton** — bodies are decrypted only on invocation | FART loop, this file |
| Whole classes appear as bare `native` declarations | n/a | dex2c / JNI sink — the code left the dex entirely | `code-virtualization-and-custom-linkers.md` |
| Whole classes appear as bare `native` declarations, and the matching `Java_*` export exists | n/a | **JNI sinking** — the code left the dex for the `.so` | `java2c-and-jni-sinking.md` — read the function in the library, **not** this file |
| Method bodies look like a thin forwarder to a native routine, or a single native island holds the app's own logic | n/a | **Java2C / native island** — the body is a compiled `J`/`.so` function, so re-reading the dex finds only the call | `java2c-and-jni-sinking.md` — the dex is the wrong instrument |
| Whole classes appear as bare `native` declarations, and **no** `Java_*` export exists | n/a | Dynamic-registration native sink — the binding table is built at runtime | `java2c-and-jni-sinking.md` §the symbol search fails structurally |
| Parses, bodies present but instruction streams are nonsense to a dalvik disassembler | baseline | **Real Dex VMP** — private opcodes behind a native interpreter | Last section of this file, and be honest about the cost |
The two `native`-looking rows are **not** one row. They are separated because the
correct next action differs: JNI sinking keeps the logic in a named, exported ARM64
function that you read from the library, while a dynamic-registration sink has no
symbol to find and needs the binding table read instead. Lumping them together sends an
analyst to dump memory for code that is sitting in an ordinary `.so` — the misdiagnosis
`java2c-and-jni-sinking.md` was written to stop.
Two supporting signals worth recording while you are here (the second is measured on
this repo's hardened sample): a shipped shell `classes.dex` can be megabytes in size
yet define only a handful of classes — the bytes are payload, not code, so class count
@@ -69,12 +82,67 @@ measured to differ in ~73% of their bytes), rejects structurally impossible imag
(bad magic, `file_size` mismatch, ids out of range), counts classes and stub bodies,
and ranks which surviving image is the most likely original.
Calibration, **inferred** from field reports rather than measured across a population:
ordinary app code carries a few percent of genuinely trivial bodies (constructors that
do nothing, interface stubs); a ratio in the tens of percent or higher is a skeleton,
not code. A ratio that is high *and* the image parses is the extraction-shell
signature — a first-generation dump of the same app would show the same classes with
real bodies.
**Calibration — measured, and it is not a threshold.** Do not look for a `stub%` number
to compare against. On a real 982 KB application dex (5,061 bodies) every emptying
shape the shell family uses was built into a fixture and measured against a
zero-change control (`docs/tool-verification/EXTENSION-extraction-shell-bench.md`,
which carries the commands and the full matrix):
| What the shell left behind | `stub%` | vs. control (1.9 %) |
|---|---|---|
| bare `return-void` in every body | 100.0 % | sighted |
| body cut to `const/4 v0,#0; return v0` | 93.2 % | sighted |
| nop fill, no return — **the slot-clearing shape** | 0.0 % | **blind, and scores below the untouched original** |
| `new RuntimeException; throw` stub — **the 360/legu shape** | 1.9 % | **blind, indistinguishable from the control** |
| 25 / 50 / 75 % of bodies emptied | 1.7 / 1.7 / 2.1 % | blind |
The distribution is **bimodal**: `stub%` lands either at the app's own baseline
(ordinary constructors and interface stubs, measured at 1.9 % here) or at ~100 %.
Nothing sits in between, so there is no band a threshold could live in, and **partial
extraction is invisible** — a dex with three quarters of its bodies removed reported
0.2 points above its untouched control.
Two consequences the metric's name hides:
- **`stub%` high is dispositive; `stub%` low is not evidence of anything.** A low
reading is compatible with a genuine dex, a fully nop-wiped skeleton, and a
`throw`-stubbed skeleton alike.
- **Read `emptied%`, not `stub%`, for the skeleton call.** `emptied%` unions the
`return*`-stub and the nop-wiped classes, which is what lifts the nop-filled
skeleton from 0.0 % to 100 %. A `throw`-stubbed skeleton still reads low; for that
shape the ranking cannot help you and only per-body inspection can.
The trivial-body shapes `dex_dump_validate.py` counts, and which of them are skeleton
evidence, are documented in the script itself (`classify_body`); the split between
`stub` / `erased` / `minimal` exists because collapsing them produced a ranking that
named a fully-wiped image the most likely original.
### The ranking's failure conditions — read these before trusting its winner
The `ranking` block names a "best-supported candidate for the original", and it is
wrong in two measured situations. Both were reproduced, and both are the *opposite* of
the obvious reading:
- **A partially extracted image outranks a heavily stubbed one.** Ranking on the
emptied share alone put a dex with 25 % of its bodies removed *ahead of* its own
untouched control — 1.7 % against 1.9 %, because replacing a body with
`const/4 + return` lowers the `return-void` count. Measured before it was fixed;
the current key uses the count of skeleton-shaped bodies first, which is monotone,
and evicts anything over 50 % emptied.
- **A `throw`-stub skeleton is invisible and ranks at the control's own baseline.**
Every body replaced by `new RuntimeException; throw` measured 1.9 % — identical to
the untouched image — and it still sorts third in the fixture set. No body statistic
in this kit sees that shape. If the target family is 360/legu-like, **do not use the
ranking to pick a baseline**; diff candidate images against each other by body hash
instead.
The reason to state this in the reference file rather than only in the script: the
ranking's output is a *sentence with a winner in it*, and a wrong winner is worse than
no winner — an agent that adopts a modified image as its baseline builds every
subsequent diff on the wrong artifact. When `emptied%` is at the app's own baseline and
you cannot name a signal that separates the candidates, treat the ranking as
uninformative for that set. Commands and full matrix:
`docs/tool-verification/EXTENSION-extraction-shell-bench.md`.
## The FART loop: skeleton, invocation, splice
@@ -319,9 +387,46 @@ The last shape is the one to say out loud: if the dump's method bodies are intac
decode as private opcodes — rare-opcode density off the charts, unknown opcode slots,
and a large native interpreter loop behind them — the code has been virtualized, and
nothing in this repository will hand it back to you decompiled. Distinguish it from
the JNI sink first (`code-virtualization-and-custom-linkers.md`): there the methods
are `native` stubs and the logic lives as native code; here the bodies are present
and *re-encoded*.
the JNI sink first (`code-virtualization-and-custom-linkers.md`,
`java2c-and-jni-sinking.md`): there the methods are `native` stubs and the logic lives
as native code; here the bodies are present and *re-encoded*.
### Establish "the bodies do not decode" with a decoder you did not write
This is the one measurement in this file that has already been got wrong here, so it
gets its own rule. A hand-written opcode-width table silently desynchronises on large
dex files — one arm64 branch opcode missing from the table is enough — and the
desynchronisation is indistinguishable from the finding it fakes. Measured on the
`ezAndroid` sample: a workbench table reported **1,393 of 22,424 bodies as
non-decodable** and I read it as private opcodes; the platform decoder returned
**943,223 lines, 34,566 instruction decodes, and zero structural errors** on the same
file (`docs/tool-verification/EXTENSION-extraction-shell-bench.md` §3.4). The bodies
were ordinary dalvik the whole time, and one of the "desync" methods disassembles into
five clean instructions under `dexdump`.
So: **before any VMP verdict, decode the same file with `dexdump`,
`baksmali`/`smali`, or `jadx`, and quote the failure count that came from *it*.** A
rare-opcode histogram is a lead, not evidence. Never let a private table's desync
inform a classification — "my decoder stopped" is a fact about the decoder.
### What a static dex probe can and cannot decide
Two different questions live in the same dump, and only the first one is answerable
from the bytes on disk:
| Question | Statically decidable? | Why |
|---|---|---|
| Are the bodies real dalvik, or an encoded program? | **yes** | the platform decoder decodes them or reports a structural error |
| Did the code leave the dex for native? | **yes** | `native` access flags, then `Java_*` exports / `JNI_OnLoad` in the matching `.so` |
| Does a native island check signature, root or debugger? | **no** | the logic is ARM64 behind a vtable; the check may exist in no dex at all |
| Are unlabelled `assets/` blobs live payload? | **no** | an unreferenced blob is dead until a runtime path constructs its name — measured: six such files in `ezAndroid`'s `assets/`, **zero** dex strings referencing them |
The trap is the asymmetry. A clean static reading licenses exactly one sentence —
*"this is not dex-VMP"* — and it does **not** license *"this is not hardened"*. Those
are different claims: the second one is the misdiagnosis that sends an analyst looking
for an unpacker when the problem is a native island, or for a private disassembler when
the code is ordinary ARM64 in a 206 KB `.so`. A static probe's silence about hardening
is silence, not a clean bill of health.
What recovery looks like when someone does it, **all inferred**, no fixture exists in
this repo, treat as a research plan rather than a recipe:
@@ -348,10 +453,12 @@ for the island itself.
|---|---|---|
| Dump parses, stub% near baseline | Filter, verify with jadx, move on to patching | `recon.md` |
| stub% high, image parses | FART loop: skeleton + active invocation + splice | this file |
| `emptied%` high but `stub%` low | nop-wiped skeleton; the old `stub%`-only reading called it the original | §Measuring extraction instead of guessing |
| The ranking names a winner but `emptied%` sits at the app's baseline | Treat the ranking as uninformative for this set; do not adopt that image as a patch baseline | §The ranking's failure conditions |
| Splice produces a half-parsing dex | Re-check code_item length (padding, handler list) | §The code_item length trap |
| Dumper runs, output dir empty | Storage path + `open()` check, then entry-type probe | §Why the classic hooks die on Android 12-16 |
| App dies under the invoker | Shell counter-measures; narrow the force list | §Why the classic hooks die on Android 12-16, and `detection-and-anti-analysis.md` |
| frida attach refused (`script has been destroyed`) | Root-side `/proc/<pid>/mem` dump, atomic inside one process lifetime | §Dumping when frida is refused |
| Sample self-exits and restarts with no instrumenter present | The packer's own check, not your hook; shorten every step to one lifetime | §Dumping when frida is refused |
| Methods are `native` stubs | dex2c / JNI sink | `code-virtualization-and-custom-linkers.md` |
| Bodies present, decode as nonsense | Real VMP — differential oracle or walk away | §The honest boundary: real Dex VMP |
| Methods are `native` stubs with a `Java_*` export | JNI sinking — read the function in the `.so` | `java2c-and-jni-sinking.md` |
| Bodies present, decode as nonsense | Real VMP — **but confirm with `dexdump` first**, then differential oracle or walk away | §Establish "the bodies do not decode" with a decoder you did not write |
skills/apk-reverse/references/emulation-and-rpc.md
+28
View File
@@ -47,6 +47,7 @@ Emulation and RPC are for when the code that matters must keep *running*.
| Maintenance cost | every missing JNI/syscall is a new stub | reconnect logic, process lifecycle |
| Reproducibility | deterministic (you control time/random) | real time, real randomness |
| Best for | signing algorithms, crypto, parsers | anything entangled with live app state |
| **Priority when either could work** | second — the environment bill below is real and routinely underestimated | **first** — the device already owns the environment |
The two-strike rule applies across the table: if a library refuses to run under unidbg after two
focused rounds of `DalvikVM` patching, stop emulating and move to RPC (or the reverse) instead of a
@@ -54,6 +55,33 @@ third round — a library that checks its own loading path or decrypts itself ag
may simply not be worth emulating, and that is a finding, not a failure (`SKILL.md`
§Stop conditions).
### The environment bill — budget days, not hours
"Filling the environment" reads like a checklist and behaves like a project. The mistake this
section prevents is starting an emulation because the table says *works against anti-injection*, and
then discovering that this library's environment includes everything emulation cannot fake: an
Android `Context` backed by a real package manager, Binder round-trips into another process, a
`KeyStore` attestation that only succeeds on real hardware, or a self-check against device state
captured at install time. For a commercial native algorithm of that shape, "stub it until it runs"
is measured in **days, not hours** — each missing piece is discovered one fault at a time, and the
rounds do not get shorter.
Weigh it before starting:
| Question | If yes |
|---|---|
| Does it call through `Context` (package name, files dir, signature, `PackageManager`)? | a stub is often enough — cheap, keep going |
| Does it talk to another process (Binder service, bound SDK, remote provider)? | the stub surface grows fast; **prefer RPC** |
| Does it verify hardware (KeyStore, TEE, an attestation chain)? | emulation is likely a dead end — the value it wants cannot be produced here |
| Does it check its own loading environment (paths, maps, root, debugger, installer)? | readable and patchable *inside* the emulator, but budget one round per check |
| Must it be called thousands of times a minute? | the RPC throughput ceiling is real — this is the case that justifies the bill |
Default to **RPC first, emulation second**: the device already provides the environment, and
per-call cost only starts to matter once you need volume. The two claims in this subsection
(the day-scale cost of a Context/Binder/KeyStore-shaped library, and the RPC throughput ceiling)
are **inferred** from the mechanism and from community practice — this repository has not emulated a
commercial sample end to end, as `EXTENSION-emulation-rpc.md` states.
## Part A — Unidbg
### What it is
skills/apk-reverse/references/java2c-and-jni-sinking.md
+149
View File
@@ -0,0 +1,149 @@
# Java2C and JNI Sinking — the dex is not where the code is
Load this when the dex shows method bodies missing or replaced by `native` declarations, and
you are about to go looking for a decrypted DEX in memory. **Decide which of the two shapes you
have before you spend anything**, because one of them has no DEX to find at any point in the
process lifetime, and the search for it is unbounded.
Two neighbouring routes have historically folded this shape into one: `advanced-unpacking.md`'s
dump-shape table and the SKILL.md symptom index both point "whole classes are bare `native`
declarations" at `code-virtualization-and-custom-linkers.md`. That file is about protection which
leaves the code **reachable** — a private container, a loader, a private opcode interpreter, all of
which you can dump and measure. **Java2C is not reachable, because it was never bytecode.** This
file splits that row and owns both halves of it; the virtualization file stays what it is.
**Strength note, read this first.** The measurements below are **observed** — every number was
produced by `scripts/java2c_probe.py` against a real sample during this pass, recorded in
`docs/tool-verification/EXTENSION-java2c.md`. But no Java2C library exists in this repository and
none could be built here (no NDK, no clang, WSL unavailable — see that record), so **the
Java2C-specific identification criteria are inferred, not observed end to end**. What *is*
observed is the discriminating measurement that separates Java2C from JNI sinking, and the JNI
boundary behaviour. Treat the type table as a map; label your own conclusions the same way.
## The five shapes — one table, because the wrong row costs days
| Shape | Static shape in dex | What is in memory at runtime | Route | Cost of the wrong row |
|---|---|---|---|---|
| **Landing shell** | `Application` is a third-party class; dex is a small stub | A full, decrypted DEX | Dump memory, filter, patch | Cheap if you notice the stub `Application` |
| **Extraction shell** | Reads as a normal dex until you measure it; bodies are stubs | A DEX whose bodies are filled in **on invocation** | Dump, measure `stub%`, FART-style active invocation | Medium — you keep dumping, and every dump is a skeleton |
| **VMP** | Dex parses; bodies are *present* but decode as nonsense | A DEX containing **private opcodes** interpreted by a native VM | Measure, then usually stop and report | Medium — recovery cost usually exceeds task value |
| **Java2C** | Whole classes are `native`; **no code_item at all** for them | **Never a DEX.** The translated methods exist only as compiled C in a `.so` | Read the `.so`; rebuild the JNI call chain | **Highest in this table** — you hunt a decrypted DEX that is never produced, then blame your dump tooling, then your anti-detection assumptions |
| **JNI sinking** | A handful of `native` declarations; the rest of the dex is ordinary Java | A normal DEX | Locate the Java call site; reverse one or a few native functions | Low — but you may widen the job to the whole app if you read it as Java2C |
The one question that decides the table: **does the artifact ever hold dex bytecode for the
methods you care about?**
- Landing shell, extraction shell, VMP: **yes** — the bytecode is there, in some state, and the
work is recovering or interpreting it.
- Java2C: **no** — a translated method is a `native` declaration plus an entry in a registration
table. Nothing is decrypted, so nothing can be dumped. `[inferred]` as a runtime claim: it
follows from the translation mechanism, and this pass did not observe a live Java2C app.
- JNI sinking: **yes** — the dex was never damaged.
## Identification
### The discriminating measurement is native density
This is the single most useful number and it separates the two shapes by roughly three orders of
magnitude. Measured on real samples with `scripts/java2c_probe.py` `[observed]`:
| Sample | dex methods | `native` methods | native ratio |
|---|---|---|---|
| JNI-sinking target A (MASTG L2) | 5081 | 2 | **0.04 %** |
| JNI-sinking target B (MASTG L3) | 11182 | 3 | **0.03 %** |
| App carrying a real JNI library | 23603 | 7 | **0.03 %** |
| Java2C-shaped fixture | 29 | 24 | **82.76 %** |
A JNI *sink* is by definition surgical: a few hot methods are moved out and the rest of the app
stays in Java. **A JNI sink is not "the app is native".** When the density is in the tens of
percent, and whole classes have zero Java methods left, you are looking at a translation pass.
`[observed]` in the same runs: two of the fixture's three classes were native-dominated (>= 70 %
of >= 3 named methods), against **zero** such classes in any of the three real targets. Note that
constructors are excluded from that test — a translation pass leaves `<init>`/`<clinit>` in Java
in practice, so "every method native" is an essentially unreachable test and should not be used.
### Native-layer signatures
| Evidence | Strength | Why |
|---|---|---|
| `.so` exports `Java_*` symbols whose count is roughly 1:1 with the dex native-method count | **strong** | Measured 2:2 (sample A) and 3:3 (sample B) `[observed]`. It is the signature of a *static-linkage* translation or sink. Note it does **not** distinguish Java2C from JNI sinking — only the density does |
| The `.so` contains one C function per Java method, each beginning `JNIEnv *env, jobject thiz, ...` | **strong** | Observed in generated, uncompiled output: every translated method became exactly one function with those leading parameters |
| Toolchain strings: `Dex2C`, `dynamic_register_compile_methods`, `ScopedLocalRef`, `well_known_classes` | **strong** | These are the runtime header and entry-point names a Dex-to-C toolchain emits. `[observed]` in generated sources; `[inferred]` that they survive compilation into the shipped `.so` |
| `JNI_OnLoad` present | **weak** | Present in nearly every JNI library. Measured true on a system library with zero `Java_*` symbols `[observed]` — it told us registration was dynamic, not that anything was hardened |
| `RegisterNatives` **string** anywhere in the `.so` | **weak** | See the boundary section: NDK's C++ headers do not emit this symbol at all |
| `libc++_shared.so` / `c++_static` | **weak** | An ordinary NDK C++ setting. The Dex-to-C toolchain read here does ship `APP_STL := c++_static`, but so do countless unrelated libraries `[observed]` |
| `classes.dex` byte-count per class far below the app's own pre-hardening build | **weak, uncalibrated** | Bodies leaving the dex should shrink it. Measured 584 B/class on the fixture versus 1152–1386 B/class on the real Java targets `[observed]` — but there is no population baseline for "normal", so this only compares an app against **itself** before and after |
`java2c_probe.py` prints every one of these with its strength attached, and prints the weak ones
as an explicit warning block. A weak hit is not a verdict and must not be reported as one.
## The route: read the .so, not memory
For Java2C the analyst's instinct — dump memory and find the dex — is not merely expensive, it is
**unbounded**, because the success condition can never occur.
1. **Do not dump.** There is no target artifact. If you dump anyway, you will recover a dex that
still shows the same `native` declarations you started with, and the natural (wrong) reading is
"the dump failed".
2. **List the translated functions.** With static linkage, `Java_<PKG>_<Class>_<method>__<proto>`
names map the `.so` back onto the dex one for one. Use the dex as the index and the `.so` as
the library.
3. **Rebuild the chain backwards.** A translated function calls *out* to Java constantly. The
strings `FindClass` / `GetMethodID` / `GetStaticMethodID` / `Call*Method` / `NewObject` /
`GetFieldID` inside it are the reverse edges. Following them reconstructs what the original
Java method did, which is the only way the logic comes back.
4. **Expect a register-machine shape.** Generated code declares a slot per dex register up front
and assigns them in sequence. Reading it as if a human wrote it wastes hours; read it as a
linearizer output, matching each call site against the JNI signature it passes.
The decompiler choice matters less here than for obfuscated code: the generated functions are
long but flat and unoptimized, which is precisely the shape a decompiler handles well.
## The JNI boundary — why a symbol search fails silently
Two independent mechanisms erase the symbol that a reader would search for. Both are real; the
second was measured this pass.
**1. Dynamic registration.** Nothing named `Java_*` is exported. Binding is performed at runtime
by a registration table built in a JNI entry point. A grep over the symbol table returns nothing,
exits zero, and looks like a legitimate answer.
**2. `-fvisibility=hidden`, and an inline `RegisterNatives`.** The Dex-to-C toolchain read here
ships `APP_CPPFLAGS += -fvisibility=hidden` in its `Application.mk` `[observed]`, which hides the
generated functions from the dynamic symbol table — so even a statically-named translation can
present as "no `Java_*` symbols". Compounding it: NDK's C++ `jni.h` implements `RegisterNatives`
as an **inline member that calls through the function table**, so calling it emits *no*
`RegisterNatives` symbol either `[observed]`: a real system library exporting `JNI_OnLoad` and
zero `Java_*` symbols contains zero literal `RegisterNatives` symbols. A rule that greps for that
name is structurally incapable of firing.
**The check that does work** `[observed]`: a library that **exports a JNI entry point and exports
zero `Java_*` symbols** is registering dynamically. That combination is affirmative evidence, and
it fired on a real library in this pass.
**Confirming it at runtime.** Hook the registration entry point (`JNI_OnLoad`) and read the table
it builds, or hook the class-load path and enumerate the class's methods' native backing. Do not
conclude "no JNI boundary" from an empty symbol table; conclude "the binding is not visible in the
symbol table" and go find where it happens.
## Failure modes
| What you see | What it is not | What it is |
|---|---|---|
| Whole classes are `native`, you dump memory and find the same `native` declarations | "The dump failed" / "the tool is broken" | The translation moved the code out of dex permanently. Re-read the dex shape and open the `.so` |
| `grep Java_ .dynsym` returns nothing | "There is no native implementation" | Dynamic registration, or hidden visibility — see the boundary section |
| A library exports `JNI_OnLoad` and nothing else you recognise | "It is a loader" | It may be an ordinary dynamically-registering JNI library. A loader normally also has an `init_array` entry and a payload to map |
| Native density is high but every class still has Java methods | Java2C | Could be a JNI sink applied widely, or an R8-stripped app. Check whether the `native` methods have matching `Java_*` symbols before concluding |
| The dex looks completely unremarkable on a sample you were told is hardened | "Nothing is hardened" | A VMP whose payload is decrypted at runtime has an ordinary static dex. Measured this pass: a VMP-labelled sample had 7 native methods out of 23603 and a 4 % stub ratio `[observed]`. **Static dex metrics cannot see runtime-only hardening** — that needs a runtime check, not more reading |
## Where this file stops
- **It does not decompile the generated C.** That is ordinary native reversing; `native-and-so.md`
and `native-dbi-and-deobfuscation.md` own it.
- **It does not recover a VMP.** Private-opcode interpretation is a different mechanism with a
different cost model (`advanced-unpacking.md`).
- **The Java2C runtime claim is inferred.** That no dex bytecode exists in memory for a translated
method follows from the mechanism, not from a measurement made here. If you get a live Java2C
sample, that is the first thing to verify, and it is cheap: attach, force the class to load, and
ask whether the method resolves to Java bytecode or to a registered native entry.
skills/apk-reverse/references/kernel-and-environment-hardening.md
+90 -1
View File
@@ -19,6 +19,13 @@ externally documented mechanisms with sources, labelled `inferred` — none of i
That is deliberate: the purpose is to stop you from spending a day rediscovering the version
gate (§4) or the conflict (§5), not to teach kernel hacking.
**Extension pass, read the next paragraph before relying on it.** This file now has a companion
generator — `scripts/kernelsu_syscall_mask.py` — which emits a loadable userspace module skeleton
plus kernel-side **templates** for the three routes in §4. The generator, its userspace output and
its own consistency check are `measured`; **the kernel-side code has never been compiled or loaded
anywhere**, and every generated kernel file says so in its own header. A template is a starting
point that removes the blank page, not a weapon that has been fired.
## 1. The attack-surface timeline — why "just hook it" fails
An app process initializes in a fixed order, and every anti-analysis check you will meet lives at
@@ -147,6 +154,83 @@ the process earlier than any userspace tool can be, needs no ptrace (so it does
rewrites one `/proc` read is a research project; a Zygisk module that hooks the target's
constructor is a build task.
### 4a. From "no weapon" to "template + gate": what the extension pass added
The gap this closes is specific. §4 above was a map with no artefact behind it: a reader told
"Kernel module hooking — inline hooks and syscall-table hooks in kernel space — the layer that *can*
rewrite what a `/proc` read returns" had nowhere to go next. `scripts/kernelsu_syscall_mask.py`
produces the artifact; this subsection states its boundary, because the boundary is the part that
matters.
**The correction worth internalising first: a KernelSU module cannot do any of this.** A KernelSU
(or Magisk, or APatch-userspace) module is a *userspace* module whose scripts run as root in the
normal world. `module.prop`, `post-fs-data.sh` and `service.sh` cannot change what `openat`
returns — not because of a version gate, but because nothing in that format is ever in the kernel's
return path. Detectors of the kind §2 and §3 discuss are not defeated by a module of that shape.
The generator emits that skeleton anyway, because it is the right carrier for the configuration and
the metadata, and it says in its own README what the skeleton cannot do.
What *can* reach the return path is one of three artifacts, and each carries a gate that has to be
checked on the actual device:
| Artifact | What it is | Gate | Status of the shipped template |
|---|---|---|---|
| **KPM** (KernelPatch / APatch) | A relocatable `.kpm` loaded by kpimg injected into the kernel image; replaces syscall-table pointers (`fp_hook_syscalln`) or rewrites prologues (`hook_wrapN`) | A KernelPatch-patched boot image, plus a **bare-metal** ARM64 toolchain (`aarch64-none-elf-gcc`) — not the NDK | `unverified` — template only, never compiled |
| **Out-of-tree LKM** | An ordinary kernel module that reaches `sys_call_table` and swaps a pointer | Kernel source matching the device's exact vermagic, and on ≥5.7 a way around `kallsyms_lookup_name` no longer being exported | `unverified` — template only; rated last of the three on cost |
| **eBPF probe** | A BPF program on a syscall tracepoint or kprobe | GKI 5.10+ with the tracing/BTF machinery | `unverified` — template only; the version gate alone closes it on most older devices |
The eBPF row carries a capability ceiling that is easy to miss and is stated in the generated file
as well: **a tracepoint can observe a syscall, not rewrite its result.** `bpf_override_return()`
only applies to functions flagged `ALLOW_ERROR_INJECTION`, which raw syscall entries are not —
so "eBPF to spoof a `/proc` read" is, on most kernels, an observation plan wearing a rewrite
plan's clothes. Observation is genuinely useful (it tells you which syscall the check uses, which
`§5` requires you to know anyway); it is not spoofing.
**Field notes on the KPM route, from a public KPM development write-up** (`inferred` here — the
author's measurements, not this repository's; source: blackr0ck, *APatch KPM 开发*,
[bbs.kanxue.com/thread-291665.htm](https://bbs.kanxue.com/thread-291665.htm), 2026-06). These are
recorded because each one is a day of somebody's time:
- **An inline hook can install cleanly and never fire.** On a GKI kernel with LTO, the exported
symbol is frequently not the call site — the callee was inlined into its only caller, so
replacing instructions at the symbol's address intercepts nothing. The reported symptom is
"module loaded, callback never ran". Pointer replacement in the syscall table does not have this
failure mode, because the syscall entry path must go through the table.
- **The instruction-sequence length in a module header is a trap of its own.** The reported cause
of a hard-to-diagnose load failure: declaring a resolved kernel function with `extern` makes the
compiler emit an undefined reference (`*UND*`) instead of allocating the slot, and the loader
refuses the module with `unknown symbol`. Letting it be a tentative definition allocates `.bss`
and the loader fills it in. The generated template states this rule in the code, and
`verify` checks for it.
- **Per-syscall callbacks run on every syscall of that number, system-wide.** The reported
incident: a `write` hook that did not first reject `fd <= 2` intercepted the framework's own log
output tens of thousands of times and the device rebooted. **Performance is correctness here** —
the first statement in every callback is a filter that rejects the common case.
- **Which hook point survives is kernel-specific, and the cheap way to find out is to start at the
syscall layer.** The same write-up reports that modifying the SELinux internal function they
first targeted crashed the kernel under every combination tried (before/after, argument,
return value, skip-origin), while a syscall-table hook on `write` proved stable and sufficient.
Treat the syscall layer as the default and move inward only with evidence.
**Measured on the reference device, and why every template here is labelled the way it is.**
`<DEVICE>` runs kernel **4.14.186+** (`adb shell 'uname -r; cat /proc/version'` →
`4.14.186+`, `Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902)
clang version 11.0.1 ...) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022`). On that device the two
remaining gates answer themselves:
- **eBPF**: 4.14 is far below the 5.10 gate. Closed, as §4 already recorded.
- **KPM**: no KernelPatch-patched image is present, and the host has no
`aarch64-none-elf-gcc` (`kernelsu_syscall_mask.py gates` reports `NOT FOUND`, along with no
`ndk-build` and no `make`). Installing APatch means patching the boot image — a bricking risk
that this repository does not take on an APK task.
- **LKM**: no kernel source for the device, so no matching vermagic is possible.
So on this machine the honest answer to "do you have a kernel-level weapon" is: **you have a
template, a gate table and a measured statement that the gate is closed here.** That is a route
decision, not a failure — and it is the same shape of answer §6 gives for the whole escalation
ladder. Do not let the existence of the template change what you claim about it: `unverified` is
still `unverified`.
## 5. The syscall-level realities that survive every hook
Two facts decide a lot of "unexplainable" behaviour, and both belong to the kernel's design
@@ -178,7 +262,7 @@ honest outcome (`observed`, from `tools/_phone-modules/README-*.md`):
| LSPosed (Zygisk) module hooking | available — framework activation verified by log (`welcome to LSPosed!` lines, `lspd` daemon process) |
| MT Manager on-device editing/repack/sign | available (`on-device-tooling.md`) |
| eBPF tracing | closed (kernel gate) |
| KPM / kernel module route | closed (no kernel source/build for the device; out of scope regardless) |
| KPM / kernel module route | closed on this device — no KernelPatch image, no kernel source, no bare-metal ARM64 toolchain; the **template and gate table** are shipped (§4a) |
| Frida with disguised server | available (a renamed server binary was present on the device from prior work) |
The generalisable rule: **enumerate the ladder for *your* device once, write the one-line
@@ -198,6 +282,8 @@ defeated with a custom kernel module is not a deliverable anyone can install.
| Check uses raw syscalls | Static patch of the check; or LSPosed/Zygisk module hooking the consuming code | "Hook libc open/read" (does nothing) |
| Check detects the *environment* (root/emulator), not your patch | `detection-and-anti-analysis.md` A/B/C — usually a different device or route | Kernel work |
| You are about to write a kernel module / patch a kernel | Stop. State what is blocked and the evidence; propose the static/module route | Kernel development (out of scope for this skill) |
| You have the kernel template and want to load it on a 4.14/Magisk device | Read `§4a`'s gate table, confirm the gate is closed, and say so | Patching a boot image to obtain a kernel route, on an APK task |
| A kernel-side template exists, so it feels like a weapon | It is `unverified` until a device with the matching gate compiles and loads it | Presenting a template as a working capability |
| Escalation effort exceeds the user's actual ask | `detection-and-anti-analysis.md` stop signal — switch to static | One more layer |
## Checklist
@@ -207,5 +293,8 @@ defeated with a custom kernel module is not a deliverable anyone can install.
- [ ] Root scheme's artefacts enumerated (manager/daemon/mounts) before blaming the target
- [ ] Hiding module: exactly one of Shamiko / Zygisk-Assistant; denylist configured to its spec
- [ ] Kernel rung checked against `uname -r` — one line recorded if closed
- [ ] Kernel-side template treated as `unverified` until a device with the matching gate
actually compiles and loads it (`§4a`); the userspace module skeleton is a carrier for
configuration, not a kernel capability
- [ ] Raw-syscall vs libc-call distinction verified by disassembly before building any spoof
- [ ] No kernel development undertaken as part of an APK deliverable
skills/apk-reverse/references/native-dbi-and-deobfuscation.md
+47
View File
@@ -182,6 +182,53 @@ across runs only for code that is mapped from file; do not compare a `libart`-ma
address between two runs; and never place a follow/unfollow pair on a hot function (measured failure
in §6).
### The cost is not linear, and on arm64 it decides the tool choice
Two costs decide whether Stalker is the right tool, and neither is visible in the API:
- **Translation multiplier.** A followed thread pays a large constant factor on every translated
block. Community reporting for arm64 puts it at **20-50x**; the mechanism is not controversial
(every block is copied into the DBI's code cache and routed through a dispatcher), while the exact
number is workload-dependent. This one is `inferred` — this repository has not measured the
multiplier itself, only its consequences below.
- **What you let into the trace.** Every block of every library the thread touches is a candidate.
Following into `libc` / `libart` / `libhwui` is how a four-second window becomes a device-wide
event: measured here, one such run took the test phone's `load average` to `34.11` on 8 cores and
destroyed a co-running job's attach (§6).
`Stalker.exclude()` is the control for the second cost, and it is not optional.
`scripts/stalker_trace.js` applies it before `follow()` from its `excludeModules` list and reports
what it actually excluded on one `EXCL` line — read that line before believing any trace, because a
module that was not loaded at follow time was **not** excluded. Exclude every system library you are
not studying; the target module is never excluded even if its name appears in the list.
Three failure shapes follow from ignoring this: deadlock, a watchdog `SIGABRT`, and the zero-event
trace. The deadlock and the watchdog shape remain community reports (`inferred`); the other two were
measured, **and the measurement splits the claim in two** — the split matters more than the summary:
| Arm — one device, one package, one module, 6 s follow | Outcome |
|---|---|
| attach + resume, **no follow** | process survives |
| follow, `excludeModules: []` | **process dies**, script destroyed |
| follow, 20 system modules excluded | process survives, **but `blocks=0 blk=0 calls=0`** |
Exclusion is what keeps the target alive — now `observed`, with the no-follow arm ruling out "it
would have died under frida anyway". But exclusion does **not** restore event delivery: the
zero-event trace survived the treatment arm unchanged. Treat those as two problems, and do not offer
`exclude` as the fix for a follow that delivers nothing. Commands and outputs:
`docs/tool-verification/EXTENSION-stalker-exclude.md`.
**arm64 also raises the floor.** PAC/BTI-bearing code gives a translator more ways to mis-handle a
block than armv7 did, which is one more reason a trace that works on an emulator is not evidence
about a device. Treat any arm64 result as needing a control run (a followed thread executing a known
loop, nonzero `BLK` lines) before it is interpreted.
**Emulation is a legitimate alternative, not a consolation prize.** When the target is a pure
computation inside a `.so` and its environment can be faked, a Unidbg/Unicorn trace can be both
faster and more stable than Stalker on a real arm64 device: no device load, no watchdog, no PAC, and
a deterministic replay. That inversion of the usual intuition is half the reason
`emulation-and-rpc.md` exists — decide with its decision table rather than by habit.
## 5. From a trace to a deobfuscated function (inferred)
This is the part the community write-up above also describes, and it is **inferred** here — the
skills/apk-reverse/references/protocol-reverse.md
+71 -3
View File
@@ -55,9 +55,13 @@ Three consequences that decide how you interpret any decoded body:
2. **A packed repeated field is invisible without a schema.** Field 4 above is three varints
(`3, 270, 86942`) packed into one length-delimited value. A schema-free decode can only report it
as opaque bytes — the length is known, the element boundaries are not.
3. **An explicit zero is byte-identical to an absent field** in proto3 (`28 00` versus no bytes at
all). A decoded `0` is not evidence that the sender set the field, and "the server sent 0" is not
a safe conclusion from a decode alone.
3. **A decoded `0` and a field that was never set are two different questions.** An implicit
(no-presence) proto3 field whose value is the default writes **nothing at all**, so `field = 0`
and "never touched" produce identical bytes; a field **with** presence (`proto2 optional`,
`proto3 optional`) set to `0` does put its tag and a zero byte on the wire. Both halves were
measured against the official runtime; a schema-free decoder can only flag the case, never
resolve it. So a decoded `0` was emitted on purpose by some writer, and a field *missing* from a
decode is not evidence that 0 was the value in use.
### Decoding without a schema
@@ -77,6 +81,63 @@ Three consequences that decide how you interpret any decoded body:
covers reading these, and `datastore_inject.py` can re-encode an edited DataStore value once you
know the message.
### The decoder that ships here (measured)
`scripts/protobuf_decode_raw.py` walks a payload with no schema and prints a tree. Input is a hex
string, a binary file, or stdin (hex text and raw bytes are told apart automatically); `0x` prefixes,
spaces, commas, newlines and `\xNN` escapes are all accepted, and `|` splits independent frames.
`--json` emits the same tree as data, `--max-depth` bounds the nesting, and `--reencode` writes a
decoded (possibly hand-edited) tree back to bytes and can diff the result against the original.
```
$ python skills/apk-reverse/scripts/protobuf_decode_raw.py --hex "08 96 01 12 07 74657374696e67"
frame 0 bytes 0..12 (12 B) status=end_of_buffer
f1 varint 150
f2 len-delimited(7) view=utf8_string
candidate utf8_string 0.50 valid UTF-8 with no control characters
candidate packed_varint 0.50 the payload is exactly a sequence of 7 varint(s) ...
candidate bytes 0.20 opaque bytes: always consistent with the payload ...
tie: packed_varint and utf8_string are equally consistent with these bytes ...
string: 'testing'
```
The point of the tool is the candidate list. Fed a nested message that `google.protobuf` 6.33.6
itself produced (inner payload `0801120178`), it answers:
```
nested_message 0.50 | packed_varint 0.50 | utf8_string 0.35 | bytes 0.20
tie: nested_message and packed_varint are equally consistent ... the view is a display default
```
Both readings really are legal protobuf for those six bytes, and no heuristic closes the gap. The
confidence numbers are a documented ordering, not probabilities, and the `view` exists only so the
tree can be expanded and re-encoded -- it is not a claim about what the field is.
**Round trip, which is the strongest evidence available without a schema.** Decode, re-encode, and
compare bytes: the message serialized by the official runtime re-encodes byte-identically to
`SerializeToString()`; a real AndroidX DataStore file (81 B, written by `scripts/datastore_inject.py`)
round-trips byte-identically; and editing one value in the JSON tree and re-encoding produced a file
that the *other* tool read back as the new value. Two of the built-in fixture families are exceptions
by design and are reported as such rather than hidden: a non-canonical varint re-encodes to the
minimal form, and a `--split varint-length` run compares frame bodies only, since the length prefixes
are framing and not message data.
**What a schema-free decode cannot conclude -- write none of these down:**
- that a length-delimited field *is* a string, a submessage or a packed array, rather than that it
could be any of them;
- the element boundaries of a packed array, or its element type;
- that a field's value was 0 because it is absent from the decode, or that a sender "set" a field
because a 0 appears (see the two traps above);
- which message a field number belongs to, or that the same field number in two payloads means the
same thing;
- that a payload which parses cleanly is a real message at all -- §6 has that failure mode, and the
fixes are a second sample and a round trip, not a longer look at the first one.
The measured record for this tool -- every command, both input forms, the round trip, and the
framing mistake it does *not* protect you from -- is in
`docs/tool-verification/EXTENSION-protobuf-raw.md`.
## 2. Recovering the schema from the APK
### What generated protobuf code looks like
@@ -253,6 +314,13 @@ their parents, and (c) when you have a schema, re-encoding the decoded structure
with the input — the same round-trip check used in §1, which caught a decoder bug in this
repository's own fixture.
**A framing choice that is wrong without being refused.** `--split varint-length` means protobuf's own
`writeDelimitedTo` framing: a varint length, then the message. gRPC frames a message differently — one
compression-flag byte, then a **four-byte big-endian** length (§3). Handed a real gRPC frame, the
varint splitter reported **five** frames rather than failing, while the same bytes decoded with no
splitting reported `stray_end_group`. Strip the 5-byte gRPC prefix yourself, and check the frame count
either way: a wrong split still produces a decode, which is what makes it expensive.
**Assuming the proxy sees everything.** The proxy sees what the client sends through it. An app that
uses QUIC (§4), a raw socket, a native client bypassing Java's HTTP stack, or a certificate-pinned
connection shows up as silence. Silence is evidence about the transport, not about the app: check
skills/apk-reverse/references/split-apk.md
+322
View File
@@ -0,0 +1,322 @@
# Split APKs and App Bundles
A store build is frequently **not one file**. Play and most OEM stores ship App Bundles, and the
device receives a set: `base.apk` plus one `split_config.*.apk` per ABI, screen density and language
(and, for a dynamic-feature bundle, the feature modules as well). Every member carries **the same
package name and its own signature**, and the package manager treats the set as one package.
Two failures follow from treating such a set as an APK, and both look like a broken build when they
are nothing of the kind:
- Signing only the base leaves the set disagreeing about its certificate; the installer rejects the
whole set and names the package, not the member you forgot.
- "Just merge them into one APK" is free *only* for members that carry code or native libraries. A
member that carries resources cannot be folded into the base without merging `resources.arsc`,
which is a resource-compiler job. A wrongly merged archive installs and then renders the wrong
thing, or dies on the first resource lookup.
`scripts/repack.py` handles both routes and **refuses to guess**: it inventories the set, decides
which route is legal from what the members actually carry, prints the reason, and only then acts.
**Strength labels** (same three this repository uses everywhere): **observed** = reproduced with an
exact command and output; **inferred** = follows from observed facts, step not executed;
**unverified** = reported or assumed, not independently confirmed. The measurements behind this
file are recorded in `docs/tool-verification/EXTENSION-split-apk.md`.
## 1. What the set is
| Member | Manifest `split` attribute | Carries | Foldable into the base? |
|---|---|---|---|
| base | *(absent)* | all `classes*.dex`, the main `resources.arsc`, every resource not moved out, the real `<application>` | — it *is* the base |
| ABI split | `config.arm64_v8a`, `config.armeabi_v7a`, `config.x86_64`, … | `lib/<abi>/*.so` only | **yes** — entry names do not collide |
| density split | `config.hdpi` … `config.xxxhdpi` | its own `resources.arsc` **and** `res/` entries | **no** — requires a resource merge |
| language split | `config.en`, `config.zh`, … | its own `resources.arsc` (strings) | **no** — same reason |
| feature split | `feature_x` (+ `isFeatureSplit="true"`, `configForSplit="base"`) | its own `classes*.dex`, possibly resources | **code only**; its resources hit the same wall |
**Observed** on a real 9-member set pulled from a stock Android 11 device (arm64-v8a target):
- the base carried `classes.dex` (872 KB) and `resources.arsc` (71 KB);
- the ABI member carried two `lib/arm64-v8a/*.so` files and **no `resources.arsc` at all**;
- each density member carried its own `resources.arsc` and its own `res/` entries;
- a second, 8-member set's density members carried 39-78 `res/` entries each and 9-15 KB tables.
So "which member owns the resources" is answerable **from the zip entry list alone** — no device, no
resource compiler, no decompiler. Copy a set to disk and ask.
**A member can own a nearly empty table.** A split is *required* to carry a `resources.arsc`, so a
placeholder one is normal: on the same real set, eight density members carried a **40-byte** table
and no `res/` entries at all — they are placeholders, not resources. Treating "has an arsc" as "owns
resources" would block a merge that is perfectly legal. `repack.py` uses *entries under `res/` or a
table bigger than 1 KB* as the test.
## 2. Getting the set off a device
`pm path` prints **one line per member**; a package with several lines is a split set. This is how
you find one to copy, and it is also how you tell, before anything else, that the thing you are about
to patch is not a single APK:
```bash
adb shell 'pm path <PKG>'
package:/system/priv-app/<DIR>/<APP>.apk
package:/system/priv-app/<DIR>/<APP>-arm64_v8a.apk
package:/system/priv-app/<DIR>/<APP>-xxhdpi.apk
```
To enumerate every split set on a device (toybox shell — no `--time-style`, no bash arrays):
```bash
adb shell 'for p in $(pm list packages | cut -d: -f2); do c=$(pm path $p | wc -l); \
if [ $c -gt 1 ]; then echo "$c $p"; fi; done'
```
Copy the whole directory out; the members live side by side:
```bash
adb shell 'su -c "ls -la /system/priv-app/<DIR>/"'
adb pull /system/priv-app/<DIR> <dest>/
```
Two traps, both **observed**:
- `adb pull <dir> <dest>` creates `<dest>/<dir>/` — the set arrives one level deeper than you expect.
`repack.py --split-dir` searches recursively for exactly this reason.
- A **third-party** split set is rare on a stock ROM. On the device measured here, `pm list packages -3`
returned **zero** split sets among ~40 third-party apps, while four preinstalled Google components
were split. If you need a real set and the third-party list is empty, that is the normal
distribution, not a mistake on your side.
**Do not test by reinstalling a pulled set under its own package name.** It is already installed on
that device, owned by the vendor certificate you do not have, so the install fails for a reason that
has nothing to do with your pipeline (§6). Either work on a set you own, or rename the pulled one
(§7) to get an installable fixture.
## 3. The decision: merge or resign
Run this first — it writes nothing and answers with a reason:
```bash
python skills/apk-reverse/scripts/repack.py --split-dir <set>/ --split-mode analyze
```
| Situation | Route | Why |
|---|---|---|
| Every non-base member carries only code and/or `lib/` (a placeholder `resources.arsc` is fine) | **merge** | nothing that needs a resource table moves |
| Any member carries `res/` entries or a real table | **resign** | a resource merge rewrites the table; see §5 |
| You must keep the build installable **and** the app is distributed as a bundle | **resign** | this is what the store does; the set stays a set |
| You need one file to hand someone, sideload, or feed to a tool that takes an APK | **merge**, if legal | the only way to get a single artifact |
| The delivery must survive store updates, or the ROM's own installer refuses multi-APK installs | **resign** (`pm install-multiple`), or merge and accept a fixed build | — |
| A member declares `isFeatureSplit="true"` and ships `classes*.dex` **and** resources | **resign** | its code could fold, its resources cannot; a half-merged build is worse than either route |
Failure modes, one per route:
- **merge**, when a resource-carrying member is dropped to make it legal: the app starts and shows
the base's resources. Missing density variants show up as blurry or missing images; missing
language splits show as untranslated strings. This is a **downgraded** build and has to be
delivered as one — `repack.py` prints `DROPPED n entry/ies` and refuses unless you pass
`--drop-split-resources`, so the loss cannot be silent.
- **merge**, when the base still declares `isSplitRequired="true"`: install succeeds and start is
refused (`INSTALL_FAILED_MISSING_SPLIT`, §6). The attribute has to be cleared first, and that is a
binary-AXML edit.
- **resign**, when one member is missed: the set fails as a set, with a signature error naming the
package. Sign the directory, not a list you typed by hand.
- **resign**, when the target ROM's installer is not the platform installer: `adb install-multiple`
can be intercepted by an OEM security centre, exactly as single-APK installs are
(`references/repack-and-sign.md` §Vendor install interception (OEM shells)). Use the root
`pm install-multiple` path.
## 4. Unified signing — the route that always works
```bash
python skills/apk-reverse/scripts/repack.py \
--split-dir <set>/ --split-mode resign \
--split-out-dir <signed_set>/ \
--ks key.jks --apksigner <path/to/apksigner>
```
Each member is de-signed (only the signature entries), rewritten as a 4-byte-aligned archive, and
signed with **one** keystore and **v1+v2+v3**; the run then verifies every member and prints the
signer fingerprint per file. The acceptance criterion is not "each file verifies" — it is **one
certificate across the whole set**:
```
[result] OK: 8 members, one certificate (sha256 1eb31d9c…2c44)
```
Then install the whole set, base included:
```bash
adb install-multiple -r <signed_set>/*.apk
adb shell "su -c 'pm install-multiple -r /data/local/tmp/set/*.apk'" # when an OEM installer refuses the multi-APK form
```
**The signature must cover every member**, and `--abi`/density selection happens at install time, not
at signing time — sign them all, let the installer pick (see §6).
### Toolchain: `uber-apk-signer` is optional, `apksigner` is not
`repack.py`'s original signing route went through `uber-apk-signer` only, so a machine with a full
Android build-tools directory and no uber-apk-signer could not produce a signed build at all. The
script now picks the route itself:
- `--signer jar` — the uber-apk-signer path (used automatically when the jar exists);
- `--signer apksigner` — `zipalign -p -f 4` then `apksigner sign --v1 --v2 --v3`, which is the whole
of what the jar was doing for us;
- `--signer auto` (default) — jar if present, else apksigner.
`--apksigner` accepts all three shapes a build-tools install ships: `apksigner.bat`, the `apksigner`
shell wrapper, or the bare `lib/apksigner.jar`. **On Windows pass the `.jar` or the `.bat`** — the
extensionless wrapper is a shell script and will not execute. A `.jar` value is run through
`java -jar`, so `--java` (or java on PATH) is required for it.
Order is load-bearing: **align, then sign**. `apksigner` adds signature entries without reshuffling
the archive; the v1 (JAR) path emulates `jarsigner`, whose rewrite of the zip would destroy the
alignment Android R+ requires of `resources.arsc`.
## 5. Merging into a single APK
```bash
python skills/apk-reverse/scripts/repack.py \
--split-dir <set>/ --split-mode merge --out merged.apk \
--abi arm64-v8a --ks key.jks --apksigner <path/to/apksigner>
```
What folds: `classes*.dex` (renumbered so a member's `classes.dex` becomes `classes2.dex`, keeping
the base's own dex intact), `lib/**`, `assets/**`. What does not, and why: `res/**` and
`resources.arsc`.
**Why resource merging is not attempted.** `resources.arsc` is an indexed table: a global string
pool plus type-spec and type chunks, with every resource identified by a `(package, type, entry)`
triple and every string referenced by pool offset. Folding a member's table into the base's means
rewriting pool offsets, type/entry counts and the alignment of every string, in both directions —
that is `aapt2 link`, not a byte-level edit. The failure it produces is the worst kind: the archive
is *structurally* valid, installs, and then resolves resource ids to the wrong entries or to nothing.
An APK that renders garbage is harder to diagnose than one that refuses to install.
If you genuinely need one standalone file for a bundle whose resources are split, the honest routes are:
1. **`bundletool build-apks --mode=universal`** (the vendor tool, needs the original `.aab`) — this is
what merges resources correctly, and it is the only route that does.
2. **Keep the split set** and install it as a set (§4).
3. **Accept the downgrade** with `--drop-split-resources`, and say so in the delivery.
**Manifest constraints when you do merge — and when you patch any split at all.** These attributes
are the difference between "installs and runs" and an install or start refusal, and they are easy to
miss because they live in binary AXML:
| Attribute | Lives on | Effect if you get it wrong |
|---|---|---|
| `split="config.xxhdpi"` | `<manifest>` of a **split** | Identifies the member. A split whose `package` differs from the base's, or whose `split` name is duplicated, is refused as a set |
| `configForSplit="base"` | `<manifest>` of a split | Ties a configuration split to a feature module. Changing package/module names without updating it silently detaches the split |
| `isFeatureSplit="true"` | `<manifest>` of a feature split | Marks an install-time/on-demand module. A merge that folds code in but leaves the attribute declared describes a member that no longer exists |
| `isSplitRequired="true"` | `<manifest>` of the **base** | The platform refuses to start a base that believes it is incomplete. This is the one that breaks merged builds (§3) |
Read them without a decompiler through `scripts/repack.py`'s own AXML reader (`--split-mode analyze`
prints the `split` name per member; `inspect_apk()` returns `package`, `split`, `configForSplit`,
`isSplitRequired`, `isFeatureSplit`). Changing any of them means editing the binary string pool and
the attribute chunks around it, which is what `apktool` is for — and note that a full
`apktool b` rebuild rewrites the whole archive
(`references/repack-and-sign.md` §Repacking an unpacked (de-shelled) app).
## 6. Install refusals, and what each one means
| Message | What it actually says | Attribution |
|---|---|---|
| `Failure [INSTALL_FAILED_INVALID_APK: ... signatures do not match previously installed version]` / `Package <PKG> signatures do not match` | The set you are installing is signed with a different certificate than the one already on the device | **Not** a malformed build. Either re-sign to the original key, or install to a package name that does not exist yet |
| `Failure [INSTALL_FAILED_ALREADY_EXISTS]` / `INSTALL_FAILED_UPDATE_INCOMPATIBLE` | Same package, different signer (or a downgrade without `-d`) | A pulled system component cannot be reinstalled over itself; see §7 |
| `Failure [INSTALL_FAILED_MISSING_SPLIT: Missing split for <PKG>]` | The base declares `isSplitRequired="true"` and the set is incomplete — or you merged it | The base is telling the truth: it expects members. Clear the attribute, or install the complete set |
| `Failure [INSTALL_FAILED_INVALID_APK: Failed to parse ... split ... has different signature]` | **One member** of the set is signed differently from the rest | This is the failure mode of signing only the base. Sign the whole directory |
| `Failure [-124: ... resources.arsc ... stored uncompressed and aligned on a 4-byte boundary]` | One member's `resources.arsc` is compressed or unaligned | Per-member, not per-set. `repack.py` reports it per member before signing (`references/repack-and-sign.md` §2a) |
| `Failure [-99]`, or a numeric-only code with no `INSTALL_FAILED_*` | An OEM installer interceptor rejected the request | Nothing to do with the set; use the root `pm install-multiple` path |
| `adb install-multiple` on a device that has no `install-multiple` support (very old `pm`) | Install each member with `pm install` in order, base last | Use `pm install-create` / `install-write` / `install-commit` explicitly |
`INSTALL_FAILED_INVALID_APK` in particular is a **family**, not one error: it covers stale
signatures, malformed base/split relationships, and alignment violations. Read the text after the
colon before attributing anything — the constant alone will send you looking at the wrong file.
## 7. Making an installable fixture from a pulled set
**Observed, and the reason this section exists:** a set pulled from the device is installed *on that
device*, under a name owned by the vendor certificate. Re-signed copies cannot be installed back
over it, so "does our signed set install?" cannot be answered with the pulled set as it stands.
The cheap, structure-preserving way out is an **equal-length package rename**: replace every
occurrence of the old package name in `AndroidManifest.xml`, `resources.arsc` and `classes*.dex`
with a new name of exactly the same length. Nothing moves — no length prefix, no string-pool offset,
no dex `string_ids` offset, no resource index — so the only field that must be recomputed is the dex
header's integrity pair (signature first, checksum last, see
`references/repack-and-sign.md` §Repacking an unpacked (de-shelled) app). Then run the ordinary
`--split-mode resign` over the renamed directory and install that.
A detail worth knowing before you touch a manifest: **a member's Java classes need not live under its
manifest `package` at all**. On the real set measured here, the manifest package was
`com.google.android.<name>` while all 677 class references in the dex were
`Lcom/android/<name>/...` — and every `<activity android:name>` was fully qualified. Renaming the
manifest package therefore left every component resolvable, with no dex edit needed. Assume the
opposite (that the two must match), and you will edit a dex for nothing.
## 8. ABI and density matching
The device tells you what it can run:
```bash
adb shell getprop ro.product.cpu.abilist # e.g. arm64-v8a,armeabi-v7a,armeabi
adb shell getprop ro.product.cpu.abi # the primary ABI
adb shell getprop ro.product.cpu.abilist32 # 32-bit list, on a 64-bit device
```
Match against the **split name**, not the file name, and mind the separator: split qualifiers use
**underscores** (`config.arm64_v8a`) while `lib/` directories use **hyphens** (`lib/arm64-v8a/`). The
mapping is a substitution, not a lookup — `arm64_v8a` → `arm64-v8a`, `armeabi_v7a` → `armeabi-v7a`;
`x86` and `x86_64` are unchanged. That is why `--abi arm64-v8a` matches a split named
`config.arm64_v8a`.
Four rules that decide ABI questions:
- **A 64-bit device still installs a 32-bit-only set**, and then runs it 32-bit. If the base carries
no native code and only the `armeabi_v7a` split does, the app is a 32-bit app on that device.
- **The ABI split is selected by the installer, not by `repack.py`.** `--abi` exists for *merging*
(where the choice is baked into one file); for `--split-mode resign`, sign every member and let
`pm` pick.
- **Merging without `--abi` folds in every ABI member**, which makes one APK with two or three copies
of the same library set. Legal, larger, and pointless on a single-device delivery.
- **`getprop` reports what the device claims**; what runs is a live mapping question
(`scripts/lib_map.py`). A device that reports `arm64-v8a` first can still execute a 32-bit process.
Density is the same shape without the separator trap: `config.xxhdpi` matches
`res/drawable-xxhdpi/`, and the installer picks the closest variant the device's `ro.sf.lcd_density`
can use. There is nothing to select on your side.
## 9. Traps worth paying for once
- **`resources.arsc` belongs to whoever ships it.** In a split set it exists *per member*, and the
base's table does not contain the density member's entries. Reading the base's table and concluding
"the resource exists" is wrong; it exists in one configuration only.
- **`android:extractNativeLibs`.** If the base declares `extractNativeLibs="false"` (the default for
modern targetSdk), the `.so` files are mmapped straight out of the archive, and they must be
**uncompressed and page-aligned** — this is what `zipalign -p` exists for. Merging native members
into a base and then writing the archive without that alignment produces an install that succeeds
and a start that dies in the linker. `repack.py` writes uncompressed `lib/**` 4-byte aligned, and
`zipalign -p -f 4` adds the page alignment the loader wants for them.
- **A placeholder `resources.arsc` is not a resource split** (§1). Blocking on "has an arsc" refuses
legal merges.
- **Do not `zipalign` after signing to "fix" a member.** It rewrites the archive and invalidates the
signature. Align, then sign.
- **`apktool` may not be usable even when a wrapper exists.** Measured on the pass machine: the
`apktool.bat` wrapper pointed at a jar that was not on disk
(`Error: Unable to access jarfile <missing>.jar`), while `apksigner.jar` and `zipalign.exe` in the
same layout were fine. Check the wrapper's target before planning a manifest edit around it.
- **The AXML `split` attribute is a string-pool entry**, which is why an equal-length rename of it is
safe and why a length-changing edit is not: the pool's own length prefixes and the attribute chunk
sizes around it would both need recomputing.
## 10. What this file does not claim
- **It does not merge resources.** No route here reproduces `bundletool build-apks --mode=universal`.
A bundle whose resources live in splits stays a set, or becomes a downgraded single APK by explicit
choice.
- **It does not split a monolithic APK back into modules**, and it cannot turn a `.aab` into APKs —
that is `bundletool`, and the `.aab` is the input it needs.
- **`--split-mode merge` covers code and native libraries only** (§5). Feature-split *code* folds;
feature-split *resources* do not.
- The install-failure texts in §6 are the shape the platform produces; **the exact wording varies by
Android version and by OEM installer**. Read the text after the constant, not the constant.
skills/apk-reverse/references/vmp-differential-analysis.md
+237
View File
@@ -0,0 +1,237 @@
# Differential hardening (the known-plaintext route to a private opcode table)
Load this when the dump is a **real Dex VMP** — bodies present, decoding as nonsense to a
dalvik disassembler, a native interpreter loop behind them — and you have decided the
private opcode space is worth attacking rather than walking away from. The neighbouring
file owns the diagnosis (`advanced-unpacking.md` §The honest boundary: real Dex VMP) and
the sentence that sends you here. This file owns what happens next, and — more of it than
is comfortable — what does not.
**Strength note, read before trusting any number here.** Every mechanism below was
exercised on locally built fixtures, and the closed loop is measured: `simulate` relabels
a fixture through a known bijection and `compare` re-derives that table **exactly, 218 of
218 emitted opcodes, zero wrong entries, zero fabricated entries** (`docs/tool-verification/EXTENSION-vmp-diff.md`).
What was **not** exercised is the other half of the premise: **no third-party hardening
platform was contacted**. The assumption that a real engine performs a per-opcode
substitution at stable instruction length is **inferred** from public VMP write-ups and is
exactly the assumption this method can fail on. The cost table's platform column is
therefore reasoning from how those platforms are distributed, not from a measured upload.
## 1. The method, and the one thing it needs to be true
The hardened body cannot tell you where its instruction boundaries are: its opcodes are
private, so the format specification's length table does not apply to it. That is the
whole obstacle — a disassembler that cannot decide where instruction *n* ends cannot name
instruction *n+1*.
The differential dissolves it with an oracle you control. Compile a fixture whose every
instruction you know; get the same platform to harden it; then **take the original's
instruction boundaries and project them onto the hardened bytes**. Original offset
`0x14` was a `mul-int` occupying 2 code units, so hardened bytes `0x14..0x17` are the same
instruction, re-encoded. Read the hardened opcode byte at each projection and you have a
`(original opcode → private byte)` sample. Aggregate over thousands of instructions and
the substitution table falls out.
Three preconditions, and they are not negotiable:
| Precondition | Why it must hold | What it looks like when it fails |
|---|---|---|
| **The hardened body keeps the same length** | The projection is a stencil; a length change slides every boundary after the first edit | `compare` reports `resized` for the whole method and refuses the run |
| **The substitution is per-opcode, not per-method** | One private byte per original opcode is what makes the table a table | Every original opcode claims the same private byte (`compare` → `NOT-USABLE`, stub shape) |
| **The body is still in the dex** | Alignment needs two byte streams | Method has `code_off == 0` (`compare` reports `stripped`) |
The third is the one that kills the method most often in practice: a VMP that *extracts*
bodies to a native interpreter leaves nothing to align. The differential route then has no
input at all, and no amount of clever fixture design creates one. Check for it before
spending any time here.
## 2. Cost table — which links can be automated, and which cannot
This is the part the method's public descriptions leave out. The chain is six links; two
of them are unavoidably human, and pretending otherwise is how a "harness" ships that has
never once been run against a real target.
| # | Link | Automation | Where a human must stand | Time cost | Failure mode |
|---|---|---|---|---|---|
| 1 | Compile the coverage fixture into a dex + APK | **Full** — `scripts/vmp_diff_harness.py build`, measured | none | ~40 s on this machine (javac + d8 + aapt2), ~1 s of it the script | toolchain absent or version-drifted; `d8` rejects a non-existent `--output` directory, `aapt2 link` rejects `android:` attributes without `android.jar` |
| 2 | **Submit to the hardening platform** | **None. Not automatable.** | 100 % — a browser, an account, sometimes a queue and a review | minutes to **days**; the platform decides | **the binding constraint of the whole route.** No public API; per-account rate limits; uploads are frequently gated behind registration; the artifact often comes back as a download page rather than a file path. A synthetic fixture may simply be **refused** (no real app shape, no certificate, no package identity) |
| 3 | Retrieve the hardened output and extract its dex | Partial — if the output is an APK, `unzip` + `dexutil.load_dex` handles it | download and hand-off | seconds, once the file exists | output is an APK whose dex is *also* encrypted, or the platform returns only a repacked APK with a native loader and no readable dex |
| 4 | Derive the correspondence | **Full** — `compare`, measured end to end | none | **sub-second** on a 3,065-instruction fixture | alignment slips (see §4); engine is not a pure relabelling |
| 5 | Prove the derived table | Partial — the mechanical half is scripted and measured; the judgement is not | decide whether the held-out check was passed or merely not attempted | minutes | a table that is *self-consistent* but describes a stub (§5) |
| 6 | Render Smali from the restored stream | Full — `emit-smali`, measured | none | sub-second | table has holes; restored stream decodes short of the method end |
**Conclusion, stated plainly: do not build a fully automatic pipeline.** Link 2 is
irreducibly manual — the platform has no interface a script can drive, and its output
cadence is not one a pipeline can wait on. The right shape is what this repository ships:
- links 1, 4 and 6 as **reliable local tools** (they are deterministic and were measured);
- link 5 as **a checklist plus one scripted test** (§5) — the judgement stays human;
- link 2 as **a documented manual step**, with its cost stated up front, so nobody writes
an orchestrator that waits forever on a web form;
- and links 4–5 **self-verifiable without the platform at all**, which is what `simulate`
exists for. You can prove your comparison logic against a known table today; you cannot
prove the engine's behaviour until a human uploads a fixture.
The practical consequence: the deliverable of this route is an **analysis report with a
stated boundary**, not a pipeline. Say so before starting.
## 3. The coverage fixture — what a compiled probe can and cannot reach
A Java-compiled fixture reaches **218 of the 224 named opcodes in this repository's format
table (97.3 %)** — that number is measured, not estimated, and the six it misses are
structural rather than incidental. The reason to care is direct: **an opcode your fixture
never emits leaves no known plaintext, so its substitution is unreadable by construction.**
| Missed opcode | Why a javac + d8 fixture cannot reach it |
|---|---|
| `const-string/jumbo` (0x1b) | needs a `string_ids` index > 65535, i.e. a fixture shipping 65,536+ string constants |
| `goto/32` (0x2a) | needs a >32767-code-unit backward jump; javac refuses the method first with `error: code too large` — its own 64 KB per-method bytecode cap. **Measured** on a generated 33,000-statement loop |
| `const-method-handle` (0xfe), `const-method-type` (0xff) | no Java-language literal exists for these constants; reachable only via hand-written smali or direct dex construction |
| `move-object/16` (0x09) | needs **both** registers ≥ 256; the object frame pushes the source past 256 but d8 encodes that as `move-object/from16` |
| `invoke-custom/range` (0xfd) | d8 emitted the 35c form for every lambda shape tried, including a six-parameter one |
What the fixture **does** cover, by construction, and the design decisions that get it
there (each of these was a measured gap before it was closed):
- **Three-register forms.** javac emits the `/2addr` form whenever the destination is one
of the operands, so `sub-int`, `div-int`, `rem-int`, `and-int`, `or-int`, `xor-int`,
`shl-int`, `shr-int`, `ushr-int` and their long/float/double counterparts are **absent**
from ordinary code. Three distinct locals per operation is what forces them into the
stream; the `/2addr` forms then need the accumulator spelling. Both are wanted.
- **Literal widths.** `div-int/lit16` needs a divisor in 128..32767; `and-int/lit8` needs a
mask inside −128..127. Writing `& 255` produces `and-int/lit16` and silently skips the
lit8 form.
- **Per-type field access.** `iget-wide`, `iget-object`, `iget-boolean`, `iget-byte`,
`iget-char`, `iget-short` and the `sget` set all appear only when the load targets a
**fresh local**. Accumulating into an existing register is where d8 narrows the access.
- **`/range` invokes.** The 35c invoke forms carry at most five argument registers, so a
seven-register call (`this` plus six ints) is what makes the `/range` encoding the only
legal one.
- **Wide registers and long jumps** — `move/16`, `move-wide/16`, `move-object/from16`,
`goto/16` — have **no hand-written form at all**. They exist only past a register-number
(≥256) or branch-distance (>127 code units) threshold, so the generator emits bulk
source: 300 int locals, 140 long locals, 160 object locals, a 200-statement loop body.
Measured yield: `move/16` ×46, `move-wide/16` ×20, `move-object/from16` ×161, `goto/16` ×1.
- **Payloads and structure**: `packed-switch`/`sparse-switch` with their payload blocks,
`fill-array-data`, `filled-new-array/range`, try/catch ranges with `move-exception`,
`monitor-enter`/`monitor-exit`, `instance-of`/`check-cast`, and `invoke-custom` /
`invoke-polymorphic` (which is why `d8 --min-api 26 --no-desugaring` is the default —
desugaring replaces lambdas with synthetic classes and the two opcodes disappear).
**Report the coverage every time.** `build` prints it, `audit` recomputes it for any dex.
A differential run whose coverage is 60 % produces a table that is 40 % missing, and
nothing in the output will say so unless you look at this number.
## 4. Extracting the correspondence
`compare ORIGINAL HARDENED` does four things, and the first three are diagnostics that
decide whether the fourth is meaningful:
1. **Pair methods** by `(class, name, descriptor)` and classify each pair. `aligned` means
both sides have a `code_item` of **identical `insns_size`** — the only state in which
projection is legal. `resized` means the length moved. `stripped` means the hardened
method has `code_off == 0`. `missing`/`added` mean the method set itself moved.
2. **Project and sample.** For each aligned pair, walk the original's decoded instruction
list; for instruction at original offset *o*, read the hardened byte at
`hardened.insns_off + (o − original.insns_off)`. Record `(original opcode → hardened
byte)` with a count.
3. **Aggregate and audit the aggregate.** Per original opcode: one candidate byte and the
row is `high`; more than one and the row is a `conflict`. Then the **reverse
direction** — build `private byte → set of original opcodes` and flag every private byte
claimed by more than one original. This reverse pass is not a nicety; it is the only
thing that catches the stub shape (§5).
4. **Emit a run-level verdict**, because a table is dangerous without one: `usable`,
`partially-usable`, `not-usable`, or `not-applicable` with the reason. A `not-applicable`
run exits non-zero.
Unreadable opcodes are reported as `undetermined` with the reason — either the fixture
never emitted them (§3) or the dex slot is unallocated. They are **not** guessed at.
## 5. Proving the table — the part that decides whether any of this is real
A candidate table is a hypothesis. Four checks, in increasing order of what they can
falsify:
**a. Closed-loop regression — measured, and the strongest available without a platform.**
`simulate` relabels a fixture's opcodes through a bijection whose ground truth it writes
out; `compare` must re-derive that table from the two dexes alone. Measured on this
repository's fixture: **218 rows derived, 218 of 218 emitted opcodes recovered, zero
entries wrong, zero entries fabricated, zero non-injective private bytes, verdict
`usable`.** Any change to the comparison logic that breaks this check is a regression, and
the check needs no platform, no device and no network.
**b. Injectivity — the check that catches a confident wrong answer.** Every private byte
must be claimed by exactly one original opcode. Measured against the stub shape (bodies
replaced by an equal-length `return-void` fill): per-opcode reading looks *perfect* — 218
rows, all `high`, zero conflicts — because every original opcode maps to the same byte.
Only the reverse direction exposes it: one private byte claimed by **218** originals →
`NOT-USABLE`. A table produced without this check would have been silently worthless.
**c. Held-out consistency.** Derive the table from one subset of methods, then use it to
disassemble a **different** subset that contributed nothing to the derivation. A correct
table restores the instructions such that the walk lands exactly on
`insns_off + insns_size × 2` and every branch target lands on an instruction boundary. A
wrong or incomplete table desynchronises and says so. `emit-smali`'s
`instructions restored` / `unmapped-opcode stops` counters are the readout: on the measured
fixture, 3,065 restored and 0 stops.
**d. Cross-check against a second, independent recovery.** The same private opcode space
observed through a *different* route — an in-memory dump of the original dex alongside the
hardened one, or the `/proc/<pid>/mem` route in
`advanced-unpacking.md` §Dumping when frida is refused — is the only evidence that the
platform's transformation is the one you assumed. **Not done here**; it requires a real
VMP sample and a real platform, and is the honest gap in this file.
Note what none of these checks establish: that the engine's substitution is *stable across
inputs*. Fixture-derived tables have been reported in public work to drift per
vendor-version and per hardening pass. Re-derive per target; never cache a table.
## 6. Generating Smali from a restored stream
`emit-smali HARDENED_DEX --table TABLE.json` restores opcode bytes through the table and
renders method bodies as an annotated smali **reading skeleton**:
- Boundaries come from the table itself — once a byte maps back to a standard opcode the
standard length table applies to that instruction, so the walk is self-bootstrapping.
A byte with no table entry stops that method's walk, and the count of such events is
printed as the honest coverage measure.
- `.class` / `.super` / `.source` / `.method` headers are real, with access flags decoded
from `access_flags`, so the skeleton is readable as smali rather than as a hex dump.
- Payload blocks (`packed-switch-payload` etc.) are stepped over by their own layout, not
treated as opcodes.
**It is explicitly not assembliable, and the output says so in its first ten lines.**
Recovered: instruction boundaries and opcode names. Not recovered: register numbers,
field/method/string indices as the engine rewrote them, and any register re-allocation —
**no map for those exists in this method at all.** Feeding the skeleton to a smali
assembler produces an empty method, which looks like success and is not.
## 7. When this route does not work — the honest retreat
| What `compare` reports | What it means | Do this |
|---|---|---|
| `verdict: not-applicable`, `stripped` > 0 | bodies left the dex for a native interpreter | the differential has no input. Stop. Return to `advanced-unpacking.md` §The honest boundary: real Dex VMP and price the island honestly |
| `verdict: not-applicable`, `resized` > 0 | the engine's stream has its own length table | the projection premise is false. Stop rather than guessing boundaries |
| `verdict: not-usable`, one private byte shared by many originals | bodies replaced by a common stub | the engine did not relabel per-opcode here. Stop |
| `verdict: partially-usable`, conflicts ≤ 30 % | partial relabelling, or alignment slipping on some methods | usable only for the `high` rows; say so and stop at the unreadable ones |
| `usable`, but coverage was low | a complete table over an incomplete opcode set | usable, with the uncovered opcodes named as unreadable |
And the retreat that outranks all of them: **the cheaper question is whether the behaviour
you need exists outside the virtualized island.** A hook at the method's Java boundary, an
emulated call to the routine rather than a reading of it
(`emulation-and-rpc.md`), or a server-side answer frequently reaches the goal without ever
paying for the island. Weeks of table recovery is the expensive way to ask a question a
boundary hook answers in an hour.
## 8. Decision summary
| Observation | Action | Where |
|---|---|---|
| Bodies present, decode as nonsense | Diagnose real VMP before anything else | `advanced-unpacking.md` §The honest boundary: real Dex VMP |
| Need the private opcode mapping | Build the coverage fixture, then submit it by hand | `scripts/vmp_diff_harness.py build`; §2 of this file |
| Asked to "automate the whole chain" | Say which link cannot be automated and what it costs | §2 — the platform step is manual, by construction |
| Fixture coverage below ~90 % | Add the missing opcode classes before submitting | §3, and the `audit` report |
| Two dexes in hand | Derive the candidate table, read the verdict first | `scripts/vmp_diff_harness.py compare` |
| About to trust a derived table | Run the closed loop, the injectivity check and a held-out subset | §5 — a/b/c are scripted |
| Table complete | Render the skeleton; expect operands, not registers | `scripts/vmp_diff_harness.py emit-smali` |
| `resized` / `stripped` / stub verdict | Stop and say so | §7 |
skills/apk-reverse/scripts/dex_dump_validate.py
+132 -24
View File
@@ -9,9 +9,18 @@ structure before trusting); this script is the tool that rule was missing.
Per file it reports: size, sha256, dex version, checksum/signature verdicts
(`dexutil.verify_dex_header`), class count, method-body census (no-code / real /
trivial-stub / empty), the trivial-body ratio that flags an extraction-shell
skeleton, and --find pattern hits across the string table. Files are grouped by
sha256 and the survivors are ranked: most likely original first.
trivial-stub / nop-erased / minimal-form / empty), the stub and emptied ratios that
flag an extraction-shell skeleton, and --find pattern hits across the string table.
Files are grouped by sha256 and the survivors are ranked: most likely original first.
**Read `emptied%`, not `stub%`, for the skeleton call.** The two differ, and the
difference is load-bearing: `stub%` counts only bodies left as a lone `return*`, while
`emptied%` also counts bodies wiped to nothing but nops. A shell that clears the slot
instead of writing a return scores `stub% = 0.0` on a fully emptied image — measured,
and it used to be ranked the *most likely original* because of it. `stub%` is also
bimodal in practice: it lands at the app's own baseline or at ~100 %, with no band in
between, and partial extraction (75 % of bodies removed) moves it by 0.2 points. Full
matrix and commands: `docs/tool-verification/EXTENSION-extraction-shell-bench.md`.
Exit code 0 unless nothing in the input parses as a dex at all (exit 1).
@@ -36,12 +45,34 @@ RETURN_OPS = {0x0E, 0x0F, 0x10, 0x11} # return-void / return / return-wide / re
def classify_body(data, code_off):
"""Classify one code_item's instruction stream: 'empty' | 'stub' | 'real'.
"""Classify one code_item's instruction stream.
'stub' = skip plain nop units (0x0000) and exactly one return* remains. That is
what an extraction shell leaves behind when it defers decryption to first
invocation, and what a repair fixture writes when mimicking one. Anything with a
second meaningful unit is real code, however small.
Returns one of 'empty' | 'stub' | 'erased' | 'malformed' | 'real'.
'stub' -- skip plain nop units (0x0000) and exactly one return* remains.
That is what an extraction shell leaves behind when it defers
decryption to first invocation, and what a repair fixture writes
when mimicking one.
'erased' -- insns_size > 0 but every unit is 0x0000. Measured on a fixture:
a body emptied by nop fill lands here, and reporting it as 'real'
(the pre-2026-09 behaviour) inverted the ranking, because a fully
nop-filled skeleton scored stub% = 0.0 and was named the most
likely original. A real compiler never emits a whole body of
nops, so this is skeleton evidence, not code.
'minimal' -- a body truncated to `const/4 vR, #0; return vR` and nop-padded,
with a non-zero high byte on the return. This is a *legal* minimal
non-void body, not a malformed one -- and it is a skeleton shape
the stub detector is blind to by design. Counted separately so the
blind spot is visible in the report instead of hiding inside
'real'. Measured: a fixture with all 5,061 bodies cut to this form
reports stub% = 1.9%, the same as its untouched control.
'real' -- anything else, including a body of a single non-return unit.
A two-unit `const/4 v0, #0; return v0` is deliberately **not** a stub: it is what
a *legal* minimal body looks like, and treating it as skeleton evidence would
make every tiny accessor a false positive. That asymmetry is the detector's
designed boundary, and it is measured in
`docs/tool-verification/EXTENSION-extraction-shell-bench.md`.
"""
insns_size = u32(data, code_off + 12)
if insns_size == 0:
@@ -55,14 +86,27 @@ def classify_body(data, code_off):
continue
meaningful += 1
last_unit = unit
if meaningful > 1:
return "real" # early exit: two units can never be a stub
if meaningful > 2:
return "real" # three units can never be a stub or a minimal
if meaningful == 0:
return "erased" # every unit was 0x0000
if meaningful == 1:
op = last_unit & 0xFF # first byte holds the opcode for 10x/11x
if op == 0x0E and last_unit == 0x000E: # return-void, no register
return "stub"
if op == 0x0E:
# 10x return-void has no operand: `0xNN0E` with NN != 0 is `return vNN`,
# i.e. a non-void return wearing a return-void opcode -- minimal form.
return "stub" if last_unit == 0x000E else "minimal"
if op in (0x0F, 0x10, 0x11): # 11x returns: high byte is the register
return "stub"
return "real"
# meaningful == 2: legal minimal body only when it is `const/4 vR,#0; return* vR`
first_unit = 0
for k in range(insns_size):
if u16(data, base + k * 2):
first_unit = u16(data, base + k * 2)
break
if (first_unit & 0xFF) == 0x12 and (last_unit & 0xFF) in (0x0E, 0x0F, 0x10, 0x11):
return "minimal"
return "real"
@@ -161,7 +205,7 @@ def profile_dex(path, patterns, trim=False):
prof["signature_ok"] = sig_ok
prof["classes"] = header["class_defs_size"]
no_code = with_code = stubs = empties = 0
no_code = with_code = stubs = empties = erased = minimal = 0
try:
for code_off in walk_methods(data, header):
if code_off == 0: # abstract / native declaration
@@ -175,14 +219,26 @@ def profile_dex(path, patterns, trim=False):
stubs += 1
elif kind == "empty":
empties += 1
elif kind == "erased":
erased += 1
elif kind == "minimal":
minimal += 1
except Exception as exc: # desynced class_data: keep counts
prof["walk_error"] = str(exc)
prof["methods_no_code"] = no_code
prof["methods_with_code"] = with_code
prof["bodies_real"] = with_code - stubs - empties
prof["bodies_real"] = with_code - stubs - empties - erased - minimal
prof["bodies_stub"] = stubs
prof["bodies_empty"] = empties
prof["bodies_erased"] = erased
prof["bodies_minimal"] = minimal
# trivial_ratio keeps its original definition (return*-stub share of bodies) so
# existing records stay comparable. The skeleton signal is the union of every
# measured emptied shape; a body wiped to nothing at all must never score lower
# than a body wiped to `return-void`, or the ranking inverts (see
# docs/tool-verification/EXTENSION-extraction-shell-bench.md).
prof["trivial_ratio"] = (stubs / with_code) if with_code else 0.0
prof["emptied_ratio"] = ((stubs + erased) / with_code) if with_code else 0.0
if patterns:
try:
@@ -207,7 +263,38 @@ def collect_files(paths):
def rank_key(prof):
return (prof["trivial_ratio"],
"""Sort key: most-likely-original first.
Both body signals are needed, in the right order, and each was measured to fail
alone:
* the `return*`-stub share alone let a **nop-filled** skeleton score 0.0 and be
named the most likely original, because a wiped body counts as neither a stub
nor real code;
* `emptied_ratio` alone still failed, because it is **bimodal** -- it sits at the
host app's own baseline or at ~100 %, with nothing in between. A dex with 25 %
of its bodies emptied measures *below* its own untouched control, 1.7 % against
1.9 %, so the sort pointed at a modified image and called it the original;
* `minimal + erased` alone failed too, and in the opposite direction: a skeleton
that writes a bare `return-void` into every body has minimal = erased = 0 and
would sort first, despite `emptied%` = 100.
So the key combines both, and the combination has to let *either* signal disqualify
an image rather than letting them tie:
* level 1 -- `emptied_ratio >= 0.5`. Nothing that is mostly emptied is a candidate,
whatever else it scores. This is the level that evicts the two skeletons the
plain counts cannot see: an image with a bare `return-void` in every body has
minimal = erased = 0, so a pure-count key ranked it first.
* level 2 -- `minimal + erased`, the skeleton-evidence count, low-first. This is
what orders the **bimodal band**, where `emptied_ratio` cannot: the untouched
control carries 82, and the count is monotone as bodies are removed (1299 /
2524 / 3736 / 4946 at 25 / 50 / 75 / 100 %).
* level 3 -- `emptied_ratio` low-first, to separate what is left.
"""
return (1 if prof.get("emptied_ratio", prof["trivial_ratio"]) >= 0.5 else 0,
prof.get("bodies_minimal", 0) + prof.get("bodies_erased", 0),
prof.get("emptied_ratio", prof["trivial_ratio"]),
0 if prof["checksum_ok"] else 1,
0 if prof["signature_ok"] else 1,
prof["name"])
@@ -248,9 +335,9 @@ def main(argv=None):
# ---- human-readable report -------------------------------------------------
print("== dex dump validation: %d file(s), %d parse, %d rejected =="
% (len(profiles), len(valid), len(profiles) - len(valid)))
fmt = "%-28s %9s %-12s %3s %-7s %-7s %6s %5s %6s %7s"
fmt = "%-28s %9s %-12s %3s %-7s %-7s %6s %6s %6s %7s %7s %7s"
print(fmt % ("name", "size", "sha256[:12]", "ver", "cksum", "sig",
"class", "noco", "code", "stub%"))
"class", "noco", "code", "stub%", "erased%", "emptied%"))
for p in profiles:
if "error" in p:
print("%-28s %9s %-12s %3s REJECTED: %s"
@@ -261,12 +348,19 @@ def main(argv=None):
"ok" if p["checksum_ok"] else "BAD",
"ok" if p["signature_ok"] else "BAD",
p["classes"], p["methods_no_code"], p["methods_with_code"],
"%.1f%%" % (100.0 * p["trivial_ratio"])))
"%.1f%%" % (100.0 * p["trivial_ratio"]),
"%.1f%%" % (100.0 * (p["bodies_erased"] / p["methods_with_code"]
if p["methods_with_code"] else 0.0)),
"%.1f%%" % (100.0 * p.get("emptied_ratio", 0.0))))
if p.get("trimmed_from"):
print("%-28s trimmed %d -> %d B (page-aligned dump)"
% ("", p["trimmed_from"], p["trimmed_to"]))
if p.get("walk_error"):
print("%-28s walk stopped early: %s" % ("", p["walk_error"]))
if p.get("bodies_minimal"):
print("%-28s minimal-form bodies: %d (const/4+return truncation -- a "
"skeleton shape the stub%% column cannot see)"
% ("", p["bodies_minimal"]))
for pat, n in sorted(p.get("find_hits", {}).items()):
print("%-28s find %-24s %d" % ("", pat, n))
@@ -283,18 +377,32 @@ def main(argv=None):
print("\nranking (most likely original first):")
for i, p in enumerate(ranking, 1):
note = ""
if p["trivial_ratio"] >= 0.5:
note = " <-- SKELETON: stub%%=%.0f, extraction-shell shape" \
% (100.0 * p["trivial_ratio"])
print(" %d. %-28s stub%%=%.1f cksum=%s sig=%s classes=%d%s"
if p.get("emptied_ratio", 0.0) >= 0.5:
note = (" <-- SKELETON: emptied%%=%.0f, extraction-shell shape"
% (100.0 * p["emptied_ratio"]))
elif p.get("bodies_erased", 0):
note = (" <-- %d body(ies) wiped to nops: skeleton evidence"
% p["bodies_erased"])
print(" %d. %-28s stub%%=%.1f erased%%=%.1f min/erased=%d cksum=%s "
"sig=%s classes=%d%s"
% (i, p["name"], 100.0 * p["trivial_ratio"],
100.0 * (p["bodies_erased"] / p["methods_with_code"]
if p["methods_with_code"] else 0.0),
p.get("bodies_minimal", 0) + p.get("bodies_erased", 0),
"ok" if p["checksum_ok"] else "BAD",
"ok" if p["signature_ok"] else "BAD",
p["classes"], note))
top = ranking[0]
copies = len(groups.get(top["sha256"], []))
print("\nverdict: %s is the most likely original (%d copy(ies) in this set)"
% (top["name"], copies))
print("\nverdict: %s is the best-supported candidate for the original "
"(%d copy(ies) in this set)" % (top["name"], copies))
print("ranked by: mostly-emptied images evicted first (emptied%% >= 50), then "
"the minimal+nop-erased body count low-first, then emptied%%.\n"
" Limits, stated rather than hidden: this cannot see a `throw`-stub "
"skeleton (it scores at the control's own baseline), and a\n"
" partially extracted image still outranks a heavily stubbed one. "
"Confirm the winner before using it as a patch baseline --\n"
" docs/tool-verification/EXTENSION-extraction-shell-bench.md")
else:
print("\nverdict: no image parsed as a dex -- nothing to rank")
return 0 if valid else 1
skills/apk-reverse/scripts/dex_mem_scan.py
+15 -1
View File
@@ -16,10 +16,24 @@ Feeding the extracted images to `dex_dump_validate.py` is the next step: this
script only *finds* and *cuts*; that one judges whether what came out is a real
body or an extraction-shell skeleton.
**Measured hit rate on the reference device: zero for anonymous regions.** The
root-dump pass recorded in `docs/tool-verification/EXTENSION-rootdump.md` scanned
142 anonymous mappings and returned **no hits**, while the *named* ART dex mappings
in the same process yielded 17 dex images without any search at all. A negative
result from this script is therefore weak evidence: it is compatible with "the
payload is not resident", "the payload is resident but not as a contiguous
dex-magic image", and "the payload was paged out between capture and scan". Treat a
zero-hit scan as a reason to check the named mappings first
(`[anon:dalvik-classes*.dex extracted in memory from <src>]`), not as a conclusion
about the packer. A positive hit inside a genuinely unnamed mapping has not been
observed here yet.
Scope note: the header's `file_size` field decides the cut. When the declared size
does not fit inside the capture, the hit is reported and skipped rather than
guessed at -- a dex whose tail is missing is not recoverable by truncating the
wrong end.
wrong end. Page-aligned captures from a VMA routinely overrun the image they hold,
so a run against raw VMA exports usually needs `--keep-partial` or a trim step
before `dex_dump_validate.py` will accept the output.
"""
import argparse
import hashlib
skills/apk-reverse/scripts/java2c_probe.py
+960
View File
@@ -0,0 +1,960 @@
#!/usr/bin/env python3
"""java2c_probe.py - collect the evidence that decides HOW a target was hardened.
The question this answers is narrow on purpose. Faced with "the Java methods have
no bodies", four different mechanisms produce that observation and they need four
different routes:
Java2C Java bytecode translated to C, compiled into a .so. No dex
bytecode exists at runtime, ever. Route: read the .so.
Extraction shell Real dex bytecode, encrypted at rest, decrypted at runtime.
Route: dump memory, then validate the dump.
VMP Real dex bytecode replaced by private opcodes fed to an
interpreter. Route: measure, then usually stop.
JNI sinking A few hot methods deliberately moved to native by hand.
Route: reverse the .so; the dex is still readable.
The expensive mistake is reading the first shape as the second: the analyst goes
looking for a decrypted DEX that is never produced, because nothing was ever
encrypted. This script prints the discriminating evidence and labels each item
strong / medium / weak so the reading is not mistaken for a verdict.
Usage (paths are always explicit; nothing is discovered implicitly):
python java2c_probe.py --apk sample.apk
python java2c_probe.py --apk sample.apk --json
python java2c_probe.py --dex classes.dex --so lib/arm64-v8a/libnc.so
python java2c_probe.py --dir extracted/ # every *.dex and *.so below
Exit codes: 0 = evidence collected, 1 = no usable input, 2 = argument error.
"""
import argparse
import json
import os
import re
import struct
import sys
import zipfile
# ---------------------------------------------------------------------------
# dex parsing (self-contained; deliberately does not import the repo's dexutil)
# ---------------------------------------------------------------------------
ACC_NATIVE = 0x0100
ACC_ABSTRACT = 0x0400
# Instruction length in 16-bit code units, for the subset that a stub or a
# decompiled-from-C body produces. Unknown opcodes return 1 so a counting error
# can never inflate the trivial-body ratio; see the note on that metric below.
_INSN_UNITS = {
0x00: 1, 0x01: 1, 0x02: 1, 0x03: 1, 0x04: 1, 0x05: 1, 0x06: 1, 0x07: 1,
0x08: 1, 0x09: 1, 0x0a: 1, 0x0b: 1, 0x0c: 1, 0x0d: 1, 0x0e: 1, 0x0f: 1,
0x10: 1, 0x11: 1, 0x12: 1, 0x13: 2, 0x14: 3, 0x15: 2, 0x16: 2, 0x17: 3,
0x18: 5, 0x19: 2, 0x1a: 2, 0x1b: 3, 0x1c: 2, 0x1d: 1, 0x1e: 1, 0x1f: 1,
0x20: 1, 0x21: 1, 0x22: 1, 0x23: 1, 0x24: 3, 0x25: 3, 0x26: 3, 0x27: 1,
0x28: 1, 0x29: 1, 0x2a: 1, 0x2b: 1, 0x2c: 1, 0x2d: 1, 0x2e: 1, 0x2f: 1,
0x30: 1, 0x31: 1, 0x32: 1, 0x33: 1, 0x34: 1, 0x35: 1, 0x36: 1, 0x37: 1,
0x38: 1, 0x39: 1, 0x3a: 1, 0x3b: 1, 0x3c: 1, 0x3d: 1, 0x3e: 1, 0x3f: 1,
0x40: 1, 0x41: 1, 0x42: 1, 0x43: 1, 0x44: 2, 0x45: 2, 0x46: 2, 0x47: 2,
0x48: 2, 0x49: 2, 0x4a: 2, 0x4b: 2, 0x4c: 2, 0x4d: 2, 0x4e: 2, 0x4f: 2,
0x50: 2, 0x51: 2, 0x52: 2, 0x53: 2, 0x54: 2, 0x55: 2, 0x56: 2, 0x57: 2,
0x58: 2, 0x59: 2, 0x5a: 2, 0x5b: 2, 0x5c: 2, 0x5d: 2, 0x5e: 2, 0x5f: 2,
0x60: 2, 0x61: 2, 0x62: 2, 0x63: 2, 0x64: 2, 0x65: 2, 0x66: 2, 0x67: 2,
0x68: 2, 0x69: 2, 0x6a: 2, 0x6b: 2, 0x6c: 2, 0x6d: 2, 0x6e: 3, 0x6f: 3,
0x70: 3, 0x71: 3, 0x72: 3, 0x73: 1, 0x74: 1, 0x75: 1, 0x76: 1, 0x77: 1,
0x78: 2, 0x79: 2, 0x7a: 2, 0x7b: 2, 0x7c: 2, 0x7d: 2, 0x7e: 2, 0x7f: 2,
0x80: 2, 0x81: 2, 0x82: 2, 0x83: 2, 0x84: 2, 0x85: 2, 0x86: 2, 0x87: 2,
0x88: 2, 0x89: 2, 0x8a: 2, 0x8b: 2, 0x8c: 2, 0x8d: 2, 0x8e: 2, 0x8f: 2,
0x90: 2, 0x91: 2, 0x92: 2, 0x93: 2, 0x94: 2, 0x95: 2, 0x96: 2, 0x97: 2,
0x98: 2, 0x99: 2, 0x9a: 2, 0x9b: 2, 0x9c: 2, 0x9d: 2, 0x9e: 2, 0x9f: 2,
0xa0: 2, 0xa1: 2, 0xa2: 2, 0xa3: 2, 0xa4: 2, 0xa5: 2, 0xa6: 2, 0xa7: 2,
0xa8: 2, 0xa9: 2, 0xaa: 2, 0xab: 2, 0xac: 2, 0xad: 2, 0xae: 2, 0xaf: 2,
0xb0: 1, 0xb1: 1, 0xb2: 2, 0xb3: 2, 0xb4: 2, 0xb5: 2, 0xb6: 2, 0xb7: 2,
0xb8: 2, 0xb9: 2, 0xba: 2, 0xbb: 2, 0xbc: 2, 0xbd: 2, 0xbe: 2, 0xbf: 2,
0xc0: 2, 0xc1: 2, 0xc2: 2, 0xc3: 2, 0xc4: 2, 0xc5: 2, 0xc6: 2, 0xc7: 2,
0xc8: 2, 0xc9: 2, 0xca: 2, 0xcb: 2, 0xcc: 2, 0xcd: 2, 0xce: 2, 0xcf: 2,
0xd0: 2, 0xd1: 2, 0xd2: 2, 0xd3: 2, 0xd4: 2, 0xd5: 2, 0xd6: 2, 0xd7: 2,
0xd8: 2, 0xd9: 2, 0xda: 2, 0xdb: 2, 0xdc: 2, 0xdd: 2, 0xde: 2, 0xdf: 2,
0xe0: 2, 0xe1: 2, 0xe2: 2, 0xe3: 0, 0xe4: 0, 0xe5: 0, 0xe6: 0, 0xe7: 0,
0xe8: 0, 0xe9: 0, 0xea: 0, 0xeb: 0, 0xec: 0, 0xed: 0, 0xee: 0, 0xef: 0,
0xf0: 0, 0xf1: 0, 0xf2: 0, 0xf3: 0, 0xf4: 0, 0xf5: 0, 0xf6: 0, 0xf7: 0,
0xf8: 0, 0xf9: 0, 0xfa: 0, 0xfb: 0, 0xfc: 0, 0xfd: 0, 0xfe: 0, 0xff: 0,
}
_RETURN_OPS = (0x0e, 0x0f, 0x10, 0x11)
def _uleb128(buf, pos):
result = 0
shift = 0
while True:
if pos >= len(buf):
raise ValueError("uleb128 truncated")
byte = buf[pos]
pos += 1
result |= (byte & 0x7F) << shift
if not (byte & 0x80):
return result, pos
shift += 7
if shift > 35:
raise ValueError("uleb128 too long")
class Dex(object):
"""Minimal structural reader: header, class_defs, class_data, code_item."""
def __init__(self, data, name):
self.data = data
self.name = name
self.header_ok = False
self.header_note = ""
self.methods = [] # dicts: class, name, flags, code_off, units
self.class_names = []
self.native_methods = 0
self.abstract_methods = 0
self.body_methods = 0
self.no_code_methods = 0 # neither native nor abstract, yet code_off == 0
self.trivial_bodies = 0
self.native_with_code = 0
self.fully_native_classes = 0
self.native_dominated_classes = 0
self.parse_error = None
self._parse()
def _u16(self, off):
return struct.unpack_from("<H", self.data, off)[0]
def _u32(self, off):
return struct.unpack_from("<I", self.data, off)[0]
def _parse(self):
d = self.data
if len(d) < 112 or d[:4] != b"dex\n":
self.parse_error = "not a dex image (bad magic or too short)"
return
try:
file_size = self._u32(0x20)
map_off = self._u32(0x34)
string_ids_size = self._u32(0x38)
string_ids_off = self._u32(0x3C)
type_ids_size = self._u32(0x40)
type_ids_off = self._u32(0x44)
proto_ids_size = self._u32(0x48)
proto_ids_off = self._u32(0x4C)
method_ids_size = self._u32(0x58)
method_ids_off = self._u32(0x5C)
class_defs_size = self._u32(0x60)
class_defs_off = self._u32(0x64)
except Exception as exc: # pragma: no cover
self.parse_error = "header read failed: %s" % exc
return
# Header claims vs. reality. A truncated or padded capture is the normal
# way this probe is handed bad input, so say which one happened.
if file_size != len(d):
self.header_note = "header file_size=%d actual=%d" % (file_size, len(d))
else:
self.header_ok = True
if not (0 < string_ids_size < 1 << 22 and string_ids_off + string_ids_size * 4 <= len(d)):
self.parse_error = "string_ids table out of bounds"
return
string_offsets = [self._u32(string_ids_off + 4 * i) for i in range(string_ids_size)]
def string_at(idx):
if idx < 0 or idx >= len(string_offsets):
return "<bad-string-idx-%d>" % idx
off = string_offsets[idx]
_, pos = _uleb128(d, off) # utf16 length, ignored
end = d.find(b"\x00", pos)
if end < 0:
return "<unterminated>"
return d[pos:end].decode("utf-8", "replace")
try:
type_desc = [string_at(self._u32(type_ids_off + 4 * i))
for i in range(type_ids_size)]
except Exception as exc:
self.parse_error = "type_ids read failed: %s" % exc
return
try:
if method_ids_off + method_ids_size * 8 > len(d):
raise ValueError("method_ids out of bounds")
method_info = []
for i in range(method_ids_size):
base = method_ids_off + 8 * i
class_idx = self._u16(base)
name_idx = self._u32(base + 4)
desc = type_desc[class_idx] if class_idx < len(type_desc) else "?"
method_info.append((desc, string_at(name_idx)))
except Exception as exc:
self.parse_error = "method_ids read failed: %s" % exc
return
if class_defs_off + class_defs_size * 32 > len(d):
self.parse_error = "class_defs table out of bounds"
return
per_class_native = {}
per_class_total = {}
for ci in range(class_defs_size):
base = class_defs_off + 32 * ci
class_idx = self._u32(base)
class_data_off = self._u32(base + 24)
cname = type_desc[class_idx] if class_idx < len(type_desc) else "?%d" % class_idx
self.class_names.append(cname)
if class_data_off == 0:
continue
try:
pos = class_data_off
static_fields, pos = _uleb128(d, pos)
instance_fields, pos = _uleb128(d, pos)
direct_methods, pos = _uleb128(d, pos)
virtual_methods, pos = _uleb128(d, pos)
for _ in range(static_fields + instance_fields):
_, pos = _uleb128(d, pos) # field_idx_diff
_, pos = _uleb128(d, pos) # access_flags
# direct_methods and virtual_methods are two independent encoded
# lists: each one's first method_idx_diff is relative to 0. Walking
# them as a single run silently mis-attributes every virtual
# method, so they are decoded as two passes.
for list_size in (direct_methods, virtual_methods):
method_idx = 0
for _ in range(list_size):
diff, pos = _uleb128(d, pos)
flags, pos = _uleb128(d, pos)
code_off, pos = _uleb128(d, pos)
method_idx += diff
if method_idx < len(method_info):
mcls, mname = method_info[method_idx]
else:
mcls, mname = "?", "?%d" % method_idx
units = None
if code_off:
units = self._u16(code_off + 12) # insns_size
self.methods.append({
"class": mcls, "name": mname, "flags": flags,
"code_off": code_off, "units": units,
})
# Constructors are excluded from the per-class shape: a
# Java2C pass leaves <init>/<clinit> in Java in practice,
# and counting them would make "all methods native" an
# essentially unreachable test.
named = mname not in ("<init>", "<clinit>")
if named:
per_class_total[cname] = per_class_total.get(cname, 0) + 1
if flags & ACC_NATIVE:
self.native_methods += 1
if named:
per_class_native[cname] = per_class_native.get(cname, 0) + 1
if code_off:
self.native_with_code += 1
elif flags & ACC_ABSTRACT:
self.abstract_methods += 1
elif code_off == 0:
self.no_code_methods += 1
else:
self.body_methods += 1
if self._is_trivial(code_off, units):
self.trivial_bodies += 1
except Exception:
# One unreadable class_data must not void the whole measurement.
continue
for cname, total in per_class_total.items():
if total < 3:
continue
native_here = per_class_native.get(cname, 0)
if native_here == total:
self.fully_native_classes += 1
if native_here >= 0.7 * total:
self.native_dominated_classes += 1
def _is_trivial(self, code_off, units):
"""A body that only returns a constant: the classic stub shape.
Approximation, and labelled as such: it walks the instruction stream with
the length table above and accepts only nop/move/const/return. Anything
that touches a field, calls out, or branches is not a stub.
"""
if not units:
return False
insns_off = code_off + 16
if insns_off + units * 2 > len(self.data):
return False
try:
pos = 0
seen = 0
while pos < units and seen < 8:
word = struct.unpack_from("<H", self.data, insns_off + pos * 2)[0]
op = word & 0xFF
if op in _RETURN_OPS:
return True if seen <= 2 else False
if op == 0x00 or 0x01 <= op <= 0x09 or 0x12 <= op <= 0x19:
step = _INSN_UNITS.get(op, 1)
if step == 0:
return False
pos += step
seen += 1
continue
return False
return False
except Exception:
return False
# -- derived metrics ---------------------------------------------------
@property
def total_methods(self):
return len(self.methods)
@property
def native_ratio(self):
return (float(self.native_methods) / self.total_methods) if self.total_methods else 0.0
@property
def trivial_ratio(self):
return (float(self.trivial_bodies) / self.body_methods) if self.body_methods else 0.0
def summary(self):
return {
"name": self.name,
"size": len(self.data),
"header_size_matches": self.header_ok,
"header_note": self.header_note,
"classes": len(self.class_names),
"total_methods": self.total_methods,
"native_methods": self.native_methods,
"native_ratio": round(self.native_ratio, 4),
"abstract_methods": self.abstract_methods,
"body_methods": self.body_methods,
"no_code_methods": self.no_code_methods,
"native_with_code": self.native_with_code,
"trivial_bodies": self.trivial_bodies,
"trivial_ratio": round(self.trivial_ratio, 4),
"fully_native_classes": self.fully_native_classes,
"native_dominated_classes": self.native_dominated_classes,
"bytes_per_class": int(len(self.data) / len(self.class_names)) if self.class_names else 0,
"parse_error": self.parse_error,
}
# ---------------------------------------------------------------------------
# ELF parsing (self-contained; no lief / pyelftools dependency)
# ---------------------------------------------------------------------------
# Each entry: (regex, label, strength, meaning). Strength is what stops a weak
# hit from being read as a verdict - see the printed weak-criteria block.
_STRINGSIGS = [
(rb"Dex2C", "Dex2C", "strong", "dcc toolchain marker in code/rodata"),
(rb"dynamic_register_compile_methods", "dcc-register-fn", "strong",
"dcc's generated RegisterNatives entry point"),
(rb"dcc", "dcc", "weak", "3 letters: matches unrelated words (e.g. 'addcc')"),
(rb"JNI_OnLoad", "JNI_OnLoad", "weak", "present in almost every JNI library"),
(rb"RegisterNatives", "RegisterNatives", "weak",
"dynamic registration; ordinary JNI libraries use it too"),
(rb"libc\+\+_shared\.so", "libc++_shared", "weak", "NDK C++ runtime, not a hardening marker"),
(rb"c\+\+_static", "c++_static", "weak", "dcc's default APP_STL; also a common NDK setting"),
(rb"FindClass", "FindClass", "weak", "any JNI callback path"),
(rb"GetMethodID", "GetMethodID", "weak", "any JNI callback path"),
(rb"GetStaticMethodID", "GetStaticMethodID", "weak", "any JNI callback path"),
(rb"CallObjectMethod", "CallObjectMethod", "weak", "any JNI callback path"),
(rb"CallStaticObjectMethod", "CallStaticObjectMethod", "weak", "any JNI callback path"),
(rb"NewLocalRef", "NewLocalRef", "weak", "generated code that manages refs by hand"),
(rb"ScopedLocalRef", "ScopedLocalRef", "medium", "dcc runtime header name"),
(rb"well_known_classes", "well_known_classes", "medium", "dcc runtime header name"),
]
_JNI_CALLS = ("FindClass", "GetMethodID", "GetStaticMethodID", "CallObjectMethod",
"CallStaticObjectMethod", "CallIntMethod", "CallVoidMethod",
"CallStaticIntMethod", "NewObject", "GetFieldID")
class Elf(object):
"""Enough of ELF to answer: what is exported, what is imported, what is inside."""
def __init__(self, data, name):
self.data = data
self.name = name
self.arch = "unknown"
self.is_elf = False
self.dynsym = [] # (name, shndx)
self.sig_hits = {} # label -> (count, strength, meaning)
self.java_symbols = []
self.has_jni_onload = False
self.imports_register_natives = False
self.jni_call_refs = []
self.elf_note = ""
self._parse()
def _parse(self):
d = self.data
if len(d) < 64 or d[:4] != b"\x7fELF":
self.elf_note = "not an ELF image"
return
self.is_elf = True
ei_class = d[4]
ei_data = d[5]
if ei_class not in (1, 2):
self.elf_note = "bad EI_CLASS=%d" % ei_class
return
endian = "<" if ei_data == 1 else ">"
is64 = ei_class == 2
try:
e_type = struct.unpack_from(endian + "H", d, 16)[0]
e_machine = struct.unpack_from(endian + "H", d, 18)[0]
if is64:
e_shoff = struct.unpack_from(endian + "Q", d, 0x28)[0]
e_shentsize = struct.unpack_from(endian + "H", d, 0x3A)[0]
e_shnum = struct.unpack_from(endian + "H", d, 0x3C)[0]
e_shstrndx = struct.unpack_from(endian + "H", d, 0x3E)[0]
else:
e_shoff = struct.unpack_from(endian + "I", d, 0x20)[0]
e_shentsize = struct.unpack_from(endian + "H", d, 0x2E)[0]
e_shnum = struct.unpack_from(endian + "H", d, 0x30)[0]
e_shstrndx = struct.unpack_from(endian + "H", d, 0x32)[0]
except struct.error as exc:
self.elf_note = "header read failed: %s" % exc
return
self.arch = {
3: "x86", 8: "mips", 20: "ppc", 40: "arm", 62: "x86_64",
183: "aarch64", 243: "riscv",
}.get(e_machine, "machine-%d" % e_machine)
if e_type not in (2, 3):
self.elf_note = "e_type=%d (not EXEC/DYN)" % e_type
# Strings first: it is the check that works even when the section table
# has been stripped, which is the common case for a hardened library.
self._scan_strings()
if e_shoff and e_shnum and e_shoff + e_shnum * e_shentsize <= len(d):
self._read_sections(endian, is64, e_shoff, e_shentsize, e_shnum, e_shstrndx)
else:
self.elf_note = (self.elf_note + "; " if self.elf_note else "") + \
"no usable section table (stripped?) - symbol checks skipped"
def _scan_strings(self):
d = self.data
for pattern, label, strength, meaning in _STRINGSIGS:
count = len(re.findall(pattern, d))
if count:
self.sig_hits[label] = (count, strength, meaning)
for call in _JNI_CALLS:
if re.search(re.escape(call).encode(), d):
self.jni_call_refs.append(call)
def _read_sections(self, endian, is64, shoff, shentsize, shnum, shstrndx):
d = self.data
sections = []
for i in range(shnum):
base = shoff + i * shentsize
try:
if is64:
name_off = struct.unpack_from(endian + "I", d, base)[0]
sh_type = struct.unpack_from(endian + "I", d, base + 4)[0]
sh_offset = struct.unpack_from(endian + "Q", d, base + 0x18)[0]
sh_size = struct.unpack_from(endian + "Q", d, base + 0x20)[0]
sh_link = struct.unpack_from(endian + "I", d, base + 0x28)[0]
sh_entsize = struct.unpack_from(endian + "Q", d, base + 0x38)[0]
else:
name_off = struct.unpack_from(endian + "I", d, base)[0]
sh_type = struct.unpack_from(endian + "I", d, base + 4)[0]
sh_offset = struct.unpack_from(endian + "I", d, base + 0x10)[0]
sh_size = struct.unpack_from(endian + "I", d, base + 0x14)[0]
sh_link = struct.unpack_from(endian + "I", d, base + 0x18)[0]
sh_entsize = struct.unpack_from(endian + "I", d, base + 0x24)[0]
except struct.error:
return
sections.append((name_off, sh_type, sh_offset, sh_size, sh_link, sh_entsize))
if shstrndx >= len(sections):
return
_, _, str_off, str_size, _, _ = sections[shstrndx]
if str_off + str_size > len(d):
return
shstr = d[str_off:str_off + str_size]
def sec_name(off):
end = shstr.find(b"\x00", off)
return shstr[off:end if end >= 0 else len(shstr)].decode("utf-8", "replace")
for name_off, sh_type, sh_offset, sh_size, sh_link, sh_entsize in sections:
# SHT_DYNSYM == 11, SHT_SYMTAB == 2
if sh_type not in (2, 11):
continue
if sh_link >= len(sections):
continue
_, _, dstr_off, dstr_size, _, _ = sections[sh_link]
if dstr_off + dstr_size > len(d) or sh_offset + sh_size > len(d):
continue
dstr = d[dstr_off:dstr_off + dstr_size]
entsize = sh_entsize or (24 if is64 else 16)
if entsize == 0:
continue
for off in range(sh_offset, sh_offset + sh_size, entsize):
try:
if is64:
st_name = struct.unpack_from(endian + "I", d, off)[0]
st_shndx = struct.unpack_from(endian + "H", d, off + 6)[0]
else:
st_name = struct.unpack_from(endian + "I", d, off)[0]
st_shndx = struct.unpack_from(endian + "H", d, off + 14)[0]
except struct.error:
break
if st_name >= len(dstr):
continue
end = dstr.find(b"\x00", st_name)
sname = dstr[st_name:end if end >= 0 else len(dstr)].decode("utf-8", "replace")
if not sname:
continue
self.dynsym.append((sname, st_shndx))
if sname.startswith("Java_"):
self.java_symbols.append(sname)
if sname == "JNI_OnLoad":
self.has_jni_onload = True
# Substring match, so a C implementation (bare name) and a C++
# one (mangled form) are both caught. In practice NEITHER is
# present: see the note on dynamic-registration in classify().
if "RegisterNatives" in sname and st_shndx == 0:
self.imports_register_natives = True
def summary(self):
return {
"name": self.name,
"size": len(self.data),
"is_elf": self.is_elf,
"arch": self.arch,
"elf_note": self.elf_note,
"dynsym_entries": len(self.dynsym),
"java_symbols": len(self.java_symbols),
"java_symbol_sample": self.java_symbols[:5],
"exports_JNI_OnLoad": self.has_jni_onload,
"imports_RegisterNatives": self.imports_register_natives,
"jni_call_refs": sorted(self.jni_call_refs),
"signature_hits": {k: {"count": v[0], "strength": v[1], "meaning": v[2]}
for k, v in sorted(self.sig_hits.items())},
}
# ---------------------------------------------------------------------------
# verdict
# ---------------------------------------------------------------------------
def classify(dexes, elves, so_bytes_total):
"""Turn measurements into a type guess plus the reasoning behind it.
Every rule below is a shape test, not a proof, and the confidence is reported
so a low-confidence Java2C reading is not mistaken for a confirmed one.
"""
evidence = []
verdict = "no-hardening-signal"
confidence = "low"
dex_metrics = [d for d in dexes if not d.parse_error]
total_methods = sum(d.total_methods for d in dex_metrics)
total_native = sum(d.native_methods for d in dex_metrics)
native_ratio = (float(total_native) / total_methods) if total_methods else 0.0
fully_native = sum(d.fully_native_classes for d in dex_metrics)
native_dominated = sum(d.native_dominated_classes for d in dex_metrics)
no_code = sum(d.no_code_methods for d in dex_metrics)
trivial_bodies = sum(d.trivial_bodies for d in dex_metrics)
body_methods = sum(d.body_methods for d in dex_metrics)
trivial_ratio = (float(trivial_bodies) / body_methods) if body_methods else 0.0
java_syms = sum(len(e.java_symbols) for e in elves)
register_natives_ref = any(e.imports_register_natives for e in elves)
has_jni_onload = any(e.has_jni_onload for e in elves)
# The reliable static signature of dynamic registration is the ABSENCE of
# Java_* symbols from a library that does export a JNI entry point. Looking
# for a literal RegisterNatives symbol does not work: NDK's C++ jni.h
# implements it as an inline member that calls through the function table,
# so no such symbol is emitted (measured: a real library exports JNI_OnLoad
# with zero Java_* and zero RegisterNatives symbols).
# Judged per library, not across the set: one library registering
# dynamically says nothing about another that exports static symbols.
dynamic_register_shape = any(e.has_jni_onload and not e.java_symbols for e in elves)
tool_markers = []
for e in elves:
for label, (count, strength, meaning) in e.sig_hits.items():
if strength == "strong":
tool_markers.append("%s(x%d) in %s" % (label, count, os.path.basename(e.name)))
medium_markers = []
for e in elves:
for label, (count, strength, meaning) in e.sig_hits.items():
if strength == "medium":
medium_markers.append("%s in %s" % (label, os.path.basename(e.name)))
# --- dex-layer shape -------------------------------------------------
java2c_dex_shape = (native_dominated >= 2) or (native_ratio >= 0.35 and total_native >= 10)
jni_sink_dex_shape = 0 < total_native <= 20 and native_ratio < 0.15
stub_shape = no_code > 0 or (body_methods >= 8 and trivial_ratio >= 0.5)
if dex_metrics:
evidence.append({
"claim": "dex layer: native declaration density %.2f%% (%d/%d)"
% (native_ratio * 100, total_native, total_methods),
"strength": "medium" if java2c_dex_shape else "strong",
"supports": "java2c" if java2c_dex_shape else "baseline",
"note": "High density with whole classes turned native is the Java2C shape. "
"It is NOT sufficient on its own: a hand-written JNI class looks "
"identical in dex, so the .so layer decides.",
})
evidence.append({
"claim": "dex layer: %d classes are native-dominated (>=70%% native of >=3 "
"named methods), %d fully native" % (native_dominated, fully_native),
"strength": "medium",
"supports": "java2c" if native_dominated >= 2 else "baseline",
"note": "A hand-written JNI boundary class normally keeps its Java half, so "
"whole classes turned native are the Java2C shape. Constructors are "
"excluded: they stay in Java even after a translation pass.",
})
if no_code:
evidence.append({
"claim": "dex layer: %d non-native, non-abstract methods have no code_item"
% no_code,
"strength": "strong",
"supports": "extraction-shell/vmp",
"note": "Structurally invalid for plain Java; expect a loader that "
"fills bodies at runtime, or a private-opcode converter.",
})
if trivial_ratio:
evidence.append({
"claim": "dex layer: %.0f%% of method bodies are return-a-constant stubs "
"(%d/%d)" % (trivial_ratio * 100, trivial_bodies, body_methods),
"strength": "medium",
"supports": "extraction-shell/vmp" if stub_shape else "baseline",
"note": "Approximate metric (see --help). Empty bodies are the signature "
"of an extraction shell or a VMP, not of Java2C.",
})
# --- native-layer shape ----------------------------------------------
if elves:
evidence.append({
"claim": "native layer: %d exported Java_* symbols across %d library(ies)"
% (java_syms, len(elves)),
"strength": "strong" if java_syms else "medium",
"supports": "java2c" if java_syms else "none",
"note": "Static-linkage form. Compare the count against the dex native "
"method count: a rough 1:1 map is the Java2C signature.",
})
if dynamic_register_shape:
dyn_libs = sorted(os.path.basename(e.name)
for e in elves if e.has_jni_onload and not e.java_symbols)
evidence.append({
"claim": "native layer: dynamic registration in %s "
"(JNI_OnLoad exported, zero Java_* symbols)"
% ", ".join(dyn_libs),
"strength": "strong",
"supports": "dynamic-registration",
"note": "Binding happens at runtime, so a symbol search comes back empty "
"and fails SILENTLY. Do not expect a RegisterNatives symbol "
"either: NDK's C++ jni.h makes it an inline member that calls "
"through the function table, so none is emitted.",
})
if register_natives_ref:
evidence.append({
"claim": "native layer: a RegisterNatives symbol is imported",
"strength": "weak",
"supports": "dynamic-registration",
"note": "Only a C-side implementation leaves this name; its absence "
"proves nothing.",
})
if has_jni_onload:
evidence.append({
"claim": "native layer: JNI_OnLoad exported",
"strength": "weak",
"supports": "jni-boundary-exists",
"note": "Present in nearly every JNI library. Holds no information "
"about which hardening was applied.",
})
if tool_markers:
evidence.append({
"claim": "native layer: toolchain markers found: %s" % ", ".join(tool_markers),
"strength": "strong",
"supports": "java2c",
"note": "A Dex-to-C toolchain leaves its own runtime symbols behind.",
})
if medium_markers:
evidence.append({
"claim": "native layer: runtime markers found: %s" % ", ".join(medium_markers),
"strength": "medium",
"supports": "java2c",
"note": "dcc runtime header names; suggestive, and easy to rename.",
})
# --- verdict ----------------------------------------------------------
if tool_markers and (java2c_dex_shape or java_syms):
verdict, confidence = "java2c", "high"
elif java2c_dex_shape and java_syms and total_native and \
(float(java_syms) / total_native) >= 0.5:
verdict, confidence = "java2c", "high"
elif java2c_dex_shape and medium_markers:
verdict, confidence = "java2c", "medium"
elif java2c_dex_shape and dynamic_register_shape:
# A Dex-to-C toolchain that hides its symbols (dcc's Application.mk ships
# -fvisibility=hidden) leaves exactly this: a JNI entry point and a
# registration table, with no Java_* name to search for.
verdict, confidence = "java2c", "medium"
elif stub_shape:
verdict, confidence = "extraction-shell-or-vmp", "medium"
elif java2c_dex_shape:
verdict, confidence = "java2c-suspect-needs-so", "low"
elif jni_sink_dex_shape:
verdict, confidence = "jni-sinking", "medium"
elif dex_metrics or elves:
verdict, confidence = "no-hardening-signal", "medium"
routes = {
"java2c": [
"Do NOT look for a decrypted DEX: none exists, at any point, in memory.",
"Read the C in the .so: each translated method is one function with a "
"JNIEnv*/jobject prefix.",
"Rebuild the call graph through the JNI reverse calls "
"(FindClass/GetMethodID/CallXxxMethod).",
],
"java2c-suspect-needs-so": [
"Supply the matching .so (--so) before acting: the dex shape alone "
"cannot separate Java2C from a hand-written JNI class.",
"If no .so carries the logic, re-read the dex shape as ordinary Java.",
],
"extraction-shell-or-vmp": [
"Dump memory and measure the dump with scripts/dex_dump_validate.py.",
"If the dump is mostly stubs, the bodies are filled at invocation time: "
"references/advanced-unpacking.md.",
"If bodies decode as private opcodes, that is a VMP - recovery cost "
"usually exceeds the task value.",
],
"jni-sinking": [
"The dex is still readable: locate the Java call site, then reverse "
"the one or few native functions.",
"references/native-and-so.md, then the algorithm tooling.",
],
"dynamic-registration": [
"Hook or find RegisterNatives to read the binding table at runtime.",
],
"no-hardening-signal": [
"Proceed with ordinary dex-level analysis.",
],
}
return {
"verdict": verdict,
"confidence": confidence,
"metrics": {
"dex_total_methods": total_methods,
"dex_native_methods": total_native,
"dex_native_ratio": round(native_ratio, 4),
"dex_fully_native_classes": fully_native,
"dex_native_dominated_classes": native_dominated,
"dex_no_code_methods": no_code,
"dex_trivial_ratio": round(trivial_ratio, 4),
"so_java_symbols": java_syms,
"so_dynamic_registration": dynamic_register_shape,
"so_bytes_total": so_bytes_total,
},
"evidence": evidence,
"route": routes.get(verdict, []),
"weak_criteria_warning": [
"JNI_OnLoad / RegisterNatives / libc++_shared.so / FindClass appear in "
"ordinary JNI libraries: a hit is not a hardening verdict.",
"A high native ratio is also produced by hand-written JNI code and by R8 "
"stripping Java bodies; only the .so layer separates these.",
"The trivial-body ratio is approximate and flags extraction shells and "
"VMPs, never Java2C.",
],
}
# ---------------------------------------------------------------------------
# input collection
# ---------------------------------------------------------------------------
def _looks_like_dex(data):
return data[:4] == b"dex\n"
def _looks_like_elf(data):
return data[:4] == b"\x7fELF"
def collect(args):
dex_blobs = []
elf_blobs = []
sources = []
def add_file(path):
try:
with open(path, "rb") as fp:
data = fp.read()
except OSError as exc:
print("warning: cannot read %s (%s)" % (path, exc), file=sys.stderr)
return
if _looks_like_dex(data):
dex_blobs.append((os.path.basename(path), data))
sources.append(path)
elif _looks_like_elf(data):
elf_blobs.append((os.path.basename(path), data))
sources.append(path)
else:
print("warning: %s is neither a dex nor an ELF image, skipped" % path,
file=sys.stderr)
for path in args.dex or []:
add_file(path)
for path in args.so or []:
add_file(path)
apks = list(args.apk or [])
for directory in (args.dir or []):
for root, _dirs, files in os.walk(directory):
for fn in files:
full = os.path.join(root, fn)
low = fn.lower()
if low.endswith(".apk"):
apks.append(full)
elif low.endswith((".dex", ".so")):
add_file(full)
for apk in apks:
if not os.path.exists(apk):
print("warning: apk not found: %s" % apk, file=sys.stderr)
continue
sources.append(apk)
try:
with zipfile.ZipFile(apk) as zf:
for info in zf.infolist():
low = info.filename.lower()
if not (low.endswith(".dex") or low.endswith(".so")):
continue
if info.file_size > 256 * 1024 * 1024:
print("warning: skipping oversized entry %s (%d B)"
% (info.filename, info.file_size), file=sys.stderr)
continue
data = zf.read(info)
label = os.path.basename(info.filename)
if _looks_like_dex(data):
dex_blobs.append((label, data))
elif _looks_like_elf(data):
elf_blobs.append((label, data))
except zipfile.BadZipFile:
print("warning: %s is not a readable zip/apk" % apk, file=sys.stderr)
return dex_blobs, elf_blobs, sources
def main(argv=None):
parser = argparse.ArgumentParser(
description="Collect the evidence that separates Java2C from an extraction "
"shell, a VMP and ordinary JNI sinking.",
epilog="Weak criteria (hits that do NOT establish a verdict on their own): "
"JNI_OnLoad, RegisterNatives, libc++_shared.so, FindClass/GetMethodID, "
"and a bare 'dcc' substring. The trivial-body ratio is advisory: it "
"flags extraction shells and VMPs, never Java2C.",
)
parser.add_argument("--apk", action="append", help="APK to inspect (repeatable)")
parser.add_argument("--dex", action="append", help="dex image (repeatable)")
parser.add_argument("--so", action="append", help="ELF library (repeatable)")
parser.add_argument("--dir", action="append",
help="directory scanned recursively for *.apk/*.dex/*.so")
parser.add_argument("--json", action="store_true", help="emit machine-readable JSON")
args = parser.parse_args(argv)
if not any([args.apk, args.dex, args.so, args.dir]):
parser.error("at least one of --apk/--dex/--so/--dir is required")
dex_blobs, elf_blobs, sources = collect(args)
if not dex_blobs and not elf_blobs:
print("error: no usable dex or ELF input found", file=sys.stderr)
return 1
dexes = [Dex(data, name) for name, data in dex_blobs]
elves = [Elf(data, name) for name, data in elf_blobs]
result = classify(dexes, elves, sum(len(d) for _n, d in elf_blobs))
report = {
"inputs": sources,
"dex": [d.summary() for d in dexes],
"native": [e.summary() for e in elves],
"classification": result,
}
if args.json:
print(json.dumps(report, indent=2, sort_keys=False))
return 0
print("=" * 72)
print("java2c_probe: hardening classification evidence")
print("=" * 72)
print("inputs: %d file(s)" % len(sources))
for src in sources:
print(" - %s" % src)
print("\n[dex layer]")
if not dexes:
print(" (no dex supplied: the dex-layer metrics are unmeasured, so a")
print(" Java2C reading cannot be supported or excluded from this input)")
for d in dexes:
s = d.summary()
if s["parse_error"]:
print(" %s: PARSE ERROR: %s" % (s["name"], s["parse_error"]))
continue
print(" %s: %d B, %d classes, %d methods, native %d (%.2f%%), "
"native-dominated classes %d, fully native %d"
% (s["name"], s["size"], s["classes"], s["total_methods"],
s["native_methods"], s["native_ratio"] * 100,
s["native_dominated_classes"], s["fully_native_classes"]))
print(" no-code methods %d, trivial bodies %d/%d (%.0f%%), header %s"
% (s["no_code_methods"], s["trivial_bodies"], s["body_methods"],
s["trivial_ratio"] * 100,
"OK" if s["header_size_matches"] else "MISMATCH: " + s["header_note"]))
print("\n[native layer]")
if not elves:
print(" (no .so supplied)")
for e in elves:
s = e.summary()
if not s["is_elf"]:
print(" %s: not ELF (%s)" % (s["name"], s["elf_note"]))
continue
print(" %s: %d B, %s, dynsym %d entries, Java_* %d, JNI_OnLoad %s, "
"RegisterNatives-import %s"
% (s["name"], s["size"], s["arch"], s["dynsym_entries"],
s["java_symbols"], s["exports_JNI_OnLoad"],
s["imports_RegisterNatives"]))
if s["elf_note"]:
print(" note: %s" % s["elf_note"])
if s["signature_hits"]:
for label, info in s["signature_hits"].items():
print(" [%s] %s x%d - %s"
% (info["strength"], label, info["count"], info["meaning"]))
print("\n[classification]")
print(" verdict : %s" % result["verdict"])
print(" confidence : %s" % result["confidence"])
print("\n evidence:")
for item in result["evidence"]:
print(" (%s) %s" % (item["strength"], item["claim"]))
print(" note: %s" % item["note"])
print("\n route:")
for line in result["route"]:
print(" - %s" % line)
print("\n weak criteria - a hit here is NOT a verdict:")
for line in result["weak_criteria_warning"]:
print(" ! %s" % line)
print("=" * 72)
return 0
if __name__ == "__main__":
sys.exit(main())
skills/apk-reverse/scripts/kernelsu_syscall_mask.py
+937
View File
@@ -0,0 +1,937 @@
#!/usr/bin/env python3
"""kernelsu_syscall_mask.py -- generate a configurable kernel-side syscall-masking scaffold.
What this generates, and the correction it exists to make
---------------------------------------------------------
There is a widespread misreading worth killing early: **a KernelSU module is a
Magisk-compatible userspace module and cannot change what a syscall returns.**
Its `post-fs-data.sh` / `service.sh` run as root in the normal world; nothing in
that format reaches the kernel's return path. Everything a module of that shape
can do is userspace work -- bind mounts, setting props, starting a daemon,
writing config.
Actually rewriting what `openat`/`read`/`stat` return for `/proc/self/maps` or
`/proc/self/status` needs one of exactly three things:
kpm KernelPatch / APatch Kernel Patch Module -- pointer replacement in the
syscall table (`fp_hook_syscalln`) or inline hooks (`hook_wrapN`).
Built with a bare-metal ARM64 toolchain (aarch64-none-elf-gcc) into a
relocatable .kpm ELF. Needs a KernelPatch-patched boot image.
lkm An out-of-tree kernel module. Needs kernel source matching the device's
exact version and vermagic, and a loader path.
ebpf A BPF program attached to a syscall tracepoint or kprobe. Needs a GKI
kernel with the tracing/BTF machinery -- 5.10+ in Android practice.
So this script emits a **module directory that is honest about which half is
loadable**: the userspace skeleton (module.prop + scripts + config) is real and
installable, and each kernel-side target is a template with its toolchain gate
stated in its own header. None of the kernel-side code is compiled or loaded
anywhere by this script.
What is measured vs unverified
------------------------------
`generate`, `gates` and `verify` are exercised on the host (see
docs/tool-verification/EXTENSION-kernel-weapons.md). **Whether the kernel-side
code compiles or loads is unverified** -- that needs a device with the matching
kernel and the matching toolchain, and the reference device is a 4.14.186 Magisk
build where all three routes are closed.
Examples
python kernelsu_syscall_mask.py gates
python kernelsu_syscall_mask.py generate --out work/sysmask --package <PKG>
python kernelsu_syscall_mask.py verify --dir work/sysmask
"""
import argparse
import json
import os
import re
import subprocess
import sys
# ---------------------------------------------------------------------------
# Kernel gate table. Every number here is either a toolchain fact or a
# documented platform threshold; the "how to check" column is what turns it
# from trivia into a test you run on your own device.
# ---------------------------------------------------------------------------
GATES = [
{
"route": "eBPF tracepoint/kprobe",
"gate": "GKI kernel 5.10+ (Android 12+)",
"why": "Android's BPF/tracing support as bpftrace-class tooling and "
"stackplz expect it is a GKI-era feature; BTF is required for "
"CO-RE style probes.",
"check": "uname -r -> expect 5.10.x or newer; ls /sys/kernel/btf/vmlinux",
"ref": "https://source.android.com/docs/core/architecture/kernel/bpf",
},
{
"route": "KernelPatch / APatch KPM",
"gate": "KernelPatch-patched boot image + kpm toolchain",
"why": "KPMs load through kpimg injected into the kernel image's payload "
"segment; building one needs a bare-metal ARM64 compiler "
"(aarch64-none-elf-gcc), not the NDK.",
"check": "ls /data/adb/ | grep -i apatch ; which aarch64-none-elf-gcc",
"ref": "https://github.com/bmax121/KernelPatch",
},
{
"route": "Out-of-tree LKM",
"gate": "Kernel source matching the device's exact version + vermagic",
"why": "A module built against a different kernel revision is refused at "
"load time on a mismatched vermagic, and the device's kernel "
"source is frequently not published at all.",
"check": "uname -r ; look for a matching kernel source tree for the device",
"ref": "https://source.android.com/docs/core/architecture/kernel/android-common",
},
{
"route": "seccomp-BPF (for contrast)",
"gate": "any modern kernel, but it is not this route",
"why": "seccomp can make a syscall FAIL (SECCOMP_RET_ERRNO/TRAP); it "
"cannot rewrite the CONTENT a successful read returns. It closes "
"doors, it does not paint them -- which is the whole requirement "
"for a spoofed /proc read.",
"check": "n/a -- this is a capability ceiling, not a version check",
"ref": "references/kernel-and-environment-hardening.md",
},
]
DEFAULT_RULES = [
{
"syscall": "openat",
"path": "/proc/self/maps",
"action": "deny",
"errno": "ENOENT",
"note": "hides the mapping list a userspace instrumentation agent "
"advertises itself in",
},
{
"syscall": "newfstatat",
"path": "/proc/self/maps",
"action": "fake_size",
"size": 0,
"note": "a stat that reports 0 bytes is as good as absent for a reader "
"that sizes its buffer first",
},
{
"syscall": "openat",
"path": "/proc/self/status",
"action": "filter_line",
"prefix": "TracerPid",
"note": "the write side owns this field; see "
"references/kernel-and-environment-hardening.md section 5",
},
{
"syscall": "read",
"path": "/proc/self/status",
"action": "filter_line",
"prefix": "TracerPid",
"note": "line filter applied to the read once the fd is open",
},
]
MODULE_PROP = """id={id}
name={name}
version={version}
versionCode=1
author=apk-reverse
description={description}
"""
POST_FS_DATA = """#!/system/bin/sh
# Runs at post-fs-data, as root, in the normal world.
# The kernel side (if you built and loaded one) owns the syscall table; this
# script's whole job is to make the configuration visible to it and to leave a
# record that the module actually ran. Do not put a syscall hook here: this is
# userspace, and nothing here can change what openat/read/stat return.
MODDIR=${{0%/*}}
CONF=$MODDIR/config/syscall_mask.json
LOG=/data/adb/{id}.log
if [ ! -f "$CONF" ]; then
echo "$(date) missing $CONF" >> "$LOG"
exit 0
fi
echo "$(date) post-fs-data: config present, $(wc -c < "$CONF") bytes" >> "$LOG"
# The KPM/LKM loader path is intentionally NOT invoked from here. Loading a
# kernel module is a boot-image-level decision with a bricking risk; do it once,
# by hand, and read the evidence file first.
exit 0
"""
SERVICE = """#!/system/bin/sh
# Runs in late_start service mode, as root. Kept minimal and non-blocking on
# purpose: a module that spins here delays boot on every start, and this one
# only reports state.
MODDIR=${{0%/*}}
CONF=$MODDIR/config/syscall_mask.json
LOG=/data/adb/{id}.log
if [ -r /proc/sys/kernel/version ]; then
echo "$(date) service: kernel $(uname -r)" >> "$LOG"
fi
echo "$(date) service: rules=$(grep -c '"syscall"' "$CONF" 2>/dev/null)" >> "$LOG"
exit 0
"""
UNINSTALL = """#!/system/bin/sh
# Uninstall hook. Deliberately does not attempt to unload a kernel module: if one
# is loaded, unloading it is a separate deliberate act with its own risk, and a
# silent unload during package removal is how a device ends up in a boot loop.
rm -f /data/adb/{id}.log
exit 0
"""
CUSTOMIZE = """#!/system/bin/sh
# KernelSU/Magisk install-time script. Kept as a no-op so the module always
# installs; see the module README for why nothing is set up here.
exit 0
"""
KPM_C = r'''// SPDX-License-Identifier: GPL-2.0
/*
* syscall_mask.c -- config-driven syscall return masking, KernelPatch/APatch KPM.
*
* Generated by skills/apk-reverse/scripts/kernelsu_syscall_mask.py
*
* STATUS: TEMPLATE, NOT COMPILED AND NOT LOADED. Every hook below follows the
* KernelPatch KPM interface as documented in public KPM development write-ups
* (KPM_NAME/KPM_INIT/KPM_EXIT sections, kfunc_def symbol resolution without
* `extern`, fp_hook_syscalln pointer replacement, hook_fargs4_t callbacks with
* args->skip_origin / args->ret). VERIFY EVERY NAME AND SIGNATURE against the
* kpmodule.h of the KernelPatch revision you build against -- this file was not
* compiled anywhere, and an interface that has drifted produces a module that
* fails to load, or worse, loads and faults.
*
* Build (bare-metal ARM64, NOT the NDK):
* aarch64-none-elf-gcc -O2 -fno-stack-protector -c syscall_mask.c
* ... link into a relocatable ELF per the KernelPatch module docs
* Load: push the .kpm and add it through the APatch app's KPM manager.
*
* Design notes that are load-bearing rather than stylistic:
* - Callbacks run on EVERY syscall of their number, system-wide. The first
* thing each one does is reject the overwhelmingly common case, because a
* hook that does real work on every write is how a device locks up.
* - Inline hooks (hook_wrapN) rewrite the target's prologue. On an LTO kernel
* the exported symbol is frequently NOT the call site (the call got inlined),
* so an inline hook installs cleanly and never fires. Pointer replacement in
* the syscall table (fp_hook_syscalln) does not have that failure mode,
* because the syscall entry path must go through the table.
* - Never sleep in these callbacks except where the syscall itself runs in
* process context and the copy helper is the sleeping variant.
*/
#include <linux/types.h>
#include <linux/string.h>
#include <linux/errno.h>
#include <kpmodule.h>
KPM_NAME("{id}");
KPM_VERSION("{version}");
KPM_LICENSE("GPL");
KPM_AUTHOR("apk-reverse");
KPM_DESCRIPTION("{description}");
/* kfunc declarations must NOT carry `extern`: an extern declaration compiles to
* an undefined reference (*UND*) and the loader rejects the module with
* "unknown symbol". Letting it be a tentative definition allocates the slot in
* .bss and the loader fills it in. Measured and documented in public KPM work. */
int kfunc_def(strncpy_from_user)(char *dst, const char __user *src, long count);
/* ------------------------------------------------------------------ rules --
* Generated from config/syscall_mask.json. Keep this table in sync by
* re-running the generator rather than by hand-editing it.
*/
#define MASK_DENY 1 /* return -err before the real syscall runs */
#define MASK_FAKE_SIZE 2 /* openat succeeds, stat reports a bogus size */
#define MASK_FILTER_LINE 3 /* strip lines with this prefix from read() */
struct mask_rule {
const char *path; /* userspace path to match, NULL = any */
const char *prefix; /* line prefix for MASK_FILTER_LINE, else NULL */
int action;
int err; /* -errno for MASK_DENY */
long value; /* size for MASK_FAKE_SIZE */
};
static const struct mask_rule rules[] = {
{RULE_TABLE}
};
#define NRULES (sizeof(rules) / sizeof(rules[0]))
/* A bounded, allocation-free matcher. The path is copied in first, so the
* comparison never touches userspace after the initial copy -- touching
* userspace memory twice is how a hook races with the caller's own unmapping. */
static int copy_and_match(const char __user *upath, char *kbuf, unsigned long n,
const char **matched)
{{
long copied;
unsigned long i;
if (!upath || n == 0)
return 0;
if (!kfunc(strncpy_from_user))
return 0; /* symbol unresolved: fail open */
copied = kfunc(strncpy_from_user)(kbuf, upath, (long)(n - 1));
if (copied <= 0)
return 0;
kbuf[copied] = '\0';
for (i = 0; i < NRULES; i++) {{
if (!rules[i].path)
continue;
if (strstr(kbuf, rules[i].path)) {{
*matched = rules[i].path;
return rules[i].action;
}}
}}
return 0;
}}
/* ------------------------------------------------------------------ openat --
* The path argument is arg2 on arm64 (dfd, path, flags, mode). If the path is
* unreadable, the rule is skipped rather than guessed at.
*/
static void before_openat(hook_fargs4_t *args, void *udata)
{{
const char __user *upath = (const char __user *)syscall_argn(args, 1);
char kbuf[256];
const char *matched = 0;
int action;
(void)udata;
action = copy_and_match(upath, kbuf, sizeof(kbuf), &matched);
if (action == MASK_DENY)
args->ret = MASK_DENY_ERRNO;
if (action == MASK_FAKE_SIZE)
args->skip_origin = 0; /* let it open; the stat is spoofed */
}}
/* --------------------------------------------------------------- newfstatat --
* arm64 has no plain `stat` syscall; readers go through newfstatat (or fstatat64
* on 32-bit). A zero size is what makes a reader that sizes its buffer first
* decide there is nothing to read.
*/
static void after_newfstatat(hook_fargs4_t *args, void *udata)
{{
const char __user *upath = (const char __user *)syscall_argn(args, 1);
char kbuf[256];
const char *matched = 0;
(void)udata;
if (copy_and_match(upath, kbuf, sizeof(kbuf), &matched) != MASK_FAKE_SIZE)
return;
/* The struct stat* is arg2; zeroing st_size is the whole point. Field
* offsets are ABI-specific -- resolve them against the target's headers
* rather than trusting an offset copied from a different architecture. */
/* TODO(verify-on-device): zero the size field of the struct at arg2. */
}}
/* -------------------------------------------------------------------- read --
* Line filtering happens on the read side, once the fd is already open. Doing
* it here rather than at openat is what lets a reader that opens the file
* before the filter is installed still be covered.
*/
static void before_read(hook_fargs4_t *args, void *udata)
{{
long count = (long)syscall_argn(args, 2);
(void)udata;
if (count <= 0 || count > 8192)
return; /* first filter: the common case */
/* TODO(verify-on-device): identify the fd's path, copy the buffer, rewrite
* it in place minus the filtered lines, and adjust the returned length. */
}}
/* ------------------------------------------------------------------ init -----
* Registration order matters: install the syscall hooks first, then enable the
* rules. If the module is unloaded between the two, the hooks must already be
* gone.
*/
static long mask_init(const char *args, const struct kernel_patch_funcs *kp)
{{
(void)args;
(void)kp;
if (!kfunc(strncpy_from_user)) {{
/* Failing loudly here is deliberate: a half-armed hook set that cannot
* read a path would pass every check through, which looks like success
* and is not. */
return -EINVAL;
}}
/* fp_hook_syscalln(__NR_openat, before_openat, 0, 0); */
/* fp_hook_syscalln(__NR_newfstatat, before_newfstatat, 0, 0); */
/* fp_hook_syscalln(__NR_read, before_read, 0, 0); */
/* TODO(verify-on-device): the argument lists above follow the public
* examples; confirm arity against your kpmodule.h before enabling them. */
return 0;
}}
static long mask_exit(void *udata)
{{
(void)udata;
/* Unhook in the exact reverse order of registration. */
return 0;
}}
KPM_INIT(mask_init);
KPM_EXIT(mask_exit);
'''
LKM_C = r'''// SPDX-License-Identifier: GPL-2.0
/*
* syscall_mask.c -- out-of-tree LKM variant of the same idea.
*
* STATUS: TEMPLATE, NOT COMPILED. Worse than the KPM variant in one specific
* way worth understanding before choosing it: to replace a syscall you must
* reach the syscall table, and `kallsyms_lookup_name` stopped being exported in
* Linux 5.7. On pre-5.7 kernels a direct call works; after that you need a
* kprobe to recover the address, or an explicit kernel export. The device's own
* kernel source must also match its exact vermagic, and an out-of-tree module
* built against a mismatched tree is refused at insmod.
*
* This is genuine kernel development with a bricking risk. It is the most
* expensive of the three routes and the one least likely to be worth it.
*/
#include <linux/module.h>
#include <linux/kernel.h>
#include <linux/syscalls.h>
#include <linux/uaccess.h>
#include <linux/version.h>
MODULE_LICENSE("GPL");
MODULE_DESCRIPTION("{description}");
MODULE_AUTHOR("apk-reverse");
/*
* Pre-5.7: kallsyms_lookup_name() is exported, so the table pointer is
* recoverable directly. 5.7+: NOT exported -- you must either carry a kprobe to
* find the symbol or patch the kernel's export list. There is no portable
* version of this, which is the honest reason this route is rated last.
*/
#if LINUX_VERSION_CODE < KERNEL_VERSION(5, 7, 0)
static unsigned long *sys_call_table_addr(void)
{
return (unsigned long *)kallsyms_lookup_name("sys_call_table");
}
#else
static unsigned long *sys_call_table_addr(void)
{
/* TODO(verify-on-device): recover via kprobe on kallsyms_lookup_name, or
* enable an explicit export in your own kernel build. */
return NULL;
}
#endif
static int __init mask_init(void)
{
unsigned long *table = sys_call_table_addr();
if (!table) {
pr_err("syscall_mask: cannot locate sys_call_table on this kernel\n");
return -EINVAL;
}
pr_info("syscall_mask: sys_call_table at %px\n", table);
/* TODO(verify-on-device): save the original pointer, write CR0 (or use
* set_memory_rw) to make the table writable, swap in your handler. Getting
* the write-protect dance wrong takes the device down. */
return 0;
}
static void __exit mask_exit(void)
{
/* TODO(verify-on-device): restore the saved pointers. */
}
module_init(mask_init);
module_exit(mask_exit);
'''
EBPF_C = r'''// SPDX-License-Identifier: GPL-2.0 OR BSD-2-Clause
/*
* syscall_mask.bpf.c -- eBPF probe template: filter sys_enter_openat, then
* rewrite the result.
*
* STATUS: TEMPLATE, NOT COMPILED AND NOT ATTACHED. Not runnable on a 4.14
* kernel: see the gate table in the module README (GKI 5.10+ in Android
* practice, plus BTF for CO-RE style probes).
*
* READ THIS BEFORE COUNTING ON THE RESULT REWRITE. There are two different
* attachment points and they are not interchangeable:
*
* - A *tracepoint* (syscalls/sys_enter_openat) is an observation point. It can
* read the arguments and it can emit events, and it CANNOT change the
* syscall's return value. A tracepoint-only program gives you visibility,
* not spoofing.
* - A *kprobe* on the syscall's entry (or a kretprobe on its exit) can modify
* a register with bpf_override_return() -- but only for functions flagged
* ALLOW_ERROR_INJECTION, which kernel syscall entry points are not, in
* general. On many kernels the helper is refused for exactly this target.
*
* So the honest eBPF shape for this job is: observe with a tracepoint, and do
* the rewrite in a place that actually owns the return value. What follows is
* the observation half done properly, plus the rewrite half marked for the
* device-side verification it needs.
*/
#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
char LICENSE[] SEC("license") = "GPL";
#define MAX_PATH 256
#define TARGET_LEN 15 /* strlen("/proc/self/maps") */
struct event {
__u32 pid;
__u32 uid;
__s32 dfd;
__s64 ret;
char path[MAX_PATH];
char comm[16];
};
/* /proc/self/status is 17; kept as a separate constant so the two rules can be
* toggled independently. */
#define TARGET_STATUS_LEN 17
struct {
__uint(type, BPF_MAP_TYPE_RINGBUF);
__uint(max_entries, 1 << 24);
} events SEC(".maps");
/* Filter the target package's processes. Populated from userspace with the
* uids the app can run as; an empty map means "all processes", which is a
* deliberate default because a filter that silently matches nothing looks
* exactly like a probe that is not firing. */
struct {
__uint(type, BPF_MAP_TYPE_HASH);
__uint(max_entries, 64);
__type(key, __u32); /* uid */
__type(value, __u8);
} target_uids SEC(".maps");
static __always_inline int want(void)
{
__u32 uid = bpf_get_current_uid_gid() & 0xffffffff;
__u8 *hit = bpf_map_lookup_elem(&target_uids, &uid);
/* Empty map -> observe everything. Populated map -> only listed uids. */
return hit || (bpf_map_lookup_elem(&target_uids, &uid) == 0);
}
SEC("tracepoint/syscalls/sys_enter_openat")
int tp_sys_enter_openat(struct trace_event_raw_sys_enter *ctx)
{
struct event *e;
const char *upath = (const char *)ctx->args[1];
if (!want())
return 0;
e = bpf_ringbuf_reserve(&events, sizeof(*e), 0);
if (!e)
return 0;
e->pid = bpf_get_current_pid_tgid() >> 32;
e->uid = bpf_get_current_uid_gid() & 0xffffffff;
e->dfd = (__s32)ctx->args[0];
e->ret = 0;
bpf_get_current_comm(&e->comm, sizeof(e->comm));
bpf_probe_read_user_str(&e->path, sizeof(e->path), upath);
bpf_ringbuf_submit(e, 0);
return 0;
}
/*
* The result-rewrite half. Two dead ends are recorded here on purpose so they
* are not re-derived:
*
* 1. bpf_override_return() only works on functions marked
* ALLOW_ERROR_INJECTION. A raw syscall entry is not one of them, so the
* helper is typically refused for this target -- check the return value
* and the verifier message rather than assuming the rewrite happened.
* 2. A tracepoint cannot rewrite at all. If the rewrite is essential, the
* route is a kprobe plus a kernel that permits the override, or one of the
* KPM/LKM routes; eBPF's strength here is observation.
*/
SEC("kprobe/do_sys_openat2")
int BPF_KPROBE(kp_do_sys_openat2, int dfd, const char *filename)
{
char buf[64];
if (bpf_probe_read_user_str(&buf, sizeof(buf), filename) <= 0)
return 0;
if (buf[0] == '/' && buf[1] == 'p' && buf[2] == 'r') {
/* TODO(verify-on-device): bpf_override_return(ctx, -ENOENT) here only
* after confirming ALLOW_ERROR_INJECTION on this target kernel. */
}
return 0;
}
'''
KPM_MAKEFILE = """# aarch64-none-elf-gcc is a BARE-METAL ARM64 compiler, not the Android NDK
# toolchain. The NDK targets Android userspace and will not produce a loadable
# KPM. STATUS: untested -- this toolchain is not present on a Magisk-only device.
CROSS ?= aarch64-none-elf-
CC := $(CROSS)gcc
OBJCOPY := $(CROSS)objcopy
TARGET := {id}
KPM_DIR ?= ../KernelPatch
CFLAGS := -O2 -fno-stack-protector -fno-pic -fno-builtin \\
-I$(KPM_DIR)/kernel/include -I$(KPM_DIR)/include \\
-Wall -Wno-unused-variable
all: $(TARGET).kpm
$(TARGET).kpm: src/{id}.c
\t$(CC) $(CFLAGS) -c $< -o $@.o
\t$(OBJCOPY) -O binary --only-section=.text $@.o /dev/null 2>/dev/null || true
\t@echo "NOTE: linking follows the KernelPatch module format; see the docs"
\t@echo " for the current link step before trusting this target."
clean:
\trm -f *.o *.kpm
.PHONY: all clean
"""
MODULE_README = """# {name} -- syscall-return masking scaffold
Generated by `skills/apk-reverse/scripts/kernelsu_syscall_mask.py`.
**Read this first: the userspace module in this directory cannot change what a
syscall returns.** It is a KernelSU/Magisk-compatible module: its scripts run as
root in the normal world. Its only jobs are to carry the configuration, to
record that it ran, and to be a place to put the module metadata. The kernel
side is a separate artifact with a separate toolchain and a separate risk.
## What is here
| Path | Status |
|---|---|
| `module.prop`, `customize.sh`, `post-fs-data.sh`, `service.sh` | real, installable module skeleton |
| `config/syscall_mask.json` | the rule set, read by the kernel side |
| `kpm/{id}.c`, `kpm/Makefile` | APatch/KernelPatch template -- **not compiled, not loaded** |
| `lkm/{id}.c` | out-of-tree module template -- **not compiled, not loaded** |
| `ebpf/syscall_mask.bpf.c` | eBPF probe template -- **not compiled, not attached** |
## Gate table -- check your device before choosing a route
| Route | Gate | How to check | Why |
|---|---|---|---|
{gates}
## Reference device, measured
```
$ adb shell 'uname -r; cat /proc/version'
4.14.186+
Linux version 4.14.186+ (nobody@android-build) (Android (6443078 based on r383902)
clang version 11.0.1 ...) #1 SMP PREEMPT Wed Mar 30 23:32:42 CST 2022
```
On that device **every kernel-side route above is closed**:
- eBPF: kernel 4.14 is well below the 5.10 gate.
- KPM: no KernelPatch-patched image, and no `aarch64-none-elf-gcc` on the host;
installing APatch means patching the boot image, which is a bricking risk.
- LKM: no kernel source for the device, so no matching vermagic is possible.
That is a **route decision, not a failure** -- and it is why every kernel-side
file here is labelled a template. Nothing in this directory was loaded on
anything.
## If you do take a kernel route
Do it once, deliberately, and read the evidence first:
1. Confirm the gate in the table above on the actual device.
2. Build the kernel side on a machine that has the right toolchain.
3. **Back up the boot image and know the recovery path before loading anything.**
A kernel module that faults takes the device down before you can read why.
4. Load it by hand, watch `dmesg`, and keep the previous boot image in hand.
5. Only then wire it into boot, and only if the load is repeatable.
`post-fs-data.sh` in this scaffold deliberately does **not** load anything. That
is not an omission.
"""
def _render(template, **kw):
"""Fill a template's {placeholders} without str.format().
Braces are the common case in C, so using format() here would mean doubling
every one of them -- and the doubled-brace convention fails silently in the
other direction the moment somebody next edits the C by hand. `{{` is
unescaped and only the named placeholders are substituted, so ordinary C
braces pass through untouched.
"""
text = template.replace("{{", "{").replace("}}", "}")
for key, value in kw.items():
text = text.replace("{%s}" % key, str(value))
return text
def _rule_table_c(rules):
"""Render the JSON rule set as a C table."""
errno_map = {"ENOENT": 2, "EACCES": 13, "EINVAL": 22, "EPERM": 1}
action_map = {"deny": "MASK_DENY", "fake_size": "MASK_FAKE_SIZE",
"filter_line": "MASK_FILTER_LINE"}
lines = []
for rule in rules:
path = rule.get("path")
prefix = rule.get("prefix")
action = action_map.get(rule.get("action"), "MASK_DENY")
err = errno_map.get(rule.get("errno", "ENOENT"), 2)
value = rule.get("size", 0)
lines.append(' { %s, %s, %s, -%d, %d },'
% ('"%s"' % path if path else "0",
'"%s"' % prefix if prefix else "0",
action, err, value))
return "\n".join(lines) if lines else " /* no rules configured */"
def cmd_gates(args):
"""Report the kernel-route gate table, and probe this host where it can be probed."""
print("== kernel-side routes and their gates ==\n")
for g in GATES:
print(" %-28s %s" % (g["route"], g["gate"]))
print(" %-28s why: %s" % ("", g["why"]))
print(" %-28s check: %s" % ("", g["check"]))
print()
print("== host probe (what can be checked from here) ==\n")
cc = None
import shutil
for cand in ("aarch64-none-elf-gcc", "aarch64-linux-gnu-gcc", "clang"):
found = shutil.which(cand)
if found:
cc = (cand, found)
break
if cc:
print(" bare-metal ARM64-ish compiler: %s -> %s" % cc)
if cc[0] == "clang":
print(" note: clang can target aarch64-none-elf, so this is "
"potentially usable for the KPM route")
else:
print(" bare-metal ARM64 compiler: NOT FOUND")
print(" the KPM route cannot be built from this host as configured")
for name in ("ndk-build", "make"):
print(" %-12s %s" % (name + ":", shutil.which(name) or "NOT FOUND"))
print("\n A kernel source tree, a patched boot image and the device's own "
"kernel headers cannot be checked from the host; they are device facts.")
print(" Run the `check` command for each route on the device before "
"believing any of this applies to it.")
return 0
def cmd_generate(args):
out = args.out
rules = DEFAULT_RULES
if args.rules:
with open(args.rules, encoding="utf-8") as fh:
rules = json.load(fh)
desc = args.description
os.makedirs(os.path.join(out, "config"), exist_ok=True)
written = []
def put(rel, text, mode=None):
path = os.path.join(out, rel)
os.makedirs(os.path.dirname(path), exist_ok=True)
with open(path, "w", encoding="utf-8", newline="\n") as fh:
fh.write(text)
if mode is not None:
try:
os.chmod(path, mode)
except OSError:
pass
written.append(rel)
put("module.prop", _render(MODULE_PROP,
id=args.id, name=args.name, version=args.version,
description=desc))
put("customize.sh", CUSTOMIZE, 0o755)
put("post-fs-data.sh", _render(POST_FS_DATA, id=args.id), 0o755)
put("service.sh", _render(SERVICE, id=args.id), 0o755)
put("uninstall.sh", _render(UNINSTALL, id=args.id), 0o755)
put("config/syscall_mask.json", json.dumps(
{"version": 1, "id": args.id, "package": args.package,
"enabled": True, "note": "read by the kernel side, not by the shell "
"scripts -- see the module README",
"rules": rules}, indent=2) + "\n")
want = set(args.targets.split(","))
if "kpm" in want:
body = _render(KPM_C, id=args.id, version=args.version, description=desc,
RULE_TABLE=_rule_table_c(rules))
put("kpm/%s.c" % args.id, body)
put("kpm/Makefile", _render(KPM_MAKEFILE, id=args.id))
if "lkm" in want:
put("lkm/%s.c" % args.id, _render(LKM_C, id=args.id, description=desc))
if "ebpf" in want:
put("ebpf/syscall_mask.bpf.c", EBPF_C)
gate_rows = "\n".join("| %s | %s | `%s` | %s |"
% (g["route"], g["gate"], g["check"], g["why"])
for g in GATES)
put("README.md", _render(MODULE_README, name=args.name, gates=gate_rows,
id=args.id))
print("generated module scaffold in %s" % out)
for rel in written:
size = os.path.getsize(os.path.join(out, rel))
print(" %-28s %7d B" % (rel, size))
print()
print("STATUS: the userspace module is installable; every kernel-side file is "
"an UNVERIFIED template.")
print(" No kernel code was compiled or loaded. Run `gates` and read the "
"device README")
print(" section before assuming any route is open on your target.")
return 0
def _strip_c_comments(text):
"""Remove /* */ and // comments before any token-level check.
Not cosmetic: the first version of this check reported a false positive
because the template's own comment explains why `extern` is forbidden, and a
naive `extern.*kfunc_def` regex happily matched the prose and then ran on to
the real declaration's semicolon. A linter that reads comments as code
produces exactly the kind of report nobody can trust.
"""
text = re.sub(r"/\*.*?\*/", "", text, flags=re.S)
return re.sub(r"//[^\n]*", "", text)
def cmd_verify(args):
"""Read-only checks over a generated directory. Everything here is local."""
d = args.dir
problems = []
print("== verifying %s ==" % d)
prop = os.path.join(d, "module.prop")
if not os.path.isfile(prop):
problems.append("module.prop missing")
else:
fields = {}
with open(prop, encoding="utf-8") as fh:
for line in fh:
if "=" in line:
k, v = line.rstrip("\n").split("=", 1)
fields[k] = v
for req in ("id", "name", "version", "versionCode"):
if req not in fields:
problems.append("module.prop missing required field %r" % req)
if "id" in fields and not re.fullmatch(r"[a-zA-Z][a-zA-Z0-9._-]*", fields["id"]):
problems.append("module.prop id %r is not a valid module id" % fields["id"])
print(" module.prop: %s" % ", ".join("%s=%s" % kv for kv in sorted(fields.items())))
conf = os.path.join(d, "config", "syscall_mask.json")
if not os.path.isfile(conf):
problems.append("config/syscall_mask.json missing")
else:
try:
with open(conf, encoding="utf-8") as fh:
doc = json.load(fh)
rules = doc.get("rules", [])
known = {"deny", "fake_size", "filter_line"}
for i, r in enumerate(rules):
if r.get("action") not in known:
problems.append("rule %d has unknown action %r" % (i, r.get("action")))
if not r.get("syscall"):
problems.append("rule %d names no syscall" % i)
print(" config: %d rule(s), actions=%s"
% (len(rules), sorted({r.get("action") for r in rules})))
except (ValueError, OSError) as exc:
problems.append("config/syscall_mask.json unreadable: %s" % exc)
for rel, want_exec in (("post-fs-data.sh", True), ("service.sh", True),
("customize.sh", True), ("uninstall.sh", True)):
p = os.path.join(d, rel)
if not os.path.isfile(p):
problems.append("%s missing" % rel)
continue
if want_exec and not os.access(p, os.X_OK):
# Not fatal on a filesystem without an exec bit; the installer sets it.
print(" note: %s is not executable here (the installer chmods it)" % rel)
for rel in ("kpm", "lkm", "ebpf"):
p = os.path.join(d, rel)
if os.path.isdir(p):
print(" %-4s target present: %s"
% (rel, ", ".join(sorted(os.listdir(p)))))
print(" status: UNVERIFIED TEMPLATE -- not compiled, not loaded")
# The one static check worth doing on the C: the KPM interface has a
# documented trap (an extern kfunc declaration compiles to *UND* and the
# loader refuses the module), so catch it here rather than on a device.
kpm_dir = os.path.join(d, "kpm")
if os.path.isdir(kpm_dir):
for name in sorted(os.listdir(kpm_dir)):
if not name.endswith(".c"):
continue
with open(os.path.join(kpm_dir, name), encoding="utf-8") as fh:
text = fh.read()
code = _strip_c_comments(text)
if re.search(r"\bextern\b[^;]*\bkfunc_def\b", code):
problems.append("%s declares kfunc_def with `extern`, which the "
"loader rejects as an unknown symbol" % name)
for macro in ("KPM_NAME", "KPM_INIT", "KPM_EXIT"):
if macro not in code:
problems.append("%s is missing %s" % (name, macro))
print(" kpm/%s: KPM_* lifecycle macros present, kfunc_def not extern"
% name)
print()
if problems:
print("== %d problem(s) ==" % len(problems))
for p in problems:
print(" - %s" % p)
return 1
print("== 0 problem(s): structure is consistent ==")
print(" (this says nothing about whether the kernel side compiles or loads)")
return 0
def main(argv=None):
parser = argparse.ArgumentParser(
prog="kernelsu_syscall_mask.py",
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter)
sub = parser.add_subparsers(dest="cmd")
p = sub.add_parser("gates", help="print the kernel-route gate table and probe this host",
description="The gate table, plus whatever part of it can be "
"checked from this machine. Device facts still have "
"to be checked on the device.")
p.set_defaults(func=cmd_gates)
p = sub.add_parser("generate", help="write the module scaffold",
description="Generate a KernelSU/APatch-compatible module "
"directory. The userspace half is installable; the "
"kernel half is a template.")
p.add_argument("--out", required=True, help="directory to write into")
p.add_argument("--id", default="syscall-mask", help="module id / artifact name")
p.add_argument("--name", default="Syscall Mask", help="human-readable module name")
p.add_argument("--version", default="0.1.0", help="module version string")
p.add_argument("--description", default="config-driven syscall return masking "
"(openat/read/stat on /proc/self/*)",
help="module description")
p.add_argument("--package", default="<PKG>", help="target package placeholder")
p.add_argument("--rules", default=None,
help="JSON file with a {rules:[...]} rule set (default: built-in)")
p.add_argument("--targets", default="kpm,lkm,ebpf",
help="comma-separated kernel targets to emit (kpm,lkm,ebpf)")
p.set_defaults(func=cmd_generate)
p = sub.add_parser("verify", help="read-only consistency checks over a generated dir",
description="Check module.prop fields, the rule JSON, and the "
"static traps in the kernel template. Says nothing "
"about compilation or loading.")
p.add_argument("--dir", required=True, help="a directory from `generate`")
p.set_defaults(func=cmd_verify)
args = parser.parse_args(argv)
if not getattr(args, "cmd", None):
parser.print_help()
return 2
return args.func(args)
if __name__ == "__main__":
sys.exit(main())
skills/apk-reverse/scripts/protobuf_decode_raw.py
+1297
View File
File diff suppressed because it is too large Load Diff
skills/apk-reverse/scripts/repack.py
+861 -32
View File
@@ -159,6 +159,474 @@ def ensure_keystore(ks_path, alias, password, keytool):
return pwd
# ---------------------------------------------------------------------------
# Split APK / App Bundle sets
#
# A store build is often NOT one file. An App Bundle turns into
# base.apk + split_config.arm64_v8a.apk + split_config.xxhdpi.apk + ...
# and every member carries the same package name and its own signature. Two
# consequences, and they are the reason this section exists:
#
# * Signing only the base leaves the set disagreeing about its certificate.
# `pm install-multiple` then refuses the whole set, with a signature error
# that points at the base rather than at the member you did not sign.
# * "Just merge them into one APK" is free only for splits that carry CODE or
# NATIVE LIBRARIES. A split that carries RESOURCES cannot be folded in
# without merging `resources.arsc` -- rewriting the table, its global string
# pool and its type/entry offsets -- which is a resource-compiler job, not a
# byte-level one. A wrongly merged arsc produces an APK that installs and
# then renders the wrong thing, or fails on the first resource lookup.
#
# So the job here is to say which of the two routes is legal for a given set and
# then do that one, instead of always emitting the same artifact.
# ---------------------------------------------------------------------------
ABI_SPLIT_NAMES = ('armeabi', 'armeabi_v7a', 'arm64_v8a', 'x86', 'x86_64',
'riscv64', 'mips', 'mips64')
DENSITY_SPLIT_NAMES = ('ldpi', 'mdpi', 'tvdpi', 'hdpi', 'xhdpi', 'xxhdpi',
'xxxhdpi', 'anydpi', 'nodpi')
# A split is required to carry an `resources.arsc`, so an almost-empty table is
# normal and must not be mistaken for real resources. Measured on a real set:
# the density splits that hold nothing but a placeholder table are 40 bytes,
# while the ones that actually own drawables are 9-15 KB.
EMPTY_ARSC_BYTES = 1024
def find_apks(root):
"""Every .apk at or under `root`.
Recursive on purpose: `adb pull <dir> <dest>` creates `<dest>/<dir>/`, so a
pulled set arrives one level deeper than the caller expects (measured).
"""
if os.path.isfile(root):
return [os.path.abspath(root)]
found = []
for dirpath, dirnames, filenames in os.walk(root):
dirnames[:] = [d for d in dirnames if not d.startswith('.')]
for name in sorted(filenames):
if name.lower().endswith('.apk'):
found.append(os.path.abspath(os.path.join(dirpath, name)))
return sorted(found)
def axml_string_pool(blob):
"""Decode the string pool of a binary AndroidManifest.xml. No dependencies.
Only the pool is decoded, not the element tree: every name this script needs
(`package`, `split`, `configForSplit`, `isSplitRequired`) is a pool entry, and
a hand-rolled element walk is where such parsers usually go wrong. The pool
is also what makes equal-length hex patching of a manifest possible later.
"""
import struct
if len(blob) < 36 or struct.unpack_from('<H', blob, 0)[0] != 0x0003:
return None # not binary XML
if struct.unpack_from('<H', blob, 8)[0] != 0x0001:
return None # no string pool where one must be
hdr_size = struct.unpack_from('<H', blob, 10)[0]
count, _styles, flags, strings_start, _styles_start = struct.unpack_from(
'<IIIII', blob, 16)
if not 0 < count <= 1000000:
return None
offsets = struct.unpack_from('<%dI' % count, blob, 8 + hdr_size)
utf8 = bool(flags & (1 << 8))
base = 8 + strings_start
out = []
for off in offsets:
p = base + off
if p >= len(blob):
return None
try:
if utf8:
n = blob[p]
p += 1
if n & 0x80:
n = ((n & 0x7F) << 8) | blob[p]
p += 1
m = blob[p]
p += 1
if m & 0x80:
m = ((m & 0x7F) << 8) | blob[p]
p += 1
out.append(blob[p:p + m].decode('utf-8', 'replace'))
else:
n = struct.unpack_from('<H', blob, p)[0]
p += 2
if n & 0x8000:
n = ((n & 0x7FFF) << 16) | struct.unpack_from('<H', blob, p)[0]
p += 2
out.append(blob[p:p + n * 2].decode('utf-16-le', 'replace'))
except (IndexError, struct.error):
return None
return out
def axml_root_attrs(blob):
"""Attributes of the root <manifest> element, as {name: value}.
Starts at the first start-element chunk, which is the root element in every
manifest Android produces. Values come back as strings where the pool has
them and as formatted scalars otherwise, so a caller can print them without
caring which encoding the compiler chose (a boolean is `0xffffffff`, not the
word "true", in the binary form).
"""
import struct
strings = axml_string_pool(blob)
if strings is None:
return None
total = struct.unpack_from('<I', blob, 4)[0] or len(blob)
end = min(len(blob), total)
off = 8
while off + 8 <= end:
ctype, _hsize, csize = struct.unpack_from('<HHI', blob, off)
if csize < 8 or off + csize > end:
break
if ctype == 0x0102: # RES_XML_START_ELEMENT
attr_start, attr_size, attr_count = struct.unpack_from(
'<HHH', blob, off + 24)
attrs = {}
first = off + 16 + attr_start
for i in range(attr_count):
p = first + i * attr_size
if p + 20 > off + csize:
break
name_i, raw_i = struct.unpack_from('<II', blob, p + 4)
_size, _res0, dtype = struct.unpack_from('<HBB', blob, p + 12)
data = struct.unpack_from('<I', blob, p + 16)[0]
name = strings[name_i] if name_i < len(strings) else '?'
if dtype == 0x03:
val = strings[data] if data < len(strings) else ''
elif raw_i != 0xFFFFFFFF and raw_i < len(strings):
val = strings[raw_i]
elif dtype == 0x10:
val = str(data - (1 << 32) if data >= (1 << 31) else data)
elif dtype == 0x11:
val = '0x%x' % data
elif dtype == 0x12:
val = 'true' if data else 'false'
elif dtype == 0x01:
val = '@ref/0x%08x' % data
else:
val = 'type=0x%02x/0x%x' % (dtype, data)
attrs[name] = val
return attrs
off += csize
return None
def inspect_apk(path):
"""Structural facts about one member of a split set, from the zip alone.
Nothing here needs a device or a resource compiler: which split owns dex,
which owns native libraries and which owns resources is visible in the entry
list, and the split's own name (and therefore its role) is a manifest
attribute. Reading it from the file is what keeps this usable on a set whose
package name is irrelevant to the caller.
"""
info = {'path': os.path.abspath(path), 'name': os.path.basename(path),
'size': os.path.getsize(path), 'dex': [], 'abis': [], 'res': 0,
'assets': 0, 'libs': [], 'arsc': None, 'arsc_stored': None,
'package': None, 'split': None, 'config_for_split': None,
'is_split_required': None, 'is_feature': None, 'readable': True}
try:
with zipfile.ZipFile(path, 'r') as z:
names = []
for item in z.infolist():
name = item.filename
names.append(name)
if item.is_dir():
continue
if re.match(r'^classes\d*\.dex$', name):
info['dex'].append(name)
elif name == 'resources.arsc':
info['arsc'] = item.file_size
info['arsc_stored'] = (item.compress_type == 0)
elif name.startswith('lib/') and name.count('/') >= 2:
abi = name.split('/')[1]
if abi not in info['abis']:
info['abis'].append(abi)
info['libs'].append(name)
elif name.startswith('res/'):
info['res'] += 1
elif name.startswith('assets/'):
info['assets'] += 1
if 'AndroidManifest.xml' in names:
attrs = axml_root_attrs(z.read('AndroidManifest.xml')) or {}
info['package'] = attrs.get('package')
info['split'] = attrs.get('split')
info['config_for_split'] = attrs.get('configForSplit')
info['is_split_required'] = attrs.get('isSplitRequired')
info['is_feature'] = attrs.get('isFeatureSplit')
except (zipfile.BadZipFile, OSError) as exc:
info['readable'] = False
info['error'] = str(exc)
return info
def split_role(info):
"""What a member of a set contributes: base / abi / code / resources.
The declared split name is consulted first (it is the authoritative label),
then the actual contents, because a set pulled off a device is often named
`Foo-arm64_v8a.apk` while its manifest still says `config.arm64_v8a`.
"""
if not info.get('readable'):
return 'unreadable'
if not info.get('split'):
return 'base'
name = info['split']
tail = name.split('.', 1)[1] if '.' in name else name
if info['abis'] or tail in ABI_SPLIT_NAMES:
return 'abi'
if name.startswith('config.'):
if tail in DENSITY_SPLIT_NAMES:
return 'density'
return 'config'
if info['dex']:
return 'code'
return 'split'
def split_has_real_resources(info):
"""True when folding this split into the base would require merging arsc."""
if info.get('res'):
return True
return (info.get('arsc') or 0) > EMPTY_ARSC_BYTES
def pick_base(items):
"""The base is the member with no `split` attribute; ties break toward dex."""
cands = [i for i in items if not i.get('split')]
if not cands:
return None, []
cands.sort(key=lambda i: (not i['dex'], -i['size']))
return cands[0], cands[1:]
def split_report(items):
"""Human-readable inventory of a set: the input to the route decision."""
rows = []
for i in items:
role = split_role(i)
extra = []
if i['dex']:
extra.append('dex=%s' % ','.join(i['dex']))
if i['abis']:
extra.append('libs=%s' % ','.join(i['abis']))
if i['res']:
extra.append('res=%d entries' % i['res'])
if i['arsc'] is not None:
extra.append('arsc=%d%s' % (i['arsc'], '' if i['arsc_stored'] else ' COMPRESSED'))
if i['assets']:
extra.append('assets=%d' % i['assets'])
rows.append('%-10s %-40s %10d B split=%-18s %s'
% (role, i['name'], i['size'], i.get('split') or '-',
' '.join(extra)))
return rows
def plan_split_set(items):
"""Decide merge vs resign for this set, and say why in one sentence each.
Returns (base, others, blockers, notes). `blockers` non-empty means merging
is not legal without explicitly accepting a loss, and the message names the
splits responsible.
"""
notes, blockers = [], []
base, extras = pick_base(items)
if base is None:
return None, [], ['no base APK in the set (no member lacks a `split` '
'attribute) -- this does not look like a split set'], notes
others = [i for i in items if i is not base]
for i in extras:
notes.append('extra candidate without a `split` attribute: %s' % i['name'])
pkgs = sorted({i['package'] for i in items if i.get('package')})
if len(pkgs) > 1:
notes.append('members disagree about the package name: %s -- a set must '
'share one, so these files probably do not belong together'
% ', '.join(pkgs))
if base.get('is_split_required') == 'true':
notes.append('base sets isSplitRequired=true: after a merge the platform '
'will refuse to start it unless the installer still knows the '
'set, so a merged build is only safe once that attribute is '
'cleared')
if not base['dex']:
notes.append('base carries no classes*.dex (unusual: %s)'
% (', '.join(base['dex']) or 'none'))
carrying = [i for i in others if split_has_real_resources(i)]
if carrying:
blockers.append('merging is not legal for %d of %d splits: %s carry their '
'own resources, and folding those in means merging '
'resources.arsc (a resource-compiler job). Use '
'--split-mode resign, or pass --drop-split-resources to '
'accept losing exactly those resources'
% (len(carrying), len(others),
', '.join(i.get('split') or i['name'] for i in carrying)))
elif others:
notes.append('every non-base split carries only code/native libraries, so '
'a merged single APK is legal')
fabis = [i for i in others if i['abis']]
for i in fabis:
notes.append('%s provides libs for %s -- keep only the ABIs the target '
'device runs, selected with --abi'
% (i.get('split') or i['name'], ','.join(i['abis'])))
return base, others, blockers, notes
def collect_split_inputs(args):
"""Every APK the caller pointed at as part of one set (empty = not a set)."""
if not (args.split_dir or args.split):
return []
paths = []
if args.split_dir:
paths += find_apks(os.path.abspath(args.split_dir))
for p in (args.split or []):
ap = os.path.abspath(p)
paths += find_apks(ap) if os.path.isdir(ap) else [ap]
if args.apk:
ap = os.path.abspath(args.apk)
if ap not in paths:
paths.append(ap)
# A set is a set: dedupe by real path so `--split-dir X --split X/base.apk`
# does not try to merge an APK into itself.
seen, out = set(), []
for p in paths:
key = os.path.normcase(p)
if key not in seen:
seen.add(key)
out.append(p)
return out
def abi_of_split(info):
"""The ABI a split provides, from its lib directories or its split name."""
if info['abis']:
return info['abis'][0]
name = (info.get('split') or '').split('.', 1)[-1]
return name.replace('_', '-') if name in ABI_SPLIT_NAMES else None
def merge_split_set(base, others, out_apk, allow_drop_resources=False, keep_abi=None):
"""Fold a set into one standalone APK: base plus every foldable member.
Folded: `classes*.dex` (renumbered so the second dex becomes classes2.dex),
`lib/**` and `assets/**`. Not folded: `res/**` and `resources.arsc`, because
a correct resource merge rewrites the table, and a wrong one is invisible
until the app renders.
Returns (stats, dropped) so the caller can report the cost out loud instead
of shipping a silently downgraded build.
"""
plan = []
taken = set()
stats = {'dex': 0, 'libs': 0, 'assets': 0, 'conflicts': 0}
dropped = []
with zipfile.ZipFile(base['path'], 'r') as z:
for item in z.infolist():
name = item.filename
if item.is_dir() or is_signature_entry(name):
continue
taken.add(name)
plan.append((name, item, z.read(name),
name in STORE_ONLY or os.path.basename(name) in STORE_ONLY))
base_dex = len([n for n in taken if re.match(r'^classes\d*\.dex$', n)])
next_dex = base_dex + 1 # a base with no dex still wants classes.dex first
for info in others:
if info['abis'] and keep_abi and keep_abi not in abi_of_split(info):
dropped.append('%s (ABI %s, keeping %s)' % (info['name'], abi_of_split(info), keep_abi))
continue
if split_has_real_resources(info) and not allow_drop_resources:
raise SystemExit(
'error: refusing to merge %s -- it carries its own resources '
'(%d res entries, arsc=%s bytes).\n'
' Folding it in would require merging resources.arsc, which this '
'script does not do.\n'
' Either use --split-mode resign (recommended for a resource '
'split), or pass --drop-split-resources to accept that the '
'resources it owns are lost.'
% (info['name'], info['res'], info['arsc']))
with zipfile.ZipFile(info['path'], 'r') as z:
for item in z.infolist():
name = item.filename
if item.is_dir() or is_signature_entry(name):
continue
if name == 'AndroidManifest.xml':
continue # the base manifest already exists
if name == 'resources.arsc' or name.startswith('res/'):
dropped.append('%s: %s' % (info['name'], name))
continue
if re.match(r'^classes\d*\.dex$', name):
new = 'classes%d.dex' % next_dex if next_dex > 1 else 'classes.dex'
while new in taken:
next_dex += 1
new = 'classes%d.dex' % next_dex
data = z.read(name)
zi = zipfile.ZipInfo(new, date_time=(2024, 1, 1, 0, 0, 0))
plan.append((new, zi, data, False))
taken.add(new)
stats['dex'] += 1
log('[merge] + %s (%d bytes <- %s:%s)' % (new, len(data), info['name'], name))
next_dex += 1
continue
data = z.read(name)
if name in taken:
existing = next((d for n, _i, d, _k in plan if n == name), None)
if existing == data:
continue
stats['conflicts'] += 1
log('[merge] CONFLICT %s: %s and the base both provide it with '
'different content; keeping the base copy' % (name, info['name']))
continue
plan.append((name, item, data, name in STORE_ONLY))
taken.add(name)
if name.startswith('lib/'):
stats['libs'] += 1
elif name.startswith('assets/'):
stats['assets'] += 1
log('[merge] + %s (%d bytes <- %s)' % (name, len(data), info['name']))
_write_aligned_zip(out_apk, plan)
return stats, dropped
def resign_split_set(items, outdir, sign_one):
"""Sign every member of a set with the same keystore, keeping the structure.
Each member is de-signed and rewritten as an aligned archive first, exactly
like a single-APK repack -- a member whose `resources.arsc` is compressed or
unaligned is refused by the installer on its own, and the error names the
member, not the set.
"""
os.makedirs(outdir, exist_ok=True)
scratch = os.path.join(outdir, '.unsigned')
if os.path.isdir(scratch):
shutil.rmtree(scratch, ignore_errors=True)
os.makedirs(scratch, exist_ok=True)
outs = []
for info in items:
unsigned = os.path.join(scratch, info['name'])
build_unsigned(info['path'], {}, unsigned)
problems = check_alignment(unsigned)
for p in problems:
log('[resign] alignment problem in %s: %s' % (info['name'], p))
signed = sign_one(unsigned, os.path.join(outdir, info['name']))
outs.append((info, signed))
log('[resign] %s -> %s (%d bytes)' % (info['name'], signed, os.path.getsize(signed)))
shutil.rmtree(scratch, ignore_errors=True)
return outs
def cert_fingerprints(apk, tools):
"""SHA-256 of each signer certificate, as reported by apksigner."""
apksigner = tools.get('apksigner')
if not apksigner:
return []
cmd = signer_invocation(apksigner, tools) + ['verify', '--print-certs', apk]
r = run(cmd)
text = (r.stdout or '') + (r.stderr or '')
return sorted({m.lower() for m in re.findall(r'SHA-256 digest:\s*([0-9a-fA-F:]+)', text)})
# ---------------------------------------------------------------------------
# Packing
# ---------------------------------------------------------------------------
@@ -422,20 +890,26 @@ def check_alignment(apk):
def zip_report(apk):
"""Entry summary, including the storage/alignment facts Android gates on."""
"""Entry summary, including the storage/alignment facts Android gates on.
Only entries that MUST be aligned are judged on alignment. `AndroidManifest.xml`
must be STORED but not aligned, and printing "NOT ALIGNED" next to it read as a
failing build when it is the normal, correct layout.
"""
import struct
rows = []
with zipfile.ZipFile(apk, 'r') as z, open(apk, 'rb') as fh:
for i in z.infolist():
base = os.path.basename(i.filename)
needs = (i.filename in STORE_ONLY or base in STORE_ONLY
or _needs_alignment(i.filename))
must_align = _needs_alignment(i.filename)
needs = (i.filename in STORE_ONLY or base in STORE_ONLY or must_align)
if needs:
fh.seek(i.header_offset)
lh = fh.read(30)
nlen, elen = struct.unpack('<HH', lh[26:30])
data_off = i.header_offset + 30 + nlen + elen
verdict = 'aligned' if data_off % ALIGN == 0 else 'NOT ALIGNED'
verdict = ('aligned' if data_off % ALIGN == 0
else 'NOT ALIGNED') if must_align else '-'
stored = 'STORED' if i.compress_type == 0 else \
'COMPRESSED(method=%d)' % i.compress_type
rows.append('%-40s %-22s offset=0x%-8x %s'
@@ -449,6 +923,88 @@ def zip_report(apk):
# ---------------------------------------------------------------------------
# Sign / verify
# ---------------------------------------------------------------------------
def signer_invocation(path, tools):
"""Command prefix that runs apksigner, whatever shape it was shipped as.
A build-tools directory offers `apksigner.bat` (Windows), `apksigner` (a
shell wrapper) or the bare `lib/apksigner.jar`. All three are legitimate
values for --apksigner, and only the jar needs java in front of it. A bare
shell wrapper does not run on Windows, which is why the jar is the value to
pass there.
"""
if path.lower().endswith('.jar'):
java = require(tools, 'java', 'run apksigner.jar')
return [java, '-jar', path]
return [path]
def sign_with_apksigner(unsigned_apk, out_apk, ks, alias, password, tools):
"""zipalign + apksigner (v1+v2+v3) -- the route that needs no uber-apk-signer.
Order is load-bearing: align first, sign second. apksigner adds signature
entries without reshuffling the archive; the v1 (JAR) path emulates
jarsigner, whose rewrite of the zip would destroy the alignment that Android
R+ requires of `resources.arsc`.
"""
apksigner = require(tools, 'apksigner', 'sign the apk')
src = unsigned_apk
aligned = None
if tools.get('zipalign'):
aligned = unsigned_apk + '.aligned'
r = run([tools['zipalign'], '-p', '-f', '4', unsigned_apk, aligned])
log('[sign:zipalign] rc=%d' % r.returncode)
if r.returncode != 0 or not os.path.exists(aligned):
log(((r.stdout or '') + (r.stderr or '')).strip())
raise SystemExit('error: zipalign failed; refusing to sign an '
'unaligned build')
src = aligned
else:
log('[sign:zipalign] skipped: zipalign not available; the archive written '
'by this script is already 4-byte aligned (see the alignment gate)')
try:
cmd = signer_invocation(apksigner, tools) + [
'sign', '--ks', ks, '--ks-key-alias', alias,
'--ks-pass', 'pass:' + password, '--key-pass', 'pass:' + password,
'--v1-signing-enabled', 'true', '--v2-signing-enabled', 'true',
'--v3-signing-enabled', 'true', '--out', out_apk, src]
r = run(cmd)
log('[sign:apksigner] rc=%d' % r.returncode)
log(((r.stdout or '') + (r.stderr or '')).strip())
if r.returncode != 0 or not os.path.exists(out_apk):
raise SystemExit('error: apksigner failed (output above)')
finally:
if aligned and os.path.exists(aligned):
os.remove(aligned)
return out_apk
def choose_signer(args, tools):
"""Pick the signing route: the jar when it exists, else apksigner directly.
The jar stays first so an existing workflow does not change behaviour. The
apksigner branch exists because a machine can have a complete build-tools
directory and no uber-apk-signer at all -- the previous single-route design
made every signed build impossible there, with an error that named a jar the
user never had.
"""
jar = args.signer_jar or os.environ.get('APK_SIGNER_JAR', 'uber-apk-signer.jar')
kind = getattr(args, 'signer', 'auto')
if kind in ('auto', 'jar') and os.path.exists(jar):
return {'kind': 'jar', 'jar': jar}
if kind == 'jar':
raise SystemExit('error: --signer jar requested but %s does not exist'
% jar)
if tools.get('apksigner'):
return {'kind': 'apksigner', 'path': tools['apksigner']}
raise SystemExit(
'error: no signing route available.\n'
' * uber-apk-signer.jar not found at %s\n'
' * apksigner not found on PATH either (pass --apksigner <path>; it may\n'
' be apksigner.bat, an apksigner wrapper, or lib/apksigner.jar)\n'
' A machine with Android build-tools and no uber-apk-signer is supported:\n'
' give it --apksigner and this script aligns and signs on its own.' % jar)
def sign_apk(unsigned_apk, workdir, ks, alias, password, tools, signer_jar):
java = require(tools, 'java', 'run the signer jar')
if not os.path.exists(signer_jar):
@@ -498,8 +1054,9 @@ def verify_apk(apk, tools, signer_jar=None, expect_signed=True):
# are meaningful.
if tools.get('apksigner'):
sig_checks += 1
r = run([tools['apksigner'], 'verify', '--print-certs', '--verbose',
'--min-sdk-version', '21', '--max-sdk-version', '34', apk])
r = run(signer_invocation(tools['apksigner'], tools) + [
'verify', '--print-certs', '--verbose',
'--min-sdk-version', '21', '--max-sdk-version', '34', apk])
log('[verify:apksigner] rc=%d\n%s' % (r.returncode, ((r.stdout or '') +
(r.stderr or '')).strip()[:3000]))
if r.returncode != 0:
@@ -559,27 +1116,295 @@ def verify_apk(apk, tools, signer_jar=None, expect_signed=True):
def fingerprint(apk, tools):
if not tools.get('keytool'):
return []
r = run([tools['keytool'], '-printcert', '-jarfile', apk])
return re.findall(r'(SHA1|SHA256):\s*([0-9A-F:]+)', r.stdout or '')
"""Signer fingerprints. keytool when there is one, apksigner otherwise.
keytool is frequently absent (it lives in a JDK bin directory that is not on
PATH), and reporting no fingerprint at all reads as "unsigned". apksigner is
the tool that actually signed the file, so its answer is the authoritative
one anyway.
"""
if tools.get('keytool'):
r = run([tools['keytool'], '-printcert', '-jarfile', apk])
found = re.findall(r'(SHA1|SHA256):\s*([0-9A-F:]+)', r.stdout or '')
if found:
return found
return [('SHA-256', f) for f in cert_fingerprints(apk, tools)]
def resolve_keystore(args, tools):
"""Return (keystore path, password), creating the keystore on first use."""
if not args.ks:
raise SystemExit(
"error: --ks is required when signing (there is no default keystore).\n"
" e.g. --ks work/out/release.keystore")
keytool = require(tools, 'keytool', 'create the keystore on first use')
ks = os.path.abspath(args.ks)
return ks, ensure_keystore(ks, args.ks_alias, args.ks_pass, keytool)
def sign_one_artifact(unsigned_apk, out_path, args, tools, signer, ks, password, workdir):
"""Sign one unsigned APK through whichever route choose_signer() picked."""
if signer['kind'] == 'jar':
signed = sign_apk(unsigned_apk, workdir, ks, args.ks_alias, password,
tools, signer['jar'])
if os.path.abspath(signed) != os.path.abspath(out_path):
shutil.copy2(signed, out_path)
else:
sign_with_apksigner(unsigned_apk, out_path, ks, args.ks_alias, password, tools)
return out_path
def scratch_dir(args, fallback):
workdir = os.path.abspath(args.workdir) if args.workdir else fallback
if not workdir:
workdir = os.getcwd()
os.makedirs(workdir, exist_ok=True)
return workdir
def run_split_mode(args, tools):
"""Handle --split-dir / --split: inventory the set, pick a legal route, act.
Nothing here guesses which route "should" work: the set is inspected, merge
legality is decided from what the splits actually carry, and the reason is
printed before anything is written.
"""
items = [inspect_apk(p) for p in collect_split_inputs(args)]
broken = [i['name'] for i in items if not i['readable']]
if broken:
raise SystemExit('error: not readable as an APK: %s' % ', '.join(broken))
log('== split set: %d apk(s), %d bytes total'
% (len(items), sum(i['size'] for i in items)))
for row in split_report(items):
log(' ' + row)
base, others, blockers, notes = plan_split_set(items)
if base is None or not others:
for b in blockers:
log('[plan] BLOCKED: %s' % b)
if base is not None:
log('[plan] %s is the only member -- a single APK: run without '
'--split/--split-dir' % base['name'])
return 2
log('[plan] base = %s (package=%s)' % (base['name'], base.get('package')))
for n in notes:
log('[plan] note: %s' % n)
for b in blockers:
log('[plan] MERGE BLOCKED: %s' % b)
if not blockers:
log('[plan] merge = legal: %d member(s) fold into the base' % len(others))
log('[plan] resign = legal: one keystore for all %d members, install the set with'
% len(items))
log('[plan] adb install-multiple -r %s'
% ' '.join(i['name'] for i in items))
mode = args.split_mode
if mode == 'auto':
if blockers:
mode = 'resign'
elif (args.out or '').lower().endswith('.apk'):
mode = 'merge'
else:
mode = 'resign'
log('[plan] --split-mode auto resolved to: %s' % mode)
if mode == 'analyze':
log('[result] analyze only: nothing was written. Pass --split-mode '
'merge|resign|auto to act on this set.')
return 0
if mode == 'merge':
return _split_merge(args, tools, base, others)
outdir = args.split_out_dir or (os.path.abspath(args.out) if args.out else None)
if not outdir:
raise SystemExit('error: --split-mode resign needs --split-out-dir <dir> '
'(or --out, which is then treated as a directory).')
outdir = os.path.abspath(outdir)
log('[resign] output directory: %s' % outdir)
workdir = scratch_dir(args, os.path.join(outdir, '.work'))
signer = None
ks = password = None
if not args.no_sign:
signer = choose_signer(args, tools)
ks, password = resolve_keystore(args, tools)
log('[resign] signer route: %s' % signer['kind'])
if args.no_sign:
def sign_member(unsigned, out_path):
shutil.copy2(unsigned, out_path)
return out_path
else:
def sign_member(unsigned, out_path):
return sign_one_artifact(unsigned, out_path, args, tools, signer,
ks, password, workdir)
outs = resign_split_set(items, outdir, sign_member)
fps = set()
ok_all = True
signer_jar = signer['jar'] if signer and signer['kind'] == 'jar' else None
for info, path in outs:
if not verify_apk(path, tools, signer_jar, expect_signed=not args.no_sign):
ok_all = False
for f in cert_fingerprints(path, tools):
fps.add(f)
log('[cert] %s %s' % (info['name'], f))
if not args.no_sign:
if len(fps) == 1:
log('[result] OK: %d members, one certificate (sha256 %s)'
% (len(outs), sorted(fps)[0]))
else:
log('[result] CHECK-FAILED: %d distinct certificates across the set '
'(a mismatch is exactly what an installer rejects): %s'
% (len(fps), ', '.join(sorted(fps))))
ok_all = False
else:
log('[result] UNSIGNED (--no-sign): de-signed and aligned set, not installable '
'as-is')
log('[next] adb install-multiple -r %s'
% ' '.join(os.path.join(outdir, i['name']) for i in items))
return 0 if ok_all else 2
def _split_merge(args, tools, base, others):
"""The merge half of run_split_mode (kept separate to stay readable)."""
if not args.out:
raise SystemExit('error: --split-mode merge needs --out <merged.apk>')
out = os.path.abspath(args.out)
folder = os.path.dirname(out)
if folder:
os.makedirs(folder, exist_ok=True)
unsigned = out + '.unsigned'
if os.path.exists(unsigned):
os.remove(unsigned)
stats, dropped = merge_split_set(base, others, unsigned,
allow_drop_resources=args.drop_split_resources,
keep_abi=args.abi)
log('[merge] folded: dex=%d libs=%d assets=%d conflicts=%d'
% (stats['dex'], stats['libs'], stats['assets'], stats['conflicts']))
if dropped:
log('[merge] DROPPED %d entry/ies. This is a DOWNGRADED build -- say so when '
'you deliver it:' % len(dropped))
for d in dropped[:40]:
log(' - %s' % d)
if len(dropped) > 40:
log(' ... and %d more' % (len(dropped) - 40))
for row in zip_report(unsigned):
log('[zip] ' + row)
problems = check_alignment(unsigned)
if problems:
log('== ALIGNMENT/STORAGE GATE FAILED -- do not ship this build:')
for p in problems:
log(' - %s' % p)
else:
log('== alignment gate: resources.arsc STORED and 4-byte aligned (OK)')
if args.no_sign:
shutil.move(unsigned, out)
log('[out] %s (%d bytes)' % (out, os.path.getsize(out)))
log('[result] UNSIGNED (--no-sign): inspection only, not installable as-is')
return 0
workdir = scratch_dir(args, folder or os.getcwd())
signer = choose_signer(args, tools)
ks, password = resolve_keystore(args, tools)
log('[merge] signer route: %s' % signer['kind'])
sign_one_artifact(unsigned, out, args, tools, signer, ks, password, workdir)
os.remove(unsigned)
log('[out] %s (%d bytes)' % (out, os.path.getsize(out)))
ok = verify_apk(out, tools,
signer['jar'] if signer['kind'] == 'jar' else None)
for algo, val in fingerprint(out, tools):
log('[cert] %s %s' % (algo, val))
log('[result] %s' % ('OK' if ok else 'CHECK-FAILED'))
log('[next] adb install -r %s (one file, no set to keep together)' % out)
return 0 if ok else 2
def main():
ap = argparse.ArgumentParser(
formatter_class=argparse.RawDescriptionHelpFormatter,
description='Repack an APK with replaced dex files, drop only the signature '
'entries, re-sign and verify.')
ap.add_argument('--apk', required=True, help='source apk to use as the template')
ap.add_argument('--out', required=True, help='output apk path')
'entries, re-sign and verify.\n'
'Also handles SPLIT APK / App Bundle sets (base.apk plus\n'
'split_config.*.apk): inventory them, then either sign the whole\n'
'set with one keystore (installable with `pm install-multiple`)\n'
'or fold the code/native members into one standalone APK.',
epilog='usage examples:\n'
' # single APK (unchanged behaviour)\n'
' python repack.py --apk original.apk --dexdir dex/ --out out.apk '
'--ks ks.jks\n'
'\n'
' # inspect a split set and get the route decision (writes nothing)\n'
' python repack.py --split-dir pulled_set/ --split-mode analyze\n'
'\n'
' # resign every member with ONE keystore -> pm install-multiple\n'
' python repack.py --split-dir pulled_set/ --split-mode resign \\\n'
' --split-out-dir signed_set/ --ks ks.jks \\\n'
' --apksigner /path/to/lib/apksigner.jar\n'
'\n'
' # fold code/native splits into one standalone APK\n'
' python repack.py --split-dir pulled_set/ --split-mode merge \\\n'
' --out merged.apk --ks ks.jks\n'
'\n'
' # no uber-apk-signer on this machine: --apksigner replaces it\n'
' python repack.py --apk app.apk --out out.apk --ks ks.jks \\\n'
' --apksigner /path/to/lib/apksigner.jar --zipalign '
'/path/to/zipalign\n'
'\n'
'split route decision (enforced, not guessed):\n'
' merge is legal only when every non-base split carries code or\n'
' native libraries. A split that carries its own resources\n'
' cannot be folded in without merging resources.arsc, which\n'
' needs a resource compiler; pass --drop-split-resources to\n'
' accept losing exactly those resources.\n'
' resign always works: one keystore, v1+v2+v3 on every member.\n')
ap.add_argument('--apk', help='source apk to use as the template (in split mode: '
'an explicit base; optional)')
ap.add_argument('--out', help='output apk path (in --split-mode resign: the '
'output directory, or use --split-out-dir)')
ap.add_argument('--dexdir', help='directory of classes*.dex to swap in')
ap.add_argument('--dex', action='append', default=[],
help='name=path, repeatable, e.g. classes8.dex=work/patch/classes8.dex')
ap.add_argument('--split-dir', default=None,
help='directory holding a split set (searched recursively for '
'*.apk); every member must share one package name')
ap.add_argument('--split', action='append', default=[],
help='one split member, repeatable; a directory is searched '
'recursively. Supplying any --split/--split-dir switches '
'this script into split-set mode')
ap.add_argument('--split-mode', choices=('analyze', 'auto', 'merge', 'resign'),
default='analyze',
help='analyze (default): print the inventory and the route '
'decision, write nothing. auto: merge when that is legal '
'and --out ends in .apk, else resign. merge: fold into one '
'standalone APK (--out). resign: sign every member with one '
'keystore (--split-out-dir)')
ap.add_argument('--split-out-dir', default=None,
help='output directory for --split-mode resign')
ap.add_argument('--drop-split-resources', action='store_true',
help='merge mode: explicitly allow dropping splits whose '
'resources cannot be merged (produces a downgraded build)')
ap.add_argument('--abi', default=None,
help='merge mode: keep only the ABI split matching this ABI '
'(e.g. arm64-v8a). Compare with the device via '
'`getprop ro.product.cpu.abilist`; split names use '
'underscores (arm64_v8a), lib/ directories use hyphens')
ap.add_argument('--workdir', default=None,
help='scratch directory (default: next to --out)')
ap.add_argument('--ks', default=None, help='keystore path (created on first use)')
ap.add_argument('--ks-alias', default='apkreverse', help='keystore alias')
ap.add_argument('--ks-pass', default=None,
help='keystore password; default: <ks>.pass.txt, else generated')
ap.add_argument('--signer', choices=('auto', 'jar', 'apksigner'), default='auto',
help='signing route. auto (default): uber-apk-signer jar when it '
'exists, otherwise zipalign+apksigner directly')
ap.add_argument('--signer-jar', default=None,
help='jar used for zipalign+sign (default: $APK_SIGNER_JAR or '
'uber-apk-signer.jar)')
@@ -588,25 +1413,32 @@ def main():
ap.add_argument('--keytool', default=None, help='path to keytool (default: from PATH)')
ap.add_argument('--jarsigner', default=None, help='path to jarsigner (default: from PATH)')
ap.add_argument('--zipalign', default=None, help='path to zipalign (default: from PATH)')
ap.add_argument('--apksigner', default=None, help='path to apksigner (default: from PATH)')
ap.add_argument('--apksigner', default=None,
help='path to apksigner (default: from PATH). Accepts '
'apksigner.bat, an apksigner wrapper, or lib/apksigner.jar')
args = ap.parse_args()
tools = resolve_tools(args)
signer_jar = args.signer_jar or os.environ.get('APK_SIGNER_JAR', 'uber-apk-signer.jar')
split_mode = bool(args.split_dir or args.split)
if not split_mode:
if not args.apk:
raise SystemExit('error: --apk is required (or supply --split/--split-dir '
'for a split set).')
if not args.out:
raise SystemExit('error: --out is required.')
if split_mode:
return run_split_mode(args, tools)
apk = os.path.abspath(args.apk)
out = os.path.abspath(args.out)
if not os.path.exists(apk):
raise SystemExit('missing apk: %s' % apk)
# Default the scratch dir next to --out; never to a machine-specific path.
workdir = os.path.abspath(args.workdir) if args.workdir else os.path.dirname(out)
if not workdir:
workdir = os.getcwd()
os.makedirs(workdir, exist_ok=True)
out_folder = os.path.dirname(out)
if out_folder:
os.makedirs(out_folder, exist_ok=True)
workdir = scratch_dir(args, os.path.dirname(out))
repl = collect_replacement_dex(args.dexdir, args.dex)
log('[in ] %s (%d bytes)' % (apk, os.path.getsize(apk)))
@@ -622,19 +1454,14 @@ def main():
for row in zip_report(unsigned):
log('[zip] ' + row)
signer = None
if args.no_sign:
shutil.copy2(unsigned, out)
else:
if not args.ks:
raise SystemExit(
"error: --ks is required when signing (there is no default keystore).\n"
" e.g. --ks work/out/release.keystore")
keytool = require(tools, 'keytool', 'create the keystore on first use')
password = ensure_keystore(os.path.abspath(args.ks), args.ks_alias,
args.ks_pass, keytool)
signed = sign_apk(unsigned, workdir, os.path.abspath(args.ks), args.ks_alias,
password, tools, signer_jar)
shutil.copy2(signed, out)
signer = choose_signer(args, tools)
log('[sign] route: %s' % signer['kind'])
ks, password = resolve_keystore(args, tools)
sign_one_artifact(unsigned, out, args, tools, signer, ks, password, workdir)
log('[out] %s (%d bytes)' % (out, os.path.getsize(out)))
log('== zip layout of final apk')
@@ -650,7 +1477,9 @@ def main():
else:
log('== alignment gate: resources.arsc STORED and 4-byte aligned (OK)')
ok = verify_apk(out, tools, signer_jar, expect_signed=not args.no_sign)
ok = verify_apk(out, tools,
signer['jar'] if signer and signer['kind'] == 'jar' else None,
expect_signed=not args.no_sign)
if not args.no_sign:
for algo, val in fingerprint(out, tools):
log('[cert] %s %s' % (algo, val))
skills/apk-reverse/scripts/spawn_patch_detach.py
+79 -12
View File
@@ -11,13 +11,29 @@ The same app launched normally, after Frida has left, renders correctly.
The fix is an ordering rather than a different hook:
1. spawn and attach,
2. let the probe write its patch into memory (``Memory.patchCode``),
3. **detach** -- the write is a plain memory write and survives; ``Interceptor``
hooks go away with the session, which is usually what you want for an
observation run because it removes the instrumentation from the picture,
1. spawn (`device.spawn()` leaves the target **paused**),
2. attach and load the probe,
3. resume, then **wait for the probe to report `PATCHED`** and only detach after
that -- the write is a plain memory write and survives; ``Interceptor`` hooks go
away with the session, which is usually what you want for an observation run
because it removes the instrumentation from the picture,
4. start the Activity normally and capture.
**The two orderings that do not work, both measured on MASTG UnCrackable-Level3
(``libfoo.so`` self-destruct, ``goodbyev`` at ``base+0x3080``):**
- *Detach immediately after ``script.load()``.* The probe's ``Process.findModuleByName``
poll has not seen the library yet, so the write never happens and the target kills
itself on its own schedule. The measured run reported ``patched=False`` and the
target was gone by the first capture.
- *Hold the target paused until ``PATCHED``.* While the process is frozen its
libraries are not mapped, so the poll can never succeed: 15 s paused produced no
module, and ``libfoo.so base=0x764c0aa000`` appeared 0.11 s after resume.
So the target must be running for the patch to be possible at all, and the time
between resume and the write is race time, not slack. Report ``patched=False`` loudly
rather than presenting an unpatched run as a result.
What the probe may contain
--------------------------
Anything self-contained. A minimal probe that only neutralises a death site and
@@ -63,7 +79,16 @@ def build_parser():
p.add_argument("--serial", default=None, help="adb device serial (needed when several are online)")
p.add_argument("--adb", default=DEFAULT_ADB, help="path to adb (default: from PATH)")
p.add_argument("--wait-patched", type=float, default=15.0,
help="seconds to wait for the probe to report PATCHED (default %(default)s)")
help="seconds to wait for the probe to report PATCHED after resume; "
"the detach happens only when it arrives (default %(default)s)")
p.add_argument("--rpc-timeout", type=float, default=20.0,
help="seconds to bound each frida RPC (attach / load), so a dead "
"device server fails instead of hanging (default %(default)s)")
p.add_argument("--playground", default=None,
help="package to ask for the launcher activity when --activity is "
"omitted (default: --package)")
p.add_argument("--launch-timeout", type=float, default=30.0,
help="seconds to bound the post-detach `am start` (default %(default)s)")
p.add_argument("--detach-settle", type=float, default=3.0,
help="pause after detach, before launching (default %(default)s)")
p.add_argument("--captures", type=int, default=3, help="number of screenshots (default %(default)s)")
@@ -86,6 +111,25 @@ def root_shell(cmd, adb, serial, timeout=200):
return adb_run(["shell", "su -c '%s'" % cmd], adb, serial, timeout)
def resolve_launcher(adb, serial, arg_activity, package, playground=None):
"""Return the component to start, asking the device when none was given.
`am start <package>` alone is not accepted on every ROM, and starting nothing
leaves the spawned process dead with the screen showing whatever was there
before -- which reads exactly like "the app died". Ask the package manager.
"""
if arg_activity:
return arg_activity
pkg = playground or package
r = adb_run(["shell", "cmd", "package", "resolve-activity", "--brief", pkg], adb, serial)
for line in (r.stdout or "").splitlines():
line = line.strip()
if "/" in line and not line.startswith("priority") and "No activity" not in line:
if line.count("/") == 1 and not line.startswith("Starting"):
return line
return None
def main(argv=None):
args = build_parser().parse_args(argv)
@@ -95,6 +139,7 @@ def main(argv=None):
return 2
patched = {"done": False}
failed = {"done": False}
def on_message(msg, data):
if msg.get("type") == "send":
@@ -102,6 +147,8 @@ def main(argv=None):
print("[MSG] %s" % payload, flush=True)
if payload == "PATCHED":
patched["done"] = True
elif payload == "PATCHFAIL":
failed["done"] = True
elif msg.get("type") == "error":
print("[ERR] %s" % (msg.get("stack") or msg.get("description")), flush=True)
else:
@@ -112,27 +159,47 @@ def main(argv=None):
dev = frida.get_device_manager().add_remote_device(args.frida_host)
pid = dev.spawn([args.package])
print("[i] spawned pid=%d" % pid, flush=True)
print("[i] spawned pid=%d (paused)" % pid, flush=True)
session = dev.attach(pid)
script = session.create_script(open(args.js, encoding="utf-8").read())
script.on("message", on_message)
script.load()
# Resume, then wait for the probe to report. Holding the target paused until
# PATCHED does NOT work for a probe that waits on a module: while the process is
# frozen its libraries are not mapped, so Process.findModuleByName() never
# returns. Measured on MASTG UnCrackable-Level3: 15 s paused -> no module, then
# "libfoo.so base=0x764c0aa000" 0.11 s after resume. What matters is that we do
# NOT detach before the write has landed, and that we report patched=False rather
# than pretending the target was patched.
dev.resume(pid)
print("[i] resumed pid=%d" % pid, flush=True)
deadline = time.time() + args.wait_patched
while time.time() < deadline and not patched["done"]:
while time.time() < deadline and not patched["done"] and not failed["done"]:
time.sleep(0.2)
if not patched["done"]:
print("[!] the probe did not report PATCHED in %.1fs. Detaching anyway -- the "
"target runs UNPATCHED, so nothing about this run tests the probe."
% args.wait_patched, flush=True)
print("[i] patched=%s" % patched["done"], flush=True)
time.sleep(1.0)
time.sleep(0.5)
session.detach()
print("[i] detached (memory writes persist, hooks are gone)", flush=True)
time.sleep(args.detach_settle)
if args.activity:
out = adb_run(["shell", "am start -n %s" % args.activity], args.adb, args.serial)
print("[i] am start: %s" % ((out.stdout or out.stderr or "").strip()), flush=True)
component = resolve_launcher(args.adb, args.serial, args.activity, args.package,
args.playground)
if component:
out = adb_run(["shell", "am start -n %s" % component], args.adb, args.serial,
timeout=args.launch_timeout)
print("[i] am start %s: %s" % (component,
(out.stdout or out.stderr or "").strip()), flush=True)
else:
print("[!] no launcher component resolved; the spawned process may be dead. "
"Pass --activity pkg/.Activity.", flush=True)
for i in range(max(1, args.captures)):
time.sleep(args.interval)
skills/apk-reverse/scripts/stalker_trace.js
+80
View File
@@ -43,6 +43,16 @@
* exec = per-instruction (ENORMOUS; only for a few hundred
* instructions around a known point; off by default);
* ret = return edges (off by default).
* excludeModules: modules Stalker must not translate (see the CONFIG comment).
* Followed through libc/libart, an arm64 device pays a large
* multiplier and the trace becomes a trace of the OS. Emitted
* as one `EXCL excluded=N/M [names]` line before the follow --
* read it, because it states what the trace was protected by.
* A module not yet loaded cannot be excluded; exclusion is
* best-effort and applies to modules already mapped.
* zeroEventWarnMs: emits `WARN zero events ...` when a follow delivered
* nothing by then, so a dead pipeline is not mistaken for
* "the code did not run".
*
* HOW TO RUN
* ----------
@@ -102,6 +112,25 @@ var CONFIG = {
maxBlocks: 100000, // stop recording past this many reported events
autoStart: true, // false = arm triggers but wait for rpc start()
// Modules Stalker must NOT translate, resolved by name at follow time.
// Cost is the reason this exists. A followed thread whose execution runs
// through libc/libart pays a large multiplier on arm64 (community reports
// 20-50x) and drags the whole device down with it, and a hot library
// reached through a follow/unfollow cycle is where target processes have
// died. Excluding the system libraries you are not studying is the
// difference between a trace of your target and a trace of the OS.
// The target module is never excluded, even if its name is listed here.
excludeModules: [
'libc.so', 'libm.so', 'libdl.so', 'libc++.so', 'libc++_shared.so',
'libart.so', 'libartbase.so', 'libnativehelper.so',
'libutils.so', 'libbinder.so', 'libcutils.so', 'libbase.so',
'libui.so', 'libgui.so', 'libinput.so', 'libhwui.so', 'libskia.so',
'libEGL.so', 'libGLESv2.so', 'libvulkan.so', 'libandroid.so',
'liblog.so', 'libziparchive.so', 'libz.so'
],
excludeModulesExtra: [], // your own: 'libfoo.so', or 'libfoo.so+0x1000' for a sub-range
zeroEventWarnMs: 1500, // warn when a follow has produced nothing by then
events: {
compile: true, // first translation of each block (skeleton)
block: true, // every executed block (weights)
@@ -224,6 +253,40 @@ function onStalkEvents(events) {
flush(true);
}
/* Exclude every configured module that is already mapped. Best-effort by
* design: a module that is not loaded yet cannot be excluded, so the count
* emitted here is what the trace is actually protected by -- read it before
* interpreting a bad trace. Exclusion must happen before follow(). */
function applyExclusions() {
var names = (CONFIG.excludeModules || []).concat(CONFIG.excludeModulesExtra || []);
var excluded = [], attempted = 0, missing = [];
for (var i = 0; i < names.length; i++) {
var spec = names[i];
if (!spec) continue;
var plus = spec.indexOf('+');
var name = plus > 0 ? spec.substring(0, plus) : spec;
if (state.target && name === state.target.name) continue; // never the target
var mod = null;
try { mod = Process.findModuleByName(name); } catch (e) { mod = null; }
if (!mod) { missing.push(name); continue; }
attempted++;
try {
if (plus > 0) {
var off = parseInt(spec.substring(plus + 1), 16) || 0;
Stalker.exclude({ base: mod.base.add(off), size: mod.size - off });
} else {
Stalker.exclude(mod);
}
excluded.push(name);
} catch (e) {
emit('EXCL-FAIL', spec + ': ' + e);
}
}
emit('EXCL', 'excluded=' + excluded.length + '/' + attempted +
' [' + excluded.join(',') + ']' +
(missing.length ? ' not-loaded=' + missing.length : ''));
}
function beginFollow(tid, why) {
if (state.following) return;
if (!state.target) {
@@ -237,6 +300,8 @@ function beginFollow(tid, why) {
line('MOD ' + state.target.name + ' base=' + state.target.base +
' size=' + state.target.size + ' path=' + state.target.path);
applyExclusions();
Stalker.follow(tid, {
events: {
call: CONFIG.events.call,
@@ -248,6 +313,20 @@ function beginFollow(tid, why) {
onReceive: onStalkEvents
});
/* A zero-event trace has two very different meanings -- "nothing executed"
* and "the pipeline never delivered" -- and the log alone cannot tell them
* apart. This warning exists so the second reading is the default one. */
if (CONFIG.zeroEventWarnMs > 0) {
state.warnTimer = setTimeout(function () {
if (state.following && state.nBB === 0 && state.nBLK === 0) {
emit('WARN', 'zero events ' + CONFIG.zeroEventWarnMs +
'ms after follow (blocks=0 blk=0 calls=0) -- the pipeline is ' +
'NOT proven; do not report this as "the code did not run". ' +
'Control: follow a thread running a known loop first.');
}
}, CONFIG.zeroEventWarnMs);
}
if (CONFIG.followMs > 0) {
state.timer = setTimeout(function () { endFollow('timeout'); },
CONFIG.followMs);
@@ -258,6 +337,7 @@ function endFollow(reason) {
if (!state.following) return;
state.following = false;
if (state.timer !== null) { clearTimeout(state.timer); state.timer = null; }
if (state.warnTimer) { clearTimeout(state.warnTimer); state.warnTimer = null; }
try { Stalker.flush(); } catch (e) { /* already gone */ }
try { Stalker.unfollow(state.followTid); } catch (e) { /* already gone */ }
flush(true);
skills/apk-reverse/scripts/vmp_diff_harness.py
+1719
View File
File diff suppressed because it is too large Load Diff
tests/benchmark.md
+55
View File
@@ -0,0 +1,55 @@
# Benchmark set — the regression baseline for this skill
This file is a **test matrix**, not a sample collection. It answers one question per row: *when an
agent meets this shape of target, does the route this repository documents actually work, and what
is the strength of the evidence?* Every claim in the `Result` column must be reproducible from the
`Target` column plus the named script — nothing here is a summary of what "should" happen.
**Samples are never committed.** Each row names a public source; download it into `tools/_work/bench/`
(ignored by git) before running the row. Record the sample hash when you run it, because two rows on
different sample revisions are not comparable. Hashes used in this pass:
```
UnCrackable-Level1.apk 66651 B 1da8bf57d266109f…
UnCrackable-Level2.apk 901022 B 4c7980ef1f6cc295…
UnCrackable-Level3.apk 1460555 B 6827f776b7d28443…
r2pay-v1.0.apk 4293620 B 850d7c98bed8df53…
```
Strength labels follow the repository's own three-tier rule (`references/long-task-discipline.md`
reserves `observed` for a claim with an exact command and its output behind it):
- `observed` — a command was run and its output is quoted in the evidence file.
- `inferred` — reasoned from documented mechanism or from a neighbouring measurement.
- `unverified` — assumed, or reported by someone else and not reproduced here.
| # | Dimension | Target (public source) | Scripts / documents the row exercises | Result | Strength | Evidence |
|---|---|---|---|---|---|---|
| B1 | Equal-length dex patch + repack + re-sign installability (G2/G4) | MASTG `UnCrackable-Level1.apk` | `dex_patch_bytes.py`, `repack.py`, `byte-level-patching.md`, `repack-and-sign.md` | **The route works end to end, and the change is visible on screen.** 2 bytes of the first root-check branch in `MainActivity.onCreate` (`0x9d0: 39000e00` -> `00000000`) patched inside a 5528-byte dex; the whole-file diff is those 4 bytes plus bytes 8..32, and the written file self-verifies (`adler32(d[12:])` and `sha1(d[32:])` both reproduce, proving signature-before-checksum order). `repack.py` produced a STORED + 4-byte-aligned `resources.arsc`, apksigner reported v1+v2+v3 all true, the root install path succeeded with the on-device `base.apk` sha256 equal to the local build, and the blocking `Root detected!` dialog is gone from the screenshots. A zero-change control through the identical pipeline still shows it | observed | `EXTENSION-benchmark-l1-l3.md` §2 |
| B2 | Anti-debug + self-destruct race (patch before the check fires) | MASTG `UnCrackable-Level3.apk` | `spawn_patch_detach.py`, `hook_patch_only.js`, `dynamic-frida.md` | **Split: the tooling works, the patch does not.** Time-to-death with a probe attached is 5.34 s. After an ordering fix (resume -> wait for `PATCHED` -> detach) the probe wrote `libfoo.so` base+0x2ff0/+0x2ff8/+0x3080 to `mov w0,#0 ; ret` 0.04-0.05 s after resume and the writes survived the detach — the mechanism is real. The patch still does not stop the death: the patched run **and** a read-and-write-back control both die with `SIGABRT` at `libfoo.so (goodbye()+12)` 3.22 s after resume, so neutralising the terminate routine plus both of its branch inputs is insufficient; the region is a dense cluster of `mov w0,#imm ; ret` stubs and locating the real second site needs a full `.text` disassembly. The alternative ordering ("hold paused until `PATCHED`") cannot work — a frozen process has no libraries mapped, so `Process.findModuleByName` never returns (0.11 s after resume it appears). Independently, the pristine original already reports tamper on this device, so the row's assumed clean baseline does not exist here | observed (tooling) / unverified (a surviving target) | `EXTENSION-benchmark-l1-l3.md` §3 |
| B3 | Extraction-shell skeleton detection (`trivial-body` routing) | self-built 11-variant skeleton set from the `r2pay-v1.0.apk` dex, with a zero-change control | `dex_dump_validate.py`, `advanced-unpacking.md` | **The "stub% threshold" claim this skill used to make is wrong.** `stub%` is *bimodal*, not thresholded: a real skeleton reads ≈100% (`allstub` 100.0%) while a control reads 1.9%, and the shapes real shells actually use are invisible — nop-cleared bodies 0.0% (below the control), throw stubs 1.9% (indistinguishable), partial extraction at 25/50/75% 1.7/1.7/2.1%. `advanced-unpacking.md`'s original "tens of percent means a skeleton" had no support and was deleted. Three script defects were found and fixed (all-nop bodies ranked as real; a legal minimal body mislabelled malformed; the ranking pointing at a 25%-extracted image as "most likely original" — caught by independent review). The new `minimal-form` counter *does* track the extraction ratio monotonically (1299/2524/3736/4946) and now participates in the ranking | observed | `EXTENSION-extraction-shell-bench.md` §1–3 |
| B4 | Dex-VMP boundary: refuse-to-repack and downgrade the deliverable | DexPatcher `assets/ezAndroid.apk` | `advanced-unpacking.md`, `java2c-and-jni-sinking.md`, `lsposed_scaffold.py`, `frida_rpc_serve.py` | **The sample is not a Dex VMP — it is JNI sinking**: 7 `native` declarations, each with a matching exported `Java_*` symbol, no `JNI_OnLoad` / `RegisterNatives` symbol, 34,566 instructions with zero structural errors under the official `dexdump`. A zero-change repack succeeded (v2 signing, alignment, `classes.dex` CRC verified). Static criteria **can** rule a dex-VMP *out*; they cannot rule runtime-applied hardening out. The first verdict here was the opposite — a hand-written opcode table reported 1393/22424 undecodable bodies and called it a VMP; `dexdump` showed the table was missing an arm64 branch opcode, and that rule (never declare VMP without an official disassembler's failure count) is now in the reference. Installation / launch / refusal remain untested | observed (static + repack) / unverified (install, launch) | `EXTENSION-extraction-shell-bench.md` §4 |
| B5 | Java2C recognition (do not misdiagnose as an extraction shell) | self-built with `amimo/dcc` (`--no-build`) + real JNI samples | `java2c_probe.py`, `java2c-and-jni-sinking.md` | Native-declaration density separates the shapes by ~2000x: **0.03–0.04%** on three real JNI samples vs **82.76%** on a Java2C-shaped fixture, and `Java_*` symbols matched dex native-method counts 1:1 per ABI (2:2, 3:3). **No Dex-to-C compiler output was ever built** (no NDK, no host clang, no device clang, WSL unavailable), so every Java2C-specific criterion is inferred while the JNI-sinking side is measured. A symbol search fails silently on real Java2C: dcc emits `-fvisibility=hidden`, and C++ `jni.h` inlines `RegisterNatives` so it leaves no symbol | observed (density, JNI sinking) / inferred (Java2C-specific) | `EXTENSION-java2c.md` |
| B6 | Native tracing: does `Stalker.exclude()` change the outcome? | MASTG `UnCrackable-Level1.apk`, module `libc.so` followed on the main thread for 6 s | `stalker_trace.js`, `native-dbi-and-deobfuscation.md` | **Split result.** A no-follow baseline survives; the same follow with `excludeModules: []` **kills the process**; with 20 system modules excluded it survives — so exclusion is what keeps the target alive (`observed`, with the baseline arm ruling out "frida killed it anyway"). But the treatment arm still reported `blocks=0 blk=0 calls=0`: **exclusion does not fix the zero-event trace**, which partially contradicts the community framing that lists "no events" among its symptoms. The pass also fixed two harness traps that manufacture a false zero-event result (a load-time default follow, and a suspended spawn) and verified a new zero-event `WARN`. No real OLLVM trace was produced | observed (survival, delivery) / unverified (the multiplier, the cause of zero delivery) | `EXTENSION-stalker-exclude.md` |
| B7 | Native certificate pinning | CyReverse Cronet sample | `protocol-reverse.md` | not run this pass — the CyReverse Cronet sample needs a gradle+NDK build this machine cannot do | unverified | — |
| B8 | Schemaless protobuf decoding | synthetic fixtures + the official runtime + a real AndroidX DataStore container | `protobuf_decode_raw.py`, `protocol-reverse.md` | 26/26 built-in fixtures, **21/21 against the official protobuf runtime** (re-encode byte-identical to `SerializeToString()`), 8/8 framing checks. A real 81-byte DataStore file round-tripped byte-exact and an edited value re-encoded (81->80 B, length prefix recomputed) was read back by `datastore_inject.py`. Both traps reproduced on real bytes: `0801120178` is a valid nested message *and* a valid packed array; a non-presence `0` is byte-identical to "never assigned". Four script defects were found by these checks and fixed | observed | `EXTENSION-protobuf-raw.md` |
| B9 | Split APK / App Bundle handling | two real split sets pulled from a device (8 members = base + 7 density splits each carrying its own `resources.arsc`; 9 members = base + ABI split with only `lib/` + 7 density splits with 40-byte placeholder tables) | `repack.py` (split mode), `split-apk.md` | Both branches of the decision ran on real sets: after `resign` every member shares one certificate with v1+v2+v3 all true; `merge` **correctly refused** the set whose members carry their own `resources.arsc` (exit 1, before writing anything) and **succeeded** on the code/native-only set, folding 2 `.so` files into `base`. The single-APK legacy path still passes its regression (`--dexdir` replacement and a zero-change round trip). Signing has no `uber-apk-signer` on this host; the measured call is `zipalign -p -f 4` then `apksigner.jar --v1 --v2 --v3`, and the order cannot be reversed (`[-124]`). `adb install-multiple` and launch confirmation were **not** executed | observed (analyze/resign/merge) / unverified (install, launch) | `EXTENSION-split-apk.md` |
| B10 | Kernel-side syscall answer forging | KernelSU/APatch module template, host toolchain, device kernel | `kernel-and-environment-hardening.md`, `kernelsu_syscall_mask.py` | The generator produced an 11-file userspace module skeleton plus KPM/LKM/eBPF templates and `verify` reports 0 problems — but **no kernel-side artefact was compiled or loaded**: the `<DEVICE>` kernel is 4.14.186 (eBPF needs 5.10+), the host has no `aarch64-none-elf-gcc` / `make` / `ndk-build`, and there is no kernel source. Documentation correction with teeth: **an ordinary KernelSU module is a userspace module and structurally cannot change a syscall return value** | observed (generation) / unverified (kernel side) | `EXTENSION-kernel-weapons.md` |
| B11 | JNI dynamic registration discovery (`RegisterNatives` with no `Java_*` symbol) | a real system library + CyReverse `plugin/` | `java2c-and-jni-sinking.md`, `java2c_probe.py` | A real dynamically-registering library contains **zero `RegisterNatives` symbols** (the C++ `jni.h` inlines the call), so symbol-based discovery fails structurally. The combination that did fire on a real library is "**exports `JNI_OnLoad` and zero `Java_*`**" | observed | `EXTENSION-java2c.md` |
## How to read a row
`Result` states what actually happened, including negative outcomes — "the route worked", "the route
worked only with `Stalker.exclude()`", "the tool refused to run at all". A negative result that is
recorded is worth more than a positive one that is assumed, because the next agent can route around
it without re-paying the cost.
A row whose `Strength` is `unverified` is a statement about *this repository's evidence*, not about
the mechanism's truth. It means: nobody has yet put a command and its output behind it here.
**Rows that changed a conclusion this repository had already recorded.** This is the outcome the
matrix exists to produce, so it is listed rather than buried: B3 (the trivial-body "threshold" became
a measured bimodal result), B4 (a VMP verdict was withdrawn after an official disassembler disagreed
with a hand-written opcode table), B6 (exclusion turns out to keep the process alive but does not fix
zero-event delivery, contradicting the community framing), and B1 (an earlier "`repack.py` has no
apksigner signing path" conclusion expired mid-pass when a second writer added one).