antior/apk-reverse
1
0
Fork 0
mirror of https://codeberg.org/reverse-skill-lab/apk-reverse synced 2026-11-15 20:44:22 +08:00
Code Issues Packages Projects Releases Wiki Activity

Absorption pass: take the mechanics from four external skill repositories, keep our discipline

Browse Source 
Four projects were cloned for comparison (reverse-skill, awesome-game-security,
android-reverse-engineering-claude-skill, MobileRE-Skill) and read for what they do that this
repository does not. None is vendored and none becomes a runtime dependency: what crosses over is a
phase order, a checklist or a command pattern, with the source URL and the retrieval date recorded in
the file that absorbed it.

The rule applied to every candidate: does this make a judgement easier or a verification harder? The
repository is a decision-and-verification layer, and the four things kept are all of that kind.

  new  scripts/scan_leaks.py            target identity scanned before publishing, with an explicit
                                        do-not-anonymize list and four RESULT=/exit states
  new  references/precedents/           the positive half of the record: route including dead ends, a
                                        grade per assertion, measured pits, and a write-back checklist
  new  scripts/svc_scan.py              does a libc-level hook structurally see this call at all
  new  scripts/anti_detect_probe.js     observer-only: name the check, and report the environment it reads
  new  references/desensitization-and-leak-scans.md
  new  references/dart-aot.md sections   the 32-bit snapshot's real string-record format

  check_repo.py now delegates the identity scan to scripts/scan_leaks.py and runs it over
  git ls-files, so the gate reads content instead of layout for the first time. An explicit file list
  cannot be narrowed by the default work-area exemptions -- that defect was caught by the gate itself.

Two claims that this repository had already written down turned out to be wrong, and both are
recorded as refuted rather than quietly rewritten:

  dart-aot.md   the armv7 string table was documented as [len*2 u32le][UTF-16LE]. It is not. The same
                extractor keeps 4,980 chained entries on arm64 and zero on armv7, and the byte layer
                shows the 32-bit record is [header u32][byte-count u32le][UTF-8 payload]: two adjacent
                literals carry 0x46 (70) and 0x48 (72), exactly their byte lengths. The zero was a
                format mismatch, not an empty table. The engine-version banner on the same target is
                66 bytes of "0.0.1" with a blank build id, so the documented regex returns a
                version-shaped lie.
  leak scan     the pass that added the scanner found 26 strong hits inside its own evidence file on
                the day it was written -- the fixture's literal match values, quoted to prove the
                rules fire. The corpus may contain values; the published surface may not.

Measured, not asserted. The check that fires was named on a public MASTG target: three arms with one
variable, the control alive at the end and the probe arm dead in about 300 ms, with strstr("frida") at
t=303 ms and no tombstone. Reproducibility is bounded honestly -- one full sequence in three arms, on
a target whose whole window is sub-second. svc_scan.py's two independent decoders agree on an
identical 214-site set, and ptrace-free dumping gained a negative boundary: a target whose dex is
deflated inside its APK has no whole-image dex in memory to export, for reasons that say nothing about
protection.

Also in this pass: the detector-locating pipeline (detection-and-anti-analysis.md Step 3) with its
branch conditions; the layered Java-to-svc descent and the dump's honesty boundary in
advanced-unpacking.md; a sourced capability-gap list in toolchain.md where every row is graded
URL-verified or measured; the RESULT=-token convention; and SKILL.md opening with an immediate-action
block, because the symptom index and the gates were sitting in the part of a long file nobody reads.

Gates: check_repo.py 0 problem, check_refs.py 0 dangling / 0 warning, check_budget.py 0 failure /
0 warning / 1 pre-existing note. The scan reports RESULT=leaks_found_strong_only, exit 0: four
endpoint/weak hits, all of them addresses from the RFC 5737 documentation range in usage examples.
This commit is contained in:
inliver233
2026-09-21 22:51:09 +08:00
parent fd2a1150be
commit 6cffc3d2ac
24 changed files with 3904 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tool-verification/EXTENSION-dart-aot-formats.md
+183
View File
@@ -0,0 +1,183 @@
# EXTENSION — Dart AOT string-table formats, measured on a real dual-ABI app
Evidence file for the `dart-aot.md` changes in this pass. Every claim below carries a grade:
- **observed** — a command was run here and its output is quoted in this file.
- **inferred** — follows from an observed fact or from documented mechanism; the step was not executed.
- **unverified** — assumed or reported elsewhere, not reproduced here.
Target identity is replaced with `<PKG>`; the artifact is referred to as "the sample". The technique
(tool, library, function, hardening product) is kept, per
`skills/apk-reverse/references/desensitization-and-leak-scans.md`.
## 1. What was measured
| Claim under test | Grade | Result |
|---|---|---|
| The arm64 Dart AOT string table is `[0x80\|(len<<1)\|two_byte][payload]` and the entries chain | **observed** | confirmed on 4,980 entries |
| The armv7 (32-bit) snapshot uses the *same* format | **refuted** | the same extractor keeps 0 entries; the tag byte is not present before the literal |
| An armv7 string record is `[length u32le][UTF-16LE chars]`, so the length field reads `len*2` | **refuted** | the u32 before the payload reads the **byte** count, and the payload is UTF-8 |
| The engine version banner always yields `x.y.z (stable) (…)` | **refuted** | a real `libflutter.so` carries `0.0.1` with a blank build id and no `(stable)` |
| An equal-length replacement of a data key is legal on both ABIs without recomputing the tag | **observed** (record layout), **unverified** (end-to-end device run) | the record is self-contained on both ABIs |
Two of the five were assumptions that would have been written into the reference as fact. Both were
killed by a single command each, which is the whole argument for measuring before documenting.
## 2. Sample and preparation
The sample is the locally supplied "complex encryption and hardening" APK (360 shell + Flutter).
The two AOT snapshots were extracted from the zip without repacking:
```
$ python -c "import zipfile; z=zipfile.ZipFile(r'存在问题和例子\<sample>.apk'); \
[print(n, len(z.read(n))) for n in ('lib/arm64-v8a/libapp.so','lib/armeabi-v7a/libapp.so')]"
lib/arm64-v8a/libapp.so 16352152
lib/armeabi-v7a/libapp.so 17629776
```
Work copies live under the ignored workspace `tools/_work/dart/`; nothing from this evidence pass is
committed except the reference text.
## 3. The arm64 packed table — confirmed
```
$ python skills/apk-reverse/scripts/dart_pool_strings.py tools/_work/dart/app_arm64.so s64.tsv --min 6
candidates : 17352
kept (run >= 3) : 4980 (one-byte 4980 / utf16le 0)
below threshold : 12372 [--keep-isolated to keep them]
written s64.tsv
```
The tag arithmetic was checked directly against real literals in the image rather than only through
the script. For each literal, the byte immediately before it must equal `0x80|(len<<1)`:
```
$ python -c "... for m in (b'AdmobBanner', b'showSplashAd', b'AdPgl_Splash'): ..."
### app_arm64.so
AdmobBanner @0x4932e len=11 expected_tag=0x96
tag_byte_at_-1=0x96 matches=True
showSplashAd @0x39deb len=12 expected_tag=0x98
tag_byte_at_-1=0x98 matches=True
AdPgl_Splash @0x493ee len=12 expected_tag=0x98
tag_byte_at_-1=0x98 matches=True
```
**observed**: three independent literals of two different lengths, three matching tag bytes. This is
also the assertion a patch script should perform (`dart-aot.md` §10), and it is what rejects a hit
that landed mid-string.
## 4. The armv7 assumption — refuted, and what is actually there
The starting hypothesis (carried over from an external case) was that the 32-bit snapshot stores a
string as `[tag 0x81][len*2 u32le][UTF-16LE chars]`. Two scanners written against that hypothesis
found nothing, which is itself the evidence:
```
$ python tools/_work/dart/probe3.py app_armv7.so raw # [8-byte hdr ending 0x81][u32le len][chars]
hits=0 chained_runs=0 entries_in_runs=0
$ python tools/_work/dart/probe3.py app_armv7.so packed # the arm64 tag scheme
hits=0 chained_runs=0 entries_in_runs=0
```
The same extractor that keeps 4,980 entries on arm64 keeps **zero** on armv7:
```
$ python skills/apk-reverse/scripts/dart_pool_strings.py tools/_work/dart/app_armv7.so s7.tsv --min 6
candidates : 1355
kept (run >= 3) : 0 (one-byte 0 / utf16le 0)
```
A zero here is a **format mismatch, not an empty string table**, and that distinction is now stated in
`dart-aot.md` §7: reading it as "the strings were stripped" is exactly the wrong conclusion the same
section warns about for UTF-16.
What the record actually looks like, from bytes around a known literal:
```
$ python -c "... b.find(b'SystemChrome') ..."
0x1ffb3c b'\x00\x00\x00\x008\x06U\x00F\x00\x00\x00\xce-\x8a@SystemChrome.setEnab...'
0x204ce4 b'\x00\x00\x00\x008\x06U\x00H\x00\x00\x00\xa45jVSystemChrome.setSyst...'
0x2185c4 b' ere8\x07U\x00J\x00\x00\x00\x8c]\xb4GSystemChrome.setPref...'
```
Read as `[header u32][length u32le][payload]`: `0x00550638` then `0x00000046` = **70**, which is the
byte length of `SystemChrome.setEnabledSystemUIMode` (36 chars) immediately followed by
`SystemChrome.setSystemUIOverlayStyle` (34 chars) — **70 bytes, UTF-8, not UTF-16**. The second
literal starts at the exact byte where the first ends, and its own length field reads `0x00000048` =
72, matching its own continuation.
**observed**: `length == payload byte count`, payload is UTF-8, and in-place equal-length replacement
therefore keeps the record valid without recomputing anything. **refuted**: the `len*2` / UTF-16 form.
The header word is a tags value with full 4-byte alignment, not a constant to hard-code:
```
$ python -c "... alignment histogram over the image ..."
alignment mod 4: {0: 13397}
```
```
$ python tools/_work/dart/perf.py app_armv7.so
scanned 17629776 B in 0.94s using 4-byte stride + int.from_bytes
aligned-header string records: 582
stride-1 micro scan over 17629776 B took 2.80s (found 2762)
```
The literal search plus a backwards length assertion is the tool-independent method, and it costs
under a second over a 17.6 MB image — so validating every candidate is affordable and the armv7
limitation does not justify a hand-built table parser. **unverified**: whether a *different* armv7
snapshot uses a different header word; the reference tells the reader to read it from the file.
## 5. The engine version banner — a version-shaped lie
```
$ python -c "... rfind NUL before the ' on \"android_arm64\"' suffix ..."
end-of-version offset 0x110821
version string start 0x1107f1 => total 66
b'0.0.1 on "android_arm64"'
```
The banner exists, is 66 bytes long, and its numeric field is `0.0.1` with a blank build-id field —
no `(stable)`, no `(Mon …)`. A regex that takes `\d+\.\d+\.\d+` out of this returns a value that would
send a reader to build a Dart VM for `0.0.1`.
**observed** (the byte string above), **inferred** (that this is a build-time population failure rather
than a deliberately rewritten banner). The actionable consequence is in `dart-aot.md` §1: assert the
`(stable) (…)` tail before using the number, and fall back to the snapshot hash otherwise. **The
sample's Dart version was not pinned in this pass** — that is an unverified item, not a finding.
## 6. Cross-validation
Every measurement above is a **raw byte search with an independent arithmetic assertion** (find the
literal in the file; check the byte or word immediately before it against the length of what follows).
It does not depend on `dart_pool_strings.py`, and the script's numbers are reported alongside the raw
probe rather than as the source of truth. The two agree on arm64 (4,980 kept entries, tag byte equal to
`0x80|(len<<1)` at every literal tested) and on armv7 (the packed scheme finds nothing, the byte layer
shows why).
**Not obtained**: a second, independently implemented Dart snapshot parser. `blutter` was not built in
this pass, so the reference's `pp.txt`-side claims were not re-derived here, and the armv7 conclusion
rests on the byte layout rather than on a tool's agreement. That is the boundary of this evidence.
## 7. What this pass changed in the repository
- `skills/apk-reverse/references/dart-aot.md` — §7 gained the armv7 record format and the
"the armv7 zero is a format mismatch" warning; §1 gained the banner-shape assertion; §10 gained the
data-key-versus-API-path rule with its measured failure mode, and the longest-first / assert-the-prefix
rule. No new file, no new index row: the finding belongs in the file that already owns this layer.
- The equal-length-replacement failure (a 404 whose body is not JSON, `jsonDecode` throwing, the app
frozen on the launch logo) is documented as the concrete instance of the top-level constraint
"do not make an API fail to suppress a UI element".
## 8. Not done, and known to be outstanding
1. **No end-to-end device run.** The string-table patch was not repacked, signed and installed in this
pass, so "an equal-length key replacement suppresses the feature while the app still starts" is
`unverified` end-to-end even though the record layout that makes it legal is `observed`.
2. **No `blutter` build**, so no `pp.txt` comparison and no confirmation of the arm64 pool space from a
second implementation (§6).
3. **The sample's Dart version is unpinned** (§5); a structural probe or a snapshot hash is the next
experiment, not a guessed number.
4. **The armv7 header word is not characterised.** It is read from the file and used as a validator;
which component assigns it was not determined.
docs/tool-verification/EXTENSION-desensitization.md
+454
View File
@@ -0,0 +1,454 @@
# Extension verification — desensitization and leak scans
Covers the pass that added `skills/apk-reverse/scripts/scan_leaks.py` and
`skills/apk-reverse/references/desensitization-and-leak-scans.md`, plus the precedent library
under `skills/apk-reverse/references/precedents/`.
Strength labels follow `docs/tool-verification/README.md`: **observed** (the exact command and
its output are reproduced here), **inferred** (follows from observed facts; the step itself was
not executed), **unverified** (assumed or reported elsewhere and not independently confirmed).
Reference environment: Windows 11 host, Python 3.14.0, repository at `E:\apk-reverse` on branch
`main`. **No device, no network and no target sample were involved** — this pass is entirely
machine-local, because the thing worth measuring is what a scanner reports against a tree whose
contents are already known. Every identifier in the fixture and in every transcript below is
synthetic; the repository's convention of `<PKG>` / `<DEVICE>` / `<serial>` is used wherever a
real one would have appeared.
Date of the recorded runs: 2026-09-21. The numbers in §3b and §4a were measured on that date, on
the tree this pass leaves behind; those two sections' transcripts are the ones this pass produced.
**Headline: the planted-leak corpus is reported in full (30 findings, 10 further shapes admitted
through the exemption path), and the repository scan converges to zero strong/certain findings —
the residue is four `endpoint/weak` hits, all four an RFC 5737 documentation address, which is a
class this scanner reports on purpose. The first version of this file was itself the
counter-example to its own subject: it quoted the planted values to prove the rules fire, and the
scanner found that as a leak (§3b). The corpus keeps its values, because a corpus that cannot be
found proves nothing; the published document keeps none. The gate is not wired into anything yet;
that is the largest gap, and it is a Lead-side action.**
---
## 1. What was built, and what it was measured against
| Artifact | Role | Measured? |
|---|---|---|
| `skills/apk-reverse/scripts/scan_leaks.py` | the scanner: 17 rules in 6 categories, exemption table, four `RESULT=` tokens, `--fail-on strong\|any` | **observed** — every run below |
| `tools/_work/leak-precedent/gen_fixtures.py` | generates the planted corpus and the exemption corpus | **observed** — 7 files, re-runnable |
| `tools/_work/leak-precedent/fixture/` | the corpus itself (git-ignored work area) | **observed** |
| `skills/apk-reverse/references/desensitization-and-leak-scans.md` | the reasoning, the do-not-anonymize table, the failure modes | not executable |
| `skills/apk-reverse/references/precedents/**` | the case library, distilled from this repository's existing evidence | see the per-case strength tables |
The fixture exists so the claim "the scanner reports planted leaks" is reproducible rather than
a transcript pasted once:
```
$ python tools/_work/leak-precedent/gen_fixtures.py
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\package.txt (411 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\device.txt (218 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\token.txt (261 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\appkey.txt (212 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\endpoint.txt (146 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\planted\path.txt (175 bytes)
wrote E:\apk-reverse\tools\_work\leak-precedent\fixture\exempt\benign.txt (1396 bytes)
== 7 fixture file(s) ==
```
## 2. `--help`, and the rule table — observed
```
$ python skills/apk-reverse/scripts/scan_leaks.py --help
usage: scan_leaks.py [-h] [--root ROOT] [--only ONLY] [--exclude EXCLUDE]
[--format {text,json}] [--show-exempt] [--list-rules]
[--max MAX] [--quiet] [--fail-on {strong,any}]
[--no-color]
Scan a skills repository for target identity that should not be published (bundle ids, device serials, tokens, SDK keys, literal endpoints, user paths). Exit 0 clean / 1 leaks_found / 2 error; prints RESULT=<token> last. `--fail-on strong` (default) keeps the gate green when the only hits are documentation-range addresses.
options:
-h, --help show this help message and exit
--root ROOT repository root to scan (default: the repo this script
lives in)
--only ONLY comma-separated categories to restrict to, e.g.
pat,appkey
--exclude EXCLUDE extra directory name to skip (repeatable)
--format {text,json}
--show-exempt also print findings that were suppressed by an
exemption
--list-rules print the rule table and exit
--max MAX stop after N findings (0 = no limit)
--quiet print only the RESULT token
--fail-on {strong,any}
which findings make the exit code non-zero: `strong`
(default) fails only on strong/certain findings, `any`
also fails on `weak` ones such as an address from the
RFC 5737 documentation range
--no-color accepted for compatibility
Categories: package, device, token, appkey, endpoint, path
Exemptions (tool names, library names, CVE ids, hardening products, public
crackme names, <PKG>-style placeholders, loopback/emulator addresses) are built
in and audited with --show-exempt.
```
`--list-rules` prints the table the exemption functions are attached to, which is what makes the
scanner arguable rather than opaque. It exits 0 and prints `RESULT=clean` (no files were read),
and the top of its output is:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --list-rules
== rules ==
bundle_pkg_attr package strong a `package=` bundle id in a manifest or transcript
bundle_pkg_decl package strong a bundle id in a `<manifest>` line
bundle_pm_path package strong a bundle id in a `pm`/`am`/`monkey` invocation
bundle_process_line package strong a bundle id in a `ps`/`pidof`/`top` capture
bundle_component package strong a bundle id in a `component=`/`-n` activity reference
device_serial device strong an `adb devices`-shaped device serial
device_serial_context device strong a serial next to a device/install/`su` context word
device_serial_tabular device strong a bare 16-char token in a device-listing table
token_github_pat token certain a GitHub fine-grained personal access token
token_github_classic token certain a GitHub classic token
token_env_assignment token strong an inline API-key/token/secret assignment
token_authorization_header token strong a literal Authorization header value
appkey_assignment appkey strong an SDK appkey/appsecret assignment with a literal value
plain_addr endpoint weak a non-loopback literal IP:port
plain_addr_ipliteral endpoint weak a non-loopback bare IPv4 literal
user_path_posix path strong an absolute POSIX user-home path
user_path_windows path strong an absolute Windows user-profile path
RESULT=clean (exit 0)
```
## 3. The planted corpus — observed, every rule fires
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --format json
{ "root": "...\\tools\\_work\\leak-precedent\\fixture", "files_scanned": 7,
"findings": [ ... 30 ... ], "exempted": [], "errors": [], "result": "leaks_found" }
RESULT=leaks_found (exit 1)
```
aggregated (the `findings` array of the same JSON run, grouped by category):
```
files 7
findings 30 {'appkey': 4, 'token': 5, 'device': 8, 'endpoint': 5, 'package': 5, 'path': 3}
```
and the 30 hits by position, rule and value **shape** — every field of the report except the
literal itself. Rule name, category, strength, position and count are the evidence; the quoted
value is a copy of the leak, and copying it into a published document re-creates what the scan
exists to prevent (§3b is where that rule was paid for):
```
planted/package.txt:2:16 bundle_pkg_decl 4-segment all-lowercase reverse domain
planted/package.txt:2:26 bundle_pkg_attr the same value, second rule on the same line
planted/package.txt:3:1 bundle_pm_path 3-segment all-lowercase reverse domain, after `pm path`
planted/package.txt:5:1 bundle_process_line 4-segment all-lowercase reverse domain, after `pidof`
planted/package.txt:6:15 bundle_component 4-segment all-lowercase reverse domain, after `component=`
planted/device.txt:4:1 device_serial 16×[A-Z0-9], tab-separated from the word `device`
planted/device.txt:4:1 device_serial_tabular the same value, second rule on the same line
planted/device.txt:5:3 device_serial_context 16×[A-Z0-9], after `$ adb -s `
planted/device.txt:5:10 device_serial the same value, second rule on the same line
planted/device.txt:6:2 device_serial_context 16×[A-Z0-9], after `[serial `
planted/device.txt:6:9 device_serial the same value, second rule on the same line
planted/device.txt:7:1 device_serial_context 16×[A-Z0-9], after `device serial: ` (a second value)
planted/device.txt:7:16 device_serial the same value, second rule on the same line
planted/token.txt:2:14 token_github_pat fixed provider prefix + 62×[A-Za-z0-9_]
planted/token.txt:3:15 token_github_classic fixed 4-char provider prefix + 36×[A-Za-z0-9]
planted/token.txt:4:1 token_env_assignment 20-char `AKIA`-prefixed key id after a key assignment
planted/token.txt:5:9 token_authorization_header 32×[A-Za-z0-9] after an `Authorization:` keyword
planted/appkey.txt:5:1 token_env_assignment the same 40-char secret, second rule on the same line
planted/appkey.txt:2:1 appkey_assignment 24×[0-9a-f] after an appkey assignment
planted/appkey.txt:3:1 appkey_assignment 24×[0-9a-f], quoted, after a camelCase app-secret assignment
planted/appkey.txt:4:1 appkey_assignment 7 dash-separated 4-character uppercase groups
planted/appkey.txt:5:1 appkey_assignment 40-char mixed-case secret after a `SECRET_KEY` assignment
planted/endpoint.txt:2:7 plain_addr an RFC 5737 documentation address, with `:port`
planted/endpoint.txt:2:7 plain_addr_ipliteral the bare form of the same address
planted/endpoint.txt:3:16 plain_addr an RFC 1918 private-range address, with `:port`
planted/endpoint.txt:3:16 plain_addr_ipliteral the bare form of the same address
planted/endpoint.txt:4:12 plain_addr_ipliteral an RFC 5737 documentation address, bare
planted/path.txt:2:10 user_path_posix two-segment personal name after `/home/`
planted/path.txt:3:6 user_path_posix one-word personal name after `/Users/`
planted/path.txt:4:6 user_path_windows capitalized personal name after `C:\Users\`
```
Two rules firing on one line (`SECRET_KEY=` matches both an appkey and a token pattern) is kept
deliberately and is visible in the count; identical duplicates (same rule, same value, same
line) are collapsed. Eight of the 30 findings share their line with a second rule — one package
line, all four serial lines, the `SECRET_KEY` line and the two endpoint lines — which is why 30
findings cover 22 distinct values (counted from the `findings` array of the JSON run above, not
by hand).
### 3a. The exemption corpus — observed, and it is the half that is easy to get wrong
The same run with `--show-exempt` shows what was *not* reported from
`fixture/exempt/benign.txt`, and the reason:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --show-exempt
== suppressed by exemption: 10 ==
exempt/benign.txt:13 [user_path_windows] <user> -> placeholder user
exempt/benign.txt:17 [plain_addr] 127.0.0.1:27042 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 127.0.0.1 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr] 0.0.0.0:8080 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 0.0.0.0 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr] 10.0.2.2:8080 -> loopback/unspecified/emulator address
exempt/benign.txt:17 [plain_addr_ipliteral] 10.0.2.2 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr] 127.0.0.1:8787 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr_ipliteral] 127.0.0.1 -> loopback/unspecified/emulator address
exempt/benign.txt:18 [plain_addr_ipliteral] 169.254.169.254 -> loopback/unspecified/emulator address
```
The earlier pass recorded 6 suppressed hits, all `plain_addr`; the current run reports the paired
`plain_addr_ipliteral` for each exempt address as well, which is what the two same-line rules
look like when nothing is being collapsed. The count moved from 6 to 10 for that reason and not
because the exemption table changed — nothing in it was touched by this pass.
Everything else on that file — tool names, library names, function names, dex/ELF constant
identifiers, protocol field names, CVE ids, hardening product names, the public crackme and its
URL, platform and SDK packages, `com.example.*` / `probe.synthetic.*` and the `<PKG>`/`<DEVICE>`/
`<hash>` placeholders — produced **no finding at all**, which is the intended outcome for the
do-not-anonymize list and is also the check that the two synthetic fixture prefixes named in
`references/desensitization-and-leak-scans.md` really are exempt: `com.example.app` sits on line
83 of that file and is absent from both the findings and the suppressed list, because
`BENIGN_PACKAGE_PREFIXES` contains `com.example`, `org.example`, `net.example`, `io.example` and
`probe.synthetic`. The file was written to hold every category of that list, one line each, so a
regression there shows up as a new finding rather than as a silent behavioural drift.
`169.254.169.254` (the cloud metadata endpoint) is exempted **only** in this documented-endpoint
form. It is a real address that matters in a different context; the exemption is a judgement
recorded in `BENIGN_IP_PREFIXES`, not a pattern match, and it is the entry most worth
re-examining if this list is ever copied to another repository.
### 3b. The leak this file carried — found by the scanner it documents
The first version of this section listed all 30 hits **with their values**, quoted out of the
JSON, because a report with the value in it is the concrete kind. The Lead then ran the tool
against the repository and got `RESULT=leaks_found`, exit 1, with **26 strong/certain findings
and 14 weak ones — all 26 strong ones inside this file**, at lines 113–146 of the version that
existed then: the 5 bundle-id hits, 2 distinct device serials (12 hits), 2 provider-format tokens,
2 inline assignment values, 2 SDK appkeys and 3 host user names it had just finished quoting.
Nothing else in the tracked surface produced a strong hit.
That is the failure mode this whole pass exists to prevent, committed by the pass itself, and it
states the boundary in one line: **the corpus is where identity is allowed; the published surface
is not.** `tools/_work/leak-precedent/` is git-ignored and excluded from the default scan root so
that it *can* hold the values — a corpus that does not contain them cannot demonstrate the rules
firing. This file is not in that category: it is tracked, it is read, and every string in it is
published.
The correction, applied in this pass:
1. The hit list in §3 now carries rule, category, strength, position, count and a **structural
description** of each value, in the repository's own placeholder notation. No evidence was
dropped: all 30 positions, all 17 rule ids, both strength labels and the per-category counts
are still here, and the corpus still produces all 30 when it is scanned directly (§3).
2. The two version-shaped literals in §4's false-positive table are now written in
`<N>.<N>.<N>.<N>` form. They are legitimate values, but leaving them would put two *different*
classes in the weak residue instead of one, and an unexplained second class is how a weak hit
starts getting ignored.
3. `references/desensitization-and-leak-scans.md` records the rule this produced — a report is a
copy of what it found, so the report is scanned like any other published surface — and its
exit table now carries the `leaks_found_strong_only` state.
The scanner's own verdict on the corrected tree is in §4a. The lesson is the one the reference
file already made about entry points, applied to an evidence file: **run the scan on the file you
are about to publish, not on the file you started from** — and then do not paste the thing it
found back into that file.
## 4. The repository scan — observed, and the false positives that produced it
The first run against the real tree reported **28 findings**, of which 27 were wrong. The
sequence is recorded in full because the corrections are the useful part: every one of them was
found by running the scanner, not by reasoning about it.
```
$ python skills/apk-reverse/scripts/scan_leaks.py # first run
== leak scan: 118 file(s) scanned under E:\apk-reverse ==
findings by category: device=4, endpoint=20, package=4
== result: 28 finding(s) == RESULT=leaks_found (exit 1)
```
| # | Reported | Why it was wrong | Fix |
|---|---|---|---|
| 1 | a `jdk`-version-shaped run (`…\jdk-<N>.<N>.<N>.<N>`) ×19, across eight files in `docs/tool-verification/` | it is the JDK version in the toolchain's own path; the token `jdk` sits immediately before the match | a 30-character context window before the literal must not end in a version word (`jdk\|java\|python\|frida\|…`) |
| 2 | `0000000040001000`, `0000000000714240` | tombstone register/`pc` values — 16 hex digits, all digits | an all-digit 16-character token is suppressed; a real platform serial carries at least one letter |
| 3 | `0807060504030201` ×3 | a protobuf `fixed64` fixture in `EXTENSION-protobuf-raw.md` and in `scripts/protobuf_decode_raw.py` | same rule as #2 |
| 4 | `sg.vantagepoint.uncrackable3` | the published package name of an OWASP MASTG crackme | `PUBLIC_TARGET_PREFIXES` (public crackme/benchmark families are reuse, not identity) |
| 5 | `probe.synthetic.vmpdiff` | the synthetic manifest inside `vmp_diff_harness.py` | added `probe.synthetic` to the fixture prefixes |
| 6 | `args.package.spli` (twice truncated to `spli`) | a property lookup in `scripts/spawn_patch_detach.py`; the exemption window was measured from the *match* start, which an unbounded `[^\n]*?` prefix pushes past the `args.` | the window starts at the **capture group** (`m.start(group)`), and the exemption reason is printed so the path can be audited |
| 7 | `<N>.<N>.<N>.<N>` | an SDK version *directory* (`…/com.kwad.components.tachikoma/<N>.<N>.<N>.<N>/base-1_apk`) inside a private-storage path | a version-directory rule: a `.`-separated digit run enclosed in path separators is not an address |
Two rows in that table are quoted in placeholder form rather than as the literal the first run
printed — row 1's four-part JDK version run and row 7's four-part SDK version run. Neither names
anybody: both are version numbers, and both were rewritten in this pass for the reason given in
§3b, because leaving them would put a second class (version runs, not addresses) into the weak
residue, where an unexplained class is the first thing a reader learns to skip. The shape is what
each row is about, and it survives the rewrite intact.
### 4a. What this pass inherited, and what it leaves — observed
Inherited state, reproduced at the start of this pass (same tree, before the §3b correction):
```
$ python skills/apk-reverse/scripts/scan_leaks.py
== leak scan: 128 file(s) scanned under E:\apk-reverse ==
findings by category: appkey=2, device=12, endpoint=14, package=5, path=3, token=4
== result: 40 finding(s) -- each one is a line to look at, not a verdict ==
RESULT=leaks_found (exit 1)
```
26 of the 40 are strong/certain (`package` 5, `device` 12, `token` 4, `appkey` 2, `path` 3) and
every one of those 26 is a line of this file's former §3 hit list, at lines 113–146. The other 14
are weak: 5 more lines of that same block (its endpoint rows), 2 version strings in §4's own
table, 3 RFC 5737 range literals in
`skills/apk-reverse/references/desensitization-and-leak-scans.md`, and `scripts/tls_check.py`'s
usage example.
Post-correction, on the tree this pass leaves behind:
```
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet
RESULT=leaks_found_strong_only (exit 0)
$ python skills/apk-reverse/scripts/scan_leaks.py
== leak scan: 129 file(s) scanned under E:\apk-reverse ==
findings by category: endpoint=4
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:10 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:26 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/references/desensitization-and-leak-scans.md:81:45 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
skills/apk-reverse/scripts/tls_check.py:44:23 [endpoint/weak] plain_addr_ipliteral
match: <RFC 5737 literal>
== result: 4 finding(s) -- each one is a line to look at, not a verdict ==
all are `weak` (the documented address ranges); the gate is green under the default --fail-on strong, and `--fail-on any` is the strict pass
RESULT=leaks_found_strong_only
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet --fail-on any
RESULT=leaks_found (exit 1)
```
The four residual hits, by position, rule and class. Their individual values are deliberately not
reprinted here — that is §3b's correction, and re-printing them is how this file leaked the first
time. All four are the same class:
| position | rule / strength | class |
|---|---|---|
| `skills/apk-reverse/references/desensitization-and-leak-scans.md:81` ×3 | `plain_addr_ipliteral` / weak | the three RFC 5737 documentation blocks, named inside that file's own statement that they are *not* exempt |
| `skills/apk-reverse/scripts/tls_check.py:44` | `plain_addr_ipliteral` / weak | the script's usage example, an RFC 5737 address |
Reading: **zero strong/certain findings and zero unexplained hits on the tracked surface; four
weak ones, one class, both sources named.** The residue is the price of the design decision
recorded in §"Deliberately *not* exempt" of the reference file: the documentation that states the
decision has to name the ranges, and the ranges are exactly what the rule reports. That is why
the gate's pass condition lives in `--fail-on` and not in the exemption table, and why
`--fail-on any` stays red on this repository on purpose.
The file count is 129 here, 128 in the inherited run above and 119 in the first pass: the tracked
surface grows while several writers work on it, and this pass added no file to the scanned
surface, so the count is a property of the run rather than of the leak. The number that is
comparable across the runs is the strong/certain one: **26 inherited, 0 now**. The inherited
figure also reproduces the Lead's own catch (§3b) exactly, which is the point of quoting the
transcript rather than the conclusion.
Two work artifacts of this pass live beside the corpus in the same git-ignored directory, and
neither is part of the scanned surface: `EXTENSION-desensitization.before.md` (the pre-correction
snapshot of this file, kept so that the correction is auditable in both directions) and
`fixture-scan-before.txt` (a captured run of the fixture scan). Both contain the planted values
by construction, exactly like the corpus, and both are excluded from every default run — the work
area is where identity is allowed to exist; that is the whole of §3b's boundary.
## 5. The exit-code contract — observed, all four states
```
$ python skills/apk-reverse/scripts/scan_leaks.py --root LICENSE --quiet
RESULT=clean (exit 0)
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet
RESULT=leaks_found_strong_only (exit 0) # the repository tree, weak hits only
$ python skills/apk-reverse/scripts/scan_leaks.py --quiet --fail-on any
RESULT=leaks_found (exit 1) # the same tree, strict pass
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/leak-precedent/fixture --quiet
RESULT=leaks_found (exit 1)
$ python skills/apk-reverse/scripts/scan_leaks.py --root tools/_work/nope-not-here
error: root does not exist: E:\apk-reverse\tools\_work\nope-not-here [stderr]
RESULT=error (exit 2)
$ python skills/apk-reverse/scripts/scan_leaks.py --only nope
error: unknown category (or categories): nope [stderr]
RESULT=error (exit 2)
```
The token and the exit code are deliberately separate channels: a wrapper that only greps
stdout cannot tell "clean" from "the scanner never ran", and one that only reads `$?` cannot
explain itself in a log.
`--fail-on` is what keeps those two channels from contradicting each other on a tree that is
documentation-shaped. Under the default (`strong`) a tree whose only findings are `weak` reports
`leaks_found_strong_only` and exits 0: the findings are still printed in full, they simply do not
fail the gate. Under `--fail-on any` the same tree reports `leaks_found` and exits 1, which is the
right verdict at publish time and the wrong one for a habit that runs on every change — a gate
that is red on its own design documentation is a gate people disable.
## 6. What was not done
- **The `RESULT=` token convention was undocumented when this pass started, and it is documented
now.** The Lead added `references/long-task-discipline.md` §Every script answers with a token,
not with prose during this pass, so the scanner is no longer the convention's only user. That
section's vocabulary is generic (`clean`, `patched`, `success`, `crash`, `timeout`,
`unavailable`, …) and does not spell out `leaks_found` / `leaks_found_strong_only`; whether
they belong in the enumeration is a Lead call. Verified with
`grep -n "RESULT=" skills/apk-reverse/references/long-task-discipline.md` → one match, line 70.
- **Nothing is wired into a gate.** No pre-commit hook, no CI job, no line in the maintenance
three-command habit. The scanner has never run automatically; every result above is a manual
invocation. Recorded as `unverified` in the sense that matters: whether it *would* be run is
untested.
- **No scan of the git-tracked list specifically.** The scan walks a filesystem root, so its
coverage is "the default surface under this root", not "the files that exist in HEAD". A file
that is present locally but ignored, or tracked but absent, is not distinguished.
- **No check of binary or archive content.** `*.apk`, `*.png`, `*.pcap` and `*.zip` are not
opened. Identity inside a screenshot or a capture file is real and this pass does not address
it.
- **No git-history scan.** A string deleted from the working tree remains in the history; nothing
here inspects commits. This pass did check the one case it created: `git status --short` shows
`docs/tool-verification/EXTENSION-desensitization.md` as `??` (untracked), so the values §3b
removed were never committed and nothing was written into history. That is a check of this
file, not a history scan.
- **No second-language rule set.** Identifiers that are not reverse-domain shaped (a nickname, a
local-part email, a CJK app name) are not covered. The rules are shape-based by construction.
- **No cross-repository validation of the exemption list.** The benign prefixes were derived from
this tree's own contents (a grep over its Markdown for reverse-domain tokens). Whether the same
list is right for another repository is `unverified`, and the reference file says so.
- **The `--max` / `--only` / `--exclude` paths were exercised only in the forms shown above.**
`--max` was not driven at all.
## 7. One-line deliverable statement, and the open items
**Deliverable (verified):** `python skills/apk-reverse/scripts/scan_leaks.py` scans a repository
root for six categories of target identity, reports each hit with a limited context window and a
strength label, exits 0/1/2 with a matching `RESULT=` token, reports all 30 planted leaks in the
reproducible corpus at `tools/_work/leak-precedent/fixture` (7 files, 10 further shapes admitted
through the exemption path), and against this repository's 129-file tracked surface returns zero
strong/certain findings — the residue is four `endpoint/weak` hits, all RFC 5737, both sources
named in §4a — so the default gate is green (`RESULT=leaks_found_strong_only`, exit 0) while
`--fail-on any` stays available as the strict pass (`RESULT=leaks_found`, exit 1).
**Open items for the Lead:**
1. Wire the scan into the maintenance gate (`check_repo.py`, `check_refs.py`, `scan_leaks.py`)
and record in `README.md` that the third one's exit 1 is a *content* finding, not a
mechanical failure like the other two.
2. Decide on a history scan. Nothing in this pass looked at commits, and a leak removed from the
tree is still in them.
docs/tool-verification/EXTENSION-detection-pipeline.md
+311
View File
@@ -0,0 +1,311 @@
# Extension verification — detection pipeline, layered descent, ptrace-free dumping
Covers the pass that added the §Step 3 locating pipeline and the §Step 2B boundaries to
`references/detection-and-anti-analysis.md`, and the §What this route cannot do /
§Is this dex image real / §Layered descent sections to `references/advanced-unpacking.md`.
It also ships `scripts/svc_scan.py` and `scripts/anti_detect_probe.js`.
Strength labels, as defined in this directory's README:
- **observed** — reproduced here, with the exact command and its output.
- **inferred** — follows from observed facts, but the step itself was not executed.
- **unverified** — assumed, or reported by a tool and not independently confirmed.
**Target identity is deliberately absent.** The hardened sample appears as `<PKG>`; the
public MASTG challenge apps appear by their package names because they are public
published targets, not a user's app. No keys, signatures or user data are recorded.
## 0. Environment and what was decided before measuring
| Item | Value |
|---|---|
| Device | Android 11 (API 30), arm64-v8a, kernel **4.14.186**, Magisk alpha + root, toybox shell |
| Host | frida 16.7.19 / frida-tools 13.7.1; capstone; lief; pyelftools |
| Instrumentation server | **a `frida-server` was already running on the device** (`frida-server -l 0.0.0.0:27099`, pid 31859) with a leftover `adb forward tcp:27099` |
| Decision taken | **Reuse it rather than restart it.** A restart is a device-wide variable and the process had a live abstract socket (`unix:abstract=/frida-...)`; it was never started by this pass, so nothing about it is attributed here. Cost: the server's listening port and its path are part of the environment a detector can see — see §4, where a probe confirms frida artefacts *are* visible in the target's own maps |
| Root hiding | `magisk --denylist status` = *enforced*, list **empty** → no app had root hidden. Not changed in this pass |
`frida-ls-devices` cannot be used as a liveness check in this harness (it requires a
Windows console and dies with `NoConsoleScreenBufferError`); the Python binding is the
reliable route, and that is what every measurement below used:
```python
dev = frida.get_device_manager().add_remote_device('127.0.0.1:27099')
dev.query_system_parameters()
# {'arch': 'arm64', 'os': {'version': '11', 'id': 'android'}, 'api-level': 30, 'access': 'full'}
```
## 1. `scripts/svc_scan.py` — measured, with the count cross-checked
`--help` works; a real call was made on five inputs. T3 cross-check first, because the
whole script is a decode claim:
```
json sites = 214
hand word-scan = 214
set difference = []
```
The hand word-scan is a 6-line script that matches the `svc` encoding
(`(w & 0xFFE0001F) == 0xD4000001`) with no dependency on capstone, so the two readings
are genuinely independent implementations. **observed.**
Device libraries, scanned from copies pulled with `su -c cp`:
| Input | Sites | by_group | Notable |
|---|---|---|---|
| `/apex/…/bin/linker64` (1,451,392 B) | 214 | termination=4, evasion=5, discovery=8, unresolved=6 | the 4 termination sites are `exit`/`exit_group` at `0xda284`/`0xda2a4` |
| `/apex/…/lib64/bionic/libc.so` (1,193,024 B) | 215 | termination=4, evasion=6, discovery=8, unresolved=6 | `kill @0xc5b94`, `tgkill @0xc5bb4`, `ptrace @0xc5bd4`, `mprotect @0xc5f54`, `exit_group @0xc7054`, `exit @0xc7074` |
| `assets/libjiagu_a64.so` from the sample (1,171,896 B) | 21 | **unresolved=21** | every site is inside `scvtf`/`udf`/NEON byte soup — data, not instructions |
| `assets/libjiagu.so` (32-bit, 861,436 B) | 11710 | unresolved=11116 | scanning both PT_LOAD segments mixes code and data; the site count is not usable without inspecting neighbours |
**The interpretation that matters, and it is the reason the script exists:** the four
termination sites in `libc.so` are libc's *own* exported implementations, so a hook on
`exit`/`kill` does see callers that go through libc. The shell library has **no**
termination site and 21 data artefacts — so on this target a libc-level hook is *not*
structurally excluded, and a missing exit event would be a finding about the hook.
**observed.**
Two defects were found by running it and both are fixed in the shipped script:
- `capstone` raises `CS_ERR_DETAIL` unless `detail = True` is set; the first run died on the
first hit (`AttributeError`-equivalent at `insn.operands`).
- `pf_x` is **not** usable as a filter on this platform. The ROM's own `libc.so` and
`linker64` carry `pf_x` set on their code segment but a first version of the probe read
`p_flags` from the wrong struct offset (reading `p_offset`'s low word instead), which
reported `pf_x=0` for everything. The shipped version reports per-segment `pf_x` and does
not filter on it by default: `--code-only` is opt-in precisely because a device that
clears the flag would otherwise return zero hits silently.
## 2. The hardened sample: what it does now, and why nothing was dumped from it
The previous pass recorded this sample as a clean self-exit on a 7-18 s cycle with no
tombstone. **That is no longer the shape.** Measured this pass, twice, 20 samples at 1 Hz
after `monkey` launch:
```
22:28:19.923 D/OppoExSurfaceFlinger: hecate traverse … Splash Screen com.gentle.ppcat#0
22:28:21.684 W/ActivityManager: Process ProcessRecord{… 14905:com.gentle.ppcat/u0a623} failed to attach
22:28:21.686 I/ActivityManager: Killing 14905:com.gentle.ppcat/u0a623 (adj -10000): start timeout
```
- **observed:** the platform kills the process on **`failed to attach` / `start timeout`**
roughly **2 s** after start, i.e. the app never finishes starting.
- **observed:** an attempt to harvest memory inside that lifetime failed on both tries —
`RESULT=NO_PID` and `RESULT=NO_PID` with `t_pid=9.99`, so a 10 s window is already longer
than the process lives. Two strikes, no third variant: **the ptrace-free dump was not
obtained from this sample and this file does not claim one.**
- **observed, and a new signal rather than a repeat:** a tombstone is now produced, and its
fault is not in the packer's own code but in the runtime, reached from JIT output:
```
signal 11 (SIGSEGV), code 2 (SEGV_ACCERR), fault addr 0x75f16ff028
backtrace:
#00 pc 0000000000574028 /apex/com.android.art/lib64/libart.so
(art::Runtime::SetJavaDebuggable(bool))
#01 pc 0000000000022d58 /memfd:jit-cache (deleted)
```
A self-terminating `SIGABRT` would be the expected shape for an anti-analysis kill; a
`SEGV` in `Runtime::SetJavaDebuggable` reached **from JIT-compiled Java** is not, and the
honest reading is limited: **the app now crashes during startup, in the runtime, on a
path a Java method entered.** Whether that is the packer's environment check firing or a
consequence of the kill arriving mid-startup is **unverified** — the sample's own
start/stop behaviour changed between passes and no control run with root hiding
configured was made, because changing root-hiding state is a device-wide variable this
pass deliberately did not touch.
## 3. Clean-target baseline: state, liveness, and a harness artefact avoided
- **observed:** `owasp.mstg.uncrackable1` launched with `monkey` stayed alive at pid 15310
for **more than a minute** of repeated checks — the stable baseline every dynamic
experiment needs, and the reason the arms below were run on it and on L3 rather than on
the hardened sample.
- **observed:** `owasp.mstg.uncrackable3` runs two processes (zygote64 + 32-bit), had its
own integrity log line, and stayed up.
- **observed, and the most expensive-looking failure that turned out to be free:** the
first attach to L1 hung and then failed —
`attach/spawn failed: unexpectedly timed out while waiting for signal from process with PID 15310`
— and a second attach said `process not found`. The process was still in `ps` and
`dev.enumerate_processes()` still listed it, but:
```
$ adb shell 'cat /proc/15310/stat | cut -d" " -f3'
D
```
**`D` (uninterruptible disk sleep)** is why. A userspace attach needs the process to run.
The one-line control that settled it in a single command: attaching to a *different* pid
from the same device succeeded (`attach 15974 OK`). **observed** — the failure was the
environment, not a detector, and this is now Stage 0 of the reference's pipeline.
**Harness self-check applied before interpreting anything** (the discipline that a spawn
which is never resumed produces zero events and reads as a hang): the driver
(`tools/_work/detect/run_anti_detect_probe.py`, a work-file, not shipped) resumes
**explicitly** after the script is loaded, and records whether it did. Both arms below
report `resumed=True`, so "no events" cannot be an un-resumed spawn here.
## 4. The three arms, on one target — and the detection actually found
All arms: `owasp.mstg.uncrackable3`, `--spawn`, script loaded before resume.
| Arm | Script | Result |
|---|---|---|
| **A. baseline** | none (launch, no instrumentation) | process alive and stable across a minute of sampling (§3) |
| **B. control** | attached/spawned, resumed, **no script loaded** | **`alive_at_end=True`**, survived the full window |
| **C. experimental** | `scripts/anti_detect_probe.js` (observer-only) | **`alive_at_end=False`**, process gone before the first heartbeat |
Control vs. experimental differs in exactly one variable — the probe script — so the death
is attributable to instrumentation. Then the observer's own stream named the check:
```
t=0 [armed] script armed
t=207ms [thread_create] pthread_create entry=0x76dc031d78
t=238ms [strcmp_probe] strcmp("Ldalvik/system/ClassExt;")
t=245ms [env_probe] open("/system/fonts/Roboto-Regular.ttf")
t=254ms [loader] android_dlopen_ext("…/oat/arm64/base.odex")
t=303ms [strstr_probe] strstr("frida") <-- the check
t~350ms process gone, heartbeats=0, no tombstone, no crash/ANR record
```
- **observed:** the target **searches for the string `frida`** (`strstr("frida")`) at ~300 ms
of its life. This is the "detection really did happen" datum the task asked for: not an
inference from a death, a recorded call with a timestamp.
- **observed:** an earlier arm of the same shape also captured the kill itself —
`[termination] raise(...) <- …` at **t=352 ms**, i.e. a signal-based self-abort roughly
50 ms after the `strstr`. No tombstone was written and no `has died: crash` record
appeared, which is the clean-self-exit signature.
- **observed, and the reason the arm is reported as unstable:** three attempts at the same
arm produced *one* full sequence, then two runs where the script received **zero**
messages before the process died (`live=0, messages=0, heartbeats=None`). On a target that
kills itself in ~300 ms the observable window is sub-second and **not reproducible arm to
arm**. One successful attribution is therefore not a pipeline, and the reference says so.
What the probe also recorded about the *environment*, before any check ran:
```
tracerpid=0 frida_named_maps=4 (four mappings whose names match /frida/i)
status: Name: tg.uncrackable3 ; State: S (sleeping) ; Seccomp: 2
/proc/net/tcp -> Permission denied /proc/self/task -> Is a directory
```
**observed:** the target's own `/proc/self/maps` contains **four frida-named mappings** while
`TracerPid` reads **0**. So a `TracerPid`-only check would pass while a maps-or-thread-name
check would not — which is exactly the "what can this process actually see" question the
Stage 1 self-report exists to answer, and it is why the probe reports both.
**One defect in the probe, fixed once and then left:** caller attribution printed
`bt-unavailable: cannot read property 'context' of undefined` on every line. Cause: the
`this` binding inside the anonymous `Interceptor` callback was not what the helper assumed;
the fix is to pass `this.context` explicitly. The measured consequence is worth stating
because it is a *plumbing* failure that reads like a hooking failure: the events and their
timings were correct throughout, only the "which module called this" half was missing, and
one of the three arms still showed no events at all. `observed`.
## 5. ptrace-free dumping: the route, its boundary, and a cross-check that passed
### 5.1 What the route yields when the target has a whole dex in memory
Not re-measured here: the earlier pass's 17 ART dex mappings exported from `/proc/<pid>/mem`
without any instrumenter present (`docs/tool-verification/EXTENSION-rootdump.md`). This pass
adds the **negative boundary** the earlier pass could not state, and it needed a clean
target to state it:
```
# both public MASTG targets, ~2,400 map lines each, one adb round-trip
owasp.mstg.uncrackable1: maps_lines=2427 named_dex=0 apk_views=2
owasp.mstg.uncrackable3: maps_lines=2377 named_dex=0 apk_views=2
```
- **observed:** `named_dex=0`. Neither target produces an
`[anon:dalvik-classes.dex extracted in memory from …]` mapping, so the filename-driven
export has **nothing to export** — and this says nothing about those targets being
protected.
- **observed, the explanation:** their `classes.dex` is **deflate-compressed inside the
on-device APK** (`compress_type=8` read from the APK pulled off the device), so the
`dexopt` "map the dex directly" mechanism cannot apply to it.
- **observed:** the `r--s` views that *do* exist point at APK **file offsets**, and the
bytes there are the APK's own bytes — at `0x11000` on L1 that content is the ZIP
**central directory** (`PK\x01\x02`), not a dex.
- **observed, and a trap that produced a false "DIFFERENT":** on L3 the view
`76dc28e000-76dc299000` has file offset `0x15a000` = 1,417,216 B, while the mapping is
44 KiB (45,056 B) long — so the range reaches 1,462,272, i.e. **past the end of the
1,460,555-byte APK**. The file-side read came back short (1,015 B) and the two sha256s
described different byte counts. Compute the length before suspecting the target.
### 5.2 The reproducibility cross-check, which passed
Two independent read paths over the same address range must agree byte-for-byte before a
capture is evidence:
| Range | Read via `/proc/<pid>/mem` | Read from the file | Verdict |
|---|---|---|---|
| L3 `765a89c000-765a8de000` (264 KiB, file offset `0x6000`) | `sha256=2eafd7cca73630c05d0b0c1b4a7709f21ee24540b5f2ba388ad3467af645aa75` | same | **IDENTICAL** |
| L1 `76dfb14000-76dfb15000` (4 KiB, file offset `0x11000`) | `sha256=74846b8cb02a764efef158194745466cf0c8ad2c1c3fecbb2bcf7cf9c58b6465` | same | **IDENTICAL** |
**observed.** The comparison is only meaningful because both sides covered the same byte
count; the one range that differed was the one whose length exceeded its backing file.
### 5.3 The cross-check that was **not** run, and why it is not silently absent
The task's cross-validation requirement is satisfied for `svc_scan.py` by two independent
decoders agreeing on an identical 214-site set (§1). For a **dump**, the equivalent check
needs two producers that emit the *same* artifact. What this pass has is `/proc/<pid>/mem`
(successful, §5.2) and, on the hardened sample, `frida-dexdump` — which was **refused** by
that sample in the previous pass (`script has been destroyed`,
`docs/tool-verification/EXTENSION-device-run.md`). The sample's lifetime then shrank to ~2 s
(§2), so the second producer could not be obtained on any target here:
| Second-producer option | Status |
|---|---|
| `frida-dexdump` on the hardened sample | **unverified** — refused in the earlier pass; not retried here on a 2 s process |
| `frida-dexdump` on L1/L3 | **unverified, and expected to be empty** — `named_dex=0`, so there is no whole image for any dumper to find |
| `process_vm_readv(2)` as a second backend | **not run** — the on-device helper would have needed a second binary on the device, and the byte-identity question it answers was already answered against the file path |
**Statement of the gap, so the next reader does not have to infer it:** the byte-identity
check between two *dumpers* is **unverified** in this repository. What is `observed` is
byte-identity between two *read paths* on the same bytes, and a 214-site decode agreement
between two independent decoders. Those are weaker claims than "two dumpers cross-checked",
and they are the ones this file makes.
**A related statement that is deliberately negative:** two dumpers agreeing is not evidence
of *recovery*. Against an extraction shell, two agreeing dumps are two copies of the same
skeleton. The judge for that is the platform decoder, per
`advanced-unpacking.md` §Establish "the bodies do not decode" with a decoder you did not write.
## 6. What was not established
- **Not** a recovered payload from the hardened sample. Its process lives ~2 s and never
reaches a state where its payload is loaded; the attempt hit the two-strike limit and was
re-routed to the stable targets.
- **Not** a fix for the sample's startup failure. Root hiding was left alone — configuring
`magisk --denylist` is a device-wide change and this pass was the only lock holder, which
is not the same as being free to change every shared variable.
- **Not** a reproducible end-to-end anti-instrumentation pipeline on a fast detector. The
full sequence was captured once out of three arms and the reference labels it that way.
- **Not** a general claim about `strstr("frida")` being the only check, or about MASTG
challenges, or about Android 11. One device, one ROM, one session, with other experiments
sharing the device.
- **Not** a kernel-layer result. Kernel 4.14.186 < 5.10 closes the eBPF route; the layered
descent in the reference stops at the `svc` boundary and says so.
## 7. Reproduction one-liners
```
python skills/apk-reverse/scripts/svc_scan.py tools/_work/detect/libc.so --context 2
python skills/apk-reverse/scripts/svc_scan.py tools/_work/detect/linker64.bin --json
adb shell 'su -c "sh /data/local/tmp/apkmaps.sh owasp.mstg.uncrackable3"' # named_dex / apk_views shape
adb shell 'su -c "sh /data/local/tmp/apkviews_verify.sh owasp.mstg.uncrackable3"'
adb shell 'su -c "sh /data/local/tmp/detlaunch.sh <pkg> /data/local/tmp/dd1"'
python tools/_work/detect/run_anti_detect_probe.py --pid <pid> --watch-ms 8000 --out arm.json
python tools/_work/detect/run_anti_detect_probe.py --pkg owasp.mstg.uncrackable3 --spawn \
--script skills/apk-reverse/scripts/anti_detect_probe.js --out arm_probe.json
```
The on-device helper scripts (`apkmaps.sh`, `apkviews_verify.sh`, `detlaunch.sh`) and the
driver live under `tools/_work/detect/` and are **work files, not shipped deliverables** —
they exist so the commands above are reproducible, and their content is summarised in the
sections that cite them. The two shipped scripts are `scripts/svc_scan.py` and
`scripts/anti_detect_probe.js`.
docs/tool-verification/README.md
+15
View File
@@ -70,6 +70,21 @@ those are called out in the matrix rather than buried here.
| `EXTENSION-split-apk.md` | Two real split sets through both branches: unified re-signing, the merge that is refused by design, the merge that succeeds, and the signer chain this host actually needs |
| `EXTENSION-protobuf-raw.md` | A schema-free decoder cross-checked against the official runtime and a real DataStore container, with the packed-boundary and proto3-zero ambiguities reproduced on real bytes |
## Absorption-pass record
A later pass absorbed mechanics from external skill repositories (`reverse-skill`,
`awesome-game-security`, `android-reverse-engineering-claude-skill`, `MobileRE-Skill`), keeping each
project's own discipline: only the phase, the checklist and the command pattern cross over, with the
source URL and the access date recorded, and nothing vendored as a runtime dependency. These files
live here rather than in the skill because two of them corrected a claim the repository had already
written down.
| File | Covers |
|---|---|
| `EXTENSION-desensitization.md` | The leak scanner's rules and exemptions, its four `RESULT=` states and exit codes, the escaping false positives, and **the pass that found its own evidence file leaking 26 strong hits** — the fixture's literal match values, quoted to prove the rules fire |
| `EXTENSION-dart-aot-formats.md` | Two string-table claims tested against a real dual-ABI Flutter app: the arm64 packed scheme confirmed against literal tag bytes, and the assumed armv7 `len*2`/UTF-16 form **refuted** — the 32-bit record is `[header u32][byte-count u32le][UTF-8]`, and the extractor's zero is a format mismatch rather than an empty table |
| `EXTENSION-detection-pipeline.md` | Naming the check that fires: three arms on a public MASTG target (control alive, probe arm dead in ≈300 ms via `strstr("frida")`, with the `TracerPid=0` versus 4-frida-mappings asymmetry), `svc_scan.py`'s two decoders agreeing on an identical 214-site set, the layered Java-to-svc descent, and the ptrace-free dump boundary on targets whose dex is deflated inside the APK |
## Adding a run
Record the tools, versions and exact commands, and label each claim. Keep target-specific facts out: