mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-15 19:44:20 +08:00
Absorb gaps from three public reverse-engineering skills
Batch 1 (full): Kotlin original-name recovery (new reference + zero-dep kotlin_name_recovery.py mining @DebugMetadata/@Metadata.d2/jadx rename comments, with -o/-p/--grep query modes); jadx advanced flags and a Fernflower/Vineflower second-engine tier in toolchain.md; BuildConfig sweep habit in recon.md; WebSocket/long-connection state-chain section in protocol-reverse.md. jsvmp-only (batches 2-3): A/B/C recovery ladder with black-box reuse boundary and a wire-parameter family index in jsvmp-recovery.md; new js-ast-deobfuscation.md pipeline layer (family detection, per-family pass order, safe-rewrite rules, minimal bootstrap eval, residue metrics) with js_obf_detect.py weighted detector. Wiring: routing.md/SKILL.md symptom+index rows, coverage-and-limits, route-inventory, evidence-summary, capability-matrix (+3 entries, labels kept honest: scripts observed on fixtures, absorbed methodology unverified). 12 new unit tests. Also: silence two environment-dependent mypy errors (yaml/frida sentinel assignments) and bump mypy python_version 3.9 -> 3.10 with the trade-off recorded. Gates: check_repo/check_refs/check_budget/check_commands/check_routing PASS; pytest 787 passed; flake8 clean; mypy clean.
This commit is contained in:
+5
-1
@@ -31,7 +31,11 @@ import sys
|
||||
try:
|
||||
import yaml
|
||||
except ImportError: # the skills CLI is the real authority; this is a local gate
|
||||
yaml = None
|
||||
# `ignore[assignment]` because the sentinel assignment is only ill-typed where the
|
||||
# module IS installed (mypy then knows the name as Module); on a bare machine the
|
||||
# override in mypy.ini makes it Any and the ignore is unused -- warn_unused_ignores
|
||||
# is off, so both environments stay clean.
|
||||
yaml = None # type: ignore[assignment]
|
||||
|
||||
ROOT = os.path.dirname(os.path.abspath(__file__))
|
||||
SKILLS_DIR = os.path.join(ROOT, 'skills')
|
||||
|
||||
@@ -17,11 +17,19 @@
|
||||
# configuration, on a bare interpreter (no frida/capstone/tiktoken/PyYAML installed), so the
|
||||
# overrides below are exercised rather than skipped.
|
||||
#
|
||||
# Verified locally on 2026-09-22: `mypy` over 55 files reports no issues with the overrides in place,
|
||||
# on a venv containing only mypy and flake8.
|
||||
# Verified locally on 2022-09-22 baseline; re-verified 2026-10-20 after the target bump below:
|
||||
# `mypy` over 64 files reports no issues with the overrides in place, on an interpreter that
|
||||
# also has yaml/frida installed (the ignore comments in check_repo.py and
|
||||
# spawn_patch_detach.py keep both the installed and bare-machine sides clean).
|
||||
|
||||
[mypy]
|
||||
python_version = 3.9
|
||||
# 3.10, not 3.9: mypy releases from 2025 on dropped `python_version = 3.9` as a supported
|
||||
# target and warn on every run. The honest cost: under a 3.10 target mypy stops rejecting
|
||||
# 3.10-only syntax (match statements, runtime `X | Y`), so the SKILL.md "Python 3.9+"
|
||||
# claim is no longer statically guarded -- nothing in the tree uses such syntax, and CI's
|
||||
# oldest actually-executed interpreter is 3.11 (see ci.yml), which is the check that would
|
||||
# catch a real 3.9 break at test time.
|
||||
python_version = 3.10
|
||||
warn_unused_ignores = False
|
||||
|
||||
[mypy-frida]
|
||||
|
||||
@@ -234,12 +234,13 @@ unread in this repository.
|
||||
| The trace segments themselves look corrupted — handler heads missing, tails glued into wrong segments | `references/native-vm-recovery.md` §7 Nested VMs — a flat segment accumulator discards the outer level; nested slices need a segment stack (`vm_handler_slice.py` flags the symptom) |
|
||||
| You need to know **where two runs of the same operation diverge** (password/license/key), not just what executed | `ete-diff-tracing.md` — ETE hardware-trace gates plus the differential method; `scripts/trace_diff.py` also diffs Stalker logs |
|
||||
| A generated KPM/LKM template is in front of you, or the kernel route feels like the plan | `kernel-event-injection.md` — the primitive map, the composition recipes, and the gates every kernel-side file must pass before it is a weapon |
|
||||
| The traffic is protobuf/gRPC/QUIC, or a proxy sees TLS but requests still fail on a Flutter app | `references/protocol-reverse.md` — schema-less protobuf, frame capture, and native-side pinning |
|
||||
| The traffic is protobuf/gRPC/QUIC, a proxy sees TLS but requests still fail on a Flutter app, **or the behaviour rides a long-lived WebSocket** | `references/protocol-reverse.md` — schema-less protobuf, frame capture, native-side pinning, and the long-connection state chain |
|
||||
| Userspace hooks land and the app still dies: the check reads `/proc/self/status` through a raw `svc`, or runs before `JNI_OnLoad` | `references/kernel-and-environment-hardening.md` — what the next layer up and down can actually do, and when to stop |
|
||||
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
|
||||
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
|
||||
| The value you need (sign, token, risk parameter) is computed in **JavaScript** — a WebView or embedded JS engine — by an entry call fed a huge hex/base64 opcode literal: a JS virtual machine | `jsvmp-recovery.md` — de-obfuscate and **call it** first, then the stack/register-VM procedures and the four JS traps; `emulation-and-rpc.md` owns the call-don't-read route |
|
||||
| You are reading a decompiled **Kotlin** app and every class name is a meaningless short token (`nq.e`, `C0123x`) | `kotlin-name-recovery.md` — the original FQNs survive R8 inside `@DebugMetadata`/`@Metadata`; `scripts/kotlin_name_recovery.py` rebuilds the mapping from any jadx export |
|
||||
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
|
||||
|
||||
## Gates — clear these before you patch, in order
|
||||
|
||||
@@ -847,6 +847,53 @@
|
||||
"three public write-ups of one reverse-engineering series, 2024-03..2024-09 (titles and authors anonymized; the absorbed record is the reference file itself)"
|
||||
],
|
||||
"limits": "The recovery procedures (recognition signals, the call-it-first route, stack- and register-machine reading, AST instruction extraction with register/immediate typing, the four JS-language evaluation traps, constant folding and control-flow recovery) are absorbed from three public write-ups and never executed by this repository on any real target -- all inferred. The shipped loader decoder is host-measured only against its own round trip (2,016 values exact, known varint/ZigZag byte layouts pinned, truncation raises) and decodes the write-up's own sample literal end to end; that is corroboration of the chain's shape, not a measurement on an independent target. Engine-specific delivery (WebView, QuickJS/JSC behind JNI, mini-program containers) was never exercised."
|
||||
},
|
||||
{
|
||||
"capability": "js-ast-pre-deobfuscation",
|
||||
"group": "unpacking",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "observed",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/js-ast-deobfuscation.md",
|
||||
"scripts/js_obf_detect.py"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"lwjjike/xbsReverseSkill (MIT), ast-deobfuscation skill: pipeline-config.js, safe-rewrite-rules.md, string-array-and-minimal-eval.md, control-flow-and-opcode-patterns.md, pattern-layering.md -- architecture and rules absorbed 2026-10-20, no code copied"
|
||||
],
|
||||
"limits": "The detection layer is observed against its own embedded fixtures only (four family fixtures, the hint-beats-content scoring rule, plain-code-is-generic, residue metric filter) -- no real obfuscated bundle was ever classified here. The rewriting passes (string-array restore, dispatcher inlining, array-CFF flattening, if-chain-to-switch, identifier renaming) need node plus @babel/parser/traverse/generator, which this skill does not ship; every rewrite is a documented route, unverified end to end. Residue metrics are textual proxies, valid for same-file before/after comparison only. The per-family pass-order caveat carried from the absorbed architecture (VM-heavy families skip late literal-inlining because it stalls large samples) is the upstream's measurement, not this repository's."
|
||||
},
|
||||
{
|
||||
"capability": "kotlin-name-recovery",
|
||||
"group": "dex",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "observed",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/kotlin-name-recovery.md",
|
||||
"scripts/kotlin_name_recovery.py"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"SimoneAvogadro/android-reverse-engineering-skill (Apache-2.0): recover-kotlin-names.sh and references/kotlin-name-recovery.md -- re-implemented 2026-10-20 as dependency-free Python"
|
||||
],
|
||||
"limits": "Parsing of all three recovery sources (@DebugMetadata c=, @Metadata d2 first non-stdlib descriptor, jadx 'renamed from' comments), the third-party skip list and the map-file round trip are observed on embedded fixtures only. The upstream recovery-rate claims (30-50% of classes overall, ~100% of *Repository/*ViewModel/*Impl, ~80% of data-class DTOs) are unverified here: no real obfuscated Kotlin app was run through the script. Method and field names are never recovered; heavily inlined shapes can surface under a wrong primary class, so every mapping row is a hint. The script reads decompiled sources, not the dex: a decompiler that drops annotations removes the source entirely."
|
||||
},
|
||||
{
|
||||
"capability": "websocket-long-connection-analysis",
|
||||
"group": "protocol",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "unverified",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/protocol-reverse.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"715494637/reverse-skill (no license file): jsr-reverse/references/protocol-and-long-connection.md -- methodology only absorbed 2026-10-20, no text copied"
|
||||
],
|
||||
"limits": "The state-chain method (capture handshake, first business frame, heartbeat/ack, renewal, error; record the connection as a dependency chain; separate message families; split envelope from payload before body semantics) is absorbed from a browser-WebSocket reference and was never run on an Android long-connection target here. The Android capture points in the section (OkHttp WebSocket listener surfacing, native/QUIC transports never transiting Java) are inferred from documented APIs, not measured. Frame bodies that are protobuf do inherit the measured schema-free decoder from protocol-reverse.md section 1; that is the only measured part of the route."
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -141,10 +141,31 @@ its own strength note at the top.
|
||||
- **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition
|
||||
signals, the call-it-first route (AST restore + environment fill), stack- and register-machine
|
||||
reading procedures, AST instruction extraction with register/immediate typing, the four
|
||||
JS-language traps, and constant-folding/control-flow recovery. Absorbed from three public
|
||||
JS-language traps, and constant-folding/control-flow recovery; the A/B/C depth ladder and the
|
||||
wire-parameter family index are later additions from a fourth public source (provenance in the
|
||||
file). Absorbed from three public
|
||||
write-ups; every procedure `inferred`, none run on a real target. The one shipped piece is the
|
||||
loader decoder (base64 → varint → ZigZag), host-measured against its own round trip plus the
|
||||
write-up's sample literal (`jsvmp-recovery.md`, `scripts/jsvmp_loader_decode.py`).
|
||||
- **JS AST pre-deobfuscation (detection + pass order)** — the pipeline layer `jsvmp-recovery.md`
|
||||
names as its step 1: weighted family detection and per-family pass ordering, safe-rewrite rules,
|
||||
minimal bootstrap eval, residue metrics. Absorbed 2026-10-20 from a public MIT-licensed skill
|
||||
(`lwjjike/xbsReverseSkill`); the rewriting passes need node+babel, which this skill does not
|
||||
ship, so the end-to-end route is `unverified`. The shipped detector is `observed` against its
|
||||
own embedded fixtures only (`js-ast-deobfuscation.md`, `scripts/js_obf_detect.py`).
|
||||
- **Kotlin original-name recovery** — the mapping R8 cannot strip: `@DebugMetadata`/`@Metadata.d2`
|
||||
carry original class FQNs into any decompiled export, and the shipped script rebuilds the
|
||||
obfuscated→original map plus `--grep` annotation. Absorbed 2026-10-20 from a public
|
||||
Apache-2.0-licensed skill (`SimoneAvogadro/android-reverse-engineering-skill`); parsing is
|
||||
`observed` against embedded fixtures, the upstream recovery-rate claims are `unverified`, and
|
||||
no real obfuscated app was run through it here (`kotlin-name-recovery.md`,
|
||||
`scripts/kotlin_name_recovery.py`).
|
||||
- **Long-connection (WebSocket) state-chain analysis** — capture beyond the first frame,
|
||||
connection state chains, message-family separation, envelope/payload split. Absorbed
|
||||
2026-10-20 from a public unlicensed JS-reverse skill's reference (methodology only); written
|
||||
for browser targets and never run on an Android long-connection app here — `unverified`
|
||||
end to end, with Android capture points `inferred` from documented APIs
|
||||
(`protocol-reverse.md` §6).
|
||||
- **Kernel-side event injection** — invisible watchpoints and register rewrites from a driver, as
|
||||
generated templates with their gates; nothing kernel-side was ever compiled or loaded here
|
||||
(`kernel-event-injection.md`, `scripts/kernel_instrument_scaffold.py`).
|
||||
@@ -153,6 +174,10 @@ its own strength note at the top.
|
||||
|
||||
## Dependencies this skill does not ship — name them before the workflow starts
|
||||
|
||||
- **JS AST rewriting needs node + babel** (`@babel/parser`/`traverse`/`generator`) — the detector
|
||||
and the pass order ship; the rewriting engine does not, and every rewrite pass is a documented
|
||||
route until that stack is installed (`js-ast-deobfuscation.md`).
|
||||
|
||||
- **Dart AOT analysis needs a snapshot dump.** `dart-aot.md`'s workflow begins at `pp.txt`;
|
||||
producing it requires a snapshot container resolver this skill does not contain and cannot
|
||||
synthesize. `dart_pool_strings.py` reports **file** offsets while `pp.txt` and `dart_pprefs.py`
|
||||
|
||||
@@ -49,6 +49,9 @@ route was not measured on this repository's evidence**. Treat them as leads with
|
||||
| Interpreter-trace de-virtualization | handler slicing + guest-opcode trajectory, calibrated on an open IR-VM's source — `vm_handler_slice.py` |
|
||||
| Native VM recovery procedure | the absorbed method end to end: dispatcher separation, register anchoring, nested stacks, operand slicing, re-VM validation — `native-vm-recovery.md` |
|
||||
| JS VMP (jsvmp) recovery | recognition, call-it-first, stack/register-VM reading, instruction extraction, the four JS traps — procedures absorbed, only the loader decoder shipped and round-trip-tested — `jsvmp-recovery.md` |
|
||||
| JS AST pre-deobfuscation | family detection + per-family pass order + safe-rewrite rules; detector shipped and fixture-tested, rewriting needs node+babel (not shipped) — `js-ast-deobfuscation.md`, `js_obf_detect.py` |
|
||||
| Kotlin original-name recovery | obfuscated→original class map from `@DebugMetadata`/`@Metadata` in any jadx export; parsing fixture-tested, recovery rates upstream-claimed — `kotlin-name-recovery.md`, `kotlin_name_recovery.py` |
|
||||
| Long-connection (WebSocket) analysis | state chains, message families, envelope/payload split — absorbed methodology, never run on an Android target here — `protocol-reverse.md` |
|
||||
| Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` |
|
||||
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
|
||||
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
|
||||
|
||||
@@ -62,6 +62,9 @@ next** — and points at the one section that says "the full record is elsewhere
|
||||
| Interpreter-trace de-virtualization | Dispatcher slicing, signature clustering and the opcode trajectory measured on synthetic interpreter traces; the calibration facts come from a public IR-VM's source, and no hardened binary was traced | partial | observed | `references/ete-diff-tracing.md` §5, `scripts/vm_handler_slice.py` |
|
||||
| Native VM recovery procedure | The full method is absorbed from two public write-ups (dispatcher-layer separation, register anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM validation); no step was executed by this repository | partial | inferred | `references/native-vm-recovery.md` |
|
||||
| JS VMP (jsvmp) recovery | Procedures absorbed from three public write-ups (stack and register machines) and never run on a real target; the shipped loader decoder (base64→varint→ZigZag) round-trips 2,016 values exactly, pins the known varint/ZigZag layouts and decodes the write-up's own sample literal end to end | partial | inferred | `references/jsvmp-recovery.md`, `scripts/jsvmp_loader_decode.py` |
|
||||
| JS AST pre-deobfuscation | Detector self-tested on embedded fixtures (4 families, hint-beats-content, plain-is-generic) and residue metrics pinned; the rewrite passes themselves need node+babel and were never run here | partial | observed | `references/js-ast-deobfuscation.md`, `scripts/js_obf_detect.py` |
|
||||
| Kotlin original-name recovery | All three annotation sources (DebugMetadata, Metadata.d2, jadx rename comments) parse correctly on pinned fixtures, third-party trees skipped, map files round-trip; upstream's 30–50%/≈100% recovery-rate claims not measured here | partial | observed | `references/kotlin-name-recovery.md`, `scripts/kotlin_name_recovery.py` |
|
||||
| Long-connection (WebSocket) analysis | Absorbed state-chain methodology (handshake→heartbeat/ack→renewal, message families, envelope/payload split); no Android long-connection app was captured through it here | partial | unverified | `references/protocol-reverse.md` §6 |
|
||||
| Kernel-side instrumentation/injection | The scaffold's generate/verify is measured and every kernel file ships unbuilt with its gate; the register-rewrite action is explicitly experimental and never ran anywhere | blocked | unverified | `references/kernel-event-injection.md`, `scripts/kernel_instrument_scaffold.py` |
|
||||
| `svc` site scanning | Two independent decoders agreed on an identical 214-site set; a byte scan also matches data, so neighbour context decides | ok | observed | `references/kernel-and-environment-hardening.md`, `scripts/svc_scan.py` |
|
||||
| On-device tooling (MT Manager MCP) | The service-down path is measured; the connected path needs the service started by hand | partial | observed | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
# JS AST pre-deobfuscation — the pipeline layer before `jsvmp-recovery.md` reads anything
|
||||
|
||||
**Load this when** the JavaScript in front of you (pulled from a WebView, an embedded engine, a
|
||||
mini-program bundle, `assets/www/`) is unreadable through *structure*, not just minification —
|
||||
string tables, self-executing decoder wrappers, flattened control flow, if-chain dispatch — and
|
||||
`jsvmp-recovery.md` §3's step 1 ("AST pre-deobfuscation") is the instruction you are executing.
|
||||
This file owns **how that step is run as a pipeline**; the VM work after it stays in
|
||||
`jsvmp-recovery.md`, and this file deliberately does not repeat it.
|
||||
|
||||
**What this answers:** which family the sample belongs to, in what order the rewrite passes run,
|
||||
when a rewrite is *safe to apply at all*, and how you prove the batch helped. Contents:
|
||||
§1 the three layers · §2 detection · §3 the pass library and order · §4 safe-rewrite rules ·
|
||||
§5 string tables and minimal bootstrap eval · §6 residue and proof · §7 extending the library.
|
||||
|
||||
**The fence — this skill ships detection and order, not the rewriter.** The passes need a real JS
|
||||
parser: node + `@babel/parser` + `@babel/traverse` + `@babel/generator`. That stack is one
|
||||
`npm install` away, but this repository does not carry it; every *rewrite* below is a documented
|
||||
route (`unverified` as end-to-end), while the **detector and its fixtures are `observed`**
|
||||
(`scripts/js_obf_detect.py --selftest`). Re-parse after every batch — output that does not re-parse
|
||||
is corruption, not progress.
|
||||
|
||||
**Provenance.** Absorbed 2026-10-20 from `lwjjike/xbsReverseSkill` (MIT), its `ast-deobfuscation`
|
||||
skill — the pipeline-config step library with per-family ordering and weighted detection, and the
|
||||
safe-rewrite-rules, string-array-and-minimal-eval, control-flow-and-opcode-patterns and
|
||||
pattern-layering notes. No code or pattern corpus was copied; the architecture and the rules are
|
||||
the absorption. The absorbed pipeline's own measured caveat
|
||||
(late literal-inlining stalls on large VM-heavy samples, so that family skips it) is carried in §3.
|
||||
|
||||
## 1. Three layers, and the rule that keeps them honest
|
||||
|
||||
1. **Generic transform layer** — stable, low-risk rewrites that hold across unrelated samples.
|
||||
2. **Detection / pipeline-selection layer** — score the sample, pick the pass order.
|
||||
3. **Family adapter layer** — narrow passes for one vendor's reproducible quirk.
|
||||
|
||||
The rule (the reason the absorbed library stayed usable): **a family rule does not move into the
|
||||
generic layer until it has recurred in at least two unrelated samples** with its safety boundary
|
||||
still clear. The failure this prevents is generic passes accreting site-specific heuristics until
|
||||
every run needs per-sample exemptions — at which point the pipeline is unmaintainable *and*
|
||||
unreviewable, and mis-rewrites become undebuggable.
|
||||
|
||||
## 2. Detection — score before you choose the order
|
||||
|
||||
`scripts/js_obf_detect.py <file.js> [--hint TEXT] [--json]` implements the absorbed scoring model:
|
||||
|
||||
- a **hint** token matched against your `--hint` text or the file path scores **10** — a human
|
||||
saying "this is the delivery platform's sign SDK" outranks any regex;
|
||||
- each **content** regex hit scores **3** — `_0x` identifiers and a top-level string array
|
||||
(string-array lineage), `while(!![]){switch(...)}` with `"3|1|2".split('|')` order tables
|
||||
(flattening), a huge hex/base64 literal plus a `[Object, Math, …]` dependency array (jsvmp
|
||||
entry — cross-check against `jsvmp-recovery.md` §2's table, which is the authoritative
|
||||
recognition step), long literal-vs-variable if-chains (opcode dispatch).
|
||||
|
||||
Any score > 0 is a detection; the top scorer sets the pipeline; nothing scored means the generic
|
||||
ladder, which is also where you fall back when family residue stays stable after two batches.
|
||||
The huge-literal threshold is `--min-literal` (default 1500) — a parameter, not a boundary: lower
|
||||
it for small samples, and say so in your record.
|
||||
|
||||
## 3. The pass library and the order that matters
|
||||
|
||||
Generic ladder (absorbed order): **normalize → prune fake branches → inline dispatchers → flatten
|
||||
array-driven control flow → if-chain to switch → prune → normalize**, with identifier renaming
|
||||
**last**, and a `string_array_pass` **first** when detection says the lineage uses one — every
|
||||
later pass reads strings through the decoder, so metrics and rewrites on undecoded text are noise.
|
||||
|
||||
Per-order rules, each paid for by someone's stall:
|
||||
|
||||
- **Decode the string array before any control-flow work** (§5).
|
||||
- **Never jump to full beautification because the code is ugly.** The stop condition is "the main
|
||||
dispatcher is readable and the key cases are individually analysable", not "the file is pretty".
|
||||
`jsvmp-recovery.md` §3's ladder decides how much deeper than that you ever go.
|
||||
- **VM-heavy families skip late literal-inlining** — the absorbed architecture's own measured
|
||||
example of a pass order that hangs a large sample; re-ordering per family is the point of the
|
||||
detection layer, not a hack.
|
||||
- **A pass that finds no match must exit clean**, not "best effort" — a visitor that half-applies
|
||||
leaves the tree in a state no later pass expects. Structure mismatch ⇒ skip and report.
|
||||
|
||||
## 4. Safe-rewrite rules — when a node is *certainly* foldable
|
||||
|
||||
Replace **only when the answer to "is this certainly recoverable" is yes**; anything less is a
|
||||
corruption lottery.
|
||||
|
||||
Certainly replaceable: array-literal indexing with a numeric literal and stable contents · calls
|
||||
whose arguments are all literals **after** the minimal decoder environment exists (§5) · pure
|
||||
expressions (`['a','b'].join('')`, `String.fromCharCode(...)`, `split/reverse/join` chains) ·
|
||||
verified alias chains (`var a='x'; var b=a;`) · statically-true/false comparisons.
|
||||
|
||||
Never fold: anything depending on `this`/global/closure-external bindings or the host environment ·
|
||||
getters/setters/Proxy objects/dynamic property access/side-effecting calls · alias chains whose
|
||||
binding stability is unproven · comma/logical expressions where order matters · anything needing
|
||||
cross-scope execution reasoning.
|
||||
|
||||
Judgement order: (1) does the result depend only on literals and stable local bindings?
|
||||
(2) does evaluation have side effects? (3) does it depend on execution order, host state, or
|
||||
environment? — **any unknown at any step ⇒ do not replace**; leave the region for runtime evidence.
|
||||
And never re-fold an already-folded literal: putting `StringLiteral`s back through the evaluator is
|
||||
how output destabilizes.
|
||||
|
||||
## 5. String tables and minimal bootstrap eval
|
||||
|
||||
The string-array family: the file's first statements build a decoder — a top-level IIFE rotating an
|
||||
array, or 2–5 statements initialising one. Order:
|
||||
|
||||
1. Locate the array literal, the decoder function, and the alias chain.
|
||||
2. **Static first**: array indexing, literal concatenation, numeric-argument calls.
|
||||
3. Only when static stops: **minimal bootstrap eval** — execute *only* the first few statements
|
||||
that construct the decoder, never the file; never in an environment with DOM/timers/network;
|
||||
with the narrowest goal ("resolve numeric-argument calls to strings").
|
||||
4. Delete the consumed bootstrap declarations, **re-parse**, then enter control-flow work.
|
||||
|
||||
Stop when the main table is restored and the decoder entry is clear; string tables beyond that are
|
||||
low-value residue. For a jsvmp sample this is `jsvmp-recovery.md` §3 step 1, which halved the
|
||||
write-up's sample before any VM reading.
|
||||
|
||||
## 6. Residue and proof — a batch helped, or it did not
|
||||
|
||||
Measure **before and after each batch** with `js_obf_detect.py`'s residue metrics (mangled-identifier
|
||||
count, hex-literal density, comma-expression count, switch/case counts, longest line). They are
|
||||
*textual proxies*, honest for same-file before/after comparison and for nothing finer — do not
|
||||
quote them as AST counts. Two rules:
|
||||
|
||||
- **Compare against a reference where one exists.** If you hold the vendor's decoded output or a
|
||||
known-good decode of one function, diff against it; residue dropping while diverging from the
|
||||
reference is a loss.
|
||||
- **A batch that did not move the metrics is a signal to stop**, not to re-run it louder — the same
|
||||
two-strike shape as `SKILL.md` §Stop conditions, at pass granularity.
|
||||
|
||||
## 7. Extending the pattern library — the three-piece rule
|
||||
|
||||
Adding a family adapter means adding **all three, in the same change**: the rule document
|
||||
(a `patterns/<family>.md` note — shape, safety boundary, measured caveats; the directory starts
|
||||
empty and fills only when a real family is absorbed), the detector entries (hint tokens + content
|
||||
regexes in `scripts/js_obf_detect.py`), and the pipeline entry (pass order in its `FAMILIES`
|
||||
record). An adapter that exists in only one of the three places is invisible,
|
||||
undetectable, or unmaintainable — and §1's promotion rule still governs what may reach the generic
|
||||
layer.
|
||||
@@ -7,9 +7,11 @@ receives a **giant literal** — thousands of hex characters or a base64 blob
|
||||
globals, and everything behind it is one closed interpreter. That shape is **jsvmp**: the original
|
||||
logic compiled to a private opcode stream and executed by a virtual machine shipped with the script.
|
||||
This file owns reading it. The neighbouring layers own the rest: `references/emulation-and-rpc.md`
|
||||
owns **calling** the routine instead of reading it, `references/vmp-differential-analysis.md` the
|
||||
dex-layer VMP, `references/native-vm-recovery.md` the native interpreter loop, and
|
||||
`references/framework-runtimes.md` deciding that the logic lives in JS at all.
|
||||
owns **calling** the routine instead of reading it, `references/js-ast-deobfuscation.md` owns the
|
||||
**pre-deobfuscation pipeline** (detection, pass order, safe-rewrite rules — the step §3.1 names),
|
||||
`references/vmp-differential-analysis.md` the dex-layer VMP, `references/native-vm-recovery.md` the
|
||||
native interpreter loop, and `references/framework-runtimes.md` deciding that the logic lives in JS
|
||||
at all.
|
||||
|
||||
**Claim strength.** Absorbed from three public write-ups of one reverse-engineering series
|
||||
(2024-03..2024-09; titles and authors anonymized in this record). One builds a minimal JS VM to
|
||||
@@ -57,6 +59,39 @@ Two cautions. A big `switch` alone proves nothing (a state machine looks the sam
|
||||
**read a cursor from a data buffer, dispatch on the value, and write the cursor back**. And the
|
||||
presence of a readable interpreter does not mean you must read it — §3 first.
|
||||
|
||||
**A parameter-name index for the families this repository has on record** (`unverified` —
|
||||
absorbed 2026-10-20 from a public JS-reverse case corpus; wire-level parameter names are kept as
|
||||
the recognition signal, vendors anonymised per this repository's desensitization convention; none
|
||||
of these was met on a real target here). Recognition aid only — confirm by §2's structural signals
|
||||
before routing on it:
|
||||
|
||||
| Wire parameter / marker | Family shape | Typical carrier |
|
||||
|---|---|---|
|
||||
| `a_bogus`, `X-Bogus` | layered sign params, heavy env probing | short-video platforms, web + app |
|
||||
| `mtgsig` | mini-program VMP with native assist | delivery/local-services apps |
|
||||
| `sensor_data` | device-fingerprint collection + packing | risk-control SDKs (app-embedded) |
|
||||
| `blackbox` | fingerprint blob, env-sensitivity | risk-control SDKs (app-embedded) |
|
||||
| `h5st` | staged params with time windows | large e-commerce clients |
|
||||
| `x-s`, `x-t` pair | jsvmp-generated sign pair | lifestyle-community platforms |
|
||||
| `__zse_ck` | JS + wasm nested obfuscation | Q&A communities |
|
||||
| `acw_sc__v2` | JS-computed cookie on first visit | e-commerce/ACW-fronted sites |
|
||||
| `F14B3CEF*`-prefixed cookies, `$_ts` | 4/5/6-generation dynamic JS ("RS") | ticketing/airline/gov sites in WebView |
|
||||
| `armorToken`, `Anti-in` | request-header token pair | fashion e-commerce |
|
||||
| `aws-waf-token` | JS challenge token | AWS-WAF-fronted endpoints |
|
||||
| `Reese84`, `___utmvc` | obfuscator.io-lineage challenge | airline/CDN-fronted sites |
|
||||
| `hexin-v` | sign with heavy obfuscation | stock-data platforms |
|
||||
| `ssxmod_itna` | multi-cookie interdependent chain | travel/booking sites |
|
||||
|
||||
A hit in this table says "expect a known shape, budget accordingly" — it does **not** say the
|
||||
algorithm is known, and it never overrides a structural check. Two or three of these families show
|
||||
up inside APKs as embedded WebView/H5 risk pages and mini-program containers rather than as the
|
||||
app's own code — which is exactly why they surface here and not in `detection-and-anti-analysis.md`.
|
||||
|
||||
**Provenance note.** The A/B/C ladder at the end of §3 and this table are the only parts of this
|
||||
file absorbed from outside the three write-ups the rest cites: both come from
|
||||
`715494637/reverse-skill` (`jsr-reverse` skill; the repository ships no license, so methodology
|
||||
only was absorbed, no text), retrieved 2026-10-20.
|
||||
|
||||
## 3. The cheaper question first: restore, fill the environment, call it
|
||||
|
||||
The stack-machine write-up's outcome, in its own order, and the step most skip:
|
||||
@@ -65,7 +100,9 @@ The stack-machine write-up's outcome, in its own order, and the step most skip:
|
||||
propagation. This halved the sample (~9,600 lines → roughly half) and made every later step
|
||||
debuggable in an IDE instead of a browser's obfuscated view (the same function was 3,000+
|
||||
unreadable lines in devtools). Debugging un-restored code in the browser is the slow path; do
|
||||
not attempt VM analysis inside it.
|
||||
not attempt VM analysis inside it. **Run as a pipeline, not ad hoc:**
|
||||
`references/js-ast-deobfuscation.md` owns the family detection, pass order and safe-rewrite
|
||||
rules, and `scripts/js_obf_detect.py` scores the sample.
|
||||
2. **Fill the environment** ("补环境"): give the script the `window`/`document`/`navigator`/canvas
|
||||
surface it probes. Canvas fingerprints are **hardware-linked** — vendors have collected real
|
||||
device fingerprints to reject forged ones; a random canvas value may still be accepted (the
|
||||
@@ -79,6 +116,28 @@ deliverable: a harness that runs the guest in your controlled environment. Read
|
||||
you must understand or alter the algorithm itself, or when the environment the VM demands is more
|
||||
expensive to fake than the VM is to read.
|
||||
|
||||
### How deep to go — the A/B/C ladder (absorbed; `inferred`)
|
||||
|
||||
"Call it first" is level **A**, and most tasks never leave it. The ladder below makes the upgrade
|
||||
a *decision with evidence*, not a mood — its shapes come from a public JS-reverse recovery-strategy
|
||||
note (see provenance above), and they match what the three write-ups actually did:
|
||||
|
||||
| Level | What you recover | You are here when… | You leave when… |
|
||||
|---|---|---|---|
|
||||
| **A — call the guest, extract only the opcodes that touch the target field** | the harness + a handful of critical-opcode semantics | the write-back boundary is known and a few opcodes explain the field | — (the deliverable is the harness) |
|
||||
| **B — recover the dispatcher plus the critical state carriers** | the dispatcher loop, register/stack/context carriers, opcode families with state flow | a critical opcode cannot be interpreted without dispatcher semantics; the field explanation depends on register/stack flow; key branches cannot be judged without the carriers | never jump here because the code is *ugly* — ugliness is not an evidence type |
|
||||
| **C — minimal decompilation / minimal re-interpreter** | a smallest verifiable re-execution of the needed path | multiple paths must be re-played, or batch/protocol rebuild requires an executable fragment, and A+B demonstrably cannot support it | — |
|
||||
|
||||
**Hard rules.** Start at A; never jump to C for aesthetics. A→B requires a *proven* dispatcher or
|
||||
state-carrier dependency, not a suspicion. B→C requires a named downstream need (replay, batch,
|
||||
protocol rebuild) — "the write-up rebuilt the whole ISA" is not that need until your task has one.
|
||||
And the **black-box reuse boundary** decides whether reading is needed at all: reuse the module
|
||||
whole when its input/output boundaries are known and the difficulty is the container or bridge
|
||||
rather than the operator; recover deeper when the module rides heavy implicit shared state, replays
|
||||
are unstable without the state carrier, or the module is *itself* another jsvmp. Write the decision
|
||||
down in four lines — reuse mode / known inputs / known outputs / still-dependent state carriers —
|
||||
so "we stopped at A" reads as a conclusion rather than a shortcut.
|
||||
|
||||
## 4. Locating the interpreter and clearing the porch
|
||||
|
||||
- **Find the entry by the literal, not by names.** Search for the huge constant being passed; follow
|
||||
@@ -238,6 +297,7 @@ integers → ~14,000 instructions, with constants visible as bulk stores into fi
|
||||
| Need | Tool / file |
|
||||
|---|---|
|
||||
| decode a jsvmp program literal (base64 → varint → ZigZag) | `scripts/jsvmp_loader_decode.py` (`--selftest` proves the round trip) |
|
||||
| classify an obfuscated JS sample and get its pass order | `scripts/js_obf_detect.py`, then `references/js-ast-deobfuscation.md` |
|
||||
| decide the logic lives in JS at all | `references/framework-runtimes.md` (runtime table; zero-hit Java hooks) |
|
||||
| call the routine instead of reading it (the §3 route) | `references/emulation-and-rpc.md`; Frida RPC over a WebView/engine boundary |
|
||||
| reach a JS engine behind JNI from the device | `references/dynamic-frida.md`; the JS bridge inputs are in `references/system-ipc-audit.md` |
|
||||
@@ -252,6 +312,7 @@ the G1 sentence accordingly before sinking days into the ISA.
|
||||
|
||||
- [ ] entry recognised by the literal + dependency-array shape, not by a big `switch` alone
|
||||
- [ ] §3 executed in order — AST restore, environment fill, call the function — before any VM reading
|
||||
- [ ] the A/B/C stop depth stated with its evidence (which field-explanation or replay need forced the level) and the reuse boundary written down
|
||||
- [ ] environment patches labelled: each one can fail server-side later (canvas especially)
|
||||
- [ ] guard closures simplified with reference-checked replacements; "dead" branches probed, not cut
|
||||
- [ ] stack vs register decided by the state idiom (§2 table), then the naming map applied
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# Kotlin name recovery — original class names survive R8 inside annotations
|
||||
|
||||
**Load this when** you are reading a decompiled Kotlin app and the class names are meaningless
|
||||
(`nq.e`, `a`, `C0123x`) — before you conclude the code is "unreadable" and before you sink hours
|
||||
into cross-referencing by shape. One pass over the export recovers the *original* names for a
|
||||
large share of the classes you actually want to read.
|
||||
|
||||
**What this answers:** which original class each obfuscated class was — from data R8 cannot strip,
|
||||
not from guessing. What it does **not** answer: method names and field names (never recovered —
|
||||
see Limits).
|
||||
|
||||
Contents: §1 why the leak exists · §2 the three sources · §3 the shipped script · §4 what you
|
||||
typically get · §5 how it plugs into this kit's index-first discipline · §6 limits.
|
||||
|
||||
**Provenance.** Absorbed 2026-10-20 from `SimoneAvogadro/android-reverse-engineering-skill`
|
||||
(Apache-2.0), its `recover-kotlin-names.sh` and `references/kotlin-name-recovery.md`; re-implemented
|
||||
here as a dependency-free Python script. The recovery-rate numbers in §4 are the upstream author's
|
||||
claims — `unverified` here. The script's parsing is `observed` against its own embedded fixtures
|
||||
only (`--selftest`); no real obfuscated app was run through it in this repository.
|
||||
|
||||
## 1. Why the names survive
|
||||
|
||||
R8/ProGuard rename JVM symbols, but the Kotlin runtime needs metadata at runtime — for reflection,
|
||||
coroutines (`@DebugMetadata` drives coroutine stack-trace recovery), and `data class` / serialization
|
||||
behaviour (`@Metadata`). The annotations are **values the runtime reads**, so obfuscators keep them,
|
||||
and the values contain the original fully-qualified names. This is a leak by necessity, the same
|
||||
class of fact as "R8 cannot rename what the manifest references" (`recon.md` §5's anchor idea).
|
||||
|
||||
## 2. The three sources, in trust order
|
||||
|
||||
1. **`@DebugMetadata(c = "...")`** — emitted for nearly every `suspend` function's `SuspendLambda`
|
||||
subclass. The `c` value is the original FQN **with `$`-suffix scope**:
|
||||
`com.example.account.AccountRepositoryImpl$fetch$1` means the declaring class is everything
|
||||
before the first `$`. Highest precision: a true declaring-class name, one per coroutine lambda —
|
||||
and modern Kotlin apps are full of them.
|
||||
2. **`@Metadata(d2 = {...})`** — every Kotlin class carries it. The `d2` array lists the file's
|
||||
internal class references as JVM descriptors (`Lcom/example/Foo;`); the **first non-stdlib,
|
||||
non-framework descriptor** is usually the file's primary class. A hint, not a declaration.
|
||||
3. **jadx `/* renamed from: X */`** — jadx's own rename bookkeeping, emitted when `--deobf` or a
|
||||
mapping renamed the class. Weakest: it records what *jadx* renamed from, which for an R8 target
|
||||
is the obfuscated name — useful mainly for jadx-generated names (`C0123x`).
|
||||
|
||||
## 3. The shipped script
|
||||
|
||||
```bash
|
||||
python scripts/kotlin_name_recovery.py <jadx-export>/sources # build the map
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> -o nq.e # resolve one name
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> -p com.example.feature # list a package
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> --grep '"api/' <jadx-export>/sources
|
||||
```
|
||||
|
||||
Outputs: `mapping.tsv` (`obf_fqn real_fqn source file`), `mapping.json`, and `by_package/`
|
||||
(one file per original package). The `--grep` mode is the everyday one: it greps the export and
|
||||
annotates **every hit with the real owning class**, so the export becomes navigable without opening
|
||||
files one by one. Third-party trees (`kotlin.*`, `androidx.*`, `okhttp3.*`, …) are skipped — their
|
||||
names are already real.
|
||||
|
||||
Resolving `-o nq.e` also prints **siblings** under the same short prefix (`nq.d`, `nq.f`): they are
|
||||
frequently one original class's split lambdas/inner classes, which is often the context you wanted.
|
||||
|
||||
## 4. What you typically get — upstream's claims, `unverified`
|
||||
|
||||
| Class kind | Recovery |
|
||||
|---|---|
|
||||
| `*Repository` / `*Impl` / `*ViewModel` / `*UseCase` | ~100 % |
|
||||
| plain `data class` DTOs | ~80 % |
|
||||
| share of all classes in a real app | 30–50 % |
|
||||
|
||||
The shape is the point: the classes a *reading* task targets are exactly the annotated ones, so the
|
||||
effective recovery on "the classes you came for" is much higher than the global share. Treat the
|
||||
numbers as directional; this repository has not measured them.
|
||||
|
||||
## 5. How it plugs into this kit's order of work
|
||||
|
||||
This is a **reading aid for the export tier**, not a replacement for the index tier
|
||||
(`SKILL.md` §Tooling — orient with an indexer, not an export): locate first with `rasc findrefs` /
|
||||
`dexsql query`, then read the located class in the export with names restored. Two combinations that pay:
|
||||
|
||||
- **Mapping onto query results:** the index hands you `nq.e`; `-o nq.e` tells you it is
|
||||
`com.example.feature.account.AccountRepositoryImpl` — which usually *is* the layer answer
|
||||
(`recon.md` §3's "where does the behaviour live" becomes a lookup instead of an inference).
|
||||
- **Endpoint hunting:** `--grep '"/' <export>` over the app's own packages gives every path literal
|
||||
with its real owning class attached — a cheap first pass for `server-api.md`'s surface list.
|
||||
|
||||
## 6. Limits — say them before trusting a mapping
|
||||
|
||||
- **Method and field names are never recovered.** Metadata preserves class-level FQNs and some
|
||||
signatures, not members. For members you still need call-site evidence (`find_refs.py`,
|
||||
`dexsql` callers) or jadx-gui's interactive rename.
|
||||
- **Pure-Java classes carry no `@Metadata`** and stay obfuscated; the fallbacks do not see them.
|
||||
- **Heavily inlined shapes** (`@JvmInline value class`, top-level functions compiled into shared
|
||||
`*Kt` classes) can surface under a wrong primary class; every mapping row is a hint, and §5's
|
||||
cross-checks are the antidote.
|
||||
- The script reads **decompiled sources**, not the dex: annotations are read as jadx printed them,
|
||||
so a decompiler that drops annotations removes the source entirely. (The raw dex annotation
|
||||
sections carry the same strings — `dexsql`'s annotation tables — as a cross-check when in doubt.)
|
||||
@@ -305,7 +305,56 @@ therefore: identify the client first (`tls-and-cert.md` §Step 4 attribution), t
|
||||
mechanism. Patching a library that was never consulting the store is work with no effect; installing
|
||||
a system CA when a native verifier ignores it produces the same empty capture you started with.
|
||||
|
||||
## 6. Failure modes
|
||||
## 6. WebSocket and long-lived connections — state chains, not packets
|
||||
|
||||
**Load this when** the behaviour you care about rides one long connection instead of request/response
|
||||
pairs: a WebSocket (OkHttp `WebSocket`, `Java-WebSocket`, a native socket), SSE, or an app-held
|
||||
TCP/QUIC stream. The unit of analysis changes: a single captured frame says almost nothing, because
|
||||
its meaning is a function of the connection's accumulated state.
|
||||
|
||||
**Claim strength.** `unverified` as a method — absorbed 2026-10-20 from `715494637/reverse-skill`
|
||||
(`jsr-reverse`'s long-connection reference; no license there, methodology only), where it was
|
||||
written for browser WebSocket targets; the Android capture points below are `inferred` from
|
||||
documented APIs, and nothing here was run on a real long-connection app in this repository.
|
||||
|
||||
**Capture more than the first packet.** The expensive mistake is analysing one business frame with
|
||||
no idea which state produced it. Minimum evidence set, in order: **handshake** (and its response —
|
||||
it usually mints the session), **first business frame**, a **heartbeat/keepalive** pair, a
|
||||
**seq/ack** exchange, a **renewal/refresh** if tokens expire, and an **error/retry** frame if one
|
||||
exists. Each later frame family is only interpretable against that chain.
|
||||
|
||||
**Record the connection as a state chain, not a frame list.** Write the dependencies down as a
|
||||
chain — handshake response grants session → heartbeat carries seq → ack confirms → token near
|
||||
expiry triggers renewal → renewal failure forces reconnect/risk state — because the question that
|
||||
actually decides replayability is *which prior state a frame consumes*. A frame that embeds the
|
||||
session id, a monotonic seq, and an ack of the previous server frame is **not replayable alone**;
|
||||
that is a finding about the protocol, not a failure of your capture.
|
||||
|
||||
**Separate message families before naming anything.** Handshake/auth · heartbeat/keepalive ·
|
||||
business · ack/retry/error · renewal. Only the business family is your target; the others are the
|
||||
state machinery you must reproduce to *reach* the business family. Mislabelling a heartbeat as a
|
||||
protocol error is the standard first misread.
|
||||
|
||||
**Split the envelope from the payload — envelope first.** Long-connection payloads are usually
|
||||
framed: message **type**, **length**, **seq**, **time window**, session identifier, and
|
||||
compression/encryption flags wrap the body. Recover that contract before touching body semantics
|
||||
(and the body itself is very often a protobuf message — §1's decoder and the round-trip check apply
|
||||
unchanged). A body read against a wrong envelope alignment produces plausible garbage, the same
|
||||
failure as §7's wrong frame split.
|
||||
|
||||
**Android capture points** (`inferred`): an OkHttp `WebSocket` surfaces through its listener and
|
||||
`okhttp3.internal.ws` — hookable from Java (`dynamic-frida.md`); a native or engine-internal socket
|
||||
(QUIC, a game/push SDK) may never transit Java at all, and the §5 attribution rules apply — check
|
||||
what is mapped (`scripts/lib_map.py`) before assuming you missed a hook. Note the transport itself
|
||||
can be pinned exactly like §5's native verifier, and `tls-and-cert.md` still owns the trust-chain
|
||||
attribution.
|
||||
|
||||
**Stop rule.** Stop when you can name the state chain, the message families, and the target
|
||||
frame's dependency set, and the envelope decodes stably across several samples. If the real blocker
|
||||
is still "which state did this frame consume", no amount of body decoding helps — go capture the
|
||||
missing family first.
|
||||
|
||||
## 7. Failure modes
|
||||
|
||||
**A decode that looks valid but is not.** protobuf is self-synchronising enough that random bytes
|
||||
often decode into a plausible-looking field list. Guard against it by (a) decoding several samples of
|
||||
|
||||
@@ -197,9 +197,10 @@ Collect: base URLs, DoH/DoT lookups, path constants, CDN hosts, custom headers.
|
||||
|
||||
Treat **unique** strings as navigation aids — they can often be searched byte-wise in the dex to find the owning class (see `references/dex-patching.md` §finding-the-call-site).
|
||||
|
||||
Two patterns worth recognizing early:
|
||||
Three patterns worth recognizing early:
|
||||
- **Dynamic gateway**: the real API host is fetched at runtime (e.g. via a DNS TXT record over DoH). The hardcoded host in dex may be only a fallback. This affects server-side analysis, not patching.
|
||||
- **Minimal headers / no request signing**: if auth is just `Authorization: Bearer` plus a static app-name header, then a repackaged client is not distinguishable to the server by request signature — which matters for `references/server-api.md`.
|
||||
- **BuildConfig sweep** (absorbed 2026-10-20 from `SimoneAvogadro/android-reverse-engineering-skill`, Apache-2.0; the habit is `inferred`, its yield unmeasured here): every Gradle module compiles one generated `BuildConfig` class, and they are a cheap first source of `BASE_URL`/`ENDPOINT` constants, API keys, flavor flags and debug switches — one `--find 'BuildConfig|BASE_URL|FLAVOR'` pass over the string dump. Generated constants are inlined at their use sites even when the class itself is renamed, so the *values* surface in `--urls` output regardless; the sweep is what attributes them to a module.
|
||||
|
||||
## Recon output
|
||||
|
||||
|
||||
@@ -75,15 +75,16 @@ reference file is not named here, or when a script is not named here.
|
||||
| The trace segments themselves look corrupted — handler heads missing, tails glued into wrong segments | `references/native-vm-recovery.md` §7 Nested VMs — a flat segment accumulator discards the outer level; nested slices need a segment stack (`vm_handler_slice.py` flags the symptom) |
|
||||
| You need to know **where two runs of the same operation diverge** (password/license/key), not just what executed | `ete-diff-tracing.md` — ETE hardware-trace gates plus the differential method; `scripts/trace_diff.py` also diffs Stalker logs |
|
||||
| A generated KPM/LKM template is in front of you, or the kernel route feels like the plan | `kernel-event-injection.md` — the primitive map, the composition recipes, and the gates every kernel-side file must pass before it is a weapon |
|
||||
| The traffic is protobuf/gRPC/QUIC, or a proxy sees TLS but requests still fail on a Flutter app | `references/protocol-reverse.md` — schema-less protobuf, frame capture, and native-side pinning |
|
||||
| The traffic is protobuf/gRPC/QUIC, a proxy sees TLS but requests still fail on a Flutter app, **or the behaviour rides a long-lived WebSocket** | `references/protocol-reverse.md` — schema-less protobuf, frame capture, native-side pinning, and the long-connection state chain |
|
||||
| Userspace hooks land and the app still dies: the check reads `/proc/self/status` through a raw `svc`, or runs before `JNI_OnLoad` | `references/kernel-and-environment-hardening.md` — what the next layer up and down can actually do, and when to stop |
|
||||
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
|
||||
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
|
||||
| The value you need (sign, token, risk parameter) is computed in **JavaScript** — a WebView or embedded JS engine — by an entry call fed a huge hex/base64 opcode literal: a JS virtual machine | `jsvmp-recovery.md` — de-obfuscate and **call it** first, then the stack/register-VM procedures and the four JS traps; `emulation-and-rpc.md` owns the call-don't-read route |
|
||||
| You are reading a decompiled **Kotlin** app and every class name is a meaningless short token (`nq.e`, `C0123x`) | `kotlin-name-recovery.md` — the original FQNs survive R8 inside `@DebugMetadata`/`@Metadata`; `scripts/kotlin_name_recovery.py` rebuilds the mapping from any jadx export |
|
||||
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
|
||||
|
||||
_55 row(s) below the header._
|
||||
_56 row(s) below the header._
|
||||
|
||||
## Reference index (no longer inline in `SKILL.md`)
|
||||
|
||||
@@ -97,7 +98,9 @@ _55 row(s) below the header._
|
||||
| `references/code-virtualization-and-custom-linkers.md` | **No packer, dex is readable, and a re-signed build still dies** — whole classes turned `native`, a private loader with a mismatched SONAME, a self-decrypting payload, a Java-layer "signature killer", and the keep-it/drop-it deadlock |
|
||||
| `references/java2c-and-jni-sinking.md` | **Whole classes are bare `native` declarations and you are about to hunt for a decrypted DEX** — Java2C has none, ever; the code is in a `.so`. Also the JNI boundary: why a `Java_*` symbol search fails silently |
|
||||
| `references/vmp-differential-analysis.md` | Method bodies are present but decode as **private opcodes** — a real Dex VMP: the known-plaintext differential, which links can and cannot be automated, how to *prove* a derived opcode table, smali generation, and when the route is closed |
|
||||
| `references/jsvmp-recovery.md` | The protected logic is **JavaScript** (WebView, embedded JS engine, mini-program): a jsvmp opcode literal plus an interpreter — recognition, the call-it-first route, stack- and register-VM reading, instruction extraction, the four JS-language traps, and the shipped loader decoder |
|
||||
| `references/jsvmp-recovery.md` | The protected logic is **JavaScript** (WebView, embedded JS engine, mini-program): a jsvmp opcode literal plus an interpreter — recognition and the wire-parameter family index, the call-it-first route with its A/B/C depth ladder, stack- and register-VM reading, instruction extraction, the four JS-language traps, and the shipped loader decoder |
|
||||
| `references/js-ast-deobfuscation.md` | The JS in front of you is unreadable through **structure** — string tables, decoder IIFEs, flattened control flow — and `jsvmp-recovery.md` §3's "AST pre-deobfuscation" step is next: family detection with `scripts/js_obf_detect.py`, the per-family pass order, safe-rewrite rules, minimal bootstrap eval, residue proof |
|
||||
| `references/kotlin-name-recovery.md` | You are reading a decompiled **Kotlin** app and the class names are meaningless short tokens: the original FQNs survive R8 inside `@DebugMetadata` and `@Metadata` — `scripts/kotlin_name_recovery.py` builds the obfuscated→original map from any jadx export, with `--grep` annotating every hit by its real class |
|
||||
| `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works |
|
||||
| `references/dart-aot.md` | The logic lives in a Dart AOT snapshot (`libapp.so`): pinning the Dart version, building a matching decompiler, the object pool and reference indexing, register/boolean conventions, locating and patching Dart code |
|
||||
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** |
|
||||
@@ -145,7 +148,7 @@ _55 row(s) below the header._
|
||||
| `references/on-device-tooling.md` | Working **from the phone itself**: MT Manager edit/repack/sign and its built-in APK MCP, LSPosed Manager, Termux+frida, and on-device data inspection |
|
||||
| `references/split-apk.md` | The target is a **split APK / App Bundle set** (`base.apk` + `split_config.*.apk`), or `pm path <PKG>` returned several files: reading a set, merge versus unified re-signing, and the install refusal each mistake produces |
|
||||
|
||||
_44 row(s) below the header._
|
||||
_46 row(s) below the header._
|
||||
|
||||
## Script index (no longer inline in `SKILL.md`)
|
||||
|
||||
@@ -163,6 +166,7 @@ _44 row(s) below the header._
|
||||
| `scripts/dex_strpatch.py` | Byte-level string constant patch with **string_ids ordering guard** |
|
||||
| `scripts/dex_classdiff.py` | Compare two dex class tables (set + access flags) to prove a patch was surgical |
|
||||
| `scripts/dex_strings.py` | Dump/extract strings and endpoints from dex without a decompiler |
|
||||
| `scripts/kotlin_name_recovery.py` | Rebuild the **obfuscated→original Kotlin class-name map** from a jadx export: `@DebugMetadata`/`@Metadata.d2`/jadx rename comments, third-party trees skipped, `mapping.tsv`/`json`/`by_package/` outputs, then `-o`/`-p`/`--grep` query modes that annotate every grep hit with the real owning class; `--selftest` pins the three sources |
|
||||
| `scripts/dart_pprefs.py` | Build/query the object-pool offset -> code-site index for a Dart AOT snapshot (arithmetic decode; seconds, not minutes) |
|
||||
| `scripts/dart_pool_strings.py` | Recover string literals from a Dart AOT snapshot: framed entries, the one-byte vs UTF-16 split, file offsets, and a run-length noise filter |
|
||||
| `scripts/dart_disasm.py` | Annotated windowed disassembly of Dart AOT code (pool + boolean annotations) plus a B/BL caller index |
|
||||
@@ -201,6 +205,7 @@ _44 row(s) below the header._
|
||||
| `scripts/protobuf_decode_raw.py` | **Schema-free protobuf decode**: hex / file / stdin to a JSON tree, every length-delimited field as a candidate set with ties labelled; `--reencode --check` for a byte-exact round trip |
|
||||
| `scripts/vmp_diff_harness.py` | Differential hardening for Dex-VMP: build a labelled opcode-coverage fixture (**218 of 224 opcodes, measured**), derive a candidate private-opcode map from an original/hardened dex pair, verify it in a closed loop, render smali |
|
||||
| `scripts/jsvmp_loader_decode.py` | Decode a **jsvmp** program literal (base64/hex → varint → ZigZag → signed opcode stream) with offsets, JSON output and a round-trip `--selftest`; raises on truncated input rather than decoding short |
|
||||
| `scripts/js_obf_detect.py` | **Score a JS bundle's obfuscation family** (hint tokens weight 10, content regexes weight 3: string-array lineage, control-flow flattening, jsvmp entry, opcode if-chains), get the per-family pass order for `js-ast-deobfuscation.md`, plus textual residue metrics for before/after comparisons; `--selftest` pins the fixtures |
|
||||
| `scripts/kernelsu_syscall_mask.py` | Generate a KernelSU/APatch syscall-masking scaffold: an installable userspace module plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label. **A userspace module cannot change a syscall return value** |
|
||||
| `scripts/hw_watch.py` | **Invisible memory watchpoints / execution breakpoints** on arm64: plan (WCR/BCR encoders with the 8-byte-window rules, host-tested), emit (self-contained on-device C helper, compiles clean as static aarch64), gates, run (build+push+stream hits), decode (module+offset per hit). ptrace route: `TracerPid` stays visible — the ledger is in the reference |
|
||||
| `scripts/trace_diff.py` | **Differential trace reduction**: reduce one trace to runs+histogram, or diff two traces into first-divergence / divergent regions / blocks-unique-to-each-side. Producer-agnostic (Stalker `BLK` logs or any address-per-line stream, incl. decoded cs_etm) |
|
||||
@@ -215,4 +220,4 @@ _44 row(s) below the header._
|
||||
| `scripts/svc_scan.py` | Name the syscall behind an inline `svc` and the segment it sits in — which decides whether a libc-level hook can observe the call at all. `--context` shows neighbours, because a byte scan also matches data |
|
||||
| `scripts/anti_detect_probe.js` | Observer-only probe (patches nothing): path/loader/thread/kill hooks with **caller module + offset**, an environment self-report (`TracerPid`, frida-named mappings), and live streaming so a sub-second target still yields evidence |
|
||||
|
||||
_54 row(s) below the header._
|
||||
_56 row(s) below the header._
|
||||
|
||||
@@ -37,10 +37,38 @@ cross-check with the program headers.
|
||||
| **`ddc`** | `ddc app.apk -c <Class>` | **fastest read of dex as Java, plus query subcommands** — reach for this SECOND, to *read*. See below |
|
||||
| **`dexsql`** | `dexsql query --input app.apk --sql "…"` | **the whole dex as SQL tables** (callers, field accesses, numeric constants, debug lines, annotations) plus manifest and `resources.arsc` queries and an MCP server. Reach for it when the question is a **join or an aggregate** rather than a class list — see below |
|
||||
| `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources |
|
||||
| `jadx` | `jadx --no-res -d <out> <apk>` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) |
|
||||
| `jadx` | `jadx --deobf -d <out> <apk>` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) |
|
||||
| Fernflower / Vineflower | `java -jar vineflower.jar -dgs=1 -mpm=60 …` (via dex2jar) | the **second engine** for the classes jadx mangles — see below |
|
||||
| `aapt2` | `aapt2 dump badging <apk>` | manifest facts, package name, versions |
|
||||
| `zipalign`, `apksigner` | from build-tools | alignment and signing |
|
||||
|
||||
### jadx and a second engine — the readable-Java tier (`unverified` unless stated)
|
||||
|
||||
jadx stays what the table says: a viewer of located classes. Four flags change what you get,
|
||||
absorbed 2026-10-20 from `SimoneAvogadro/android-reverse-engineering-skill` (Apache-2.0):
|
||||
|
||||
- **`--deobf`** renames obfuscated identifiers to *synthetic* readable ones and writes the mapping
|
||||
to `<out>/deobf-mapping.txt`. It does **not** recover original names — for a Kotlin target the
|
||||
original names live in annotations, and `references/kotlin-name-recovery.md` recovers them.
|
||||
- **`--deobf-map <mapping.txt>`** consumes a real ProGuard/R8 mapping — and an APK sometimes
|
||||
**ships one** (search `assets/` and the dex strings for `mapping.txt`/`.map` before assuming
|
||||
there is none). When a mapping exists this beats every heuristic.
|
||||
- **`--show-bad-code`** emits partially-decompiled bodies instead of error comments — useful
|
||||
exactly where you need to see *something* of a class jadx struggles with.
|
||||
- **`-e` / `--export-gradle`** exports a Gradle project (IDE navigation); **per-dex targeting**
|
||||
(`unzip` the APK, run jadx on one `classes*.dex`) keeps cost down on multi-dex giants.
|
||||
|
||||
**Fernflower / Vineflower as the second engine.** Where jadx's output on a class is wrong or
|
||||
empty (complex lambdas, generics, streams — the shapes a Kotlin app is full of), the analytical
|
||||
decompiler often produces a readable body jadx cannot. It reads class files, not dex: the chain
|
||||
is `d2j-dex2jar -f -o app.jar app.apk` → `java -jar vineflower.jar -dgs=1 -mpm=60 -ren=1
|
||||
-e=$ANDROID_HOME/platforms/android-<N>/android.jar app.jar out/` → unzip the source jar. `-dgs=1`
|
||||
decompiles generic signatures, `-mpm=60` bounds each method (a decompiler hang becomes a skip),
|
||||
`-e=android.jar` gives it the framework types. **Run both engines and compare** when a class is
|
||||
load-bearing: two independent bodies agreeing is cheap corroboration a single body cannot give.
|
||||
All of this tier is `unverified` here — no APK was decompiled by this route in this repository —
|
||||
and it stays a *reading* route: the patch decision still belongs to the byte/smali tier above.
|
||||
|
||||
### rasc — the same index in Rust, when you have built it
|
||||
|
||||
`rasc` is the `rust` branch of the same project (`MG1937/ASC`). Same CLI shape — `classes`,
|
||||
|
||||
@@ -0,0 +1,278 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Detect which obfuscation family a JS bundle belongs to, by weighted content probes.
|
||||
|
||||
This is the detection layer of the JS AST pre-deobfuscation route documented in
|
||||
`references/js-ast-deobfuscation.md`: before running any rewrite pass you want to know
|
||||
*which family* you face, because the pass order differs per family (a wrong order stalls
|
||||
or corrupts -- the absorbed architecture's own measured example: late literal-inlining
|
||||
hangs on large VM-heavy samples, so that family skips it).
|
||||
|
||||
Design (absorbed, see the reference for provenance): every family carries
|
||||
|
||||
* ``hint`` tokens -- matched against the file path and the user-supplied ``--hint``
|
||||
text; a hit scores 10, because a human saying "this is the delivery platform's SDK"
|
||||
is stronger evidence than any content regex;
|
||||
* ``content`` regexes -- matched against the source text; each hit scores 3.
|
||||
|
||||
Any family scoring > 0 is a detection; the top scorer is the recommended pipeline. No
|
||||
hit means the generic pipeline, which is also the escalation target when residue stays
|
||||
stable after the family passes.
|
||||
|
||||
What this script deliberately does NOT do: rewrite anything. The passes need a real JS
|
||||
parser (node + @babel/parser + traverse + generator); this skill ships the detector and
|
||||
the pass order, not the rewriting engine -- the fence is stated in the reference.
|
||||
|
||||
The residue metrics are *textual proxies* (identifier shapes, literal densities, chain
|
||||
counts), not AST counts; they are honest for comparing before/after a pass batch on the
|
||||
same file and for nothing finer.
|
||||
|
||||
Evidence: `observed` only for `--selftest` against embedded fixtures. No real obfuscated
|
||||
target was ever run through this detector here; every threshold is a starting point, not
|
||||
a boundary.
|
||||
|
||||
Examples:
|
||||
python scripts/js_obf_detect.py bundle.js
|
||||
python scripts/js_obf_detect.py bundle.js --hint 'delivery-platform sign sdk' --json
|
||||
python scripts/js_obf_detect.py --selftest
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
|
||||
FAMILIES = [
|
||||
{
|
||||
"id": "ob_string_array",
|
||||
"display": "string-array / obfuscator.io lineage",
|
||||
"hint": ["obfuscator", "javascript-obfuscator", "ob.io"],
|
||||
"pipeline": ["normalize", "prune_fake_branches", "string_array_pass",
|
||||
"inline_dispatchers", "flatten_array_cff", "prune_fake_branches",
|
||||
"normalize"],
|
||||
"content": [
|
||||
(r'\b_0x[a-f0-9]{3,}\b', "hex-mangled identifiers"),
|
||||
(r'var\s+_0x[a-f0-9]+\s*=\s*\[', "top-level string array"),
|
||||
(r'\b(?:push\s*\(\s*(?:shift|pop)\s*\(\s*\)\s*\)|shift\s*\(\s*\)\s*\))', "rotate/pull decoder idiom"),
|
||||
(r'String\s*\.\s*fromCharCode', "charCode decoder"),
|
||||
(r'\bparseInt\s*\(\s*[A-Za-z_$][\w$]*\s*\+\s*[^)]*?,\s*16\s*\)', "radix-16 decoder math"),
|
||||
],
|
||||
"notes": "Decode the string array before any control-flow work: every later pass "
|
||||
"reads strings through it, and residue metrics on undecoded samples are noise.",
|
||||
},
|
||||
{
|
||||
"id": "control_flow_flattening",
|
||||
"display": "control-flow flattening (switch dispatch)",
|
||||
"hint": ["flatten", "cff", "ollvm-js"],
|
||||
"pipeline": ["normalize", "prune_fake_branches", "inline_dispatchers",
|
||||
"flatten_array_cff", "if_chain_to_switch", "prune_fake_branches",
|
||||
"normalize"],
|
||||
"content": [
|
||||
(r'while\s*\(\s*(?:!!\[\]|\(?!!\[\]\)?|!0|0x1|1)\s*\)\s*\{', "infinite-loop driver"),
|
||||
(r'["\'][0-9]+\|[0-9|]+["\']\s*\.\s*split\s*\(\s*["\']\|["\']\s*\)', "sequence-string order table"),
|
||||
(r'switch\s*\(\s*_?0x[a-f0-9]+', "switch on mangled var"),
|
||||
],
|
||||
"notes": "Recover the real block order from the state variable's assignment chain; "
|
||||
"the dispatcher block is noise, not structure.",
|
||||
},
|
||||
{
|
||||
"id": "jsvmp_entry",
|
||||
"display": "jsvmp entry (opcode literal + interpreter)",
|
||||
"hint": ["jsvmp", "js vmp", "sign", "risk"],
|
||||
"pipeline": ["string_array_pass", "normalize", "prune_fake_branches",
|
||||
"inline_dispatchers", "if_chain_to_switch"],
|
||||
"content": [
|
||||
(r'[_$][\w$]{0,12}\'?\]?\s*\(\s*[\'"][0-9a-fA-F]{%d,}[\'"]', "huge hex opcode literal"),
|
||||
(r'[_$][\w$]{0,12}\'?\]?\s*\(\s*[\'"][A-Za-z0-9+/=]{%d,}[\'"]', "huge base64 opcode literal"),
|
||||
(r'\[\s*(?:Object|Function|Math)\s*,', "globals dependency array"),
|
||||
(r'_\$\w+', "_$-prefixed vendor identifiers"),
|
||||
],
|
||||
"notes": "Do NOT beautify the interpreter into 'readable' form before reading the "
|
||||
"dispatcher: the pass order here stops at making the dispatch readable. "
|
||||
"See references/jsvmp-recovery.md for the VM work itself.",
|
||||
# The huge-literal thresholds are parameters, not constants: a selftest or a small
|
||||
# sample legitimately lowers them.
|
||||
"literal_min": 1500,
|
||||
},
|
||||
{
|
||||
"id": "opcode_if_chain",
|
||||
"display": "opcode if-chain dispatch",
|
||||
"hint": ["if chain", "opcode chain"],
|
||||
"pipeline": ["normalize", "prune_fake_branches", "if_chain_to_switch",
|
||||
"prune_fake_branches", "normalize"],
|
||||
"content": [
|
||||
(r'if\s*\(\s*(?:0x[0-9a-f]+|\d+)\s*[=!]==?\s*[A-Za-z_$][\w$]*', "literal-vs-var comparison"),
|
||||
(r'else\s+if\s*\(\s*(?:0x[0-9a-f]+|\d+)\s*[=!]==?\s*[A-Za-z_$][\w$]*', "chained else-if"),
|
||||
],
|
||||
"notes": "Rewrite to switch ONLY when the whole chain tests one discriminant against "
|
||||
"literals; a mixed chain left as-is beats a mangled rewrite.",
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def score_families(source_text, hint_text, path_text, literal_min):
|
||||
"""Return [detection dicts] sorted by score desc then id; every family with score > 0."""
|
||||
detections = []
|
||||
haystack = " ".join(filter(None, [hint_text, path_text])).lower()
|
||||
for fam in FAMILIES:
|
||||
score = 0
|
||||
reasons = []
|
||||
for token in fam.get("hint", []):
|
||||
if token.lower() and token.lower() in haystack:
|
||||
score += 10
|
||||
reasons.append("hint:" + token)
|
||||
for rx_src, what in fam["content"]:
|
||||
rx_src = _bind_literal_min(rx_src, literal_min)
|
||||
if re.search(rx_src, source_text):
|
||||
score += 3
|
||||
reasons.append("content:" + what)
|
||||
if score > 0:
|
||||
detections.append({"id": fam["id"], "score": score, "reasons": reasons})
|
||||
detections.sort(key=lambda d: (-d["score"], d["id"]))
|
||||
return detections
|
||||
|
||||
|
||||
def _bind_literal_min(rx_src, literal_min):
|
||||
"""Families declare huge-literal thresholds with %d so callers can lower them."""
|
||||
return rx_src % literal_min if "%d" in rx_src else rx_src
|
||||
|
||||
|
||||
def residue_metrics(source_text):
|
||||
"""Textual proxies for 'how obfuscated does this still look'. Compare same-file only."""
|
||||
lines = source_text.splitlines()
|
||||
identifiers = set(re.findall(r'\b_0x[a-f0-9]{3,}\b', source_text))
|
||||
hex_lits = re.findall(r'\b0x[a-fA-F0-9]{2,}\b', source_text)
|
||||
comma_expr = re.findall(r'(?<=[;{}])\s*[^;{}()]{1,80},[^;{}]{1,80}\s*(?=[;{}])', source_text)
|
||||
switches = len(re.findall(r'\bswitch\s*\(', source_text))
|
||||
cases = len(re.findall(r'\bcase\s+', source_text))
|
||||
longest = max((len(ln) for ln in lines), default=0)
|
||||
evals = len(re.findall(r'\beval\s*\(', source_text))
|
||||
return {
|
||||
"lines": len(lines),
|
||||
"mangled_identifiers": len(identifiers),
|
||||
"hex_literals": len(hex_lits),
|
||||
"comma_expressions": len(comma_expr),
|
||||
"switches": switches,
|
||||
"cases": cases,
|
||||
"longest_line": longest,
|
||||
"eval_calls": evals,
|
||||
}
|
||||
|
||||
|
||||
def pipeline_for(family_id):
|
||||
"""The ordered pass list for a family (or the generic ladder when nothing scored)."""
|
||||
fam = next((f for f in FAMILIES if f["id"] == family_id), None)
|
||||
if fam is None:
|
||||
return ["normalize", "prune_fake_branches", "inline_dispatchers",
|
||||
"flatten_array_cff", "if_chain_to_switch", "prune_fake_branches", "normalize"]
|
||||
return list(fam["pipeline"])
|
||||
|
||||
|
||||
def notes_for(family_id):
|
||||
fam = next((f for f in FAMILIES if f["id"] == family_id), None)
|
||||
if fam is None:
|
||||
return "Generic ladder only. Escalate to a family adapter when residue is stable and reproducible."
|
||||
return fam["notes"]
|
||||
|
||||
|
||||
FIXTURES = {
|
||||
"ob_string_array": (
|
||||
"var _0x4ab1 = ['\\x68\\x65\\x6c\\x6c\\x6f', 'push', 'shift'];\n"
|
||||
"(function (_0x1, _0x2) { var _0x3 = function (_0x4) { while (--_0x4) { _0x1.push(_0x1.shift()); } }; _0x3(++_0x2); }(_0x4ab1, 0x1a4));\n"
|
||||
"var _0x9f = function (_0x1, _0x2) { _0x1 = _0x1 - 0x0; var _0x3 = _0x4ab1[_0x1]; return _0x3; };\n"
|
||||
"console[_0x9f('0x0')](_0x9f('0x1'));\n"
|
||||
),
|
||||
"control_flow_flattening": (
|
||||
"function f(_0x1, _0x2) { var _0x3 = '3|1|2'.split('|'), _0x4 = 0; "
|
||||
"while (!![]) { switch (_0x3[_0x4++]) { case '1': var _0x5 = _0x1 + _0x2; continue; "
|
||||
"case '2': return _0x5; case '3': _0x1 = _0x1 | 0; continue; } break; } }\n"
|
||||
),
|
||||
"opcode_if_chain": (
|
||||
"function d(o) { if (0x31 === o) { return 1; } else if (0x32 === o) { return 2; } "
|
||||
"else if (0x33 === o) { return 3; } else if (0x34 === o) { return 4; } return 0; }\n"
|
||||
),
|
||||
}
|
||||
|
||||
|
||||
def _jsvmp_fixture(literal_min):
|
||||
literal = "ab" * (literal_min // 2 + 10)
|
||||
return "global['_$webrt_%s']('%s', [Object, Math, setTimeout, void 0, false, entry]);\n" % ("d2a1", literal)
|
||||
|
||||
|
||||
def selftest(literal_min):
|
||||
"""Each embedded fixture must select its own family as the TOP pick.
|
||||
|
||||
Exclusivity is NOT asserted: families genuinely overlap by design (hex-mangled
|
||||
identifiers appear in the CFF fixture too), so the test pins the winner, not the
|
||||
loser list. The hint-beats-content rule (10 vs 3) and the plain-code-is-generic
|
||||
case are pinned as well, because both decide real routing.
|
||||
"""
|
||||
for fam_id, body in FIXTURES.items():
|
||||
detections = score_families(body, "", "", literal_min)
|
||||
assert detections, "no detection for %s" % fam_id
|
||||
assert detections[0]["id"] == fam_id, (fam_id, detections)
|
||||
jsvmp_body = _jsvmp_fixture(literal_min)
|
||||
detections = score_families(jsvmp_body, "", "", literal_min)
|
||||
assert detections and detections[0]["id"] == "jsvmp_entry", detections
|
||||
# A hint alone is worth 10 and must beat a single content hit (3).
|
||||
hinted = score_families("plain text, nothing obfuscated", "delivery-platform obfuscator", "x.js", literal_min)
|
||||
assert hinted and hinted[0]["id"] == "ob_string_array" and hinted[0]["score"] == 10, hinted
|
||||
# Plain code selects nothing: generic ladder.
|
||||
plain = score_families("function add(a, b) { return a + b; }\n", "", "", literal_min)
|
||||
assert plain == [], plain
|
||||
residues = residue_metrics(FIXTURES["ob_string_array"])
|
||||
# The 3+ hex-char filter counts _0x4ab1 but not short names (_0x1, _0x9f) -- pinned so
|
||||
# the filter does not silently loosen to counting every _0x-prefixed token.
|
||||
assert residues["mangled_identifiers"] == 1 and residues["eval_calls"] == 0, residues
|
||||
print("selftest: 4 family fixtures + hint-beats-content + plain-is-generic + residues: ok")
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("input", nargs="?", help="JS source file to classify")
|
||||
ap.add_argument("--hint", default="", help="free text: target name, vendor, parameter name")
|
||||
ap.add_argument("--json", action="store_true", help="machine-readable output")
|
||||
ap.add_argument("--min-literal", type=int, default=1500,
|
||||
help="char length that makes a literal 'huge' (default 1500; lower for small samples)")
|
||||
ap.add_argument("--selftest", action="store_true", help="run the embedded fixture tests")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.selftest:
|
||||
return selftest(args.min_literal)
|
||||
if not args.input:
|
||||
ap.print_usage()
|
||||
print("classify: script bundle.js [--hint TEXT] families: %s" %
|
||||
", ".join(f["id"] for f in FAMILIES))
|
||||
return 2
|
||||
if not os.path.isfile(args.input):
|
||||
print("not a file: %s" % args.input, file=sys.stderr)
|
||||
return 1
|
||||
with open(args.input, "r", errors="replace") as fh:
|
||||
text = fh.read()
|
||||
|
||||
detections = score_families(text, args.hint, args.input, args.min_literal)
|
||||
best = detections[0]["id"] if detections else "generic"
|
||||
result = {
|
||||
"input": os.path.abspath(args.input),
|
||||
"best": best,
|
||||
"detections": detections,
|
||||
"pipeline": pipeline_for(best),
|
||||
"notes": notes_for(best),
|
||||
"residues": residue_metrics(text),
|
||||
}
|
||||
if args.json:
|
||||
print(json.dumps(result, indent=1))
|
||||
else:
|
||||
print("family: %s" % best)
|
||||
for d in detections:
|
||||
print(" %-3d %-22s %s" % (d["score"], d["id"], ", ".join(d["reasons"])))
|
||||
if not detections:
|
||||
print(" (no family scored; generic ladder)")
|
||||
print("pipeline order: %s" % " -> ".join(result["pipeline"]))
|
||||
print("note: %s" % result["notes"])
|
||||
print("residues: %s" % json.dumps(result["residues"], sort_keys=True))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,317 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Rebuild an (obfuscated -> original) Kotlin class-name map from decompiled sources.
|
||||
|
||||
R8/ProGuard rename JVM symbols but cannot strip the Kotlin metadata the Kotlin runtime
|
||||
itself needs for reflection and coroutines, so two annotations survive into any decompiled
|
||||
Java and still carry the *original* fully-qualified names:
|
||||
|
||||
* ``@DebugMetadata(c = "com.example.Foo$bar$1", ...)`` -- emitted for nearly every
|
||||
``suspend`` function's SuspendLambda; everything after the first ``$`` is inner/lambda
|
||||
scope, so the prefix is the declaring class.
|
||||
* ``@Metadata(d2 = {"...", "Lcom/example/Foo;", ...})`` -- every Kotlin class; the first
|
||||
non-stdlib ``L...;`` descriptor in ``d2`` is usually the file's primary class.
|
||||
* jadx's ``/* renamed from: X */`` comments as a last-resort fallback.
|
||||
|
||||
This is a *reading aid*, not ground truth: it feeds names back into a jadx export you are
|
||||
already navigating, and every mapping is a hint. Method and field names are NOT recovered --
|
||||
metadata preserves class-level FQNs only.
|
||||
|
||||
Provenance: absorbed 2026-10-20 from SimoneAvogadro/android-reverse-engineering-skill
|
||||
(Apache-2.0), its ``recover-kotlin-names.sh`` + ``references/kotlin-name-recovery.md``;
|
||||
re-implemented here as a dependency-free Python script. The upstream recovery-rate claims
|
||||
("30-50% of classes, ~100% of *Repository/*ViewModel/*Impl") are the upstream author's and
|
||||
are ``unverified`` in this repository. The script itself is measured only by ``--selftest``
|
||||
on embedded fixtures (``observed`` for the parsing, nothing more).
|
||||
|
||||
Examples:
|
||||
python scripts/kotlin_name_recovery.py <jadx-export>/sources # build the map
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> -o nq.e # obf -> real
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> -p com.example # list a package
|
||||
python scripts/kotlin_name_recovery.py --map <map-dir> --grep '"api/' <jadx-export>/sources
|
||||
python scripts/kotlin_name_recovery.py --selftest
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import tempfile
|
||||
from collections import defaultdict
|
||||
|
||||
# @DebugMetadata(c = "com.foo.Bar$Inner$1", ...) -- `c` carries the original outer FQN.
|
||||
RE_DEBUG = re.compile(r'@DebugMetadata\([^)]*?c\s*=\s*"([^"]+)"', re.S)
|
||||
# @Metadata(... d2 = { "...", "Lcom/foo/Bar;", ... } ...) -- internal class refs of the file.
|
||||
RE_DTWO = re.compile(r'@Metadata\([^)]*?d2\s*=\s*\{([^}]*)\}', re.S)
|
||||
# A JVM class descriptor inside d2: Lcom/foo/Bar;
|
||||
RE_LCLASS = re.compile(r'L([A-Za-z][\w/$]+);')
|
||||
# jadx emits this comment above classes it renamed during deobfuscation.
|
||||
RE_RENAMED = re.compile(r'/\*\s*renamed from:\s*([\w.$]+)\s*\*/')
|
||||
|
||||
# Third-party / framework trees whose names are already real, so a "recovery" there is noise.
|
||||
SKIP_PREFIXES = (
|
||||
"kotlin.", "kotlinx.", "androidx.", "android.", "java.", "javax.",
|
||||
"com.google.", "com.facebook.", "com.appsflyer.", "com.datadog.",
|
||||
"io.ktor.", "io.sentry.", "io.realm.", "okhttp3.", "okio.",
|
||||
"com.squareup.", "com.bumptech.", "com.airbnb.",
|
||||
"com.amplitude.", "com.segment.", "com.mixpanel.", "com.onesignal.",
|
||||
"com.stripe.", "retrofit2.", "dagger.", "javax.inject.", "org.jetbrains.",
|
||||
)
|
||||
|
||||
MAP_TSV = "mapping.tsv"
|
||||
MAP_JSON = "mapping.json"
|
||||
BY_PACKAGE = "by_package"
|
||||
|
||||
|
||||
def recover_real_name(text):
|
||||
"""Return (real_fqn, source) for one decompiled file's text, or (None, None).
|
||||
|
||||
Order is deliberate: @DebugMetadata is the most precise (a true declaring-class FQN),
|
||||
d2 descriptors are a strong hint, jadx's rename comment is the weakest.
|
||||
"""
|
||||
m = RE_DEBUG.search(text)
|
||||
if m:
|
||||
# Everything after the first `$` is inner/lambda scope of the original source tree.
|
||||
return m.group(1).split("$", 1)[0], "debug_metadata"
|
||||
m = RE_DTWO.search(text)
|
||||
if m:
|
||||
for lm in RE_LCLASS.finditer(m.group(1)):
|
||||
cand = lm.group(1).replace("/", ".").split("$", 1)[0]
|
||||
# A real package has at least one dot and is not a stdlib/framework ref.
|
||||
if "." in cand and not cand.startswith(SKIP_PREFIXES):
|
||||
return cand, "metadata_d2"
|
||||
m = RE_RENAMED.search(text)
|
||||
if m:
|
||||
return m.group(1).split("$", 1)[0], "renamed_comment"
|
||||
return None, None
|
||||
|
||||
|
||||
def mine_tree(sources_dir):
|
||||
"""Walk every *.java under sources_dir; return {obf_fqn: {real, file, source}}."""
|
||||
mapping = {}
|
||||
counts = defaultdict(int)
|
||||
for dirpath, _dirs, files in os.walk(sources_dir):
|
||||
for fname in files:
|
||||
if not fname.endswith(".java"):
|
||||
continue
|
||||
path = os.path.join(dirpath, fname)
|
||||
rel = os.path.relpath(path, sources_dir)
|
||||
obf = rel[:-len(".java")].replace(os.sep, ".")
|
||||
if obf.startswith(SKIP_PREFIXES):
|
||||
continue
|
||||
try:
|
||||
with open(path, "r", errors="replace") as fh:
|
||||
text = fh.read()
|
||||
except OSError:
|
||||
continue
|
||||
real, source = recover_real_name(text)
|
||||
if real:
|
||||
mapping[obf] = {"real": real, "file": path, "source": source}
|
||||
counts[source] += 1
|
||||
return mapping, dict(counts)
|
||||
|
||||
|
||||
def default_map_dir(sources_dir):
|
||||
"""The mapping lands next to the sources tree, matching an export-then-query flow."""
|
||||
return os.path.join(os.path.dirname(os.path.abspath(sources_dir)), "kotlin-name-map")
|
||||
|
||||
|
||||
def write_map(mapping, counts, out_dir):
|
||||
os.makedirs(os.path.join(out_dir, BY_PACKAGE), exist_ok=True)
|
||||
with open(os.path.join(out_dir, MAP_TSV), "w") as fh:
|
||||
fh.write("obf_fqn\treal_fqn\tsource\tfile\n")
|
||||
for obf in sorted(mapping):
|
||||
entry = mapping[obf]
|
||||
fh.write("%s\t%s\t%s\t%s\n" % (obf, entry["real"], entry["source"], entry["file"]))
|
||||
with open(os.path.join(out_dir, MAP_JSON), "w") as fh:
|
||||
json.dump({"counts": counts, "mapping": mapping}, fh, indent=1, sort_keys=True)
|
||||
fh.write("\n")
|
||||
by_pkg = defaultdict(list)
|
||||
for obf, entry in mapping.items():
|
||||
by_pkg[entry["real"].rsplit(".", 1)[0]].append((entry["real"], obf, entry["file"]))
|
||||
for pkg, rows in by_pkg.items():
|
||||
safe = pkg.replace(".", "_")
|
||||
with open(os.path.join(out_dir, BY_PACKAGE, safe + ".tsv"), "w") as fh:
|
||||
fh.write("real_fqn\tobf_fqn\tfile\n")
|
||||
for row in sorted(rows):
|
||||
fh.write("%s\t%s\t%s\n" % row)
|
||||
return len(by_pkg)
|
||||
|
||||
|
||||
def load_map(map_dir):
|
||||
with open(os.path.join(map_dir, MAP_JSON), "r") as fh:
|
||||
return json.load(fh)["mapping"]
|
||||
|
||||
|
||||
def resolve_query(args, mapping):
|
||||
"""Run one query mode against a loaded mapping; returns the lines to print."""
|
||||
lines = []
|
||||
if args.obf:
|
||||
if args.obf in mapping:
|
||||
lines.append("%s -> %s (%s)" % (args.obf, mapping[args.obf]["real"],
|
||||
mapping[args.obf]["source"]))
|
||||
else:
|
||||
# Sibling classes under the same short prefix are useful context: they are
|
||||
# frequently the same original class's split lambdas/inner classes.
|
||||
prefix = args.obf.rsplit(".", 1)[0] + "."
|
||||
siblings = sorted(k for k in mapping if k.startswith(prefix))
|
||||
for sib in siblings:
|
||||
lines.append("%s -> %s (%s)" % (sib, mapping[sib]["real"],
|
||||
mapping[sib]["source"]))
|
||||
if not siblings:
|
||||
lines.append("not in mapping: %s" % args.obf)
|
||||
elif args.package:
|
||||
for obf in sorted(mapping):
|
||||
real = mapping[obf]["real"]
|
||||
if real == args.package or real.startswith(args.package + "."):
|
||||
lines.append("%s\t%s" % (real, obf))
|
||||
return lines
|
||||
|
||||
|
||||
def grep_with_map(pattern, sources_dir, mapping):
|
||||
"""Grep sources, annotating each hit with the real owning class when known."""
|
||||
rx = re.compile(pattern)
|
||||
hits = 0
|
||||
for dirpath, _dirs, files in os.walk(sources_dir):
|
||||
for fname in files:
|
||||
if not fname.endswith(".java"):
|
||||
continue
|
||||
path = os.path.join(dirpath, fname)
|
||||
rel = os.path.relpath(path, sources_dir)
|
||||
obf = rel[:-len(".java")].replace(os.sep, ".")
|
||||
try:
|
||||
with open(path, "r", errors="replace") as fh:
|
||||
for lineno, line in enumerate(fh, 1):
|
||||
if rx.search(line):
|
||||
real = mapping.get(obf, {}).get("real", "?")
|
||||
print("%s:%d: %s // %s" % (rel, lineno, line.rstrip(), real))
|
||||
hits += 1
|
||||
except OSError:
|
||||
continue
|
||||
if not hits:
|
||||
print("(no hits for %r under %s)" % (pattern, sources_dir))
|
||||
|
||||
|
||||
FIXTURE_DEBUG = '''package nq;
|
||||
|
||||
@DebugMetadata(
|
||||
c = "com.example.feature.account.AccountRepositoryImpl$fetch$1",
|
||||
f = "AccountRepositoryImpl.kt",
|
||||
l = {42, 51},
|
||||
m = "invokeSuspend"
|
||||
)
|
||||
public final class a extends kotlin.coroutines.jvm.internal.SuspendLambda {
|
||||
}
|
||||
'''
|
||||
|
||||
FIXTURE_METADATA = '''package p0.q;
|
||||
|
||||
@Metadata(
|
||||
mvp = 1,
|
||||
d1 = {"..." },
|
||||
d2 = {"", "", "Lcom/example/feature/login/LoginViewModel;", "", "", ""}
|
||||
)
|
||||
public final class b {
|
||||
}
|
||||
'''
|
||||
|
||||
FIXTURE_RENAMED = '''package z;
|
||||
|
||||
/* renamed from: com.example.core.network.ApiClient */
|
||||
public class C0123x {
|
||||
}
|
||||
'''
|
||||
|
||||
FIXTURE_PLAIN = '''package z;
|
||||
|
||||
public class Untouched {
|
||||
}
|
||||
'''
|
||||
|
||||
|
||||
def selftest():
|
||||
"""Parse the three recovery sources plus a negative fixture; assert each outcome.
|
||||
|
||||
The fixtures pin the *shapes* (annotation layout, `$`-suffix scope, d2 descriptor list,
|
||||
rename comment), so a regex regression fails here rather than on a real export.
|
||||
"""
|
||||
real, source = recover_real_name(FIXTURE_DEBUG)
|
||||
assert real == "com.example.feature.account.AccountRepositoryImpl", real
|
||||
assert source == "debug_metadata"
|
||||
real, source = recover_real_name(FIXTURE_METADATA)
|
||||
assert real == "com.example.feature.login.LoginViewModel", real
|
||||
assert source == "metadata_d2"
|
||||
real, source = recover_real_name(FIXTURE_RENAMED)
|
||||
assert real == "com.example.core.network.ApiClient", real
|
||||
assert source == "renamed_comment"
|
||||
real, source = recover_real_name(FIXTURE_PLAIN)
|
||||
assert real is None and source is None
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
src = os.path.join(tmp, "sources")
|
||||
for rel, body in (("nq/a.java", FIXTURE_DEBUG),
|
||||
("p0/q/b.java", FIXTURE_METADATA),
|
||||
("z/C0123x.java", FIXTURE_RENAMED),
|
||||
("z/Untouched.java", FIXTURE_PLAIN),
|
||||
("kotlin/x/Keep.java", FIXTURE_DEBUG)):
|
||||
path = os.path.join(src, rel)
|
||||
os.makedirs(os.path.dirname(path), exist_ok=True)
|
||||
with open(path, "w") as fh:
|
||||
fh.write(body)
|
||||
mapping, counts = mine_tree(src)
|
||||
# kotlin.* is skipped as already-real: exactly three mappings must come back.
|
||||
assert len(mapping) == 3, mapping
|
||||
assert counts == {"debug_metadata": 1, "metadata_d2": 1, "renamed_comment": 1}, counts
|
||||
out = os.path.join(tmp, "map")
|
||||
packages = write_map(mapping, counts, out)
|
||||
assert packages == 3, packages
|
||||
assert load_map(out)["nq.a"]["real"] == "com.example.feature.account.AccountRepositoryImpl"
|
||||
print("selftest: 3 recovery sources + skip-list + round trip through the map files: ok")
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(description=__doc__.splitlines()[0])
|
||||
ap.add_argument("sources", nargs="?", help="decompiled sources root (build mode)")
|
||||
ap.add_argument("--out", help="map output dir (default: next to the sources dir)")
|
||||
ap.add_argument("--map", dest="map_dir", help="map dir for query modes (-o/-p)")
|
||||
ap.add_argument("-o", "--obf", help="resolve one obfuscated FQN (siblings listed as context)")
|
||||
ap.add_argument("-p", "--package", help="list real classes under one original package")
|
||||
ap.add_argument("--grep", help="regex; grep sources with the real class annotated per hit")
|
||||
ap.add_argument("--selftest", action="store_true", help="run the embedded fixture tests")
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.selftest:
|
||||
return selftest()
|
||||
if args.obf or args.package:
|
||||
if not args.map_dir:
|
||||
ap.error("-o/-p need --map <dir> (build it first with: script <sources-dir>)")
|
||||
for line in resolve_query(args, load_map(args.map_dir)):
|
||||
print(line)
|
||||
return 0
|
||||
if args.grep:
|
||||
if not (args.map_dir and args.sources):
|
||||
ap.error("--grep needs --map <dir> and a sources dir")
|
||||
grep_with_map(args.grep, args.sources, load_map(args.map_dir))
|
||||
return 0
|
||||
if not args.sources:
|
||||
ap.print_usage()
|
||||
print("build: script <sources-dir> [--out DIR] query: --map DIR -o nq.e / -p com.pkg / --grep")
|
||||
return 2
|
||||
if not os.path.isdir(args.sources):
|
||||
print("not a directory: %s" % args.sources, file=sys.stderr)
|
||||
return 1
|
||||
out_dir = args.out or default_map_dir(args.sources)
|
||||
mapping, counts = mine_tree(args.sources)
|
||||
packages = write_map(mapping, counts, out_dir)
|
||||
total = 0
|
||||
for dirpath, _dirs, files in os.walk(args.sources):
|
||||
total += sum(1 for f in files if f.endswith(".java"))
|
||||
print("mapped %d of %d .java files (%d original packages) -> %s" %
|
||||
(len(mapping), total, packages, out_dir))
|
||||
print("by source: %s" % json.dumps(counts, sort_keys=True))
|
||||
print("next: --map %s -o <obf.Fqn> | -p <com.example.pkg> | --grep <pattern> <sources-dir>" % out_dir)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -55,7 +55,9 @@ import time
|
||||
try:
|
||||
import frida
|
||||
except ImportError: # keep --help working on a machine without frida
|
||||
frida = None
|
||||
# Same measured asymmetry as check_repo.py's yaml: only ill-typed where frida is
|
||||
# installed; unused-ignored on a bare machine (warn_unused_ignores is off).
|
||||
frida = None # type: ignore[assignment]
|
||||
|
||||
DEFAULT_ADB = "adb"
|
||||
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Pinned-shape tests for the JS obfuscation family detector.
|
||||
|
||||
The detector (`skills/apk-reverse/scripts/js_obf_detect.py`) scores families by weighted
|
||||
probes (hint token = 10, content regex = 3). These tests pin the routing decisions that
|
||||
matter: each fixture selects its own family as the top pick, a hint outranks a single
|
||||
content hit, plain code routes to the generic ladder, and the huge-literal threshold is
|
||||
a parameter the caller can lower for small samples.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', 'skills',
|
||||
'apk-reverse', 'scripts'))
|
||||
|
||||
from js_obf_detect import ( # noqa: E402
|
||||
FIXTURES, _jsvmp_fixture, pipeline_for, residue_metrics, score_families,
|
||||
)
|
||||
|
||||
|
||||
def test_each_fixture_wins_its_own_family():
|
||||
for fam_id, body in FIXTURES.items():
|
||||
detections = score_families(body, '', 'sample.js', 1500)
|
||||
assert detections, 'no detection for %s' % fam_id
|
||||
assert detections[0]['id'] == fam_id, (fam_id, detections)
|
||||
|
||||
|
||||
def test_jsvmp_entry_needs_the_literal_threshold():
|
||||
# The huge-literal probes only fire at the configured length: with the default 1500
|
||||
# threshold a 210-char literal scores nothing for the literal itself, and the fixture
|
||||
# is detected (if at all) through the dependency-array probe alone. Lowering
|
||||
# --min-literal is the documented way to classify small samples.
|
||||
body = _jsvmp_fixture(200)
|
||||
default_detections = score_families(body, '', 's.js', 1500)
|
||||
reasons = [r for d in default_detections if d['id'] == 'jsvmp_entry' for r in d['reasons']]
|
||||
assert all('opcode literal' not in r for r in reasons), reasons
|
||||
detections = score_families(body, '', 's.js', 200)
|
||||
assert detections and detections[0]['id'] == 'jsvmp_entry', detections
|
||||
assert any('opcode literal' in r for r in detections[0]['reasons'])
|
||||
|
||||
|
||||
def test_hint_beats_a_single_content_hit():
|
||||
# A human hint is worth 10; one content regex is worth 3. The routing consequence:
|
||||
# a named vendor hint wins over incidental structural overlap.
|
||||
detections = score_families('plain text', 'some obfuscator lineage', 'x.js', 1500)
|
||||
assert detections[0]['id'] == 'ob_string_array'
|
||||
assert detections[0]['score'] == 10
|
||||
|
||||
|
||||
def test_plain_code_is_generic_ladder():
|
||||
assert score_families('function add(a, b) { return a + b; }\n', '', 'x.js', 1500) == []
|
||||
assert pipeline_for('generic')[0] == 'normalize' and pipeline_for('generic')[-1] == 'normalize'
|
||||
|
||||
|
||||
def test_pipelines_differ_per_family():
|
||||
# The whole point of detection: the string-array lineage decodes FIRST, the
|
||||
# opcode-if-chain family goes straight to if-chain conversion.
|
||||
assert pipeline_for('ob_string_array')[2] == 'string_array_pass'
|
||||
assert 'if_chain_to_switch' in pipeline_for('opcode_if_chain')
|
||||
assert pipeline_for('ob_string_array') != pipeline_for('opcode_if_chain')
|
||||
|
||||
|
||||
def test_residue_metrics_are_textual_proxies():
|
||||
r = residue_metrics(FIXTURES['ob_string_array'])
|
||||
# The 3+ hex-char filter counts _0x4ab1 but not short names (_0x1, _0x9f).
|
||||
assert r['mangled_identifiers'] == 1
|
||||
assert r['eval_calls'] == 0
|
||||
assert r['lines'] >= 3
|
||||
@@ -0,0 +1,73 @@
|
||||
"""Pinned-shape tests for the Kotlin name recovery parser.
|
||||
|
||||
The shipped script (`skills/apk-reverse/scripts/kotlin_name_recovery.py`) is measured by
|
||||
its embedded fixtures; these tests pin the same shapes independently so a regex regression
|
||||
fails in CI rather than on a real jadx export. Every fixture here is a *shape*, written by
|
||||
this repository, not a fragment of any real target.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', 'skills',
|
||||
'apk-reverse', 'scripts'))
|
||||
|
||||
from kotlin_name_recovery import ( # noqa: E402
|
||||
FIXTURE_DEBUG, FIXTURE_METADATA, FIXTURE_PLAIN, FIXTURE_RENAMED, mine_tree,
|
||||
recover_real_name, write_map,
|
||||
)
|
||||
|
||||
|
||||
def test_debug_metadata_strips_dollar_scope():
|
||||
# `$fetch$1` is inner/lambda scope; the declaring class is everything before the first $.
|
||||
real, source = recover_real_name(FIXTURE_DEBUG)
|
||||
assert real == 'com.example.feature.account.AccountRepositoryImpl'
|
||||
assert source == 'debug_metadata'
|
||||
|
||||
|
||||
def test_metadata_d2_takes_first_non_stdlib_descriptor():
|
||||
real, source = recover_real_name(FIXTURE_METADATA)
|
||||
assert real == 'com.example.feature.login.LoginViewModel'
|
||||
assert source == 'metadata_d2'
|
||||
|
||||
|
||||
def test_renamed_comment_is_the_weakest_fallback():
|
||||
real, source = recover_real_name(FIXTURE_RENAMED)
|
||||
assert real == 'com.example.core.network.ApiClient'
|
||||
assert source == 'renamed_comment'
|
||||
|
||||
|
||||
def test_plain_java_recovers_nothing():
|
||||
# No annotation and no rename comment: the honest result is (None, None), not a guess.
|
||||
real, source = recover_real_name(FIXTURE_PLAIN)
|
||||
assert real is None and source is None
|
||||
|
||||
|
||||
def test_debug_metadata_outranks_everything():
|
||||
# A file carrying both sources must resolve through @DebugMetadata: it is a true
|
||||
# declaring-class FQN, while d2 descriptors are a hint.
|
||||
both = FIXTURE_DEBUG + '\n@Metadata(d2 = {"Lcom/example/other/WrongHint;"})\n'
|
||||
real, source = recover_real_name(both)
|
||||
assert real == 'com.example.feature.account.AccountRepositoryImpl'
|
||||
assert source == 'debug_metadata'
|
||||
|
||||
|
||||
def test_mine_tree_skips_third_party_and_maps_the_rest(tmp_path):
|
||||
src = tmp_path / 'sources'
|
||||
for rel, body in (('nq/a.java', FIXTURE_DEBUG),
|
||||
('z/C0123x.java', FIXTURE_RENAMED),
|
||||
('kotlin/x/Keep.java', FIXTURE_DEBUG)):
|
||||
path = src / rel
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_text(body, encoding='utf-8')
|
||||
mapping, counts = mine_tree(str(src))
|
||||
# kotlin.* is skipped as already-real even though its fixture carries @DebugMetadata.
|
||||
assert set(mapping) == {'nq.a', 'z.C0123x'}
|
||||
assert counts == {'debug_metadata': 1, 'renamed_comment': 1}
|
||||
|
||||
out = tmp_path / 'map'
|
||||
packages = write_map(mapping, counts, str(out))
|
||||
assert packages == 2
|
||||
tsv = (out / 'mapping.tsv').read_text(encoding='utf-8')
|
||||
assert 'nq.a\tcom.example.feature.account.AccountRepositoryImpl\tdebug_metadata' in tsv
|
||||
# by_package files are keyed by the ORIGINAL package, which is the lookup a reader wants.
|
||||
assert (out / 'by_package' / 'com_example_core_network.tsv').exists()
|
||||
Reference in New Issue
Block a user