mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-15 20:04:22 +08:00
Adopt dexsql as a third indexer, parity-checked against rasc, droidasc and ddc
LSPosed/dexsql exposes the whole dex as SQL tables (callers, field accesses,
numeric constants, debug lines, annotations, hiddenapi flags) plus the APK
manifest and resources.arsc, plus an MCP server. It is a third indexer at the
same stage as ASC, so it is registered the same way:
* scripts/capabilities.py gains tool:dexsql and the dex_query_sql capability,
so doctor.py reports it instead of guessing. INSTALL_HINTS replaces the
rasc-only hardcoded path list, so a release download or a from-source build
that landed outside PATH is not reported BLOCKED.
* references/dexsql-sql-query-layer.md carries the table model, the recipes,
the MCP tool list and the boundaries; references/toolchain.md gains the
Tier-1 row and a section; routing.md, SKILL.md, both READMEs and the
evidence record are updated in step.
What was measured, on four archives (1.4 MB / 1,396 classes, 4.3 MB / 647,
38.7 MB / 4 hardened, 34.8 MB / 30,768):
* class-definition sets identical to rasc, droidasc and ddc on all four,
zero differences in either direction;
* string cross-reference agrees with ASC site for site -- and the match
semantics differ, which is recorded as a trap (`=` is not `findrefs`);
* decode emits the same instruction sequence as ddc disasm, line for line;
* the persistent cache is NOT a speed switch: 0.121 s -> 0.098 s on a 34.8 MB
archive, a net loss on a small one. Recorded as measured, not as sold.
Full class listing, best of 3: dexsql 0.136 s against droidasc 0.200 s and
rasc 0.055 s on the 34.8 MB archive, so it does not replace rasc for a class
list -- it answers the joins and aggregates the other two cannot express.
This commit is contained in:
@@ -458,6 +458,7 @@ Linux; where a snippet is POSIX-only it is labelled. Nothing here assumes a Unix
|
||||
| Python 3.9+ | all scripts |
|
||||
| **`droidasc`** (ASC) (**optional but strongly recommended — install this first**) | whole-APK cross-reference index: `findrefs` / `listclass` / `getclass` / `getmanifest`. One `pip install droidasc`, no JVM, no SDK, no index build. Turns "which of N thousand classes mentions this string" into a sub-second query, and it is the only route to a class whose name R8 mangled. **This is the tool an agent should reach for before any full decompile** — see `skills/apk-reverse/references/toolchain.md` §droidasc (ASC) — ask an APK "who references this?", in one query |
|
||||
| `ddc` (optional but strongly recommended) | single-binary dex→Java decompiler with query subcommands (`info`, `findrefs`, `strings --with-locations`, per-class decompile). No JVM. **Reads** what ASC **locates**; also reports package identity reliably — see `skills/apk-reverse/references/toolchain.md` §ddc — dex-to-Java with query subcommands (worth adopting) |
|
||||
| **`dexsql`** (optional but strongly recommended) | single-binary **SQL over the whole dex**, plus the APK manifest and `resources.arsc`, plus an MCP server. Its class-definition set matched `rasc`/`droidasc`/`ddc` exactly on four archives, and it answers what the other two cannot express: callers of a callee, field accesses, numeric constants, debug lines, annotations, hiddenapi flags. Reach for it when the question is a **join or an aggregate** — see `skills/apk-reverse/references/dexsql-sql-query-layer.md` |
|
||||
| `baksmali` / `smali` + `dexlib2` jars | disassembly, assembly, surgical patching |
|
||||
| JDK (`javac`, `java`) | building/running the dexlib2 patcher; also provides `keytool`/`jarsigner` |
|
||||
| Android SDK build-tools (`aapt`, `zipalign`, `apksigner`) | manifest info, alignment, signing. **`apksigner` is the signer to use** — `jarsigner` rewrites the archive and breaks the alignment Android R+ requires |
|
||||
|
||||
@@ -375,6 +375,7 @@ export APKREV_TOOLS=<dir>:<dir> # POSIX
|
||||
| Python 3.9+ | 所有脚本 |
|
||||
| **`droidasc`**(ASC)(**可选,但强烈建议——先装这个**) | 整 APK 交叉引用索引:`findrefs` / `listclass` / `getclass` / `getmanifest`。一句 `pip install droidasc`,无 JVM、无 SDK、无需建索引。把「这几千个类里哪一个提到过这个字符串」变成亚秒级查询,也是找到名字被 R8 混淆过的类的唯一路线。**这是 agent 在做任何完整反编译之前应当伸手去拿的工具** —— 见 `skills/apk-reverse/references/toolchain.md` §droidasc (ASC) —— 一次查询就能问一个 APK「谁引用了这个?」 |
|
||||
| `ddc`(可选,但强烈建议) | 单文件 dex→Java 反编译器,带查询子命令(`info`、`findrefs`、`strings --with-locations`、按类反编译)。无 JVM。**读** ASC 所**定位**的东西;也能可靠地报告包身份 —— 见 `skills/apk-reverse/references/toolchain.md` §ddc —— 带查询子命令的 dex 转 Java(值得采用) |
|
||||
| **`dexsql`**(可选,但强烈建议) | 单文件,**用 SQL 查询整个 dex**,外加 APK manifest 与 `resources.arsc`,并内置 MCP server。它的类定义集合在四个归档上与 `rasc`/`droidasc`/`ddc` 完全一致,而且能回答另两个表达不了的问题:某被调方法的调用者、字段访问、数值常量、调试行表、注解、hiddenapi 标志。当问题是**连接或聚合**时用它 —— 见 `skills/apk-reverse/references/dexsql-sql-query-layer.md` |
|
||||
| `baksmali` / `smali` + `dexlib2` jars | 反汇编、汇编、外科式打补丁 |
|
||||
| JDK(`javac`、`java`) | 构建/运行 dexlib2 补丁器;同时提供 `keytool`/`jarsigner` |
|
||||
| Android SDK build-tools(`aapt`、`zipalign`、`apksigner`) | manifest 信息、对齐、签名。**`apksigner` 才是该用的签名器** —— `jarsigner` 会重写归档并破坏 Android R+ 所要求的对齐 |
|
||||
|
||||
@@ -0,0 +1,149 @@
|
||||
# Extension verification — dexsql, the SQL query layer (parity, decode, cache, MCP)
|
||||
|
||||
Covers the pass that adopted `dexsql` as a third indexer beside `rasc`/`droidasc`/`ddc` and
|
||||
registered it as the `dex_query_sql` capability. Strength labels as this directory's README defines
|
||||
them.
|
||||
|
||||
Reference environment: Windows 10.0.26100, PowerShell 7, Python 3.14.0, no device attached. The tool
|
||||
is a release download (`dexsql-windows-amd64.exe`, v0.1.0, 21,365,760 B, sha256
|
||||
`f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448`) — no build step, no Python, no
|
||||
JVM, no Android SDK.
|
||||
|
||||
**Headline: the class-definition parity check passes on all four archives — identical to `rasc`,
|
||||
`droidasc` and `ddc`, zero differences either way — and the string cross-reference agrees site for
|
||||
site. Three claims came back *smaller* than they could have been sold: the persistent cache is not a
|
||||
speed switch, `decode` emits smali rather than Java, and the SQL match semantics are not `findrefs`'s.
|
||||
The MCP surface was exercised to a tool listing, not end to end from a client.**
|
||||
|
||||
---
|
||||
|
||||
## 1. Class-definition parity — observed
|
||||
|
||||
The check that decides whether a new indexer may be trusted for location questions at all, and the
|
||||
same comparison `rasc_build.py --verify` performs between `rasc` and `droidasc`. Each tool prints
|
||||
class identity in its own shape, so each output is parsed per format and normalised to a DEX
|
||||
descriptor before comparing — a single regex over all four outputs measures the regex, not the tools
|
||||
(this pass's own first defect: an unescaped `$` in a descriptor pattern silently dropped classes whose
|
||||
names contain `$` or non-ASCII characters, and reported `droidasc` as 38 classes where it returns 647).
|
||||
|
||||
| Archive | dexsql | rasc | droidasc | ddc | verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| 1.4 MB MASTG challenge | 1,396 | 1,396 | 1,396 | 1,396 | identical |
|
||||
| 4.3 MB hardened | 647 | 647 | 647 | 647 | identical |
|
||||
| 38.7 MB hardened | 4 | 4 | 4 | 4 | identical |
|
||||
| 34.8 MB app | 30,768 | 30,768 | 30,768 | 30,768 | identical |
|
||||
|
||||
Zero differences in either direction on every row. The two small counts are the hardened targets, not
|
||||
the tools: the business dex is encrypted inside a shell and is in no DEX any of the four can read.
|
||||
|
||||
Full class listing, best of 3, stdout redirected to a file — the same shape for all four tools:
|
||||
|
||||
| Archive | dexsql | rasc | droidasc | ddc |
|
||||
|---|---|---|---|---|
|
||||
| 1.4 MB / 1,396 classes | 0.035 s | 0.028 s | 0.149 s | 0.109 s |
|
||||
| 4.3 MB / 647 classes | 0.031 s | 0.025 s | 0.139 s | 0.099 s |
|
||||
| 38.7 MB / 4 classes | 0.040 s | 0.021 s | 0.156 s | 0.092 s |
|
||||
| 34.8 MB / 30,768 classes | 0.136 s | 0.055 s | 0.200 s | 0.156 s |
|
||||
|
||||
The `rasc` and `droidasc` columns reproduce this repository's earlier figures for the same 34.8 MB
|
||||
archive (0.048 s / 0.190 s), which is what makes the new column comparable rather than merely new.
|
||||
`dexsql` is 1.5× faster than the Python indexer and 2.5× slower than the Rust one, so **it does not
|
||||
replace `rasc` for a plain class list** — it answers questions the other two cannot express.
|
||||
|
||||
## 2. Cross-reference parity, and the semantics that differ — observed
|
||||
|
||||
```
|
||||
$ dexsql query --input <1.4 MB challenge> --sql \
|
||||
"SELECT DISTINCT method_class, method_name FROM method_strings WHERE value LIKE '%http%'"
|
||||
Landroid/support/v4/content/res/TypedArrayUtils; | hasAttribute
|
||||
Landroid/support/v4/text/util/LinkifyCompat; | addLinks
|
||||
Landroid/support/v4/util/PatternsCompat; | <clinit>
|
||||
```
|
||||
|
||||
`rasc findrefs … string http` and `droidasc findrefs … string http` return exactly those three sites,
|
||||
and `ddc findrefs` agrees. **The matching semantics differ, and the difference is a trap worth
|
||||
recording:** `findrefs` matches a *substring*, while SQL needs `LIKE '%…%'`. `WHERE value = 'http'`
|
||||
returns **zero rows** and is not a broken query — no method in that archive holds a literal that *is*
|
||||
`http`. A first pass read that empty result as a tool defect; the control was the same query with
|
||||
`LIKE`, which reproduced ASC's answer exactly.
|
||||
|
||||
## 3. `decode` against `ddc disasm` — observed
|
||||
|
||||
```
|
||||
$ dexsql decode --input <challenge> --method 'Lsg/vantagepoint/uncrackable3/MainActivity;->verifyLibs()V'
|
||||
.method private verifyLibs()V
|
||||
.registers 14
|
||||
const-string v0, "UnCrackable3"
|
||||
new-instance v1, Ljava/util/HashMap;
|
||||
...
|
||||
```
|
||||
|
||||
The instruction sequence matches `ddc disasm <challenge> sg.vantagepoint.uncrackable3.MainActivity.verify`
|
||||
line for line on that method. `decode` emits **smali, not Java** — that is a feature for the
|
||||
bytecode-level decision `dex-patching.md` requires, and a limitation when the question is Java-level,
|
||||
where `ddc getclass` still owns it. A missing class exits 1 with `Error: not_found: selected file,
|
||||
method, or class not found`.
|
||||
|
||||
## 4. The persistent cache is not a speed switch — observed
|
||||
|
||||
```
|
||||
$ dexsql prepare-cache --input <34.8 MB app> --output cache # 0.196 s, writes cache.dxc2 34,481,119 B
|
||||
$ dexsql query --input <app> --full --sql "SELECT count(*) FROM method_calls" # 0.121 s
|
||||
$ dexsql query-cache --cache-file cache/cache.dxc2 --sql "SELECT count(*) FROM method_calls" # 0.098 s
|
||||
$ dexsql query --input <app> --full --sql "<top-20 GROUP BY over method_calls>" # 0.470 s
|
||||
$ dexsql query-cache --cache-file cache/cache.dxc2 --sql "<same aggregate>" # 0.458 s
|
||||
```
|
||||
|
||||
**Under 20%, not an order of magnitude**, because parse-and-build is already 0.12 s on that archive.
|
||||
On the 38.7 MB sample the cache is a net *loss* (0.071 s from cache against 0.042 s direct). The
|
||||
honest claim is "hold one dataset across many queries in a long session", not "queries get fast" —
|
||||
and the cache file is roughly the size of the source archive.
|
||||
|
||||
## 5. Error surface and read-only enforcement — observed
|
||||
|
||||
Every failure exits 1 with a named cause, and writes are refused:
|
||||
|
||||
| Input | Result |
|
||||
|---|---|
|
||||
| missing class | `Error: not_found: selected file, method, or class not found` |
|
||||
| missing file, or a non-dex file | `Error: load_failed at load_inputs: failed to prepare native input cache` |
|
||||
| unknown column | `Error: invalid_query at execute_sql: unknown column `nope`` |
|
||||
| unknown relation | `Error: invalid_query at execute_sql: unknown relation `nope`` |
|
||||
| `DELETE FROM classes` | `Error: SQL statement 1 failed` — the statement is rejected |
|
||||
|
||||
## 6. The MCP surface, exercised to its boundary — observed (listing), unverified (client)
|
||||
|
||||
`dexsql mcp` speaks JSON-RPC over stdio; an `initialize` + `tools/list` handshake returns 13 tools:
|
||||
`open_dataset`, `list_datasets`, `close_dataset`, `list_tables`, `describe_table`, `query`, `decode`,
|
||||
`get_manifest`, `get_manifest_components`, `list_resource_files`, `read_resource`, `query_resources`,
|
||||
plus an embedded English documentation catalog on `resources/list`. Every list-shaped result carries
|
||||
`next_offset`, and the resource tools read `resources.arsc` and archive entries without extracting
|
||||
them to disk. Quotas are explicit (`--max-datasets`, `--max-input-bytes`, `--max-dex-bytes`,
|
||||
`--max-output-bytes`, `--timeout-ms`), and the server's own instructions say a running timeout closes
|
||||
the affected dataset. **No client session was driven against it in this pass** — the tool list and the
|
||||
schemas are what is measured.
|
||||
|
||||
## 7. What this pass deliberately did not do
|
||||
|
||||
- No Linux or macOS asset was downloaded or run: the release publishes `linux-amd64`, `linux-arm64`
|
||||
and `darwin-arm64` binaries and none was exercised.
|
||||
- No MCP client session: the handshake and tool inventory were read, the tools were not called.
|
||||
- No patching through `dexsql`: it has no write path, and nothing here claims one.
|
||||
- No claim about a packed target beyond the parity row: on both hardened archives it returns the same
|
||||
class count as the other three tools, and the payload is not in any readable DEX.
|
||||
|
||||
## 8. Facts this pass pinned for the next one
|
||||
|
||||
- **A new indexer earns trust by the class-set comparison, not by its feature list.** The parity check
|
||||
is cheap (four tools, four archives, seconds) and it is the only evidence that separates "answers
|
||||
location questions correctly" from "parses most of the file". Do it before adopting the next one.
|
||||
- **Parse each tool's own output format.** The first pass compared four differently-shaped outputs
|
||||
with one regex and produced two false verdicts — one an unescaped `$`, one a dotted-vs-descriptor
|
||||
mismatch. A comparison harness is itself a tool that needs a control.
|
||||
- **`doctor.py` is the registration surface.** Adding `tool:dexsql` plus a `dex_query_sql` capability
|
||||
makes G2 report the tool instead of guessing, and `INSTALL_HINTS` is where a release download or a
|
||||
from-source build declares the location this kit's own install step produces, so a machine that
|
||||
followed the instructions is not reported BLOCKED.
|
||||
- **The three indexers now have a stated division of labour:** `rasc` locates fastest, `droidasc` is
|
||||
the Python cross-check, `dexsql` answers joins and aggregates (and carries the resource model).
|
||||
Writing that down is what keeps the next pass from re-deriving it.
|
||||
@@ -87,6 +87,7 @@ written down.
|
||||
| `EXTENSION-desensitization.md` | The leak scanner's rules and exemptions, its four `RESULT=` states and exit codes, the escaping false positives, and **the pass that found its own evidence file leaking 26 strong hits** — the fixture's literal match values, quoted to prove the rules fire |
|
||||
| `EXTENSION-dart-aot-formats.md` | Two string-table claims tested against a real dual-ABI Flutter app: the arm64 packed scheme confirmed against literal tag bytes, and the assumed armv7 `len*2`/UTF-16 form **refuted** — the 32-bit record is `[header u32][byte-count u32le][UTF-8]`, and the extractor's zero is a format mismatch rather than an empty table |
|
||||
| `EXTENSION-detection-pipeline.md` | Naming the check that fires: three arms on a public MASTG target (control alive, probe arm dead in ≈300 ms via `strstr("frida")`, with the `TracerPid=0` versus 4-frida-mappings asymmetry), `svc_scan.py`'s two decoders agreeing on an identical 214-site set, the layered Java-to-svc descent, and the ptrace-free dump boundary on targets whose dex is deflated inside the APK |
|
||||
| `EXTENSION-dexsql.md` | The SQL query layer adopted as a **third indexer** beside rasc/droidasc/ddc: identical class-definition sets on four archives (zero differences either way), the string xref agreement and the `=` versus `LIKE` match-semantics trap, `decode` against `ddc disasm` line for line, and the persistent cache measured as **not** a speed switch |
|
||||
|
||||
## Adding a run
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ invisible from the inside, because the weak route still produces output.
|
||||
|
||||
| Obligation | Do this | Cost of ignoring it |
|
||||
|---|---|---|
|
||||
| **Orient with an indexer, not an export** | Build the ability to *ask the artifact questions* before reading code: `droidasc findrefs` / `ddc findrefs` (string/type/method → every reference site, sub-second). A decompile reads a class you have **already located** — it is not how you locate it. `jadx` is a readable viewer of last resort, never the source of truth and never the entry point of a recon | hours of `grep` per question asked, over an export you paid for first |
|
||||
| **Orient with an indexer, not an export** | Build the ability to *ask the artifact questions* before reading code: `droidasc findrefs` / `ddc findrefs` (string/type/method → every reference site, sub-second), and `dexsql query` when the question is a **join or an aggregate** — callers of a callee, methods holding a numeric constant, the debug line table — rather than a class list. A decompile reads a class you have **already located** — it is not how you locate it. `jadx` is a readable viewer of last resort, never the source of truth and never the entry point of a recon | hours of `grep` per question asked, over an export you paid for first |
|
||||
| **Install the missing tool; it is part of the task** | A missing arm64 decompiler is the next step, not a constraint to route around. Ask a human only when installation is genuinely impossible | hours re-derived by hand for output a decompiler gives in minutes |
|
||||
| **Name the gap before spending against it** | State the capability the blocker requires and whether this machine has it, **out loud**. This is a G2 item | designing around a tool that installs in ten minutes |
|
||||
| **Reach for a shipped script before writing one** | Parsing, hashing, alignment and hot-plug probes are already written; a bespoke script in place of `scripts/dex_find_insn.py` is how offsets get **guessed instead of computed** | a wrong offset that decodes cleanly and behaves wrongly |
|
||||
|
||||
@@ -415,6 +415,23 @@
|
||||
"evidence": ["references/server-api.md", "references/handoff-boundaries.md"],
|
||||
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
|
||||
"limits": "Determining who owns a gate is covered; breaking an authorization the server performs is not. There is no client-side patch that reaches it, and the honest deliverable is the residual: which authority decides, and why the artifact cannot change it."
|
||||
},
|
||||
{
|
||||
"capability": "dex-sql-query-layer-dexsql",
|
||||
"group": "dex",
|
||||
"status": "ok",
|
||||
"route_measured": true,
|
||||
"strength": "observed",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/dexsql-sql-query-layer.md",
|
||||
"references/toolchain.md",
|
||||
"scripts/capabilities.py"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"docs/tool-verification/EXTENSION-dexsql.md (repository root, not shipped)"
|
||||
],
|
||||
"limits": "Measured on dexsql v0.1.0, Windows amd64 release asset only; no other version or platform was exercised. Class-definition sets were identical to rasc, droidasc and ddc on four archives (1,396 / 647 / 4 / 30,768 classes, zero differences either way), and a string cross-reference query reproduced ASC's hits site for site on one archive. Three boundaries are load-bearing: `decode` emits smali rather than Java, SQL `=` is not `findrefs` (ASC matches a substring, so `LIKE` is the equivalent), and the persistent cache is not a speed switch (under 20% on a 34.8 MB archive, a net loss on a small one). The MCP server was exercised to a tool listing and a resource-model inventory, not end to end from a client."
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -61,6 +61,12 @@
|
||||
{"name": "lxml", "version": "6.0.2", "strength": "observed"},
|
||||
{"name": "mitmproxy", "version": "12.2.3", "strength": "observed", "notes": "the library version"}
|
||||
],
|
||||
"dex_indexers": [
|
||||
{"name": "dexsql", "version": "0.1.0", "probe": "dexsql --version", "strength": "observed", "used_by": "the SQL query layer over dex and APK resources; registered as the dex_query_sql capability", "notes": "Windows amd64 release asset, 21,365,760 bytes, sha256 f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448, downloaded from the project's release host (no build step). Class-definition sets were compared against the three indexers below on four archives (1,396 / 647 / 4 / 30,768 classes): identical on all four, zero differences in either direction. Full measurements, recipes and boundaries: references/dexsql-sql-query-layer.md."},
|
||||
{"name": "rasc (Rust ASC)", "version": "0.1.0", "probe": "rasc --version", "strength": "observed", "notes": "built from source by scripts/rasc_build.py; the class-set comparison quoted above is the same one rasc_build.py --verify performs between rasc and droidasc"},
|
||||
{"name": "droidasc (Python ASC)", "version": "0.1.1.post1", "probe": "python -c \"import importlib.metadata as m; print(m.version('droidasc'))\"", "strength": "observed"},
|
||||
{"name": "ddc", "version": "0.1.8", "probe": "ddc -V", "strength": "observed", "notes": "single Rust binary, not on PATH here; the measured calls pass it by explicit path"}
|
||||
],
|
||||
"proxy": [
|
||||
{"name": "mitmdump (standalone binary in the local tools directory)", "version": "11.1.2 binary, bundling Python 3.13.1 and OpenSSL 3.4.0", "probe": "<binary> --version", "strength": "observed", "notes": "This is NOT the same version as the mitmproxy library above. A local workbench note recorded 12.2.3 against this binary's path; the binary reports 11.1.2. If a captured-flow shape matters, read the version from the binary you actually invoke."}
|
||||
],
|
||||
@@ -98,6 +104,7 @@
|
||||
"Kernel side: no kernel module was compiled or loaded at any version, on any kernel. The generator was exercised; the artefact was not.",
|
||||
"Gradle and the Android NDK: never used, at any version.",
|
||||
"Dart: the string-table formats were checked on two ABIs of one build (a Dart 3.6.0 snapshot), through a front end this skill does not ship.",
|
||||
"Mitmproxy: the library and the standalone binary measured different versions (12.2.3 and 11.1.2); no capture in the record is tied to a mitmproxy release."
|
||||
"Mitmproxy: the library and the standalone binary measured different versions (12.2.3 and 11.1.2); no capture in the record is tied to a mitmproxy release.",
|
||||
"dexsql: only 0.1.0 on Windows amd64 was measured. The same release publishes Linux and macOS assets, and none of them was exercised; no other dexsql version was tested at all. The MCP surface was exercised to a tool listing, not from a real client session."
|
||||
]
|
||||
}
|
||||
|
||||
@@ -32,6 +32,15 @@ but nobody here has paid for the counters yet.
|
||||
- **dex-level surgical patching**: equal-length byte edits and dexlib2 method rewrites, plus the
|
||||
header, verifier and alignment rules that decide whether the build loads (`dex-patching.md`,
|
||||
`byte-level-patching.md`, `patch-audit.md`).
|
||||
- **Whole-APK dex indexing and query — locating a class or a reference before reading code.** Three
|
||||
indexers ship as documented routes (`rasc`, `droidasc`, `dexsql`) and their class-definition sets
|
||||
were compared against each other and against `ddc` on four archives (1,396 / 647 / 4 / 30,768
|
||||
classes): **identical on all four, zero differences in either direction**, and a string
|
||||
cross-reference query agreed site for site. `dexsql` additionally exposes the dex as SQL — callers,
|
||||
field accesses, numeric constants, debug lines, annotations, hiddenapi flags — plus the APK
|
||||
resources and an MCP server (`rasc-and-droidsaw.md`, `dexsql-sql-query-layer.md`). **Boundary:** on a
|
||||
hardened target all four return the same small class count because the payload is encrypted inside a
|
||||
shell; no query language reaches that, and the unpacking routes own it.
|
||||
- **Repacking, signing, installing**, and the install refusals that look like a broken build
|
||||
(`repack-and-sign.md`).
|
||||
- **Packers, custom loaders and code virtualization**: identifying them, measuring the validation
|
||||
|
||||
@@ -18,6 +18,7 @@ the evidence record condensed in `references/evidence-summary.md` §Where the fu
|
||||
| Client-side ads, promos, splash/popup/tab config | includes server-issued UI config that has no SDK to find | `ad-removal.md`, `server-config-and-updates.md` |
|
||||
| Membership / paywall / feature gates | decide whether it is *client-enforceable* at all, and say so when it is not | `membership-and-limits.md`, `account-gates.md` |
|
||||
| dex-level surgical patching | equal-length byte edits, dexlib2 method rewrite, header/verifier/alignment rules | `dex-patching.md`, `byte-level-patching.md`, `patch-audit.md` |
|
||||
| Whole-APK dex indexing and query | locate classes and references **before** reading code: `rasc`/`droidasc` for `findrefs`, `dexsql` for SQL joins and aggregates (callers, field accesses, numeric constants, debug lines, annotations) — all four indexers cross-checked for identical class-definition sets on four archives | `rasc-and-droidsaw.md`, `dexsql-sql-query-layer.md` |
|
||||
| Repack, sign, install | including the install refusals that look like a broken build | `repack-and-sign.md` |
|
||||
| Packers, custom loaders, code virtualization | identification, the validation boundary, the routes that survive it | `packers.md`, `code-virtualization-and-custom-linkers.md` |
|
||||
| Java2C vs extraction shell vs JNI sinking | the misdiagnosis that sends you hunting a decrypted DEX that never exists | `java2c-and-jni-sinking.md`, `scripts/java2c_probe.py` |
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
# dexsql — the whole dex as SQL tables, and the APK's resources beside it
|
||||
|
||||
Load this when: you are about to answer a question about a dex by grepping an export, or you already
|
||||
use `droidasc`/`rasc` for `findrefs` and want the same answers as a relational query — or when you
|
||||
want the artifact's manifest and resources in the same query surface as its code.
|
||||
|
||||
**In one line:** a third indexer at the same stage as ASC, with the same class-definition answers
|
||||
(measured identical on four archives), slower than `rasc` on a plain class list and faster than
|
||||
`droidasc`, and one thing neither of the others has: the entire dex as SQL tables plus an MCP server.
|
||||
|
||||
## What it is
|
||||
|
||||
`LSPosed/dexsql` is a single self-contained binary (`dexsql-windows-amd64.exe`, 21.4 MB, v0.1.0). No
|
||||
Python, no JVM, no SDK, no first-run index build. It parses the dex once into a relational model and
|
||||
answers queries against it:
|
||||
|
||||
```bash
|
||||
dexsql tables --input app.apk # every table and column type
|
||||
dexsql query --input app.apk --sql "SELECT count(*) AS n FROM methods"
|
||||
dexsql query --input app.apk --interactive # keep the dataset loaded, read SQL from stdin
|
||||
dexsql decode --input app.apk --class 'Lcom/x/Y;' # smali for a class or one method
|
||||
dexsql prepare-cache --input app.apk --output cache # persistent cache.dxc2 for repeated queries
|
||||
dexsql query-cache --cache-file cache/cache.dxc2 --sql "SELECT ..."
|
||||
dexsql mcp # MCP server over stdio
|
||||
```
|
||||
|
||||
Acquire it the same way as `ddc` — it is a release download, not a build:
|
||||
|
||||
```bash
|
||||
# https://github.com/LSPosed/dexsql-release/releases -> dexsql-<platform>-<arch>
|
||||
```
|
||||
|
||||
**It is a registered capability.** `doctor.py` reports `dex_query_sql` as `OK` when the binary is on
|
||||
PATH, on `APKREV_TOOLS`, or at the location this kit's own install step uses (`tools/_work/bin/`,
|
||||
`DEXSQL`), and `BLOCKED` with the release URL as the next action when it is not.
|
||||
|
||||
## The three indexers, and which one to reach for
|
||||
|
||||
| | `rasc` | `droidasc` | `dexsql` |
|
||||
|---|---|---|---|
|
||||
| Answers class/reference questions | yes | yes | yes |
|
||||
| Plain class list, 34.8 MB app | **0.055 s** | 0.200 s | 0.136 s |
|
||||
| Reference query | `findrefs` (substring) | `findrefs` (substring) | SQL `WHERE value LIKE` |
|
||||
| Read one class | `getclass` (Java) | `getclass` (Java) | `decode` (**smali**, not Java) |
|
||||
| Relational model (callers, field accesses, numeric constants, debug lines, annotations, hiddenapi) | no | no | **yes** |
|
||||
| APK manifest / resources / `resources.arsc` as queries | manifest only | manifest only | **yes, as SQL + MCP tools** |
|
||||
| MCP server | no | no | **yes (13 tools)** |
|
||||
|
||||
**The division of labour is now three-way:** `rasc` locates fastest, `droidasc` is the Python
|
||||
cross-check, and `dexsql` answers the questions the other two cannot express — *which* methods call a
|
||||
callee, which method holds a numeric constant above a threshold, what the debug line table says, which
|
||||
annotations resolve on which owner. Reach for `dexsql` when the question is a **join or an
|
||||
aggregate**, and for `rasc` when the question is a **class list**.
|
||||
|
||||
**`decode` is not a decompiler.** It emits smali, and that is a feature: it is the bytecode-level
|
||||
answer, which is what `dex-patching.md` tells you to decide with when a decompiler's Java looks
|
||||
surprising.
|
||||
|
||||
## The table model — what a query can ask
|
||||
|
||||
~60 tables, grouped by what they answer (the full list is `dexsql tables`):
|
||||
|
||||
| Group | Tables worth knowing |
|
||||
|---|---|
|
||||
| Identity / structure | `dex_files` (per-dex header fields, `checksum`, `signature_hex`, counts), `classes`, `class_defs`, `class_interfaces`, `fields`, `methods`, `protos`, `type_lists` |
|
||||
| Code | `code_items` (`registers_size`, `ins_size`, `tries_size`), `method_opcodes`, `try_items`, `catch_handlers` |
|
||||
| **Cross-reference** | `method_calls` (caller → callee), `method_field_accesses`, `method_strings` (method → literal), `method_used_types`, `method_numbers` (numeric constants), `method_using_numbers`, `method_refs` |
|
||||
| Debug info | `debug_info`, `debug_positions` (address → line), `debug_locals`, `debug_parameters`, `debug_opcodes` |
|
||||
| Annotations | `annotations`, `annotation_elements`, `annotation_values`, `annotation_elements_resolved` (owner + annotation + element flattened) |
|
||||
| Runtime surface | `hiddenapi_flags`, `call_sites`, `method_handles` |
|
||||
|
||||
The cross-reference group is the part that replaces an hour of grepping: `method_calls`,
|
||||
`method_field_accesses`, `method_strings`, `method_used_types` and `method_numbers` are all
|
||||
pre-joined views carrying `method`, `method_class` and `method_name` on the calling side.
|
||||
|
||||
## Measured here
|
||||
|
||||
Environment: Windows 10.0.26100, `dexsql 0.1.0` (Windows amd64 release asset, sha256
|
||||
`f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448`), PowerShell 7. Four archives:
|
||||
a 1.4 MB MASTG challenge (1,396 classes), a 4.3 MB hardened app (647), a 38.7 MB hardened app (4) and
|
||||
a 34.8 MB app (30,768 classes / 268,446 methods).
|
||||
|
||||
**Class-definition sets — identical to all three other tools, on every archive.** This is the check
|
||||
that decides whether a new indexer may be trusted for location questions at all, and it is the same
|
||||
comparison `rasc_build.py --verify` performs between `rasc` and `droidasc`:
|
||||
|
||||
| Archive | dexsql | rasc | droidasc | ddc | verdict |
|
||||
|---|---|---|---|---|---|
|
||||
| 1.4 MB MASTG challenge | 1,396 | 1,396 | 1,396 | 1,396 | **identical** |
|
||||
| 4.3 MB hardened | 647 | 647 | 647 | 647 | **identical** |
|
||||
| 38.7 MB hardened | 4 | 4 | 4 | 4 | **identical** |
|
||||
| 34.8 MB app | 30,768 | 30,768 | 30,768 | 30,768 | **identical** |
|
||||
|
||||
Zero differences in either direction on all four — including the two archives where the count is
|
||||
small because the payload is encrypted inside a shell, which is the target and not the tool.
|
||||
|
||||
**Full class listing, best of 3, stdout to a file (the same shape for every tool):**
|
||||
|
||||
| Archive | dexsql | rasc | droidasc | ddc |
|
||||
|---|---|---|---|---|
|
||||
| 1.4 MB / 1,396 classes | 0.035 s | 0.028 s | 0.149 s | 0.109 s |
|
||||
| 4.3 MB / 647 classes | 0.031 s | 0.025 s | 0.139 s | 0.099 s |
|
||||
| 38.7 MB / 4 classes | 0.040 s | 0.021 s | 0.156 s | 0.092 s |
|
||||
| 34.8 MB / 30,768 classes | 0.136 s | **0.055 s** | 0.200 s | 0.156 s |
|
||||
|
||||
So on a real app `dexsql` is **1.5× faster than `droidasc`** and **2.5× slower than `rasc`**; the
|
||||
`rasc` and `droidasc` columns reproduce this repository's earlier figures (0.048 / 0.190 s on the
|
||||
same 34.8 MB archive), which is what makes the row comparable rather than merely new. A count-only
|
||||
query on the same archive is 0.056 s — the listing itself is most of the difference.
|
||||
|
||||
**String cross-reference agrees with ASC.** On the 1.4 MB challenge, `SELECT DISTINCT method_class,
|
||||
method_name FROM method_strings WHERE value LIKE '%http%'` returns exactly the three sites
|
||||
`rasc findrefs … string http` and `droidasc findrefs … string http` return, and `ddc findrefs` agrees.
|
||||
**The matching semantics differ and the difference is a trap:** `findrefs` matches a *substring*,
|
||||
while SQL needs `LIKE '%…%'` — `value = 'http'` legitimately returns zero rows, because no method
|
||||
holds a literal that *is* `http`.
|
||||
|
||||
**`decode` agrees with `ddc disasm`.** `dexsql decode --method
|
||||
'Lsg/vantagepoint/uncrackable3/MainActivity;->verifyLibs()V'` emits the same instruction sequence, in
|
||||
the same order, as `ddc disasm … MainActivity.verify` — checked line by line on that method.
|
||||
|
||||
**Persistent cache: a real feature with a small measured payoff here.** `prepare-cache` on the
|
||||
34.8 MB archive takes 0.196 s and writes a 34.5 MB `cache.dxc2`. Repeating a query that needs the
|
||||
full dataset — `method_calls` count, and a top-20 `GROUP BY` aggregate over it — went from 0.121 s to
|
||||
0.098 s and from 0.470 s to 0.458 s. That is **under 20%, not an order of magnitude**, because the
|
||||
parse-and-build path is already 0.12 s on this archive. Treat the cache as a way to hold one dataset
|
||||
across many queries in a long session, not as a speed switch; on small archives it is a net loss
|
||||
(cache read 0.071 s against a 0.042 s direct query on the 38.7 MB sample).
|
||||
|
||||
**Read-only is enforced, and the error surface is uniform.** Missing class → `Error: not_found:
|
||||
selected file, method, or class not found`, exit 1. Missing input, a non-dex input, an unknown column,
|
||||
an unknown relation and a `DELETE` all exit 1 with a named cause; the write is refused.
|
||||
|
||||
## Recipes worth keeping
|
||||
|
||||
```sql
|
||||
-- who calls what, ranked (the question `findrefs` cannot express)
|
||||
SELECT callee_class, callee_name, count(*) AS n FROM method_calls
|
||||
GROUP BY 1,2 ORDER BY n DESC LIMIT 20;
|
||||
|
||||
-- every literal a method holds, instead of one needle at a time
|
||||
SELECT method_class, method_name, value FROM method_strings
|
||||
WHERE method_class LIKE 'Lcom/target/%' ORDER BY method_class;
|
||||
|
||||
-- numeric constants above a threshold: config values, protocol magic, crypto parameters
|
||||
SELECT method_class, method_name, number_type, long_value FROM method_numbers
|
||||
WHERE long_value > 1000000 ORDER BY long_value DESC LIMIT 50;
|
||||
|
||||
-- methods with no code (native, abstract) versus the ones that carry it
|
||||
SELECT class_descriptor, name, method_kind, opcode_count FROM methods
|
||||
WHERE opcode_count IS NULL OR opcode_count = 0;
|
||||
|
||||
-- which owners carry a given annotation
|
||||
SELECT owner, annotation, count(*) AS n FROM annotation_elements_resolved
|
||||
GROUP BY 1,2 ORDER BY n DESC LIMIT 20;
|
||||
|
||||
-- debug line table size: a cheap signal for "was this dex built with debug info"
|
||||
SELECT count(*) AS positions FROM debug_positions;
|
||||
```
|
||||
|
||||
`--sql` repeats, so several statements run against one loaded dataset; `--cache name=SELECT …` defines
|
||||
a temporary table for the invocation; `--interactive` keeps the dataset loaded for a session. A
|
||||
`method_idx` is DEX-local, so on a multi-dex APK carry `file_id` (from `dex_files`) into the join —
|
||||
`-i` accepts several inputs and they share one dataset.
|
||||
|
||||
**The MCP surface is the same model for an agent.** `dexsql mcp` speaks JSON-RPC over stdio and
|
||||
publishes 13 tools — `open_dataset`, `list_tables`, `describe_table`, `query`, `decode`,
|
||||
`get_manifest`, `get_manifest_components`, `list_resource_files`, `read_resource`, `query_resources`,
|
||||
`list_datasets`, `close_dataset`, plus an embedded documentation catalog on `resources/list`. The
|
||||
resource tools read `resources.arsc` and archive entries **without extracting them to disk**, and
|
||||
every list-shaped result is paged by `next_offset`. Datasets are opened by path and are capped
|
||||
(`--max-datasets`, `--max-input-bytes`, `--timeout-ms`), so a client that forgets `close_dataset`
|
||||
degrades rather than grows.
|
||||
|
||||
## Boundaries — what this does not settle
|
||||
|
||||
- **A hit is a reference, not a call, and SQL does not change that.** `method_strings` proves a method
|
||||
holds a literal; whether that method runs on the path you care about is still a disassembly or
|
||||
runtime question (`dynamic-frida.md`).
|
||||
- **`decode` gives smali, not Java.** When the question is Java-level, `ddc getclass` or `jadx` is
|
||||
still the tool; `dexsql` is where you *locate* and where you check bytecode.
|
||||
- **One version, one platform family was measured.** v0.1.0 on Windows amd64. Linux/macOS assets exist
|
||||
in the same release but were not exercised here (`unverified`), and no other dexsql version was
|
||||
tested at all.
|
||||
- **The cache is not a speed switch** — see the measurement above; the honest claim is "hold a dataset
|
||||
across queries", not "queries get fast".
|
||||
- **`--trusted-input` skips input-consistency checks.** It exists for controlled build pipelines; do
|
||||
not reach for it to make a suspicious file load.
|
||||
- **A packed target stays packed.** On both hardened archives `dexsql` returns the same 4 and 647
|
||||
classes the other three tools return: the business dex is not in the file, and no query language
|
||||
changes that. Take the question to the unpacking routes.
|
||||
- **`quickened`/odex opcodes are unsupported** by `decode` (`odex` input is out of scope).
|
||||
- **This is not a patching tool.** Nothing here writes a dex; `dex-patching.md` and
|
||||
`byte-level-patching.md` own that, and `dexsql` is where you find the site to patch.
|
||||
|
||||
## Provenance
|
||||
|
||||
`https://github.com/LSPosed/dexsql-release` (release `v0.1.0`, published 2026-10-15; the repository
|
||||
is the release host — it carries binaries, not source), retrieved and measured 2026-10-16. Nothing is
|
||||
vendored into this repository: the binary lives in the gitignored `tools/` work area, and the
|
||||
measurements above name the version, the platform and the hashes they came from.
|
||||
@@ -31,6 +31,7 @@ next** — and points at the one section that says "the full record is elsewhere
|
||||
| Dex method-level rewrite (dexlib2) | Documented, not measured: the smali round-trip blind spot is real and a tree round trip can pass every table check and still fail at load | partial | unverified | `references/dex-patching.md`, `references/patch-audit.md` |
|
||||
| Dex header integrity and verifier legality | Recompute order proven on one written file; `dex_classdiff` passing is necessary, not sufficient — it cannot see code-item damage | ok | observed | `references/patch-audit.md`, `scripts/dexutil.py`, `scripts/dex_check_verifier.py` |
|
||||
| Dex string-constant patch | Equal-length only; the shipped script refuses unequal lengths, so a different-length edit is a different, unmeasured route | partial | unverified | `references/byte-level-patching.md`, `scripts/dex_strpatch.py` |
|
||||
| Whole-APK SQL query layer (dexsql) | Class-definition sets identical to rasc/droidasc/ddc on four archives, zero differences either way; a string xref query agrees site for site; `decode` emits smali, not Java, and the persistent cache is not a speed switch | ok | observed | `references/dexsql-sql-query-layer.md`, `references/toolchain.md` |
|
||||
| Repack, sign, install (single APK) | End to end: STORED and 4-byte-aligned `resources.arsc`, v1+v2+v3 true, install succeeded, on-device hash matched the local build, control build still fails the old way | ok | observed | `references/repack-and-sign.md`, `references/verification.md`, `scripts/repack.py` |
|
||||
| Split APK / App Bundle sets | Analyze, unified re-sign and merge measured on two real sets; merge correctly refuses a member carrying its own `resources.arsc` | partial | observed | `references/split-apk.md`, `scripts/repack.py` |
|
||||
| Third-party build audit | Not measured: the APK differ is on the list of scripts no pass has run | partial | unverified | `references/third-party-builds.md`, `scripts/apk_diff.py` |
|
||||
|
||||
@@ -112,6 +112,7 @@ _54 row(s) below the header._
|
||||
| `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, **deliverable-form drift (rooted-only vs shippable)**, bound-your-waits, **captures-you-never-looked-at**, **long-context decay**, handover |
|
||||
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
|
||||
| `references/rasc-and-droidsaw.md` | You are about to reach for the ASC indexer, or the Python one is the bottleneck: the Rust re-implementation, its measured speedup and **identical class sets**, the one class shape where it silently drops code, and how to build it |
|
||||
| `references/dexsql-sql-query-layer.md` | You are about to answer a dex question by grepping an export, or you need a **join or an aggregate** rather than a class list: the whole dex as SQL tables (callers, field accesses, numeric constants, debug lines, annotations, hiddenapi), manifest and `resources.arsc` queries, the MCP server, the measured standing against rasc/droidasc/ddc, and the two traps (`decode` is smali; `=` is not a substring match) |
|
||||
| `references/coverage-and-limits.md` | You need to weigh a claim before trusting it: the evidence behind each covered item, the dependencies this skill does not ship, and the record of what was never exercised |
|
||||
| `references/coverage/route-inventory.md` | You must answer "**can this skill actually do X**" in one pass: the per-route inventory — covered by verified mechanism, covered only by a documented route, the dependencies not shipped, and what is out of scope |
|
||||
| `references/evidence-summary.md` | You need to know **whether a route was verified and how strong the evidence is**, from inside an installed copy: capability → one-line conclusion → strength → where the evidence lives |
|
||||
|
||||
@@ -35,6 +35,7 @@ cross-check with the program headers.
|
||||
| **`rasc`** (Rust ASC) | `rasc findrefs app.apk string <S>` | **the same index, 4–15× faster**, no Python runtime, a Rust DEX decompiler behind `getclass`. Reach for this FIRST when it is built — its one blind spot is documented in `rasc-and-droidsaw.md` |
|
||||
| **`droidasc`** (ASC) | `droidasc findrefs app.apk string <S>` | the original Python index, one `pip install` away, and the cross-check for `rasc`. Reach for it FIRST when `rasc` is not built |
|
||||
| **`ddc`** | `ddc app.apk -c <Class>` | **fastest read of dex as Java, plus query subcommands** — reach for this SECOND, to *read*. See below |
|
||||
| **`dexsql`** | `dexsql query --input app.apk --sql "…"` | **the whole dex as SQL tables** (callers, field accesses, numeric constants, debug lines, annotations) plus manifest and `resources.arsc` queries and an MCP server. Reach for it when the question is a **join or an aggregate** rather than a class list — see below |
|
||||
| `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources |
|
||||
| `jadx` | `jadx --no-res -d <out> <apk>` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) |
|
||||
| `aapt2` | `aapt2 dump badging <apk>` | manifest facts, package name, versions |
|
||||
@@ -213,6 +214,48 @@ matters: a full `ddc <apk> -o out/` export is a reasonable thing to *have*, but
|
||||
not a substitute for one indexed query, and it is the slower of the two by a wide margin on any APK
|
||||
worth analyzing.
|
||||
|
||||
### dexsql — the whole dex as SQL, and the resources beside it
|
||||
|
||||
A third indexer at the same stage as ASC, and the one to reach for when the question is a **join or an
|
||||
aggregate** rather than a class list. One self-contained binary (v0.1.0, 21.4 MB on Windows amd64, no
|
||||
Python, no JVM, no index build), acquired as a release download from
|
||||
`https://github.com/LSPosed/dexsql-release/releases`:
|
||||
|
||||
```bash
|
||||
dexsql tables --input app.apk # every table and its column types
|
||||
dexsql query --input app.apk --sql "SELECT count(*) AS n FROM methods"
|
||||
dexsql query --input app.apk --sql "…" --sql "…" # several statements, one loaded dataset
|
||||
dexsql query --input app.apk --interactive # keep it loaded, read SQL from stdin
|
||||
dexsql decode --input app.apk --class 'Lcom/x/Y;' # smali, not Java
|
||||
dexsql prepare-cache --input app.apk --output cache # then: query-cache --cache-file cache/cache.dxc2
|
||||
dexsql mcp # MCP server over stdio
|
||||
```
|
||||
|
||||
**What it is for.** `method_calls` (caller → callee), `method_field_accesses`, `method_strings`,
|
||||
`method_used_types` and `method_numbers` are pre-joined views carrying the calling method's class and
|
||||
name — so "which methods call `X`", "which methods hold a numeric constant above N", "what does the
|
||||
debug line table say", "which owners carry this annotation" are single queries instead of greps.
|
||||
`hiddenapi_flags` and the annotation tables have no equivalent in the other two indexers.
|
||||
|
||||
**Where it sits against the other two, measured on four archives (identical class-definition sets on
|
||||
all four, zero differences either way):** on a 34.8 MB / 30,768-class app a full class listing takes
|
||||
`dexsql` **0.136 s** against `droidasc` 0.200 s and `rasc` 0.055 s — 1.5× faster than the Python
|
||||
indexer, 2.5× slower than the Rust one. **So it does not replace `rasc` for a class list**; it answers
|
||||
questions `rasc` cannot express. Its string cross-reference was checked against ASC's on a real
|
||||
archive and agrees site for site.
|
||||
|
||||
**Two traps.** `decode` emits **smali**, not Java — when the question is Java-level, `ddc getclass`
|
||||
still owns it. And SQL's `=` is not `findrefs`: ASC matches a *substring*, so `WHERE value = 'http'`
|
||||
legitimately returns nothing where `findrefs string http` returns three sites; use `LIKE '%…%'`.
|
||||
|
||||
**The persistent cache is not a speed switch.** `prepare-cache` on the 34.8 MB archive writes a
|
||||
34.5 MB `cache.dxc2` in 0.196 s, and repeating a full-dataset query against it moved 0.121 s → 0.098 s
|
||||
— **under 20%**, because parse-and-build is already 0.12 s there. It is a way to hold one dataset
|
||||
across many queries in a long session; on a small archive it is a net loss.
|
||||
|
||||
Full table inventory, recipes, the MCP tool list and the boundaries:
|
||||
`references/dexsql-sql-query-layer.md`.
|
||||
|
||||
**Classpath gotcha.** `baksmali`/`smali` need their dependency jars on the classpath together
|
||||
(`smali`, `antlr-runtime`, `stringtemplate`, `baksmali`, `dexlib2`, `util`, `jcommander`, `guava`).
|
||||
Missing one produces `ClassNotFoundException` that reads like a broken target. `scripts/smtool.py`
|
||||
|
||||
@@ -211,6 +211,17 @@ ATOMS = {
|
||||
'est_minutes': 1,
|
||||
'basis': 'measured',
|
||||
},
|
||||
'tool:dexsql': {
|
||||
'name': 'dexsql (SQL over dex and APK resources)',
|
||||
'kind': 'tool',
|
||||
'tool': 'dexsql',
|
||||
'install': 'download the release binary for this platform from '
|
||||
'https://github.com/LSPosed/dexsql-release/releases and put it on PATH '
|
||||
'or on APKREV_TOOLS (no Python, no JVM, no crate, no build step)',
|
||||
'est_minutes': 2,
|
||||
# Measured here: release download plus first query, on the Windows amd64 asset.
|
||||
'basis': 'observed',
|
||||
},
|
||||
'tool:apktool': {
|
||||
'name': 'apktool (resource round-trip)',
|
||||
'kind': 'tool',
|
||||
@@ -356,6 +367,19 @@ CAPABILITIES = {
|
||||
'note': 'one `pip install droidasc` away, and the cross-check for the Rust one: '
|
||||
'their class-definition sets are compared in scripts/rasc_build.py --verify',
|
||||
},
|
||||
'dex_query_sql': {
|
||||
'name': 'Whole-APK SQL query layer over dex and APK resources (dexsql)',
|
||||
'required': ['tool:dexsql'],
|
||||
'optional': [],
|
||||
'note': 'one self-contained binary: no Python, no JVM, no index build. Its '
|
||||
'class-definition set matched rasc, droidasc and ddc exactly on four archives '
|
||||
'(1,396 / 647 / 4 / 30,768 classes, zero differences either way). What it adds '
|
||||
'is the relational model -- method_calls, method_field_accesses, method_strings, '
|
||||
'method_numbers, debug_positions, hiddenapi_flags, annotation_elements_resolved '
|
||||
'-- plus APK resource queries and an MCP server. It is a third indexer at the '
|
||||
'same stage as ASC, not a replacement for it: rasc is still the fastest for a '
|
||||
'plain class list. See references/dexsql-sql-query-layer.md',
|
||||
},
|
||||
'static_dex': {
|
||||
'name': 'Static dex / zip / strings analysis',
|
||||
'required': ['python39'],
|
||||
@@ -535,6 +559,24 @@ def extra_tool_dirs():
|
||||
return [d for d in dirs if d and os.path.isdir(d)]
|
||||
|
||||
|
||||
# Tools whose *documented install step in this kit* lands somewhere PATH does not know about.
|
||||
# A machine that followed those instructions has the tool, and reporting it BLOCKED would make
|
||||
# the capability table lie in the pessimistic direction -- the same reasoning as APKREV_TOOLS,
|
||||
# applied to the tools acquired by a build or a release download rather than a package manager.
|
||||
INSTALL_HINTS = {
|
||||
'rasc': [
|
||||
os.path.join(REPO_ROOT, 'tools', '_work', 'rust', 'target', 'release',
|
||||
'rasc' + ('.exe' if os.name == 'nt' else '')),
|
||||
os.environ.get('RASC', ''),
|
||||
],
|
||||
'dexsql': [
|
||||
os.path.join(REPO_ROOT, 'tools', '_work', 'bin',
|
||||
'dexsql' + ('.exe' if os.name == 'nt' else '')),
|
||||
os.environ.get('DEXSQL', ''),
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
class Probe:
|
||||
"""Host facts, probed once per process and cached.
|
||||
|
||||
@@ -559,15 +601,12 @@ class Probe:
|
||||
if name in self._tool_cache:
|
||||
return self._tool_cache[name]
|
||||
hit = shutil.which(name)
|
||||
if not hit and name == 'rasc':
|
||||
# rasc ships no artifact, so a machine that followed this kit's own build step has it
|
||||
# under the repository's ignored work area rather than on PATH. Accepting that location
|
||||
if not hit:
|
||||
# The kit's own install step for this tool lands outside PATH (rasc is built into the
|
||||
# ignored work area, dexsql is a downloaded release binary). Accepting that location
|
||||
# keeps the capability table from reporting BLOCKED at an agent that already did the
|
||||
# work -- the same reasoning as APKREV_TOOLS, for the one tool whose install step is a
|
||||
# build (scripts/rasc_build.py).
|
||||
for cand in (os.path.join(REPO_ROOT, 'tools', '_work', 'rust', 'target', 'release',
|
||||
'rasc' + ('.exe' if os.name == 'nt' else '')),
|
||||
os.environ.get('RASC', '')):
|
||||
# work -- see INSTALL_HINTS.
|
||||
for cand in INSTALL_HINTS.get(name, []):
|
||||
if cand and os.path.isfile(cand):
|
||||
hit = cand
|
||||
break
|
||||
|
||||
Reference in New Issue
Block a user