Adopt dexsql as a third indexer, parity-checked against rasc, droidasc and ddc

LSPosed/dexsql exposes the whole dex as SQL tables (callers, field accesses,
numeric constants, debug lines, annotations, hiddenapi flags) plus the APK
manifest and resources.arsc, plus an MCP server. It is a third indexer at the
same stage as ASC, so it is registered the same way:

  * scripts/capabilities.py gains tool:dexsql and the dex_query_sql capability,
    so doctor.py reports it instead of guessing. INSTALL_HINTS replaces the
    rasc-only hardcoded path list, so a release download or a from-source build
    that landed outside PATH is not reported BLOCKED.
  * references/dexsql-sql-query-layer.md carries the table model, the recipes,
    the MCP tool list and the boundaries; references/toolchain.md gains the
    Tier-1 row and a section; routing.md, SKILL.md, both READMEs and the
    evidence record are updated in step.

What was measured, on four archives (1.4 MB / 1,396 classes, 4.3 MB / 647,
38.7 MB / 4 hardened, 34.8 MB / 30,768):

  * class-definition sets identical to rasc, droidasc and ddc on all four,
    zero differences in either direction;
  * string cross-reference agrees with ASC site for site -- and the match
    semantics differ, which is recorded as a trap (`=` is not `findrefs`);
  * decode emits the same instruction sequence as ddc disasm, line for line;
  * the persistent cache is NOT a speed switch: 0.121 s -> 0.098 s on a 34.8 MB
    archive, a net loss on a small one. Recorded as measured, not as sold.

Full class listing, best of 3: dexsql 0.136 s against droidasc 0.200 s and
rasc 0.055 s on the 34.8 MB archive, so it does not replace rasc for a class
list -- it answers the joins and aggregates the other two cannot express.
This commit is contained in:
inliver233
2026-10-16 02:01:36 +08:00
parent 7b6c6932a9
commit cbbeb58343
14 changed files with 481 additions and 10 deletions
+1
View File
@@ -458,6 +458,7 @@ Linux; where a snippet is POSIX-only it is labelled. Nothing here assumes a Unix
| Python 3.9+ | all scripts |
| **`droidasc`** (ASC) (**optional but strongly recommended — install this first**) | whole-APK cross-reference index: `findrefs` / `listclass` / `getclass` / `getmanifest`. One `pip install droidasc`, no JVM, no SDK, no index build. Turns "which of N thousand classes mentions this string" into a sub-second query, and it is the only route to a class whose name R8 mangled. **This is the tool an agent should reach for before any full decompile** — see `skills/apk-reverse/references/toolchain.md` §droidasc (ASC) — ask an APK "who references this?", in one query |
| `ddc` (optional but strongly recommended) | single-binary dex→Java decompiler with query subcommands (`info`, `findrefs`, `strings --with-locations`, per-class decompile). No JVM. **Reads** what ASC **locates**; also reports package identity reliably — see `skills/apk-reverse/references/toolchain.md` §ddc — dex-to-Java with query subcommands (worth adopting) |
| **`dexsql`** (optional but strongly recommended) | single-binary **SQL over the whole dex**, plus the APK manifest and `resources.arsc`, plus an MCP server. Its class-definition set matched `rasc`/`droidasc`/`ddc` exactly on four archives, and it answers what the other two cannot express: callers of a callee, field accesses, numeric constants, debug lines, annotations, hiddenapi flags. Reach for it when the question is a **join or an aggregate** — see `skills/apk-reverse/references/dexsql-sql-query-layer.md` |
| `baksmali` / `smali` + `dexlib2` jars | disassembly, assembly, surgical patching |
| JDK (`javac`, `java`) | building/running the dexlib2 patcher; also provides `keytool`/`jarsigner` |
| Android SDK build-tools (`aapt`, `zipalign`, `apksigner`) | manifest info, alignment, signing. **`apksigner` is the signer to use** — `jarsigner` rewrites the archive and breaks the alignment Android R+ requires |
+1
View File
@@ -375,6 +375,7 @@ export APKREV_TOOLS=<dir>:<dir> # POSIX
| Python 3.9+ | 所有脚本 |
| **`droidasc`**(ASC)(**可选,但强烈建议——先装这个**) | 整 APK 交叉引用索引:`findrefs` / `listclass` / `getclass` / `getmanifest`。一句 `pip install droidasc`,无 JVM、无 SDK、无需建索引。把「这几千个类里哪一个提到过这个字符串」变成亚秒级查询,也是找到名字被 R8 混淆过的类的唯一路线。**这是 agent 在做任何完整反编译之前应当伸手去拿的工具** —— 见 `skills/apk-reverse/references/toolchain.md` §droidasc (ASC) —— 一次查询就能问一个 APK「谁引用了这个?」 |
| `ddc`(可选,但强烈建议) | 单文件 dex→Java 反编译器,带查询子命令(`info`、`findrefs`、`strings --with-locations`、按类反编译)。无 JVM。**读** ASC 所**定位**的东西;也能可靠地报告包身份 —— 见 `skills/apk-reverse/references/toolchain.md` §ddc —— 带查询子命令的 dex 转 Java(值得采用) |
| **`dexsql`**(可选,但强烈建议) | 单文件,**用 SQL 查询整个 dex**,外加 APK manifest 与 `resources.arsc`,并内置 MCP server。它的类定义集合在四个归档上与 `rasc`/`droidasc`/`ddc` 完全一致,而且能回答另两个表达不了的问题:某被调方法的调用者、字段访问、数值常量、调试行表、注解、hiddenapi 标志。当问题是**连接或聚合**时用它 —— 见 `skills/apk-reverse/references/dexsql-sql-query-layer.md` |
| `baksmali` / `smali` + `dexlib2` jars | 反汇编、汇编、外科式打补丁 |
| JDK(`javac`、`java`) | 构建/运行 dexlib2 补丁器;同时提供 `keytool`/`jarsigner` |
| Android SDK build-tools(`aapt`、`zipalign`、`apksigner`) | manifest 信息、对齐、签名。**`apksigner` 才是该用的签名器** —— `jarsigner` 会重写归档并破坏 Android R+ 所要求的对齐 |
+149
View File
@@ -0,0 +1,149 @@
# Extension verification — dexsql, the SQL query layer (parity, decode, cache, MCP)
Covers the pass that adopted `dexsql` as a third indexer beside `rasc`/`droidasc`/`ddc` and
registered it as the `dex_query_sql` capability. Strength labels as this directory's README defines
them.
Reference environment: Windows 10.0.26100, PowerShell 7, Python 3.14.0, no device attached. The tool
is a release download (`dexsql-windows-amd64.exe`, v0.1.0, 21,365,760 B, sha256
`f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448`) — no build step, no Python, no
JVM, no Android SDK.
**Headline: the class-definition parity check passes on all four archives — identical to `rasc`,
`droidasc` and `ddc`, zero differences either way — and the string cross-reference agrees site for
site. Three claims came back *smaller* than they could have been sold: the persistent cache is not a
speed switch, `decode` emits smali rather than Java, and the SQL match semantics are not `findrefs`'s.
The MCP surface was exercised to a tool listing, not end to end from a client.**
---
## 1. Class-definition parity — observed
The check that decides whether a new indexer may be trusted for location questions at all, and the
same comparison `rasc_build.py --verify` performs between `rasc` and `droidasc`. Each tool prints
class identity in its own shape, so each output is parsed per format and normalised to a DEX
descriptor before comparing — a single regex over all four outputs measures the regex, not the tools
(this pass's own first defect: an unescaped `$` in a descriptor pattern silently dropped classes whose
names contain `$` or non-ASCII characters, and reported `droidasc` as 38 classes where it returns 647).
| Archive | dexsql | rasc | droidasc | ddc | verdict |
|---|---|---|---|---|---|
| 1.4 MB MASTG challenge | 1,396 | 1,396 | 1,396 | 1,396 | identical |
| 4.3 MB hardened | 647 | 647 | 647 | 647 | identical |
| 38.7 MB hardened | 4 | 4 | 4 | 4 | identical |
| 34.8 MB app | 30,768 | 30,768 | 30,768 | 30,768 | identical |
Zero differences in either direction on every row. The two small counts are the hardened targets, not
the tools: the business dex is encrypted inside a shell and is in no DEX any of the four can read.
Full class listing, best of 3, stdout redirected to a file — the same shape for all four tools:
| Archive | dexsql | rasc | droidasc | ddc |
|---|---|---|---|---|
| 1.4 MB / 1,396 classes | 0.035 s | 0.028 s | 0.149 s | 0.109 s |
| 4.3 MB / 647 classes | 0.031 s | 0.025 s | 0.139 s | 0.099 s |
| 38.7 MB / 4 classes | 0.040 s | 0.021 s | 0.156 s | 0.092 s |
| 34.8 MB / 30,768 classes | 0.136 s | 0.055 s | 0.200 s | 0.156 s |
The `rasc` and `droidasc` columns reproduce this repository's earlier figures for the same 34.8 MB
archive (0.048 s / 0.190 s), which is what makes the new column comparable rather than merely new.
`dexsql` is 1.5× faster than the Python indexer and 2.5× slower than the Rust one, so **it does not
replace `rasc` for a plain class list** — it answers questions the other two cannot express.
## 2. Cross-reference parity, and the semantics that differ — observed
```
$ dexsql query --input <1.4 MB challenge> --sql \
"SELECT DISTINCT method_class, method_name FROM method_strings WHERE value LIKE '%http%'"
Landroid/support/v4/content/res/TypedArrayUtils; | hasAttribute
Landroid/support/v4/text/util/LinkifyCompat; | addLinks
Landroid/support/v4/util/PatternsCompat; | <clinit>
```
`rasc findrefs … string http` and `droidasc findrefs … string http` return exactly those three sites,
and `ddc findrefs` agrees. **The matching semantics differ, and the difference is a trap worth
recording:** `findrefs` matches a *substring*, while SQL needs `LIKE '%…%'`. `WHERE value = 'http'`
returns **zero rows** and is not a broken query — no method in that archive holds a literal that *is*
`http`. A first pass read that empty result as a tool defect; the control was the same query with
`LIKE`, which reproduced ASC's answer exactly.
## 3. `decode` against `ddc disasm` — observed
```
$ dexsql decode --input <challenge> --method 'Lsg/vantagepoint/uncrackable3/MainActivity;->verifyLibs()V'
.method private verifyLibs()V
.registers 14
const-string v0, "UnCrackable3"
new-instance v1, Ljava/util/HashMap;
...
```
The instruction sequence matches `ddc disasm <challenge> sg.vantagepoint.uncrackable3.MainActivity.verify`
line for line on that method. `decode` emits **smali, not Java** — that is a feature for the
bytecode-level decision `dex-patching.md` requires, and a limitation when the question is Java-level,
where `ddc getclass` still owns it. A missing class exits 1 with `Error: not_found: selected file,
method, or class not found`.
## 4. The persistent cache is not a speed switch — observed
```
$ dexsql prepare-cache --input <34.8 MB app> --output cache # 0.196 s, writes cache.dxc2 34,481,119 B
$ dexsql query --input <app> --full --sql "SELECT count(*) FROM method_calls" # 0.121 s
$ dexsql query-cache --cache-file cache/cache.dxc2 --sql "SELECT count(*) FROM method_calls" # 0.098 s
$ dexsql query --input <app> --full --sql "<top-20 GROUP BY over method_calls>" # 0.470 s
$ dexsql query-cache --cache-file cache/cache.dxc2 --sql "<same aggregate>" # 0.458 s
```
**Under 20%, not an order of magnitude**, because parse-and-build is already 0.12 s on that archive.
On the 38.7 MB sample the cache is a net *loss* (0.071 s from cache against 0.042 s direct). The
honest claim is "hold one dataset across many queries in a long session", not "queries get fast" —
and the cache file is roughly the size of the source archive.
## 5. Error surface and read-only enforcement — observed
Every failure exits 1 with a named cause, and writes are refused:
| Input | Result |
|---|---|
| missing class | `Error: not_found: selected file, method, or class not found` |
| missing file, or a non-dex file | `Error: load_failed at load_inputs: failed to prepare native input cache` |
| unknown column | `Error: invalid_query at execute_sql: unknown column `nope`` |
| unknown relation | `Error: invalid_query at execute_sql: unknown relation `nope`` |
| `DELETE FROM classes` | `Error: SQL statement 1 failed` — the statement is rejected |
## 6. The MCP surface, exercised to its boundary — observed (listing), unverified (client)
`dexsql mcp` speaks JSON-RPC over stdio; an `initialize` + `tools/list` handshake returns 13 tools:
`open_dataset`, `list_datasets`, `close_dataset`, `list_tables`, `describe_table`, `query`, `decode`,
`get_manifest`, `get_manifest_components`, `list_resource_files`, `read_resource`, `query_resources`,
plus an embedded English documentation catalog on `resources/list`. Every list-shaped result carries
`next_offset`, and the resource tools read `resources.arsc` and archive entries without extracting
them to disk. Quotas are explicit (`--max-datasets`, `--max-input-bytes`, `--max-dex-bytes`,
`--max-output-bytes`, `--timeout-ms`), and the server's own instructions say a running timeout closes
the affected dataset. **No client session was driven against it in this pass** — the tool list and the
schemas are what is measured.
## 7. What this pass deliberately did not do
- No Linux or macOS asset was downloaded or run: the release publishes `linux-amd64`, `linux-arm64`
and `darwin-arm64` binaries and none was exercised.
- No MCP client session: the handshake and tool inventory were read, the tools were not called.
- No patching through `dexsql`: it has no write path, and nothing here claims one.
- No claim about a packed target beyond the parity row: on both hardened archives it returns the same
class count as the other three tools, and the payload is not in any readable DEX.
## 8. Facts this pass pinned for the next one
- **A new indexer earns trust by the class-set comparison, not by its feature list.** The parity check
is cheap (four tools, four archives, seconds) and it is the only evidence that separates "answers
location questions correctly" from "parses most of the file". Do it before adopting the next one.
- **Parse each tool's own output format.** The first pass compared four differently-shaped outputs
with one regex and produced two false verdicts — one an unescaped `$`, one a dotted-vs-descriptor
mismatch. A comparison harness is itself a tool that needs a control.
- **`doctor.py` is the registration surface.** Adding `tool:dexsql` plus a `dex_query_sql` capability
makes G2 report the tool instead of guessing, and `INSTALL_HINTS` is where a release download or a
from-source build declares the location this kit's own install step produces, so a machine that
followed the instructions is not reported BLOCKED.
- **The three indexers now have a stated division of labour:** `rasc` locates fastest, `droidasc` is
the Python cross-check, `dexsql` answers joins and aggregates (and carries the resource model).
Writing that down is what keeps the next pass from re-deriving it.
+1
View File
@@ -87,6 +87,7 @@ written down.
| `EXTENSION-desensitization.md` | The leak scanner's rules and exemptions, its four `RESULT=` states and exit codes, the escaping false positives, and **the pass that found its own evidence file leaking 26 strong hits** — the fixture's literal match values, quoted to prove the rules fire |
| `EXTENSION-dart-aot-formats.md` | Two string-table claims tested against a real dual-ABI Flutter app: the arm64 packed scheme confirmed against literal tag bytes, and the assumed armv7 `len*2`/UTF-16 form **refuted** — the 32-bit record is `[header u32][byte-count u32le][UTF-8]`, and the extractor's zero is a format mismatch rather than an empty table |
| `EXTENSION-detection-pipeline.md` | Naming the check that fires: three arms on a public MASTG target (control alive, probe arm dead in ≈300 ms via `strstr("frida")`, with the `TracerPid=0` versus 4-frida-mappings asymmetry), `svc_scan.py`'s two decoders agreeing on an identical 214-site set, the layered Java-to-svc descent, and the ptrace-free dump boundary on targets whose dex is deflated inside the APK |
| `EXTENSION-dexsql.md` | The SQL query layer adopted as a **third indexer** beside rasc/droidasc/ddc: identical class-definition sets on four archives (zero differences either way), the string xref agreement and the `=` versus `LIKE` match-semantics trap, `decode` against `ddc disasm` line for line, and the persistent cache measured as **not** a speed switch |
## Adding a run
+1 -1
View File
@@ -81,7 +81,7 @@ invisible from the inside, because the weak route still produces output.
| Obligation | Do this | Cost of ignoring it |
|---|---|---|
| **Orient with an indexer, not an export** | Build the ability to *ask the artifact questions* before reading code: `droidasc findrefs` / `ddc findrefs` (string/type/method → every reference site, sub-second). A decompile reads a class you have **already located** — it is not how you locate it. `jadx` is a readable viewer of last resort, never the source of truth and never the entry point of a recon | hours of `grep` per question asked, over an export you paid for first |
| **Orient with an indexer, not an export** | Build the ability to *ask the artifact questions* before reading code: `droidasc findrefs` / `ddc findrefs` (string/type/method → every reference site, sub-second), and `dexsql query` when the question is a **join or an aggregate** — callers of a callee, methods holding a numeric constant, the debug line table — rather than a class list. A decompile reads a class you have **already located** — it is not how you locate it. `jadx` is a readable viewer of last resort, never the source of truth and never the entry point of a recon | hours of `grep` per question asked, over an export you paid for first |
| **Install the missing tool; it is part of the task** | A missing arm64 decompiler is the next step, not a constraint to route around. Ask a human only when installation is genuinely impossible | hours re-derived by hand for output a decompiler gives in minutes |
| **Name the gap before spending against it** | State the capability the blocker requires and whether this machine has it, **out loud**. This is a G2 item | designing around a tool that installs in ten minutes |
| **Reach for a shipped script before writing one** | Parsing, hashing, alignment and hot-plug probes are already written; a bespoke script in place of `scripts/dex_find_insn.py` is how offsets get **guessed instead of computed** | a wrong offset that decodes cleanly and behaves wrongly |
@@ -415,6 +415,23 @@
"evidence": ["references/server-api.md", "references/handoff-boundaries.md"],
"evidence_not_shipped": ["docs/tool-verification/README.md (repository root, not shipped)"],
"limits": "Determining who owns a gate is covered; breaking an authorization the server performs is not. There is no client-side patch that reaches it, and the honest deliverable is the residual: which authority decides, and why the artifact cannot change it."
},
{
"capability": "dex-sql-query-layer-dexsql",
"group": "dex",
"status": "ok",
"route_measured": true,
"strength": "observed",
"benchmark_rows": [],
"evidence": [
"references/dexsql-sql-query-layer.md",
"references/toolchain.md",
"scripts/capabilities.py"
],
"evidence_not_shipped": [
"docs/tool-verification/EXTENSION-dexsql.md (repository root, not shipped)"
],
"limits": "Measured on dexsql v0.1.0, Windows amd64 release asset only; no other version or platform was exercised. Class-definition sets were identical to rasc, droidasc and ddc on four archives (1,396 / 647 / 4 / 30,768 classes, zero differences either way), and a string cross-reference query reproduced ASC's hits site for site on one archive. Three boundaries are load-bearing: `decode` emits smali rather than Java, SQL `=` is not `findrefs` (ASC matches a substring, so `LIKE` is the equivalent), and the persistent cache is not a speed switch (under 20% on a 34.8 MB archive, a net loss on a small one). The MCP server was exercised to a tool listing and a resource-model inventory, not end to end from a client."
}
]
}
@@ -61,6 +61,12 @@
{"name": "lxml", "version": "6.0.2", "strength": "observed"},
{"name": "mitmproxy", "version": "12.2.3", "strength": "observed", "notes": "the library version"}
],
"dex_indexers": [
{"name": "dexsql", "version": "0.1.0", "probe": "dexsql --version", "strength": "observed", "used_by": "the SQL query layer over dex and APK resources; registered as the dex_query_sql capability", "notes": "Windows amd64 release asset, 21,365,760 bytes, sha256 f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448, downloaded from the project's release host (no build step). Class-definition sets were compared against the three indexers below on four archives (1,396 / 647 / 4 / 30,768 classes): identical on all four, zero differences in either direction. Full measurements, recipes and boundaries: references/dexsql-sql-query-layer.md."},
{"name": "rasc (Rust ASC)", "version": "0.1.0", "probe": "rasc --version", "strength": "observed", "notes": "built from source by scripts/rasc_build.py; the class-set comparison quoted above is the same one rasc_build.py --verify performs between rasc and droidasc"},
{"name": "droidasc (Python ASC)", "version": "0.1.1.post1", "probe": "python -c \"import importlib.metadata as m; print(m.version('droidasc'))\"", "strength": "observed"},
{"name": "ddc", "version": "0.1.8", "probe": "ddc -V", "strength": "observed", "notes": "single Rust binary, not on PATH here; the measured calls pass it by explicit path"}
],
"proxy": [
{"name": "mitmdump (standalone binary in the local tools directory)", "version": "11.1.2 binary, bundling Python 3.13.1 and OpenSSL 3.4.0", "probe": "<binary> --version", "strength": "observed", "notes": "This is NOT the same version as the mitmproxy library above. A local workbench note recorded 12.2.3 against this binary's path; the binary reports 11.1.2. If a captured-flow shape matters, read the version from the binary you actually invoke."}
],
@@ -98,6 +104,7 @@
"Kernel side: no kernel module was compiled or loaded at any version, on any kernel. The generator was exercised; the artefact was not.",
"Gradle and the Android NDK: never used, at any version.",
"Dart: the string-table formats were checked on two ABIs of one build (a Dart 3.6.0 snapshot), through a front end this skill does not ship.",
"Mitmproxy: the library and the standalone binary measured different versions (12.2.3 and 11.1.2); no capture in the record is tied to a mitmproxy release."
"Mitmproxy: the library and the standalone binary measured different versions (12.2.3 and 11.1.2); no capture in the record is tied to a mitmproxy release.",
"dexsql: only 0.1.0 on Windows amd64 was measured. The same release publishes Linux and macOS assets, and none of them was exercised; no other dexsql version was tested at all. The MCP surface was exercised to a tool listing, not from a real client session."
]
}
@@ -32,6 +32,15 @@ but nobody here has paid for the counters yet.
- **dex-level surgical patching**: equal-length byte edits and dexlib2 method rewrites, plus the
header, verifier and alignment rules that decide whether the build loads (`dex-patching.md`,
`byte-level-patching.md`, `patch-audit.md`).
- **Whole-APK dex indexing and query — locating a class or a reference before reading code.** Three
indexers ship as documented routes (`rasc`, `droidasc`, `dexsql`) and their class-definition sets
were compared against each other and against `ddc` on four archives (1,396 / 647 / 4 / 30,768
classes): **identical on all four, zero differences in either direction**, and a string
cross-reference query agreed site for site. `dexsql` additionally exposes the dex as SQL — callers,
field accesses, numeric constants, debug lines, annotations, hiddenapi flags — plus the APK
resources and an MCP server (`rasc-and-droidsaw.md`, `dexsql-sql-query-layer.md`). **Boundary:** on a
hardened target all four return the same small class count because the payload is encrypted inside a
shell; no query language reaches that, and the unpacking routes own it.
- **Repacking, signing, installing**, and the install refusals that look like a broken build
(`repack-and-sign.md`).
- **Packers, custom loaders and code virtualization**: identifying them, measuring the validation
@@ -18,6 +18,7 @@ the evidence record condensed in `references/evidence-summary.md` §Where the fu
| Client-side ads, promos, splash/popup/tab config | includes server-issued UI config that has no SDK to find | `ad-removal.md`, `server-config-and-updates.md` |
| Membership / paywall / feature gates | decide whether it is *client-enforceable* at all, and say so when it is not | `membership-and-limits.md`, `account-gates.md` |
| dex-level surgical patching | equal-length byte edits, dexlib2 method rewrite, header/verifier/alignment rules | `dex-patching.md`, `byte-level-patching.md`, `patch-audit.md` |
| Whole-APK dex indexing and query | locate classes and references **before** reading code: `rasc`/`droidasc` for `findrefs`, `dexsql` for SQL joins and aggregates (callers, field accesses, numeric constants, debug lines, annotations) — all four indexers cross-checked for identical class-definition sets on four archives | `rasc-and-droidsaw.md`, `dexsql-sql-query-layer.md` |
| Repack, sign, install | including the install refusals that look like a broken build | `repack-and-sign.md` |
| Packers, custom loaders, code virtualization | identification, the validation boundary, the routes that survive it | `packers.md`, `code-virtualization-and-custom-linkers.md` |
| Java2C vs extraction shell vs JNI sinking | the misdiagnosis that sends you hunting a decrypted DEX that never exists | `java2c-and-jni-sinking.md`, `scripts/java2c_probe.py` |
@@ -0,0 +1,201 @@
# dexsql — the whole dex as SQL tables, and the APK's resources beside it
Load this when: you are about to answer a question about a dex by grepping an export, or you already
use `droidasc`/`rasc` for `findrefs` and want the same answers as a relational query — or when you
want the artifact's manifest and resources in the same query surface as its code.
**In one line:** a third indexer at the same stage as ASC, with the same class-definition answers
(measured identical on four archives), slower than `rasc` on a plain class list and faster than
`droidasc`, and one thing neither of the others has: the entire dex as SQL tables plus an MCP server.
## What it is
`LSPosed/dexsql` is a single self-contained binary (`dexsql-windows-amd64.exe`, 21.4 MB, v0.1.0). No
Python, no JVM, no SDK, no first-run index build. It parses the dex once into a relational model and
answers queries against it:
```bash
dexsql tables --input app.apk # every table and column type
dexsql query --input app.apk --sql "SELECT count(*) AS n FROM methods"
dexsql query --input app.apk --interactive # keep the dataset loaded, read SQL from stdin
dexsql decode --input app.apk --class 'Lcom/x/Y;' # smali for a class or one method
dexsql prepare-cache --input app.apk --output cache # persistent cache.dxc2 for repeated queries
dexsql query-cache --cache-file cache/cache.dxc2 --sql "SELECT ..."
dexsql mcp # MCP server over stdio
```
Acquire it the same way as `ddc` — it is a release download, not a build:
```bash
# https://github.com/LSPosed/dexsql-release/releases -> dexsql-<platform>-<arch>
```
**It is a registered capability.** `doctor.py` reports `dex_query_sql` as `OK` when the binary is on
PATH, on `APKREV_TOOLS`, or at the location this kit's own install step uses (`tools/_work/bin/`,
`DEXSQL`), and `BLOCKED` with the release URL as the next action when it is not.
## The three indexers, and which one to reach for
| | `rasc` | `droidasc` | `dexsql` |
|---|---|---|---|
| Answers class/reference questions | yes | yes | yes |
| Plain class list, 34.8 MB app | **0.055 s** | 0.200 s | 0.136 s |
| Reference query | `findrefs` (substring) | `findrefs` (substring) | SQL `WHERE value LIKE` |
| Read one class | `getclass` (Java) | `getclass` (Java) | `decode` (**smali**, not Java) |
| Relational model (callers, field accesses, numeric constants, debug lines, annotations, hiddenapi) | no | no | **yes** |
| APK manifest / resources / `resources.arsc` as queries | manifest only | manifest only | **yes, as SQL + MCP tools** |
| MCP server | no | no | **yes (13 tools)** |
**The division of labour is now three-way:** `rasc` locates fastest, `droidasc` is the Python
cross-check, and `dexsql` answers the questions the other two cannot express — *which* methods call a
callee, which method holds a numeric constant above a threshold, what the debug line table says, which
annotations resolve on which owner. Reach for `dexsql` when the question is a **join or an
aggregate**, and for `rasc` when the question is a **class list**.
**`decode` is not a decompiler.** It emits smali, and that is a feature: it is the bytecode-level
answer, which is what `dex-patching.md` tells you to decide with when a decompiler's Java looks
surprising.
## The table model — what a query can ask
~60 tables, grouped by what they answer (the full list is `dexsql tables`):
| Group | Tables worth knowing |
|---|---|
| Identity / structure | `dex_files` (per-dex header fields, `checksum`, `signature_hex`, counts), `classes`, `class_defs`, `class_interfaces`, `fields`, `methods`, `protos`, `type_lists` |
| Code | `code_items` (`registers_size`, `ins_size`, `tries_size`), `method_opcodes`, `try_items`, `catch_handlers` |
| **Cross-reference** | `method_calls` (caller → callee), `method_field_accesses`, `method_strings` (method → literal), `method_used_types`, `method_numbers` (numeric constants), `method_using_numbers`, `method_refs` |
| Debug info | `debug_info`, `debug_positions` (address → line), `debug_locals`, `debug_parameters`, `debug_opcodes` |
| Annotations | `annotations`, `annotation_elements`, `annotation_values`, `annotation_elements_resolved` (owner + annotation + element flattened) |
| Runtime surface | `hiddenapi_flags`, `call_sites`, `method_handles` |
The cross-reference group is the part that replaces an hour of grepping: `method_calls`,
`method_field_accesses`, `method_strings`, `method_used_types` and `method_numbers` are all
pre-joined views carrying `method`, `method_class` and `method_name` on the calling side.
## Measured here
Environment: Windows 10.0.26100, `dexsql 0.1.0` (Windows amd64 release asset, sha256
`f354ebcf715de9a66ef8ebe08b2b36807ab58e25cd5a1c0812e497fc1fcfc448`), PowerShell 7. Four archives:
a 1.4 MB MASTG challenge (1,396 classes), a 4.3 MB hardened app (647), a 38.7 MB hardened app (4) and
a 34.8 MB app (30,768 classes / 268,446 methods).
**Class-definition sets — identical to all three other tools, on every archive.** This is the check
that decides whether a new indexer may be trusted for location questions at all, and it is the same
comparison `rasc_build.py --verify` performs between `rasc` and `droidasc`:
| Archive | dexsql | rasc | droidasc | ddc | verdict |
|---|---|---|---|---|---|
| 1.4 MB MASTG challenge | 1,396 | 1,396 | 1,396 | 1,396 | **identical** |
| 4.3 MB hardened | 647 | 647 | 647 | 647 | **identical** |
| 38.7 MB hardened | 4 | 4 | 4 | 4 | **identical** |
| 34.8 MB app | 30,768 | 30,768 | 30,768 | 30,768 | **identical** |
Zero differences in either direction on all four — including the two archives where the count is
small because the payload is encrypted inside a shell, which is the target and not the tool.
**Full class listing, best of 3, stdout to a file (the same shape for every tool):**
| Archive | dexsql | rasc | droidasc | ddc |
|---|---|---|---|---|
| 1.4 MB / 1,396 classes | 0.035 s | 0.028 s | 0.149 s | 0.109 s |
| 4.3 MB / 647 classes | 0.031 s | 0.025 s | 0.139 s | 0.099 s |
| 38.7 MB / 4 classes | 0.040 s | 0.021 s | 0.156 s | 0.092 s |
| 34.8 MB / 30,768 classes | 0.136 s | **0.055 s** | 0.200 s | 0.156 s |
So on a real app `dexsql` is **1.5× faster than `droidasc`** and **2.5× slower than `rasc`**; the
`rasc` and `droidasc` columns reproduce this repository's earlier figures (0.048 / 0.190 s on the
same 34.8 MB archive), which is what makes the row comparable rather than merely new. A count-only
query on the same archive is 0.056 s — the listing itself is most of the difference.
**String cross-reference agrees with ASC.** On the 1.4 MB challenge, `SELECT DISTINCT method_class,
method_name FROM method_strings WHERE value LIKE '%http%'` returns exactly the three sites
`rasc findrefs … string http` and `droidasc findrefs … string http` return, and `ddc findrefs` agrees.
**The matching semantics differ and the difference is a trap:** `findrefs` matches a *substring*,
while SQL needs `LIKE '%…%'` — `value = 'http'` legitimately returns zero rows, because no method
holds a literal that *is* `http`.
**`decode` agrees with `ddc disasm`.** `dexsql decode --method
'Lsg/vantagepoint/uncrackable3/MainActivity;->verifyLibs()V'` emits the same instruction sequence, in
the same order, as `ddc disasm … MainActivity.verify` — checked line by line on that method.
**Persistent cache: a real feature with a small measured payoff here.** `prepare-cache` on the
34.8 MB archive takes 0.196 s and writes a 34.5 MB `cache.dxc2`. Repeating a query that needs the
full dataset — `method_calls` count, and a top-20 `GROUP BY` aggregate over it — went from 0.121 s to
0.098 s and from 0.470 s to 0.458 s. That is **under 20%, not an order of magnitude**, because the
parse-and-build path is already 0.12 s on this archive. Treat the cache as a way to hold one dataset
across many queries in a long session, not as a speed switch; on small archives it is a net loss
(cache read 0.071 s against a 0.042 s direct query on the 38.7 MB sample).
**Read-only is enforced, and the error surface is uniform.** Missing class → `Error: not_found:
selected file, method, or class not found`, exit 1. Missing input, a non-dex input, an unknown column,
an unknown relation and a `DELETE` all exit 1 with a named cause; the write is refused.
## Recipes worth keeping
```sql
-- who calls what, ranked (the question `findrefs` cannot express)
SELECT callee_class, callee_name, count(*) AS n FROM method_calls
GROUP BY 1,2 ORDER BY n DESC LIMIT 20;
-- every literal a method holds, instead of one needle at a time
SELECT method_class, method_name, value FROM method_strings
WHERE method_class LIKE 'Lcom/target/%' ORDER BY method_class;
-- numeric constants above a threshold: config values, protocol magic, crypto parameters
SELECT method_class, method_name, number_type, long_value FROM method_numbers
WHERE long_value > 1000000 ORDER BY long_value DESC LIMIT 50;
-- methods with no code (native, abstract) versus the ones that carry it
SELECT class_descriptor, name, method_kind, opcode_count FROM methods
WHERE opcode_count IS NULL OR opcode_count = 0;
-- which owners carry a given annotation
SELECT owner, annotation, count(*) AS n FROM annotation_elements_resolved
GROUP BY 1,2 ORDER BY n DESC LIMIT 20;
-- debug line table size: a cheap signal for "was this dex built with debug info"
SELECT count(*) AS positions FROM debug_positions;
```
`--sql` repeats, so several statements run against one loaded dataset; `--cache name=SELECT …` defines
a temporary table for the invocation; `--interactive` keeps the dataset loaded for a session. A
`method_idx` is DEX-local, so on a multi-dex APK carry `file_id` (from `dex_files`) into the join —
`-i` accepts several inputs and they share one dataset.
**The MCP surface is the same model for an agent.** `dexsql mcp` speaks JSON-RPC over stdio and
publishes 13 tools — `open_dataset`, `list_tables`, `describe_table`, `query`, `decode`,
`get_manifest`, `get_manifest_components`, `list_resource_files`, `read_resource`, `query_resources`,
`list_datasets`, `close_dataset`, plus an embedded documentation catalog on `resources/list`. The
resource tools read `resources.arsc` and archive entries **without extracting them to disk**, and
every list-shaped result is paged by `next_offset`. Datasets are opened by path and are capped
(`--max-datasets`, `--max-input-bytes`, `--timeout-ms`), so a client that forgets `close_dataset`
degrades rather than grows.
## Boundaries — what this does not settle
- **A hit is a reference, not a call, and SQL does not change that.** `method_strings` proves a method
holds a literal; whether that method runs on the path you care about is still a disassembly or
runtime question (`dynamic-frida.md`).
- **`decode` gives smali, not Java.** When the question is Java-level, `ddc getclass` or `jadx` is
still the tool; `dexsql` is where you *locate* and where you check bytecode.
- **One version, one platform family was measured.** v0.1.0 on Windows amd64. Linux/macOS assets exist
in the same release but were not exercised here (`unverified`), and no other dexsql version was
tested at all.
- **The cache is not a speed switch** — see the measurement above; the honest claim is "hold a dataset
across queries", not "queries get fast".
- **`--trusted-input` skips input-consistency checks.** It exists for controlled build pipelines; do
not reach for it to make a suspicious file load.
- **A packed target stays packed.** On both hardened archives `dexsql` returns the same 4 and 647
classes the other three tools return: the business dex is not in the file, and no query language
changes that. Take the question to the unpacking routes.
- **`quickened`/odex opcodes are unsupported** by `decode` (`odex` input is out of scope).
- **This is not a patching tool.** Nothing here writes a dex; `dex-patching.md` and
`byte-level-patching.md` own that, and `dexsql` is where you find the site to patch.
## Provenance
`https://github.com/LSPosed/dexsql-release` (release `v0.1.0`, published 2026-10-15; the repository
is the release host — it carries binaries, not source), retrieved and measured 2026-10-16. Nothing is
vendored into this repository: the binary lives in the gitignored `tools/` work area, and the
measurements above name the version, the platform and the hashes they came from.
@@ -31,6 +31,7 @@ next** — and points at the one section that says "the full record is elsewhere
| Dex method-level rewrite (dexlib2) | Documented, not measured: the smali round-trip blind spot is real and a tree round trip can pass every table check and still fail at load | partial | unverified | `references/dex-patching.md`, `references/patch-audit.md` |
| Dex header integrity and verifier legality | Recompute order proven on one written file; `dex_classdiff` passing is necessary, not sufficient — it cannot see code-item damage | ok | observed | `references/patch-audit.md`, `scripts/dexutil.py`, `scripts/dex_check_verifier.py` |
| Dex string-constant patch | Equal-length only; the shipped script refuses unequal lengths, so a different-length edit is a different, unmeasured route | partial | unverified | `references/byte-level-patching.md`, `scripts/dex_strpatch.py` |
| Whole-APK SQL query layer (dexsql) | Class-definition sets identical to rasc/droidasc/ddc on four archives, zero differences either way; a string xref query agrees site for site; `decode` emits smali, not Java, and the persistent cache is not a speed switch | ok | observed | `references/dexsql-sql-query-layer.md`, `references/toolchain.md` |
| Repack, sign, install (single APK) | End to end: STORED and 4-byte-aligned `resources.arsc`, v1+v2+v3 true, install succeeded, on-device hash matched the local build, control build still fails the old way | ok | observed | `references/repack-and-sign.md`, `references/verification.md`, `scripts/repack.py` |
| Split APK / App Bundle sets | Analyze, unified re-sign and merge measured on two real sets; merge correctly refuses a member carrying its own `resources.arsc` | partial | observed | `references/split-apk.md`, `scripts/repack.py` |
| Third-party build audit | Not measured: the APK differ is on the list of scripts no pass has run | partial | unverified | `references/third-party-builds.md`, `scripts/apk_diff.py` |
+1
View File
@@ -112,6 +112,7 @@ _54 row(s) below the header._
| `references/long-task-discipline.md` | The task will run long, or you are resuming one. Live record, conclusion grading, drift checkpoints, **deliverable-form drift (rooted-only vs shippable)**, bound-your-waits, **captures-you-never-looked-at**, **long-context decay**, handover |
| `references/pitfalls.md` | Always worth a skim before building. This is the failure catalogue. |
| `references/rasc-and-droidsaw.md` | You are about to reach for the ASC indexer, or the Python one is the bottleneck: the Rust re-implementation, its measured speedup and **identical class sets**, the one class shape where it silently drops code, and how to build it |
| `references/dexsql-sql-query-layer.md` | You are about to answer a dex question by grepping an export, or you need a **join or an aggregate** rather than a class list: the whole dex as SQL tables (callers, field accesses, numeric constants, debug lines, annotations, hiddenapi), manifest and `resources.arsc` queries, the MCP server, the measured standing against rasc/droidasc/ddc, and the two traps (`decode` is smali; `=` is not a substring match) |
| `references/coverage-and-limits.md` | You need to weigh a claim before trusting it: the evidence behind each covered item, the dependencies this skill does not ship, and the record of what was never exercised |
| `references/coverage/route-inventory.md` | You must answer "**can this skill actually do X**" in one pass: the per-route inventory — covered by verified mechanism, covered only by a documented route, the dependencies not shipped, and what is out of scope |
| `references/evidence-summary.md` | You need to know **whether a route was verified and how strong the evidence is**, from inside an installed copy: capability → one-line conclusion → strength → where the evidence lives |
@@ -35,6 +35,7 @@ cross-check with the program headers.
| **`rasc`** (Rust ASC) | `rasc findrefs app.apk string <S>` | **the same index, 4–15× faster**, no Python runtime, a Rust DEX decompiler behind `getclass`. Reach for this FIRST when it is built — its one blind spot is documented in `rasc-and-droidsaw.md` |
| **`droidasc`** (ASC) | `droidasc findrefs app.apk string <S>` | the original Python index, one `pip install` away, and the cross-check for `rasc`. Reach for it FIRST when `rasc` is not built |
| **`ddc`** | `ddc app.apk -c <Class>` | **fastest read of dex as Java, plus query subcommands** — reach for this SECOND, to *read*. See below |
| **`dexsql`** | `dexsql query --input app.apk --sql "…"` | **the whole dex as SQL tables** (callers, field accesses, numeric constants, debug lines, annotations) plus manifest and `resources.arsc` queries and an MCP server. Reach for it when the question is a **join or an aggregate** rather than a class list — see below |
| `apktool` | `java -jar apktool.jar d/b` | whole-app decode including resources |
| `jadx` | `jadx --no-res -d <out> <apk>` | readable Java for orientation; **not** a source of truth, and **not** a recon entry point (see below) |
| `aapt2` | `aapt2 dump badging <apk>` | manifest facts, package name, versions |
@@ -213,6 +214,48 @@ matters: a full `ddc <apk> -o out/` export is a reasonable thing to *have*, but
not a substitute for one indexed query, and it is the slower of the two by a wide margin on any APK
worth analyzing.
### dexsql — the whole dex as SQL, and the resources beside it
A third indexer at the same stage as ASC, and the one to reach for when the question is a **join or an
aggregate** rather than a class list. One self-contained binary (v0.1.0, 21.4 MB on Windows amd64, no
Python, no JVM, no index build), acquired as a release download from
`https://github.com/LSPosed/dexsql-release/releases`:
```bash
dexsql tables --input app.apk # every table and its column types
dexsql query --input app.apk --sql "SELECT count(*) AS n FROM methods"
dexsql query --input app.apk --sql "…" --sql "…" # several statements, one loaded dataset
dexsql query --input app.apk --interactive # keep it loaded, read SQL from stdin
dexsql decode --input app.apk --class 'Lcom/x/Y;' # smali, not Java
dexsql prepare-cache --input app.apk --output cache # then: query-cache --cache-file cache/cache.dxc2
dexsql mcp # MCP server over stdio
```
**What it is for.** `method_calls` (caller → callee), `method_field_accesses`, `method_strings`,
`method_used_types` and `method_numbers` are pre-joined views carrying the calling method's class and
name — so "which methods call `X`", "which methods hold a numeric constant above N", "what does the
debug line table say", "which owners carry this annotation" are single queries instead of greps.
`hiddenapi_flags` and the annotation tables have no equivalent in the other two indexers.
**Where it sits against the other two, measured on four archives (identical class-definition sets on
all four, zero differences either way):** on a 34.8 MB / 30,768-class app a full class listing takes
`dexsql` **0.136 s** against `droidasc` 0.200 s and `rasc` 0.055 s — 1.5× faster than the Python
indexer, 2.5× slower than the Rust one. **So it does not replace `rasc` for a class list**; it answers
questions `rasc` cannot express. Its string cross-reference was checked against ASC's on a real
archive and agrees site for site.
**Two traps.** `decode` emits **smali**, not Java — when the question is Java-level, `ddc getclass`
still owns it. And SQL's `=` is not `findrefs`: ASC matches a *substring*, so `WHERE value = 'http'`
legitimately returns nothing where `findrefs string http` returns three sites; use `LIKE '%…%'`.
**The persistent cache is not a speed switch.** `prepare-cache` on the 34.8 MB archive writes a
34.5 MB `cache.dxc2` in 0.196 s, and repeating a full-dataset query against it moved 0.121 s → 0.098 s
— **under 20%**, because parse-and-build is already 0.12 s there. It is a way to hold one dataset
across many queries in a long session; on a small archive it is a net loss.
Full table inventory, recipes, the MCP tool list and the boundaries:
`references/dexsql-sql-query-layer.md`.
**Classpath gotcha.** `baksmali`/`smali` need their dependency jars on the classpath together
(`smali`, `antlr-runtime`, `stringtemplate`, `baksmali`, `dexlib2`, `util`, `jcommander`, `guava`).
Missing one produces `ClassNotFoundException` that reads like a broken target. `scripts/smtool.py`
+47 -8
View File
@@ -211,6 +211,17 @@ ATOMS = {
'est_minutes': 1,
'basis': 'measured',
},
'tool:dexsql': {
'name': 'dexsql (SQL over dex and APK resources)',
'kind': 'tool',
'tool': 'dexsql',
'install': 'download the release binary for this platform from '
'https://github.com/LSPosed/dexsql-release/releases and put it on PATH '
'or on APKREV_TOOLS (no Python, no JVM, no crate, no build step)',
'est_minutes': 2,
# Measured here: release download plus first query, on the Windows amd64 asset.
'basis': 'observed',
},
'tool:apktool': {
'name': 'apktool (resource round-trip)',
'kind': 'tool',
@@ -356,6 +367,19 @@ CAPABILITIES = {
'note': 'one `pip install droidasc` away, and the cross-check for the Rust one: '
'their class-definition sets are compared in scripts/rasc_build.py --verify',
},
'dex_query_sql': {
'name': 'Whole-APK SQL query layer over dex and APK resources (dexsql)',
'required': ['tool:dexsql'],
'optional': [],
'note': 'one self-contained binary: no Python, no JVM, no index build. Its '
'class-definition set matched rasc, droidasc and ddc exactly on four archives '
'(1,396 / 647 / 4 / 30,768 classes, zero differences either way). What it adds '
'is the relational model -- method_calls, method_field_accesses, method_strings, '
'method_numbers, debug_positions, hiddenapi_flags, annotation_elements_resolved '
'-- plus APK resource queries and an MCP server. It is a third indexer at the '
'same stage as ASC, not a replacement for it: rasc is still the fastest for a '
'plain class list. See references/dexsql-sql-query-layer.md',
},
'static_dex': {
'name': 'Static dex / zip / strings analysis',
'required': ['python39'],
@@ -535,6 +559,24 @@ def extra_tool_dirs():
return [d for d in dirs if d and os.path.isdir(d)]
# Tools whose *documented install step in this kit* lands somewhere PATH does not know about.
# A machine that followed those instructions has the tool, and reporting it BLOCKED would make
# the capability table lie in the pessimistic direction -- the same reasoning as APKREV_TOOLS,
# applied to the tools acquired by a build or a release download rather than a package manager.
INSTALL_HINTS = {
'rasc': [
os.path.join(REPO_ROOT, 'tools', '_work', 'rust', 'target', 'release',
'rasc' + ('.exe' if os.name == 'nt' else '')),
os.environ.get('RASC', ''),
],
'dexsql': [
os.path.join(REPO_ROOT, 'tools', '_work', 'bin',
'dexsql' + ('.exe' if os.name == 'nt' else '')),
os.environ.get('DEXSQL', ''),
],
}
class Probe:
"""Host facts, probed once per process and cached.
@@ -559,15 +601,12 @@ class Probe:
if name in self._tool_cache:
return self._tool_cache[name]
hit = shutil.which(name)
if not hit and name == 'rasc':
# rasc ships no artifact, so a machine that followed this kit's own build step has it
# under the repository's ignored work area rather than on PATH. Accepting that location
if not hit:
# The kit's own install step for this tool lands outside PATH (rasc is built into the
# ignored work area, dexsql is a downloaded release binary). Accepting that location
# keeps the capability table from reporting BLOCKED at an agent that already did the
# work -- the same reasoning as APKREV_TOOLS, for the one tool whose install step is a
# build (scripts/rasc_build.py).
for cand in (os.path.join(REPO_ROOT, 'tools', '_work', 'rust', 'target', 'release',
'rasc' + ('.exe' if os.name == 'nt' else '')),
os.environ.get('RASC', '')):
# work -- see INSTALL_HINTS.
for cand in INSTALL_HINTS.get(name, []):
if cand and os.path.isfile(cand):
hit = cand
break