diff --git a/README.md b/README.md index 5201c2b..873cf6e 100644 --- a/README.md +++ b/README.md @@ -200,6 +200,22 @@ references/ loaded on demand, one topic each dnSpy-Unity-mono patch that makes a retail Mono player accept a debugger, a run-time class lookup over frida-il2cpp-bridge, and the audit probes when the deliverable is a report + il2cpp-protected-metadata.md + when that pair is not readable: absent, 0-byte, encrypted or + embedded metadata, the consumer-side dump, version-independent + header sizing, XOR/VM/XXTEA container shapes, the live-image dump + and the ELF repair that makes it usable + linker-and-soinfo-repair.md loading, dumping or repairing a .so by hand: the `dlopen` pipeline + as a loader spec, load_bias and segment arithmetic, DT_* parsing, + relocation rules and the two biases, minimal `soinfo` repair, + anti-dump header XOR, and where a dump is valid + vm-protector-analysis.md a native VM protector with virtual eflags: no cmp/sub/jcc, a flags + slot, NAND lowering, branchless select, data-anchored breakpoints + and trace-then-search-by-constant; handler slicing stays in + native-vm-recovery.md + python-bytecode-recovery.md Python bytecode behind Pyarmor/PyInstaller: key schedule, AES-CTR + nonce rules, the 12-byte VM, version-gated pyc parsing, the + extra-section TLV and per-object/per-string decryption native-and-so.md .so hosts, DT_NEEDED vs JNI_OnLoad, relocation limits, relocation-free bootstrapping, replacing Java methods natively, and which ABI/library is *actually loaded and executing* @@ -602,9 +618,10 @@ whose measurements disagreed with them. A **hardware pass** then added the layer software tooling: **hardware watchpoints** with their honest `TracerPid` ledger, **ETE/TRBE differential tracing** with its five gates, and **kernel-side injection templates** — every host-checkable piece measured, every device-bound piece labelled `unverified` until a gate -opens. Unity/IL2CPP logic recovery and -attaching a real debugger to a `.so` are **documented routes, not measurements** — absorbed from -public write-ups and labelled `unverified` throughout. React Native/Hermes bytecode internals and +opens. Unity/IL2CPP logic recovery (including +the protected-metadata case), linker-level `.so` repair, VM-protector idioms, Python/Pyarmor +bytecode recovery and attaching a real debugger to a `.so` are **documented routes, not +measurements** — absorbed from public write-ups and labelled `unverified` throughout. React Native/Hermes bytecode internals and defeating a server-side authority are **not** covered, and the skill is written to say so and stop rather than apply the nearest documented procedure to a target it was not written for. diff --git a/README.zh-CN.md b/README.zh-CN.md index 5c08a7b..e114c0e 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -120,6 +120,20 @@ references/ 按需加载，每个文件一个主题 用 Il2CppDumper 导出这一对文件、按能成功的顺序把表导入 IDA、 让零售 Mono 播放器接受调试器的 dnSpy-Unity-mono 补丁、 基于 frida-il2cpp-bridge 的运行时类查找，以及交付物是报告时的审计探针 + il2cpp-protected-metadata.md + 当这一对不可读时：缺失、0 字节、加密或内嵌的 metadata、 + 从消费端 dump、不信任版本号的 header 定长、XOR/VM/XXTEA + 容器形态、活体镜像 dump，以及让 dump 可用的 ELF 修复 + linker-and-soinfo-repair.md 手工加载、dump 或修复 .so：把 `dlopen` 管线当作 loader + 规范、load_bias 与段映射算术、DT_* 解析、重定位规则与 + 两个 bias、最小 `soinfo` 修复、反 dump 的 header XOR， + 以及 dump 在哪个窗口内有效 + vm-protector-analysis.md 带虚拟 eflags 的 native VM 保护：没有 cmp/sub/jcc、只有 + flags 槽、NAND 降级、无分支选择、数据锚定断点与 + 「先 trace 再按常量搜索」；handler 切片仍在 native-vm-recovery.md + python-bytecode-recovery.md Pyarmor/PyInstaller 之后的 Python 字节码：密钥调度、 + AES-CTR nonce 规则、12 字节 VM、按版本分支的 pyc 解析、 + extra section TLV 与按 code object/字符串解密 native-and-so.md .so 宿主、DT_NEEDED vs JNI_OnLoad、重定位限制、 免重定位的 bootstrap、用 native 替换 Java 方法， 以及*实际加载并执行*的是哪个 ABI/库 @@ -463,7 +477,7 @@ export APKREV_TOOLS=: # POSIX 为处理你自己的应用、你被授权分析的样本，以及 CTF/竞赛沙箱而构建。它不含任何内置的第三方二进制，也不含任何目标特定数据。 -它覆盖什么、刻意不覆盖什么，写在 `SKILL.md` 顶部 **Coverage** 一节。简版：仅 Android（不含 iOS），并且在经过真实打磨的层上深入 —— dex 补丁、重打包、壳与自定义加载器、native 篡改响应，以及 Flutter/Dart AOT。一次扩展迭代加入了第二梯队的已记录路线：重打包受阻时的**模块侧交付**、**抽取壳恢复**及其 VMP 边界、用于「调用而不是阅读」的**模拟执行与实时 RPC**、针对 OLLVM 的**指令级追踪**、REST 之外的**协议逆向**、当用户态 hooking 可证明够不到时用的**内核路线**图，以及**设备上工具**。随后一次**基准迭代**把公开目标放到这些路线之下（`tests/benchmark.md`）：它加入了 **Java2C 判别**（那种会让 agent 去追一个从不存在的解密 DEX 的误诊）、**split APK / App Bundle 处理**、**无 schema 的 protobuf 解码**、一个 **Dex-VMP 差分**测试台，以及**带版本闸门的内核模块模板** —— 并纠正了先前两条与自身测量结果不一致的说法。随后一次**硬件迭代**加入了软件工具之下的那一层：带诚实 `TracerPid` 账本的**硬件观察点**、带五道闸门的 **ETE/TRBE 差分追踪**，以及**内核侧注入模板** —— 凡是宿主可验证的部分都实测过，凡是设备绑定的部分在闸门打开前一律标注 `unverified`。Unity/IL2CPP 逻辑恢复与**给 `.so` 接真实调试器**属于**已记录但未实测**的路线 —— 吸收自公开文章，全文标注 `unverified`。React Native/Hermes 字节码内部，以及击败服务端权威**不**在覆盖范围内，skill 的写法就是明说这一点并停下，而不是把最近的一条已记录流程套用到一个它并非为之编写的目标上。 +它覆盖什么、刻意不覆盖什么，写在 `SKILL.md` 顶部 **Coverage** 一节。简版：仅 Android（不含 iOS），并且在经过真实打磨的层上深入 —— dex 补丁、重打包、壳与自定义加载器、native 篡改响应，以及 Flutter/Dart AOT。一次扩展迭代加入了第二梯队的已记录路线：重打包受阻时的**模块侧交付**、**抽取壳恢复**及其 VMP 边界、用于「调用而不是阅读」的**模拟执行与实时 RPC**、针对 OLLVM 的**指令级追踪**、REST 之外的**协议逆向**、当用户态 hooking 可证明够不到时用的**内核路线**图，以及**设备上工具**。随后一次**基准迭代**把公开目标放到这些路线之下（`tests/benchmark.md`）：它加入了 **Java2C 判别**（那种会让 agent 去追一个从不存在的解密 DEX 的误诊）、**split APK / App Bundle 处理**、**无 schema 的 protobuf 解码**、一个 **Dex-VMP 差分**测试台，以及**带版本闸门的内核模块模板** —— 并纠正了先前两条与自身测量结果不一致的说法。随后一次**硬件迭代**加入了软件工具之下的那一层：带诚实 `TracerPid` 账本的**硬件观察点**、带五道闸门的 **ETE/TRBE 差分追踪**，以及**内核侧注入模板** —— 凡是宿主可验证的部分都实测过，凡是设备绑定的部分在闸门打开前一律标注 `unverified`。Unity/IL2CPP 逻辑恢复（含受保护的 metadata 情形）、linker 层的 `.so` 修复、VM 保护器惯用法、Python/Pyarmor 字节码恢复，以及**给 `.so` 接真实调试器**属于**已记录但未实测**的路线 —— 吸收自公开文章，全文标注 `unverified`。React Native/Hermes 字节码内部，以及击败服务端权威**不**在覆盖范围内，skill 的写法就是明说这一点并停下，而不是把最近的一条已记录流程套用到一个它并非为之编写的目标上。 Coverage 一节完整陈述、同时也应当放在这里的四条限定： diff --git a/docs/tool-verification/EXTENSION-ref-absorption-2026-10.md b/docs/tool-verification/EXTENSION-ref-absorption-2026-10.md new file mode 100644 index 0000000..82ff53a --- /dev/null +++ b/docs/tool-verification/EXTENSION-ref-absorption-2026-10.md @@ -0,0 +1,113 @@ +# EXTENSION — reference absorption from `ref/*` (2026-10) + +This pass read sixteen public Chinese write-ups dropped in `ref/` at the repository root and +absorbed their *mechanisms* into the skill. It did not run any of them. The record exists so a later +reader can tell what changed and on what footing. + +## Sources + +Sixteen files in four groups. Titles, authors, product names and package identifiers are omitted; +the descriptions are enough to identify the shape without carrying target identity onto the tracked +surface. + +| Group | Files | What they are | +|---|---|---| +| Unity / IL2CPP | `unity1.md`–`unity6.md`, `unity8.md`, `unity10.md`, `unity12.md` | Nine write-ups on protected Unity builds: absent/encrypted/embedded metadata, a custom container VM, a two-stage native loader, HybridCLR IL-body encryption, a version-lying metadata header, live-image dumping and ELF repair, a runtime IL2CPP bridge, an AssetBundle wrapper and a CDN manifest | +| SO / linker | `so1.md`, `so2.md`, `so4.md` | Two source-reading walkthroughs of the Android `dlopen` pipeline and packer design (Android 5.0 and 8.x), plus a Frida cookbook for native/Java hooks | +| VM protector | `vmp1.md`, `vmp2.md`, `vmp3.md` | A teaching series and a practical crack on x86 VMProtect-style targets: virtualized dispatch, virtual eflags, NAND lowering, trace search | +| Python bytecode | `py1.md` | Static unpacking of a Pyarmor build with a modified `pycdc` | + +All sixteen are **public secondary sources**. Their own claims are `observed` by their authors when +they show a command and output; nothing in them is reproduced in this repository. The absorbed +mechanics are therefore `unverified` here, and the *ordering judgement* built on them is +`inferred`. No text, code, key material, offset table or screenshot was copied. + +## What was added + +Four reference files, each with its own claim-strength paragraph and boundary: + +| File | Absorbs | Deliberately does not | +|---|---|---| +| `references/il2cpp-protected-metadata.md` | Metadata triage (absent/0-byte/encrypted/embedded/version-lie), consumer-side dump, 0x10C header sizing, repeating-key XOR, custom bytecode-VM container, XXTEA, length-derived string keys, live-image dump and ELF repair, HybridCLR IL-body decryption, AssetBundle header transforms | Recover IL2CPP native code or build an opcode map; explain a specific dumper's internals | +| `references/linker-and-soinfo-repair.md` | `dlopen` pipeline as a loader spec, `load_size`/`load_bias`, `seg_page_offset`, `DT_HASH`/`DT_GNU_HASH`, relocation targets and the two biases, APS2, minimal `soinfo` repair, anti-dump header XOR, dump points, constructor timing | Implement an unpacker; guarantee `soinfo` offsets across Android versions | +| `references/vm-protector-analysis.md` | Envelope triage, VM_DATA/VM_CONTEXT/vEIP/vStack, central vs direct-threaded dispatch, data-anchored breakpoints, forged returns, trace-by-constant, virtual eflags, NAND lowering, branchless select, static rebase caveats | Reconstruct the guest instruction stream; replace handler slicing in `native-vm-recovery.md` | +| `references/python-bytecode-recovery.md` | Pyarmor key schedule, AES-CTR nonce rules, the 12-byte nonce-XOR VM, version-gated `pyc` parsing, extra-section TLV, per-code-object and per-string decryption | Run the recovered module; generalise field offsets across Pyarmor versions | + +### Existing files touched + +- `unity-runtime.md` — the "metadata does not parse" branch now hands the protected case to the new + file instead of implying that a retry with another dumper is the answer. +- `framework-runtimes.md` — Python runtimes (Chaquopy/Kivy/BeeWare) added to the identification + table, with the new file as the next step. +- `native-vm-recovery.md` — a cross-link for the case where the missing piece is *what a handler + computes* (virtual eflags), not *where the dispatcher is*. +- `code-virtualization-and-custom-linkers.md` — a cross-link for when the loader itself becomes the + object of study rather than its validation check. + +## Registration surface updated + +- `SKILL.md` symptom index and the `routing.md` mirror: five rows, identical first cells. +- `routing.md` reference index: four rows. +- `evidence/capability-matrix.json`: four rows, all `partial` / `route_measured: false` / + `unverified`; the existing Unity row's evidence list now names the companion file. +- `references/evidence-summary.md`: four rows. +- `references/coverage-and-limits.md` and `references/coverage/route-inventory.md`: four documented + routes, and the not-covered statement now names both Unity files. +- `README.md` and `README.zh-CN.md`: tree entries and a scope clause. + +## Second pass — gaps found by auditing the first pass + +The first pass kept the four new references at the level of the primary mechanism and left several +secondary mechanisms in the sources. An explicit file-by-file audit found these, and they are now +absorbed into the files that already own the topic (same `unverified` / `inferred` labels): + +| Missed mechanism | Now in | +|---|---| +| Staged loader that decrypts a DEX from an asset, selects a payload by ABI, and patches an 8-byte tail | `code-virtualization-and-custom-linkers.md` §staged loaders | +| IL2CPP live object graph (`Il2Cpp.gc.choose`, getter invocation) and `Il2Cpp.dump(name, path)` | `unity-runtime.md` §5 | +| ARM64 app on an x86 host: translation runs the app but not an ARM64 `frida-server` | `environment.md` §emulator notes | +| MessagePack as a self-describing body; mutate request before response | `protocol-reverse.md` §1.1, `server-api.md` | +| A delayed detection that is a `sleep`/`nanosleep` timing measurement | `detection-and-anti-analysis.md` §timing-delayed checks | +| A dynamically loaded dex visible as a `(deleted)` mapping | `advanced-unpacking.md` §read the map | +| `RegisterNatives` table layout, `dlopen` gating, instruction+4 reads, Thumb NOP patch inside `call_function`, JNI argument shapes, OLLVM call-graph triage | `dynamic-frida.md` §native registration | +| Trace call-site anchoring, `rel32` resolution, back-edge hot-loop suppression | `vm-protector-analysis.md` §4.1 | +| Branchless `max` arithmetic | `vm-protector-analysis.md` §6.2 | +| Encoded handler tables (`table[i] + constant`) and the one-register/benign-landing dispatcher invariant | `native-vm-recovery.md` §2.1-2.2 | +| The `29.1` `genericMethodPointersCount` misdetection on a large `.so` | `il2cpp-protected-metadata.md` §5 | + +### Deliberately not absorbed + +- **Target-specific constants**: per-sample addresses, AES keys, section names, package ids, CDN + hosts, device serials and the write-ups' own scripts. They would be reused wrongly and they carry + identity; the reference files say the constants must be re-derived. +- **One-target asset pipelines**: Live2D `moc3` parsing, USM video reconstruction, the CDN manifest + delta and the Blender import conventions. They are not APK-reversing mechanisms and belong to an + asset-extraction skill; `unity-runtime.md` §6 keeps the audit question, not the pipeline. +- **Middleware-specific audio decryption** (CRIWARE ACB) and the GUI-only steps of the cited + debugger sessions. +- **Secure-coding advice** (constant-time comparison) that is not a reverse-engineering route. +- **Exact opcode tables** from a challenge VM. The procedure for deriving one is covered in + `native-vm-recovery.md`; copying the table would invite applying it to a different ISA. + +## What would change a label + +Per file, one cheap experiment turns the `unverified` mechanism into a local `observed`: + +- **IL2CPP metadata:** decrypt one file to a valid `0xFAB11BAF` and have Il2CppDumper accept it; + separately dump one live image and name one function in the disassembler. +- **Linker:** load a trivial `.so` from memory with the arithmetic in the file, resolve one symbol + through the repaired `soinfo`, and dump one protected image at `LoadSegments`. +- **VM protector:** trace one protected comparison, find the virtual flags slot, force the value, and + show the failure path disappear while the VM remains intact. +- **Python:** parse one `.pyc`, show the extra-section counts matching the file, decrypt one code + object, and execute the recovered module to the same output as the original. + +Until one of these is recorded, the capability matrix rows must stay `partial` / `unverified`, and +a reader must not cite them as support for a claim about a target. + +## Gate status + +The structural gates were run after the change: `check_repo.py`, `check_refs.py`, +`check_routing.py`, `check_commands.py`, `check_budget.py`, and `pytest -q`. A green run proves +the inventory resolves and the index mirrors agree; it says nothing about whether the mechanisms +work. That distinction is the point of the labels above. diff --git a/docs/tool-verification/README.md b/docs/tool-verification/README.md index 1bd5bee..6427728 100644 --- a/docs/tool-verification/README.md +++ b/docs/tool-verification/README.md @@ -88,6 +88,7 @@ written down. | `EXTENSION-dart-aot-formats.md` | Two string-table claims tested against a real dual-ABI Flutter app: the arm64 packed scheme confirmed against literal tag bytes, and the assumed armv7 `len*2`/UTF-16 form **refuted** — the 32-bit record is `[header u32][byte-count u32le][UTF-8]`, and the extractor's zero is a format mismatch rather than an empty table | | `EXTENSION-detection-pipeline.md` | Naming the check that fires: three arms on a public MASTG target (control alive, probe arm dead in ≈300 ms via `strstr("frida")`, with the `TracerPid=0` versus 4-frida-mappings asymmetry), `svc_scan.py`'s two decoders agreeing on an identical 214-site set, the layered Java-to-svc descent, and the ptrace-free dump boundary on targets whose dex is deflated inside the APK | | `EXTENSION-dexsql.md` | The SQL query layer adopted as a **third indexer** beside rasc/droidasc/ddc: identical class-definition sets on four archives (zero differences either way), the string xref agreement and the `=` versus `LIKE` match-semantics trap, `decode` against `ddc disasm` line for line, and the persistent cache measured as **not** a speed switch | +| `EXTENSION-ref-absorption-2026-10.md` | Sixteen public write-ups in four groups (Unity/IL2CPP metadata protection, linker-level SO packing, VM protector idioms, Pyarmor bytecode), the four reference files they became, the registration surface that changed, and the one cheap experiment that would move each row off `unverified` | ## Hardware-layer pass record diff --git a/skills/apk-reverse/SKILL.md b/skills/apk-reverse/SKILL.md index a75d218..ee4b3d4 100644 --- a/skills/apk-reverse/SKILL.md +++ b/skills/apk-reverse/SKILL.md @@ -7,7 +7,7 @@ metadata: version: "1.0" capability_registry: skills/apk-reverse/scripts/capabilities.py evidence_summary: skills/apk-reverse/references/evidence-summary.md - last_reconstruction_pass: "2026-09-22" + last_reconstruction_pass: "2026-10-24" strength_labels: "observed | inferred | unverified | unknown" --- @@ -140,9 +140,10 @@ requires a module this skill only scaffolds. **Not covered — say so rather than improvise:** React Native / Hermes / Cordova · iOS · defeating a server-side authority · an off-the-shelf unpacker or an anti-detection arms race · building and -shipping a kernel module. **Documented but never measured here** — Unity (Mono and IL2CPP) and +shipping a kernel module. **Documented but never measured here** — Unity (Mono and IL2CPP, including +protected metadata), linker-level `.so` repair, VM-protector idioms, Python/Pyarmor bytecode, and attaching a real debugger to a `.so`: a route, not a result (`unity-runtime.md`, -`ida-so-debugging.md`). Reasons and evidence boundaries are in `references/coverage-and-limits.md`; +`il2cpp-protected-metadata.md`, `ida-so-debugging.md`). Reasons and evidence boundaries are in `references/coverage-and-limits.md`; **silence in that record is not support.** **The fallback, as an instruction — this is a rule, not advice.** If the target does not match the @@ -246,6 +247,11 @@ unread in this repository. | A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK | | A debugger lists and attaches to the process, but the target `.so` is absent from the module list and every breakpoint in it refuses or never hits, while its feature works | `ida-so-debugging.md` §the `.so` was never extracted — `android:extractNativeLibs="false"` maps the library from `base.apk`, so there is no `/data/app//lib//` file to bind to. Not an anti-debug finding | | A Unity package has no `Assembly-CSharp.dll` — only `libil2cpp.so` and `global-metadata.dat` — and every Java-layer hook turns out to be glue | `unity-runtime.md` — that is IL2CPP, the normal shipping configuration, not a stripped build: dump the pair with Il2CppDumper and import `script.json` before `stringliteral.json`, or the database is silently half-named | +| A Unity IL2CPP package has **no readable `global-metadata.dat`** — 0 bytes, absent, wrong magic, a custom container, or a metadata version that disagrees with its own struct | `il2cpp-protected-metadata.md` — anchor on the consumer's output pointer, size the header without trusting its version, and recognise the XOR/VM/XXTEA shapes; `unity-runtime.md` owns the readable case | +| You dumped `libil2cpp.so` from memory but the disassembler shows no symbols, or every registration search returns 0 | `il2cpp-protected-metadata.md` §4.1 — `p_offset=p_vaddr`, `p_filesz=p_memsz`, `e_shoff` from the last `PT_LOAD` end, `e_shstrndx`, `sh_offset=sh_addr`; the dump is fine, the ELF metadata is not | +| A `dlopen`-able `.so` will not load after a custom loader, or a repaired `soinfo` resolves symbols to garbage | `linker-and-soinfo-repair.md` — bias versus base, `seg_page_offset`, the defining library's bias, `APS2` packed relocs, the minimal `soinfo` field set, and the dump window after `LoadSegments` | +| A native trace has no `cmp`/`sub`/`jcc` — only `and/or/not/shr/neg`, a flags slot, and a `ret` that lands at a VM entry | `vm-protector-analysis.md` — virtual eflags is data, NAND lowering, branchless select, data-anchored breakpoints, and trace-then-search-by-constant; dispatcher slicing stays in `native-vm-recovery.md` | +| The logic you need is Python bytecode — a `.pyc`/PyInstaller/Pyarmor payload rather than Java, Dart or native | `python-bytecode-recovery.md` — key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing and the extra-section TLV; static and offline, then call the recovered module | ## Gates — clear these before you patch, in order diff --git a/skills/apk-reverse/evidence/capability-matrix.json b/skills/apk-reverse/evidence/capability-matrix.json index 64bda01..5d51e5b 100644 --- a/skills/apk-reverse/evidence/capability-matrix.json +++ b/skills/apk-reverse/evidence/capability-matrix.json @@ -681,12 +681,83 @@ "benchmark_rows": [], "evidence": [ "references/unity-runtime.md", - "references/framework-runtimes.md" + "references/framework-runtimes.md", + "references/il2cpp-protected-metadata.md" ], "evidence_not_shipped": [ - "ref/unity1.md, ref/unity2.md, ref/unity3.md -- public Unity reverse-engineering write-ups (2026-02, 2026-06), methodology absorbed, no text copied" + "ref/unity1.md .. ref/unity12.md -- public Unity/IL2CPP reverse-engineering write-ups (2024-2026), methodology absorbed, no text copied" ], - "limits": "Absorbed, never run here: no Unity package was dumped and no patched Mono runtime was built in this record. What is documented: Mono versus IL2CPP identification by shipped files, Il2CppDumper over libil2cpp.so + global-metadata.dat with the metadata magic (0xFAB11BAF) as the encryption check, the four-step IDA import order (ida_py3.py, script.json, stringliteral.json, ida_with_struct_py3.py), a frida-il2cpp-bridge lookup for a run-time or encrypted case, and the dnSpy-Unity-mono patch set that makes a retail Mono player accept a debugger. Unverified details: the exact dnSpy patch hunks, the Android libmono.so equivalent (inferred from a desktop-only source), whether the loader prefers an on-disk .so on a hardened build, and every tool's behaviour on a stripped or protected package. No opcode or offset claim is made here." + "limits": "Absorbed, never run here: no Unity package was dumped and no patched Mono runtime was built in this record. What is documented: Mono versus IL2CPP identification by shipped files, Il2CppDumper over libil2cpp.so + global-metadata.dat with the metadata magic (0xFAB11BAF) as the encryption check, the four-step IDA import order (ida_py3.py, script.json, stringliteral.json, ida_with_struct_py3.py), a frida-il2cpp-bridge lookup for a run-time or encrypted case, and the dnSpy-Unity-mono patch set that makes a retail Mono player accept a debugger. Unverified details: the exact dnSpy patch hunks, the Android libmono.so equivalent (inferred from a desktop-only source), whether the loader prefers an on-disk .so on a hardened build, and every tool's behaviour on a stripped or protected package. No opcode or offset claim is made here. The protected-metadata and memory-dump half moved to the il2cpp-protected-metadata-recovery row; this row covers the readable case." + }, + { + "capability": "il2cpp-protected-metadata-recovery", + "group": "runtime", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/il2cpp-protected-metadata.md", + "references/unity-runtime.md" + ], + "evidence_not_shipped": [ + "ref/unity1.md, ref/unity2.md, ref/unity3.md, ref/unity4.md, ref/unity5.md, ref/unity6.md, ref/unity8.md, ref/unity10.md, ref/unity12.md -- public Unity/IL2CPP write-ups (2024-2026), methodology absorbed, no text copied", + "docs/tool-verification/EXTENSION-ref-absorption-2026-10.md (repository root, not shipped)" + ], + "limits": "Absorbed, never run here: no metadata was decrypted and no libil2cpp.so was dumped in this record. Covered as a route: deciding between absent, 0-byte, encrypted and embedded metadata; anchoring on the consumer's return pointer (MetadataLoader::LoadMetadataFile or GlobalMetadata::Initialize) instead of the path; the 0x10C descending-by-8 header sizing rule; repeating-key XOR (i & 0x7F), a custom-magic bytecode-VM container, XXTEA with a trailing length word, and length-derived string ciphers; dumping the second mapping and repairing p_offset/p_filesz/e_shoff/e_shstrndx/sh_offset; and treating the metadata version as a claim when the struct layout disagrees. Every offset and magic beyond 0xFAB11BAF belongs to one cited sample and is not a constant." + }, + { + "capability": "linker-level-so-unpacking", + "group": "native", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/linker-and-soinfo-repair.md", + "references/code-virtualization-and-custom-linkers.md", + "references/advanced-unpacking.md" + ], + "evidence_not_shipped": [ + "ref/so1.md, ref/so2.md, ref/so4.md -- public Android linker and SO-packing write-ups (Android 5.0/8.x source review plus a Frida cookbook), methodology absorbed, no text copied", + "docs/tool-verification/EXTENSION-ref-absorption-2026-10.md (repository root, not shipped)" + ], + "limits": "Source-reading synthesis plus a Frida cookbook, not a measurement: no loader was reimplemented and no protected .so was dumped here. The route covers the dlopen pipeline as a loader specification, load_size/load_bias, seg_page_offset when loading from memory, DT_HASH and DT_GNU_HASH layouts, RELATIVE/GLOB_DAT/JUMP_SLOT/IRELATIVE targets and the defining-library-versus-caller bias, APS2 packed relocations, the minimal soinfo repair set, anti-dump header XOR detected by ELF invariants, and the dump window at LoadSegments. soinfo offsets cited in the sources are Android-version-specific; resolving solist by symbol and taking layout from the matching AOSP tag is the durable form. Timing (constructors, one-shot RegisterNatives) is a documented boundary, not an end-to-end run." + }, + { + "capability": "vm-protector-analysis", + "group": "native-dynamic", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/vm-protector-analysis.md", + "references/native-vm-recovery.md", + "references/native-dbi-and-deobfuscation.md" + ], + "evidence_not_shipped": [ + "ref/vmp1.md, ref/vmp2.md, ref/vmp3.md -- public VMProtect teaching and crack write-ups (x86), methodology absorbed, no text copied", + "docs/tool-verification/EXTENSION-ref-absorption-2026-10.md (repository root, not shipped)" + ], + "limits": "Absorbed from x86 VMProtect-style samples and not executed against an Android target here. Covered: envelope triage (vmp/VMP0 section, entry jmp, push-imm/call, non-compiler prologue), VM_DATA/VM_CONTEXT/vEIP/vStack anatomy and central versus direct-threaded dispatch, data-anchored breakpoints with stack unwind, forged-return recognition, trace-then-search-by-constant with instruction-shape and operand-role filters, virtual eflags as a writable stack slot (eflags1 & 0x815 + eflags2 & ~0x815; ZF = & 0x40), NAND lowering and arithmetic identities, branchless select, and static rebase/xref caveats. Trace counts and addresses are one sample's and must not be carried across. It tells you which value a protected branch consumes; it does not reconstruct the original instruction stream." + }, + { + "capability": "python-bytecode-recovery", + "group": "unpacking", + "status": "partial", + "route_measured": false, + "strength": "unverified", + "benchmark_rows": [], + "evidence": [ + "references/python-bytecode-recovery.md", + "references/emulation-and-rpc.md" + ], + "evidence_not_shipped": [ + "ref/py1.md -- public Pyarmor/pycdc static-unpacking write-up (2026), methodology absorbed, no text copied", + "docs/tool-verification/EXTENSION-ref-absorption-2026-10.md (repository root, not shipped)" + ], + "limits": "Absorbed from one Pyarmor build and never executed here. Covered: build-tag/RSA-blob/AST_nonce key derivation through MD5, AES-128-CTR with the 12-byte nonce and nonce[15]=2, the 12-byte nonce-XOR VM (8 registers, valid indices 0-5 and 7, opcodes 1-0xB, idiv clobbering register 0), version-gated pyc field widths and the pre-3.8 flag remap, the extra-section TLV (fn_count/descriptor_count, (byte>>6)+2 item length, big-endian consts index), per-code-object decryption including copy_prologue and 0x09 fill, and per-string tag rules. Every offset is version-specific; the file's own claim strength says so. Static route only: no runtime, device or root, and no recovered module was executed here." }, { "capability": "native-so-remote-debugger-attach", diff --git a/skills/apk-reverse/evidence/known-limitations.md b/skills/apk-reverse/evidence/known-limitations.md index 0b55fe8..4f70f0f 100644 --- a/skills/apk-reverse/evidence/known-limitations.md +++ b/skills/apk-reverse/evidence/known-limitations.md @@ -74,9 +74,16 @@ built on them has nothing to check itself against. - **React Native / Hermes bytecode and Cordova internals, and every iOS or `.ipa` workflow.** The runtime can be identified; the internals are not covered. - **A measured Unity / IL2CPP route.** `unity-runtime.md` documents backend identification, - Il2CppDumper, the IDA import order, the Frida bridge and the dnSpy-Unity-mono patch — but nothing - in it was executed here, there is no byte-level reading of generated native code, and no package - was tested protected. Do not report a successful dump as a located method. + Il2CppDumper, the IDA import order, the Frida bridge and the dnSpy-Unity-mono patch, and + `il2cpp-protected-metadata.md` documents the absent/encrypted/embedded/version-lying cases plus the + live-image dump and ELF repair — but nothing in either was executed here, there is no byte-level + reading of generated native code, and no package was tested protected. Do not report a successful + dump as a located method. +- **A measured linker-level `.so` repair, VM-protector analysis or Python bytecode recovery.** + `linker-and-soinfo-repair.md`, `vm-protector-analysis.md` and `python-bytecode-recovery.md` are + absorbed routes: the arithmetic and idioms are documented, but no loader was reimplemented, no + protected VM was traced, and no Pyarmor sample was unpacked here. Every field offset is + version-specific; a repair or a key that works on one build is not evidence for the next. - **A measured debugger attach.** `ida-so-debugging.md` documents the `dbgsrv` route and the `am start -D` + JDWP handshake, never run here; the `extractNativeLibs="false"` chapter is the recorded failure mode, not a fix verified on a hardened build. diff --git a/skills/apk-reverse/references/advanced-unpacking.md b/skills/apk-reverse/references/advanced-unpacking.md index 4af8ffe..b0c5495 100644 --- a/skills/apk-reverse/references/advanced-unpacking.md +++ b/skills/apk-reverse/references/advanced-unpacking.md @@ -331,6 +331,14 @@ located, already sized, no search required: Copy those ranges first. Everything else — anonymous `rw-p`, `libc_malloc`, `.bss` — is a *search* target, and searching is the expensive half. +**A `(deleted)` mapping is still readable while it is mapped.** When a JNI-reachable Java method has +no implementation anywhere in the APK, look for a dex mapping whose path ends in `(deleted)` — a +loader wrote the file, mapped or loaded it, then unlinked it. Copy that range before the process +exits; `/proc//mem` does not care that the name is gone. On one sample this recovered the only +copy of a dynamically loaded dex and the Java method that had been invisible to jadx. Treat the +range's `r--p` permission and a plausible `dex\n035` header as the control that it is code, not a +cache. + ### Address arithmetic is where this route silently breaks Three device-shell traps cost real time on the first run, and each one produces a diff --git a/skills/apk-reverse/references/code-virtualization-and-custom-linkers.md b/skills/apk-reverse/references/code-virtualization-and-custom-linkers.md index 89caf38..1bbccd8 100644 --- a/skills/apk-reverse/references/code-virtualization-and-custom-linkers.md +++ b/skills/apk-reverse/references/code-virtualization-and-custom-linkers.md @@ -159,6 +159,51 @@ payload and jumping to a zero entry. Treat "`Invalid ELF magic` appears on my bu original" as **proof the loader is validating the build**, which is exactly what makes the string-redirect route worth trying. +When the loader itself becomes the object of study — you need to load, dump or repair its payload +rather than neutralise its check — `linker-and-soinfo-repair.md` owns the pipeline arithmetic and +the `soinfo` hand-off; this file owns recognising that the loader is present and deciding whether +to keep it. + +## Staged loaders that decrypt a DEX from assets + +A second family does not keep the payload in the APK's dex at all. Recognise it by the *shape*, not +by a product name: + +1. A Java `Application`/`attachBaseContext` loads a small stage-1 `.so`. +2. That library's `JNI_OnLoad` calls `RegisterNatives` for two or three methods and returns. +3. `attachBaseContext` then calls one of them with `dataDir`, `nativeLibraryDir` and the + `AssetManager` — the arguments are the tell that the native side is about to write a file and + load code from it. +4. The native method opens an asset whose name looks random, derives a key, decrypts an AES payload + into Dalvik bytecode, and returns a direct `ByteBuffer`. +5. Java passes that buffer to a `DexClassLoader`/`loadDex` path. The stage-1 `.so` may then launch a + stage-2 `.so` from the decrypted code. + +The stage-2 payload often lives in `assets` as an extensionless `.sec` file. One sample selected it +by CPU family — including mapping **ARM64 to an `x64`-named payload** — then: + +- hex-decoded a constant string into an AES-256 key and a per-block IV; +- decrypted 16-byte blocks with AES, then XORed each with the IV; +- read an **8-byte tail** from the `.sec` file as an offset, copied `[offset, EOF)` as the real + payload, and XORed that region with the file's own prefix; +- decrypted the exported symbol name with the same construction before `dlsym`, and `unlink`ed the + temporary file after `dlopen`. + +**What generalises:** the asset name, key, IV and tail layout are per-build and must be read from +the loader, not carried across. What does carry is the diagnostic chain — *asset with a random name + +`AssetManager` passed into a registered native method + a returned direct buffer + `DexClassLoader`* +— and the fact that the second stage picks its payload by ABI, so an x86 host may never see the +ARM64 branch. + +**Preconditions to record before spending time:** whether the loader writes to `dataDir` or +`nativeLibraryDir`, whether the payload is deleted after load, and which ABI the running process +reports. A dump taken after the `unlink` can only come from memory; a dump taken before it can come +from the file, which is cheaper and gives an exact length. + +**Claim strength.** One family, one sample, absorbed from a public write-up; keys and offsets are +deliberately not reproduced. Treat the sequence as a recognition template and recover the constants +from the binary in front of you. + ## What the native check actually reads — measure it, don't guess A validation payload that reads "the signature" can be reading several different things, and the diff --git a/skills/apk-reverse/references/coverage-and-limits.md b/skills/apk-reverse/references/coverage-and-limits.md index a031080..e55412e 100644 --- a/skills/apk-reverse/references/coverage-and-limits.md +++ b/skills/apk-reverse/references/coverage-and-limits.md @@ -166,12 +166,31 @@ its own strength note at the top. for browser targets and never run on an Android long-connection app here — `unverified` end to end, with Android capture points `inferred` from documented APIs (`protocol-reverse.md` §6). -- **Unity logic recovery (Mono and IL2CPP)** — telling the backends apart by what they ship, - dumping `libil2cpp.so` + `global-metadata.dat` with Il2CppDumper, the metadata magic that decides - whether the dump is even possible, the four-step IDA import order, a run-time class/method lookup - over frida-il2cpp-bridge, and the dnSpy-Unity-mono patch that makes a retail Mono player accept a - debugger. Absorbed from three public write-ups; no package was dumped here and no patched runtime - was built, so every step is `unverified` (`unity-runtime.md`). +- **Unity logic recovery (Mono and readable IL2CPP)** — telling the backends apart by what they + ship, dumping `libil2cpp.so` + `global-metadata.dat` with Il2CppDumper, the metadata magic that + decides whether the dump is even possible, the four-step IDA import order, a run-time class/method + lookup over frida-il2cpp-bridge, and the dnSpy-Unity-mono patch that makes a retail Mono player + accept a debugger. Absorbed from public write-ups; no package was dumped here and no patched + runtime was built, so every step is `unverified` (`unity-runtime.md`). +- **Protected IL2CPP metadata** — when the pair is absent, 0 bytes, encrypted, embedded or version- + inconsistent: anchoring on the consumer's output pointer, the version-independent header-sizing + rule, the repeating-key XOR / custom bytecode-VM / XXTEA container shapes, the second-mapping + live dump, and the ELF repair that names symbols again. Absorbed from public write-ups and never + executed here (`il2cpp-protected-metadata.md`). +- **Linker-level SO unpacking and `soinfo` repair** — the `dlopen` pipeline as a loader + specification, `load_bias` and segment arithmetic including `seg_page_offset`, `DT_*` parsing, + relocation targets and the defining-library-versus-caller bias, the minimal `soinfo` repair set, + anti-dump header XOR detected by ELF invariants, and the dump window at `LoadSegments`. Source- + reading synthesis plus a Frida cookbook; no loader was reimplemented here + (`linker-and-soinfo-repair.md`). +- **VM protector analysis** — envelope triage, VM_DATA/VM_CONTEXT/vEIP/vStack anatomy, data-anchored + breakpoints, forged-return recognition, trace-then-search-by-constant, virtual eflags as writable + data, NAND lowering and branchless select. Absorbed from x86 VMProtect samples; never executed + against an Android target here (`vm-protector-analysis.md`). +- **Python bytecode / Pyarmor recovery** — the MD5 key schedule, AES-CTR nonce construction, the + 12-byte nonce-XOR VM, version-gated `pyc` parsing and pre-3.8 flag remap, the extra-section TLV, + per-code-object and per-string decryption. Absorbed from one public write-up; no sample was + unpacked here (`python-bytecode-recovery.md`). - **Attaching IDA's remote debug server to a `.so`** — ABI-matched `dbgsrv`, the root server, the wait-for-debugger start plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where the library is mapped from `base.apk` and no breakpoint @@ -200,12 +219,12 @@ its own strength note at the top. ## Not covered — say so rather than improvise -- **A *verified* Unity / IL2CPP route.** `unity-runtime.md` carries the documented route — - backend identification, Il2CppDumper, the IDA import order, the Frida bridge, the dnSpy-Unity-mono - patch — but nothing in it was run in this record, and it is not the IL2CPP equivalent of - `dart-aot.md`: there is no byte-level reading of generated native code here, no opcode map, and no - measurement on a stripped or protected package. Do not present a dump as proof that a method was - located. +- **A *verified* Unity / IL2CPP route.** `unity-runtime.md` and + `il2cpp-protected-metadata.md` carry documented routes — backend identification, Il2CppDumper, the + IDA import order, the Frida bridge, consumer-side metadata recovery and ELF repair — but nothing in + either was run in this record, and neither is the IL2CPP equivalent of `dart-aot.md`: there is no + byte-level reading of generated native code here, no opcode map, and no measurement on a stripped + or protected package. Do not present a dump as proof that a method was located. - **React Native / Hermes bytecode** and Cordova/hybrid internals, beyond runtime identification and the generic "find the string, then find what references it" approach. One carve-out: if the JS layer's problem is **VMP-protected JavaScript** (a jsvmp opcode literal plus interpreter), that is diff --git a/skills/apk-reverse/references/coverage/route-inventory.md b/skills/apk-reverse/references/coverage/route-inventory.md index d945bb8..baf04f3 100644 --- a/skills/apk-reverse/references/coverage/route-inventory.md +++ b/skills/apk-reverse/references/coverage/route-inventory.md @@ -53,6 +53,10 @@ route was not measured on this repository's evidence**. Treat them as leads with | Kotlin original-name recovery | obfuscated→original class map from `@DebugMetadata`/`@Metadata` in any jadx export; parsing fixture-tested, recovery rates upstream-claimed — `kotlin-name-recovery.md`, `kotlin_name_recovery.py` | | Long-connection (WebSocket) analysis | state chains, message families, envelope/payload split — absorbed methodology, never run on an Android target here — `protocol-reverse.md` | | Unity (Mono and IL2CPP) | backend identification, Il2CppDumper over `libil2cpp.so` + `global-metadata.dat`, the IDA import order, a run-time lookup over frida-il2cpp-bridge, the dnSpy-Unity-mono patch — absorbed, nothing run here — `unity-runtime.md` | +| Protected IL2CPP metadata | when the pair is not readable: consumer-side dump, header sizing without trusting the version, XOR/VM/XXTEA containers, the live-image dump and ELF repair — absorbed, nothing run here — `il2cpp-protected-metadata.md` | +| Linker-level SO unpacking and repair | the `dlopen` pipeline as a loader spec, load_bias/segment arithmetic, `DT_*` and relocation rules, `soinfo` repair, anti-dump header XOR — source-reading synthesis, no loader reimplemented — `linker-and-soinfo-repair.md` | +| VM protector analysis | virtual eflags, NAND lowering, branchless select, data-anchored breakpoints and trace search by constant — absorbed from x86 VMProtect samples, never run on Android here — `vm-protector-analysis.md` | +| Python bytecode / Pyarmor | key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing and the extra-section TLV — absorbed, no sample unpacked here — `python-bytecode-recovery.md` | | Attaching a real debugger to a `.so` | IDA `dbgsrv` with the ABI match, wait-for-debugger plus the JDWP handshake, and the `extractNativeLibs="false"` failure where no breakpoint binds — `ida-so-debugging.md` | | Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` | | Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` | @@ -81,7 +85,8 @@ dependency yourself. React Native, Hermes bytecode and Cordova internals · iOS of any kind · defeating a server-side authority · an off-the-shelf unpacker · an anti-detection arms race · building and shipping a kernel module · a Stalker trace guaranteed on every device · **a measured** Unity / -IL2CPP route (the documented one is `unity-runtime.md`, and it carries no measurement). +IL2CPP route (the documented ones are `unity-runtime.md` and `il2cpp-protected-metadata.md`, and neither +carries a measurement). Each has a reason and, where one exists, an evidence boundary — in `references/coverage-and-limits.md`. **Silence in that record is not support**, and a route that is only "documented" is not a route that diff --git a/skills/apk-reverse/references/detection-and-anti-analysis.md b/skills/apk-reverse/references/detection-and-anti-analysis.md index fc41fd1..397dab1 100644 --- a/skills/apk-reverse/references/detection-and-anti-analysis.md +++ b/skills/apk-reverse/references/detection-and-anti-analysis.md @@ -279,6 +279,16 @@ hand showed 21 byte-scan "svc sites" that were all **data**, with no call-number them: a byte scan for `svc` matches inside data, so read the neighbours before believing a count. That distinction — structural absence vs. observed absence — is what Stage 6 is for. +### Timing-delayed checks: hook the delay, not the branch + +A check that appears only after you hook a particular library, and appears **seconds later**, may be +measuring elapsed time rather than reacting immediately. The primitive is usually `sleep` or +`nanosleep`: hook it, rewrite `args[0]` to a large value, and take a backtrace from the hook. If the +delayed popup disappears and the backtrace names one module plus a handful of offsets, those callers +are the next static target. This is lower-risk than patching the checker because it changes the +measurement, not the decision — and it gives you the module to read. (`unverified`: absorbed from one +write-up; not run here.) + ### The routing chain, on one page ``` diff --git a/skills/apk-reverse/references/dynamic-frida.md b/skills/apk-reverse/references/dynamic-frida.md index 9054f24..416dc0e 100644 --- a/skills/apk-reverse/references/dynamic-frida.md +++ b/skills/apk-reverse/references/dynamic-frida.md @@ -223,6 +223,46 @@ Interceptor.attach(f, { }); ``` +### Native registration, load order and late mapping + +Five mechanics that a plain `Interceptor.attach` misses. They are absorbed from a public Frida +cookbook and not measured here; the failure each one prevents is. + +**A `RegisterNatives` table is walked once.** Enumerate `libart.so` symbols, keep names containing +`JNI` but **not** `CheckJNI` (the debug wrapper duplicates the export), attach the `RegisterNatives` +you find, and read `args[2]` as an array of `JNINativeMethod`: name pointer at `+0`, signature +pointer at `+ptr_size`, native entry at `+2*ptr_size`. Because registration happens before +`dlopen` returns, attach-to-running-process sees nothing; spawn with `-f`. + +**A module may not be mapped when the script loads.** Gate the real hook on module load: attach both +`dlopen` and `android_dlopen_ext` (the latter for newer Android), match the path in `onEnter`, and +install the target hook in `onLeave`. A `Module.findBaseAddress` that returns `null` at injection +time is an ordering problem, not a missing library. + +**Read registers at the instruction *after* the one that sets them.** Attaching at `0x...1C` to read +a register written by the instruction at `0x...18` sees the old value on some compilers. Attach to +the next instruction and read `this.context.xN`. For Thumb, pass `base.add(off).add(1)` to the +disassembler, or it decodes in ARM mode and every mnemonic is nonsense. + +**JNI argument shapes are readable without a decompiler.** A pointer argument wants `hexdump`; a +small positive integer is often a length; `&x` is an output buffer; a pointer-to-pointer needs one +more `readPointer()` before the dump. For a `jstring`, use +`ptr(Java.vm.tryGetEnv().getStringUtfChars(args[n])).readCString()`, and the same on `retval`. +If the routine is a C++ member, active invocation needs the right convention: +`new NativeFunction(addr, 'int', ['pointer','int'], 'thiscall')`. + +**Patch the init function, not the exit it calls.** When a library's `DT_INIT` is the check, hooking +`kill`/`exit` is often too late or never reached. Enumerate the linker's symbols for +`__dl__ZL13call_functionPKcPFviPPcS2_ES0_`; on enter, `args[0]` is the tag (`DT_INIT`), `args[1]` +the function, `args[2]` the library path. Match the path and patch the function's first bytes to two +Thumb NOPs (`00 bf 00 bf`) **while `call_function` is running**, before the loader calls it. Patching +after `dlopen` returns is by definition after the check has run. + +**OLLVM does not flatten the call graph.** When the decompiler output is unreadable, prioritize calls +that sit outside `if`/`while` bodies, infer the algorithm family from constants used on +non-conditional paths, and hook the candidate rather than reading the graph. A hook that prints the +input/output shapes is cheaper evidence than a deobfuscated listing. + ### Network / domain observation (cheap and very informative) ```javascript const Inet = Java.use('java.net.InetAddress'); diff --git a/skills/apk-reverse/references/environment.md b/skills/apk-reverse/references/environment.md index 1e90921..d322e76 100644 --- a/skills/apk-reverse/references/environment.md +++ b/skills/apk-reverse/references/environment.md @@ -123,6 +123,16 @@ wrong conclusion is the one that gets written down and trusted later. Emulators are the right place to iterate and the wrong place to conclude. +- **Architecture is not interchangeable.** A VirtualBox-derived AVD is x86_64; an app that ships only + ARM64 libraries may install and then crash at load. Windows Subsystem for Android can run an ARM64 + *app* through its ARM translation layer, but it cannot run an ARM64 `frida-server`; pointing an + x86_64 server at the same process commonly yields no process in the list and no hooks — which looks + like an anti-debug finding and is a host/guest mismatch. Dynamic work on an ARM64-only target + therefore needs a rooted physical ARM device or an Apple-silicon host. For an AVD, a `rootAVD`-style + ramdisk patch plus `-writable-system -selinux disabled` is the documented route; verify with + `su -c id` rather than trusting the boot flag. (Absorbed from a public walkthrough, not measured + here.) + - **Verify the app can install at all.** INSTALL failures on emulators are common (ABI, min SDK, vendor checks, a stale copy with a different signature) and are usually **not** related to your patch. Get a clean install of the *unmodified* build working before you change anything. diff --git a/skills/apk-reverse/references/evidence-summary.md b/skills/apk-reverse/references/evidence-summary.md index ef79914..235ef91 100644 --- a/skills/apk-reverse/references/evidence-summary.md +++ b/skills/apk-reverse/references/evidence-summary.md @@ -74,6 +74,10 @@ next** — and points at the one section that says "the full record is elsewhere | Signature-derived keys | Documented only: neither the offline candidate path nor the live read has a public-target measurement here | partial | unverified | `references/signature-derived-keys.md`, `scripts/sig_probe.py` | | Publishing sanitisation and leak scanning | Every rule fired on a planted corpus with zero false positives on the do-not-anonymize list, and the scan found 26 strong hits in its own evidence file | ok | observed | `references/desensitization-and-leak-scans.md`, `scripts/scan_leaks.py` | | Unity / IL2CPP logic recovery | Documented route only: backend identification, Il2CppDumper over `libil2cpp.so` + `global-metadata.dat`, the IDA import order, the Frida bridge, and the dnSpy-Unity-mono patch. No Unity package was dumped here and no patched Mono runtime was built | partial | unverified | `references/unity-runtime.md`, `references/framework-runtimes.md` | +| IL2CPP protected metadata recovery | Absorbed route only: consumer-side dump, version-independent header sizing, the repeating-key XOR / bytecode-VM / XXTEA container shapes, the live-image dump and the ELF repair that makes it usable. No Unity metadata was decrypted or dumped here | partial | unverified | `references/il2cpp-protected-metadata.md`, `references/unity-runtime.md` | +| Linker-level SO unpacking and `soinfo` repair | Absorbed route only: the `dlopen` pipeline as a loader spec, load_bias and segment arithmetic, `DT_*` and relocation rules, the minimal `soinfo` repair set, anti-dump header XOR, and the dump points. No loader was reimplemented here | partial | unverified | `references/linker-and-soinfo-repair.md`, `references/code-virtualization-and-custom-linkers.md` | +| VM protector analysis (virtual eflags, NAND lowering) | Absorbed from three public VMProtect write-ups: envelope triage, VM anatomy, data-anchored breakpoints, trace-then-search-by-constant, virtual-eflags patching, branchless select and NAND lowering. No VM-protected binary was traced here | partial | unverified | `references/vm-protector-analysis.md`, `references/native-vm-recovery.md` | +| Python bytecode / Pyarmor recovery | Absorbed from one public write-up: the key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing, the extra-section TLV, and per-object/per-string decryption. No sample was unpacked here | partial | unverified | `references/python-bytecode-recovery.md` | | Attaching IDA's debug server to a `.so` | Documented route only: ABI-matched `dbgsrv`, root server, wait-for-debugger plus the JDWP handshake that releases the process, and the `extractNativeLibs="false"` failure where no breakpoint binds. Never run here; neither source targeted a hardened app | partial | unverified | `references/ida-so-debugging.md` | | React Native / Hermes, iOS | Outside this skill: the runtime can be identified, the logic recovery is not covered | blocked | unverified | `references/framework-runtimes.md`, `references/coverage-and-limits.md` | | Defeating a server-side authority | Out of scope by design; report it as a residual rather than patching harder | blocked | unverified | `references/server-api.md`, `references/handoff-boundaries.md` | diff --git a/skills/apk-reverse/references/framework-runtimes.md b/skills/apk-reverse/references/framework-runtimes.md index 4c41e86..b4519ec 100644 --- a/skills/apk-reverse/references/framework-runtimes.md +++ b/skills/apk-reverse/references/framework-runtimes.md @@ -17,6 +17,7 @@ Signals in the APK: | Unity | `libunity.so`, `libil2cpp.so`, `assets/bin/Data/`, `globalgamemanagers` → `unity-runtime.md` for Mono versus IL2CPP | | Cordova / Ionic | `assets/www/`, `cordova.js` | | Xamarin / .NET | `libmonodroid.so`, `assemblies/` in the APK | +| Python (Chaquopy / Kivy / BeeWare) | `libpython3.*.so`, `assets/` with `.pyc`/`.pyz`, `pyarmor`/`__pyarmor__` bootstrap → `python-bytecode-recovery.md` | A `classes.dex` that is small relative to the total payload is another strong hint that the real code lives elsewhere. diff --git a/skills/apk-reverse/references/il2cpp-protected-metadata.md b/skills/apk-reverse/references/il2cpp-protected-metadata.md new file mode 100644 index 0000000..8a00ce6 --- /dev/null +++ b/skills/apk-reverse/references/il2cpp-protected-metadata.md @@ -0,0 +1,278 @@ +# IL2CPP when the metadata is not a readable file + +Load this when a Unity package is IL2CPP but the usual pair is not usable: `global-metadata.dat` is +0 bytes, absent, has the wrong magic, carries a custom container, or Il2CppDumper rejects the +`libil2cpp.so` it came with. This file owns getting *a* parseable metadata image and *a* dumpable +`.so`; `unity-runtime.md` owns backend identification, the normal dump, the IDA import order and +the runtime bridge. Load that first if the pair is plainly readable. + +**Claim strength.** Absorbed from eight public write-ups (2024-2026) and not reproduced here: no +Unity package was decrypted, dumped or re-imported in this record. Every offset, magic value and +tool behaviour below is `unverified` until you reproduce it on your sample; the *ordering rules* +(which anchor to read, which layer to attack, what a failure means) are `inferred` from those +sources. Nothing here is a measurement. Where a number identifies one sample rather than a format, +the text says "one sample" and you must not carry it to another build. + +## 1. Triage: four shapes that all look like "encrypted metadata" + +Before choosing a route, decide which shape you have. They need different work, and retrying the +same dumper is the wrong response to three of the four. + +| Observation | Shape | First move | +|---|---|---| +| File is 0 bytes, or absent, while the game clearly runs | metadata is supplied from memory, a PE resource, or an appended blob | find the consumer, read its output pointer (§2) | +| File exists, `xxd -l 4` is not `af 1b b1 fa` | encrypted or wrapped container | identify the cipher shape (§3) | +| Magic is right, but the dumper reports a version and then produces garbage fields | version field lies, or the layout is spliced | treat the version as a claim, validate the struct (§5) | +| `libil2cpp.so` is present, but every registration search returns 0 | protected/repacked `.so`, or the wrong module is mapped | dump the live image (§4) | + +Two cheap checks come before any reverse engineering: + +```bash +xxd -l 16 assets/bin/Data/Managed/Metadata/global-metadata.dat # 0xFAB11BAF +xxd -l 16 lib/arm64-v8a/libil2cpp.so # 7f 45 4c 46 +``` + +A correct magic proves the first 4 bytes only. A file can pass that check and still carry an +encrypted string blob, a wrong header size field, or a custom program table (§3, §5). Conversely, +a zeroed magic does not prove the file is encrypted: one shipping build wipes magic and version and +embeds the metadata in the `.so` with the rest of the header intact. + +**A 0-byte metadata file is a positive finding, not a missing file.** It means the build does not +use the file at all, so a static decryptor has nothing to decrypt. Look for the loader's output. + +## 2. Anchor on the consumer, not the path + +The reliable address is the pointer *after* decryption and de-obfuscation. Two anchors exist, and +they are not interchangeable: + +- `il2cpp::vm::MetadataLoader::LoadMetadataFile` — the function that returns the metadata base. + Its return value (or the global it stores into) is the plaintext image. +- `il2cpp::GlobalMetadata::Initialize` — a second entry point some builds use for a custom + decrypt-and-validate step. One write-up wasted a diffing pass inside `LoadMetadataFile` before + finding the cipher in `GlobalMetadata::Initialize`. +- `il2cpp::utils::MemoryMappedFile::Map` — the file-mapping layer. A `jmp` from here into a + non-standard section is a signal that the mapping was replaced by a custom loader. + +Procedure: + +1. Find the metadata path string (`global-metadata.dat`) and its xrefs. The consumer is usually + one or two calls away. +2. Diff the function against the matching upstream `libil2cpp` source tag. A short added loop over + the mapped buffer is the decryption, stated in the open. +3. Hook the *return* of the loader, or read the global it writes after the call. Do not hook the + `open`/`mmap` path: that is the ciphertext. +4. Dump `[base, base + size)` using the size rule in §2.1 rather than a guessed length. + +### 2.1 Sizing a header whose fields you do not trust + +The metadata header is an array of 64 `int32` values. For a version whose tail counts are unknown, +walk backwards from `0x10C` in steps of 8: + +```text +i = 0x10C +count = read_i32(base + i) +offset = read_i32(base + i - 4) +if count != 0: size = offset + count; stop +i -= 8 +``` + +This works when magic and version are zeroed, and it survives a version mismatch because it uses +only offset/count pairs that are still laid out in the same order. Bound the loop (for example +`i > 0`), wrap reads in the platform's fault handler on Windows, and treat a non-monotone result as +"this is not the header you think it is", not as a size. + +### 2.2 Metadata outside the filesystem + +Three placements recur: + +- **PE resource, located by a hashed logical name.** One loader hashes the requested name with two + accumulators (an FNV-1a-style `hash *= 0x1000193; hash ^= byte` and a roll + `hash = hash*0x83 + byte`), combines them into a 64-bit key, and compares against constants that + map to a resource id. It then calls `FindResourceA(hModule, id, "CFG")`, `SizeofResource`, + `LoadResource`, `LockResource`. The hash table is small and reimplementable offline with + `pefile`; once the resource is extracted, the container cipher decides the next step. +- **Appended or embedded in `libil2cpp.so`.** No separate file, and no section name that announces + itself. Find the offset from the loader's pointer arithmetic, then cut the blob from the mapped + image rather than from the file if the loader applies a relocation-relative offset. +- **A non-standard section reached by a `jmp`.** The entry stub replaces the mapping function; the + payload lives in a section whose name is product-specific. Treat it as a custom container, not as + ELF data, and use the loader's own parse order. + +## 3. Container and cipher shapes worth trying first + +Do not begin by assuming AES. Public Unity protections in this set are dominated by cheap, +self-describing transforms, and each has a diagnostic that distinguishes it from "real crypto". + +### 3.1 Whole-file repeating-key XOR + +The most common shape: after the mapping call, a loop does + +```c +for (i = 0; i < mapped_size; i++) + mapped[i] ^= key[i & 0x7F]; // 128-byte table, index masked +``` + +Look for `^=` on the mapped buffer in the loader diff. The key is often a fixed byte table in +`.rodata`, not a derived secret. Extraction is mechanical: copy the table, apply it to the file, +check for `0xFAB11BAF`. If the file grows by a few bytes between versions, do not assume the same +key or the same offset; re-diff. + +### 3.2 Custom magic plus a bytecode VM + +One family wraps the header in a container with its own magic (`0x1357FEDA` in one sample) and a +program table, then runs a 255-case `switch` over the body. Each case computes an index into the +destination and applies one of: XOR with an immediate, add, subtract, byte swap, or rotate by a +key byte masked with 7. The layout in that sample was: magic at 0, length at 4, header at 8, +program at `0x108` (64 bytes), data at `0x148`, and the plaintext began 8 bytes into the +decrypted region. + +The generalisable part is not the opcode set; it is that **the VM is a dispatcher over a +self-describing container**. Dump the program table, name each case by its effect, and replay the +program in Python. Do not single-step the VM instruction by instruction in a debugger; the table +is the specification. + +### 3.3 XXTEA with a length word + +A resource-container sample used XXTEA: `delta = 0x9E3779B9`, `rounds = 6 + 52/n`, decrypt from +the tail, key a short ASCII string zero-padded to 16 bytes. The plaintext length was stored in the +**last 32-bit word** and accepted only when it fell in `[total-7, total-4]`. Check that range before +trusting a decrypted buffer; a wrong key usually fails it. + +### 3.4 Length-derived string ciphers + +String literals are often encrypted separately from the header (and may be the only encrypted +region). Two shapes recur: + +- A single-byte key derived from the literal length, for example `key = length ^ 0x2E` applied to + every byte. Long literals may use a vectorised bulk path and short ones a scalar path; both must + produce the same byte-wise result. +- A key derived from counters that are recoverable from the method body, such as + `ilLength * exceptionClauseCount * localVarCount` duplicated into a 64-bit value and rendered as + a decimal string. This appeared in a HybridCLR Interpreter build. + +Try length-derived key material before attacking the cipher. It is cheap, and it explains why a +generic XOR brute force finds nothing. + +## 4. A live image is the fallback, and it is usually cheaper + +When the metadata and the `.so` are both protected per version, reimplementing the loader is more +work than dumping the process. The constraint is that the dump must be *the image the process +mapped*, not the file on disk. + +1. Read `/proc//maps` and take the **second** mapping of `libil2cpp.so`. The first is the + file-backed, still-encrypted view; the second is the anonymous/decrypted image. Dumping the + first produces a file that looks plausible and fails every registration search. +2. Record the mapping base. Dump the whole mapping to a file. +3. Run Il2CppDumper on the dump **with the base address** when auto-detection fails. On one sample + the base `0x7ad74c3000` produced `CodeRegistration 0x7ad85254d0` and + `MetadataRegistration 0x7ad856ff38`; those offsets are sample-specific. +4. If the metadata was decrypted in place at load, dump metadata from the same process as well: + comparing the APK file with the memory image is how the in-place transform is proved. + +The dump loses section headers and dynamic symbols. A disassembler will open it and show no +symbols; that is an ELF-metadata problem, not a bad dump. + +### 4.1 Repairing the dumped ELF + +Set, for every program header: + +```text +p_offset = p_vaddr +p_filesz = p_memsz +``` + +The end of the last `PT_LOAD` is the original section-header offset. In one sample the eleventh +program header ended at `0x13BC000 + 63352 = 0x13CB778`, and that value belonged in `e_shoff`; +that arithmetic is the reusable part, not the address. + +Then: + +- `e_shstrndx` names a section whose `sh_addr` is 0 because it is not file-resident. Find the + neighbouring section with the same `sh_offset` and a valid `sh_addr`; copy that address over the + broken field. +- Set every section's `sh_offset = sh_addr`. +- Copy the section-header bytes into the file if they were not dumped, then re-analyse. +- Rebase the database to the dump's load address **before** importing headers or scripts. + +After repair, import the IL2CPP tables in the order `unity-runtime.md` §4 gives. Running the +struct script twice (once with `script.json` + `il2cpp.h`, once with `stringliteral.json`) is the +order that names both functions and literals. + +## 5. When the version field lies + +A header can say version 31 while its tail is a long run of zeros and it lacks the fields that +version introduced. In one build, three separate failures traced to one integer: + +- The dumper auto-located `CodeRegistration` at a wrong address; setting it manually was required. +- Metadata claimed 31, but the struct was a 29 layout; patching the version constant to 29 made the + dumper proceed. +- A `methodDef` parse failed because one field was read at `+0x20` where the build used `+0x24`. + Lowering the version constant in the dumper source fixed the stride. +- A large `libil2cpp.so` can make `genericMethodPointersCount` look like a 29.1 layout even when the + version was already forced to 29; force the constant in the dumper rather than accepting the + heuristic's label. Size is not a version. + +**The version integer is a claim; the struct layout is evidence.** Validate a candidate layout by +plausibility: string offsets inside the file, counts that fit the remaining bytes, section tables +that are monotone, and a first string that is a real identifier (`mscorlib` is a useful landmark). +Do not "upgrade" a dump because the version says so. + +A related trap: after a successful dump, only one region may be wrong. If the string blob's first +entry is garbage while the rest of the header is coherent, the blob is decrypted in place at load +(§4); do not discard the whole metadata image. + +## 6. String literals: index space, encoding, re-import + +When only literals are encrypted, the header arithmetic is stable: + +- entry size is `8` bytes: ``; +- entry `i` is at `stringLiteralOffset + i*8`; +- the bytes are at `stringLiteralDataOffset + dataIndex`, `length` bytes; +- the count is `stringLiteralSize / 8`. + +The recovered value is UTF-8 in the metadata, but the runtime `Il2CppString` is UTF-16LE. When +hooking `il2cpp::vm::String::NewLen`, convert UTF-16LE to UTF-8 **before** writing back, or the +re-imported metadata is silently corrupt and the dumper still accepts it. + +Dump `index,hex` to CSV, decrypt offline, patch the metadata file, then re-run Il2CppDumper. This +keeps the proof in one place: the tool either parses the patched file or it does not. + +## 7. Adjacent artifacts that fail the same way + +These are not metadata, but they are the next files the same build protects, and the diagnostic +shape is shared. + +- **AssetBundle header patched by the engine.** One build reports a fabricated Unity version + (suffix `p1`) and XORs the first 32-bit field of each block and each directory node's size with + `0x1024`. The fix is in the reader (AssetStudio), not in the bundle: apply the same XOR before + parsing. +- **Repeating-key XOR over bundles.** With one plaintext sibling, XOR the two headers to recover + the 16-byte key. Only the first few tens of KB need decrypting for the tooling to accept the + file; the body can stay encrypted until an asset is actually needed. +- **Byte-inversion AssetBundle wrapper.** A custom `Stream` skips 4 bytes and inverts the next 5, + then a second wrapper inverts bytes 0..4 of every 8-byte block, applied only to the 128-byte + header. Script it as a header transform; do not invert the whole file. +- **CRC32-named paths and blend shapes.** Names in animation data are CRC32 of the full path, + displayed as bare integers. Recover the prefix from uncompiled sources or the middleware's SDK, + then build the name→CRC map. A hash is not encryption, but without the prefix it is not a name. + +## 8. Order of attack + +1. Decide the shape (§1). +2. If the metadata is file-resident and cheaply wrapped, decrypt statically (§3) and validate the + magic and the first string. +3. If the loader is per-version or the file is absent, dump the consumer's output and the second + `.so` mapping (§2, §4). +4. Repair the ELF only if a disassembler must consume the dump (§4.1). +5. Validate the metadata version against the struct before trusting any field (§5). +6. Recover literals last, and re-import them into the tool that must parse the result (§6). + +## 9. Boundary + +Every procedure here stops at a dependency you must supply: a device that can run the game, a +dumper version that matches the metadata layout, or the loader's own container format. None was +exercised in this record. The steps that would make a row `observed`: decrypt one file and show +the magic and first string, dump one live image and show a named function in the disassembler, and +re-import one literal table and show the dumper accepting it. Until then, treat the offsets as +examples, not as constants, and do not present a repaired dump as a working target without running +the feature it belongs to. diff --git a/skills/apk-reverse/references/linker-and-soinfo-repair.md b/skills/apk-reverse/references/linker-and-soinfo-repair.md new file mode 100644 index 0000000..4d1369d --- /dev/null +++ b/skills/apk-reverse/references/linker-and-soinfo-repair.md @@ -0,0 +1,210 @@ +# Loading, dumping and repairing a protected `.so` at the linker level + +Load this when a custom loader or packer sits between the APK and a usable native image: an +unpacked `.so` will not `dlopen`, a dump validates as ELF but crashes on load, a repaired +`soinfo` points at garbage, or you need to know *where* in the load pipeline it is safe to take a +dump. `native-and-so.md` owns patching and which library is actually loaded; +`code-virtualization-and-custom-linkers.md` owns recognising the loader and the keep-it/drop-it +deadlock; `advanced-unpacking.md` owns the dump routes above this layer. This file is the linker +contract those files assume. + +**Claim strength.** The load/link pipeline and field layouts are absorbed from two public +source-reading write-ups (Android 5.0 and 8.x `bionic/linker`) and one Frida cookbook; no loader +was reimplemented and no dump was produced here. Source citations are `unverified` against your +tree until you open the matching AOSP tag. The *arithmetic* (bias, page offset, relocation target) +is `inferred` from that source and should be checked with one control load before it is trusted. +The `soinfo` offsets quoted below are explicitly version-specific and are examples, not constants. + +## 1. The pipeline, as a specification for a loader + +A loader must reproduce what `bionic` does without assuming a filesystem-backed file. The chain, +with the units that matter: + +```text +dlopen + -> do_dlopen + -> find_library + -> find_library_internal + -> find_loaded_library_by_soname (already loaded? return its soinfo) + -> load_library + -> ElfReader::Read ReadElfHeader / VerifyElfHeader + ReadProgramHeaders / ReadSectionHeaders + ReadDynamicSection + -> ElfReader::Load ReserveAddressSpace / LoadSegments / FindPhdr + -> soinfo_alloc link into the global soinfo list + -> soinfo_link_image prelink + relocate + -> CallConstructors dependencies first, then own init/init_array +``` + +A custom loader that mirrors these eight calls is functionally a linker. That is why "the shell +decrypts a `.so` and calls `dlopen` on the plaintext" is the easy case, and "the shell never +writes a valid file" is the case that needs the field-level repair below. + +`VerifyElfHeader` rejects: wrong magic, class mismatch, endianness, `e_type`, version != 1, +machine mismatch. A packer that corrupts `e_ident` cannot rely on system loading at all, so it +must carry its own loader. That is a useful diagnostic: if the file's header is deliberately +invalid, look for a loader before trying to repair the file for `dlopen`. + +`load_library` reads `DT_RUNPATH` and `DT_SONAME` from the dynamic array *before* `prelink_image` +overwrites them from `PT_LOAD` data. A reimplementation that skips this loses the `soname` used by +`find_loaded_library_by_soname`, and a second `dlopen` of the same library then loads a duplicate. + +## 2. Address arithmetic that every repair depends on + +### 2.1 Load size and bias + +```text +min_vaddr = min(p_vaddr for PT_LOAD) +max_vaddr = max(p_vaddr + p_memsz for PT_LOAD) +load_size = page_align_up(max_vaddr) - page_align_down(min_vaddr) +load_bias = load_start - page_align_down(min_vaddr) +``` + +For an ordinary shared object `min_vaddr == 0`, so `load_bias == load_start`. For a protected +image with a non-zero first segment, the difference is exactly the bias. **Every** `vaddr` to +runtime conversion adds `load_bias`, never the mapping base, and a repair that writes `base` where +`bias` is required produces addresses off by the first segment's page. + +### 2.2 Mapping one segment + +```text +seg_start = page_align_down(p_vaddr + load_bias) +seg_end = page_align_up(p_vaddr + load_bias + p_memsz) +file_page_start = page_align_down(p_offset) +file_length = page_align_up(p_offset + p_filesz) - file_page_start +mmap(seg_start, file_length, prot, flags, fd, file_page_start) +``` + +For a memory-sourced payload the sequence is: reserve anonymous memory, then `memcpy` into +`seg_start + seg_page_offset`, where `seg_page_offset = p_offset & (page_size - 1)`. Omitting +`seg_page_offset` is the classic silent corruption: the segment is page-aligned on the file side +but its first bytes belong at a non-zero offset inside the destination page. + +## 3. Parse `DT_*` first; treat `soinfo` as volatile + +The dynamic section is the stable ABI. `prelink_image` is a `switch(d_tag)` terminated by +`DT_NULL`: + +| Tag | Meaning | Address form | +|---|---|---| +| `DT_HASH` | SysV hash table: `nbucket`, `nchain`, `bucket[]`, `chain[]` | `load_bias + d_ptr` | +| `DT_GNU_HASH` | GNU hash: `nbucket`, `symndx`, `maskwords`, `shift2`, bloom, buckets, chain | `load_bias + d_ptr` | +| `DT_STRTAB` / `DT_STRSZ` | string table and size | pointer uses bias, size is a value | +| `DT_SYMTAB` / `DT_SYMENT` | symbol table; `DT_SYMENT` must equal `sizeof(ElfW(Sym))` | pointer uses bias | + +Both hash formats must be handled: a repaired `soinfo` that carries only the one the shell happens +to use will fail `dlsym` for symbols that exist. `DT_GNU_HASH` validates that `maskwords` is a +power of two and stores `maskwords - 1`. The symbol count is not stored in the GNU format; it is +derived from the highest chain index, which is why "the symbol table looks one entry short" is a +parse bug, not a stripped export. + +## 4. Relocation rules and the two biases + +| Type | Value written | Note | +|---|---|---| +| `R_*_RELATIVE` | `load_bias + addend` | no symbol | +| `R_*_GLOB_DAT` | `sym_addr + addend` | data import | +| `R_*_JUMP_SLOT` | `sym_addr + addend` | `bionic` processes it like a normal reloc; lazy binding is not a separate path | +| `R_*_IRELATIVE` | `ifunc_resolver(load_bias + addend)` | may need temporary exec permission when the page is read-only | + +Two addresses are in play and mixing them is the most common silent bug: + +- the **definition**: `sym_addr = symtab[sym].st_value + defining_soinfo->load_bias`; +- the **slot**: `write_addr = r_offset + caller_soinfo->load_bias`. + +A symbol resolved from a dependency uses the *dependency's* bias. A relocation in the caller uses +the *caller's* bias. They are equal only when both libraries loaded at the same base. + +### 4.1 Packed relocations + +If the table pointed to by `DT_ANDROID_REL`/`DT_ANDROID_RELA` begins with `"APS2"`, the body is +SLEB128-packed and must be read with the packed-relocation iterator starting after the 4-byte magic. A loader that assumes +plain `ElfW(Rel)` arrays produces a GOT full of garbage that still "looks like" relocated data. +This is an easy accidental symptom after hand-writing a loader; check the tag's first four bytes +before interpreting the table. + +## 5. Repairing `soinfo` after an in-memory load + +A loader that maps and relocates the payload still has to hand the result to the runtime. The +minimal field set one write-up reports as sufficient: + +```text +base, size, load_bias +sym_tab, str_tab +nbucket, nchain, bucket, chain +ARM_exidx, ARM_exidx_count +``` + +The acquisition method is `dlopen(self, RTLD_NOW)` for the loader's own `soinfo`, then overwrite +the fields with the protected image's values. Two cautions: + +- **Offsets are version-specific.** One 64-bit example walked the global `solist` with `next` at + `+40`, `name` at `+408`, `base` at `+16`, `size` at `+24`, `load_bias` at `+256`. Those + numbers hold for one cited build. Resolve `solist` by symbol + (`__dl__ZL6solist`) and take the layout from the matching AOSP tag, or use the `dlopen` handle + instead of walking the list. +- **A repair is not a load.** Overwriting `soinfo` makes symbols resolvable through the linker; + it does not run constructors, apply `DT_NEEDED` dependencies, or register the image with the + runtime's own bookkeeping. If the feature calls into the library before constructors would have + run, it fails in a way that looks like a bad relocation. + +## 6. Where it is safe to take a dump + +The dump is valid only in the window where segments are mapped but relocation is not complete, or +after relocation if you also record the bias. Four practical hook points: + +1. **`mprotect`** — flag calls granting `PROT_EXEC` on a large range. The "allocate then make it + executable" pattern is a stager. The same call is used offensively: skip the exec grant when a + debugger is present, which is a reason a hook may see the allocation and not the execution. +2. **`mmap`** — large file-backed mappings are likely the payload; anonymous large mappings are + likely the decrypted image. Record `(addr, len, prot)`. +3. **`ElfReader::LoadSegments` / the loader's segment-copy loop** — after the copy, before link + fixups. This is the cleanest point because the image is complete and the loader's own state is + still available. +4. **The `soinfo` list** — enumerate every loaded object and record `base`, `size`, `load_bias`. + This gives the dump bounds without knowing the packer, and it is version-portable if you resolve + the list head by symbol. + +Record the bias with the dump. A dump without its bias is only usable if you can recover the first +segment's `p_vaddr` from the program headers and prove the alignment. + +### 6.1 Anti-dump by header corruption + +A shell may XOR selected `PT_LOAD` fields after loading (`p_vaddr`, `p_memsz`, `p_filesz`, +`p_offset` with one constant) and restore them at use time. The dump then fails ELF validation +even though the code is intact. Detect it with invariants rather than by guessing the constant: + +- `p_vaddr` is page-aligned for a loadable segment; +- `p_filesz <= p_memsz`; +- `p_offset` is monotone across program headers and page-aligned in the usual way; +- the first `PT_LOAD` covers the ELF header for a normal executable. + +Test a candidate constant by applying it to all four fields and re-checking those invariants. A +uniform XOR survives every check that looks at one field alone. + +Deleting the header instead of XORing it is a weaker defence but a nastier symptom: the file is +*cleanly* rejected, so the analyst blames the dump site. If the dump site is right and the header +is missing, the correct conclusion is "the shell removed it", not "the dump was early". + +## 7. Timing: constructors and one-shot registrations + +`dlopen` does not return until `CallConstructors` has run the dependencies' constructors, then the +library's `init`, then its `init_array`. Two consequences: + +- A breakpoint on `do_dlopen` or `CallConstructors` catches unpacking before user code runs. +- A hook installed after the library is already loaded misses every one-shot registration. A + `RegisterNatives` table is walked once; spawn the process under instrumentation or gate on + `dlopen`/`android_dlopen_ext`, and run the real hook in `onLeave` when the module is mapped. + +The `dlopen` symbol name differs by Android version; hook both `dlopen` and +`android_dlopen_ext`. A module that is mapped but not yet resolvable returns no base address, and +that is a timing failure, not a missing module. + +## 8. Boundary + +This file describes the contract, not a working unpacker. Nothing here was compiled against a +device's `linker` or used to load a real protected library. The steps that would make a row +`observed`: load a trivial `.so` from memory with the arithmetic above, show one symbol resolved +through the repaired `soinfo`, and dump one protected image at `LoadSegments` and open it in a +disassembler with symbols. Until then, treat every offset as a lead and verify the bias with a +control library that loads normally. diff --git a/skills/apk-reverse/references/native-vm-recovery.md b/skills/apk-reverse/references/native-vm-recovery.md index dee6b7b..baf3d19 100644 --- a/skills/apk-reverse/references/native-vm-recovery.md +++ b/skills/apk-reverse/references/native-vm-recovery.md @@ -21,6 +21,9 @@ None of the signals below proves a VM alone — a big switch, a state machine, a table and an event dispatcher all look similar. The judgement is **several appearing together**: 1. one large native function, called often, taking a chunk of external data; + +If the missing piece is not the dispatcher but *what a handler computes* — no `cmp`/`sub`, a flags +slot written to the virtual context, `and/or/not` runs — `vm-protector-analysis.md` owns that idiom. 2. a loop that reads a cursor from one buffer, dispatches on the value read, and writes the cursor back — the `while(running){ opcode=code[pc]; execute(opcode); pc=next_pc; }` shape; 3. a **virtual register file**: many paths access `base + index*8` (or *4, *2) where *index* @@ -49,6 +52,22 @@ proves nothing about which layer it is. Judge by behaviour, not shape: Mislabelling the CFF table as `VM_HANDLER_TABLE` poisons everything downstream: every opcode→handler pair after it is wrong. The write-up's first correction was exactly this. +### 2.1 Encoded handler tables + +A table entry does not have to be a pointer. One ARM64 sample stored 64-bit values that only become +targets after adding a constant (`table[i] + 0x740078FC`, yielding offsets such as `0x3BA04`). +Resolve both arms arithmetically before deciding which one is semantically required, then patch the +conditional, not the table. When a callee is patched, undefine and rebuild the caller, or the +decompiler keeps the stale target and the next reading is wrong. + +### 2.2 A dispatcher invariant worth exploiting + +When the obfuscator routes every transfer through one register (`r9` in one sample, `rdx` in +another) and the landing instruction is a benign `nop`, enumerate real blocks by filtering the trace +for "indirect jump through that register whose target begins with `nop`". That skips symbolic +simplification of the MBA expressions entirely. The register is sample-specific; the invariant is the +benign landing instruction. + ## 3. Do not fully deflatten first — chase the VM data flow Full native-CFG recovery of a flattened interpreter is expensive and unnecessary. What the VM diff --git a/skills/apk-reverse/references/protocol-reverse.md b/skills/apk-reverse/references/protocol-reverse.md index 94f2218..61a004f 100644 --- a/skills/apk-reverse/references/protocol-reverse.md +++ b/skills/apk-reverse/references/protocol-reverse.md @@ -138,6 +138,21 @@ The measured record for this tool -- every command, both input forms, the round framing mistake it does *not* protect you from -- is in `references/evidence-summary.md` §The capability matrix. +### 1.1 If the body is self-describing, skip the schema hunt + +Not every binary body is protobuf. **MessagePack** is self-describing: the first byte is a type and +length tag, and maps/arrays carry their own shape, so a decoder needs no `.proto` and no generated +code. Plausible leading tags are fixmap `0x80`-`0x8f`, fixarray `0x90`-`0x9f`, positive fixint +`0x00`-`0x7f`, `str8 0xd9` and `map16 0xde`; the confirmation is a full decode with no leftover +bytes. Re-encode with the same library (`packb`/`unpackb`) and re-encrypt with the app's own +routine. + +**Prefer mutating the outbound request over the inbound response.** A response edit can make the UI +render an unlocked state while the server still authorizes the live entry, so the change is visible +and ineffective; a request edit changes what the server is asked to do. When both directions are +available, test both once and record which one the server re-derives. (`unverified`: absorbed from +one write-up; no MessagePack target was captured here.) + ## 2. Recovering the schema from the APK ### What generated protobuf code looks like diff --git a/skills/apk-reverse/references/python-bytecode-recovery.md b/skills/apk-reverse/references/python-bytecode-recovery.md new file mode 100644 index 0000000..e486ae6 --- /dev/null +++ b/skills/apk-reverse/references/python-bytecode-recovery.md @@ -0,0 +1,205 @@ +# Python bytecode behind a packer: Pyarmor and the `.pyc` route + +Load this when the logic you need is not Java, Kotlin, Dart or native, but Python: the APK embeds a +Python runtime (Chaquopy, Kivy, BeeWare), ships a `.pyc`/`.pyz`, or the companion desktop tool +that produces the data you are analysing is a PyInstaller/Pyarmor build. This file covers static +recovery of Pyarmor-protected bytecode; `emulation-and-rpc.md` owns calling the recovered routine +instead of reading it, and `framework-runtimes.md` owns deciding whether the runtime is Python at +all. + +**Claim strength.** Absorbed from one detailed public write-up on a Pyarmor build and a modified +`pycdc` (2026). The field layouts, flags and opcode numbers are `unverified` here; no Pyarmor +sample was unpacked in this record. The general rules (static-first, self-describing TLV, +mirror-the-host-codegen) are `inferred` from that source. Version numbers matter: every branch +below is gated on the `pycdc`/Python version, and applying a newer layout to an older file fails +silently rather than loudly. + +## 1. Recognition + +| Signal | Meaning | +|---|---| +| `pyarmor-vax-\d{6}` string | Pyarmor build tag; `000000` is the free build, other values are a customer id | +| A `.pyc` whose code object starts with a private opcode or a high-entropy body | encrypted code object | +| `pyarmor` runtime module, `__pyarmor__` import, or a bootstrap that loads a shared object | protected distribution | +| `m_flags & 0x20000000` on a code object | Pyarmor's `CO_PYARMOR_OBFUSCATED` marker | +| A string blob where the first byte has a small tag and the rest is random | per-string encryption | + +The packer is not a general-purpose VM. It is a key schedule plus AES-CTR plus a small bytecode +VM that derives one XOR key. That makes the static route the cheap one: you do not need to run the +program to recover its code. + +## 2. Key schedule + +One observed build derives the master key with MD5 over four concatenated inputs: + +1. the ASCII build tag `pyarmor-vax-{6 digits}`; +2. `RSA_KEY1`: the blob at file offset 0, length `0x10E`; +3. `RSA_KEY2`: the blob at `0x110 + 0x20 = 0x130`, length taken from the 4-byte field at + header offset `+4`; +4. a 16-byte value XORed with `0xF1` first. + +`RSA_KEY2`'s first 12 bytes are the `AST_nonce` used later. The resulting 16-byte digest is the +AES-128 key; the nonce plus the code metadata select the CTR offset. The exact blob offsets belong +to the observed version; read them from the bootstrap, do not assume them for another build. + +### 2.1 The 12-byte nonce-XOR key + +When a header field `xor_key_procedure_length` is non-zero, the packer runs a tiny VM over a +12-byte output. The VM is worth knowing because its bugs become your key: + +```text +registers[8], only indices 0..5 and 7 are valid (one build remapped 15 -> 7) +cursor starts at in + 16 +opcode 1: terminate +opcode 2,3,4,6,7: += -= *= ^= = (destination is cur[1] >> 4) +opcode 5: /=, then overwrites registers[0] with the quotient — the x86_64 lowering + keeps rax as both the dividend and the result +opcode 8: load, cursor += 2 +opcode 9: store registers[cur[1] & 7] as 4 bytes at out+0 +opcode 0xA: address setup/store, cursor += 6 +opcode 0xB: store 4 bytes at out + cur[2], cursor += 3 +unknown opcode: zero the 12-byte output and abort +``` + +The `idiv` side effect is the general lesson: when reimplementing a packer's VM, mirror the *host +code generation*, including register clobbers the author did not intend. A clean abstract +interpreter will produce a different key and every downstream field will decrypt to noise. + +## 3. Parsing the code object + +The protected file is a normal `.pyc` with extra fields, so the parser must be version-aware. +Fields that change and the versions that introduced them: + +| Field | Width / condition | +|---|---| +| `m_argCount` | 16-bit for 1.3-2.2, 32-bit from 2.3 | +| `m_posOnlyArgCount` | present from 3.8 | +| `m_numLocals` | present below 3.11 | +| `m_localKinds` | present from 3.11 | +| `m_qualName` | present from 3.11 | +| `m_exceptTable` | present from 3.11 | + +For files older than 3.8, the flags field is remapped before use: + +```text +m_flags = (m_flags & 0x1FFF) + | ((m_flags & 0xFFFE000) << 4) + | (m_flags & 0x20000000) +``` + +Warn if `m_flags & 0xD0000000` is set: those bits are not part of the layout this parser knows. +Getting the remap wrong moves every subsequent field by four bytes and the failure looks like a +corrupt string table, not a version error. + +## 4. The extra section: a self-describing TLV + +After the code object's usual fields: + +```text +1 byte extra_length +256 bytes extra_data +``` + +Decode it as a small TLV, not as fixed offsets: + +```text +if extra_data[0] & 0xF0: reject (unknown feature bits) +fn_count = extra_data[0] & 3 +descriptor_count = (extra_data[0] >> 2) & 3 +ptr = extra_data + 4 +for each item: + item_len = (*ptr >> 6) + 2 ; low 6 bits are flags + ... +``` + +Skip `fn_count` items, then walk `descriptor_count` items. For a descriptor item, ignore the low +6 bits of each byte and big-endian accumulate the remaining bits into a `consts_index`. That index +selects a string in the code object's constants; the string is the `descriptor`, and a descriptor +shorter than 20 bytes is invalid. The layout is self-describing precisely so the packer can change +the counts without changing the parser; hard-coding `fn_count = 1` breaks on the next build. + +## 5. Decrypting one code object + +The descriptor's first 8 bytes are not data: at run time they hold `&PyCodeObject`, and the +unpacker replaces them with the literal string `""`. The rest is a struct at +`descriptor.data() + 8`: + +```text +flags = *(u32*)struct +copy_prologue = flags & 0x8 +xor_aes_nonce = flags & 0x4 +short_code = flags & 0x2 +decrypt_begin = ... +decrypt_length = ... +short_nonce_index= ... +``` + +Then: + +1. Choose the nonce source index: + `short_code ? short_nonce_index : short_nonce_index + decrypt_begin + decrypt_length`. +2. Copy 12 bytes from `m_code + nonce_index` into a zeroed 16-byte nonce, then set + `nonce[15] = 2`. +3. If `xor_aes_nonce`, XOR those 12 bytes with the 12-byte VM key from §2.1. +4. AES-128-CTR decrypt `code[decrypt_begin .. decrypt_begin + decrypt_length]`. +5. If `copy_prologue`, copy `decrypt_begin_index` bytes from `code + decrypt_length` to `code + 0`, + then fill `code[decrypt_length .. decrypt_length + decrypt_begin_index]` with `0x09` (NOP). +6. Replace the descriptor's first 8 bytes with the `""` placeholder so a later re-run does + not treat runtime data as a key. + +The nonce construction is the part that is easy to get wrong: `nonce[15] = 2` is a domain +separator, and using an all-zero tail produces a stream that decrypts the first block plausibly and +then diverges. + +## 6. Decrypting strings + +Strings are separate from code and use a tag byte: + +```text +v[0] & 0x80 == 0 -> not encrypted (skip) +v[0] & 0x7F == 0 -> empty/plain (skip) +(v[0] & 0x7F) > 4 -> not an encrypted string (skip) +otherwise: + drop byte 0 + nonce = 12 bytes of pyarmor_mix_str_aes_nonce, nonce[15] = 2 + AES-CTR with the same master key +``` + +The tag is a type code, not a length. Treating the low 7 bits as a length is the usual way a +reimplementation decrypts some strings and corrupts the rest. + +## 7. What to do with the result + +- A recovered `.pyc` can be decompiled by a version-matched decompiler, but the bodies are only + useful if the decompiler understands the Python version's opcodes. Pin both. +- If the decompiler fails on one code object, keep the decrypted bytes and read the disassembly + (`dis`) rather than re-attacking the packer. The packer is solved once; the decompiler is a + separate problem. +- If the goal is *calling* the routine rather than reading it, load the recovered module in the + matching Python and call the entry point, or use the emulation/RPC route. Do not rebuild the + cipher from a partial read when the decrypted function is in front of you. + +## 8. Failure modes + +| Symptom | Likely cause | +|---|---| +| Descriptor shorter than 20 bytes | wrong `consts_index` (bit accumulation ignored the low-6-bit skip) | +| Header fields shift by 4 after `m_flags` | wrong version remap (pre-3.8) | +| First block decrypts, rest is noise | nonce tail not zeroed + `nonce[15]` not set, or CTR offset wrong | +| VM key differs from the packer's | host-codegen clobber (`idiv`) not modelled | +| Some strings decrypt, others do not | tag byte treated as length, or the `> 4` guard missing | +| Code object decrypts but prologue is garbage | `copy_prologue` flag ignored, or NOP fill used the wrong length | + +## 9. Boundary + +This route is static and offline, which is its advantage: no device, no runtime, no root. It also +means every claim about a field is a claim about one Pyarmor version. Nothing here was run against +a real sample in this record. The steps that would make a row `observed`: parse one `.pyc`, show +the extra section counts matching the file, decrypt one code object, and execute the recovered +module to the same output as the original. Until then, treat the offsets as a template and keep the +original bytes so a wrong branch can be retried without re-extracting the sample. + +For a packaged Android app, the Python payload may sit in `assets/` or be extracted on first run. +Locate it the same way as any other runtime: find the loader (the Python runtime's initialization +call), then the payload path it builds. `recon.md` owns that search; this file begins once a +protected `.pyc` is in hand. diff --git a/skills/apk-reverse/references/routing.md b/skills/apk-reverse/references/routing.md index e64bebc..1d965e5 100644 --- a/skills/apk-reverse/references/routing.md +++ b/skills/apk-reverse/references/routing.md @@ -85,8 +85,13 @@ reference file is not named here, or when a script is not named here. | A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK | | A debugger lists and attaches to the process, but the target `.so` is absent from the module list and every breakpoint in it refuses or never hits, while its feature works | `ida-so-debugging.md` §the `.so` was never extracted — `android:extractNativeLibs="false"` maps the library from `base.apk`, so there is no `/data/app//lib//` file to bind to. Not an anti-debug finding | | A Unity package has no `Assembly-CSharp.dll` — only `libil2cpp.so` and `global-metadata.dat` — and every Java-layer hook turns out to be glue | `unity-runtime.md` — that is IL2CPP, the normal shipping configuration, not a stripped build: dump the pair with Il2CppDumper and import `script.json` before `stringliteral.json`, or the database is silently half-named | +| A Unity IL2CPP package has **no readable `global-metadata.dat`** — 0 bytes, absent, wrong magic, a custom container, or a metadata version that disagrees with its own struct | `il2cpp-protected-metadata.md` — anchor on the consumer's output pointer, size the header without trusting its version, and recognise the XOR/VM/XXTEA shapes; `unity-runtime.md` owns the readable case | +| You dumped `libil2cpp.so` from memory but the disassembler shows no symbols, or every registration search returns 0 | `il2cpp-protected-metadata.md` §4.1 — `p_offset=p_vaddr`, `p_filesz=p_memsz`, `e_shoff` from the last `PT_LOAD` end, `e_shstrndx`, `sh_offset=sh_addr`; the dump is fine, the ELF metadata is not | +| A `dlopen`-able `.so` will not load after a custom loader, or a repaired `soinfo` resolves symbols to garbage | `linker-and-soinfo-repair.md` — bias versus base, `seg_page_offset`, the defining library's bias, `APS2` packed relocs, the minimal `soinfo` field set, and the dump window after `LoadSegments` | +| A native trace has no `cmp`/`sub`/`jcc` — only `and/or/not/shr/neg`, a flags slot, and a `ret` that lands at a VM entry | `vm-protector-analysis.md` — virtual eflags is data, NAND lowering, branchless select, data-anchored breakpoints, and trace-then-search-by-constant; dispatcher slicing stays in `native-vm-recovery.md` | +| The logic you need is Python bytecode — a `.pyc`/PyInstaller/Pyarmor payload rather than Java, Dart or native | `python-bytecode-recovery.md` — key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing and the extra-section TLV; static and offline, then call the recovered module | -_65 row(s) below the header._ +_70 row(s) below the header._ ## Reference index (no longer inline in `SKILL.md`) @@ -150,9 +155,13 @@ _65 row(s) below the header._ | `references/on-device-tooling.md` | Working **from the phone itself**: MT Manager edit/repack/sign and its built-in APK MCP, LSPosed Manager, Termux+frida, and on-device data inspection | | `references/split-apk.md` | The target is a **split APK / App Bundle set** (`base.apk` + `split_config.*.apk`), or `pm path ` returned several files: reading a set, merge versus unified re-signing, and the install refusal each mistake produces | | `references/unity-runtime.md` | The package is **Unity**: telling Mono from IL2CPP, dumping `libil2cpp.so` + `global-metadata.dat` with Il2CppDumper and importing it into IDA in the order that works, a run-time lookup over frida-il2cpp-bridge, the dnSpy-Unity-mono patch for a Mono player, and the audit probes when the deliverable is a report | +| `references/il2cpp-protected-metadata.md` | A Unity IL2CPP pair is **not readable**: `global-metadata.dat` is absent, 0 bytes, encrypted, embedded, or its version lies. Consumer-side dump, version-independent header sizing, the XOR/VM/XXTEA container shapes, the live-image dump, and the ELF repair that makes a dump usable | +| `references/python-bytecode-recovery.md` | The logic is **Python bytecode** behind Pyarmor or PyInstaller: key derivation, AES-CTR nonce rules, the 12-byte VM, a version-gated `pyc` parser, the extra-section TLV, and per-code-object / per-string decryption | +| `references/linker-and-soinfo-repair.md` | A protected `.so` must be loaded or dumped **by hand**: the `dlopen` pipeline as a loader spec, load_bias and segment arithmetic, `DT_*` parsing, relocation rules and the two biases, the minimal `soinfo` repair set, anti-dump header XOR, and where a dump is valid | +| `references/vm-protector-analysis.md` | A native function is a VM protector with **virtual eflags**: no `cmp`/`sub`/`jcc`, only `and/or/not/shr/neg` and a flags slot. Envelope triage, VM anatomy, data-anchored breakpoints, trace-then-search-by-constant, NAND lowering, and branchless select. Handler slicing stays in `native-vm-recovery.md` | | `references/ida-so-debugging.md` | You need to **single-step or breakpoint a `.so`** rather than hook it: IDA's `dbgsrv` server with the ABI match, the wait-for-debugger start plus the JDWP handshake that actually releases the app, and the `extractNativeLibs="false"` failure where no breakpoint can bind | -_58 row(s) below the header._ +_62 row(s) below the header._ ## Script index (no longer inline in `SKILL.md`) diff --git a/skills/apk-reverse/references/server-api.md b/skills/apk-reverse/references/server-api.md index 959ebe4..47ec61f 100644 --- a/skills/apk-reverse/references/server-api.md +++ b/skills/apk-reverse/references/server-api.md @@ -2,6 +2,11 @@ If the behavior you want to change is produced by a response, patch the client all you like: nothing real changes. Establishing server authority early is the cheapest high-value step in any "crack this" task. +**Test the outbound direction first.** Rewriting a response can make the client render a state the +server never granted; rewriting a request changes what the server is asked to do. When you have both +proxies available, mutate the request once and the response once, and keep the one whose effect +survives a reload. A response-only success is a UI fact, not an authority fact. + ## Harvest the API surface from the client ```bash diff --git a/skills/apk-reverse/references/unity-runtime.md b/skills/apk-reverse/references/unity-runtime.md index 5178b15..d8d1f18 100644 --- a/skills/apk-reverse/references/unity-runtime.md +++ b/skills/apk-reverse/references/unity-runtime.md @@ -130,6 +130,9 @@ the same rule as `pitfalls.md` P47. - **Encrypted metadata** (`xxd` shows the wrong magic): dump at run time with Zygisk-Il2CppDumper, or recover the loader's decryption (`packers.md`). A static dumper will never succeed here, and retrying with another one is the wrong move. + When the file is absent, 0 bytes, embedded, or wrapped in a custom container, `il2cpp-protected-metadata.md` + owns the consumer-side dump, the header sizing rule and the container shapes; this file resumes once a + parseable pair exists. - **Version newer than the dumper**: pin the dumper to the Unity version. A dump from the wrong version produces plausible names with wrong struct offsets, so treat every field offset as suspect until one is confirmed at run time. @@ -156,6 +159,30 @@ or one method, with `Il2Cpp.trace(true).methods(cls.method("SetShield")).and().a the usual answer to an encrypted `global-metadata.dat`. Il2CppHookScripts (`il2cpp-hooker`) is a scripted alternative with the same dependency. +**Ask the live object graph before rebuilding crypto.** When the question is a decrypted value +rather than a class's shape, the cheapest route is often to let the app construct it: + +```ts +Il2Cpp.perform(() => { + const api = Il2Cpp.domain.assembly("Assembly-CSharp").image.class("Game.APIManager"); + Il2Cpp.gc.choose(api).forEach(instance => { + const cipher = instance.method("get_Crypt").invoke(); + console.log(cipher.method("get_Key").invoke().toString()); + console.log(cipher.method("get_IV").invoke().toString()); + }); +}); +``` + +`Il2Cpp.gc.choose` enumerates live instances without knowing the allocation site; getters and +fields then return the object the app actually uses. This is `inferred` from a public write-up and +not exercised here: the method names and class names are target-specific, and a build that hides +the il2cpp exports breaks the bridge exactly as in the paragraph above. + +`Il2Cpp.dump("name", "/data/data//files")` takes an explicit name and an app-writable +directory; with no arguments it uses defaults that may not be writable on a hardened build. A dump +written before the metadata is decrypted is the encrypted file again — check the magic before +importing it. + The dependency is the boundary: a build that hides or strips the il2cpp exports breaks the bridge, and the failure presents as *class not found* — indistinguishable from a wrong class name. Confirm the exports exist before believing a negative lookup. diff --git a/skills/apk-reverse/references/vm-protector-analysis.md b/skills/apk-reverse/references/vm-protector-analysis.md new file mode 100644 index 0000000..9d53441 --- /dev/null +++ b/skills/apk-reverse/references/vm-protector-analysis.md @@ -0,0 +1,241 @@ +# Reading a VM protector: envelope, virtual eflags, and trace search + +Load this when a native function turns out to be an interpreter but the usual "find the dispatcher" +advice does not fit: `cmp`, `sub` and conditional jumps are missing, the trace is dominated by +`and/or/not/shr/neg`, a flags value is written into a stack slot, or a `ret` lands back at a VM +entry instead of the next instruction. `native-vm-recovery.md` owns the Android-side dispatcher +slicing and register anchoring; `vmp-differential-analysis.md` owns the Dex-level opcode +differential; `native-dbi-and-deobfuscation.md` owns the trace mechanics. This file owns the +*protector idiom* that sits on top of all three. + +**Claim strength.** Absorbed from three public write-ups on x86 VMProtect-style targets and one +practical crack (2024-2026). The envelopes, instruction sequences and trace counts are quoted from +those samples and are `unverified` here; the general rules (anchor on data, trace then search by +constant, patch the flag value rather than the branch) are `inferred` from the same sources. No +VM-protected binary was traced in this record. Addresses and probabilities below are one sample's, +not constants. + +## 1. Envelope triage + +A VM protector is recognisable before any handler is read: + +| Signal | What to look for | +|---|---| +| A new section | `vmp`/`VMP0`-named, often hundreds of KB on a small binary; the section's raw size is a large fraction of the image | +| Entry stub | first instruction is a `jmp`, followed by `push ; call `; the prologue does not match the compiler's idiom (no `push ebp; mov ebp, esp`) | +| Saved physical state | a block of `push` instructions with no matching `pop` before the transfer | +| Dispatcher shape | `mov reg, [index*4 + table]`, or handler-to-handler `jmp reg` / `push reg; ret` | +| Data pointer | a register loaded from the stack and then adjusted by an addend; this is the VM bytecode pointer, not a normal local | + +Procedure: open the section in the debugger, set a **memory access breakpoint** on it, run, and +catch the first `push imm; call`. Delete the section breakpoint once you are inside; it fires on +every handler and will drown the session. The same entry is reachable from a hardware breakpoint on +the section's first page when software breakpoints are detected. + +## 2. VM anatomy, in the order the fields appear + +The names are conventional; the layout is what matters. + +- **VM_DATA**: the encrypted bytecode. A pointer is staged through the stack, then adjusted by an + addend to the real base. Handler dispatch reads the next opcode from it. +- **VM_CONTEXT**: virtual registers. One sample carved `0xC0` bytes with `sub esp, 0xC0`; slots + are addressed as `[esp + virtual_reg_offset]`, with values staged through a scratch register. + `0xC0 / 4 = 48` virtual registers is a useful sanity check. +- **vEIP**: the virtual instruction pointer. In one family it is `ESI`; in another it is a field in + VM_CONTEXT. It points into VM_DATA or at the next handler, depending on the threading style. +- **vStack**: `EBP` marks its top; slots are `[ebp + off]`. The VM checks stack fullness by + comparing top against bottom, which is the one place a normal stack discipline survives. +- **Physical saves**: below the virtual context, so that leaving the VM can restore the host state. + +Two dispatch styles appear in the same product line: + +- **Central dispatcher**: a jump table indexed by opcode, with every handler returning to the + dispatcher. Easy to find, easy to instrument. +- **Direct threading** (later versions): each handler ends with `jmp next` or + `push next; ret`, so there is no single dispatcher. The stable signal is not a table but the + repeated epilogue that loads vEIP and transfers. + +Do not key a detector on a fixed register. One write-up saw `r9` in one sample and `rdx` in +another for the same role; the invariant is "one register carries the next handler and the landing +instruction is benign", not the register's name. + +## 3. Anchor on data, not on code + +Virtualisation destroys the call graph you would normally walk. The method that survives is +data-anchored: + +1. Run the target with a distinctive input: a long repeated string, a unique serial, a value you + can recognise in any buffer. +2. Let the expected failure path execute (a wrong-license dialog, an error string). +3. Find the failure string or your input in memory and set an **access breakpoint** on it. +4. At the break, unwind the stack's return addresses. Keep frames that belong to the target; drop + loader, CRT and system frames. +5. Set breakpoints on those frames, re-run with a second distinctive input, and step until one frame + shows your bytes. That frame is the comparison or the caller that owns it. +6. Replace the code-section breakpoint with a **read breakpoint on your input address** as soon as + you have it. The VM section breakpoint fires constantly; the data breakpoint fires once per use. + +The same technique finds the string decryptor: break on the buffer, then walk to the code that +writes it. + +### 3.1 A forged return is a landmark, not a bug + +If stepping out of a callee lands at a VM entry instead of the instruction after the call, the +return address was forged deliberately. That site is protected logic. Do **not** NOP the region: +one write-up crashed immediately because the VM state expected the forged transfer. The correct +next move is to keep the transfer intact and change the value the protected branch consumes (§5), +or to trace forward from the real entry. + +## 4. Trace, then search by constant + +A full trace is cheap to produce and impossible to read. The useful unit is a *constant* that the +emulated semantics must materialise. In one crack: + +- the trace was 259,607 instructions and took hours; +- searching for `0x40` (the ZF bit) produced about 100 hits; +- filtering to `and` instructions and correlating operand values left three sites, whose + execution counts were 1, 1 and 30; +- the discriminating context was `eax = 0x40`, `ecx = eflags`, and a preceding + `not eax; not ecx` pair; +- forcing `eax = 0x40` at those three sites removed the registration check. + +The reusable recipe: + +1. Trace with a tool that stores disassembly, not only addresses and registers. A trace you cannot + search is a log, not evidence. +2. Search for the semantic constant (`0x40` for ZF, `0x815` for the arithmetic flag mask, + `0x80000000` for sign, a known comparison value). +3. Filter by instruction shape: the constant is usually an operand of `and`, not of `cmp`. +4. Correlate operand roles, not just values. `and eax, ecx` with `eax = 0x40` is a flag test; + `and eax, ecx` with `eax = 0xFFFF...` is a mask. +5. Check execution counts. The handler that runs once per operation is not the one that runs once + per check. +6. Patch the *value written back* to the virtual flags slot, not the instruction that computes it, + when the same handler serves many operations. + +### 4.1 Trace one call, not the process + +A whole-process trace of a VM is unreadable and slow. Anchor it to a call site: + +- resolve the call target from the instruction, including `rel32` manually when the tracer needs an + absolute address (`e8`/`e9`, `rel = read_s32(addr+1)`, `target = addr+5+rel`); follow short + `jmp` thunks (a handful of hops) rather than stopping at the first one; +- start at the callee entry and stop at the call's return address, so the trace contains exactly the + operation you care about; +- keep per-thread state and log `module:VA!RVA insn | reg=before->after`; a trace without register + deltas cannot answer "what did this handler compute"; +- **suppress hot loops.** Count back-edges (a jump whose target is lower than the current address); + after a small number of iterations `unfollow` the thread and garbage-collect, then install a + one-shot interceptor at the loop's fall-through to resume. Without this, a VM's dispatch loop + produces thousands of identical segments and buries the one iteration that differs. + +This recipe is `unverified` here (one public write-up); the hot-loop suppression is the part that +decides whether the trace is usable at all. + +## 5. Virtual eflags is data; patch the data + +On a real CPU only a few instructions write flags directly. In a VM, eflags is a stack slot, so the +protector can synthesise any value it wants. A recurring synthesis is: + +```text +eflags = (eflags1 & 0x815) + (eflags2 & ~0x815) +``` + +The two halves carry the arithmetic flags and the control flags; the sum combines them. The ZF +test is then `zf = eflags & 0x40`. A debugger breakpoint on the test sees the branch only after the +value is already wrong, which is why patching the branch is fragile: the same handler computes many +conditions. Patch the value stored into the virtual flags slot (force `0x40`) and every consumer of +that slot agrees. + +This is also why a search for `cmp`/`sub` fails. Subtraction is lowered as +`a - b = ~(~a + b)`, and comparison is a flag synthesis, so the instruction mix contains no +`sub` at all. Recognise the identity before concluding the function is not a comparison. + +## 6. The lowering idioms that make a VM look like crypto + +### 6.1 The universal gate, and the naming trap + +The sequence that dominates these traces is `not a; not b; and`, i.e. `~a & ~b`. That is **NOR**, +not NAND (`~(a & b)`), even though the write-ups call it NAND. Use the identities as emitted: + +```text +f(a,b) = ~a & ~b ; the actual primitive (NOR) +Not(a) = f(a,a) +Or(a,b) = f(f(a,b), f(a,b)) +And(a,b) = f(f(a,a), f(b,b)) +Xor(a,b) = f(f(a, f(a,b)), f(b, f(a,b))) +``` + +Defining `f` as NAND and keeping the same right-hand sides swaps AND and OR — a mistake that +produces a plausible but wrong truth table. A protector that lowers everything through this gate +produces long runs of `not; not; and`, which reads as noise. Arithmetic is built the same way: + +```text +add(a,b) = loop: sum = a ^ b; carry = (a & b) << 1; until carry == 0 +neg(a) = add(~a, 1) +sub(a,b) = add(a, neg(b)) +mul(a,b) = shift-add on b & 1 +div(a,b) = restoring division over 32 bits +``` + +The value of knowing this is diagnostic: a trace with no `sub` but thousands of `not/and` is not +"encrypted", it is lowered. Stop looking for the missing instruction. + +### 6.2 Branchless select + +A conditional move/select is lowered to: + +```text +sub a,b; pushfd; pop x; and x,0x40; shr x,6 ; ZF -> bit 0 +mov y,1; sub y,x ; complement +neg x; neg y ; 0 -> 0, 1 -> 0xFFFFFFFF +and x,f1; and y,f2; or x,y; call x ; select one function pointer +``` + +Exactly one mask is all ones, so the AND/OR pair selects a target without a jump. + +The same construction appears as a branchless `max`/`min`: + +```text +flip(n) = n ^ 1 +sign(n) = flip((n >> 31) & 1) +c = a - b; sa = sign(a); sb = sign(b); sc = sign(c) +dif = sa ^ sb; same = flip(dif) +returnA = dif*sa + same*sc +returnB = flip(returnA) +result = a*returnA + b*returnB +``` + +There is no branch to patch. Change the operands or the value the final select consumes, and verify +with both orderings of the input — a max built this way is easy to mistake for a checksum. The tell is a +register that holds `0x40` and a `shr` by 6; the control flow graph shows neither a `je` nor a +`jne`. If you patch the `sub` that feeds the flag, you change the selected function; if you patch +the final `call`, you change only one branch of the select. + +### 6.3 Why this matters for an APK + +The same idioms appear in native Android protectors, especially where a bytecode VM emulates +comparisons for a license or signature check. The envelope differs (ELF section, JNI entry), but +the trace signature is identical: no `cmp`, a flags slot, and NAND-shaped arithmetic. Use +`native-vm-recovery.md` for the handler-slicing procedure once you have decided it is a VM; use +this file to decide what the handler is *computing*. + +## 7. Static checks around the dynamic route + +- **Rebase before trusting offsets.** Compute the section-relative offset + (`runtime_addr - image_base`) and apply it in both debugger and disassembler. One sample needed + `0x3E6000 - 0x381000 = 0x65000` and a rebase to `0x380000` to land on the function. +- **Xrefs can be empty and still be correct.** In the same sample, 24 "callers" from IDA never hit + at run time because the site was reached by `jmp` or `call reg`. An empty xref list is not + evidence that the function is unused; it is evidence that the call is indirect. +- **Address stability is not guaranteed.** Some debuggers reopen a sample at a different base each + run. Trust execution order and relative structure; re-derive absolute addresses per session. + +## 8. Boundary + +This is a reading method, not a devirtualiser. It tells you which value the protected branch +consumes and how to change it at run time; it does not reconstruct the original instruction stream. +Nothing here was run against an Android target in this record. The steps that would make a row +`observed`: trace one protected comparison, find the flags slot, force the value, and show the +failure path disappear with the VM still intact. Until then, treat instruction counts and offsets +as illustrative and keep a control run of the unmodified target.