mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-15 20:04:22 +08:00
Absorb the jsvmp series: JS VMP recovery, and a round-trip-tested loader decoder
A new documented route for logic protected as a private JS opcode stream (jsvmp): recognition signals, the call-it-first route, stack- and register-machine reading, AST instruction extraction, the four JS evaluation traps, constant folding and control-flow recovery. Every procedure is inferred (absorbed from three public write-ups, anonymized; none run on a real target) and labelled so. The one shipped piece is the loader decoder (base64 -> varint -> ZigZag), measured against its own round trip (2,016 values exact, canonical varint/ZigZag layouts pinned, truncation raises) and it decodes the write-up's own sample literal end to end. Wired into the symptom index, routing tables, coverage ledgers and the capability matrix; RN/Hermes still not covered, with the jsvmp carve-out stated. Also clears two warnings: an E302 in the new script, and the long-standing check_commands warning where a tool-verification transcript quoted check_budget.py with its result inline (annotations moved to a continuation line, content intact). Ran: check_repo/check_refs/check_routing/check_commands/check_budget (all exit 0), flake8 clean, pytest 763 passed / 199 skipped / 25 xfailed.
This commit is contained in:
@@ -128,7 +128,8 @@ and every command it runs was executed by hand on this machine:
|
||||
|
||||
```
|
||||
$ python check_refs.py exit 0 observed
|
||||
$ python check_budget.py exit 0 ("0 failure(s), 0 warning(s), 3 note(s)") observed
|
||||
$ python check_budget.py
|
||||
exit 0 ("0 failure(s), 0 warning(s), 3 note(s)") observed
|
||||
$ python check_repo.py exit 0 (with PYTHONUTF8=1; see §5.1) observed
|
||||
$ python -m pytest -q exit 0 ("596 passed, 153 skipped, 26 xfailed") observed
|
||||
```
|
||||
|
||||
@@ -239,6 +239,7 @@ unread in this repository.
|
||||
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
|
||||
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
|
||||
| The value you need (sign, token, risk parameter) is computed in **JavaScript** — a WebView or embedded JS engine — by an entry call fed a huge hex/base64 opcode literal: a JS virtual machine | `jsvmp-recovery.md` — de-obfuscate and **call it** first, then the stack/register-VM procedures and the four JS traps; `emulation-and-rpc.md` owns the call-don't-read route |
|
||||
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
|
||||
|
||||
## Gates — clear these before you patch, in order
|
||||
|
||||
@@ -831,6 +831,22 @@
|
||||
"docs/tool-verification/EXTENSION-absorption-kx.md (repository root, not shipped)"
|
||||
],
|
||||
"limits": "The procedure (VM-existence signals, CFF-vs-VM dispatcher separation, relocation-recovered tables, register anchoring with triple verification, nested segment stacks, STORE-backward operand slicing with byte provenance, clean-re-VM validation) is absorbed from two public write-ups; every step is inferred here. The shipped slicer's nested-loop diagnostic is host-tested on a synthetic nested fixture. The xvmp family now has a working, verified runtime on the workbench branches, making a synthetic fixture route available end to end."
|
||||
},
|
||||
{
|
||||
"capability": "jsvmp-recovery",
|
||||
"group": "unpacking",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "inferred",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/jsvmp-recovery.md",
|
||||
"scripts/jsvmp_loader_decode.py"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"three public write-ups of one reverse-engineering series, 2024-03..2024-09 (titles and authors anonymized; the absorbed record is the reference file itself)"
|
||||
],
|
||||
"limits": "The recovery procedures (recognition signals, the call-it-first route, stack- and register-machine reading, AST instruction extraction with register/immediate typing, the four JS-language evaluation traps, constant folding and control-flow recovery) are absorbed from three public write-ups and never executed by this repository on any real target -- all inferred. The shipped loader decoder is host-measured only against its own round trip (2,016 values exact, known varint/ZigZag byte layouts pinned, truncation raises) and decodes the write-up's own sample literal end to end; that is corroboration of the chain's shape, not a measurement on an independent target. Engine-specific delivery (WebView, QuickJS/JSC behind JNI, mini-program containers) was never exercised."
|
||||
}
|
||||
]
|
||||
}
|
||||
@@ -138,6 +138,13 @@ its own strength note at the top.
|
||||
anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM
|
||||
validation) absorbed from two public write-ups; every step `inferred`, none run here
|
||||
(`native-vm-recovery.md`).
|
||||
- **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition
|
||||
signals, the call-it-first route (AST restore + environment fill), stack- and register-machine
|
||||
reading procedures, AST instruction extraction with register/immediate typing, the four
|
||||
JS-language traps, and constant-folding/control-flow recovery. Absorbed from three public
|
||||
write-ups; every procedure `inferred`, none run on a real target. The one shipped piece is the
|
||||
loader decoder (base64 → varint → ZigZag), host-measured against its own round trip plus the
|
||||
write-up's sample literal (`jsvmp-recovery.md`, `scripts/jsvmp_loader_decode.py`).
|
||||
- **Kernel-side event injection** — invisible watchpoints and register rewrites from a driver, as
|
||||
generated templates with their gates; nothing kernel-side was ever compiled or loaded here
|
||||
(`kernel-event-injection.md`, `scripts/kernel_instrument_scaffold.py`).
|
||||
@@ -162,7 +169,10 @@ its own strength note at the top.
|
||||
There is no verified recipe here for locating a method inside `libil2cpp.so` plus
|
||||
`global-metadata.dat`.
|
||||
- **React Native / Hermes bytecode** and Cordova/hybrid internals, beyond runtime identification and
|
||||
the generic "find the string, then find what references it" approach.
|
||||
the generic "find the string, then find what references it" approach. One carve-out: if the JS
|
||||
layer's problem is **VMP-protected JavaScript** (a jsvmp opcode literal plus interpreter), that is
|
||||
a covered documented route (`jsvmp-recovery.md`) — it is orthogonal to which engine runs the JS;
|
||||
the engines' own bytecode formats remain uncovered.
|
||||
- **iOS / `.ipa` of any kind.** Every device, signing and packaging instruction here is Android.
|
||||
- **Defeating a server-side authority.** `server-api.md` determines *who owns a gate*, not how to
|
||||
break an authorization the server performs.
|
||||
|
||||
@@ -48,6 +48,7 @@ route was not measured on this repository's evidence**. Treat them as leads with
|
||||
| SPE/BRBE sampling and crash forensics | data flow without a known address, branch records past an untrustworthy stack, addresses→symbols — `hardware-sampling-and-crash.md` |
|
||||
| Interpreter-trace de-virtualization | handler slicing + guest-opcode trajectory, calibrated on an open IR-VM's source — `vm_handler_slice.py` |
|
||||
| Native VM recovery procedure | the absorbed method end to end: dispatcher separation, register anchoring, nested stacks, operand slicing, re-VM validation — `native-vm-recovery.md` |
|
||||
| JS VMP (jsvmp) recovery | recognition, call-it-first, stack/register-VM reading, instruction extraction, the four JS traps — procedures absorbed, only the loader decoder shipped and round-trip-tested — `jsvmp-recovery.md` |
|
||||
| Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` |
|
||||
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
|
||||
| Publishing without publishing the target | the leak scanner, the do-not-anonymize list, and the graded precedent library — `desensitization-and-leak-scans.md`, `scripts/scan_leaks.py`, `precedents/` |
|
||||
|
||||
@@ -61,6 +61,7 @@ next** — and points at the one section that says "the full record is elsewhere
|
||||
| SPE/BRBE sampling and crash forensics | The address→symbol loop is host-verified on a real cross-compiled `.so`; SPE/BRBE mechanisms are documented-and-gated, never captured here | partial | unverified | `references/hardware-sampling-and-crash.md`, `scripts/trace_addr2sym.py` |
|
||||
| Interpreter-trace de-virtualization | Dispatcher slicing, signature clustering and the opcode trajectory measured on synthetic interpreter traces; the calibration facts come from a public IR-VM's source, and no hardened binary was traced | partial | observed | `references/ete-diff-tracing.md` §5, `scripts/vm_handler_slice.py` |
|
||||
| Native VM recovery procedure | The full method is absorbed from two public write-ups (dispatcher-layer separation, register anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM validation); no step was executed by this repository | partial | inferred | `references/native-vm-recovery.md` |
|
||||
| JS VMP (jsvmp) recovery | Procedures absorbed from three public write-ups (stack and register machines) and never run on a real target; the shipped loader decoder (base64→varint→ZigZag) round-trips 2,016 values exactly, pins the known varint/ZigZag layouts and decodes the write-up's own sample literal end to end | partial | inferred | `references/jsvmp-recovery.md`, `scripts/jsvmp_loader_decode.py` |
|
||||
| Kernel-side instrumentation/injection | The scaffold's generate/verify is measured and every kernel file ships unbuilt with its gate; the register-rewrite action is explicitly experimental and never ran anywhere | blocked | unverified | `references/kernel-event-injection.md`, `scripts/kernel_instrument_scaffold.py` |
|
||||
| `svc` site scanning | Two independent decoders agreed on an identical 214-site set; a byte scan also matches data, so neighbour context decides | ok | observed | `references/kernel-and-environment-hardening.md`, `scripts/svc_scan.py` |
|
||||
| On-device tooling (MT Manager MCP) | The service-down path is measured; the connected path needs the service started by hand | partial | observed | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
|
||||
@@ -21,6 +21,10 @@ Signals in the APK:
|
||||
A `classes.dex` that is small relative to the total payload is another strong hint that the real code
|
||||
lives elsewhere.
|
||||
|
||||
When the runtime runs **JavaScript** and the JS you find is one entry call fed a huge hex/base64
|
||||
opcode literal, that is not a runtime-identification question at all — it is a JS virtual machine
|
||||
(jsvmp). Load `references/jsvmp-recovery.md` for that shape; the table above only names the engine.
|
||||
|
||||
## The layer trap
|
||||
|
||||
The expensive mistake: seeing a UI element (dialog, paywall, gate) and assuming it is a native control,
|
||||
|
||||
@@ -0,0 +1,262 @@
|
||||
# JS VMP recovery — when the protected logic is JavaScript (jsvmp)
|
||||
|
||||
Load this when the value you need — a sign, a token, a device fingerprint, a risk parameter — is
|
||||
produced inside **JavaScript the app controls** (a WebView page, a JS engine embedded behind JNI, a
|
||||
mini-program container, a hot-updated bundle), and the JS is not merely minified: one entry call
|
||||
receives a **giant literal** — thousands of hex characters or a base64 blob — plus an array of
|
||||
globals, and everything behind it is one closed interpreter. That shape is **jsvmp**: the original
|
||||
logic compiled to a private opcode stream and executed by a virtual machine shipped with the script.
|
||||
This file owns reading it. The neighbouring layers own the rest: `references/emulation-and-rpc.md`
|
||||
owns **calling** the routine instead of reading it, `references/vmp-differential-analysis.md` the
|
||||
dex-layer VMP, `references/native-vm-recovery.md` the native interpreter loop, and
|
||||
`references/framework-runtimes.md` deciding that the logic lives in JS at all.
|
||||
|
||||
**Claim strength.** Absorbed from three public write-ups of one reverse-engineering series
|
||||
(2024-03..2024-09; titles and authors anonymized in this record). One builds a minimal JS VM to
|
||||
expose the shape; one reverses a **stack-machine** jsvmp (a short-video platform's web sign SDK);
|
||||
one reverses a **register-machine** jsvmp (a messaging giant's refreshed risk-control VMP, which
|
||||
migrated from stack to register and grew from ~60 to 91 instructions). The write-ups are `observed`
|
||||
by their authors; this repository has executed **none** of these steps on a real target — every
|
||||
procedure below is `inferred`. The one shipped tool, `scripts/jsvmp_loader_decode.py`, is
|
||||
host-measured against its own round trip (`--selftest`). The JS-engine details (QuickJS,
|
||||
JavaScriptCore, V8-in-WebView) were never exercised here either.
|
||||
|
||||
## 1. Why this shape exists, and what it changes about your plan
|
||||
|
||||
jsvmp compiles the original JS to a private instruction stream and interprets it, exactly like a
|
||||
native VM does for machine code — but the interpreter itself is JS you can **read, run, instrument
|
||||
and re-run in a REPL**. That is the structural advantage over `native-vm-recovery.md`: no trace
|
||||
hardware, no arm64 semantics, and a wrong hypothesis is a one-line probe away from a correction.
|
||||
|
||||
Two facts about the shape drive the plan:
|
||||
|
||||
- **The literal is the program.** Every character (stack machine) or decoded integer (register
|
||||
machine) is an opcode or operand byte. Its length is a lower bound on program size, and its
|
||||
format names the loader you must undo first (§2).
|
||||
- **The dependency array is an import table and an environment probe at once.** The write-ups' entry
|
||||
calls pass `[Object, Math, setTimeout, ..., void 0 !== x ? x : void 0, false, entryFn, arg]` — the
|
||||
VM reads its globals through this array, and the `typeof`/`void 0` guards double as an environment
|
||||
fingerprint check. Hostile values here are how a harness is detected.
|
||||
|
||||
Why the vendor did not use wasm: wasm cannot touch JS objects directly without a bridge, and the
|
||||
crossing cost plus the bridge code outweigh the protection. Which tells you the **whole point of the
|
||||
JS-layer VM is environment access** — expect fingerprint collection (canvas, screen, fonts) woven
|
||||
into the guest program, not only crypto.
|
||||
|
||||
## 2. Recognition — decide "jsvmp" before committing
|
||||
|
||||
| Signal | What it looks like | Confidence |
|
||||
|---|---|---|
|
||||
| Entry shape | `global['_$webrt_<epoch>'](hugeString, [deps...], this)` — one call, one huge constant, one globals array | near-certain with the other two |
|
||||
| Interpreter shape | a `while`/`for` loop containing a large `switch` or if-chain; the **deepest indentation** in the file (loops nested in flow control) | strong |
|
||||
| Opcode fetch | `parseInt(s[i] + s[i+1], 16); i += 2` (two hex chars = one byte), or `opcode[++PC]` cascades inside each `case` | strong |
|
||||
| State | an array plus an index used as stack (`stack[++sp] = v` push, `v = stack[sp--]` pop), or an array indexed by fetched operands as a register file | decides stack vs register (§5, §6) |
|
||||
| Loader fingerprint | base64 charset ops, 7-bit masking with high-bit continuation (varint), or the ZigZag formula `(A >> 1) ^ -(1 & A)` | the program is encoded, not encrypted — decode it, don't cryptanalyse it |
|
||||
|
||||
Two cautions. A big `switch` alone proves nothing (a state machine looks the same); the loop must
|
||||
**read a cursor from a data buffer, dispatch on the value, and write the cursor back**. And the
|
||||
presence of a readable interpreter does not mean you must read it — §3 first.
|
||||
|
||||
## 3. The cheaper question first: restore, fill the environment, call it
|
||||
|
||||
The stack-machine write-up's outcome, in its own order, and the step most skip:
|
||||
|
||||
1. **AST pre-deobfuscation.** String-array restoration, folding self-executing wrappers, constant
|
||||
propagation. This halved the sample (~9,600 lines → roughly half) and made every later step
|
||||
debuggable in an IDE instead of a browser's obfuscated view (the same function was 3,000+
|
||||
unreadable lines in devtools). Debugging un-restored code in the browser is the slow path; do
|
||||
not attempt VM analysis inside it.
|
||||
2. **Fill the environment** ("补环境"): give the script the `window`/`document`/`navigator`/canvas
|
||||
surface it probes. Canvas fingerprints are **hardware-linked** — vendors have collected real
|
||||
device fingerprints to reject forged ones; a random canvas value may still be accepted (the
|
||||
write-up's target did not server-validate it *at that time*), which is `unverified` as a
|
||||
general rule. Treat every environment patch as a variable that can fail server-side later.
|
||||
3. **Export the target function and call it.** In the write-up this already produced the parameter
|
||||
the task needed — **before any VM analysis at all**.
|
||||
|
||||
This is `references/emulation-and-rpc.md`'s call-don't-read route, and it is the correct first
|
||||
deliverable: a harness that runs the guest in your controlled environment. Read the VM only when
|
||||
you must understand or alter the algorithm itself, or when the environment the VM demands is more
|
||||
expensive to fake than the VM is to read.
|
||||
|
||||
## 4. Locating the interpreter and clearing the porch
|
||||
|
||||
- **Find the entry by the literal, not by names.** Search for the huge constant being passed; follow
|
||||
the call stack one frame to the VM entry (`_$webrt_`-style name). Entry is ~900 lines even after
|
||||
de-obfuscation in the stack-machine sample.
|
||||
- **Copy it to a new file and use an IDE outline.** The entry decomposes into a handful of closures;
|
||||
the outline view names them and the reference view tells you which are dead. The write-up's entry
|
||||
was seven closures, one of them the interpreter.
|
||||
- **Simplify the guard closures before touching the interpreter.** Two measured reductions from the
|
||||
write-up, both anti-hook scaffolding rather than VM logic:
|
||||
- a `Reflect.construct` availability probe + a bind-based construct fallback + a
|
||||
`setPrototypeOf`/`__proto__` fallback — three functions whose only role is preventing
|
||||
reflection hooks; after checking references, replace with `const construct = Reflect.construct;`
|
||||
- an iterable-to-array spread helper (`Array.isArray` copy / `Symbol.iterator` / Arguments check,
|
||||
throwing `TypeError("Invalid attempt to spread non-iterable instance")`) — replace with
|
||||
`Array.from`.
|
||||
- **Do not delete "fake" branches on aesthetics.** The write-up marked a suspicious dual-`if` flow
|
||||
as probably-dead, instrumented the condition variables, and found the second branch **did** fire
|
||||
(values almost always 0, occasionally 1). Probe before cutting; a deleted live branch fails
|
||||
silently pages later.
|
||||
|
||||
## 5. Reading a stack-machine jsvmp
|
||||
|
||||
The operands live on an explicit stack; instructions pop and push. Recognition: `stack[++sp] = v`
|
||||
and `v = stack[sp--]` pairs, binary results written back in place — `stack[sp] = stack[sp] OP reg`
|
||||
after one pop. Rename first, everything becomes readable:
|
||||
|
||||
| Original idiom | Rename to |
|
||||
|---|---|
|
||||
| `_0xcc6308 = []` | `stack` — the operand stack |
|
||||
| `_0x2e1055 = 0` | `stackPointer` |
|
||||
| the fetch index, initialised from an argument | `PC` (start index), with `endIndex` computed from the program length |
|
||||
| `_0x3f0f70` — `parseInt(s[i]+s[i+1],16)` | `operationCode` (0x00..0xFF by construction) |
|
||||
| first popped operand in binary cases | `operandReg`; longer cases: `regA`, `regB` |
|
||||
| the pushed `this`/global at call boundaries | `context` — the scope object the VM executes inside |
|
||||
|
||||
The dispatch value is *computed*, not read directly — in the sample:
|
||||
`dispatch = 3 & ((13 * operationCode) % 241)`, giving an initial class 0..3 that then varies during
|
||||
execution. Derive such ranges from the arithmetic, never from secondary prose: the write-up caught
|
||||
a whole lineage of copy-paste articles asserting "the loop variable is 0..255" — that is the *byte*
|
||||
range, not the dispatch sequence. (This is the label discipline of `SKILL.md` §Coverage applied to
|
||||
someone else's blog.)
|
||||
|
||||
After renaming, rewrite the if-chain dispatch into a real `switch` via AST — the case bodies are the
|
||||
instruction semantics, and the set of cases is the ISA.
|
||||
|
||||
## 6. Reading a register-machine jsvmp
|
||||
|
||||
The newer shape (91 instructions in the sample): no stack. Each `case` fetches its own operands
|
||||
inline — `case 3: h[opcode[++PC]] = opcode[++PC];` is `MOV reg, imm` — and `h` is the
|
||||
register/context file. Verify by instrumenting: log `h`'s indices at one binary op; fixed indices
|
||||
for operand/operand/result (`h[17]`, `h[22]`, `h[26]` in the sample) prove register addressing.
|
||||
|
||||
**Undo the loader before anything else.** The program arrives encoded; the write-up's chain, each
|
||||
stage recognisable by its fingerprint (§2), and shipped as `scripts/jsvmp_loader_decode.py`
|
||||
(corroboration, `observed`: the write-up's own sample literal decodes end to end under it — no
|
||||
truncation error, values in a plausible opcode range):
|
||||
|
||||
1. **base64** string → unsigned byte array (strip `=`, map chars to indices, recombine bits);
|
||||
2. **varint** — per byte, `o |= (b & 0x7F) << shift; shift += 7` while the high bit continues;
|
||||
3. **ZigZag** decode each accumulated integer: `(n >> 1) ^ -(n & 1)` maps unsigned back to signed.
|
||||
|
||||
A flattened loader loop reads as 4 `if`s each ending in `continue` — treat every later block as the
|
||||
`else` of the earlier one and it reduces to the three-stage pipeline above. In an APK, the same
|
||||
chain may sit behind JNI or load from `assets/`; the recognition problem is identical.
|
||||
|
||||
**Expect twins.** The sample carried **two executors** selected by a condition that constant-folds
|
||||
to one side — but the twins call each other, so both must be accounted for. Text-diffing the two
|
||||
interpreters showed them byte-identical except `"use strict"`. Strict-mode code runs in non-strict
|
||||
too, so deleting the non-strict twin is safe *if you verify by running* — the write-up replaced the
|
||||
response through a proxy rewriter and confirmed the output unchanged. Two executors also means two
|
||||
places a patch must land; a patch that "had no effect" may have edited the dead twin.
|
||||
|
||||
**The exception path is the call stack.** The `catch` saves the exception, and if a callback stack
|
||||
is non-empty returns to the previous frame, else rethrows (optionally through a handler argument
|
||||
that was `void 0` in the sample — probably a vendor debugger/log hook). The VM's **PC save across
|
||||
nested guest calls rides on JS closures** — each invocation carries its own PC, and returning from
|
||||
the inner function restores the outer one. There is no explicit restore instruction, so any
|
||||
executor you build must not leak scope state across guest calls (§8).
|
||||
|
||||
## 7. Instruction extraction — from switch cases to an ISA table
|
||||
|
||||
Extract mechanically with AST, not by hand:
|
||||
|
||||
1. Pull each `case` body into its own function (drop the `break`s); the write-up used a
|
||||
`SwitchStatement` visitor that renames each to `op_<code>` and lifts it to a declaration.
|
||||
2. **Type every operand** as it is used: an operand read as `h[...]` is a **register** (`R0`,
|
||||
`R1`, …); an operand consumed directly as a value is an **immediate** (`imm`) — and an immediate
|
||||
is a **constant by construction**, because it comes from the program text. That single property
|
||||
powers constant folding later (§9).
|
||||
3. Record `argnums` per opcode — the operand count — and let a loader (`loadOP`) fetch exactly that
|
||||
many before dispatch. Instructions whose operand count is data-dependent (varargs calls, the
|
||||
sample's ops 18/19) stay unparameterised until understood.
|
||||
4. Name what you understand (`MOV`, `ADD`, `JMP`, `STOSB`…), reuse one name for ops that differ
|
||||
only in operand width, and name the rest `unknown_<code>` rather than guessing. The sample
|
||||
reached ~91 instructions with 8 `unknown` — resolved later by running.
|
||||
5. Watch for **semantic disguises**: the sample's op 10 is `h[r] = h[r] - 0` — not arithmetic but
|
||||
`ToNumber` coercion. Name intent, not syntax.
|
||||
|
||||
An operand may have **two possible widths** under one opcode — the sample's conditional jump reads
|
||||
one operand when the condition is true and two when false (`PC += h[op1] ? op2 : op3`). Encode that
|
||||
in the table; a disassembler that assumes one width desynchronises from that instruction onward.
|
||||
|
||||
## 8. Four JS-language traps with no native counterpart
|
||||
|
||||
Rewriting an interpreter's cases into clean functions breaks in ways native VM work never meets.
|
||||
All four cost the write-up a day or more; all four are now checkable in minutes.
|
||||
|
||||
1. **Expression evaluation is strictly left-to-right.** In
|
||||
`h[opcode[++PC]][h[opcode[++PC]]] = h[opcode[++PC]]`, the object index is `args[0]`, the member
|
||||
index `args[1]`, the RHS `args[2]` — *not* the Python order (RHS first, then outer-then-inner on
|
||||
the left). Verifiable with side-effect logging: JS prints 0,1,2,3. Reconstructing operand order
|
||||
"by execution priority" inverts it.
|
||||
2. **`PC += opcode[++PC]` is not `arg = opcode[++PC]; PC += arg`.** Compound assignment reads the
|
||||
LHS value *before* evaluating the RHS (`PC=1`: first form → 3, second → 4). The V8 bytecode shows
|
||||
it — the slot is loaded into the accumulator before `AddSmi`. For any PC-accumulating
|
||||
instruction, either keep the original expression verbatim in the extracted function or
|
||||
compensate by hand; a wrapped `jmp` that off-by-ones PC executes the wrong opcode and dies
|
||||
silently pages later.
|
||||
3. **`this` pollution.** Calling the extracted op functions with the caller's `this` leaks
|
||||
`undefined` (strict) or `globalThis` (sloppy) into the VM's environment. Call with
|
||||
`opFn.apply(undefined, args)` or spread — and remember the strict/sloppy difference is exactly
|
||||
why the vendor shipped twin interpreters (§6).
|
||||
4. **A `RET` may legitimately return `undefined`.** An executor's `if (result !== undefined) return
|
||||
result;` therefore never exits. Branch on the opcode (`if (bytecode === RET) return result;`),
|
||||
not on the value.
|
||||
|
||||
The acceptance test for any rebuilt executor is the write-up's: run the **real captured program**
|
||||
end to end (the sample took ~880,000 guest instructions) and compare outputs — a clean exit after
|
||||
the fixes is the only proof the extraction preserved semantics.
|
||||
|
||||
## 9. Disassembly, constant folding, control-flow recovery
|
||||
|
||||
With a loader and an ISA table you have a static disassembler. Pipeline the artefacts as files —
|
||||
`program.b64` (input) → `program.bin` (decoded integer stream) → `program.asm` (disassembled) →
|
||||
`program.ir` (lifted) — so each stage is diffable and re-runnable. In the sample: ~48,000 decoded
|
||||
integers → ~14,000 instructions, with constants visible as bulk stores into fixed program space
|
||||
(the same layout discipline as a real binary's `.rodata`).
|
||||
|
||||
- **Fold constant-building idioms with pattern matching, not by reading.** The sample's dominant
|
||||
idiom — `SETEMP [r]` then a run of `STOSB [r, imm]` then a member store — *is* a string
|
||||
declaration: `locals[r] = String.fromCharCode(imm, imm, …)` (there: `"__lastWatirConfirm"`).
|
||||
`STOSB` alone was 5,300 of 14,000 instructions; folding collapsed a third of the program. Match
|
||||
instruction sequences like regexes (the write-up used an NFA over the instruction stream) and
|
||||
emit **extended pseudo-instructions** from a reserved opcode range (999+) for each folded idiom.
|
||||
- **Control flow falls out of compare-and-branch pairs.** `EQ` + `JNZ` with two targets → `if/else`;
|
||||
`LT` + `JNZ` back-edge → `while`; `GETMEMBERIMM` + `GTIMM` + `JNZ` → `for`. This is
|
||||
`native-vm-recovery.md` §10's reading discipline with an easier instruction set — and the honest
|
||||
limit is the same: what you can recover is bounded by how much of the ISA you have named.
|
||||
- **Know what your disassembly does not give you.** The write-up's own list: register roles not
|
||||
analysed (still `locals[n]`, not variable names); constants foldable but not yet folded; no
|
||||
re-assembler, so asm edits cannot round-trip into runnable bytecode. Say which of these your
|
||||
output suffers from before treating it as source.
|
||||
|
||||
## 10. Where this plugs into this skill
|
||||
|
||||
| Need | Tool / file |
|
||||
|---|---|
|
||||
| decode a jsvmp program literal (base64 → varint → ZigZag) | `scripts/jsvmp_loader_decode.py` (`--selftest` proves the round trip) |
|
||||
| decide the logic lives in JS at all | `references/framework-runtimes.md` (runtime table; zero-hit Java hooks) |
|
||||
| call the routine instead of reading it (the §3 route) | `references/emulation-and-rpc.md`; Frida RPC over a WebView/engine boundary |
|
||||
| reach a JS engine behind JNI from the device | `references/dynamic-frida.md`; the JS bridge inputs are in `references/system-ipc-audit.md` |
|
||||
| the same VM problem, one layer down (native interpreter) | `references/native-vm-recovery.md` |
|
||||
| the same VM problem, one layer up (dex private opcodes) | `references/vmp-differential-analysis.md` |
|
||||
|
||||
Deliverable shape: a jsvmp target almost never ends in a repack — the artifact is a **harness that
|
||||
calls the guest program** (G1's third form) or an **analysis report with a stated boundary**. Write
|
||||
the G1 sentence accordingly before sinking days into the ISA.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] entry recognised by the literal + dependency-array shape, not by a big `switch` alone
|
||||
- [ ] §3 executed in order — AST restore, environment fill, call the function — before any VM reading
|
||||
- [ ] environment patches labelled: each one can fail server-side later (canvas especially)
|
||||
- [ ] guard closures simplified with reference-checked replacements; "dead" branches probed, not cut
|
||||
- [ ] stack vs register decided by the state idiom (§2 table), then the naming map applied
|
||||
- [ ] loader undone before semantics; range claims derived from arithmetic, not from secondary prose
|
||||
- [ ] twins diffed; patches proven against the live twin by running, not by reading the selector
|
||||
- [ ] operand typing R/imm recorded per opcode; unknown opcodes named `unknown_<code>`, not guessed
|
||||
- [ ] the four §8 traps checked in any rebuilt executor; real program run end to end as acceptance
|
||||
- [ ] labels: procedures `inferred` (absorbed write-ups), shipped decoder `observed` (round trip only)
|
||||
@@ -80,9 +80,10 @@ reference file is not named here, or when a script is not named here.
|
||||
| You must edit, repack, sign or inspect the APK **from the phone itself** | `references/on-device-tooling.md`, `scripts/mt_mcp_probe.py` |
|
||||
| A captured body decodes to nothing readable, or you cannot tell whether a length-delimited field is a string, a nested message or a packed array | `protocol-reverse.md`. Protobuf on the wire (measured) — run `scripts/protobuf_decode_raw.py`; the candidate list and its `tie:` lines are the answer |
|
||||
| Method bodies are present but decode as **private opcodes**, and you need the mapping rather than an explanation of why VMP is hard | `vmp-differential-analysis.md`, then `advanced-unpacking.md` for the shape diagnosis |
|
||||
| The value you need (sign, token, risk parameter) is computed in **JavaScript** — a WebView or embedded JS engine — by an entry call fed a huge hex/base64 opcode literal: a JS virtual machine | `jsvmp-recovery.md` — de-obfuscate and **call it** first, then the stack/register-VM procedures and the four JS traps; `emulation-and-rpc.md` owns the call-don't-read route |
|
||||
| A store build arrives as `base.apk` + `split_config.*.apk`, or a rebuilt build is refused **as a set** although every file verifies on its own | `split-apk.md` — one keystore across every member for `pm install-multiple`, and check that a merge is legal before trusting a merged single APK |
|
||||
|
||||
_54 row(s) below the header._
|
||||
_55 row(s) below the header._
|
||||
|
||||
## Reference index (no longer inline in `SKILL.md`)
|
||||
|
||||
@@ -96,6 +97,7 @@ _54 row(s) below the header._
|
||||
| `references/code-virtualization-and-custom-linkers.md` | **No packer, dex is readable, and a re-signed build still dies** — whole classes turned `native`, a private loader with a mismatched SONAME, a self-decrypting payload, a Java-layer "signature killer", and the keep-it/drop-it deadlock |
|
||||
| `references/java2c-and-jni-sinking.md` | **Whole classes are bare `native` declarations and you are about to hunt for a decrypted DEX** — Java2C has none, ever; the code is in a `.so`. Also the JNI boundary: why a `Java_*` symbol search fails silently |
|
||||
| `references/vmp-differential-analysis.md` | Method bodies are present but decode as **private opcodes** — a real Dex VMP: the known-plaintext differential, which links can and cannot be automated, how to *prove* a derived opcode table, smali generation, and when the route is closed |
|
||||
| `references/jsvmp-recovery.md` | The protected logic is **JavaScript** (WebView, embedded JS engine, mini-program): a jsvmp opcode literal plus an interpreter — recognition, the call-it-first route, stack- and register-VM reading, instruction extraction, the four JS-language traps, and the shipped loader decoder |
|
||||
| `references/framework-runtimes.md` | The UI is not native (Flutter / React Native / Unity / Cordova), or Java-layer hooks fire zero times while the UI clearly works |
|
||||
| `references/dart-aot.md` | The logic lives in a Dart AOT snapshot (`libapp.so`): pinning the Dart version, building a matching decompiler, the object pool and reference indexing, register/boolean conventions, locating and patching Dart code |
|
||||
| `references/native-and-so.md` | Patching in a `.so`, needing code to run before the app's own code, hand-built native payloads that crash inside the linker, or **deciding which library/ABI is actually loaded and executing** |
|
||||
@@ -143,7 +145,7 @@ _54 row(s) below the header._
|
||||
| `references/on-device-tooling.md` | Working **from the phone itself**: MT Manager edit/repack/sign and its built-in APK MCP, LSPosed Manager, Termux+frida, and on-device data inspection |
|
||||
| `references/split-apk.md` | The target is a **split APK / App Bundle set** (`base.apk` + `split_config.*.apk`), or `pm path <PKG>` returned several files: reading a set, merge versus unified re-signing, and the install refusal each mistake produces |
|
||||
|
||||
_43 row(s) below the header._
|
||||
_44 row(s) below the header._
|
||||
|
||||
## Script index (no longer inline in `SKILL.md`)
|
||||
|
||||
@@ -198,6 +200,7 @@ _43 row(s) below the header._
|
||||
| `scripts/java2c_probe.py` | The evidence that separates **Java2C** from an extraction shell, a VMP and JNI sinking: per-class `native` density and stub ratio from the dex, plus `Java_*` / `JNI_OnLoad` / registration evidence from the `.so` |
|
||||
| `scripts/protobuf_decode_raw.py` | **Schema-free protobuf decode**: hex / file / stdin to a JSON tree, every length-delimited field as a candidate set with ties labelled; `--reencode --check` for a byte-exact round trip |
|
||||
| `scripts/vmp_diff_harness.py` | Differential hardening for Dex-VMP: build a labelled opcode-coverage fixture (**218 of 224 opcodes, measured**), derive a candidate private-opcode map from an original/hardened dex pair, verify it in a closed loop, render smali |
|
||||
| `scripts/jsvmp_loader_decode.py` | Decode a **jsvmp** program literal (base64/hex → varint → ZigZag → signed opcode stream) with offsets, JSON output and a round-trip `--selftest`; raises on truncated input rather than decoding short |
|
||||
| `scripts/kernelsu_syscall_mask.py` | Generate a KernelSU/APatch syscall-masking scaffold: an installable userspace module plus KPM/LKM/eBPF kernel-side templates, each with its version gate and an explicit unverified label. **A userspace module cannot change a syscall return value** |
|
||||
| `scripts/hw_watch.py` | **Invisible memory watchpoints / execution breakpoints** on arm64: plan (WCR/BCR encoders with the 8-byte-window rules, host-tested), emit (self-contained on-device C helper, compiles clean as static aarch64), gates, run (build+push+stream hits), decode (module+offset per hit). ptrace route: `TracerPid` stays visible — the ledger is in the reference |
|
||||
| `scripts/trace_diff.py` | **Differential trace reduction**: reduce one trace to runs+histogram, or diff two traces into first-divergence / divergent regions / blocks-unique-to-each-side. Producer-agnostic (Stalker `BLK` logs or any address-per-line stream, incl. decoded cs_etm) |
|
||||
@@ -212,4 +215,4 @@ _43 row(s) below the header._
|
||||
| `scripts/svc_scan.py` | Name the syscall behind an inline `svc` and the segment it sits in — which decides whether a libc-level hook can observe the call at all. `--context` shows neighbours, because a byte scan also matches data |
|
||||
| `scripts/anti_detect_probe.js` | Observer-only probe (patches nothing): path/loader/thread/kill hooks with **caller module + offset**, an environment self-report (`TracerPid`, frida-named mappings), and live streaming so a sub-second target still yields evidence |
|
||||
|
||||
_53 row(s) below the header._
|
||||
_54 row(s) below the header._
|
||||
|
||||
@@ -1039,6 +1039,7 @@ SCRIPT_CAPABILITIES = {
|
||||
'find_refs.py': ['static_dex'],
|
||||
'apk_diff.py': ['static_dex'],
|
||||
'blob_decode.py': ['static_dex'],
|
||||
'jsvmp_loader_decode.py': ['static_dex'],
|
||||
'elf_plt.py': ['static_native'],
|
||||
'so_constpatch.py': ['static_native'],
|
||||
'svc_scan.py': ['static_native'],
|
||||
|
||||
@@ -0,0 +1,199 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Decode a jsvmp program literal: base64/hex -> varint -> ZigZag -> signed integers.
|
||||
|
||||
The register-machine jsvmp family documented in `references/jsvmp-recovery.md` ships its
|
||||
guest program as an encoded integer stream. The write-up's loader chain is three standard
|
||||
stages, each recognisable by fingerprint and each invertible without the target:
|
||||
|
||||
1. base64 (or hex) text -> unsigned byte array
|
||||
2. varint groups: each byte contributes its low 7 bits; a set high bit continues
|
||||
3. ZigZag decode per accumulated integer: (n >> 1) ^ -(n & 1) (unsigned -> signed)
|
||||
|
||||
This script implements the decode side **and** the encode side, so the chain can be proven
|
||||
against its own round trip (`--selftest`), which is the only evidence this repository has
|
||||
for it: no real jsvmp target was ever decoded here. When a target's loader differs, expect
|
||||
the difference in stage order or byte-signedness conventions (the write-up read the byte
|
||||
array as *signed* int8, where "high bit set" surfaces as a negative value -- the same bits),
|
||||
not in the stage identities.
|
||||
|
||||
Output: one decoded integer per line with its input offset, plus a summary. Feed the result
|
||||
to your disassembler stage (`program.bin` in `references/jsvmp-recovery.md` §9).
|
||||
|
||||
Examples:
|
||||
python scripts/jsvmp_loader_decode.py --b64-string 'JgQCFhoEBBZEHgASPkQEZp/4A4PzAg=='
|
||||
python scripts/jsvmp_loader_decode.py --hex-string 484e4f4a403f5243 --json
|
||||
python scripts/jsvmp_loader_decode.py --selftest
|
||||
"""
|
||||
import argparse
|
||||
import base64
|
||||
import json
|
||||
import sys
|
||||
|
||||
|
||||
def zigzag_decode(n):
|
||||
"""Unsigned -> signed ((n >> 1) ^ -(n & 1)), the standard ZigZag mapping."""
|
||||
return (n >> 1) ^ -(n & 1)
|
||||
|
||||
|
||||
def zigzag_encode(n):
|
||||
"""Signed -> unsigned ZigZag. Python ints are unbounded, so any width works."""
|
||||
return (n << 1) ^ (n >> 63) if n < 0 else (n << 1)
|
||||
|
||||
|
||||
def b64_to_bytes(text):
|
||||
"""base64 text -> bytes, tolerant of stripped padding and whitespace."""
|
||||
cleaned = ''.join(text.split())
|
||||
pad = (-len(cleaned)) % 4
|
||||
return base64.b64decode(cleaned + '=' * pad)
|
||||
|
||||
|
||||
def hex_to_bytes(text):
|
||||
cleaned = ''.join(text.split())
|
||||
if len(cleaned) % 2:
|
||||
raise ValueError('hex string has an odd length: %d' % len(cleaned))
|
||||
return bytes.fromhex(cleaned)
|
||||
|
||||
|
||||
def varint_decode(data, shift_step=7):
|
||||
"""Byte stream -> list of (offset, unsigned_value).
|
||||
|
||||
Each byte contributes its low `shift_step` bits; a set high bit means "continue".
|
||||
Raises on a stream that ends mid-value -- truncation is a finding, not a zero.
|
||||
"""
|
||||
out = []
|
||||
value = 0
|
||||
shift = 0
|
||||
start = 0
|
||||
for i, byte in enumerate(data):
|
||||
value |= (byte & 0x7F) << shift
|
||||
if byte & 0x80:
|
||||
shift += shift_step
|
||||
continue
|
||||
out.append((start, value))
|
||||
value = 0
|
||||
shift = 0
|
||||
start = i + 1
|
||||
if value or shift:
|
||||
raise ValueError('varint stream ends mid-value at offset %d '
|
||||
'(truncated input, or this is not varint)' % start)
|
||||
return out
|
||||
|
||||
|
||||
def varint_encode(values):
|
||||
"""Signed integers -> varint byte stream (ZigZag first), the encode side of the chain."""
|
||||
out = bytearray()
|
||||
for n in values:
|
||||
u = zigzag_encode(n)
|
||||
while True:
|
||||
byte = u & 0x7F
|
||||
u >>= 7
|
||||
if u:
|
||||
out.append(byte | 0x80)
|
||||
else:
|
||||
out.append(byte)
|
||||
break
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def decode_program(data):
|
||||
"""Raw bytes -> decoded signed integers (the full chain stages 2+3)."""
|
||||
return [(off, zigzag_decode(u)) for off, u in varint_decode(data)]
|
||||
|
||||
|
||||
def selftest():
|
||||
"""Round-trip proof: encode -> decode must be exact on fixed and random vectors."""
|
||||
import random
|
||||
cases = [
|
||||
[0],
|
||||
[1, -1, 2, -2], # zigzag neighbours: 0,1,2,3 unsigned
|
||||
[127, 128, -128, 129, -129], # varint width boundaries
|
||||
[300, -300, 16383, 16384], # 2- and 3-byte varints
|
||||
[2 ** 31 - 1, -(2 ** 31)], # int32 extremes
|
||||
]
|
||||
rng = random.Random(20240901)
|
||||
cases.append([rng.randrange(-(2 ** 31), 2 ** 31) for _ in range(2000)])
|
||||
for values in cases:
|
||||
encoded = varint_encode(values)
|
||||
decoded = [v for _, v in decode_program(encoded)]
|
||||
if decoded != values:
|
||||
print('FAIL round trip: %r -> %d bytes -> %d values (first divergence at index %d)'
|
||||
% (values[:8], len(encoded), len(decoded),
|
||||
next(i for i, (a, b) in enumerate(zip(values, decoded)) if a != b)))
|
||||
return 1
|
||||
# Known byte layouts pin the layout, not just self-consistency. Raw varint(300) = AC 02
|
||||
# (pinned through varint_decode below is the decode side); the full chain zigzags FIRST,
|
||||
# so encoding signed 300 means encoding zigzag(300)=600 -> D8 04.
|
||||
assert varint_encode([1, -1]) == b'\x02\x01', varint_encode([1, -1])
|
||||
assert varint_encode([300]) == b'\xd8\x04', varint_encode([300])
|
||||
assert varint_decode(b'\xac\x02') == [(0, 300)]
|
||||
# Truncated input must raise rather than silently decode short.
|
||||
try:
|
||||
varint_decode(b'\xac')
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
print('FAIL: truncated varint did not raise')
|
||||
return 1
|
||||
total = sum(len(c) for c in cases)
|
||||
print('OK: %d values round-tripped exactly (encode->varint->zigzag->decode), '
|
||||
'known byte layouts pinned, truncation raises' % total)
|
||||
return 0
|
||||
|
||||
|
||||
def main(argv=None):
|
||||
ap = argparse.ArgumentParser(
|
||||
description='Decode a jsvmp program literal (base64/hex -> varint -> ZigZag). '
|
||||
'See references/jsvmp-recovery.md for the loader chain and its limits.')
|
||||
src = ap.add_mutually_exclusive_group(required=True)
|
||||
src.add_argument('--b64-string', help='base64 program literal on the command line')
|
||||
src.add_argument('--b64-file', help='file containing the base64 program literal')
|
||||
src.add_argument('--hex-string', help='hex program literal (two chars per byte) on the command line')
|
||||
src.add_argument('--hex-file', help='file containing the hex program literal')
|
||||
src.add_argument('--bytes-file', help='file containing the raw already-decoded byte stream')
|
||||
src.add_argument('--selftest', action='store_true',
|
||||
help='prove the encode/decode round trip on fixed and random vectors, then exit')
|
||||
ap.add_argument('--json', action='store_true', help='emit a JSON object instead of lines')
|
||||
ap.add_argument('--limit', type=int, default=0,
|
||||
help='stop after N decoded values (0 = all)')
|
||||
args = ap.parse_args(argv)
|
||||
|
||||
if args.selftest:
|
||||
return selftest()
|
||||
|
||||
if args.b64_string is not None:
|
||||
data = b64_to_bytes(args.b64_string)
|
||||
elif args.b64_file is not None:
|
||||
data = b64_to_bytes(open(args.b64_file, encoding='utf-8').read())
|
||||
elif args.hex_string is not None:
|
||||
data = hex_to_bytes(args.hex_string)
|
||||
elif args.hex_file is not None:
|
||||
data = hex_to_bytes(open(args.hex_file, encoding='utf-8').read())
|
||||
else:
|
||||
data = open(args.bytes_file, 'rb').read()
|
||||
|
||||
try:
|
||||
decoded = decode_program(data)
|
||||
except ValueError as exc:
|
||||
print('error: %s' % exc, file=sys.stderr)
|
||||
return 2
|
||||
|
||||
if args.limit:
|
||||
decoded = decoded[:args.limit]
|
||||
if args.json:
|
||||
print(json.dumps({
|
||||
'input_bytes': len(data),
|
||||
'values': [v for _, v in decoded],
|
||||
'min': min((v for _, v in decoded), default=None),
|
||||
'max': max((v for _, v in decoded), default=None),
|
||||
}))
|
||||
else:
|
||||
for off, value in decoded:
|
||||
print('%6d %d' % (off, value))
|
||||
print('# %d values from %d bytes (opcode stream starts here; '
|
||||
'stage identities and caveats: references/jsvmp-recovery.md §6)'
|
||||
% (len(decoded), len(data)), file=sys.stderr)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,78 @@
|
||||
"""Round-trip and known-vector tests for the jsvmp loader decoder.
|
||||
|
||||
The decoder implements the loader chain documented in
|
||||
`skills/apk-reverse/references/jsvmp-recovery.md` (base64/hex -> varint -> ZigZag).
|
||||
Its evidence is the round trip plus pinned byte layouts -- no real jsvmp target was
|
||||
ever decoded in this repository, which is why these tests pin encodings that do not
|
||||
depend on the encoder under test having written them.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
sys.path.insert(0, os.path.join(os.path.dirname(__file__), '..', '..', 'skills',
|
||||
'apk-reverse', 'scripts'))
|
||||
|
||||
from jsvmp_loader_decode import ( # noqa: E402
|
||||
b64_to_bytes, decode_program, hex_to_bytes, main, varint_decode, varint_encode,
|
||||
zigzag_decode, zigzag_encode,
|
||||
)
|
||||
|
||||
|
||||
def test_zigzag_known_vectors():
|
||||
# The write-up's formula (n >> 1) ^ -(n & 1): neighbours interleave 0,1,2,3...
|
||||
assert [zigzag_decode(u) for u in (0, 1, 2, 3)] == [0, -1, 1, -2]
|
||||
assert [zigzag_encode(n) for n in (0, -1, 1, -2)] == [0, 1, 2, 3]
|
||||
assert zigzag_encode(-1) == 1 and zigzag_decode(1) == -1
|
||||
|
||||
|
||||
def test_varint_known_layout():
|
||||
# Raw varint(300) = AC 02 is the canonical protobuf example; the decode-side layout
|
||||
# must not depend on this repository's encoder having produced it.
|
||||
assert varint_decode(b'\xac\x02') == [(0, 300)]
|
||||
assert varint_decode(b'\x7f\x80\x01') == [(0, 127), (1, 128)]
|
||||
# varint_encode is the full-chain encode side: it zigzags FIRST, so signed 300
|
||||
# encodes as zigzag(300)=600 -> D8 04 (and raw 600 would be AC 04, not AC 02).
|
||||
assert varint_encode([1, -1]) == b'\x02\x01'
|
||||
assert varint_encode([300]) == b'\xd8\x04'
|
||||
|
||||
|
||||
def test_varint_truncation_raises():
|
||||
# A stream that ends mid-value is truncation (or not varint) -- it must raise
|
||||
# rather than silently decode short.
|
||||
for truncated in (b'\xac', b'\x80'):
|
||||
try:
|
||||
varint_decode(truncated)
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError('truncated varint %r decoded silently' % truncated)
|
||||
|
||||
|
||||
def test_round_trip_random_and_extremes():
|
||||
import random
|
||||
rng = random.Random(7)
|
||||
values = [0, 1, -1, 127, -127, 128, -128, 2 ** 31 - 1, -(2 ** 31)]
|
||||
values += [rng.randrange(-(2 ** 31), 2 ** 31) for _ in range(500)]
|
||||
decoded = [v for _, v in decode_program(varint_encode(values))]
|
||||
assert decoded == values
|
||||
|
||||
|
||||
def test_b64_and_hex_frontends():
|
||||
blob = varint_encode([1, -1, 300])
|
||||
import base64
|
||||
assert b64_to_bytes(base64.b64encode(blob).decode()) == blob
|
||||
# Stripped padding (the write-up's loader strips '=') must still decode.
|
||||
text = base64.b64encode(blob).decode().rstrip('=')
|
||||
assert b64_to_bytes(text) == blob
|
||||
assert hex_to_bytes(blob.hex().upper()) == blob
|
||||
|
||||
|
||||
def test_cli_selftest_and_decode(capsys):
|
||||
assert main(['--selftest']) == 0
|
||||
assert 'OK' in capsys.readouterr().out
|
||||
encoded = varint_encode([5, -7])
|
||||
import base64
|
||||
rc = main(['--b64-string', base64.b64encode(encoded).decode(), '--json'])
|
||||
out = capsys.readouterr()
|
||||
assert rc == 0
|
||||
assert '"values": [5, -7]' in out.out
|
||||
Reference in New Issue
Block a user