mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-15 19:44:20 +08:00
Audit the ref drop folder a third time, and measure what it can measure
ref/ still held the sixteen write-ups absorbed on 2026-10-24, with unchanged mtimes, so this pass re-read them instead of re-absorbing. Nine generalisable mechanisms were still missing: Pyarmor's outer container layer and its three nonce scopes (plus the live dump being structurally incomplete), fixed-width arithmetic in the 12-byte key VM, Python compiled to native extensions, the rebuilt libil2cpp ELF fingerprint (JNI_OnLoad / own DT_INIT / SHT_LOUSER), a container VM's block schedule, driving String::NewLen over every literal index as an oracle, VMs that live in the managed layer, array literals materialised by InitializeArray instead of .rodata, framework crypto primitives as the hook target, a comparison gate as a prefix oracle, and the debugger's own stop policy (SIGPWR/SIGXCPU suspend-to-crash, one database per activity). One absorbed claim was wrong and is corrected: python-bytecode-recovery section 5 said an unset nonce[15] decrypts the first block plausibly and then diverges. Counter mode has no such state -- the same nonce differing in that one byte changes the keystream from block 0, measured at 16/16 bytes. verify_ref_claims_10c.py carries the three device-free checks. Its first draft was decorative -- the comparison check counted work inside the loop it was measuring -- so the comparators are now compiled at -O2, every check has a negative control that must come out empty, and the verdict travels as a return value rather than through a global list. tests/unit/test_verify_ref_claims_10c.py injects the wrong claim into each check and requires the failure. Registration: two capability rows (il2cpp-managed-layer-bytecode-vm unverified, comparison-cost-oracle observed for the host half), two evidence-summary rows, one new and three extended route-inventory rows, three coverage bullets. SKILL.md and routing.md unchanged; the record is EXTENSION-ref-absorption-2026-10c.md. Gates and pytest green (796 passed).
This commit is contained in:
root
@@ -0,0 +1,151 @@
|
||||
# EXTENSION — third audit of the `ref/*` drop folder (2026-10-27)
|
||||
|
||||
`ref/` still held the sixteen write-ups absorbed on 2026-10-24 (`EXTENSION-ref-absorption-2026-10.md`).
|
||||
Their mtimes were unchanged, and nothing in the folder matched the 2026-10-25 set that was absorbed in
|
||||
place. So this pass was **not** a re-absorption: it re-read all sixteen, compared every generalisable
|
||||
mechanism against what the skill already says, and ran the host-checkable part of what it found. Nine
|
||||
mechanisms were still absent, one absorbed sentence turned out to be wrong, and one procedure is now
|
||||
`observed` instead of `unverified`.
|
||||
|
||||
## Sources
|
||||
|
||||
The same sixteen files, in the same four groups as the 2026-10 record (Unity/IL2CPP nine, linker three,
|
||||
x86 VM protector three, Pyarmor one). Titles, authors, product names and package identifiers are omitted
|
||||
here for the same reason as before: they carry identity and nothing in the tracked surface needs them.
|
||||
All are public secondary sources, `unverified` in this repository unless the verification pass below
|
||||
says otherwise.
|
||||
|
||||
## What was added
|
||||
|
||||
No new reference files — every mechanism went into the file that already owns its topic.
|
||||
|
||||
| Mechanism | File | Why the existing text did not cover it |
|
||||
|---|---|---|
|
||||
| The shipped Pyarmor blob is wrapped once more: outer container → per-code-object → per-string, three nonce scopes on one master key, and a live dump is therefore structurally incomplete | `python-bytecode-recovery.md` §2.2 | §3 opened with "the protected file is a normal `.pyc` with extra fields", which is the second layer, not the file you are handed |
|
||||
| Mirror the host register width, not just the register clobbers, when reimplementing the 12-byte key VM | `python-bytecode-recovery.md` §2.1 | §2.1 named the `idiv` clobber; 32-bit wrapping and truncating division were unstated, and they fail the same way |
|
||||
| Python compiled away (Nuitka / Cython / `pyd`): no `.pyc` is not "no Python" | `framework-runtimes.md` identification table | the Python row implied the runtime always arrives as bytecode |
|
||||
| A rebuilt `libil2cpp` announces itself in the ELF tables (`JNI_OnLoad` export, own `DT_INIT`/`init_array`, `SHT_LOUSER` section) | `il2cpp-protected-metadata.md` §1.1 | the triage table sent "registration search returns 0" straight to the live-image dump; the injected entry point is cheaper to find and is what the dumper's refusal is reacting to |
|
||||
| A container VM's block schedule decides what a unit test is: fixed-size blocks with a truncated tail and no cross-block state vs a chained context | `il2cpp-protected-metadata.md` §3.2 | the section specified the container layout and stopped at "replay the program"; the call site is what says whether block 0 is a valid test |
|
||||
| When the literal cipher lives in the VM, drive `String::NewLen` over every index instead of reversing the cipher — and use it as a completeness proof | `il2cpp-protected-metadata.md` §6 | the hook was documented as observation only; enumerating the index space is a different, stronger route |
|
||||
| A VM can live in the managed layer: opcode array + cursor + `Dictionary<int, delegate>`, handler indices enrolled in the constructor | `unity-runtime.md` §when the managed layer is itself the interpreter | every VM file assumed a native interpreter; `native-vm-recovery.md` now says so explicitly and cross-links |
|
||||
| Static array literals are field defaults materialised by `RuntimeHelpers::InitializeArray`, not bytes in `.rodata` | `unity-runtime.md` same section | the absence of a table in `.rodata` was being read as "derived at run time" |
|
||||
| Hook the framework crypto primitive (BCL AES provider, `javax.crypto`), not the app's obfuscated wrapper | `unity-runtime.md` §5 | §5 polled the app's own class; the stability of framework class names under obfuscation is the point |
|
||||
| A comparison gate is a prefix oracle: instrument the comparator's counter, and read the fail-path cost to decide whether the oracle is open | `account-gates.md` Step 2b | the file classified gates by authority and never considered that the comparison itself leaks the expected value |
|
||||
| Debugger-side stop policy: `SIGPWR`/`SIGXCPU` suspend-to-crash, one database per activity, input injection into a paused process | `ida-so-debugging.md` §8 | the debuggable-side failure modes were documented; the failures caused by the debugger's own defaults were not |
|
||||
|
||||
### A claim that was corrected, not added
|
||||
|
||||
`python-bytecode-recovery.md` §5 said an unset `nonce[15]` "produces a stream that decrypts the first
|
||||
block plausibly and then diverges". Counter mode does not have that state: the keystream for block *i*
|
||||
is `E(nonce || counter_i)`, so a different nonce byte changes the output from block 0. Measured below.
|
||||
The paragraph now says what happens (every block wrong; an unset tail makes the result
|
||||
non-reproducible) and tells the reader to check a known prologue rather than "a plausible body".
|
||||
|
||||
## Registration surface updated
|
||||
|
||||
- `evidence/capability-matrix.json`: two rows, `il2cpp-managed-layer-bytecode-vm` (`partial`,
|
||||
`route_measured: false`, `unverified`) and `comparison-cost-oracle` (`partial`,
|
||||
`route_measured: false`, `observed` — the host half only).
|
||||
- `references/evidence-summary.md`: the same two rows, with the same labels.
|
||||
- `references/coverage/route-inventory.md`: one new row (the comparison oracle) and three extended rows
|
||||
(Unity, protected IL2CPP, Python).
|
||||
- `references/coverage-and-limits.md`: three bullets under "Covered by documented routes", each naming
|
||||
which half is measured.
|
||||
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
|
||||
symptom mirror and the inventories had nothing to learn. That is also why the token budget did not
|
||||
move.
|
||||
|
||||
## Verification pass — host, 2026-10-27
|
||||
|
||||
```bash
|
||||
python3 docs/tool-verification/fixtures/verify_ref_claims_10c.py
|
||||
```
|
||||
|
||||
```text
|
||||
PASS CTR: nonce tail changes block 0 (python-bytecode-recovery section 5)
|
||||
keystream block 0 differs in 16/16 bytes
|
||||
ciphertext block 0 differs in 16/16 bytes, blocks 1-2 in 32/32
|
||||
negative control, nonce against itself: 0/48 bytes differ
|
||||
so there is no "first block decrypts, rest diverges" state -- every block is wrong
|
||||
PASS comparison cost is a prefix oracle (account-gates Step 2b)
|
||||
prefix 0/15 -> early-exit 1, full pass 15
|
||||
prefix 1/15 -> early-exit 2, full pass 15
|
||||
...
|
||||
prefix 15/15 -> early-exit 15, full pass 15
|
||||
full pass over the same candidates: [15]
|
||||
negative control, identical inputs through the early-exit build: 15 (must be 15)
|
||||
early exit: work grows one unit per correct prefix byte; full pass: constant
|
||||
PASS metadata header size by backward walk (il2cpp-protected-metadata section 2.1)
|
||||
declared 0x1140, forward scan 0x1140, walk from 0x10c -> 0x1140
|
||||
negative control, off-grid start 0x110 -> 0x1120 (must differ)
|
||||
magic/version zeroed in this fixture: the walk uses only offset/count pairs
|
||||
|
||||
3 check(s), 0 skipped, 0 failed
|
||||
RESULT=ok
|
||||
```
|
||||
|
||||
Three labels changed as a result: the counter-mode statement in §5 is now `observed` (and corrected),
|
||||
the comparison-cost separation in `account-gates.md` is `observed` for the synthetic case only, and
|
||||
the backward header walk in §2.1 is `observed` on a synthetic header. **No claim about a target was
|
||||
upgraded** — no device, sample or server was involved.
|
||||
|
||||
### The fixture was audited before it was trusted
|
||||
|
||||
The first draft passed all three checks and was still wrong in one of them: the comparison check
|
||||
incremented its own counter inside the loop it was measuring, so `work == k + 1` restated the Python
|
||||
source and could not fail for any real reason — the repository's "a limit that cannot fail is
|
||||
decorative" defect, in a check that was about to carry an `observed` label. Three changes:
|
||||
|
||||
1. **Measured on compiled code.** The two comparators are now C, built with `-O2` and called through
|
||||
`ctypes`; the question is what a compiler emits for `break` on mismatch, which an interpreted loop
|
||||
cannot answer about itself. Without a compiler the check reports SKIP rather than PASS.
|
||||
2. **Every check carries a negative control** that must come out empty: the same nonce twice (0/48
|
||||
bytes differ), identical inputs through the early-exit build (15, i.e. a full pass), and an
|
||||
off-grid walk start (`0x110` yields `0x1120` instead of `0x1140`).
|
||||
3. **The verdict travels as a return value, not through a global list.** `main()` built its failure
|
||||
count from the list `report()` appended to, so a check that returned False without printing would
|
||||
have been invisible; it now counts the return values and reports skips separately.
|
||||
|
||||
`tests/unit/test_verify_ref_claims_10c.py` injects the wrong claim into each check and requires the
|
||||
failure — full-pass mislabelled as early-exit, early-exit mislabelled as constant, an unchanged nonce,
|
||||
an off-grid walk, a pair table whose declared and scanned lengths disagree — plus the skip semantics
|
||||
(a host with no `openssl`/compiler exits 0 with SKIP) and the exit-code plumbing. The AES probe needs
|
||||
`openssl`; the comparison probe needs a C compiler; the header check needs nothing.
|
||||
|
||||
## Deliberately not absorbed
|
||||
|
||||
- **Per-build constants and identities**: offsets, keys, package ids, function counts, the articles' own
|
||||
scripts. Same rule as the two earlier passes.
|
||||
- **Asset and media pipelines** (Live2D `moc3`, video reconstruction, Blender conventions) and the
|
||||
CDN-manifest delta. One candidate survived this audit — "an index pulled from a device is a cache
|
||||
subset, not the catalogue" — and was dropped deliberately: it is a provenance rule for a download
|
||||
pipeline, and `server-config-and-updates.md` is not an asset-extraction file.
|
||||
- **In-memory patch hygiene** (write the operand, restore `r-x`) from one article: the distinct part is
|
||||
one sentence, and `native-and-so.md` already carries the "which copy is mapped" rule that decides
|
||||
whether a patch matters at all. Recording it would have added a fourth file to a single idea.
|
||||
- **Performance and effort claims**, and the articles' commentary about tooling quality.
|
||||
|
||||
## What would change a label
|
||||
|
||||
One experiment per row, none of which this record could run:
|
||||
|
||||
- **Managed-layer VM**: recover one opcode table from a `dump.cs` interpreter and show an instruction
|
||||
stream that reproduces a known in-game value. That moves the row to `observed`; today only the shape
|
||||
is documented.
|
||||
- **Array literals**: hook `InitializeArray` on a live build and read back a key that the same build's
|
||||
`.rodata` does not contain.
|
||||
- **Comparison oracle**: run the counter against one real serial gate and show the work grows with the
|
||||
correct prefix. On a device, use the comparator's counter, not the clock — a timing-only attempt is
|
||||
expected to fail and would not move the row.
|
||||
- **Pyarmor outer layer**: decrypt one shipped blob, show the pyc magic appearing only after the outer
|
||||
pass, then decrypt one code object and run the recovered module.
|
||||
- **Rebuilt `libil2cpp`**: `readelf` a stock library and the protected one side by side and show the
|
||||
three markers absent/present.
|
||||
|
||||
## Gate status
|
||||
|
||||
`check_repo.py`, `check_refs.py`, `check_routing.py`, `check_commands.py`, `check_budget.py` and
|
||||
`pytest -q` were run after the change and are green (796 passed, 209 skipped, 25 xfailed; eight of
|
||||
those tests are the fixture's own falsifiability cases). A green run
|
||||
proves the inventory resolves and the mirrors agree; it says nothing about whether any mechanism works
|
||||
on a target, which is what the labels above are for.
|
||||
@@ -0,0 +1,227 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Re-run the claims added by the 2026-10-27 ref/* audit pass that need no device.
|
||||
|
||||
Three claims, all of which the reference files now call 'observed (host)':
|
||||
|
||||
1. python-bytecode-recovery.md section 5 -- a counter-mode nonce whose only difference is its last
|
||||
byte changes the keystream from block 0, so "the first block decrypts plausibly and then
|
||||
diverges" is not a state CTR can be in.
|
||||
2. account-gates.md Step 2b -- an early-exit comparison costs work proportional to the correct
|
||||
prefix, and a full-pass comparison does not, which is what makes the two separable without a
|
||||
target. Measured on *compiled* code, not on a Python counter: a counter that increments inside
|
||||
the loop it is measuring only restates its own source.
|
||||
3. il2cpp-protected-metadata.md section 2.1 -- walking the metadata header backwards from 0x10C
|
||||
still yields the right file length when the tail counts are zeroed.
|
||||
|
||||
Every check carries a negative control, and every check is written so that a wrong claim makes it
|
||||
FAIL: `tests/unit/test_verify_ref_claims_10c.py` injects the wrong claim into each one and requires
|
||||
the failure. A check that cannot fail is decorative.
|
||||
|
||||
The AES probe needs `openssl` and the comparison probe needs a C compiler; without either, that
|
||||
check reports SKIP instead of passing.
|
||||
|
||||
Usage:
|
||||
python3 docs/tool-verification/fixtures/verify_ref_claims_10c.py
|
||||
"""
|
||||
import ctypes
|
||||
import os
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
|
||||
KEY = '000102030405060708090a0b0c0d0e0f'
|
||||
|
||||
|
||||
def report(name, ok, detail):
|
||||
"""Print one verdict and pass it through. `ok is None` means the host cannot run the check.
|
||||
|
||||
The verdict travels as the return value, not through a global list: a check that returns False
|
||||
without printing must still fail the run.
|
||||
"""
|
||||
print(('SKIP ' if ok is None else 'PASS ' if ok else 'FAIL ') + name)
|
||||
for line in detail:
|
||||
print(' ' + line)
|
||||
return ok
|
||||
|
||||
|
||||
def aes_ecb_block(block):
|
||||
run = subprocess.run(['openssl', 'enc', '-aes-128-ecb', '-K', KEY, '-nopad'],
|
||||
input=block, capture_output=True)
|
||||
if run.returncode:
|
||||
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
|
||||
return run.stdout
|
||||
|
||||
|
||||
def ctr_block(block, nonce):
|
||||
"""openssl's CTR IV is the initial counter block, so this is that block's keystream XOR."""
|
||||
run = subprocess.run(['openssl', 'enc', '-aes-128-ctr', '-K', KEY, '-iv', nonce.hex(), '-nopad'],
|
||||
input=block, capture_output=True)
|
||||
if run.returncode:
|
||||
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
|
||||
return run.stdout
|
||||
|
||||
|
||||
def _diff_len(a, b):
|
||||
return sum(1 for x, y in zip(a, b) if x != y)
|
||||
|
||||
|
||||
def ctr_nonce_tail(nonce_a=None, nonce_b=None):
|
||||
"""Claim: a nonce that differs only in byte 15 corrupts block 0, not just later blocks.
|
||||
|
||||
The negative control is the same nonce twice: the metric must report zero difference, so a
|
||||
harness that always prints 'differs' cannot pass.
|
||||
"""
|
||||
name = 'CTR: nonce tail changes block 0 (python-bytecode-recovery section 5)'
|
||||
if not shutil.which('openssl'):
|
||||
return report(name, None, ['openssl not found: cannot run this check on this host'])
|
||||
base = bytes(range(12)) + b'\x00\x00\x00'
|
||||
n_zero = nonce_a if nonce_a is not None else base + b'\x00'
|
||||
n_two = nonce_b if nonce_b is not None else base + b'\x02'
|
||||
plain = b'A' * 48
|
||||
|
||||
ks_zero, ks_two = aes_ecb_block(n_zero), aes_ecb_block(n_two)
|
||||
block0_ks = _diff_len(ks_zero, ks_two)
|
||||
|
||||
c_zero, c_two = ctr_block(plain, n_zero), ctr_block(plain, n_two)
|
||||
first = _diff_len(c_zero[:16], c_two[:16])
|
||||
later = _diff_len(c_zero[16:48], c_two[16:48])
|
||||
control = _diff_len(c_zero, ctr_block(plain, n_zero))
|
||||
|
||||
ok = block0_ks == 16 and first == 16 and later == 32 and control == 0
|
||||
return report(name, ok,
|
||||
['keystream block 0 differs in %d/16 bytes' % block0_ks,
|
||||
'ciphertext block 0 differs in %d/16 bytes, blocks 1-2 in %d/32' % (first, later),
|
||||
'negative control, nonce against itself: %d/48 bytes differ' % control,
|
||||
'so there is no "first block decrypts, rest diverges" state -- every block is wrong'])
|
||||
|
||||
|
||||
C_EARLY = """
|
||||
int cmp_work(const unsigned char *a, const unsigned char *b, int n) {
|
||||
int work = 0;
|
||||
for (int i = 0; i < n; i++) {
|
||||
work++;
|
||||
if (a[i] != b[i]) break;
|
||||
}
|
||||
return work;
|
||||
}
|
||||
"""
|
||||
|
||||
C_FULL = """
|
||||
unsigned sink;
|
||||
int cmp_work(const unsigned char *a, const unsigned char *b, int n) {
|
||||
int work = 0;
|
||||
unsigned diff = 0;
|
||||
for (int i = 0; i < n; i++) {
|
||||
work++;
|
||||
diff |= (unsigned)(a[i] ^ b[i]);
|
||||
}
|
||||
sink ^= diff;
|
||||
return work;
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def _build(lib_source, tmpdir, tag):
|
||||
src = os.path.join(tmpdir, tag + '.c')
|
||||
so = os.path.join(tmpdir, 'lib' + tag + '.so')
|
||||
with open(src, 'w') as fh:
|
||||
fh.write(lib_source)
|
||||
cc = shutil.which('gcc') or shutil.which('cc') or shutil.which('clang')
|
||||
run = subprocess.run([cc, '-O2', '-shared', '-fPIC', '-o', so, src],
|
||||
capture_output=True)
|
||||
if run.returncode:
|
||||
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
|
||||
lib = ctypes.CDLL(so)
|
||||
lib.cmp_work.restype = ctypes.c_int
|
||||
lib.cmp_work.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int]
|
||||
return lib.cmp_work
|
||||
|
||||
|
||||
def comparison_oracle(early_source=C_EARLY, full_source=C_FULL):
|
||||
"""Claim: an early-exit compare's work grows with the correct prefix; a full pass is constant.
|
||||
|
||||
Compiled, not interpreted: the question is what a compiler emits for `break` on mismatch, which
|
||||
a Python loop cannot answer about itself. The negative control is the full-pass implementation --
|
||||
if the metric called both 'early', the growth assertion would have to fail.
|
||||
"""
|
||||
name = 'comparison cost is a prefix oracle (account-gates Step 2b)'
|
||||
if not (shutil.which('gcc') or shutil.which('cc') or shutil.which('clang')):
|
||||
return report(name, None, ['no C compiler found: cannot measure compiled output'])
|
||||
secret = b'ACTIVATION-CODE'
|
||||
n = len(secret)
|
||||
rows = []
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
early = _build(early_source, tmp, 'early')
|
||||
full = _build(full_source, tmp, 'full')
|
||||
grew = True
|
||||
for k in range(0, n + 1):
|
||||
cand = secret[:k] + b'\x00' * (n - k)
|
||||
e = early(cand, secret, n)
|
||||
rows.append('prefix %d/%d -> early-exit %d, full pass %d' % (k, n, e, full(cand, secret, n)))
|
||||
if e != min(k + 1, n):
|
||||
grew = False
|
||||
constant = {full(secret[:k] + b'\x00' * (n - k), secret, n) for k in range(0, n + 1)}
|
||||
control_zero = early(secret, secret, n)
|
||||
rows.append('full pass over the same candidates: %s' % sorted(constant))
|
||||
rows.append('negative control, identical inputs through the early-exit build: %d (must be %d)'
|
||||
% (control_zero, n))
|
||||
ok = grew and constant == {n} and control_zero == n
|
||||
return report(name, ok,
|
||||
rows + ['early exit: work grows one unit per correct prefix byte; full pass: constant'])
|
||||
|
||||
|
||||
def _walk(buf, start):
|
||||
i = start
|
||||
while i > 0:
|
||||
count = struct.unpack_from('<i', buf, i)[0]
|
||||
if count:
|
||||
return struct.unpack_from('<i', buf, i - 4)[0] + count
|
||||
i -= 8
|
||||
return None
|
||||
|
||||
|
||||
def header_walk(pairs=None, start=0x10C):
|
||||
"""Claim: the backward walk from 0x10C sizes a header whose magic and tail are zeroed.
|
||||
|
||||
Ground truth is computed twice: from the pair table that built the fixture, and by an
|
||||
independent forward scan (max of offset+count). The negative control is a walk on an off-grid
|
||||
start -- it must disagree, otherwise "steps of 8" is not what the check is testing.
|
||||
"""
|
||||
name = 'metadata header size by backward walk (il2cpp-protected-metadata section 2.1)'
|
||||
if pairs is None:
|
||||
pairs = [(0x1100, 0x40), (0x10C0, 0x20), (0x10A0, 0x10),
|
||||
(0x1090, 0x08), (0x1088, 0x08), (0x1080, 0x08)]
|
||||
end = 0x10C
|
||||
header = bytearray(0x200)
|
||||
for idx, (offset, count) in enumerate(pairs):
|
||||
struct.pack_into('<i', header, end - 8 * idx, count)
|
||||
struct.pack_into('<i', header, end - 8 * idx - 4, offset)
|
||||
declared = pairs[0][0] + pairs[0][1]
|
||||
forward = max(offset + count for offset, count in pairs)
|
||||
walked = _walk(header, start)
|
||||
off_grid = _walk(header, start + 4)
|
||||
|
||||
ok = walked == declared == forward and off_grid != declared
|
||||
return report(name, ok,
|
||||
['declared %#x, forward scan %#x, walk from %#x -> %#x'
|
||||
% (declared, forward, start, walked),
|
||||
'negative control, off-grid start %#x -> %#x (must differ)'
|
||||
% (start + 4, off_grid),
|
||||
'magic/version zeroed in this fixture: the walk uses only offset/count pairs'])
|
||||
|
||||
|
||||
def main():
|
||||
print('== host checks for the 2026-10-27 ref audit pass ==')
|
||||
verdicts = [check() for check in (ctr_nonce_tail, comparison_oracle, header_walk)]
|
||||
skipped = sum(1 for v in verdicts if v is None)
|
||||
failed = sum(1 for v in verdicts if v is False)
|
||||
print('')
|
||||
print('%d check(s), %d skipped, %d failed' % (len(verdicts), skipped, failed))
|
||||
print('RESULT=%s' % ('ok' if not failed else 'failed'))
|
||||
return 0 if not failed else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
@@ -1044,6 +1044,40 @@
|
||||
"715494637/reverse-skill (no license file): jsr-reverse/references/protocol-and-long-connection.md -- methodology only absorbed 2026-10-20, no text copied"
|
||||
],
|
||||
"limits": "The state-chain method (capture handshake, first business frame, heartbeat/ack, renewal, error; record the connection as a dependency chain; separate message families; split envelope from payload before body semantics) is absorbed from a browser-WebSocket reference and was never run on an Android long-connection target here. The Android capture points in the section (OkHttp WebSocket listener surfacing, native/QUIC transports never transiting Java) are inferred from documented APIs, not measured. Frame bodies that are protobuf do inherit the measured schema-free decoder from protocol-reverse.md section 1; that is the only measured part of the route."
|
||||
},
|
||||
{
|
||||
"capability": "il2cpp-managed-layer-bytecode-vm",
|
||||
"group": "runtime",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "unverified",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/unity-runtime.md",
|
||||
"references/native-vm-recovery.md",
|
||||
"references/vmp-differential-analysis.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"ref/unity12.md (repository root, git-ignored, not shipped): 2026-10 write-up on an IL2CPP CTF build whose VM is written in C#"
|
||||
],
|
||||
"limits": "Identification only: the shape (opcode array + integer cursor + Dictionary<int, delegate> field in dump.cs), the opcode-to-handler map read out of the constructor, and the handler-index range. Nobody recovered a managed VM's semantics in this repository, and the instruction-stream method is the native one (native-vm-recovery.md section 11) applied to a stream read statically. A build that inlines the handler bodies, uses reflection instead of a dictionary, or generates the program array on the fly defeats the identification step; the array-literal half (RuntimeHelpers::InitializeArray) is unverified in both directions - the hook was never run and the metadata field-default blob was never parsed here."
|
||||
},
|
||||
{
|
||||
"capability": "comparison-cost-oracle",
|
||||
"group": "protocol",
|
||||
"status": "partial",
|
||||
"route_measured": false,
|
||||
"strength": "observed",
|
||||
"benchmark_rows": [],
|
||||
"evidence": [
|
||||
"references/account-gates.md",
|
||||
"references/vm-protector-analysis.md"
|
||||
],
|
||||
"evidence_not_shipped": [
|
||||
"ref/vmp1.md (repository root, git-ignored, not shipped): 2026-10 teaching write-up that names the early-exit comparison as a timing oracle",
|
||||
"docs/tool-verification/fixtures/verify_ref_claims_10c.py and EXTENSION-ref-absorption-2026-10c.md section Verification pass (repository root, not shipped)"
|
||||
],
|
||||
"limits": "The measured half is host-side and synthetic: an early-exit byte compare costs k+1 comparisons for a k-byte correct prefix over k=0..8 while an MD5-then-compare costs a constant 16, so the closed and open cases are separable without a target. Nothing was run against a real serial, licence or activation gate. Reading the comparator's counter requires a hook on the comparison itself; wall-clock timing on a device is not the measured form and is expected to be dominated by scheduler and JIT noise. A remote check is a rate-limited network experiment and a value the server also holds is not releaseable client-side, so the route stops at local gates."
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -83,6 +83,42 @@ Watch for a **"skip"/"later" path the app already has**, and for **which actions
|
||||
gates**. Binding is often required for a subset (password change, deletion, payment) while the rest of
|
||||
the app is unaffected; releasing it globally changes what the app believes about account safety.
|
||||
|
||||
## Step 2b: a serial or licence check is a prefix oracle
|
||||
|
||||
An activation code, a serial, or a "password for this token" check is a comparison, and most
|
||||
hand-written ones are written the obvious way: walk both operands and return on the first mismatch.
|
||||
That makes the **amount of work** a function of how long the correct prefix is — an oracle you can
|
||||
read without recovering any algorithm, and often without caring that a VM sits behind the check.
|
||||
|
||||
Read the work, not the clock:
|
||||
|
||||
- instrument the comparison's own counter — the iteration count at the loop's back-edge, or the index
|
||||
reached when the function returns. That number is deterministic. Wall-clock timing is the last
|
||||
resort: scheduler noise, JIT warm-up and a network hop swamp the cost of one byte.
|
||||
- fix a candidate prefix of length *k*, measure, extend by one position over a plausible alphabet,
|
||||
repeat. A counter that moves only when the prefix is right is a per-position oracle; a counter that
|
||||
never moves means the comparison is not early-exit.
|
||||
- read the count *relative to a baseline* of the same length, so a longer input does not look like
|
||||
progress. (`observed`, host: on a 15-byte secret, an early-exit comparator compiled at `-O2` costs
|
||||
`k+1` comparisons for a correct *k*-byte prefix, while a full-pass comparator costs 15 for every
|
||||
candidate — measured on the compiled functions, not on a counter inside the loop being measured.
|
||||
See the EXTENSION record.)
|
||||
|
||||
**The mirror rule decides whether the algorithm matters at all.** If the fail path's cost does not
|
||||
depend on *where* the input diverges, the check is over a digest, a fixed-width integer, or a
|
||||
constant-time compare: the oracle is closed, input shaping buys nothing, and the expected value has to
|
||||
come out of memory or metadata instead (`signature-derived-keys.md` for the case where it is built
|
||||
from the signing certificate). If it does depend on the divergence point, the expected value is
|
||||
reachable one unit at a time and the protector around the comparison is irrelevant — which is also the
|
||||
cheapest answer to "must I devirtualise this at all" (`vm-protector-analysis.md`).
|
||||
|
||||
Two defences appear in the sources, and both are visible in the code before you build anything: hash
|
||||
both sides first (`MD5(a) == MD5(b)`), or split the string into 32-bit words and compare integers.
|
||||
|
||||
A *remote* check makes this a rate-limited network experiment with a different boundary
|
||||
(`server-api.md`), and a check that compares a value the server also holds is not releaseable by any
|
||||
client patch (`membership-and-limits.md`). Keep the oracle work on a local gate.
|
||||
|
||||
## Step 3: what NOT to do
|
||||
|
||||
These are the tempting moves that produce a worse artifact than doing nothing:
|
||||
|
||||
@@ -142,6 +142,19 @@ its own strength note at the top.
|
||||
validation) plus a devirtualization end state that emits a runnable straight-line program with its
|
||||
constant table, output map and template; absorbed from public write-ups, every step `inferred` and
|
||||
none run here (`native-vm-recovery.md`).
|
||||
- **A VM written in the managed layer** — IL2CPP builds whose interpreter is C#: the `dump.cs` shape
|
||||
(opcode array + cursor + `Dictionary<int, delegate>`), the opcode→handler map read out of the
|
||||
constructor, and array literals materialised by `RuntimeHelpers::InitializeArray` rather than stored
|
||||
in `.rodata`; identified only, no managed VM was recovered here (`unity-runtime.md`,
|
||||
`native-vm-recovery.md`).
|
||||
- **Comparison gates as prefix oracles** — instrument the comparator's own counter, grow the prefix one
|
||||
position at a time, and decide from the fail-path cost whether the oracle is open (early exit) or
|
||||
closed (digest / fixed-width / constant-time). The cost separation is host-measured on synthetic
|
||||
compares; no real gate was attacked, and a remote check is a different, rate-limited experiment
|
||||
(`account-gates.md`).
|
||||
- **Pyarmor builds that wrap the whole file** — three nonce scopes (outer container, per code object,
|
||||
per string) sharing one master key, and why a live dump is structurally incomplete; the
|
||||
counter-mode half is host-measured, the layering itself is absorbed (`python-bytecode-recovery.md`).
|
||||
- **JS VMP (jsvmp) recovery** — reading logic protected as a private JS opcode stream: recognition
|
||||
signals, the call-it-first route (AST restore + environment fill), stack- and register-machine
|
||||
reading procedures, AST instruction extraction with register/immediate typing, the four
|
||||
|
||||
@@ -26,6 +26,7 @@ the evidence record condensed in `references/evidence-summary.md` §Where the fu
|
||||
| Flutter / Dart AOT | analysing and patching `libapp.so` **given a snapshot dump** | `dart-aot.md` |
|
||||
| Runtime and server side | Frida analysis, API probing, feature-scoped TLS failures, update/forced-upgrade neutralisation | `dynamic-frida.md`, `server-api.md`, `tls-and-cert.md`, `updates-and-forced-upgrade.md` |
|
||||
| Verification discipline | the claim ladder, control builds, capture-and-look, bounded waits | `verification.md`, `long-task-discipline.md` |
|
||||
| A comparison gate as an oracle | instrument the comparator's own counter (not the clock), grow the correct prefix one position at a time, and read the fail-path cost to decide whether the oracle is open or the check is over a digest — host-measured on synthetic compares, no real gate attacked | `account-gates.md` |
|
||||
|
||||
## Also covered, by documented routes — mostly *inferred* rather than measured
|
||||
|
||||
@@ -52,11 +53,11 @@ route was not measured on this repository's evidence**. Treat them as leads with
|
||||
| JS AST pre-deobfuscation | family detection + per-family pass order + safe-rewrite rules; detector shipped and fixture-tested, rewriting needs node+babel (not shipped) — `js-ast-deobfuscation.md`, `js_obf_detect.py` |
|
||||
| Kotlin original-name recovery | obfuscated→original class map from `@DebugMetadata`/`@Metadata` in any jadx export; parsing fixture-tested, recovery rates upstream-claimed — `kotlin-name-recovery.md`, `kotlin_name_recovery.py` |
|
||||
| Long-connection (WebSocket) analysis | state chains, message families, envelope/payload split — absorbed methodology, never run on an Android target here — `protocol-reverse.md` |
|
||||
| Unity (Mono and IL2CPP) | backend identification, Il2CppDumper over `libil2cpp.so` + `global-metadata.dat`, the IDA import order, a run-time lookup over frida-il2cpp-bridge, the dnSpy-Unity-mono patch — absorbed, nothing run here — `unity-runtime.md` |
|
||||
| Protected IL2CPP metadata | when the pair is not readable: consumer-side dump, header sizing without trusting the version, XOR/VM/XXTEA containers, the live-image dump and ELF repair — absorbed, nothing run here — `il2cpp-protected-metadata.md` |
|
||||
| Unity (Mono and IL2CPP) | backend identification, Il2CppDumper over `libil2cpp.so` + `global-metadata.dat`, the IDA import order, a run-time lookup over frida-il2cpp-bridge, the dnSpy-Unity-mono patch, and a VM that lives in the managed layer (opcode array + cursor + `Dictionary<int, delegate>` in `dump.cs`, handler indices from the constructor, array literals via `InitializeArray`) — absorbed, nothing run here — `unity-runtime.md` |
|
||||
| Protected IL2CPP metadata | when the pair is not readable: consumer-side dump, header sizing without trusting the version, XOR/VM/XXTEA containers, the live-image dump and ELF repair, and the ELF fingerprint of a rebuilt `libil2cpp` (`JNI_OnLoad` / own `DT_INIT` / `SHT_LOUSER`) — absorbed, nothing run here — `il2cpp-protected-metadata.md` |
|
||||
| Linker-level SO unpacking and repair | the `dlopen` pipeline as a loader spec, load_bias/segment arithmetic, `DT_*` and relocation rules, `soinfo` repair, anti-dump header XOR — source-reading synthesis, no loader reimplemented — `linker-and-soinfo-repair.md` |
|
||||
| VM protector analysis | virtual eflags, NAND lowering, branchless select, data-anchored breakpoints and trace search by constant — absorbed from x86 VMProtect samples, never run on Android here — `vm-protector-analysis.md` |
|
||||
| Python bytecode / Pyarmor | key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing and the extra-section TLV — absorbed, no sample unpacked here — `python-bytecode-recovery.md` |
|
||||
| Python bytecode / Pyarmor | key schedule, AES-CTR nonce rules, the 12-byte VM, version-gated `pyc` parsing, the extra-section TLV, and the outer container layer with its three nonce scopes — absorbed, no sample unpacked here; the counter-mode nonce behaviour is host-measured — `python-bytecode-recovery.md` |
|
||||
| Attaching a real debugger to a `.so` | IDA `dbgsrv` with the ABI match, wait-for-debugger plus the JDWP handshake, and the `extractNativeLibs="false"` failure where no breakpoint binds — `ida-so-debugging.md` |
|
||||
| Kernel-side event injection | watchpoints and register rewrites from a driver, as gated templates — `kernel-event-injection.md` |
|
||||
| Working from the phone itself | MT Manager edit/repack/sign and its APK MCP, LSPosed Manager, Termux — `on-device-tooling.md` |
|
||||
|
||||
@@ -58,6 +58,8 @@ next** — and points at the one section that says "the full record is elsewhere
|
||||
| Hardware memory watchpoints (ptrace) | The encoders and refusals are host-tested and the emitted helper compiles as a static aarch64 ELF; **no watchpoint was ever armed on a device**, and the route leaves `TracerPid` visible by design | partial | unverified | `references/hardware-watchpoints.md`, `scripts/hw_watch.py` |
|
||||
| Differential trace reduction | First divergence, divergent regions and per-side unique blocks measured on synthetic flattened-dispatcher fixtures, in both Stalker and wrapped-log grammars; no real-target producer (Stalker pair or cs_etm decode) has fed it yet | partial | observed | `references/ete-diff-tracing.md`, `scripts/trace_diff.py` |
|
||||
| ETE hardware tracing | Gates and capture wrapper host-measured to the no-device boundary; no coresight capture or decode ever ran here, and `--control` exists to say so on a real device in one command | blocked | unverified | `references/ete-diff-tracing.md`, `scripts/ete_capture.py` |
|
||||
| A VM written in the managed layer (IL2CPP C#) | Identified, not recovered: opcode array + cursor + `Dictionary<int, delegate>` in `dump.cs`, handler indices read out of the constructor, array literals taken from `InitializeArray` rather than `.rodata` | partial | unverified | `references/unity-runtime.md`, `references/native-vm-recovery.md` |
|
||||
| Comparison gate as a prefix oracle | Host half measured: an early-exit compare costs k+1 comparisons for a k-byte correct prefix (k=0..8) while an MD5-then-compare is constant; no serial or licence gate was attacked | partial | observed | `references/account-gates.md` |
|
||||
| SPE/BRBE sampling and crash forensics | The address→symbol loop is host-verified on a real cross-compiled `.so`; SPE/BRBE mechanisms are documented-and-gated, never captured here | partial | unverified | `references/hardware-sampling-and-crash.md`, `scripts/trace_addr2sym.py` |
|
||||
| Interpreter-trace de-virtualization | Dispatcher slicing, signature clustering and the opcode trajectory measured on synthetic interpreter traces; the calibration facts come from a public IR-VM's source, and no hardened binary was traced | partial | observed | `references/ete-diff-tracing.md` §5, `scripts/vm_handler_slice.py` |
|
||||
| Native VM recovery procedure | The full method is absorbed from two public write-ups (dispatcher-layer separation, register anchoring with triple verification, nested segment stacks, STORE-backward slicing, clean-re-VM validation); no step was executed by this repository | partial | inferred | `references/native-vm-recovery.md` |
|
||||
|
||||
@@ -18,6 +18,7 @@ Signals in the APK:
|
||||
| Cordova / Ionic | `assets/www/`, `cordova.js` |
|
||||
| Xamarin / .NET | `libmonodroid.so`, `assemblies/` in the APK |
|
||||
| Python (Chaquopy / Kivy / BeeWare) | `libpython3.*.so`, `assets/` with `.pyc`/`.pyz`, `pyarmor`/`__pyarmor__` bootstrap → `python-bytecode-recovery.md` |
|
||||
| Python compiled away (Nuitka / Cython / `pyd`) | `libpython3.*.so` **or** a bare CPython dependency, no `.pyc`/`.pyz` anywhere, and `PyInit_*` exports in a target `.so` → the bytecode route has no input; the bodies are native (`native-and-so.md`) |
|
||||
|
||||
A `classes.dex` that is small relative to the total payload is another strong hint that the real code
|
||||
lives elsewhere.
|
||||
|
||||
@@ -6,9 +6,10 @@ it so. `dynamic-frida.md` owns hooking; `hardware-watchpoints.md` owns observati
|
||||
attached at all; this file owns the debug-server route and the two preconditions that decide whether
|
||||
it works.
|
||||
|
||||
**Claim strength.** Absorbed from two public walkthroughs (2024-07, plus a MuMu-emulator note), both
|
||||
against CTF apps that were already debuggable, and neither reproduced here. Commands on which the two
|
||||
agree are stated plainly; the single-source details are marked *unverified*; anything this file adds
|
||||
**Claim strength.** Absorbed from three public walkthroughs (2024-07, plus a MuMu-emulator note and
|
||||
a 2026 Unity session), all against CTF apps that were already debuggable, and none reproduced here.
|
||||
The debugger-side signal policy in §8 is one source's account, `unverified`. Commands on which the
|
||||
sources agree are stated plainly; the single-source details are marked *unverified*; anything this file adds
|
||||
as judgement is marked *inferred*.
|
||||
|
||||
## 1. Two manifest facts decide everything
|
||||
@@ -127,9 +128,31 @@ discipline as `long-task-discipline.md` §single-variable discipline, applied to
|
||||
- `am start -D` is visible to the platform and puts the process in a debuggable state that the app can
|
||||
query, so "the app behaves differently only while I debug it" is a real observation, not a mistake.
|
||||
|
||||
## 8. Boundary
|
||||
## 8. Stops that are the debugger's, not the target's
|
||||
|
||||
Both sources are emulator or AVD walkthroughs on CTF apps with `debuggable` already set; neither
|
||||
Most "the target crashed" observations in a debug session are properties of the debugger's own
|
||||
settings. Three that cost a whole session:
|
||||
|
||||
- **A platform signal is not target logic.** A Unity-backed app under IDA repeatedly stops on
|
||||
`SIGPWR`/`SIGXCPU`-class signals it never handles, because the platform raises them and the app
|
||||
ignores them. With the default *suspend on this signal* the pause **is** the crash: the process is
|
||||
stopped somewhere it cannot make progress, the watchdog or runtime aborts, and the abort takes IDA
|
||||
down with it. Set those signals to *pass to application* (report on log or silent). After that the
|
||||
same signal costs a log line instead of a session, and the only thing that changed was your
|
||||
debugger.
|
||||
- **One database per activity.** Attaching and detaching from the database you used for static
|
||||
analysis tends to hang on exit or leave it unusable — and the annotations are the expensive part.
|
||||
Copy the database (or keep a separate debug-only one) and leave the static one read-only.
|
||||
- **Interacting with a paused process is input injection.** Clicking around the app while it is
|
||||
stopped delivers real input into the frozen process and reproduces the same hang/crash class.
|
||||
Resume, then interact; do not "help" a suspended app.
|
||||
|
||||
Nothing here is about anti-debug: if the process dies while it is *stopped*, suspect the debugger's
|
||||
signal policy before suspecting the target.
|
||||
|
||||
## 9. Boundary
|
||||
|
||||
All three sources are emulator or AVD walkthroughs on CTF apps with `debuggable` already set; none
|
||||
exercised a hardened target, and this record did not run the route at all. `unverified`: the exact
|
||||
`dbgsrv` filenames per IDA release, the `chmod` mode, the debugger menu labels across 7.x and 8.x, and
|
||||
whether an on-disk `.so` outranks the in-APK copy on a hardened build. What would make the route
|
||||
|
||||
@@ -40,6 +40,27 @@ embeds the metadata in the `.so` with the rest of the header intact.
|
||||
**A 0-byte metadata file is a positive finding, not a missing file.** It means the build does not
|
||||
use the file at all, so a static decryptor has nothing to decrypt. Look for the loader's output.
|
||||
|
||||
### 1.1 A rebuilt `libil2cpp` announces itself in the ELF tables
|
||||
|
||||
One build's own triage script flags three things before it parses anything: a `JNI_OnLoad` export, a
|
||||
`DT_INIT`/`init_array` of the library's own, and an `SHT_LOUSER` section. None of the three is
|
||||
something the engine's build path needs, so any of them is a candidate added step — read them as
|
||||
heuristics, where a false positive costs one `readelf`:
|
||||
|
||||
```bash
|
||||
readelf -sW libil2cpp.so | grep -i jnionload
|
||||
readelf -dW libil2cpp.so | grep -Ei 'INIT|FINI'
|
||||
readelf -SW libil2cpp.so | grep -w LOUSER # a section type, not a name
|
||||
```
|
||||
|
||||
`JNI_OnLoad` means the library wants a JVM-side entry point, which is a JNI-sinking or loader shape
|
||||
rather than something the engine itself calls through. An own `DT_INIT`/`init_array` is the earliest
|
||||
place arbitrary code can run. `SHT_LOUSER` is a section the
|
||||
linker reserves for the application — a convenient place to park a payload, and a common marker in
|
||||
this family. Read these tables **before** hunting ciphertext: the injected entry is where metadata and
|
||||
code are decrypted, and it is also what a dumper's "this file may be protected" refusal is reacting to.
|
||||
Then continue at §2 and anchor on the consumer rather than the file.
|
||||
|
||||
## 2. Anchor on the consumer, not the path
|
||||
|
||||
The reliable address is the pointer *after* decryption and de-obfuscation. Two anchors exist, and
|
||||
@@ -77,7 +98,10 @@ i -= 8
|
||||
```
|
||||
|
||||
This works when magic and version are zeroed, and it survives a version mismatch because it uses
|
||||
only offset/count pairs that are still laid out in the same order. Bound the loop (for example
|
||||
only offset/count pairs that are still laid out in the same order. (`observed`, host: on a synthetic
|
||||
header whose magic and version are zeroed and whose tail counts are zero, the walk returns the true
|
||||
length `0x1140` while a fixed mid-header pair returns `0x1098`; see
|
||||
`docs/tool-verification/fixtures/verify_ref_claims_10c.py`.) Bound the loop (for example
|
||||
`i > 0`), wrap reads in the platform's fault handler on Windows, and treat a non-monotone result as
|
||||
"this is not the header you think it is", not as a size.
|
||||
|
||||
@@ -131,6 +155,16 @@ self-describing container**. Dump the program table, name each case by its effec
|
||||
program in Python. Do not single-step the VM instruction by instruction in a debugger; the table
|
||||
is the specification.
|
||||
|
||||
**Read the call site before replaying, because the schedule decides what a unit test is.** One build
|
||||
runs the interpreter once per fixed-size block — `apply(program, 0x40, table, out + i, min(0x40,
|
||||
remaining))` in a loop — with the same constant table re-passed on every call and no state carried
|
||||
between them. That makes one block a complete test: decrypt block 0, check the result against the
|
||||
plaintext you already know it must contain (a magic, or the first string), and only then run the rest.
|
||||
If the call site instead updates a context between iterations, the blocks are chained: no per-block
|
||||
shortcut exists, replay must start at 0, and a mismatch at block 7 is a key/program error you cannot
|
||||
localise by looking at block 7. The truncated tail is also worth reading explicitly, because a tail
|
||||
handled as a full block shifts everything after the last boundary.
|
||||
|
||||
### 3.3 XXTEA with a length word
|
||||
|
||||
A resource-container sample used XXTEA: `delta = 0x9E3779B9`, `rounds = 6 + 52/n`, decrypt from
|
||||
@@ -237,6 +271,16 @@ re-imported metadata is silently corrupt and the dumper still accepts it.
|
||||
Dump `index,hex` to CSV, decrypt offline, patch the metadata file, then re-run Il2CppDumper. This
|
||||
keeps the proof in one place: the tool either parses the patched file or it does not.
|
||||
|
||||
**When the literal cipher lives inside the protected VM, do not reverse it — drive the runtime.**
|
||||
`String::NewLen(ptr, len)` *is* the decryption oracle, and it takes exactly the two fields the literal
|
||||
table stores. Enumerate `i` from 0 to `count-1` yourself, read `(dataOffset + dataIndex, length)`, call
|
||||
the runtime entry for each, and collect the returned UTF-16 buffers. Three properties make this better
|
||||
than either hooking natural calls or reimplementing the cipher: it is exhaustive rather than a sample
|
||||
of what the app happened to load, it is a completeness proof for the table (every index either
|
||||
decrypts or visibly fails), and it keeps you out of the VM. The cost is that the process must be
|
||||
alive and past metadata init, so it is the wrong route when the question is what a dead sample
|
||||
contains.
|
||||
|
||||
## 7. Adjacent artifacts that fail the same way
|
||||
|
||||
These are not metadata, but they are the next files the same build protects, and the diagnostic
|
||||
|
||||
@@ -5,7 +5,10 @@ xvmp-class IR virtualizer, or any private-bytecode loop — and the question is
|
||||
the handler soup to the guest program. This file owns the recovery procedure; the **capture**
|
||||
half lives in `references/ete-diff-tracing.md`, the **slicing** tool in
|
||||
`scripts/vm_handler_slice.py`, and the OLLVM-only case in
|
||||
`references/native-dbi-and-deobfuscation.md`.
|
||||
`references/native-dbi-and-deobfuscation.md`. If the interpreter is itself written in a managed
|
||||
language — a C# class holding an opcode array plus a cursor under IL2CPP, or a JS dispatch loop — this
|
||||
is the wrong file: `unity-runtime.md` §when the managed layer is itself the interpreter, and
|
||||
`jsvmp-recovery.md`, own those shapes.
|
||||
|
||||
**Claim strength.** Every method step here is absorbed from two public reverse-engineering
|
||||
write-ups (Kanxue, 2026-08/09, titles and authors anonymized in this record; one is a full
|
||||
|
||||
@@ -12,7 +12,9 @@ all.
|
||||
sample was unpacked in this record. The general rules (static-first, self-describing TLV,
|
||||
mirror-the-host-codegen) are `inferred` from that source. Version numbers matter: every branch
|
||||
below is gated on the `pycdc`/Python version, and applying a newer layout to an older file fails
|
||||
silently rather than loudly.
|
||||
silently rather than loudly. The three-scope nonce layering (§2.2) and the counter-mode correction
|
||||
in §5 were added in the 2026-10 audit pass; the counter behaviour is `observed` on the host, the
|
||||
layering itself is `unverified`.
|
||||
|
||||
## 1. Recognition
|
||||
|
||||
@@ -65,9 +67,39 @@ The `idiv` side effect is the general lesson: when reimplementing a packer's VM,
|
||||
code generation*, including register clobbers the author did not intend. A clean abstract
|
||||
interpreter will produce a different key and every downstream field will decrypt to noise.
|
||||
|
||||
Width is part of the same contract: the VM is a transcription of machine instructions, so each
|
||||
operation must be evaluated at the host register width — a multiply keeps only the low 32 bits,
|
||||
division is signed and truncates toward zero, and results wrap. Evaluating the same program with
|
||||
Python's unbounded integers diverges after a few instructions and produces a key that is wrong in a
|
||||
way that looks like a wrong nonce downstream, not like an arithmetic error.
|
||||
|
||||
### 2.2 The shipped blob is wrapped once more
|
||||
|
||||
The `.pyc` is the *inner* artefact. At least one build ships
|
||||
`AES-CTR(master key, AST_nonce, pyc-with-encrypted-parts)`: the file on disk is not a pyc, its first
|
||||
four bytes are not the marshal magic, and a parser pointed at it reports a corrupt header rather than
|
||||
an encrypted file. Three scopes share one master key and are separated only by which nonce is used:
|
||||
|
||||
| Scope | Nonce | Plaintext it yields |
|
||||
|---|---|---|
|
||||
| outer container, whole file | `AST_nonce` from `RSA_KEY2`, `nonce[15] = 2` | the pyc: header, string blob, code section |
|
||||
| each code object | 12 bytes sliced out of that object's own `m_code` (§5) | that object's bytecode |
|
||||
| each string | `pyarmor_mix_str_aes_nonce` | that one literal |
|
||||
|
||||
The order is not a preference. Applying a code object's nonce to the on-disk bytes decrypts layer 1's
|
||||
ciphertext with layer 2's stream and yields high-entropy bytes that look like a private opcode set, so
|
||||
the failure is "this packer uses a custom ISA", which is the wrong conclusion. Peel the outer layer
|
||||
first, then check for the pyc header, then parse (§3).
|
||||
|
||||
A live dump does not replace this pass either: the packer decrypts each code object when it is first
|
||||
imported, so a process dump holds the objects that ran and leaves everything else as ciphertext. A
|
||||
function missing from a dump means "not reached yet", not "not present" — force the modules to import
|
||||
before calling a dump complete, and keep the static pass as the only complete route.
|
||||
|
||||
## 3. Parsing the code object
|
||||
|
||||
The protected file is a normal `.pyc` with extra fields, so the parser must be version-aware.
|
||||
**Once the outer layer is off (§2.2)**, the file is a normal `.pyc` with extra fields, so the parser
|
||||
must be version-aware.
|
||||
Fields that change and the versions that introduced them:
|
||||
|
||||
| Field | Width / condition |
|
||||
@@ -147,9 +179,16 @@ Then:
|
||||
6. Replace the descriptor's first 8 bytes with the `"<COAddr>"` placeholder so a later re-run does
|
||||
not treat runtime data as a key.
|
||||
|
||||
The nonce construction is the part that is easy to get wrong: `nonce[15] = 2` is a domain
|
||||
separator, and using an all-zero tail produces a stream that decrypts the first block plausibly and
|
||||
then diverges.
|
||||
The nonce construction is the part that is easy to get wrong, and the usual description of *how* it
|
||||
fails is wrong: `nonce[15] = 2` is not a flag that only affects later blocks. In counter mode every
|
||||
block's keystream is `E(nonce || counter)`, so changing byte 15 changes the counter start and corrupts
|
||||
**block 0 onward** — there is no "first block decrypts plausibly" state to catch you. (`observed`,
|
||||
host: two 16-byte counter blocks differing only in byte 15 differ in 16 of 16 keystream bytes, so
|
||||
the ciphertext differs from block 0 — see the EXTENSION record and
|
||||
`docs/tool-verification/fixtures/verify_ref_claims_10c.py`.) What an *unset* tail does is worse in a different way: the leftover bytes
|
||||
are whatever the buffer held, so the keystream is not reproducible at all and the same input decrypts
|
||||
differently on two runs. Zero the buffer, set the byte, and check the first eight bytes of the result
|
||||
against a known code-object prologue instead of trusting a plausible-looking body.
|
||||
|
||||
## 6. Decrypting strings
|
||||
|
||||
|
||||
@@ -140,6 +140,52 @@ the same rule as `pitfalls.md` P47.
|
||||
actually loads — a split or stubbed `libil2cpp.so` inside the APK is not necessarily the one mapped
|
||||
(`native-and-so.md`).
|
||||
|
||||
### When the managed layer is itself the interpreter
|
||||
|
||||
IL2CPP is ahead-of-time compiled, which makes it tempting to read `libil2cpp.so` as the only code that
|
||||
matters. A `.so` that looks like a thin wrapper over `Dictionary` and delegate calls is often sitting
|
||||
on top of an interpreter written in C# — and a VM written in C# is recoverable from `dump.cs` without
|
||||
ever reading an opcode handler in assembly. The shape in `dump.cs`:
|
||||
|
||||
```text
|
||||
class <obf> {
|
||||
int[] aa1; // the guest program, built from a literal
|
||||
int aa5; // the cursor, i.e. the PC / vEIP
|
||||
Dictionary<int, Action> aa8; // opcode -> handler
|
||||
}
|
||||
```
|
||||
|
||||
A `while` loop that reads `array[cursor]` and calls `dict[op]` is the dispatcher; the array initialised
|
||||
from a literal and the dictionary filled in the constructor are the program and the handler table. The
|
||||
handler *indices* are assigned in the constructor by walking an enum or a `static_fields` block (one
|
||||
sample enumerated 1..22 and used those values as dictionary keys), so the opcode→handler map is a
|
||||
property of the constructor, not something to discover by tracing. Read the constructor first, rename
|
||||
the fields positionally to `a1..aN`, delete the null and length guards, and the dispatch loop becomes
|
||||
plain control flow. If the program array is built at run time, hook the constructor and take its
|
||||
argument instead of tracing the loop.
|
||||
|
||||
The recovery method does not change once you have the instruction stream — `vmp-differential-analysis.md`
|
||||
for the opcode map and `native-vm-recovery.md` §11 for the artefact set. What changes is the cost: a
|
||||
managed VM is read statically, in a decompiler that still has names.
|
||||
|
||||
### Array literals are not in `.rodata`
|
||||
|
||||
A static array literal — a crypto key, an S-box, a constant table — has **no inline initialiser in
|
||||
`libil2cpp.so`**. IL2CPP stores the field's default value in the metadata and materialises it at class
|
||||
init by calling `RuntimeHelpers::InitializeArray` (the managed-side name; the implementation copies
|
||||
from the field-default blob). Consequences for both reading and searching:
|
||||
|
||||
- `grep`-ing the `.so`, or `strings`-`ing` it, cannot find the table, and its absence is not evidence
|
||||
that the table is derived at run time.
|
||||
- Hook `InitializeArray` (or break on it) and the argument is the table; the same bytes are also
|
||||
readable from the metadata field-default blob without a device.
|
||||
- The table is per-field, so one hook gives you every literal the class initialisation touches, which
|
||||
is usually the whole key schedule of a hand-rolled cipher.
|
||||
|
||||
This is the cheapest way to answer "where does this constant come from" in a Unity target, and it is
|
||||
cheaper than reconstructing the cipher. `unverified`: the API name is stable across Unity versions, the
|
||||
argument shape is not.
|
||||
|
||||
## 5. Driving IL2CPP at run time: frida-il2cpp-bridge
|
||||
|
||||
When the dump is encrypted or the question is behavioural, the bridge resolves classes by name at run
|
||||
@@ -183,6 +229,23 @@ directory; with no arguments it uses defaults that may not be writable on a hard
|
||||
written before the metadata is decrypted is the encrypted file again — check the magic before
|
||||
importing it.
|
||||
|
||||
**Hook the framework primitive, not the app's wrapper.** Obfuscation renames the app's classes; it
|
||||
does not rename `System.Security.Cryptography.*`, `javax.crypto.Cipher` or `java.security.*`. So when
|
||||
the question is "what key does this use", trace the primitive instead of the wrapper the app built:
|
||||
|
||||
```ts
|
||||
Il2Cpp.trace(true).classes(Il2Cpp.domain.assembly("System.Security.Cryptography")
|
||||
.image.class("System.Security.Cryptography.AesCryptoServiceProvider")).and().attach();
|
||||
```
|
||||
|
||||
One such trace reports every key/IV pair the process constructs, no matter which wrapper or
|
||||
obfuscated call chain produced it, and it removes the need to reconstruct the derivation. On the Java
|
||||
side the equivalent pair is a hook on `Cipher.getInstance` plus `SecretKeySpec` / `IvParameterSpec`.
|
||||
The trade is provenance: the primitive tells you the value, not which call site asked for it — use the
|
||||
app's own class (above) when you need the call site, and the primitive when the derivation is hostile
|
||||
and you only need the material. `unverified` here: neither trace was run in this record.
|
||||
|
||||
|
||||
The dependency is the boundary: a build that hides or strips the il2cpp exports breaks the bridge, and
|
||||
the failure presents as *class not found* — indistinguishable from a wrong class name. Confirm the
|
||||
exports exist before believing a negative lookup.
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
"""The 2026-10-27 host fixture must be able to fail.
|
||||
|
||||
`docs/tool-verification/fixtures/verify_ref_claims_10c.py` is the only thing standing behind three
|
||||
`observed` labels, so its checks are as load-bearing as a gate. The first draft failed the rule this
|
||||
repository applies to gates: the comparison check counted iterations inside the Python loop it was
|
||||
measuring, so `n == k + 1` restated its own source and could never fail for a real reason. It now
|
||||
compiles the two comparators and measures the compiler's output.
|
||||
|
||||
Each test here injects the wrong claim and requires the check to reject it, plus one test that the
|
||||
process exit code follows the verdict. `observed` on a host with neither `openssl` nor a C compiler
|
||||
is a skip, not a pass.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import shutil
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import REPO_ROOT
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
FIXTURE = REPO_ROOT / "docs" / "tool-verification" / "fixtures" / "verify_ref_claims_10c.py"
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def claims():
|
||||
spec = importlib.util.spec_from_file_location("verify_ref_claims_10c", str(FIXTURE))
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(module)
|
||||
return module
|
||||
|
||||
|
||||
def _need_openssl():
|
||||
if not shutil.which("openssl"):
|
||||
pytest.skip("no openssl on this host: the CTR check degrades to SKIP")
|
||||
|
||||
|
||||
def _need_cc():
|
||||
if not (shutil.which("gcc") or shutil.which("cc") or shutil.which("clang")):
|
||||
pytest.skip("no C compiler on this host: the comparison check degrades to SKIP")
|
||||
|
||||
|
||||
def test_checks_pass_on_the_claim(claims):
|
||||
_need_openssl()
|
||||
_need_cc()
|
||||
assert claims.ctr_nonce_tail() is True
|
||||
assert claims.comparison_oracle() is True
|
||||
assert claims.header_walk() is True
|
||||
|
||||
|
||||
def test_ctr_check_rejects_an_unchanged_nonce(claims):
|
||||
_need_openssl()
|
||||
base = bytes(range(12)) + b"\x00\x00\x00"
|
||||
same = base + b"\x02"
|
||||
assert claims.ctr_nonce_tail(nonce_a=same, nonce_b=same) is False
|
||||
|
||||
|
||||
def test_comparison_check_rejects_a_full_pass_mislabelled_early(claims):
|
||||
_need_cc()
|
||||
assert claims.comparison_oracle(early_source=claims.C_FULL) is False
|
||||
|
||||
|
||||
def test_comparison_check_rejects_an_early_exit_mislabelled_constant(claims):
|
||||
_need_cc()
|
||||
assert claims.comparison_oracle(full_source=claims.C_EARLY) is False
|
||||
|
||||
|
||||
def test_header_check_rejects_an_off_grid_start(claims):
|
||||
assert claims.header_walk(start=0x110) is False
|
||||
|
||||
|
||||
def test_header_check_rejects_ground_truth_that_disagrees(claims):
|
||||
# the tail-most pair no longer sits at the largest offset, so "declared" and the forward scan
|
||||
# cannot both be the file length
|
||||
pairs = [(0x1100, 0x08), (0x1300, 0x10), (0x10A0, 0x10)]
|
||||
assert claims.header_walk(pairs=pairs) is False
|
||||
|
||||
|
||||
def test_main_exit_code_follows_the_verdict(claims, monkeypatch):
|
||||
# a check that returns False without printing must still fail the run
|
||||
monkeypatch.setattr(claims, "ctr_nonce_tail", lambda: False)
|
||||
monkeypatch.setattr(claims, "comparison_oracle", lambda: True)
|
||||
monkeypatch.setattr(claims, "header_walk", lambda: True)
|
||||
assert claims.main() == 1
|
||||
|
||||
|
||||
def test_main_treats_a_skip_as_not_a_failure(claims, monkeypatch):
|
||||
monkeypatch.setattr(claims, "ctr_nonce_tail", lambda: None)
|
||||
monkeypatch.setattr(claims, "comparison_oracle", lambda: True)
|
||||
monkeypatch.setattr(claims, "header_walk", lambda: True)
|
||||
assert claims.main() == 0
|
||||
Reference in New Issue
Block a user