antior/apk-reverse
1
0
Fork 0
mirror of https://codeberg.org/reverse-skill-lab/apk-reverse synced 2026-11-15 20:44:22 +08:00
Code Issues Packages Projects Releases Wiki Activity

Audit the ref drop folder a third time, and measure what it can measure

Browse Source 
ref/ still held the sixteen write-ups absorbed on 2026-10-24, with unchanged mtimes, so this pass
re-read them instead of re-absorbing. Nine generalisable mechanisms were still missing: Pyarmor's
outer container layer and its three nonce scopes (plus the live dump being structurally incomplete),
fixed-width arithmetic in the 12-byte key VM, Python compiled to native extensions, the rebuilt
libil2cpp ELF fingerprint (JNI_OnLoad / own DT_INIT / SHT_LOUSER), a container VM's block schedule,
driving String::NewLen over every literal index as an oracle, VMs that live in the managed layer,
array literals materialised by InitializeArray instead of .rodata, framework crypto primitives as
the hook target, a comparison gate as a prefix oracle, and the debugger's own stop policy
(SIGPWR/SIGXCPU suspend-to-crash, one database per activity).

One absorbed claim was wrong and is corrected: python-bytecode-recovery section 5 said an unset
nonce[15] decrypts the first block plausibly and then diverges. Counter mode has no such state --
the same nonce differing in that one byte changes the keystream from block 0, measured at 16/16
bytes.

verify_ref_claims_10c.py carries the three device-free checks. Its first draft was decorative -- the
comparison check counted work inside the loop it was measuring -- so the comparators are now
compiled at -O2, every check has a negative control that must come out empty, and the verdict
travels as a return value rather than through a global list. tests/unit/test_verify_ref_claims_10c.py
injects the wrong claim into each check and requires the failure.

Registration: two capability rows (il2cpp-managed-layer-bytecode-vm unverified,
comparison-cost-oracle observed for the host half), two evidence-summary rows, one new and three
extended route-inventory rows, three coverage bullets. SKILL.md and routing.md unchanged; the record
is EXTENSION-ref-absorption-2026-10c.md. Gates and pytest green (796 passed).
This commit is contained in:
root
2026-10-27 00:33:49 +08:00
parent 068280a180
commit 90d9bd3e57
14 changed files with 745 additions and 15 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/tool-verification/EXTENSION-ref-absorption-2026-10c.md
+151
View File
@@ -0,0 +1,151 @@
# EXTENSION — third audit of the `ref/*` drop folder (2026-10-27)
`ref/` still held the sixteen write-ups absorbed on 2026-10-24 (`EXTENSION-ref-absorption-2026-10.md`).
Their mtimes were unchanged, and nothing in the folder matched the 2026-10-25 set that was absorbed in
place. So this pass was **not** a re-absorption: it re-read all sixteen, compared every generalisable
mechanism against what the skill already says, and ran the host-checkable part of what it found. Nine
mechanisms were still absent, one absorbed sentence turned out to be wrong, and one procedure is now
`observed` instead of `unverified`.
## Sources
The same sixteen files, in the same four groups as the 2026-10 record (Unity/IL2CPP nine, linker three,
x86 VM protector three, Pyarmor one). Titles, authors, product names and package identifiers are omitted
here for the same reason as before: they carry identity and nothing in the tracked surface needs them.
All are public secondary sources, `unverified` in this repository unless the verification pass below
says otherwise.
## What was added
No new reference files — every mechanism went into the file that already owns its topic.
| Mechanism | File | Why the existing text did not cover it |
|---|---|---|
| The shipped Pyarmor blob is wrapped once more: outer container → per-code-object → per-string, three nonce scopes on one master key, and a live dump is therefore structurally incomplete | `python-bytecode-recovery.md` §2.2 | §3 opened with "the protected file is a normal `.pyc` with extra fields", which is the second layer, not the file you are handed |
| Mirror the host register width, not just the register clobbers, when reimplementing the 12-byte key VM | `python-bytecode-recovery.md` §2.1 | §2.1 named the `idiv` clobber; 32-bit wrapping and truncating division were unstated, and they fail the same way |
| Python compiled away (Nuitka / Cython / `pyd`): no `.pyc` is not "no Python" | `framework-runtimes.md` identification table | the Python row implied the runtime always arrives as bytecode |
| A rebuilt `libil2cpp` announces itself in the ELF tables (`JNI_OnLoad` export, own `DT_INIT`/`init_array`, `SHT_LOUSER` section) | `il2cpp-protected-metadata.md` §1.1 | the triage table sent "registration search returns 0" straight to the live-image dump; the injected entry point is cheaper to find and is what the dumper's refusal is reacting to |
| A container VM's block schedule decides what a unit test is: fixed-size blocks with a truncated tail and no cross-block state vs a chained context | `il2cpp-protected-metadata.md` §3.2 | the section specified the container layout and stopped at "replay the program"; the call site is what says whether block 0 is a valid test |
| When the literal cipher lives in the VM, drive `String::NewLen` over every index instead of reversing the cipher — and use it as a completeness proof | `il2cpp-protected-metadata.md` §6 | the hook was documented as observation only; enumerating the index space is a different, stronger route |
| A VM can live in the managed layer: opcode array + cursor + `Dictionary<int, delegate>`, handler indices enrolled in the constructor | `unity-runtime.md` §when the managed layer is itself the interpreter | every VM file assumed a native interpreter; `native-vm-recovery.md` now says so explicitly and cross-links |
| Static array literals are field defaults materialised by `RuntimeHelpers::InitializeArray`, not bytes in `.rodata` | `unity-runtime.md` same section | the absence of a table in `.rodata` was being read as "derived at run time" |
| Hook the framework crypto primitive (BCL AES provider, `javax.crypto`), not the app's obfuscated wrapper | `unity-runtime.md` §5 | §5 polled the app's own class; the stability of framework class names under obfuscation is the point |
| A comparison gate is a prefix oracle: instrument the comparator's counter, and read the fail-path cost to decide whether the oracle is open | `account-gates.md` Step 2b | the file classified gates by authority and never considered that the comparison itself leaks the expected value |
| Debugger-side stop policy: `SIGPWR`/`SIGXCPU` suspend-to-crash, one database per activity, input injection into a paused process | `ida-so-debugging.md` §8 | the debuggable-side failure modes were documented; the failures caused by the debugger's own defaults were not |
### A claim that was corrected, not added
`python-bytecode-recovery.md` §5 said an unset `nonce[15]` "produces a stream that decrypts the first
block plausibly and then diverges". Counter mode does not have that state: the keystream for block *i*
is `E(nonce || counter_i)`, so a different nonce byte changes the output from block 0. Measured below.
The paragraph now says what happens (every block wrong; an unset tail makes the result
non-reproducible) and tells the reader to check a known prologue rather than "a plausible body".
## Registration surface updated
- `evidence/capability-matrix.json`: two rows, `il2cpp-managed-layer-bytecode-vm` (`partial`,
`route_measured: false`, `unverified`) and `comparison-cost-oracle` (`partial`,
`route_measured: false`, `observed` — the host half only).
- `references/evidence-summary.md`: the same two rows, with the same labels.
- `references/coverage/route-inventory.md`: one new row (the comparison oracle) and three extended rows
(Unity, protected IL2CPP, Python).
- `references/coverage-and-limits.md`: three bullets under "Covered by documented routes", each naming
which half is measured.
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
symptom mirror and the inventories had nothing to learn. That is also why the token budget did not
move.
## Verification pass — host, 2026-10-27
```bash
python3 docs/tool-verification/fixtures/verify_ref_claims_10c.py
```
```text
PASS CTR: nonce tail changes block 0 (python-bytecode-recovery section 5)
keystream block 0 differs in 16/16 bytes
ciphertext block 0 differs in 16/16 bytes, blocks 1-2 in 32/32
negative control, nonce against itself: 0/48 bytes differ
so there is no "first block decrypts, rest diverges" state -- every block is wrong
PASS comparison cost is a prefix oracle (account-gates Step 2b)
prefix 0/15 -> early-exit 1, full pass 15
prefix 1/15 -> early-exit 2, full pass 15
...
prefix 15/15 -> early-exit 15, full pass 15
full pass over the same candidates: [15]
negative control, identical inputs through the early-exit build: 15 (must be 15)
early exit: work grows one unit per correct prefix byte; full pass: constant
PASS metadata header size by backward walk (il2cpp-protected-metadata section 2.1)
declared 0x1140, forward scan 0x1140, walk from 0x10c -> 0x1140
negative control, off-grid start 0x110 -> 0x1120 (must differ)
magic/version zeroed in this fixture: the walk uses only offset/count pairs
3 check(s), 0 skipped, 0 failed
RESULT=ok
```
Three labels changed as a result: the counter-mode statement in §5 is now `observed` (and corrected),
the comparison-cost separation in `account-gates.md` is `observed` for the synthetic case only, and
the backward header walk in §2.1 is `observed` on a synthetic header. **No claim about a target was
upgraded** — no device, sample or server was involved.
### The fixture was audited before it was trusted
The first draft passed all three checks and was still wrong in one of them: the comparison check
incremented its own counter inside the loop it was measuring, so `work == k + 1` restated the Python
source and could not fail for any real reason — the repository's "a limit that cannot fail is
decorative" defect, in a check that was about to carry an `observed` label. Three changes:
1. **Measured on compiled code.** The two comparators are now C, built with `-O2` and called through
`ctypes`; the question is what a compiler emits for `break` on mismatch, which an interpreted loop
cannot answer about itself. Without a compiler the check reports SKIP rather than PASS.
2. **Every check carries a negative control** that must come out empty: the same nonce twice (0/48
bytes differ), identical inputs through the early-exit build (15, i.e. a full pass), and an
off-grid walk start (`0x110` yields `0x1120` instead of `0x1140`).
3. **The verdict travels as a return value, not through a global list.** `main()` built its failure
count from the list `report()` appended to, so a check that returned False without printing would
have been invisible; it now counts the return values and reports skips separately.
`tests/unit/test_verify_ref_claims_10c.py` injects the wrong claim into each check and requires the
failure — full-pass mislabelled as early-exit, early-exit mislabelled as constant, an unchanged nonce,
an off-grid walk, a pair table whose declared and scanned lengths disagree — plus the skip semantics
(a host with no `openssl`/compiler exits 0 with SKIP) and the exit-code plumbing. The AES probe needs
`openssl`; the comparison probe needs a C compiler; the header check needs nothing.
## Deliberately not absorbed
- **Per-build constants and identities**: offsets, keys, package ids, function counts, the articles' own
scripts. Same rule as the two earlier passes.
- **Asset and media pipelines** (Live2D `moc3`, video reconstruction, Blender conventions) and the
CDN-manifest delta. One candidate survived this audit — "an index pulled from a device is a cache
subset, not the catalogue" — and was dropped deliberately: it is a provenance rule for a download
pipeline, and `server-config-and-updates.md` is not an asset-extraction file.
- **In-memory patch hygiene** (write the operand, restore `r-x`) from one article: the distinct part is
one sentence, and `native-and-so.md` already carries the "which copy is mapped" rule that decides
whether a patch matters at all. Recording it would have added a fourth file to a single idea.
- **Performance and effort claims**, and the articles' commentary about tooling quality.
## What would change a label
One experiment per row, none of which this record could run:
- **Managed-layer VM**: recover one opcode table from a `dump.cs` interpreter and show an instruction
stream that reproduces a known in-game value. That moves the row to `observed`; today only the shape
is documented.
- **Array literals**: hook `InitializeArray` on a live build and read back a key that the same build's
`.rodata` does not contain.
- **Comparison oracle**: run the counter against one real serial gate and show the work grows with the
correct prefix. On a device, use the comparator's counter, not the clock — a timing-only attempt is
expected to fail and would not move the row.
- **Pyarmor outer layer**: decrypt one shipped blob, show the pyc magic appearing only after the outer
pass, then decrypt one code object and run the recovered module.
- **Rebuilt `libil2cpp`**: `readelf` a stock library and the protected one side by side and show the
three markers absent/present.
## Gate status
`check_repo.py`, `check_refs.py`, `check_routing.py`, `check_commands.py`, `check_budget.py` and
`pytest -q` were run after the change and are green (796 passed, 209 skipped, 25 xfailed; eight of
those tests are the fixture's own falsifiability cases). A green run
proves the inventory resolves and the mirrors agree; it says nothing about whether any mechanism works
on a target, which is what the labels above are for.
docs/tool-verification/fixtures/verify_ref_claims_10c.py
+227
View File
@@ -0,0 +1,227 @@
#!/usr/bin/env python3
"""Re-run the claims added by the 2026-10-27 ref/* audit pass that need no device.
Three claims, all of which the reference files now call 'observed (host)':
1. python-bytecode-recovery.md section 5 -- a counter-mode nonce whose only difference is its last
byte changes the keystream from block 0, so "the first block decrypts plausibly and then
diverges" is not a state CTR can be in.
2. account-gates.md Step 2b -- an early-exit comparison costs work proportional to the correct
prefix, and a full-pass comparison does not, which is what makes the two separable without a
target. Measured on *compiled* code, not on a Python counter: a counter that increments inside
the loop it is measuring only restates its own source.
3. il2cpp-protected-metadata.md section 2.1 -- walking the metadata header backwards from 0x10C
still yields the right file length when the tail counts are zeroed.
Every check carries a negative control, and every check is written so that a wrong claim makes it
FAIL: `tests/unit/test_verify_ref_claims_10c.py` injects the wrong claim into each one and requires
the failure. A check that cannot fail is decorative.
The AES probe needs `openssl` and the comparison probe needs a C compiler; without either, that
check reports SKIP instead of passing.
Usage:
python3 docs/tool-verification/fixtures/verify_ref_claims_10c.py
"""
import ctypes
import os
import shutil
import struct
import subprocess
import sys
import tempfile
KEY = '000102030405060708090a0b0c0d0e0f'
def report(name, ok, detail):
"""Print one verdict and pass it through. `ok is None` means the host cannot run the check.
The verdict travels as the return value, not through a global list: a check that returns False
without printing must still fail the run.
"""
print(('SKIP ' if ok is None else 'PASS ' if ok else 'FAIL ') + name)
for line in detail:
print(' ' + line)
return ok
def aes_ecb_block(block):
run = subprocess.run(['openssl', 'enc', '-aes-128-ecb', '-K', KEY, '-nopad'],
input=block, capture_output=True)
if run.returncode:
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
return run.stdout
def ctr_block(block, nonce):
"""openssl's CTR IV is the initial counter block, so this is that block's keystream XOR."""
run = subprocess.run(['openssl', 'enc', '-aes-128-ctr', '-K', KEY, '-iv', nonce.hex(), '-nopad'],
input=block, capture_output=True)
if run.returncode:
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
return run.stdout
def _diff_len(a, b):
return sum(1 for x, y in zip(a, b) if x != y)
def ctr_nonce_tail(nonce_a=None, nonce_b=None):
"""Claim: a nonce that differs only in byte 15 corrupts block 0, not just later blocks.
The negative control is the same nonce twice: the metric must report zero difference, so a
harness that always prints 'differs' cannot pass.
"""
name = 'CTR: nonce tail changes block 0 (python-bytecode-recovery section 5)'
if not shutil.which('openssl'):
return report(name, None, ['openssl not found: cannot run this check on this host'])
base = bytes(range(12)) + b'\x00\x00\x00'
n_zero = nonce_a if nonce_a is not None else base + b'\x00'
n_two = nonce_b if nonce_b is not None else base + b'\x02'
plain = b'A' * 48
ks_zero, ks_two = aes_ecb_block(n_zero), aes_ecb_block(n_two)
block0_ks = _diff_len(ks_zero, ks_two)
c_zero, c_two = ctr_block(plain, n_zero), ctr_block(plain, n_two)
first = _diff_len(c_zero[:16], c_two[:16])
later = _diff_len(c_zero[16:48], c_two[16:48])
control = _diff_len(c_zero, ctr_block(plain, n_zero))
ok = block0_ks == 16 and first == 16 and later == 32 and control == 0
return report(name, ok,
['keystream block 0 differs in %d/16 bytes' % block0_ks,
'ciphertext block 0 differs in %d/16 bytes, blocks 1-2 in %d/32' % (first, later),
'negative control, nonce against itself: %d/48 bytes differ' % control,
'so there is no "first block decrypts, rest diverges" state -- every block is wrong'])
C_EARLY = """
int cmp_work(const unsigned char *a, const unsigned char *b, int n) {
int work = 0;
for (int i = 0; i < n; i++) {
work++;
if (a[i] != b[i]) break;
}
return work;
}
"""
C_FULL = """
unsigned sink;
int cmp_work(const unsigned char *a, const unsigned char *b, int n) {
int work = 0;
unsigned diff = 0;
for (int i = 0; i < n; i++) {
work++;
diff |= (unsigned)(a[i] ^ b[i]);
}
sink ^= diff;
return work;
}
"""
def _build(lib_source, tmpdir, tag):
src = os.path.join(tmpdir, tag + '.c')
so = os.path.join(tmpdir, 'lib' + tag + '.so')
with open(src, 'w') as fh:
fh.write(lib_source)
cc = shutil.which('gcc') or shutil.which('cc') or shutil.which('clang')
run = subprocess.run([cc, '-O2', '-shared', '-fPIC', '-o', so, src],
capture_output=True)
if run.returncode:
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
lib = ctypes.CDLL(so)
lib.cmp_work.restype = ctypes.c_int
lib.cmp_work.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int]
return lib.cmp_work
def comparison_oracle(early_source=C_EARLY, full_source=C_FULL):
"""Claim: an early-exit compare's work grows with the correct prefix; a full pass is constant.
Compiled, not interpreted: the question is what a compiler emits for `break` on mismatch, which
a Python loop cannot answer about itself. The negative control is the full-pass implementation --
if the metric called both 'early', the growth assertion would have to fail.
"""
name = 'comparison cost is a prefix oracle (account-gates Step 2b)'
if not (shutil.which('gcc') or shutil.which('cc') or shutil.which('clang')):
return report(name, None, ['no C compiler found: cannot measure compiled output'])
secret = b'ACTIVATION-CODE'
n = len(secret)
rows = []
with tempfile.TemporaryDirectory() as tmp:
early = _build(early_source, tmp, 'early')
full = _build(full_source, tmp, 'full')
grew = True
for k in range(0, n + 1):
cand = secret[:k] + b'\x00' * (n - k)
e = early(cand, secret, n)
rows.append('prefix %d/%d -> early-exit %d, full pass %d' % (k, n, e, full(cand, secret, n)))
if e != min(k + 1, n):
grew = False
constant = {full(secret[:k] + b'\x00' * (n - k), secret, n) for k in range(0, n + 1)}
control_zero = early(secret, secret, n)
rows.append('full pass over the same candidates: %s' % sorted(constant))
rows.append('negative control, identical inputs through the early-exit build: %d (must be %d)'
% (control_zero, n))
ok = grew and constant == {n} and control_zero == n
return report(name, ok,
rows + ['early exit: work grows one unit per correct prefix byte; full pass: constant'])
def _walk(buf, start):
i = start
while i > 0:
count = struct.unpack_from('<i', buf, i)[0]
if count:
return struct.unpack_from('<i', buf, i - 4)[0] + count
i -= 8
return None
def header_walk(pairs=None, start=0x10C):
"""Claim: the backward walk from 0x10C sizes a header whose magic and tail are zeroed.
Ground truth is computed twice: from the pair table that built the fixture, and by an
independent forward scan (max of offset+count). The negative control is a walk on an off-grid
start -- it must disagree, otherwise "steps of 8" is not what the check is testing.
"""
name = 'metadata header size by backward walk (il2cpp-protected-metadata section 2.1)'
if pairs is None:
pairs = [(0x1100, 0x40), (0x10C0, 0x20), (0x10A0, 0x10),
(0x1090, 0x08), (0x1088, 0x08), (0x1080, 0x08)]
end = 0x10C
header = bytearray(0x200)
for idx, (offset, count) in enumerate(pairs):
struct.pack_into('<i', header, end - 8 * idx, count)
struct.pack_into('<i', header, end - 8 * idx - 4, offset)
declared = pairs[0][0] + pairs[0][1]
forward = max(offset + count for offset, count in pairs)
walked = _walk(header, start)
off_grid = _walk(header, start + 4)
ok = walked == declared == forward and off_grid != declared
return report(name, ok,
['declared %#x, forward scan %#x, walk from %#x -> %#x'
% (declared, forward, start, walked),
'negative control, off-grid start %#x -> %#x (must differ)'
% (start + 4, off_grid),
'magic/version zeroed in this fixture: the walk uses only offset/count pairs'])
def main():
print('== host checks for the 2026-10-27 ref audit pass ==')
verdicts = [check() for check in (ctr_nonce_tail, comparison_oracle, header_walk)]
skipped = sum(1 for v in verdicts if v is None)
failed = sum(1 for v in verdicts if v is False)
print('')
print('%d check(s), %d skipped, %d failed' % (len(verdicts), skipped, failed))
print('RESULT=%s' % ('ok' if not failed else 'failed'))
return 0 if not failed else 1
if __name__ == '__main__':
sys.exit(main())