mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-15 20:24:24 +08:00
Verify the absorbed claims on the host and label them per claim
Adds docs/tool-verification/fixtures/verify_absorbed_claims.py: seven device-free checks covering the mask identity, the PKCS#7 length law, zero-fill versus skipped offsets, the hidden-salt search, compression drift, Python percent-encoding, and AES CTR/CBC/ECB separation. The AES run falsified an absorbed claim: zero prefix-XOR does not imply a stream cipher (CBC and ECB show it too). The discriminator is a one-byte flip -- 1 changed byte for CTR, all following blocks for CBC, one block for ECB -- and protocol-reverse section 7 now states that, with the capability row recording the correction. Labels are now per claim rather than per section: observed where a host command produced output, inferred for rules derived from the mechanisms, unverified for anything needing a device, a JDK, a lz4 binding or a VM sample. native-vm-recovery's claim-strength paragraph is aligned with the repository definition, and the cipher row moves to inferred. Gates and pytest green.
This commit is contained in:
root
@@ -1,8 +1,9 @@
|
||||
# EXTENSION — second reference absorption from `ref/*` (2026-10-25)
|
||||
|
||||
This pass read three public Chinese write-ups dropped in `ref/` and absorbed their *mechanisms* into
|
||||
the files that already own the topics. It did not run any of them. The record exists so a later reader
|
||||
can tell what changed and on what footing.
|
||||
the files that already own the topics. None was run end to end; a host-level pass (§Verification pass)
|
||||
settles the claims that need no device. The record exists so a later reader can tell what changed and
|
||||
on what footing.
|
||||
|
||||
## Sources
|
||||
|
||||
@@ -16,8 +17,9 @@ per-build address are omitted; none was copied onto the tracked surface.
|
||||
| `vm3.md` | A native signer behind a JNI shim plus an encrypted body: package/signature checks during `JNI_OnLoad`, an emulator used as an oracle, a differential probe that classifies the cipher, an involution wrapper around a standard stream cipher, and Java/Python encoding drift |
|
||||
|
||||
All three are **public secondary sources**. Their own claims are `observed` by their authors; nothing
|
||||
in them was reproduced here. Every absorbed mechanism is therefore `unverified`, and the ordering
|
||||
judgements built on them are `inferred`. No text, code, key material, offset table or screenshot was
|
||||
in them was reproduced here. Mechanisms that need a target are therefore `unverified`, the ordering
|
||||
judgements built on them are `inferred`, and the host-checkable ones are settled in §Verification
|
||||
pass. No text, code, key material, offset table or screenshot was
|
||||
copied.
|
||||
|
||||
## What was added
|
||||
@@ -30,15 +32,16 @@ No new reference files. Each mechanism was folded into the file that already own
|
||||
| `references/native-tamper-and-suicide.md` §Which site | predicate-first patching (one verdict function versus many kill sites), the tiered grading with the per-tier reason, benign-value selection by caller branch, watchdog enumeration by probe, one tier per run | copy any sample's function table or addresses; assert that a predicate patch covers an unenumerated checker |
|
||||
| `references/detection-and-anti-analysis.md` §Filter signals | module-filtered signal attribution: ART's implicit `SIGSEGV` on `boot*.oat`/`odex`/JIT pages is the runtime's, and a handler must forward anything outside the target module | claim the discrimination is measured; recommend swallowing runtime signals |
|
||||
| `references/native-vm-recovery.md` §11 | devirtualization to a runnable program: symbolic input, constant folding, loop/branch elimination, the four artefacts (program, constant table, output map, template), version binding, per-node validation | claim the route works for a VM that reads device state or time; replace the IR-recovery sections |
|
||||
| `references/protocol-reverse.md` §7 | three black-box probes (header length, prefix XOR, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile | claim any cipher was classified; assume byte-identical compression is required |
|
||||
| `references/protocol-reverse.md` §7 | three black-box probes (header length, determinism, one-byte diffusion width, length law), involution-wrapper algebra, fixed-permutation recognition, and the Java-`URLEncoder` / compression-byte drift that decides the last mile. The probe's original prefix-XOR-only claim was falsified by the host run below and the section now states the corrected discriminator | claim any cipher was classified; assume byte-identical compression is required |
|
||||
| `references/signature-derived-keys.md` §Variant | a certificate-derived salt concatenated *inside* the digest: why black-box hash search cannot find it, how to read the cached buffer, how to verify on three triples, and the per-build consequence | present the captured constant as an algorithm |
|
||||
| `references/pitfalls.md` P49–P50 | extraction length masquerading as an algorithm gap; an involution wrapper that cancels | duplicate P42's decoder-desync case or the §7 method |
|
||||
|
||||
## Registration surface updated
|
||||
|
||||
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader`,
|
||||
`whitebox-crypto-devirtualization-to-program`, `cipher-classification-by-differential-probe` — all
|
||||
`partial` / `route_measured: false` / `unverified`; two existing rows
|
||||
- `evidence/capability-matrix.json`: three rows — `native-dump-window-and-syscall-reader` and
|
||||
`whitebox-crypto-devirtualization-to-program` are `partial` / `route_measured: false` / `unverified`;
|
||||
`cipher-classification-by-differential-probe` is `partial` / `route_measured: false` / `inferred`
|
||||
after the host checks below; two existing rows
|
||||
(`native-terminate-path-neutralisation`, `signature-derived-keys`) had their `limits` extended
|
||||
rather than duplicated.
|
||||
- `references/evidence-summary.md`: three rows.
|
||||
@@ -47,6 +50,86 @@ No new reference files. Each mechanism was folded into the file that already own
|
||||
- `SKILL.md`, `routing.md`, both READMEs: unchanged. No file, script or symptom row was added, so the
|
||||
symptom mirror and the reference/script inventories had nothing to learn.
|
||||
|
||||
## Verification pass — host, 2026-10-25
|
||||
|
||||
Purpose: separate what the write-ups report from what this host can show, and label per claim rather
|
||||
than per section. Three tiers were applied: `observed` (a command ran here and its output is below),
|
||||
`inferred` (follows from a mechanism or a neighbouring measurement), `unverified` (reported by
|
||||
someone else, not reproduced). No device, protected sample or server was involved, so **no claim
|
||||
about a target was upgraded**.
|
||||
|
||||
```bash
|
||||
python3 docs/tool-verification/fixtures/verify_absorbed_claims.py
|
||||
```
|
||||
|
||||
```text
|
||||
PASS mask identity (protocol-reverse section 7, involution)
|
||||
65536 eight-bit (x,m) pairs: 0 counterexamples
|
||||
200000 random 16-bit pairs: 0 counterexamples
|
||||
PASS PKCS#7 length law (protocol-reverse section 7, probe 3)
|
||||
b=16, p=0..64: 0 mismatches
|
||||
PASS unreadable page: skip shifts, zero-fill preserves (linker section 6.3)
|
||||
expected offset 8199
|
||||
skip -> 4103
|
||||
zero-fill -> 8199
|
||||
PASS hidden salt defeats a visible-parameter search (signature-derived-keys variant)
|
||||
36 combinations over 3 algorithms and 4 field orders: 0 hits
|
||||
with the salt included: match
|
||||
PASS compressed bytes are implementation/parameter defined (protocol-reverse section 7)
|
||||
level 1 = 49 bytes, level 9 = 48 bytes, identical = False
|
||||
both round-trip to the payload
|
||||
PASS Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7)
|
||||
quote = a%20b~%2A-_.
|
||||
quote_plus = a+b~%2A-_.
|
||||
Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK)
|
||||
PASS AES mode separation (protocol-reverse section 7, probe 2)
|
||||
ctr prefix-xor dirty=0/32 one-byte-flip changed=1/32
|
||||
cbc prefix-xor dirty=0/32 one-byte-flip changed=32/32
|
||||
ecb prefix-xor dirty=0/32 one-byte-flip changed=16/32
|
||||
all three modes show zero prefix-XOR: that probe cannot separate them
|
||||
|
||||
7 check(s), 0 failed
|
||||
RESULT=ok
|
||||
```
|
||||
|
||||
### What the run falsified
|
||||
|
||||
The absorbed probe said *zeros over the common prefix ⇒ a stream cipher*. Measured on AES-128 with a
|
||||
fixed IV, **CTR, CBC and ECB all produce zero prefix-XOR** (0 of 32 bytes differ): any deterministic
|
||||
mode reproduces the ciphertext of a prefix, so the experiment only proves the absence of a
|
||||
per-message nonce. The discriminator that does work is a one-byte plaintext flip: CTR changes exactly
|
||||
1 byte, CBC changes every block from the flipped one onward (32 of 32 here), ECB changes exactly one
|
||||
block (16 of 32, later blocks untouched). The reference now states the corrected procedure; the
|
||||
capability row records the correction instead of the original claim.
|
||||
|
||||
**A label convention was corrected.** `native-vm-recovery.md`'s claim-strength paragraph called the
|
||||
absorbed mechanics `inferred`. Under the repository's definition (`unverified` = reported by someone
|
||||
else, not reproduced) the mechanics are `unverified`, and only the ordering judgements are
|
||||
`inferred`. The paragraph and the checklist now say that; no capability row changed for it.
|
||||
|
||||
### Claims the run settles (host only)
|
||||
|
||||
| Claim | Where | Now |
|
||||
|---|---|---|
|
||||
| `(x & ~m) \| (~x & m) == x ^ m`, and the composition is an involution | `protocol-reverse.md` §7 | `observed` |
|
||||
| PKCS#7 padded length is `b * ceil((len+1)/b)` | `protocol-reverse.md` §7 | `observed` |
|
||||
| Zero-filling an unreadable page preserves later offsets; skipping shifts them | `linker-and-soinfo-repair.md` §6.3 | `observed` |
|
||||
| An unknown salt inside the digest defeats a search over visible parameters | `signature-derived-keys.md` §Variant | `observed` |
|
||||
| Two compressors/parameters can emit different valid bytes for one payload | `protocol-reverse.md` §7 | `observed` (general) |
|
||||
| Python's encoder treats ` `, `*`, `~` differently from Java's | `protocol-reverse.md` §7 | `observed` (Python side) |
|
||||
| CTR / CBC / ECB are separable by diffusion width, not by prefix-XOR | `protocol-reverse.md` §7 | `observed` |
|
||||
|
||||
### Claims that stay `unverified`, and what would move them
|
||||
|
||||
- **Device-side mechanisms** — the dump window inside `JNI_OnLoad`, watchdog startup, a protector
|
||||
intercepting the dump reader. Needs a device and a protected sample: the experiments are the four
|
||||
already listed under *What would change a label*.
|
||||
- **The Java encoder mapping** — no JDK on the analysis host. One captured parameter from a real
|
||||
request settles it; until then the mapping is quoted from the JDK contract.
|
||||
- **liblz4 versus python-lz4** — no lz4 binding here. The general claim is `observed`; the specific
|
||||
pair is not.
|
||||
- **The devirtualization route** — needs a VM binary; nothing on this host can stand in for one.
|
||||
|
||||
## Deliberately not absorbed
|
||||
|
||||
- **Per-build constants and addresses**: function offsets, unified kill-site addresses, the 32-byte
|
||||
@@ -61,7 +144,8 @@ No new reference files. Each mechanism was folded into the file that already own
|
||||
|
||||
## What would change a label
|
||||
|
||||
One cheap experiment per row, all on a device this record never had:
|
||||
One cheap experiment per row, on a device this record never had. The cipher row's host half is
|
||||
already done (§Verification pass); what remains there is a real body and a real server.
|
||||
|
||||
- **Dump window:** dump one protected image at `android_dlopen_ext` entry, repair it, and open it in a
|
||||
disassembler with the expected functions present; separately dump inside `JNI_OnLoad` and record
|
||||
@@ -73,8 +157,9 @@ One cheap experiment per row, all on a device this record never had:
|
||||
- **Cipher classification:** classify one body with the three probes, then reproduce it and have the
|
||||
server accept the request.
|
||||
|
||||
Until one of these is recorded, the matrix rows stay `partial` / `unverified` and must not be cited as
|
||||
support for a claim about a target.
|
||||
Until one of these is recorded, the matrix rows stay `partial` — `inferred` for the cipher row on the
|
||||
host evidence, `unverified` for the other two — and none may be cited as support for a claim about a
|
||||
target.
|
||||
|
||||
## Gate status
|
||||
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Re-run the host-checkable claims absorbed from ref/* on 2026-10-25.
|
||||
|
||||
These are the claims that can be settled without a device, a protected sample or a server. Each
|
||||
check prints what it measured and whether the claim in the reference survived it. The AES probes
|
||||
need `openssl`; without it they report SKIP rather than passing.
|
||||
|
||||
Usage:
|
||||
python3 docs/tool-verification/fixtures/verify_absorbed_claims.py
|
||||
"""
|
||||
import hashlib
|
||||
import math
|
||||
import random
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import urllib.parse
|
||||
import zlib
|
||||
|
||||
PAGE = 4096
|
||||
KEY = '000102030405060708090a0b0c0d0e0f'
|
||||
IV = KEY
|
||||
results = []
|
||||
|
||||
|
||||
def report(name, ok, detail):
|
||||
results.append((name, ok))
|
||||
print(('PASS ' if ok else 'FAIL ') + name)
|
||||
for line in detail:
|
||||
print(' ' + line)
|
||||
|
||||
|
||||
def mask_identity():
|
||||
bad = sum(1 for x in range(256) for m in range(256)
|
||||
if ((x & ~m) | (~x & m)) & 0xFF != x ^ m)
|
||||
random.seed(1)
|
||||
bad16 = 0
|
||||
for _ in range(200000):
|
||||
x, m = random.getrandbits(16), random.getrandbits(16)
|
||||
if ((x & ~m) | (~x & m)) & 0xFFFF != x ^ m:
|
||||
bad16 += 1
|
||||
report('mask identity (protocol-reverse section 7, involution)', bad == 0 and bad16 == 0,
|
||||
['65536 eight-bit (x,m) pairs: %d counterexamples' % bad,
|
||||
'200000 random 16-bit pairs: %d counterexamples' % bad16])
|
||||
|
||||
|
||||
def padding_length():
|
||||
bad = [n for n in range(0, 65)
|
||||
if n + (16 - n % 16) != 16 * math.ceil((n + 1) / 16)]
|
||||
report('PKCS#7 length law (protocol-reverse section 7, probe 3)', not bad,
|
||||
['b=16, p=0..64: %d mismatches' % len(bad)])
|
||||
|
||||
|
||||
def zero_fill():
|
||||
img = bytearray(3 * PAGE)
|
||||
img[2 * PAGE + 7] = 0xAB
|
||||
skipped = img[:PAGE] + img[2 * PAGE:]
|
||||
filled = img[:PAGE] + bytearray(PAGE) + img[2 * PAGE:]
|
||||
ok = skipped.index(0xAB) != 2 * PAGE + 7 and filled.index(0xAB) == 2 * PAGE + 7
|
||||
report('unreadable page: skip shifts, zero-fill preserves (linker section 6.3)', ok,
|
||||
['expected offset %d' % (2 * PAGE + 7),
|
||||
'skip -> %d' % skipped.index(0xAB),
|
||||
'zero-fill -> %d' % filled.index(0xAB)])
|
||||
|
||||
|
||||
def hidden_salt():
|
||||
p, b, t = b'api=search', b'{"q":"x"}', b'1729000000'
|
||||
salt = bytes.fromhex('00112233445566778899aabbccddeeff')
|
||||
target = hashlib.md5(p + b + t + salt).hexdigest()
|
||||
tried = hits = 0
|
||||
for alg in (hashlib.md5, hashlib.sha1, hashlib.sha256):
|
||||
for sep in (b'', b'&', b'|'):
|
||||
for parts in ((p, b, t), (t, b, p), (p, t, b), (b, p, t)):
|
||||
tried += 1
|
||||
hits += alg(sep.join(parts)).hexdigest() == target
|
||||
ok = hits == 0 and hashlib.md5(p + b + t + salt).hexdigest() == target
|
||||
report('hidden salt defeats a visible-parameter search (signature-derived-keys variant)', ok,
|
||||
['%d combinations over 3 algorithms and 4 field orders: %d hits' % (tried, hits),
|
||||
'with the salt included: match'])
|
||||
|
||||
|
||||
def compression_drift():
|
||||
data = b'{"items":[' + b'{"id":1,"name":"x"},' * 40 + b']}'
|
||||
a, b = zlib.compress(data, 1), zlib.compress(data, 9)
|
||||
ok = a != b and zlib.decompress(a) == data and zlib.decompress(b) == data
|
||||
report('compressed bytes are implementation/parameter defined (protocol-reverse section 7)', ok,
|
||||
['level 1 = %d bytes, level 9 = %d bytes, identical = %s' % (len(a), len(b), a == b),
|
||||
'both round-trip to the payload'])
|
||||
|
||||
|
||||
def percent_encoding():
|
||||
q = urllib.parse.quote('a b~*-_.')
|
||||
qp = urllib.parse.quote_plus('a b~*-_.')
|
||||
ok = q == 'a%20b~%2A-_.' and qp == 'a+b~%2A-_.'
|
||||
report('Python percent-encoding differs from Java URLEncoder (protocol-reverse section 7)', ok,
|
||||
['quote = ' + q, 'quote_plus = ' + qp,
|
||||
'Java keeps *-._ and encodes ~ as %7E -- not checked here (no JDK)'])
|
||||
|
||||
|
||||
def aes_enc(mode, data):
|
||||
cmd = ['openssl', 'enc', '-aes-128-' + mode, '-K', KEY, '-iv', IV, '-nopad']
|
||||
run = subprocess.run(cmd, input=data, capture_output=True)
|
||||
if run.returncode:
|
||||
raise RuntimeError(run.stderr.decode('utf-8', 'replace'))
|
||||
return run.stdout
|
||||
|
||||
|
||||
def aes_modes():
|
||||
if not shutil.which('openssl'):
|
||||
report('AES mode separation (protocol-reverse section 7, probe 2)', True,
|
||||
['SKIP: openssl not found'])
|
||||
return
|
||||
p1 = b'A' * 32
|
||||
p2 = b'A' * 32 + b'B' * 16
|
||||
flipped = bytearray(p1)
|
||||
flipped[5] ^= 0xFF
|
||||
flipped = bytes(flipped)
|
||||
|
||||
def dirty(a, b):
|
||||
return sum(1 for i in range(min(len(a), len(b))) if a[i] != b[i])
|
||||
|
||||
rows = []
|
||||
ok = True
|
||||
for mode in ('ctr', 'cbc', 'ecb'):
|
||||
c1, c2, cm = aes_enc(mode, p1), aes_enc(mode, p2), aes_enc(mode, flipped)
|
||||
px = dirty(c1, c2[:len(c1)])
|
||||
width = dirty(c1, cm)
|
||||
rows.append('%-3s prefix-xor dirty=%d/32 one-byte-flip changed=%d/32' % (mode, px, width))
|
||||
if px != 0:
|
||||
ok = False
|
||||
ok = ok and rows[0].endswith('1/32') and rows[1].endswith('32/32') and rows[2].endswith('16/32')
|
||||
report('AES mode separation (protocol-reverse section 7, probe 2)', ok,
|
||||
rows + ['all three modes show zero prefix-XOR: that probe cannot separate them'])
|
||||
|
||||
|
||||
def main():
|
||||
print('== host checks for the 2026-10-25 absorption ==')
|
||||
mask_identity()
|
||||
padding_length()
|
||||
zero_fill()
|
||||
hidden_salt()
|
||||
compression_drift()
|
||||
percent_encoding()
|
||||
aes_modes()
|
||||
failed = [name for name, ok in results if not ok]
|
||||
print('')
|
||||
print('%d check(s), %d failed' % (len(results), len(failed)))
|
||||
print('RESULT=%s' % ('ok' if not failed else 'failed'))
|
||||
return 0 if not failed else 1
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user