mirror of
https://codeberg.org/reverse-skill-lab/apk-reverse
synced 2026-11-17 00:04:23 +08:00
Reconstruction pass: make the gates tell the truth, make the package self-contained, keep the one-hop symptom index
An external audit of this repository landed on four structural findings, and this pass is the
response to them. Everything below was re-measured on this tree before being acted on; two of the
audit's claims needed a correction of my own, and both corrections are noted in place.
**The gates were optimistic, and one of them did not read content at all.**
doctor.py the capability table was hard-coded (`repack + sign = bool(java) and python`), so a
machine with a JDK and no zipalign/apksigner was told it could re-sign; the script
inventory was a hand-written list of 28 against 51 files on disk, omitting the nine
most important scripts. It now scans the directory (`registered/checked = 52 / 52`,
unjudgeable entries counted as `unknown`, never skipped) and resolves capability as a
real requires-closure, reporting BLOCKED with the missing piece and an installable
next step. Measured: the old doctor reports `[OK] repack + sign` on a JDK-only PATH
where keytool, jarsigner, zipalign, apksigner and all eight smali jars are absent.
The report is allowed to look bad; that is the point.
check_commands.py (new)
anchor checks prove a section exists, never that a command runs. This parses each
script's argparse table with `ast` and checks every documented invocation against it.
146 commands / 41 interfaces, 0 drift. It caught `sig_probe.py --live --pkg`, which
does not exist -- the reference had been wrong since it was written.
check_routing.py (new)
the symptom index now has one home (`SKILL.md`, because symptom -> file must stay one
hop) and the inventory has another (`references/routing.md`). This gate proves the
mirror agrees, that every reference file is named, and that every script is named.
**so_constpatch.py was corrupting APKs it rebuilt.** The writer copied filename and timestamp onto a
fresh ZipInfo and recompressed the whole archive: 435-entry real package, `extra` lost on 205 entries,
`external_attr` rewritten on all 435, 132 untouched entries recompressed, and `resources.arsc` pushed
to an unaligned offset. Android's own `zipalign -c -v 4` fails the old output and passes the new one;
a 26-case fixture suite pins it, and `extractNativeLibs="false"` is now refused by default with a
pointer at `repack.py`. Fixing it exposed the same defect in `repack.py`, where `^lib/[^/]+\.so$`
cannot match `lib/arm64-v8a/*.so` -- so its alignment handling had never once fired on a real APK.
**The package was not self-contained.** 59 references pointed at the repository root, which `npx
skills add` does not install. They now converge on `references/evidence-summary.md` (capability,
conclusion, strength, evidence that ships) plus `evidence/` with the machine-readable companions: 8
references remain and each is explicitly labelled "repository root, not shipped". A drill in which
the new files were copied to an isolated directory and asked three questions answered all three.
**SKILL.md: 572 lines / 15,733 tokens -> 398 lines / ~10,900 tokens**, without losing the two things
that had to stay. The symptom index remains inline; the reference and script indexes moved to the
on-demand routing file. The freed budget went into making the rules *stronger*, not shorter: the
constraints section is now a table of thirteen non-negotiable constraints each paired with the check
that catches a violation, and the tooling obligations are instructions whose violation is a defect
(hedging language in SKILL.md: 1 line, down from a paragraph of hedged advice). Long references gained
a navigation head so a reader can decide from the top whether to load one.
**New: an executable test layer.** `tests/` (unit, CLI contract, no-device integration) plus
`pytest.ini`, `tests/run_tests.py`, a minimal CI job, and `evals/evals.json` written but never run --
stated as such in the file. 608 passed / 158 skipped / 25 xfailed. The 25 xfails are a work order, not
a pass: thirteen scripts report a missing input by exiting 1 instead of 2, seven let a traceback
escape, four treat `--help` as an error, and `preflight --json` lacks a `status` field. A regression
row for `dex_patch_bytes.py` reproduces a previous artifact byte-for-byte
(`67ae1e58e7a44777e5ac8f9f823ab21e…`) and additionally pins the four bytes that may move.
**Two real defects the new tests found in the tooling itself.** `check_repo.py` decoded subprocess
output with the locale encoding, so on a GBK console a script printing an em dash in its usage line
was reported as crashed; it now decodes UTF-8 explicitly. And a CLI contract test drove a *real
attached device* through `coldstart.py` -- logcat, screenshots, five orphan `adb logcat` processes --
which is fixed twice over: child processes run in a private temporary cwd, and the fifteen
device-bound scripts are never executed beyond `--help`/argument parsing.
**Quoting on the device side is now one implementation.** `devsh.su` escaped only double quotes, so
`;`, `$( )` and backticks in a caller-supplied value still reached the device shell while
`coldstart.Device.shell` quoted correctly. Both use `scripts/device_shell.py` now: POSIX single-quote
escaping, argv arrays by default, and identifier validation that refuses a malformed package, path or
component with exit 2 instead of a traceback. The integration test that used to pin the *defect* (it
asserted `su -c "…"` with `\"` escapes) now asserts the property, across five metacharacter payloads.
Gates: check_repo 0 problems (including the content-reading identity scan over git ls-files),
check_refs 0 dangling / 0 warning, check_routing coherent (53 symptom / 45 reference / 54 script rows),
check_budget 0 failure / 0 warning with the always-loaded body now measured in tokens rather than in
"narrative lines with the index excluded", check_commands 146 checked / 0 drift, pytest 608 passed.
Not done, and stated rather than implied: CI has never run on a runner; the evals have never been
executed; the 25 xfails are open; no device-side install/launch was performed in this pass.
This commit is contained in:
@@ -0,0 +1,75 @@
|
||||
# tests/ - the executable half of this repository's evidence
|
||||
|
||||
`tests/benchmark.md` is a **record**: what happened when a route was run once, by hand,
|
||||
against a real sample and a real device. This directory is the other half: checks that run
|
||||
on every commit with nothing but the standard library and pytest.
|
||||
|
||||
The two are not substitutes. A benchmark row can establish that a route works on a hardened
|
||||
target; it cannot tell you that yesterday's refactor broke the dex width table. A test in
|
||||
here can tell you that every time, and it can never tell you whether the route achieves the
|
||||
goal on a real app.
|
||||
|
||||
```
|
||||
python -m pytest -q # whole suite
|
||||
python tests/run_tests.py # same thing, with a clear message if pytest is missing
|
||||
python -m pytest -q -m unit # one marker
|
||||
python -m pytest -q -m cli # exit codes, flags, --json
|
||||
python -m pytest -q -m integration # no-device flows
|
||||
python -m pytest -q -ra # list every skip and xfail reason
|
||||
```
|
||||
|
||||
## What the markers mean
|
||||
|
||||
- `unit` - pure functions and parsers of real shipped scripts: the dex width table and branch
|
||||
arithmetic, the scheme-less protobuf decoder, the blob rotation search, the leak rules, and
|
||||
the established `dex_patch_bytes` regression hash.
|
||||
- `cli` - one subprocess run per script per invocation shape: `--help`, an unknown flag, no
|
||||
arguments, and every input pointed at a file that does not exist.
|
||||
- `integration` - flows that were made of several parts: adb argv construction under a fake
|
||||
runner, logcat signal counting, tombstone parsing, the two-sample launch-health verdict.
|
||||
|
||||
## Nothing here needs a device
|
||||
|
||||
No test in this suite opens a frida session or touches the network. The device-dependent
|
||||
behaviour is tested by intercepting the command that *would* have been run and asserting on the
|
||||
argv list.
|
||||
|
||||
The exception is deliberate and enforced: `contract.DEVICE_OR_NETWORK_BOUND` names the 15
|
||||
scripts that reach a device once their arguments parse, and those are **never executed** with
|
||||
complete arguments — only `--help` and an unknown flag, which argparse answers before any
|
||||
device call. That blocklist exists because a development host may well have a phone attached
|
||||
(the machine this harness was written on did), and an earlier version of the CLI contract test
|
||||
drove it for real. General safety net: every child runs in a private temporary working
|
||||
directory, so a script that defaults an output path relative to cwd cannot write into the
|
||||
shipped tree.
|
||||
|
||||
Anything that genuinely cannot be reached this way is not faked into a pass — it is listed as a
|
||||
coverage gap in `docs/tool-verification/EXTENSION-test-harness.md`.
|
||||
|
||||
## Skipped is not passed
|
||||
|
||||
Two kinds of non-passing outcome are kept separate on purpose:
|
||||
|
||||
- **`skip`** - the check does not apply here. `tools/_work/bench/**` is git-ignored, so the
|
||||
`dex_patch_bytes` regression hash self-skips on a fresh checkout and in CI, printing the
|
||||
reason. A --json script that needs a sample to reach its JSON path skips too.
|
||||
- **`xfail`** - the script is present and does **not** meet the repository contract. The
|
||||
reason carries the measured value, for example
|
||||
`contract says exit 2, measured 1`. These are work items, not flaky tests.
|
||||
|
||||
`pytest -q -ra` prints both lists. The xfail list for the CLI layer is the exit-code
|
||||
unification work order; it is reproduced in `EXTENSION-test-harness.md` with owners.
|
||||
|
||||
## Ground rules
|
||||
|
||||
- **This suite never edits anything under `skills/`.** A defect found here is registered in
|
||||
the harness report. Fixing a script's exit code or quoting strategy is its owner's change,
|
||||
and doing it from a test would hide the drift the test exists to expose.
|
||||
- **The bench tree is evidence and is read-only here.** The regression test copies its input
|
||||
into `tmp_path` before patching, so `tools/_work/bench/l1l3/dex/classes.patched.dex` cannot
|
||||
be overwritten by a test run.
|
||||
- **No real target identity is committed.** The leak-scanner decoys are synthesised in
|
||||
`tmp_path` at run time, and token-shaped decoys are assembled from fragments so that this
|
||||
directory cannot itself become a finding.
|
||||
- Fixtures in `tests/fixtures/` are small, synthetic sample files. `tests/fixtures/apk/**` is
|
||||
owned by a different work item and is not touched from here.
|
||||
@@ -0,0 +1,181 @@
|
||||
"""Helpers for the CLI contract tests.
|
||||
|
||||
Everything here is derived at run time from the scripts' own `--help` output
|
||||
rather than from a hard-coded list of argument names, so a new script is covered
|
||||
automatically and a renamed flag does not silently stop being tested.
|
||||
|
||||
The contract under test is the repository-wide one recorded in
|
||||
tools/_work/REBUILD-BRIEF.md section 4:
|
||||
|
||||
0 = success
|
||||
1 = the target result failed
|
||||
2 = bad arguments or input
|
||||
3 = the environment lacks a capability
|
||||
4 = internal tool error
|
||||
|
||||
A script that does not follow it yet is not a red test here: the test calls
|
||||
``pytest.xfail`` with the measured code in the reason, so the summary line stays
|
||||
green and the xfail list is exactly the work order for unifying exit codes.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
# A path that does not exist anywhere. Used to drive the "bad input" scenario.
|
||||
NONEXISTENT = str(Path("__apkrev_missing__") / "does-not-exist.bin")
|
||||
|
||||
# The four contract codes, plus None for "did not return".
|
||||
CONTRACT_EXIT_CODES = {0, 1, 2, 3, 4}
|
||||
|
||||
|
||||
def parse_leading_json(text):
|
||||
"""Parse the first JSON document in `text`, ignoring anything after it.
|
||||
|
||||
Several scripts print a `RESULT=<token>` line after their JSON document (the
|
||||
repository's long-task discipline requires that token last), so a plain
|
||||
json.loads on the whole stream fails. Only the first document is returned.
|
||||
"""
|
||||
starts = [i for i in (text.find("{"), text.find("[")) if i >= 0]
|
||||
if not starts:
|
||||
return None
|
||||
try:
|
||||
doc, _end = json.JSONDecoder().raw_decode(text[min(starts):])
|
||||
except ValueError:
|
||||
return None
|
||||
return doc
|
||||
|
||||
|
||||
def usage_block(help_text):
|
||||
"""Return the joined `usage:` block of an argparse-rendered help text."""
|
||||
lines = help_text.splitlines()
|
||||
start = None
|
||||
for i, line in enumerate(lines):
|
||||
if line.lower().startswith("usage:"):
|
||||
start = i
|
||||
break
|
||||
if start is None:
|
||||
return None
|
||||
block = [lines[start][len("usage:"):].strip()]
|
||||
for line in lines[start + 1:]:
|
||||
if not line.strip() or not line.startswith((" ", "\t")):
|
||||
break
|
||||
block.append(line.strip())
|
||||
return " ".join(block)
|
||||
|
||||
|
||||
def _strip_bracketed(block):
|
||||
"""Split a usage line into tokens, dropping every bracketed (optional) group."""
|
||||
tokens, depth, cur = [], 0, ""
|
||||
for ch in block:
|
||||
if ch in "[(":
|
||||
depth += 1
|
||||
continue
|
||||
if ch in ")]":
|
||||
depth -= 1
|
||||
continue
|
||||
if depth:
|
||||
continue
|
||||
if ch.isspace():
|
||||
if cur:
|
||||
tokens.append(cur)
|
||||
cur = ""
|
||||
else:
|
||||
cur += ch
|
||||
if cur:
|
||||
tokens.append(cur)
|
||||
return tokens
|
||||
|
||||
|
||||
def usage_state(help_text):
|
||||
"""Classify a script's usage line.
|
||||
|
||||
Returns one of:
|
||||
"unreadable" - no argparse-style `usage:` block at all (hand-written usage)
|
||||
"none-required" - a usage block exists but names no positional and no required option
|
||||
"required" - at least one positional or required option is named
|
||||
"""
|
||||
block = usage_block(help_text)
|
||||
if block is None:
|
||||
return "unreadable"
|
||||
return "required" if _strip_bracketed(block)[1:] else "none-required"
|
||||
|
||||
|
||||
def missing_input_args(help_text):
|
||||
"""Build an argv that points every positional and required option at a missing file.
|
||||
|
||||
Returns None when there is nothing to point at: either the usage block cannot
|
||||
be recovered (the four scripts with a hand-written parser print their own
|
||||
usage, not argparse's) or the script takes no required argument at all.
|
||||
"""
|
||||
if usage_state(help_text) != "required":
|
||||
return None
|
||||
tokens = _strip_bracketed(usage_block(help_text))
|
||||
args, i = [], 1 # tokens[0] is the program name
|
||||
while i < len(tokens):
|
||||
token = tokens[i]
|
||||
if token.startswith("-"):
|
||||
args.append(token)
|
||||
if i + 1 < len(tokens) and not tokens[i + 1].startswith("-"):
|
||||
args.append(NONEXISTENT)
|
||||
i += 1
|
||||
else:
|
||||
args.append(NONEXISTENT)
|
||||
i += 1
|
||||
return args or None
|
||||
|
||||
|
||||
def has_option(help_text, option):
|
||||
return option in help_text
|
||||
|
||||
|
||||
# Scripts that reach a device, a live frida server or a network peer once their
|
||||
# arguments parse. They are listed explicitly for one reason: this suite must not
|
||||
# touch a device, and a development host may well have one attached. Measured on
|
||||
# the machine this harness was written on, `adb devices` reported one online
|
||||
# device, and running `coldstart.py --pkg <name>` really did start
|
||||
# `adb logcat -v threadtime` and capture frames. So the exclusion is not
|
||||
# theoretical: it is the difference between a unit test and a device test.
|
||||
#
|
||||
# `--help` and an unknown flag are still exercised for these scripts: argparse
|
||||
# answers both before any device call is reached, which was verified for every
|
||||
# one of them (`--help` returned in well under a second, an unknown flag returned
|
||||
# 2). It is the argument-complete scenarios that are refused.
|
||||
DEVICE_OR_NETWORK_BOUND = {
|
||||
"coldstart.py", "datastore_inject.py", "devsh.py", "frida_rpc_serve.py",
|
||||
"grab_crash.py", "install_test.py", "lib_map.py", "mt_mcp_probe.py",
|
||||
"probe_api.py", "run_probe.py", "sig_probe.py", "snap.py",
|
||||
"spawn_patch_detach.py", "tls_check.py", "usb_net_proxy.py",
|
||||
}
|
||||
|
||||
|
||||
def execution_refusal(name, scenario, help_text):
|
||||
"""Why this script must not be executed for `scenario`, or None if it may be.
|
||||
|
||||
Returns a skip reason rather than a verdict: not executing a script is a
|
||||
coverage limit, not a contract violation, and the two must not be reported
|
||||
with the same word.
|
||||
"""
|
||||
if name in DEVICE_OR_NETWORK_BOUND:
|
||||
return ("%s reaches a device or the network once its arguments parse; %s is not "
|
||||
"executed because this suite does not touch a device" % (name, scenario))
|
||||
state = usage_state(help_text)
|
||||
if state == "unreadable":
|
||||
return ("%s prints its own usage instead of argparse's, so %s cannot be derived "
|
||||
"automatically" % (name, scenario))
|
||||
if state == "none-required":
|
||||
return ("%s requires no argument, so %s is legal and the contract does not apply"
|
||||
% (name, scenario))
|
||||
return None
|
||||
|
||||
# The scripts whose --json path is reachable on a bare checkout. Everything else
|
||||
# that advertises --json needs a sample or a device to get that far; those are
|
||||
# skipped with that reason rather than asserted against a guess.
|
||||
JSON_RUNNABLE = {
|
||||
"doctor.py": ["--json"],
|
||||
"preflight.py": ["--json"],
|
||||
}
|
||||
|
||||
# Keys the unified JSON contract requires at minimum.
|
||||
JSON_REQUIRED_KEYS = ("status",)
|
||||
JSON_CONTEXT_KEYS = ("exit_code", "capability", "evidence", "warnings", "next_action")
|
||||
@@ -0,0 +1,262 @@
|
||||
"""CLI contract tests: every shipped script, four invocation shapes each.
|
||||
|
||||
What is asserted, per the repository-wide contract in REBUILD-BRIEF section 4:
|
||||
|
||||
1. `--help` succeeds and never prints an uncaught traceback.
|
||||
2. An unknown flag is a usage error: exit 2.
|
||||
3. An invocation with required arguments missing, or with every input pointed
|
||||
at a file that does not exist, is a usage error: exit 2.
|
||||
4. A script that advertises `--json` emits a parseable JSON document that
|
||||
states a `status`.
|
||||
5. Every observed exit code is one of 0/1/2/3/4.
|
||||
|
||||
Two clean outcomes are distinguished, and the distinction is the whole point:
|
||||
|
||||
* `xfail` means the script is on the tree and does **not** meet the contract.
|
||||
The reason carries the measured value. The xfail list this file produces is
|
||||
the work order for exit-code unification -- it is not a defect to be fixed
|
||||
here, because every one of those scripts has an owner.
|
||||
* `skip` means the scenario is not applicable or not reachable on a machine
|
||||
with no device, no network and no frida. Those are coverage gaps, and they
|
||||
are reported as such in docs/tool-verification/EXTENSION-test-harness.md.
|
||||
|
||||
Each scenario runs once per script and is cached for the session, so the suite
|
||||
spends its time on assertions rather than on re-spawning interpreters.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import all_scripts, run_script
|
||||
from contract import (
|
||||
CONTRACT_EXIT_CODES, DEVICE_OR_NETWORK_BOUND, JSON_REQUIRED_KEYS,
|
||||
JSON_RUNNABLE, execution_refusal, has_option, missing_input_args,
|
||||
parse_leading_json,
|
||||
)
|
||||
|
||||
pytestmark = pytest.mark.cli
|
||||
|
||||
SCRIPTS = all_scripts()
|
||||
BAD_FLAG = "--zzz-not-a-real-flag"
|
||||
|
||||
HELP_TIMEOUT = 60
|
||||
SCENARIO_TIMEOUT = 12
|
||||
|
||||
|
||||
def why(name):
|
||||
if name in DEVICE_OR_NETWORK_BOUND:
|
||||
return " (device/network bound: it may legitimately block)"
|
||||
return ""
|
||||
|
||||
|
||||
def require_returned(result, what):
|
||||
if result.timed_out:
|
||||
pytest.xfail("%s did not return within %ss%s; there is no exit code to check"
|
||||
% (what, SCENARIO_TIMEOUT, why(result.argv[1])))
|
||||
if result.returncode not in CONTRACT_EXIT_CODES:
|
||||
pytest.fail("%s returned %r, outside the 0/1/2/3/4 contract vocabulary"
|
||||
% (what, result.returncode))
|
||||
|
||||
|
||||
def expect_exit(result, expected, what):
|
||||
"""Pass when the measured code matches; otherwise record it as an xfail."""
|
||||
require_returned(result, what)
|
||||
if result.returncode != expected:
|
||||
pytest.xfail("%s: contract says exit %d, measured %d%s"
|
||||
% (what, expected, result.returncode, why(result.argv[1])))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# one cached run per script per scenario
|
||||
# --------------------------------------------------------------------------- #
|
||||
@pytest.fixture(scope="session")
|
||||
def help_results():
|
||||
return {n: run_script(n, ["--help"], timeout=HELP_TIMEOUT) for n in SCRIPTS}
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def badflag_results():
|
||||
return {n: run_script(n, [BAD_FLAG], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS}
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def empty_results():
|
||||
"""Bare invocations, for the scripts that may be run at all.
|
||||
|
||||
Device-bound scripts are excluded here too: their no-argument path is exactly
|
||||
where several of them start doing device work (`usb_net_proxy.py` did not
|
||||
return within 20 s when probed by hand), and this suite does not touch a device.
|
||||
"""
|
||||
return {n: run_script(n, [], timeout=SCENARIO_TIMEOUT) for n in SCRIPTS
|
||||
if n not in DEVICE_OR_NETWORK_BOUND}
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def missing_results(help_results):
|
||||
"""Argument-complete runs against a missing file, for the scripts that may be run at all.
|
||||
|
||||
Device-bound scripts are skipped here rather than executed and timed out: on
|
||||
the host this harness was written on, `adb devices` reported an online device,
|
||||
so running them would have touched real hardware.
|
||||
"""
|
||||
results = {}
|
||||
for name in SCRIPTS:
|
||||
if name in DEVICE_OR_NETWORK_BOUND:
|
||||
continue
|
||||
args = missing_input_args(help_results[name].output)
|
||||
if args is None:
|
||||
continue
|
||||
results[name] = run_script(name, args, timeout=SCENARIO_TIMEOUT)
|
||||
return results
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 1. --help
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestHelp:
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_help_never_crashes(self, name, help_results):
|
||||
"""Hard assertion: an uncaught traceback from --help is a defect, not a drift."""
|
||||
result = help_results[name]
|
||||
assert not result.timed_out, "--help did not return for %s" % name
|
||||
assert not result.has_traceback, result.output[-2000:]
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_help_prints_something(self, name, help_results):
|
||||
assert help_results[name].output.strip(), "--help printed nothing"
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_help_exits_zero(self, name, help_results):
|
||||
expect_exit(help_results[name], 0, "%s --help" % name)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 2. unknown flag
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestUnknownFlag:
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_unknown_flag_is_never_success(self, name, badflag_results):
|
||||
result = badflag_results[name]
|
||||
require_returned(result, "%s %s" % (name, BAD_FLAG))
|
||||
assert result.returncode != 0, (
|
||||
"an unknown flag was reported as success:\n%s" % result.output[-2000:])
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_unknown_flag_exits_two(self, name, badflag_results):
|
||||
expect_exit(badflag_results[name], 2, "%s %s" % (name, BAD_FLAG))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 3a. empty invocation
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestEmptyInvocation:
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_no_arguments_is_a_usage_error(self, name, help_results, empty_results):
|
||||
"""Only applies when the script actually requires arguments.
|
||||
|
||||
A script whose usage block has no positional and no required option may
|
||||
legitimately succeed with no arguments; demanding exit 2 from those would
|
||||
invent a rule the contract does not state. Device-bound scripts are not run
|
||||
at all -- see `contract.execution_refusal`.
|
||||
"""
|
||||
refusal = execution_refusal(name, "a bare invocation", help_results[name].output)
|
||||
if refusal:
|
||||
pytest.skip(refusal)
|
||||
expect_exit(empty_results[name], 2, "%s with no arguments" % name)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 3b. bad input
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestMissingInput:
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_missing_input_file_is_a_usage_error(self, name, help_results, missing_results):
|
||||
if name not in missing_results:
|
||||
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
|
||||
or "not executed for %s" % name)
|
||||
expect_exit(missing_results[name], 2, "%s with a missing input" % name)
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_missing_input_is_never_success(self, name, help_results, missing_results):
|
||||
if name not in missing_results:
|
||||
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
|
||||
or "not executed for %s" % name)
|
||||
result = missing_results[name]
|
||||
require_returned(result, "%s with a missing input" % name)
|
||||
assert result.returncode != 0, (
|
||||
"a missing input was reported as success:\n%s" % result.output[-2000:])
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_bad_input_reports_cleanly_not_as_an_exception(self, name, help_results,
|
||||
missing_results):
|
||||
"""A missing file is an expected failure mode; it must not escape as an exception.
|
||||
|
||||
Several scripts currently let `FileNotFoundError` propagate. Each is
|
||||
recorded here as an xfail naming the exception; the fix belongs to that
|
||||
script's owner, not to this suite.
|
||||
"""
|
||||
if name not in missing_results:
|
||||
pytest.skip(execution_refusal(name, "a missing input", help_results[name].output)
|
||||
or "not executed for %s" % name)
|
||||
result = missing_results[name]
|
||||
require_returned(result, "%s with a missing input" % name)
|
||||
if result.has_traceback:
|
||||
tail = [l for l in result.output.splitlines() if l.strip()]
|
||||
pytest.xfail("%s reports a missing input by letting an exception escape "
|
||||
"instead of exiting 2 cleanly: %s"
|
||||
% (name, (tail[-1] if tail else "uncaught exception")[:140]))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 4. the --json contract
|
||||
# --------------------------------------------------------------------------- #
|
||||
def json_capable(help_results):
|
||||
return sorted(n for n, r in help_results.items() if has_option(r.output, "--json"))
|
||||
|
||||
|
||||
def test_json_capable_scripts_are_discoverable(help_results):
|
||||
found = json_capable(help_results)
|
||||
assert found, "no script advertises --json; the discovery method is broken"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,args", sorted(JSON_RUNNABLE.items()))
|
||||
def test_json_output_is_parseable_and_states_a_status(name, args):
|
||||
result = run_script(name, args, timeout=120)
|
||||
require_returned(result, "%s %s" % (name, " ".join(args)))
|
||||
doc = parse_leading_json(result.stdout)
|
||||
assert doc is not None, (
|
||||
"%s %s printed no parseable JSON document:\n%s"
|
||||
% (name, " ".join(args), result.output[-2000:]))
|
||||
missing = [k for k in JSON_REQUIRED_KEYS if not isinstance(doc, dict) or k not in doc]
|
||||
if missing:
|
||||
present = sorted(doc) if isinstance(doc, dict) else type(doc).__name__
|
||||
pytest.xfail("%s --json does not carry %s; top-level keys are %s"
|
||||
% (name, missing, present))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_json_capable_but_unreachable_scripts_are_named(name, help_results):
|
||||
"""Coverage honesty: a --json script is either exercised here or explicitly not."""
|
||||
if not has_option(help_results[name].output, "--json"):
|
||||
pytest.skip("%s does not advertise --json" % name)
|
||||
if name in JSON_RUNNABLE:
|
||||
return
|
||||
pytest.skip("%s advertises --json, but reaching that path needs a device or a "
|
||||
"sample; the gap is listed in EXTENSION-test-harness.md" % name)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 5. the exit-code vocabulary
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestExitCodeVocabulary:
|
||||
@pytest.mark.parametrize("name", SCRIPTS)
|
||||
def test_observed_codes_stay_inside_the_vocabulary(self, name, help_results,
|
||||
badflag_results):
|
||||
for label, result in (("--help", help_results[name]),
|
||||
(BAD_FLAG, badflag_results[name])):
|
||||
if result.returncode is not None:
|
||||
assert result.returncode in CONTRACT_EXIT_CODES, (
|
||||
"%s %s returned %r" % (name, label, result.returncode))
|
||||
assert result.returncode is None or result.returncode >= 0, (
|
||||
"a negative code is a crash signal, not an exit status: %s %s"
|
||||
% (name, label))
|
||||
@@ -0,0 +1,174 @@
|
||||
"""Shared fixtures for the apk-reverse regression suite.
|
||||
|
||||
The rules this directory follows (they are the point of the directory, not a
|
||||
style preference):
|
||||
|
||||
* Nothing here may need a device, adb, frida, or the network. Routes that do
|
||||
are recorded as experiments in ``tests/benchmark.md``; their coverage gap is
|
||||
stated in ``docs/tool-verification/EXTENSION-test-harness.md`` rather than
|
||||
simulated into a green tick.
|
||||
* A missing fixture is ``pytest.skip`` with the reason on stdout, never a red
|
||||
test. ``tools/_work/bench/**`` is git-ignored, so CI legitimately does not
|
||||
have it and the regression hash test must degrade to a skip there.
|
||||
* This suite observes behaviour. It never edits anything under
|
||||
``skills/apk-reverse/scripts/`` -- a defect found here is written into the
|
||||
report as a work item, not patched in place.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib.util
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
SCRIPTS_DIR = REPO_ROOT / "skills" / "apk-reverse" / "scripts"
|
||||
FIXTURES_DIR = Path(__file__).resolve().parent / "fixtures"
|
||||
BENCH_L1L3 = REPO_ROOT / "tools" / "_work" / "bench" / "l1l3"
|
||||
|
||||
# Some scripts import their siblings by bare name (`from dexutil import ...`);
|
||||
# loading them as modules requires that directory on sys.path. The suite's own
|
||||
# directory is added too, so `from conftest import load_script` works regardless
|
||||
# of how pytest resolves the rootdir.
|
||||
for _p in (SCRIPTS_DIR, Path(__file__).resolve().parent):
|
||||
if str(_p) not in sys.path:
|
||||
sys.path.insert(0, str(_p))
|
||||
|
||||
PYTHON = sys.executable
|
||||
|
||||
# Children run with this as their working directory, never the scripts directory.
|
||||
# Measured reason: `coldstart.py` writes its capture under `--out` (default
|
||||
# `coldstart/`), so a contract test that ran it with the scripts directory as cwd
|
||||
# left a `skills/apk-reverse/scripts/coldstart/logcat.txt` behind. This suite must
|
||||
# not write anywhere near the shipped tree, and a script that defaults an output
|
||||
# path relative to cwd is exactly the case that catches a harness out.
|
||||
ISOLATED_CWD = tempfile.mkdtemp(prefix="apkrev-suite-cwd-")
|
||||
|
||||
# The regression marker established before this harness existed. It is a hash,
|
||||
# not a golden-file rewrite: scripts/dex_patch_bytes.py applied to the
|
||||
# l1l3 fixture must still reproduce the file that the recorded benchmark row
|
||||
# produced.
|
||||
L1L3_PATCHED_SHA256 = "67ae1e58e7a44777e5ac8f9f823ab21e"
|
||||
|
||||
_loaded: dict[str, object] = {}
|
||||
|
||||
|
||||
class ScriptResult:
|
||||
"""Result of running one script as a subprocess."""
|
||||
|
||||
def __init__(self, argv, returncode, stdout, stderr, timed_out=False):
|
||||
self.argv = argv
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
self.timed_out = timed_out
|
||||
|
||||
@property
|
||||
def output(self) -> str:
|
||||
return self.stdout + self.stderr
|
||||
|
||||
@property
|
||||
def has_traceback(self) -> bool:
|
||||
return "Traceback (most recent call last)" in self.output
|
||||
|
||||
def __repr__(self) -> str: # keeps assertion failures readable
|
||||
return "ScriptResult(argv=%r, rc=%r, timed_out=%r)" % (
|
||||
self.argv, self.returncode, self.timed_out)
|
||||
|
||||
|
||||
def script_path(name: str) -> Path:
|
||||
return SCRIPTS_DIR / name
|
||||
|
||||
|
||||
def all_scripts() -> list[str]:
|
||||
"""Every shipped script, sorted. Used to parameterise the CLI contract tests."""
|
||||
return sorted(p.name for p in SCRIPTS_DIR.glob("*.py"))
|
||||
|
||||
|
||||
def run_script(name, args=(), timeout=60, cwd=None, stdin=None, env=None):
|
||||
"""Run one shipped script in a subprocess and capture everything.
|
||||
|
||||
The child runs with ``PYTHONIOENCODING=utf-8`` so that a non-UTF-8 console
|
||||
(a Windows CP936 shell, a CI runner with C locale) does not turn a UTF-8
|
||||
print into a UnicodeEncodeError and change the exit code under test, and with
|
||||
an isolated working directory so that a script defaulting an output path
|
||||
relative to cwd cannot write into the shipped tree.
|
||||
"""
|
||||
argv = [PYTHON, str(script_path(name))] + [str(a) for a in args]
|
||||
child_env = dict(os.environ)
|
||||
child_env["PYTHONIOENCODING"] = "utf-8"
|
||||
child_env["PYTHONDONTWRITEBYTECODE"] = "1"
|
||||
if env:
|
||||
child_env.update(env)
|
||||
try:
|
||||
proc = subprocess.run(
|
||||
argv, capture_output=True, text=True, encoding="utf-8", errors="replace",
|
||||
timeout=timeout, cwd=str(cwd or ISOLATED_CWD), input=stdin, env=child_env,
|
||||
)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
return ScriptResult(argv, None, _text(exc.stdout), _text(exc.stderr),
|
||||
timed_out=True)
|
||||
return ScriptResult(argv, proc.returncode, proc.stdout or "", proc.stderr or "")
|
||||
|
||||
|
||||
def _text(blob):
|
||||
if blob is None:
|
||||
return ""
|
||||
if isinstance(blob, bytes):
|
||||
return blob.decode("utf-8", "replace")
|
||||
return blob
|
||||
|
||||
|
||||
def load_script(name):
|
||||
"""Import a shipped script as a module, so pure functions can be unit-tested.
|
||||
|
||||
The module is registered under a namespaced name (``apkrev_<stem>``) because
|
||||
several scripts have siblings with the same basename in other trees.
|
||||
"""
|
||||
if name in _loaded:
|
||||
return _loaded[name]
|
||||
path = script_path(name)
|
||||
mod_name = "apkrev_" + path.stem
|
||||
spec = importlib.util.spec_from_file_location(mod_name, str(path))
|
||||
if spec is None or spec.loader is None:
|
||||
raise ImportError("cannot load %s" % path)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[mod_name] = module
|
||||
spec.loader.exec_module(module)
|
||||
_loaded[name] = module
|
||||
return module
|
||||
|
||||
|
||||
def require_path(path: Path, why: str):
|
||||
"""Skip with a printed reason when a git-ignored fixture is absent."""
|
||||
if not Path(path).exists():
|
||||
pytest.skip("fixture absent: %s (%s)" % (path, why))
|
||||
return Path(path)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def repo_root():
|
||||
return REPO_ROOT
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def scripts_dir():
|
||||
return SCRIPTS_DIR
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def fixtures_dir():
|
||||
return FIXTURES_DIR
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def bench_l1l3():
|
||||
return require_path(
|
||||
BENCH_L1L3,
|
||||
"tools/_work/bench is git-ignored; re-download the l1l3 sample to run the "
|
||||
"dex_patch_bytes regression hash test",
|
||||
)
|
||||
@@ -0,0 +1,9 @@
|
||||
# The APKs in this directory are committed on purpose.
|
||||
#
|
||||
# They are synthesised by ../make_fixture_apk.py (a few kilobytes each, no target data)
|
||||
# and they are the input of tests/test_so_constpatch.py, which pins the contract that
|
||||
# scripts/so_constpatch.py must keep when it rebuilds an archive: STORED stays STORED,
|
||||
# extra / external_attr / create_system survive, and resources.arsc keeps its 4-byte
|
||||
# alignment. The repository-wide `*.apk` rule exists to keep pulled samples out of the
|
||||
# tree, which is a different thing from a generated fixture.
|
||||
!*.apk
|
||||
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
BIN
Binary file not shown.
Vendored
+11
@@ -0,0 +1,11 @@
|
||||
--------- beginning of main
|
||||
01-01 10:00:01.100 1234 1234 I ActivityManager: Start proc 1234:com.example.app/u0a123 for activity
|
||||
01-01 10:00:01.500 1234 1234 D AndroidRuntime: Shutting down VM
|
||||
01-01 10:00:01.501 1234 1234 E AndroidRuntime: FATAL EXCEPTION: main
|
||||
01-01 10:00:01.501 1234 1234 E AndroidRuntime: Process: com.example.app, PID: 1234
|
||||
01-01 10:00:01.502 1234 1234 E AndroidRuntime: java.lang.VerifyError: Rejecting class Lprobe/Synthetic;
|
||||
01-01 10:00:01.503 1234 1234 W dalvikvm: Failure to verify dex file: Bad checksum (12345678, expected 9abcdef0)
|
||||
01-01 10:00:01.504 1234 1234 E AndroidRuntime: Caused by: java.lang.UnsatisfiedLinkError: dlopen failed
|
||||
01-01 10:00:02.000 1000 1500 I ActivityManager: Displayed com.example.app/.MainActivity: +842ms
|
||||
01-01 10:00:03.100 1000 1500 W ActivityManager: ANR in com.example.app (com.example.app/.MainActivity)
|
||||
01-01 10:00:03.200 1000 1500 E ActivityManager: Reason: Input dispatching timed out
|
||||
Vendored
+369
@@ -0,0 +1,369 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Build the four tiny APKs that pin the APK-rebuild contract of `so_constpatch.py`.
|
||||
|
||||
WHY THIS EXISTS
|
||||
---------------
|
||||
`so_constpatch.py` rebuilt an APK through `zipfile.ZipInfo(filename, date_time)`, which
|
||||
drops `extra`, `external_attr`, `create_system` and re-deflates every entry. An entry
|
||||
that must stay STORED or 4-byte aligned (`resources.arsc`, an uncompressed `lib/*.so`)
|
||||
can therefore come back compressed or unaligned, and the result is refused by the
|
||||
installer with `[-124]` -- or simply not loadable when the manifest asks for
|
||||
`extractNativeLibs="false"`. A regression test for that fix needs inputs that actually
|
||||
carry the metadata which was being dropped, at a size worth keeping in the repository:
|
||||
|
||||
plain.apk DEFLATED entries, no `resources.arsc` -> an unsafe rebuild is not
|
||||
in play, so this is the "rebuild is allowed" control
|
||||
arsc_aligned.apk `resources.arsc` STORED and 4-byte aligned, plus a STORED and a
|
||||
DEFLATED `lib/*.so` -> the storage/alignment contract must survive
|
||||
noextract.apk the manifest declares `android:extractNativeLibs="false"` and both
|
||||
`resources.arsc` and `lib/*.so` are STORED+aligned -> a rebuild must
|
||||
be REFUSED unless `--unsafe-rebuild` is passed
|
||||
services.apk `META-INF/services/**` entries, a UTF-8 entry name and a non-empty
|
||||
archive comment -> untouched entries must come back byte-identical
|
||||
|
||||
Nothing here is target data: every fixture is synthesised from the constants below. Two
|
||||
details exist so the fixtures are not weaker than real input:
|
||||
|
||||
* `AndroidManifest.xml` is a real binary AXML document (string pool + start element), so
|
||||
the manifest check reads the same bytes a compiler-produced manifest would give it.
|
||||
* `resources.arsc` is a valid `RES_TABLE_TYPE` header plus filler: enough to pin the
|
||||
storage/alignment contract, not a renderable resource table.
|
||||
* the `lib/*.so` payload is a real (tiny) aarch64 shared object with `.rodata` and
|
||||
`.shstrtab` section headers, so `--section-aware` exercises the section map instead of
|
||||
falling back to "no sections".
|
||||
|
||||
USAGE
|
||||
-----
|
||||
python make_fixture_apk.py --out tests/fixtures/apk
|
||||
|
||||
The generated files are committed. Regenerate only when the contract itself changes;
|
||||
the output is deterministic (fixed timestamps, fixed compression level, no randomness).
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import os
|
||||
import struct
|
||||
import zlib
|
||||
|
||||
ALIGN = 4
|
||||
STAMP_UNIX = 1577836800 # 2020-01-01T00:00:00Z
|
||||
FORCE_UTF8 = 0x800 # general purpose bit 11
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ small builders
|
||||
|
||||
def _dos(dt):
|
||||
"""(time, date) MS-DOS words, the way a zip local header stores them."""
|
||||
y, mo, d, h, mi, s = dt
|
||||
return (h << 11) | (mi << 5) | (s // 2), ((y - 1980) << 9) | (mo << 5) | d
|
||||
|
||||
|
||||
def _deflate(data):
|
||||
"""Raw deflate (no zlib wrapper) -- what a method-8 zip entry holds."""
|
||||
c = zlib.compressobj(9, zlib.DEFLATED, -15)
|
||||
return c.compress(data) + c.flush()
|
||||
|
||||
|
||||
def ext_stamp(unix=STAMP_UNIX):
|
||||
"""An extended-timestamp extra field (id 0x5455), so `extra` is never empty.
|
||||
|
||||
Real APKs carry this on entries written by a toolchain that stamps mtimes, and it is
|
||||
exactly the field a rebuilt archive used to lose.
|
||||
"""
|
||||
return b'\x55\x54' + struct.pack('<H', 5) + b'\x01' + struct.pack('<I', unix)
|
||||
|
||||
|
||||
def align_record(pad):
|
||||
"""A private extra record of `4 + pad` bytes, i.e. `pad` bytes of alignment.
|
||||
|
||||
A zip extra area is a sequence of (id, 2-byte size, payload) records, so the smallest
|
||||
record is 4 bytes and a 1-3 byte pad cannot be expressed on its own. Appending a
|
||||
record of length `4 + pad` shifts the data offset by `pad` (mod 4), which is exactly
|
||||
what is needed, without inserting a filler entry.
|
||||
"""
|
||||
if pad <= 0:
|
||||
return b''
|
||||
return b'\xca\xca' + struct.pack('<H', pad) + b'\x00' * pad
|
||||
|
||||
|
||||
def make_elf(const=b'apkhuan', decoy=b'libapkhuanx.so'):
|
||||
"""A minimal aarch64 ET_DYN object: ELF header, `.rodata`, `.shstrtab`, shdrs.
|
||||
|
||||
`const` sits alone between NULs (an isolated constant, safe to rewrite in place);
|
||||
`decoy` contains the same bytes as a substring, which is the case `--find` must
|
||||
report as NOT isolated.
|
||||
"""
|
||||
rodata = b'\x00' + const + b'\x00' + decoy + b'\x00'
|
||||
shstr = b'\x00.rodata\x00.shstrtab\x00'
|
||||
|
||||
ehsize, shentsize, shnum, shstrndx = 64, 64, 3, 2
|
||||
ro_off = ehsize
|
||||
shstr_off = ro_off + len(rodata)
|
||||
sh_off = (shstr_off + len(shstr) + 7) & ~7
|
||||
|
||||
ident = b'\x7fELF' + bytes([2, 1, 1, 0]) + b'\x00' * 8
|
||||
ehdr = ident + struct.pack('<HHIQQQIHHHHHH',
|
||||
3, # e_type = ET_DYN
|
||||
183, # e_machine= EM_AARCH64
|
||||
1, # e_version
|
||||
0, 0, # e_entry, e_phoff
|
||||
sh_off, # e_shoff
|
||||
0, # e_flags
|
||||
ehsize, 0, 0, shentsize, shnum, shstrndx)
|
||||
|
||||
def shdr(name, typ, flags, addr, off, size, align=1):
|
||||
return struct.pack('<IIQQQQIIQQ', name, typ, flags, addr, off, size,
|
||||
0, 0, align, 0)
|
||||
|
||||
shdrs = (shdr(0, 0, 0, 0, 0, 0)
|
||||
+ shdr(1, 1, 0x2, 0x1000, ro_off, len(rodata)) # .rodata PROGBITS
|
||||
+ shdr(9, 3, 0, 0, shstr_off, len(shstr))) # .shstrtab STRTAB
|
||||
pad = b'\x00' * (sh_off - (shstr_off + len(shstr)))
|
||||
return ehdr + rodata + shstr + pad + shdrs
|
||||
|
||||
|
||||
def _string_pool(strings):
|
||||
"""A UTF-8 (flag 0x100) RES_STRING_POOL chunk."""
|
||||
offsets, blob = [], b''
|
||||
for s in strings:
|
||||
raw = s.encode('utf-8')
|
||||
assert len(raw) < 0x80 and len(s) < 0x80, 'fixture pool is ASCII/short on purpose'
|
||||
offsets.append(len(blob))
|
||||
blob += bytes([len(raw), len(s)]) + raw + b'\x00'
|
||||
strings_start = 28 + 4 * len(strings)
|
||||
header = struct.pack('<HHI', 0x0001, 28, strings_start + len(blob))
|
||||
header += struct.pack('<IIIII', len(strings), 0, 0x100, strings_start, 0)
|
||||
return header + b''.join(struct.pack('<I', o) for o in offsets) + blob
|
||||
|
||||
|
||||
def make_axml(package='com.example.fixture', extract_native_libs=None):
|
||||
"""A minimal but structurally valid binary AndroidManifest.xml.
|
||||
|
||||
Layout: file header, one string pool, one START_ELEMENT chunk for <manifest>. The
|
||||
attribute shape matches what aapt2 emits: `extractNativeLibs` is a boolean
|
||||
(dataType 0x12) with no raw string, namespaced to the android URI.
|
||||
"""
|
||||
android_uri = 'http://schemas.android.com/apk/res/android'
|
||||
strings = ['android', android_uri, 'manifest', 'package', 'extractNativeLibs', package]
|
||||
s = {v: i for i, v in enumerate(strings)}
|
||||
|
||||
attrs = [(0xFFFFFFFF, s['package'], 0xFFFFFFFF, 0x03, s[package])]
|
||||
if extract_native_libs is not None:
|
||||
attrs.append((s[android_uri], s['extractNativeLibs'], 0xFFFFFFFF, 0x12,
|
||||
1 if extract_native_libs else 0))
|
||||
|
||||
pool = _string_pool(strings)
|
||||
|
||||
def chunk(ctype, node, tail):
|
||||
# chunk size covers the 8-byte chunk header as well as `node` and `tail`.
|
||||
body = node + tail
|
||||
return struct.pack('<HHI', ctype, 16, 8 + len(body)) + body
|
||||
|
||||
node = struct.pack('<II', 1, 0xFFFFFFFF) # lineNumber, comment
|
||||
body = struct.pack('<II', 0xFFFFFFFF, s['manifest']) # ns, name
|
||||
body += struct.pack('<HHHHHH', 20, 20, len(attrs), 0, 0, 0)
|
||||
for ns, name, raw, dtype, data in attrs:
|
||||
body += struct.pack('<III', ns, name, raw)
|
||||
body += struct.pack('<HBBI', 8, 0, dtype, data)
|
||||
element = chunk(0x0102, node, body)
|
||||
end = chunk(0x0103, node, struct.pack('<II', 0xFFFFFFFF, s['manifest']))
|
||||
|
||||
total = 8 + len(pool) + len(element) + len(end)
|
||||
return struct.pack('<HHI', 0x0003, 8, total) + pool + element + end
|
||||
|
||||
|
||||
def make_arsc(filler=60):
|
||||
"""A `RES_TABLE_TYPE` header with zero packages, plus filler bytes."""
|
||||
body = struct.pack('<I', 0) + b'\x00' * filler
|
||||
return struct.pack('<HHI', 0x0002, 12, 12 + len(body)) + body
|
||||
|
||||
|
||||
def make_dex(classes=2):
|
||||
"""A dex header whose file_size matches, so a reader does not reject it outright.
|
||||
|
||||
Only the header is written: the fixtures exist to pin zip behaviour, not dex
|
||||
semantics, and a full dex would be the largest file in the repository's test tree.
|
||||
"""
|
||||
header_size, endian = 0x70, 0x12345678
|
||||
size = header_size + 4 * classes
|
||||
h = bytearray(header_size)
|
||||
h[0:8] = b'dex\n035\x00'
|
||||
h[32:36] = struct.pack('<I', size)
|
||||
h[36:40] = struct.pack('<I', header_size)
|
||||
h[40:44] = struct.pack('<I', endian)
|
||||
h[104:108] = struct.pack('<I', 0) # data_size
|
||||
h[108:112] = struct.pack('<I', header_size)
|
||||
return bytes(h) + b'\x00' * (size - header_size)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ zip writer
|
||||
|
||||
class Entry(object):
|
||||
"""One fixture entry, with the metadata the rebuild must carry over."""
|
||||
|
||||
def __init__(self, name, data, method=8, date_time=(2020, 1, 1, 0, 0, 0),
|
||||
extra=None, external_attr=0o100644 << 16, internal_attr=0,
|
||||
create_system=3, comment=b'', align=False, utf8=False):
|
||||
self.name = name
|
||||
self.data = data
|
||||
self.method = method
|
||||
self.date_time = date_time
|
||||
self.extra = ext_stamp() if extra is None else extra
|
||||
self.external_attr = external_attr
|
||||
self.internal_attr = internal_attr
|
||||
self.create_system = create_system
|
||||
self.comment = comment
|
||||
self.align = align
|
||||
self.utf8 = utf8
|
||||
|
||||
@property
|
||||
def flags(self):
|
||||
return FORCE_UTF8 if self.utf8 else 0
|
||||
|
||||
|
||||
def write_apk(path, entries, archive_comment=b''):
|
||||
"""Write a zip the way zipalign does: STORED stays STORED, aligned stays aligned.
|
||||
|
||||
Written by hand rather than through `zipfile` because alignment is a property of the
|
||||
local header length, which `zipfile` cannot express -- the same reason `repack.py`
|
||||
hand-rolls its writer.
|
||||
"""
|
||||
out = bytearray()
|
||||
cd = []
|
||||
for e in entries:
|
||||
name_b = e.name.encode('utf-8')
|
||||
payload = e.data if e.method == 0 else _deflate(e.data)
|
||||
crc = zlib.crc32(e.data) & 0xFFFFFFFF
|
||||
extra = e.extra
|
||||
if e.align:
|
||||
head = len(out) + 30 + len(name_b) + len(extra)
|
||||
extra += align_record((ALIGN - head % ALIGN) % ALIGN)
|
||||
t, d = _dos(e.date_time)
|
||||
local_off = len(out)
|
||||
out += struct.pack('<IHHHHHIIIHH', 0x04034B50, 20, e.flags, e.method, t, d,
|
||||
crc, len(payload), len(e.data), len(name_b), len(extra))
|
||||
out += name_b + extra + payload
|
||||
data_off = local_off + 30 + len(name_b) + len(extra)
|
||||
if e.align and data_off % ALIGN:
|
||||
raise RuntimeError('%s is not %d-byte aligned' % (e.name, ALIGN))
|
||||
cd.append((e, name_b, crc, len(payload), len(extra), local_off))
|
||||
|
||||
cd_off = len(out)
|
||||
for e, name_b, crc, csize, elen, local_off in cd:
|
||||
t, d = _dos(e.date_time)
|
||||
out += struct.pack('<IHHHHHHIIIHHHHHII', 0x02014B50,
|
||||
(e.create_system << 8) | 20, 20, e.flags, e.method, t, d,
|
||||
crc, csize, len(e.data), len(name_b), len(e.extra),
|
||||
len(e.comment), 0, e.internal_attr, e.external_attr,
|
||||
local_off)
|
||||
out += name_b + e.extra + e.comment
|
||||
cd_size = len(out) - cd_off
|
||||
out += struct.pack('<IHHHHIIH', 0x06054B50, 0, 0, len(cd), len(cd), cd_size,
|
||||
cd_off, len(archive_comment))
|
||||
out += archive_comment
|
||||
|
||||
folder = os.path.dirname(os.path.abspath(path))
|
||||
if folder:
|
||||
os.makedirs(folder, exist_ok=True)
|
||||
with open(path, 'wb') as fh:
|
||||
fh.write(bytes(out))
|
||||
return bytes(out)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the fixtures
|
||||
|
||||
def definitions():
|
||||
"""name -> (entries, archive comment). Deterministic; no timestamps from the clock.
|
||||
|
||||
Every STORED entry is 4-byte aligned, the way `zipalign` lays a real APK out: that is
|
||||
what makes "the rebuild did not break alignment" a checkable claim, and what keeps a
|
||||
rebuilt fixture acceptable to `zipalign -c`.
|
||||
"""
|
||||
elf = make_elf()
|
||||
dex = make_dex()
|
||||
strings_xml = b'<?xml version="1.0" encoding="utf-8"?>\n<resources/>\n'
|
||||
plain_manifest = make_axml('com.example.plain')
|
||||
noextract_manifest = make_axml('com.example.noextract', extract_native_libs=False)
|
||||
|
||||
st = dict(method=0, align=True, date_time=(2021, 6, 15, 12, 34, 56))
|
||||
plain = [
|
||||
Entry('AndroidManifest.xml', plain_manifest, **st),
|
||||
Entry('classes.dex', dex, external_attr=0o100600 << 16),
|
||||
Entry('res/values/strings.xml', strings_xml),
|
||||
Entry('lib/arm64-v8a/libfoo.so', elf),
|
||||
Entry('META-INF/MANIFEST.MF', b'Manifest-Version: 1.0\r\n\r\n'),
|
||||
]
|
||||
|
||||
arsc_aligned = [
|
||||
Entry('AndroidManifest.xml', plain_manifest, **st),
|
||||
Entry('classes.dex', dex),
|
||||
# DEFLATED and patched: its compressed size moves, so resources.arsc has to be
|
||||
# re-aligned on output. This is the case the old writer silently broke.
|
||||
Entry('lib/arm64-v8a/libfoo.so', elf),
|
||||
# STORED: must stay STORED, must not be re-compressed, must stay aligned.
|
||||
Entry('lib/arm64-v8a/libbar.so', elf, method=0, align=True,
|
||||
external_attr=0o100755 << 16),
|
||||
Entry('resources.arsc', make_arsc(), method=0, align=True),
|
||||
Entry('res/values/strings.xml', strings_xml),
|
||||
Entry('META-INF/services/com.example.Provider',
|
||||
b'com.example.ProviderImpl\n'),
|
||||
]
|
||||
|
||||
noextract = [
|
||||
Entry('AndroidManifest.xml', noextract_manifest, **st),
|
||||
Entry('classes.dex', dex),
|
||||
Entry('lib/arm64-v8a/libfoo.so', elf, method=0, align=True),
|
||||
Entry('resources.arsc', make_arsc(), method=0, align=True),
|
||||
]
|
||||
|
||||
services = [
|
||||
Entry('AndroidManifest.xml', plain_manifest, **st),
|
||||
Entry('META-INF/services/com.example.Provider', b'com.example.ProviderImpl\n'),
|
||||
Entry('META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory',
|
||||
b'kotlinx.coroutines.internal.MainDispatcherFactoryImpl\n', method=0,
|
||||
align=True),
|
||||
Entry('lib/x86_64/libfoo.so', elf, external_attr=0),
|
||||
# A non-ASCII name with the UTF-8 flag (bit 11) set: the raw name bytes and the
|
||||
# flag must both survive a rebuild, and this entry must not be touched at all.
|
||||
Entry('res/raw/\u6d4b\u8bd5.bin', b'fixture-payload\x00', utf8=True,
|
||||
create_system=0, external_attr=0, internal_attr=1,
|
||||
comment=b'fixture comment'),
|
||||
]
|
||||
|
||||
return {
|
||||
'plain.apk': (plain, b''),
|
||||
'arsc_aligned.apk': (arsc_aligned, b''),
|
||||
'noextract.apk': (noextract, b''),
|
||||
'services.apk': (services, b'fixture-archive-comment'),
|
||||
}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser(description=__doc__.split('\n')[0])
|
||||
ap.add_argument('--out', default=os.path.join(os.path.dirname(os.path.abspath(__file__)),
|
||||
'apk'),
|
||||
help='directory to write the fixtures into')
|
||||
ap.add_argument('--list', action='store_true', help='print names and exit')
|
||||
a = ap.parse_args()
|
||||
|
||||
defs = definitions()
|
||||
if a.list:
|
||||
for name in sorted(defs):
|
||||
print(name)
|
||||
return 0
|
||||
|
||||
for name in sorted(defs):
|
||||
entries, comment = defs[name]
|
||||
path = os.path.join(a.out, name)
|
||||
blob = write_apk(path, entries, comment)
|
||||
print('%-18s %6d bytes %2d entries sha256=%s'
|
||||
% (name, len(blob), len(entries), hashlib.sha256(blob).hexdigest()[:16]))
|
||||
print('wrote %d fixture(s) to %s' % (len(defs), os.path.abspath(a.out)))
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
raise SystemExit(main())
|
||||
Vendored
+20
@@ -0,0 +1,20 @@
|
||||
*** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
|
||||
Build fingerprint: 'google/sdk_gphone64_arm64/sdk_gphone64_arm64:14/UE1A.230829.036/11228888:userdebug/dev-keys'
|
||||
Revision: '0'
|
||||
ABI: 'arm64'
|
||||
Timestamp: 2026-01-01 10:00:01.500000000+0000
|
||||
Process uptime: 5
|
||||
Cmdline: com.example.app
|
||||
pid: 12345, tid: 12345, name: com.example.app >>> com.example.app <<<
|
||||
uid: 10123
|
||||
signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x0000000000000010
|
||||
x0 0000000000000010 x1 0000007a12345678 x2 0000000000000000 x3 0000000000000020
|
||||
x4 0000007a98765432 x5 0000000000000001 x6 0000000000000000 x7 0000000000000008
|
||||
x8 0000000000000098 x9 0000000000000000 x10 0000007ffabc1234 x11 0000000000000000
|
||||
sp 0000007ffabc1110 lr 0000007a11112222 pc 0000007a111128cc pst 0000000060001000
|
||||
backtrace:
|
||||
#00 pc 00000000000128cc /data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so
|
||||
#01 pc 0000000000020f40 /data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so (Java_com_example_app_Native_check+120)
|
||||
#02 pc 00000000000cccd0 /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start+256)
|
||||
#03 pc 0000000000055488 /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68)
|
||||
Cause: null pointer dereference? no -- read of a small integer address
|
||||
@@ -0,0 +1,334 @@
|
||||
"""Device-side command construction, exercised without a device.
|
||||
|
||||
The behaviour under test is the one the repository's own documentation says is
|
||||
the reason these helpers exist: a device command is built as an **argv list** and
|
||||
handed to `subprocess` directly, so the host shell never expands `$`, `|`, `>`,
|
||||
`;` or quotes before adb sees them. Every assertion here inspects the argv list
|
||||
that would have been executed; none of them launches adb.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script
|
||||
|
||||
devsh = load_script("devsh.py")
|
||||
coldstart = load_script("coldstart.py")
|
||||
install_test = load_script("install_test.py")
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
|
||||
class Recorder:
|
||||
"""Captures the argv lists handed to a patched subprocess runner."""
|
||||
|
||||
def __init__(self, stdout="", stderr="", returncode=0):
|
||||
self.calls = []
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
self.returncode = returncode
|
||||
|
||||
def __call__(self, cmd, *args, **kwargs):
|
||||
self.calls.append((list(cmd), kwargs))
|
||||
return subprocess.CompletedProcess(cmd, self.returncode, self.stdout, self.stderr)
|
||||
|
||||
@property
|
||||
def last(self):
|
||||
return self.calls[-1][0]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# devsh.py
|
||||
# --------------------------------------------------------------------------- #
|
||||
@pytest.fixture
|
||||
def devsh_recorder(monkeypatch):
|
||||
rec = Recorder(stdout="ok\n")
|
||||
monkeypatch.setattr(devsh.subprocess, "run", rec)
|
||||
monkeypatch.delenv("ADB_PATH", raising=False)
|
||||
monkeypatch.delenv("ADB_SERIAL", raising=False)
|
||||
return rec
|
||||
|
||||
|
||||
def run_devsh(monkeypatch, rec, argv):
|
||||
monkeypatch.setattr(sys, "argv", ["devsh.py"] + argv)
|
||||
return devsh.main()
|
||||
|
||||
|
||||
class TestDevshCommandConstruction:
|
||||
def test_shell_command_is_one_argv_element_not_a_shell_string(self, monkeypatch, devsh_recorder):
|
||||
"""`|` and `grep` must survive as literal text inside a single argument."""
|
||||
assert run_devsh(monkeypatch, devsh_recorder, ["sh", "pm list packages | grep myapp"]) == 0
|
||||
assert devsh_recorder.last == ["adb", "shell", "pm list packages | grep myapp"]
|
||||
|
||||
def test_serial_is_injected_as_dash_s(self, monkeypatch, devsh_recorder):
|
||||
run_devsh(monkeypatch, devsh_recorder, ["--serial", "SERIAL123", "sh", "id"])
|
||||
assert devsh_recorder.last == ["adb", "-s", "SERIAL123", "shell", "id"]
|
||||
|
||||
def test_serial_equals_form_is_accepted(self, monkeypatch, devsh_recorder):
|
||||
run_devsh(monkeypatch, devsh_recorder, ["--adb=x", "--serial=SERIAL123", "sh", "id"])
|
||||
assert devsh_recorder.last == ["x", "-s", "SERIAL123", "shell", "id"]
|
||||
|
||||
def test_adb_path_comes_from_the_environment_when_not_given(self, monkeypatch, devsh_recorder):
|
||||
monkeypatch.setenv("ADB_PATH", "/opt/platform-tools/adb")
|
||||
monkeypatch.setenv("ADB_SERIAL", "ENVSERIAL")
|
||||
run_devsh(monkeypatch, devsh_recorder, ["sh", "id"])
|
||||
assert devsh_recorder.last == ["/opt/platform-tools/adb", "-s", "ENVSERIAL", "shell", "id"]
|
||||
|
||||
def test_device_listing_explicitly_ignores_the_serial(self, monkeypatch, devsh_recorder):
|
||||
run_devsh(monkeypatch, devsh_recorder, ["dev"])
|
||||
assert devsh_recorder.last == ["adb", "devices", "-l"]
|
||||
|
||||
def test_pull_and_push_keep_their_arguments_positional(self, monkeypatch, devsh_recorder):
|
||||
run_devsh(monkeypatch, devsh_recorder, ["pull", "/data/local/tmp/x.apk", "./x.apk"])
|
||||
assert devsh_recorder.last == ["adb", "pull", "/data/local/tmp/x.apk", "./x.apk"]
|
||||
run_devsh(monkeypatch, devsh_recorder, ["push", "./x.apk", "/data/local/tmp/x.apk"])
|
||||
assert devsh_recorder.last == ["adb", "push", "./x.apk", "/data/local/tmp/x.apk"]
|
||||
|
||||
def test_a_nonzero_device_status_is_reported_as_one(self, monkeypatch, devsh_recorder):
|
||||
devsh_recorder.returncode = 1
|
||||
assert run_devsh(monkeypatch, devsh_recorder, ["sh", "false"]) == 1
|
||||
|
||||
def test_the_su_wrapper_is_posix_single_quoted(self, monkeypatch, devsh_recorder):
|
||||
# Assert the *property*, not the representation. The previous version of this test pinned
|
||||
# `su -c "..."` with `\"` escapes, which is exactly the form that let `;`, `$( )` and
|
||||
# backticks through to the device shell -- the test was guarding the defect.
|
||||
run_devsh(monkeypatch, devsh_recorder, ["su", 'pm grant com.example.app "android.permission.X"'])
|
||||
payload = devsh_recorder.last[-1]
|
||||
assert payload.startswith("su -c '") and payload.endswith("'")
|
||||
inner = payload[len("su -c '"):-1]
|
||||
# A POSIX single-quoted string: no bare quote may appear except as the '\'' escape.
|
||||
assert re.fullmatch(r"[^']*('\\''[^']*)*", inner), payload
|
||||
assert '"' in inner # the caller's double quotes survive as literal characters
|
||||
|
||||
def test_an_unknown_mode_prints_usage_and_exits_two(self, monkeypatch, devsh_recorder):
|
||||
assert run_devsh(monkeypatch, devsh_recorder, ["frobnicate", "x"]) == 2
|
||||
|
||||
def test_no_arguments_prints_usage_and_exits_two(self, monkeypatch, devsh_recorder):
|
||||
assert run_devsh(monkeypatch, devsh_recorder, []) == 2
|
||||
|
||||
|
||||
class TestDevshSuQuotingGap:
|
||||
"""The quoting contract, now unified through `scripts/device_shell.py`.
|
||||
|
||||
This class used to record a *gap*: `devsh.su` wrapped the command in double quotes and escaped
|
||||
only `"`, so `$`, backticks, `;` and `|` from a caller-supplied value still reached the device
|
||||
shell, while `coldstart.Device.shell` used the POSIX single-quote escape and did neutralise them.
|
||||
Both now go through the shared module; the tests below pin the property from both sides, so a
|
||||
future edit that reintroduces double-quote escaping fails here instead of on a device.
|
||||
"""
|
||||
|
||||
@pytest.mark.parametrize("payload", [
|
||||
"id; rm -rf /sdcard/x",
|
||||
"id$(touch /data/local/tmp/pwned)",
|
||||
"id`touch /data/local/tmp/pwned`",
|
||||
"id | tee /data/local/tmp/pwned",
|
||||
"id 'quoted'",
|
||||
])
|
||||
def test_su_wrapper_neutralises_shell_metacharacters(self, monkeypatch, devsh_recorder, payload):
|
||||
run_devsh(monkeypatch, devsh_recorder, ["su", payload])
|
||||
wrapped = devsh_recorder.last[-1]
|
||||
assert wrapped.startswith("su -c '"), wrapped
|
||||
inner = wrapped[len("su -c '"):-1]
|
||||
# The payload must be present verbatim, with only the '\'' sequence able to break out --
|
||||
# i.e. the whole payload is one argument to sh, not syntax.
|
||||
assert re.fullmatch(r"[^']*('\\''[^']*)*", inner), wrapped
|
||||
assert payload.replace("'", "'\\''") == inner
|
||||
|
||||
def test_a_malformed_path_is_refused_before_the_device_shell(self, monkeypatch, devsh_recorder):
|
||||
"""`suf` interpolates a path, so it validates rather than quotes-and-hopes."""
|
||||
rc = run_devsh(monkeypatch, devsh_recorder, ["suf", "/data/local/tmp/x;id", "200"])
|
||||
assert rc == 2
|
||||
assert not devsh_recorder.calls, "the refused command must never reach adb"
|
||||
|
||||
def test_coldstart_uses_the_posix_single_quote_escape(self, monkeypatch):
|
||||
"""The contrasting implementation: this one does neutralise metacharacters."""
|
||||
rec = Recorder()
|
||||
monkeypatch.setattr(coldstart, "run", rec)
|
||||
dev = coldstart.Device(None, use_su=False)
|
||||
dev.su = True
|
||||
dev.shell("echo 'hi'; rm -rf /sdcard/x")
|
||||
|
||||
payload = rec.last[-1]
|
||||
assert payload.startswith("su -c '")
|
||||
assert payload.endswith("'")
|
||||
assert "'\\''" in payload, "the POSIX single-quote escape is missing"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# coldstart.Device: command shape and screenshot handling
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestDeviceHelperCommands:
|
||||
def make_device(self, monkeypatch, serial=None):
|
||||
rec = Recorder()
|
||||
monkeypatch.setattr(coldstart, "run", rec)
|
||||
dev = coldstart.Device(serial, use_su=False)
|
||||
return dev, rec
|
||||
|
||||
def test_shell_without_root_passes_the_command_as_one_argument(self, monkeypatch):
|
||||
dev, rec = self.make_device(monkeypatch, serial="SERIAL123")
|
||||
dev.shell("dumpsys window | grep mCurrentFocus")
|
||||
assert rec.last == ["adb", "-s", "SERIAL123", "shell", "dumpsys window | grep mCurrentFocus"]
|
||||
|
||||
def test_shell_without_a_serial_omits_dash_s(self, monkeypatch):
|
||||
dev, rec = self.make_device(monkeypatch)
|
||||
dev.shell("id")
|
||||
assert rec.last == ["adb", "shell", "id"]
|
||||
|
||||
def test_foreground_activity_parses_the_resumed_component(self, monkeypatch):
|
||||
dev, rec = self.make_device(monkeypatch, serial="S")
|
||||
rec.stdout = ("mResumedActivity: ActivityRecord{9f2c1 u0 "
|
||||
"com.example.app/.MainActivity t42}\n")
|
||||
assert coldstart.foreground_activity(dev) == "com.example.app/.MainActivity"
|
||||
|
||||
def test_foreground_activity_returns_none_when_absent(self, monkeypatch):
|
||||
dev, rec = self.make_device(monkeypatch, serial="S")
|
||||
rec.stdout = "no matches\n"
|
||||
assert coldstart.foreground_activity(dev) is None
|
||||
|
||||
def test_installed_facts_extracts_version_and_uid(self, monkeypatch):
|
||||
dev, rec = self.make_device(monkeypatch, serial="S")
|
||||
|
||||
def respond(cmd, *args, **kwargs):
|
||||
rec.calls.append((list(cmd), kwargs))
|
||||
if "versionCode" in cmd[-1]:
|
||||
out = " versionCode=42 minSdk=24 targetSdk=34\n versionName=1.2.3\n"
|
||||
elif "sha256sum" in cmd[-1]:
|
||||
out = "a" * 64 + " /data/app/base.apk\n"
|
||||
elif "base.apk" in cmd[-1]:
|
||||
out = "/data/app/~~x==/com.example.app-1/base.apk\n"
|
||||
elif "userId=" in cmd[-1]:
|
||||
out = " userId=10123\n"
|
||||
else:
|
||||
out = ""
|
||||
return subprocess.CompletedProcess(cmd, 0, out, "")
|
||||
|
||||
monkeypatch.setattr(coldstart, "run", respond)
|
||||
facts = coldstart.installed_facts(dev, "com.example.app")
|
||||
assert facts["versionName"] == "1.2.3"
|
||||
assert facts["uid"] == "10123"
|
||||
assert facts["apk_sha256"] == "a" * 64
|
||||
|
||||
def test_version_code_swallows_the_rest_of_the_line(self, monkeypatch):
|
||||
"""Measured sub-string behaviour of `installed_facts`, recorded as a work item.
|
||||
|
||||
The extraction is `r'\\s*(versionCode|...)=(.*)'`, so on a real
|
||||
`dumpsys package` line -- `versionCode=42 minSdk=24 targetSdk=34` --
|
||||
`facts['versionCode']` is the whole remainder, not `42`. It is still a
|
||||
usable before/after fingerprint, which is how the caller uses it, but the
|
||||
field name overstates what it holds.
|
||||
"""
|
||||
dev = coldstart.Device(None, use_su=False)
|
||||
|
||||
def respond(cmd, *args, **kwargs):
|
||||
if "versionCode" in cmd[-1]:
|
||||
out = " versionCode=42 minSdk=24 targetSdk=34\n"
|
||||
else:
|
||||
out = ""
|
||||
return subprocess.CompletedProcess(cmd, 0, out, "")
|
||||
|
||||
monkeypatch.setattr(coldstart, "run", respond)
|
||||
facts = coldstart.installed_facts(dev, "com.example.app")
|
||||
assert facts["versionCode"] == "42 minSdk=24 targetSdk=34"
|
||||
|
||||
|
||||
class TestScreenshotValidation:
|
||||
"""`screencap` is judged by the bytes on disk, not by the command's exit status."""
|
||||
|
||||
PNG_MAGIC = b"\x89PNG\r\n\x1a\n"
|
||||
|
||||
def make_device(self, monkeypatch):
|
||||
dev = coldstart.Device(None, use_su=False)
|
||||
return dev
|
||||
|
||||
def test_a_png_header_is_accepted_and_the_size_returned(self, monkeypatch, tmp_path):
|
||||
dev = self.make_device(monkeypatch)
|
||||
payload = self.PNG_MAGIC + b"\x00" * 100
|
||||
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
kwargs["stdout"].write(payload)
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
|
||||
out = tmp_path / "shot.png"
|
||||
assert dev.screencap(str(out)) == len(payload)
|
||||
assert out.read_bytes() == payload
|
||||
|
||||
def test_the_capture_uses_exec_out_not_shell_redirect(self, monkeypatch, tmp_path):
|
||||
dev = self.make_device(monkeypatch)
|
||||
seen = []
|
||||
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
seen.append(list(cmd))
|
||||
kwargs["stdout"].write(self.PNG_MAGIC + b"\x00" * 100)
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
|
||||
dev.screencap(str(tmp_path / "shot.png"))
|
||||
assert seen[0] == ["adb", "exec-out", "screencap", "-p"]
|
||||
|
||||
def test_a_non_png_body_is_rejected(self, monkeypatch, tmp_path):
|
||||
"""A device that printed an error into the stream must not be read as a frame."""
|
||||
dev = self.make_device(monkeypatch)
|
||||
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
kwargs["stdout"].write(b"error: device offline\n")
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
|
||||
assert dev.screencap(str(tmp_path / "shot.png")) == -1
|
||||
|
||||
def test_a_truncated_body_is_rejected(self, monkeypatch, tmp_path):
|
||||
dev = self.make_device(monkeypatch)
|
||||
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
kwargs["stdout"].write(b"\x89PNG")
|
||||
return subprocess.CompletedProcess(cmd, 0, "", "")
|
||||
|
||||
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
|
||||
assert dev.screencap(str(tmp_path / "shot.png")) == -1
|
||||
|
||||
def test_a_timeout_is_reported_as_a_failed_capture(self, monkeypatch, tmp_path):
|
||||
dev = self.make_device(monkeypatch)
|
||||
|
||||
def fake_run(cmd, *args, **kwargs):
|
||||
raise subprocess.TimeoutExpired(cmd, 30)
|
||||
|
||||
monkeypatch.setattr(coldstart.subprocess, "run", fake_run)
|
||||
assert dev.screencap(str(tmp_path / "shot.png")) == -1
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# install_test.py: argument plumbing
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestInstallTestAdbWrappers:
|
||||
def test_serial_is_injected_as_dash_s(self, monkeypatch):
|
||||
rec = Recorder()
|
||||
monkeypatch.setattr(install_test.subprocess, "run", rec)
|
||||
install_test.adb(["shell", "id"], "SERIAL")
|
||||
assert rec.last == ["adb", "-s", "SERIAL", "shell", "id"]
|
||||
|
||||
def test_without_a_serial_no_dash_s_is_added(self, monkeypatch):
|
||||
rec = Recorder()
|
||||
monkeypatch.setattr(install_test.subprocess, "run", rec)
|
||||
install_test.adb(["devices", "-l"])
|
||||
assert rec.last == ["adb", "devices", "-l"]
|
||||
|
||||
def test_stderr_is_appended_so_device_errors_are_not_silently_dropped(self, monkeypatch):
|
||||
rec = Recorder(stdout="out", stderr="error: device offline")
|
||||
monkeypatch.setattr(install_test.subprocess, "run", rec)
|
||||
combined = install_test.adb(["devices"])
|
||||
assert "out" in combined
|
||||
assert "device offline" in combined
|
||||
|
||||
def test_su_wrapper_keeps_the_command_as_one_argument(self, monkeypatch):
|
||||
rec = Recorder()
|
||||
monkeypatch.setattr(install_test.subprocess, "run", rec)
|
||||
install_test.su('pm grant com.example.app "android.permission.X"', "S")
|
||||
assert rec.last == [
|
||||
"adb", "-s", "S", "shell",
|
||||
'su -c "pm grant com.example.app \\"android.permission.X\\""',
|
||||
]
|
||||
@@ -0,0 +1,252 @@
|
||||
"""No-device flows: launch-health verdicts, log parsing, tombstone parsing.
|
||||
|
||||
Everything here runs on a bare checkout. The device commands are intercepted, so
|
||||
what is being tested is the *decision logic* -- which is exactly the part that a
|
||||
device test would otherwise hide behind "it looked fine on screen".
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script, run_script
|
||||
|
||||
install_test = load_script("install_test.py")
|
||||
coldstart = load_script("coldstart.py")
|
||||
native_crash = load_script("native_crash.py")
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
TOMBSTONE = "tombstone_sample.txt"
|
||||
LOGCAT = "logcat_sample.txt"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# install_test.py: the two-sample pid verdict
|
||||
# --------------------------------------------------------------------------- #
|
||||
def drive(monkeypatch, pids, log="", capsys=None):
|
||||
"""Run install_test.main() with the device replaced by a scripted responder.
|
||||
|
||||
`pids` is consumed in order by successive `pidof` calls, which is the whole
|
||||
point of the check: one sample cannot tell a running app from a crash loop.
|
||||
"""
|
||||
pids = list(pids)
|
||||
calls = []
|
||||
|
||||
def fake_adb(args, serial=None, timeout=600):
|
||||
calls.append(list(args))
|
||||
if list(args)[:1] == ["logcat"] and "-d" in args:
|
||||
return log
|
||||
return ""
|
||||
|
||||
def fake_sh(cmd, serial=None):
|
||||
calls.append(cmd)
|
||||
if cmd.startswith("pidof"):
|
||||
return pids.pop(0) if pids else ""
|
||||
if "mCurrentFocus" in cmd:
|
||||
return "mCurrentFocus=Window{a1b2 u0 com.example.app/.MainActivity}"
|
||||
return ""
|
||||
|
||||
monkeypatch.setattr(install_test, "adb", fake_adb)
|
||||
monkeypatch.setattr(install_test, "sh", fake_sh)
|
||||
monkeypatch.setattr(install_test.time, "sleep", lambda _s: None)
|
||||
monkeypatch.setattr(sys, "argv", [
|
||||
"install_test.py", "--apk", "out/app.apk", "--pkg", "com.example.app",
|
||||
"--activity", ".MainActivity",
|
||||
])
|
||||
return install_test.main(), calls
|
||||
|
||||
|
||||
class TestLaunchHealthVerdict:
|
||||
def test_a_stable_pid_with_no_fatal_signature_is_ok(self, monkeypatch, capsys):
|
||||
code, calls = drive(monkeypatch, ["1234", "1234"])
|
||||
assert code == 0, capsys.readouterr().out
|
||||
|
||||
def test_a_changed_pid_is_a_failure_even_though_the_app_is_running(self, monkeypatch, capsys):
|
||||
"""The measured shape of a crash loop: alive at the first sample, restarted by the second."""
|
||||
code, _calls = drive(monkeypatch, ["1234", "5678"])
|
||||
out = capsys.readouterr().out
|
||||
assert code == 1
|
||||
assert "pid @+6s : 1234" in out
|
||||
assert "5678" in out
|
||||
|
||||
def test_a_dead_app_is_a_failure(self, monkeypatch, capsys):
|
||||
code, _calls = drive(monkeypatch, ["", ""])
|
||||
assert code == 1
|
||||
assert "(dead)" in capsys.readouterr().out
|
||||
|
||||
def test_a_fatal_signature_fails_even_with_a_stable_pid(self, monkeypatch, capsys):
|
||||
"""Stability is not health: a caught-but-fatal error still fails the check."""
|
||||
code, _calls = drive(monkeypatch, ["1234", "1234"],
|
||||
log="01-01 10:00:01 E AndroidRuntime: FATAL EXCEPTION: main\n")
|
||||
assert code == 1
|
||||
assert "FATAL EXCEPTION" in capsys.readouterr().out
|
||||
|
||||
def test_both_pid_samples_are_actually_taken(self, monkeypatch, capsys):
|
||||
_code, calls = drive(monkeypatch, ["1234", "1234"])
|
||||
capsys.readouterr()
|
||||
pidof_calls = [c for c in calls if isinstance(c, str) and c.startswith("pidof")]
|
||||
assert len(pidof_calls) == 2, "the second sample is what catches a crash loop"
|
||||
|
||||
def test_the_launch_uses_the_package_slash_activity_component(self, monkeypatch, capsys):
|
||||
_code, calls = drive(monkeypatch, ["1234", "1234"])
|
||||
capsys.readouterr()
|
||||
assert any("am start -n com.example.app/.MainActivity" == c for c in calls
|
||||
if isinstance(c, str))
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# coldstart.py: logcat signal counting
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestLogcatSignalCounting:
|
||||
@pytest.fixture
|
||||
def counts(self, fixtures_dir):
|
||||
path = fixtures_dir / LOGCAT
|
||||
assert path.exists(), "the logcat fixture is missing from tests/fixtures"
|
||||
return coldstart.signal_counts(str(path), coldstart.SIGNAL_PATTERNS)
|
||||
|
||||
@pytest.mark.parametrize("name,expected", [
|
||||
("FATAL EXCEPTION", 1),
|
||||
("VerifyError", 1),
|
||||
("Bad checksum", 1),
|
||||
("ANR", 1),
|
||||
("Displayed", 1),
|
||||
("IncompatibleClassChangeError", 0),
|
||||
("ClassNotFoundException", 0),
|
||||
("uncaughtException", 0),
|
||||
("am_crash", 0),
|
||||
])
|
||||
def test_counts(self, counts, name, expected):
|
||||
assert counts[name] == expected
|
||||
|
||||
def test_every_pattern_in_the_table_is_reported(self, counts):
|
||||
assert set(counts) == set(coldstart.SIGNAL_PATTERNS)
|
||||
|
||||
def test_a_missing_log_yields_no_counts_rather_than_raising(self, tmp_path):
|
||||
assert coldstart.signal_counts(str(tmp_path / "nope.txt"),
|
||||
coldstart.SIGNAL_PATTERNS) == {}
|
||||
|
||||
def test_the_displayed_pattern_captures_the_milliseconds(self):
|
||||
import re
|
||||
match = re.search(coldstart.SIGNAL_PATTERNS["Displayed"],
|
||||
"I ActivityManager: Displayed com.example.app/.MainActivity: +842ms")
|
||||
assert match is not None
|
||||
assert match.group(1) == "842"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# native_crash.py: tombstone parsing
|
||||
# --------------------------------------------------------------------------- #
|
||||
@pytest.fixture
|
||||
def crash(fixtures_dir):
|
||||
text = (fixtures_dir / TOMBSTONE).read_text(encoding="utf-8")
|
||||
crashes = native_crash.parse(text)
|
||||
assert len(crashes) == 1, "the fixture holds exactly one crash block"
|
||||
return crashes[0]
|
||||
|
||||
|
||||
class TestTombstoneParsing:
|
||||
def test_signal_and_code(self, crash):
|
||||
assert crash["signal"] == 11
|
||||
assert crash["signame"] == "SIGSEGV"
|
||||
assert crash["si_code"] == "SEGV_MAPERR"
|
||||
assert crash["fault"] == "0x0000000000000010"
|
||||
|
||||
def test_process_identity_is_picked_up(self, crash):
|
||||
assert crash["pid"] == "12345"
|
||||
assert crash["tid"] == "12345"
|
||||
assert crash["proc"] == "com.example.app"
|
||||
|
||||
def test_uptime_and_cause(self, crash):
|
||||
assert crash["uptime"] == "5"
|
||||
assert "null pointer" in crash["cause"]
|
||||
|
||||
def test_backtrace_frames_are_complete(self, crash):
|
||||
assert len(crash["frames"]) == 4
|
||||
assert crash["frames"][0]["path"].endswith("libfoo.so")
|
||||
assert crash["frames"][0]["symbol"] == ""
|
||||
assert crash["frames"][1]["symbol"] == "Java_com_example_app_Native_check+120"
|
||||
assert crash["frames"][2]["path"].startswith("/apex/")
|
||||
assert crash["frames"][2]["symbol"] == "__pthread_start+256"
|
||||
|
||||
def test_registers_are_collected_from_the_dump_line(self, crash):
|
||||
assert crash["regs"]["x0"] == 0x10
|
||||
assert crash["regs"]["x1"] == 0x7A12345678
|
||||
assert crash["regs"]["sp"] == 0x7FFABC1110
|
||||
assert crash["regs"]["lr"] == 0x7A11112222
|
||||
|
||||
def test_the_pc_key_is_overwritten_by_the_last_backtrace_frame(self, crash):
|
||||
"""Measured behaviour, recorded rather than smoothed over.
|
||||
|
||||
Register extraction scans every line of the block with one permissive
|
||||
pattern, so a `#NN pc <hex>` backtrace line also matches and the last one
|
||||
wins. The register dump's own `pc 0000007a111128cc` is therefore not what
|
||||
`regs['pc']` holds. It is a reporting wart, not a wrong verdict -- but it
|
||||
is a wart, so it is pinned here and listed in the harness report.
|
||||
"""
|
||||
assert crash["regs"]["pc"] == 0x55488, "the last frame's pc"
|
||||
assert crash["regs"]["pc"] != 0x7A111128CC, "the register dump's pc"
|
||||
|
||||
def test_empty_input_produces_no_crashes(self):
|
||||
assert native_crash.parse("") == []
|
||||
|
||||
def test_a_log_without_a_signal_line_produces_no_crashes(self):
|
||||
assert native_crash.parse("I/whatever: nothing to see\n") == []
|
||||
|
||||
|
||||
class TestLibraryClassification:
|
||||
@pytest.mark.parametrize("path,expected", [
|
||||
("/apex/com.android.runtime/lib64/bionic/libc.so", True),
|
||||
("/system/lib64/libart.so", True),
|
||||
("/vendor/lib64/libqcom.so", True),
|
||||
("/data/dalvik-cache/arm64/system@framework@boot.oat", True),
|
||||
("/data/app/~~AbCdEf==/com.example.app-1/lib/arm64/libfoo.so", False),
|
||||
("/data/local/tmp/libpayload.so", False),
|
||||
])
|
||||
def test_classification(self, path, expected):
|
||||
assert native_crash.is_system_lib(path) is expected
|
||||
|
||||
def test_a_bare_name_is_treated_as_system(self):
|
||||
"""Measured behaviour: a path that is not absolute cannot be attributed to the app."""
|
||||
assert native_crash.is_system_lib("libfoo.so") is True
|
||||
|
||||
|
||||
class TestVerdictHint:
|
||||
def test_a_small_fault_address_held_in_a_register_is_flagged_as_arranged(self, crash):
|
||||
hints = " ".join(native_crash.verdict_hint(crash))
|
||||
assert "small integer" in hints
|
||||
assert "ARRANGED" in hints
|
||||
assert "x0" in hints
|
||||
|
||||
def test_an_ordinary_null_dereference_gets_no_arranged_hint(self):
|
||||
ordinary = {"fault": "0x0", "regs": {"x1": 0x7A111128CC}, "signame": "SIGSEGV"}
|
||||
hints = native_crash.verdict_hint(ordinary)
|
||||
assert any("small integer" in h for h in hints)
|
||||
assert not any("ARRANGED" in h for h in hints)
|
||||
|
||||
def test_sigkill_is_called_out_as_a_deliberate_termination(self):
|
||||
hints = native_crash.verdict_hint({"fault": "0x0", "regs": {}, "signame": "SIGKILL"})
|
||||
assert any("deliberate termination" in h for h in hints)
|
||||
|
||||
|
||||
class TestNativeCrashCli:
|
||||
def test_the_fixture_tombstone_is_parsed_by_the_cli(self, fixtures_dir):
|
||||
result = run_script("native_crash.py", [str(fixtures_dir / TOMBSTONE)], timeout=120)
|
||||
assert not result.timed_out
|
||||
assert not result.has_traceback, result.output
|
||||
assert result.returncode == 0, result.output
|
||||
assert "libfoo.so" in result.stdout
|
||||
assert "SIGSEGV" in result.stdout
|
||||
|
||||
def test_a_missing_log_is_reported_without_a_traceback(self, tmp_path):
|
||||
result = run_script("native_crash.py", [str(tmp_path / "absent.txt")])
|
||||
assert not result.has_traceback, result.output
|
||||
assert result.returncode != 0
|
||||
assert "no such file" in result.output
|
||||
|
||||
def test_a_malformed_lib_spec_is_refused(self, fixtures_dir):
|
||||
result = run_script("native_crash.py",
|
||||
[str(fixtures_dir / TOMBSTONE), "--lib", "notapair"])
|
||||
assert result.returncode != 0
|
||||
assert "NAME=PATH" in result.output
|
||||
@@ -0,0 +1,41 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Single entry point for the regression suite.
|
||||
|
||||
Why this file exists in addition to ``pytest.ini``: a fresh checkout without
|
||||
pytest should say so in one line, with the install command, instead of dying on
|
||||
an import error. It exits 3 (environment capability insufficient) per the
|
||||
repository's exit-code policy, so a wrapper script can tell "no pytest here"
|
||||
apart from "tests failed" (1).
|
||||
|
||||
Usage:
|
||||
python tests/run_tests.py # whole suite, quiet summary
|
||||
python tests/run_tests.py -m unit # one marker
|
||||
python tests/run_tests.py -k dexutil -vv # anything pytest accepts
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
|
||||
def main(argv):
|
||||
try:
|
||||
import pytest
|
||||
except ImportError:
|
||||
sys.stderr.write(
|
||||
"pytest is not installed; this suite cannot run.\n"
|
||||
" install: python -m pip install pytest\n"
|
||||
" the suite is stdlib-only otherwise (no device, no network, no adb)\n")
|
||||
return 3
|
||||
|
||||
# pytest.ini lives at the repository root and is found from there.
|
||||
os.chdir(REPO_ROOT)
|
||||
args = list(argv[1:]) or ["-q"]
|
||||
return int(pytest.main(args))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv))
|
||||
@@ -0,0 +1,686 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""Regression tests for `skills/apk-reverse/scripts/so_constpatch.py`.
|
||||
|
||||
What is being locked down
|
||||
-------------------------
|
||||
The script rewrites one zip entry inside an APK. The first implementation rebuilt the
|
||||
whole archive through `zipfile.ZipInfo(filename, date_time)` and re-deflated everything,
|
||||
which dropped `extra`, `external_attr` and `create_system`, and could push
|
||||
`resources.arsc` off its STORED/4-byte-aligned layout -- an APK the installer refuses
|
||||
with `[-124]`, and one that may not load at all when the manifest asks for
|
||||
`extractNativeLibs="false"`.
|
||||
|
||||
These tests run the CLI the way a caller does (subprocess, exit code, RESULT token, JSON)
|
||||
and re-read the written archive to check the contract, so they cannot pass by trusting
|
||||
the writer's own bookkeeping. No device, no Android toolchain, no network.
|
||||
|
||||
The fixtures in `tests/fixtures/apk/` are committed; `tests/fixtures/make_fixture_apk.py`
|
||||
regenerates them and is exercised too, so a fixture that stops carrying the metadata
|
||||
under test fails here rather than silently weakening the suite.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
import zipfile
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
HERE = Path(__file__).resolve().parent
|
||||
REPO = HERE.parent
|
||||
SCRIPT = REPO / 'skills' / 'apk-reverse' / 'scripts' / 'so_constpatch.py'
|
||||
SCRIPTS_DIR = SCRIPT.parent
|
||||
FIXTURES = HERE / 'fixtures' / 'apk'
|
||||
MAKE_FIXTURES = HERE / 'fixtures' / 'make_fixture_apk.py'
|
||||
|
||||
APKHUNAN = b'apkhuan'
|
||||
ANDROID = b'android'
|
||||
SO_ENTRY = 'lib/arm64-v8a/libfoo.so'
|
||||
STORE_ENTRY = 'lib/arm64-v8a/libbar.so'
|
||||
X64_SO_ENTRY = 'lib/x86_64/libfoo.so'
|
||||
SERVICES_ENTRIES = (
|
||||
'META-INF/services/com.example.Provider',
|
||||
'META-INF/services/kotlinx.coroutines.internal.MainDispatcherFactory',
|
||||
)
|
||||
UTF8_ENTRY = 'res/raw/\u6d4b\u8bd5.bin'
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ helpers
|
||||
|
||||
def run_cli(*args, cwd=None):
|
||||
return subprocess.run([sys.executable, str(SCRIPT)] + [str(a) for a in args],
|
||||
capture_output=True, text=True, encoding='utf-8',
|
||||
errors='replace', cwd=cwd)
|
||||
|
||||
|
||||
def token_of(proc):
|
||||
for line in reversed(proc.stdout.splitlines()):
|
||||
if line.startswith('RESULT='):
|
||||
return line.split('=', 1)[1].strip()
|
||||
return None
|
||||
|
||||
|
||||
def json_of(proc):
|
||||
lines = proc.stdout.strip().splitlines()
|
||||
assert lines and lines[-1].startswith('RESULT='), proc.stdout
|
||||
return json.loads('\n'.join(lines[:-1]))
|
||||
|
||||
|
||||
def load_named(name, path):
|
||||
"""Import a module by path, so the tests do not depend on sys.path state."""
|
||||
spec = importlib.util.spec_from_file_location(name, path)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
def load_module():
|
||||
return load_named('so_constpatch_under_test', SCRIPT)
|
||||
|
||||
|
||||
def local_extra(path, name):
|
||||
"""The local-header extra field, which is what alignment lives in."""
|
||||
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
|
||||
info = z.getinfo(name)
|
||||
fh.seek(info.header_offset)
|
||||
head = fh.read(30)
|
||||
nlen, elen = struct.unpack_from('<HH', head, 26)
|
||||
return fh.read(nlen + elen)[nlen:]
|
||||
|
||||
|
||||
def data_offset(path, name):
|
||||
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
|
||||
info = z.getinfo(name)
|
||||
fh.seek(info.header_offset)
|
||||
head = fh.read(30)
|
||||
nlen, elen = struct.unpack_from('<HH', head, 26)
|
||||
return info.header_offset + 30 + nlen + elen
|
||||
|
||||
|
||||
def zip_dump(path):
|
||||
"""Everything a rebuild must preserve, keyed by entry name."""
|
||||
out = {}
|
||||
with zipfile.ZipFile(path) as z, open(path, 'rb') as fh:
|
||||
for info in z.infolist():
|
||||
fh.seek(info.header_offset)
|
||||
head = fh.read(30)
|
||||
nlen, elen = struct.unpack_from('<HH', head, 26)
|
||||
raw_name = fh.read(nlen)
|
||||
local_extra_bytes = fh.read(elen)
|
||||
out[info.filename] = {
|
||||
'info': info,
|
||||
'raw_name': raw_name,
|
||||
'local_extra': local_extra_bytes,
|
||||
'method': info.compress_type,
|
||||
'extra': info.extra,
|
||||
'external_attr': info.external_attr,
|
||||
'internal_attr': info.internal_attr,
|
||||
'create_system': info.create_system,
|
||||
'comment': info.comment,
|
||||
'flags': info.flag_bits,
|
||||
'date_time': info.date_time,
|
||||
'crc': info.CRC,
|
||||
'compress_size': info.compress_size,
|
||||
'file_size': info.file_size,
|
||||
'data_offset': info.header_offset + 30 + nlen + elen,
|
||||
'sha256': hashlib.sha256(z.read(info.filename)).hexdigest(),
|
||||
}
|
||||
return out
|
||||
|
||||
|
||||
def fixture(name):
|
||||
p = FIXTURES / name
|
||||
assert p.is_file(), 'missing fixture %s; run tests/fixtures/make_fixture_apk.py' % p
|
||||
return p
|
||||
|
||||
|
||||
def so_bytes(apk, entry=SO_ENTRY):
|
||||
with zipfile.ZipFile(apk) as z:
|
||||
return z.read(entry)
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ fixtures themselves
|
||||
|
||||
def test_fixtures_carry_the_metadata_under_test():
|
||||
"""A fixture that lost its `extra`/`external_attr` would make the suite vacuous."""
|
||||
for name in ('plain.apk', 'arsc_aligned.apk', 'noextract.apk', 'services.apk'):
|
||||
dump = zip_dump(fixture(name))
|
||||
assert dump, name
|
||||
for entry, row in dump.items():
|
||||
assert row['extra'], '%s!%s has no extra field to preserve' % (name, entry)
|
||||
assert any(row['external_attr'] for row in dump.values()), \
|
||||
'%s has no non-zero external_attr to preserve' % name
|
||||
# And one entry that must keep a zero external_attr / create_system=0 too.
|
||||
zeroed = zip_dump(fixture('services.apk'))[UTF8_ENTRY]
|
||||
assert zeroed['external_attr'] == 0 and zeroed['create_system'] == 0
|
||||
|
||||
arsc = zip_dump(fixture('arsc_aligned.apk'))['resources.arsc']
|
||||
assert arsc['method'] == 0
|
||||
assert arsc['data_offset'] % 4 == 0
|
||||
|
||||
noextract = zip_dump(fixture('noextract.apk'))
|
||||
assert noextract[SO_ENTRY]['method'] == 0
|
||||
assert noextract[SO_ENTRY]['data_offset'] % 4 == 0
|
||||
|
||||
|
||||
def test_fixture_manifest_is_binary_axml_with_the_declaration():
|
||||
"""The manifest is real AXML, and the script's reader agrees with repack.py's.
|
||||
|
||||
Two independent readers on the same bytes: if either drifts, this fails instead of
|
||||
one tool quietly disagreeing with the other about `extractNativeLibs`.
|
||||
"""
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
try:
|
||||
import repack
|
||||
finally:
|
||||
sys.path.pop(0)
|
||||
|
||||
with zipfile.ZipFile(fixture('noextract.apk')) as z:
|
||||
blob = z.read('AndroidManifest.xml')
|
||||
assert blob[:2] == b'\x03\x00', 'not a binary AXML document'
|
||||
|
||||
attrs = repack.axml_root_attrs(blob)
|
||||
assert attrs['extractNativeLibs'] == 'false'
|
||||
assert attrs['package'] == 'com.example.noextract'
|
||||
|
||||
mod = load_module()
|
||||
assert mod.axml_root_attributes(blob)['extractNativeLibs'] == 'false'
|
||||
assert mod.manifest_extract_native_libs(blob) == 'false'
|
||||
|
||||
with zipfile.ZipFile(fixture('plain.apk')) as z:
|
||||
assert mod.manifest_extract_native_libs(z.read('AndroidManifest.xml')) is None
|
||||
assert mod.manifest_extract_native_libs(
|
||||
b'<manifest xmlns:android="x" android:extractNativeLibs="false">') == 'false'
|
||||
assert mod.manifest_extract_native_libs(b'</bad>') == 'unreadable'
|
||||
|
||||
|
||||
def test_alignment_predicate_matches_a_real_abi_directory():
|
||||
"""`lib/<abi>/x.so` has two slashes; a `[^/]+` pattern would match nothing."""
|
||||
mod = load_module()
|
||||
for name in ('lib/arm64-v8a/libfoo.so', 'lib/x86/libfoo.so', 'lib/x86_64/a.so'):
|
||||
assert mod.needs_alignment(name, 0, 8) is True, name
|
||||
# `resources.arsc` is mandatory whatever its input offset is.
|
||||
assert mod.needs_alignment('resources.arsc', 0, 10) is True
|
||||
assert mod.needs_alignment('resources.arsc', 8, 10) is True
|
||||
# An uncompressed .so is mandatory too (extractNativeLibs=false maps it in place).
|
||||
assert mod.needs_alignment('lib/arm64-v8a/libfoo.so', 0, 10) is True
|
||||
# Any other STORED entry keeps the alignment it arrived with, and is not re-laid out.
|
||||
assert mod.needs_alignment('assets/blob.bin', 0, 8) is True
|
||||
assert mod.needs_alignment('assets/blob.bin', 0, 10) is False
|
||||
# Deflated entries have no alignment requirement.
|
||||
assert mod.needs_alignment('classes.dex', 8, 8) is False
|
||||
assert mod.needs_alignment('lib/arm64-v8a/libfoo.so', 8, 8) is False
|
||||
|
||||
|
||||
def test_fixture_generator_reproduces_the_committed_archives(tmp_path):
|
||||
proc = subprocess.run([sys.executable, str(MAKE_FIXTURES), '--out', str(tmp_path)],
|
||||
capture_output=True, text=True)
|
||||
assert proc.returncode == 0, proc.stderr
|
||||
for name in ('plain.apk', 'arsc_aligned.apk', 'noextract.apk', 'services.apk'):
|
||||
regenerated = tmp_path / name
|
||||
assert regenerated.is_file()
|
||||
assert list(zip_dump(regenerated)) == list(zip_dump(fixture(name))), name
|
||||
# Structure must match even if a different zlib produced different bytes.
|
||||
assert os.path.getsize(regenerated) < 8192, 'fixtures must stay tiny'
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the bare-.so path
|
||||
|
||||
def test_bare_so_rewrite_is_byte_exact(tmp_path):
|
||||
"""The `.so` path predates the container work and must not move a single byte.
|
||||
|
||||
The expected output is constructed here from the input, so this is not a
|
||||
self-consistency check: patching `apkhuan` at its reported offset is the whole
|
||||
contract. The sha256 of this same output was measured identical under the
|
||||
pre-change script (see tools/_work/zip-safety-EVIDENCE.md).
|
||||
"""
|
||||
src = so_bytes(fixture('plain.apk'))
|
||||
so_path = tmp_path / 'libfoo.so'
|
||||
so_path.write_bytes(src)
|
||||
out = tmp_path / 'libfoo.patched.so'
|
||||
|
||||
proc = run_cli(so_path, '--replace', 'apkhuan=android', '-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
assert token_of(proc) == 'patched'
|
||||
|
||||
got = out.read_bytes()
|
||||
expected = bytearray(src)
|
||||
expected[0x41:0x41 + len(APKHUNAN)] = ANDROID
|
||||
assert got == bytes(expected)
|
||||
assert len(got) == len(src)
|
||||
assert hashlib.sha256(got).hexdigest() == \
|
||||
hashlib.sha256(bytes(expected)).hexdigest()
|
||||
|
||||
|
||||
def test_bare_so_reports_the_bytes_actually_changed(tmp_path):
|
||||
"""`N byte(s) actually changed`: the window is 7, the changed bytes are 6."""
|
||||
so_path = tmp_path / 'libfoo.so'
|
||||
so_path.write_bytes(so_bytes(fixture('plain.apk')))
|
||||
proc = run_cli(so_path, '--replace', 'apkhuan=android', '-o', tmp_path / 'o.so')
|
||||
assert '6 byte(s) actually changed' in proc.stdout, proc.stdout
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ APK rebuild: metadata
|
||||
|
||||
def test_deflated_entry_rebuild_preserves_every_other_entry(tmp_path):
|
||||
src = fixture('plain.apk')
|
||||
out = tmp_path / 'plain.patched.apk'
|
||||
before = zip_dump(src)
|
||||
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
|
||||
'--verbose')
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
assert list(after) == list(before), 'entry order changed'
|
||||
assert len(after) == len(before)
|
||||
|
||||
changed = after[SO_ENTRY]
|
||||
assert changed['sha256'] != before[SO_ENTRY]['sha256'], 'the patched entry did not change'
|
||||
assert changed['method'] == 8
|
||||
|
||||
for name, row in before.items():
|
||||
if name == SO_ENTRY:
|
||||
continue
|
||||
got = after[name]
|
||||
assert got['sha256'] == row['sha256'], '%s payload changed' % name
|
||||
assert got['crc'] == row['crc'], '%s CRC changed' % name
|
||||
assert got['compress_size'] == row['compress_size'], '%s compress_size changed' % name
|
||||
assert got['extra'] == row['extra'], '%s central-directory extra changed' % name
|
||||
assert got['local_extra'] == row['local_extra'], '%s local extra changed' % name
|
||||
assert got['external_attr'] == row['external_attr'], '%s external_attr changed' % name
|
||||
assert got['internal_attr'] == row['internal_attr']
|
||||
assert got['create_system'] == row['create_system']
|
||||
assert got['comment'] == row['comment']
|
||||
assert got['date_time'] == row['date_time']
|
||||
assert got['flags'] == row['flags']
|
||||
|
||||
|
||||
def test_store_stays_store_and_is_not_recompressed(tmp_path):
|
||||
"""A STORED entry must come back STORED with the same layout, not re-deflated."""
|
||||
src = fixture('arsc_aligned.apk')
|
||||
out = tmp_path / 'arsc_store.patched.apk'
|
||||
before = zip_dump(src)
|
||||
|
||||
proc = run_cli(src, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android', '-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
got, want = after[STORE_ENTRY], before[STORE_ENTRY]
|
||||
assert want['method'] == 0
|
||||
assert got['method'] == 0, 'a STORED entry was re-compressed'
|
||||
assert got['compress_size'] == want['compress_size'] == want['file_size']
|
||||
assert got['file_size'] == want['file_size']
|
||||
assert got['sha256'] != want['sha256']
|
||||
|
||||
# Nothing moved, so alignment needed no new padding and every extra field is
|
||||
# byte-identical -- the strongest form of "metadata preserved".
|
||||
assert got['data_offset'] == want['data_offset']
|
||||
assert got['extra'] == want['extra']
|
||||
assert got['local_extra'] == want['local_extra']
|
||||
for name, row in before.items():
|
||||
assert after[name]['data_offset'] == row['data_offset'], name
|
||||
assert after[name]['local_extra'] == row['local_extra'], name
|
||||
|
||||
|
||||
def test_deflated_change_keeps_arsc_and_stored_so_aligned(tmp_path):
|
||||
"""The case the old writer broke: a size change before `resources.arsc`."""
|
||||
src = fixture('arsc_aligned.apk')
|
||||
out = tmp_path / 'arsc_deflate.patched.apk'
|
||||
before = zip_dump(src)
|
||||
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
assert after[SO_ENTRY]['compress_size'] != before[SO_ENTRY]['compress_size'], \
|
||||
'this fixture must make the patched entry change size, or padding is not exercised'
|
||||
|
||||
for name, row in before.items():
|
||||
got = after[name]
|
||||
if row['method'] != 0:
|
||||
continue
|
||||
# Every uncompressed entry that was aligned stays aligned (zipalign semantics),
|
||||
# and `resources.arsc` is aligned even if it had not been.
|
||||
assert got['data_offset'] % 4 == 0, '%s lost 4-byte alignment' % name
|
||||
assert got['method'] == 0, '%s is no longer STORED' % name
|
||||
if name != SO_ENTRY:
|
||||
# The entry itself is untouched...
|
||||
assert got['sha256'] == row['sha256'], name
|
||||
assert got['crc'] == row['crc'], name
|
||||
assert got['extra'] == row['extra'], name
|
||||
assert got['external_attr'] == row['external_attr'], name
|
||||
# ...and its original local extra survives as a prefix of the new one:
|
||||
# padding is appended, never substituted.
|
||||
assert got['local_extra'].startswith(row['local_extra']), name
|
||||
|
||||
# `libbar.so` is STORED and sits before `resources.arsc`, so the size change shifts
|
||||
# it by an unaligned amount: the writer must restore its alignment with a pad record.
|
||||
bar_pad = len(after[STORE_ENTRY]['local_extra']) \
|
||||
- len(before[STORE_ENTRY]['local_extra'])
|
||||
assert bar_pad > 0, 'a shifted STORED .so was left unaligned without a pad record'
|
||||
pad = bar_pad - 4
|
||||
assert 1 <= pad <= 3, 'the appended record claims a pad of %d bytes' % pad
|
||||
assert bar_pad % 4 == pad, 'the appended record does not land on the boundary'
|
||||
appended = after[STORE_ENTRY]['local_extra'][
|
||||
len(before[STORE_ENTRY]['local_extra']):]
|
||||
assert struct.unpack_from('<H', appended, 2)[0] == pad, 'malformed extra record'
|
||||
assert 'alignment padding' in proc.stdout
|
||||
assert 'resources.arsc' in proc.stdout
|
||||
assert '4-byte aligned' in proc.stdout
|
||||
|
||||
|
||||
def test_an_unaligned_stored_entry_is_not_silently_relaid(tmp_path):
|
||||
"""This tool preserves alignment; it does not re-lay-out an archive it was not asked
|
||||
to fix. A STORED entry that arrived unaligned stays that way, with its extra and
|
||||
offsets untouched -- silently rewriting it would be the same class of change the
|
||||
STORED/alignment work exists to prevent."""
|
||||
fx = load_named('make_fixture_apk_under_test', MAKE_FIXTURES)
|
||||
src = tmp_path / 'handmade.apk'
|
||||
entries = [
|
||||
fx.Entry('classes.dex', fx.make_dex(), method=8),
|
||||
fx.Entry('lib/arm64-v8a/libfoo.so', fx.make_elf(), method=8),
|
||||
# STORED, deliberately NOT aligned (the generator aligns only when asked).
|
||||
fx.Entry('AndroidManifest.xml', fx.make_axml('com.example.unaligned'), method=0),
|
||||
]
|
||||
fx.write_apk(str(src), entries)
|
||||
before = zip_dump(src)
|
||||
assert before['AndroidManifest.xml']['data_offset'] % 4 != 0, \
|
||||
'the handmade archive is supposed to be unaligned here'
|
||||
|
||||
out = tmp_path / 'handmade.patched.apk'
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
assert after[SO_ENTRY]['compress_size'] != before[SO_ENTRY]['compress_size']
|
||||
got = after['AndroidManifest.xml']
|
||||
assert got['data_offset'] % 4 != 0, 'an unaligned STORED entry was silently moved'
|
||||
assert got['local_extra'] == before['AndroidManifest.xml']['local_extra']
|
||||
assert got['sha256'] == before['AndroidManifest.xml']['sha256']
|
||||
assert got['extra'] == before['AndroidManifest.xml']['extra']
|
||||
assert 'NOT 4-byte aligned in the input' in proc.stdout
|
||||
|
||||
|
||||
def test_self_check_is_printed_and_reads_back_the_written_file(tmp_path):
|
||||
src = fixture('arsc_aligned.apk')
|
||||
out = tmp_path / 'selfcheck.apk'
|
||||
proc = run_cli(src, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android', '-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
text = proc.stdout
|
||||
assert 'rebuild self-check' in text
|
||||
assert 'entry count' in text and 'identical' in text
|
||||
assert 'CRC+size identical' in text
|
||||
assert 'sha256 identical' in text
|
||||
assert 'header_offset' in text
|
||||
|
||||
|
||||
def test_services_and_utf8_entries_are_untouched(tmp_path):
|
||||
src = fixture('services.apk')
|
||||
out = tmp_path / 'services.patched.apk'
|
||||
before = zip_dump(src)
|
||||
|
||||
proc = run_cli(src, '--entry', X64_SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
|
||||
'--verbose')
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
for name in SERVICES_ENTRIES + (UTF8_ENTRY, 'AndroidManifest.xml'):
|
||||
assert after[name]['sha256'] == before[name]['sha256'], '%s changed' % name
|
||||
assert after[name]['extra'] == before[name]['extra'], '%s extra changed' % name
|
||||
assert after[name]['external_attr'] == before[name]['external_attr']
|
||||
assert after[name]['create_system'] == before[name]['create_system']
|
||||
assert after[name]['comment'] == before[name]['comment']
|
||||
|
||||
utf8 = after[UTF8_ENTRY]
|
||||
assert utf8['info'].filename == UTF8_ENTRY
|
||||
assert utf8['flags'] & 0x800, 'the UTF-8 name flag was dropped'
|
||||
assert utf8['create_system'] == 0
|
||||
assert utf8['internal_attr'] == 1
|
||||
assert utf8['comment'] == b'fixture comment'
|
||||
|
||||
with zipfile.ZipFile(src) as a, zipfile.ZipFile(out) as b:
|
||||
assert a.comment == b.comment == b'fixture-archive-comment'
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ the refusal gate
|
||||
|
||||
def test_extract_native_libs_false_is_refused_by_default(tmp_path):
|
||||
src = fixture('noextract.apk')
|
||||
out = tmp_path / 'noextract.patched.apk'
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out)
|
||||
|
||||
assert proc.returncode == 1, proc.stdout + proc.stderr
|
||||
assert token_of(proc) == 'refused_unsafe_rebuild'
|
||||
assert not out.exists(), 'a refused rebuild must write nothing'
|
||||
assert 'REFUSED' in proc.stdout
|
||||
assert 'extractNativeLibs' in proc.stdout
|
||||
assert 'scripts/repack.py' in proc.stdout, 'the refusal must name the right tool'
|
||||
assert '--unsafe-rebuild' in proc.stdout
|
||||
|
||||
|
||||
def test_unsafe_rebuild_is_explicit_and_reported(tmp_path):
|
||||
src = fixture('noextract.apk')
|
||||
out = tmp_path / 'noextract.unsafe.apk'
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--replace', 'apkhuan=android', '-o', out,
|
||||
'--unsafe-rebuild')
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
assert token_of(proc) == 'patched'
|
||||
assert 'WARNING: --unsafe-rebuild accepted' in proc.stdout
|
||||
|
||||
after = zip_dump(out)
|
||||
assert after[SO_ENTRY]['method'] == 0
|
||||
assert after[SO_ENTRY]['data_offset'] % 4 == 0
|
||||
assert after['resources.arsc']['method'] == 0
|
||||
assert after['resources.arsc']['data_offset'] % 4 == 0
|
||||
|
||||
|
||||
def test_a_compressed_arsc_is_refused(tmp_path):
|
||||
"""An arsc that is not STORED+aligned cannot be made compliant by this writer."""
|
||||
mod = load_module()
|
||||
broken = tmp_path / 'compressed_arsc.apk'
|
||||
with zipfile.ZipFile(fixture('arsc_aligned.apk')) as src_z:
|
||||
plan = [(i.filename, src_z.read(i.filename), i.compress_type, i.extra,
|
||||
i.external_attr, i.create_system, i.date_time)
|
||||
for i in src_z.infolist()]
|
||||
with zipfile.ZipFile(broken, 'w') as dst:
|
||||
for name, blob, method, extra, eattr, csys, dt in plan:
|
||||
zi = zipfile.ZipInfo(name, date_time=dt)
|
||||
zi.compress_type = zipfile.ZIP_DEFLATED if name == 'resources.arsc' else method
|
||||
zi.extra = extra
|
||||
zi.external_attr = eattr
|
||||
zi.create_system = csys
|
||||
dst.writestr(zi, blob)
|
||||
|
||||
out = tmp_path / 'x.apk'
|
||||
proc = run_cli(broken, '--entry', STORE_ENTRY, '--replace', 'apkhuan=android',
|
||||
'-o', out)
|
||||
assert proc.returncode == 1, proc.stdout + proc.stderr
|
||||
assert token_of(proc) == 'refused_unsafe_rebuild'
|
||||
assert 'resources.arsc' in proc.stdout
|
||||
assert not out.exists()
|
||||
|
||||
# The audit reads the storage state without needing a rebuild at all.
|
||||
facts = mod.audit_apk_for_rebuild(str(broken), mod.Reporter(as_json=True))
|
||||
assert facts['arsc']['stored'] is False
|
||||
assert facts['unsafe']
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ find / usage errors
|
||||
|
||||
def test_find_reports_isolated_and_non_isolated(tmp_path):
|
||||
src = fixture('plain.apk')
|
||||
proc = run_cli(src, '--entry', SO_ENTRY, '--find', 'apkhuan')
|
||||
assert proc.returncode == 0
|
||||
assert token_of(proc) == 'found'
|
||||
assert 'isolated=True' in proc.stdout and 'isolated=False' in proc.stdout
|
||||
assert 'isolated (safe to rewrite in place): 1 / 2' in proc.stdout
|
||||
|
||||
|
||||
def test_find_without_hits_is_a_negative_finding():
|
||||
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--find', 'nosuchthing')
|
||||
assert proc.returncode == 1
|
||||
assert token_of(proc) == 'not_found'
|
||||
|
||||
|
||||
def test_equal_length_is_mandatory():
|
||||
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=and',
|
||||
'-o', 'out.apk')
|
||||
assert proc.returncode == 2
|
||||
assert 'Equal length is mandatory' in proc.stdout
|
||||
|
||||
|
||||
def test_non_isolated_and_multi_hits_need_an_explicit_flag():
|
||||
# The decoy `libapkhuanx.so` makes a second, non-isolated hit.
|
||||
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
|
||||
'-o', 'out.apk', '--allow-nonisolated')
|
||||
assert proc.returncode == 2
|
||||
assert '--force' in proc.stdout
|
||||
assert token_of(proc) == 'usage_error'
|
||||
|
||||
|
||||
def test_missing_entry_and_missing_out_are_usage_errors(tmp_path):
|
||||
p = run_cli(fixture('plain.apk'), '--entry', 'lib/arm64-v8a/nope.so',
|
||||
'--replace', 'apkhuan=android', '-o', tmp_path / 'x.apk')
|
||||
assert p.returncode == 2
|
||||
assert 'available .so entries' in p.stdout
|
||||
|
||||
p = run_cli(fixture('plain.apk'), '--replace', 'apkhuan=android')
|
||||
assert p.returncode == 2
|
||||
|
||||
p = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android')
|
||||
assert p.returncode == 2, 'patching an APK without -o must be rejected'
|
||||
|
||||
p = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
|
||||
'-o', fixture('plain.apk'))
|
||||
assert p.returncode == 2, 'overwriting the input in place must be rejected'
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ json contract
|
||||
|
||||
def test_json_contract_on_success(tmp_path):
|
||||
out = tmp_path / 'j.apk'
|
||||
proc = run_cli(fixture('arsc_aligned.apk'), '--entry', STORE_ENTRY, '--replace',
|
||||
'apkhuan=android', '-o', out, '--json')
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
payload = json_of(proc)
|
||||
assert payload['status'] == 'ok'
|
||||
assert payload['exit_code'] == 0
|
||||
assert payload['capability'] == 'native_const_patch'
|
||||
assert payload['result'] == 'patched'
|
||||
for key in ('evidence', 'warnings', 'next_action'):
|
||||
assert key in payload
|
||||
assert isinstance(payload['evidence'], list) and payload['evidence']
|
||||
assert payload['verify']['entries'] == payload['verify']['untouched'] + 1
|
||||
assert payload['verify']['sha256_identical'] == payload['verify']['untouched']
|
||||
assert payload['audit']['arsc']['stored'] is True
|
||||
assert payload['audit']['unsafe'] == []
|
||||
|
||||
|
||||
def test_json_contract_on_refusal(tmp_path):
|
||||
proc = run_cli(fixture('noextract.apk'), '--entry', SO_ENTRY, '--replace',
|
||||
'apkhuan=android', '-o', tmp_path / 'r.apk', '--json')
|
||||
assert proc.returncode == 1
|
||||
payload = json_of(proc)
|
||||
assert payload['status'] == 'refused'
|
||||
assert payload['exit_code'] == 1
|
||||
assert payload['result'] == 'refused_unsafe_rebuild'
|
||||
assert payload['audit']['extract_native_libs'] == 'false'
|
||||
assert payload['audit']['unsafe']
|
||||
assert 'repack.py' in payload['next_action']
|
||||
|
||||
|
||||
def test_json_contract_on_negative_find():
|
||||
proc = run_cli(fixture('plain.apk'), '--entry', SO_ENTRY, '--find', 'nope', '--json')
|
||||
assert proc.returncode == 1
|
||||
payload = json_of(proc)
|
||||
assert payload['status'] == 'negative'
|
||||
assert payload['result'] == 'not_found'
|
||||
assert payload['exit_code'] == 1
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ streaming archives
|
||||
|
||||
def test_data_descriptors_are_resolved_into_the_local_header(tmp_path):
|
||||
"""An entry written while streaming carries flag bit 3 and no CRC in its local
|
||||
header. The rebuild resolves it from the central directory, so the output is
|
||||
readable by tools that never look at the central directory."""
|
||||
class _Sink(object):
|
||||
"""A file object without seek/tell, which is what makes zipfile stream."""
|
||||
|
||||
def __init__(self, fh):
|
||||
self._fh = fh
|
||||
|
||||
def write(self, data):
|
||||
return self._fh.write(data)
|
||||
|
||||
def flush(self):
|
||||
self._fh.flush()
|
||||
|
||||
def close(self):
|
||||
self._fh.close()
|
||||
|
||||
streaming = tmp_path / 'streaming.apk'
|
||||
with zipfile.ZipFile(fixture('plain.apk')) as zin:
|
||||
with open(streaming, 'wb') as raw:
|
||||
with zipfile.ZipFile(_Sink(raw), 'w', zipfile.ZIP_DEFLATED) as zout:
|
||||
for info in zin.infolist():
|
||||
zi = zipfile.ZipInfo(info.filename, date_time=info.date_time)
|
||||
zi.compress_type = info.compress_type
|
||||
with zout.open(zi, 'w') as fh:
|
||||
fh.write(zin.read(info.filename))
|
||||
|
||||
before = zip_dump(streaming)
|
||||
assert all(row['flags'] & 0x08 for row in before.values()), \
|
||||
'this fixture must carry data descriptors, or the branch is not covered'
|
||||
assert all(row['local_extra'] == b'' or True for row in before.values())
|
||||
|
||||
out = tmp_path / 'streaming.patched.apk'
|
||||
proc = run_cli(streaming, '--entry', SO_ENTRY, '--replace', 'apkhuan=android',
|
||||
'-o', out)
|
||||
assert proc.returncode == 0, proc.stdout + proc.stderr
|
||||
after = zip_dump(out)
|
||||
|
||||
assert 'data descriptors' in proc.stdout
|
||||
for name, row in before.items():
|
||||
got = after[name]
|
||||
assert not (got['flags'] & 0x08), '%s still declares a data descriptor' % name
|
||||
if name != SO_ENTRY:
|
||||
assert got['sha256'] == row['sha256']
|
||||
assert got['crc'] == row['crc']
|
||||
with zipfile.ZipFile(out) as z:
|
||||
assert z.testzip() is None
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ internal invariants
|
||||
|
||||
def test_padding_record_is_four_byte_expressible():
|
||||
"""A zip extra record is (id, size, payload) and its minimum length is 4 bytes."""
|
||||
mod = load_module()
|
||||
assert mod._pad_extra(0) == b''
|
||||
for pad in (1, 2, 3):
|
||||
rec = mod._pad_extra(pad)
|
||||
assert len(rec) == 4 + pad
|
||||
assert len(rec) % 4 == pad
|
||||
assert struct.unpack_from('<H', rec, 2)[0] == pad
|
||||
|
||||
|
||||
def test_eocd_and_unsupported_containers_are_reported(tmp_path):
|
||||
mod = load_module()
|
||||
junk = tmp_path / 'not-a-zip.apk'
|
||||
junk.write_bytes(b'not a zip at all')
|
||||
with pytest.raises(mod.Refused) as exc:
|
||||
mod.audit_apk_for_rebuild(str(junk), mod.Reporter(as_json=True))
|
||||
assert exc.value.exit_code == 3
|
||||
assert exc.value.token == 'unsupported_container'
|
||||
|
||||
proc = run_cli(junk, '--entry', 'anything', '--replace', 'a=b', '-o', tmp_path / 'o.apk')
|
||||
assert proc.returncode == 3
|
||||
assert token_of(proc) == 'unsupported_container'
|
||||
@@ -0,0 +1,186 @@
|
||||
"""Unit tests for skills/apk-reverse/scripts/blob_decode.py.
|
||||
|
||||
The encoded-blob search is the part worth pinning: the whole method exists
|
||||
because a single un-rotated inflate attempt fails and the failure looks like
|
||||
"this is encrypted". The tests build blobs with known parameters and require the
|
||||
search to recover them.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import gzip
|
||||
import json
|
||||
import zlib
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script, run_script
|
||||
|
||||
blob_decode = load_script("blob_decode.py")
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
PAYLOAD = json.dumps({
|
||||
"flags": {"newCheckout": True, "minVersion": 4210},
|
||||
"endpoints": ["https://api.example.invalid/v3/config"],
|
||||
"notes": "synthetic payload for the rotation search regression" * 3,
|
||||
}).encode("utf-8")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# outer encodings
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestOuterEncodings:
|
||||
@pytest.mark.parametrize("which", ["base64", "base64url", "hex"])
|
||||
def test_round_trip(self, which):
|
||||
data = bytes(range(64))
|
||||
encoded = blob_decode.outer_encode(data, which)
|
||||
assert blob_decode.outer_decode(encoded, which) == data
|
||||
|
||||
def test_unknown_encoding_raises(self):
|
||||
with pytest.raises(ValueError):
|
||||
blob_decode.outer_encode(b"x", "rot13")
|
||||
|
||||
def test_undecodable_hex_returns_empty_not_an_exception(self):
|
||||
"""The search calls this on candidate encodings; a raise would abort it."""
|
||||
assert blob_decode.outer_decode("zzzz", "hex") == b""
|
||||
|
||||
def test_whitespace_and_newlines_are_stripped_before_decoding(self):
|
||||
encoded = blob_decode.outer_encode(b"hello world", "base64")
|
||||
wrapped = "\n".join([encoded[:4], encoded[4:]])
|
||||
assert blob_decode.outer_decode(wrapped, "base64") == b"hello world"
|
||||
|
||||
def test_sniff_offers_all_three_encodings(self):
|
||||
guesses = blob_decode.sniff_outer("aGVsbG8gd29ybGQgdGhpcyBpcyBhIHRlc3Q=")
|
||||
assert set(guesses) >= {"base64", "base64url", "hex"}
|
||||
|
||||
def test_sniff_puts_hex_first_for_hex_shaped_text(self):
|
||||
assert blob_decode.sniff_outer("deadbeef" * 8)[0] == "hex"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# inner transforms
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestInnerTransforms:
|
||||
def test_inflate_raw_round_trips_through_raw_deflate(self):
|
||||
body = blob_decode.inflate_raw(PAYLOAD)
|
||||
assert zlib.decompress(body, -15) == PAYLOAD
|
||||
|
||||
def test_inflate_raw_output_has_no_zlib_header(self):
|
||||
"""A raw stream is what the rotation search needs: no header to mis-align."""
|
||||
body = blob_decode.inflate_raw(b"x" * 200)
|
||||
with pytest.raises(zlib.error):
|
||||
zlib.decompress(body) # default wbits expects a zlib header
|
||||
|
||||
@pytest.mark.parametrize("name,blob", [
|
||||
("raw", zlib.compressobj(9, zlib.DEFLATED, -15)),
|
||||
("zlib", zlib.compressobj(9, zlib.DEFLATED, 15)),
|
||||
("gzip", zlib.compressobj(9, zlib.DEFLATED, 31)),
|
||||
])
|
||||
def test_try_inflate_names_each_inner_transform(self, name, blob):
|
||||
body = blob.compress(PAYLOAD) + blob.flush()
|
||||
got = blob_decode.try_inflate(body)
|
||||
assert got is not None
|
||||
assert got[0] == name
|
||||
assert got[1] == PAYLOAD
|
||||
|
||||
def test_try_inflate_returns_none_for_noise(self):
|
||||
assert blob_decode.try_inflate(bytes(range(256))) is None
|
||||
|
||||
def test_gzip_is_accepted_via_the_stdlib_helper(self):
|
||||
got = blob_decode.try_inflate(gzip.compress(PAYLOAD))
|
||||
assert got is not None and got[0] == "gzip" and got[1] == PAYLOAD
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# scoring
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestScoring:
|
||||
def test_structured_text_scores_above_random_bytes(self):
|
||||
structured = blob_decode.score(PAYLOAD)
|
||||
noise = blob_decode.score(bytes(range(256)) * 4)
|
||||
assert structured > noise
|
||||
|
||||
def test_a_json_header_is_rewarded(self):
|
||||
assert blob_decode.score(b'{"a": 1}') > blob_decode.score(b"aaaaaaaa")
|
||||
|
||||
def test_empty_output_scores_negative(self):
|
||||
assert blob_decode.score(b"") < 0
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the rotation search
|
||||
# --------------------------------------------------------------------------- #
|
||||
def rotate(body, cut):
|
||||
return body[-cut:] + body[:-cut] if cut else body
|
||||
|
||||
|
||||
class TestRotationSearch:
|
||||
def _hits(self, blob, max_skip=0):
|
||||
return blob_decode.search(blob, max_skip, lambda _msg: None)
|
||||
|
||||
def test_search_recovers_a_rotated_raw_stream(self):
|
||||
body = blob_decode.inflate_raw(PAYLOAD)
|
||||
cut = 7
|
||||
hits = self._hits(rotate(body, cut))
|
||||
matching = [h for h in hits if h["data"] == PAYLOAD]
|
||||
assert matching, "the rotation with cut=%d was not recovered" % cut
|
||||
assert matching[0]["cut"] == cut
|
||||
assert matching[0]["inner"] == "raw"
|
||||
assert matching[0]["size"] == len(PAYLOAD)
|
||||
|
||||
def test_search_recovers_a_prefixed_and_rotated_stream(self):
|
||||
"""The `skip` arm: a short header in front of the compressed stream."""
|
||||
body = blob_decode.inflate_raw(PAYLOAD)
|
||||
blob = b"\x01\x02" + rotate(body, 5)
|
||||
hits = self._hits(blob, max_skip=4)
|
||||
matching = [h for h in hits if h["data"] == PAYLOAD]
|
||||
assert matching, "skip=2 + cut=5 was not recovered"
|
||||
assert matching[0]["skip"] == 2
|
||||
|
||||
def test_search_on_noise_returns_no_hits(self):
|
||||
assert self._hits(bytes(range(64))) == []
|
||||
|
||||
def test_scores_are_reported_rounded(self):
|
||||
body = blob_decode.inflate_raw(PAYLOAD)
|
||||
hits = self._hits(body)
|
||||
assert hits
|
||||
assert all(isinstance(h["score"], float) for h in hits)
|
||||
|
||||
def test_the_shortest_accepted_body_is_eight_bytes(self):
|
||||
"""Documented floor: `m < 8` breaks the loop, so tiny blobs yield nothing."""
|
||||
assert self._hits(b"\x01\x02\x03") == []
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the CLI-level write-back contract
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestEncodePath:
|
||||
def test_encode_then_decode_recovers_the_payload(self, tmp_path):
|
||||
"""The write-back line the tool prints must actually round-trip."""
|
||||
payload_file = tmp_path / "payload.bin"
|
||||
payload_file.write_bytes(PAYLOAD)
|
||||
|
||||
encoded_file = tmp_path / "value.txt"
|
||||
enc = run_script("blob_decode.py",
|
||||
["--encode", "--file", str(payload_file), "--inner", "raw",
|
||||
"--outer", "base64", "--cut", "0", "--out", str(encoded_file)])
|
||||
assert enc.returncode == 0, enc.output
|
||||
|
||||
back = run_script("blob_decode.py",
|
||||
["--file", str(encoded_file), "--outer", "base64",
|
||||
"--min-size", "1", "--out", str(tmp_path / "back.bin")])
|
||||
assert back.returncode == 0, back.output
|
||||
assert (tmp_path / "back.bin").read_bytes() == PAYLOAD
|
||||
|
||||
def test_an_undecodable_blob_exits_one(self, tmp_path):
|
||||
value = tmp_path / "value.txt"
|
||||
value.write_text("not-a-blob-at-all-just-prose-text\n", encoding="utf-8")
|
||||
result = run_script("blob_decode.py", ["--file", str(value), "--outer", "hex"])
|
||||
assert result.returncode == 1, result.output
|
||||
assert not result.has_traceback
|
||||
|
||||
def test_empty_stdin_is_reported_as_invalid_input(self):
|
||||
"""Exit code 2, not 1: an empty input is invalid, not a failed decode."""
|
||||
result = run_script("blob_decode.py", [], stdin=" ")
|
||||
assert result.returncode == 2, result.output
|
||||
assert "empty input" in result.output
|
||||
@@ -0,0 +1,216 @@
|
||||
"""The established regression for scripts/dex_patch_bytes.py.
|
||||
|
||||
This is the repository's pre-existing red line, not a test invented here: the
|
||||
l1l3 benchmark row recorded that patching the first root check in
|
||||
`MainActivity.onCreate` produces a specific file, and any refactor of the dex
|
||||
reader, the spec matcher, the header fixer or the writer must reproduce it
|
||||
byte for byte.
|
||||
|
||||
`tools/_work/bench/**` is git-ignored, so these tests skip with a printed reason
|
||||
on a fresh checkout. That is deliberate: a regression marker that turns red
|
||||
because a sample is absent is a marker people learn to ignore.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import L1L3_PATCHED_SHA256, load_script, require_path, run_script
|
||||
|
||||
dexutil = load_script("dexutil.py")
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
SPEC_NAME = "l1_root.spec.json"
|
||||
SITE_OFF = 0x9D0
|
||||
OLD_BYTES = "39000e00"
|
||||
NEW_BYTES = "00000000"
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def sample(bench_l1l3):
|
||||
"""Paths to the recorded input, spec and expected output."""
|
||||
dex = require_path(bench_l1l3 / "dex" / "classes.dex",
|
||||
"git-ignored bench tree")
|
||||
spec = require_path(bench_l1l3 / SPEC_NAME, "git-ignored bench tree")
|
||||
expected = require_path(bench_l1l3 / "dex" / "classes.patched.dex",
|
||||
"git-ignored bench tree")
|
||||
return {"dex": dex, "spec": spec, "expected": expected}
|
||||
|
||||
|
||||
def patch(sample, tmp_path, extra=()):
|
||||
out = tmp_path / "out.patched.dex"
|
||||
report = tmp_path / "report.json"
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(sample["dex"]), "--spec", str(sample["spec"]),
|
||||
"-o", str(out), "--report", str(report)] + list(extra))
|
||||
return result, out, report
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the hash marker
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestRecordedRegression:
|
||||
def test_the_patch_runs_and_exits_zero(self, sample, tmp_path):
|
||||
result, _out, _report = patch(sample, tmp_path)
|
||||
assert not result.timed_out, repr(result)
|
||||
assert not result.has_traceback, result.output
|
||||
assert result.returncode == 0, result.output
|
||||
|
||||
def test_output_matches_the_recorded_sha256(self, sample, tmp_path):
|
||||
_result, out, _report = patch(sample, tmp_path)
|
||||
digest = hashlib.sha256(out.read_bytes()).hexdigest()
|
||||
assert digest.startswith(L1L3_PATCHED_SHA256), (
|
||||
"the recorded marker moved: got %s, expected prefix %s"
|
||||
% (digest, L1L3_PATCHED_SHA256))
|
||||
|
||||
def test_output_is_byte_identical_to_the_stored_expectation(self, sample, tmp_path):
|
||||
_result, out, _report = patch(sample, tmp_path)
|
||||
assert out.read_bytes() == sample["expected"].read_bytes()
|
||||
|
||||
def test_only_the_four_patch_bytes_and_the_header_change(self, sample, tmp_path):
|
||||
"""Pin *where* the difference is, so a stray write cannot pass as a hash match.
|
||||
|
||||
The allowed window is bytes 8..32 (checksum + signature, recomputed) plus
|
||||
the four bytes of the patched instruction. Two of those four keep their
|
||||
original value (`0e00` -> `0000` clears one byte and leaves the zero
|
||||
byte), and a recomputed checksum may keep a byte by coincidence, so the
|
||||
assertion is containment plus the changes that must occur.
|
||||
"""
|
||||
_result, out, _report = patch(sample, tmp_path)
|
||||
before = sample["dex"].read_bytes()
|
||||
after = out.read_bytes()
|
||||
assert len(before) == len(after), "an equal-length patch must not resize the file"
|
||||
|
||||
diffs = {i for i, (a, b) in enumerate(zip(before, after)) if a != b}
|
||||
window = set(range(8, 32)) | {SITE_OFF, SITE_OFF + 1, SITE_OFF + 2, SITE_OFF + 3}
|
||||
|
||||
assert diffs <= window, "bytes changed outside the owned window: %r" % sorted(
|
||||
diffs - window)
|
||||
# The two bytes that differ between 39000e00 and 00000000 must have moved.
|
||||
assert {SITE_OFF, SITE_OFF + 2} <= diffs
|
||||
# Both header fields are recomputed; each must have changed somewhere.
|
||||
assert diffs & set(range(8, 12)), "no checksum byte changed"
|
||||
assert diffs & set(range(12, 32)), "no signature byte changed"
|
||||
|
||||
def test_the_written_dex_self_verifies(self, sample, tmp_path):
|
||||
_result, out, _report = patch(sample, tmp_path)
|
||||
data = bytearray(out.read_bytes())
|
||||
assert dexutil.verify_dex_header(data) == (True, True)
|
||||
|
||||
def test_the_result_still_parses_and_decodes_cleanly(self, sample, tmp_path):
|
||||
_result, out, _report = patch(sample, tmp_path)
|
||||
dex, _entry = dexutil.load_dex(str(out))
|
||||
assert dex.check() == []
|
||||
found = dex.find_method("Lsg/vantagepoint/uncrackable1/MainActivity;",
|
||||
"onCreate", "(Landroid/os/Bundle;)V")
|
||||
assert found
|
||||
_section, _idx, code_off = found
|
||||
_insns, clean, expected = dex.decode_all(code_off)
|
||||
assert clean, "the patched method must still decode to exactly insns_size units"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the report the tool writes
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestPatchReport:
|
||||
def test_report_records_the_site_and_the_polarity_check(self, sample, tmp_path):
|
||||
_result, _out, report = patch(sample, tmp_path)
|
||||
reports = json.loads(report.read_text(encoding="utf-8"))
|
||||
assert len(reports) == 1
|
||||
entry = reports[0]
|
||||
assert entry["ok"] is True
|
||||
assert entry["site_off"] == SITE_OFF
|
||||
assert entry["old_bytes"] == OLD_BYTES
|
||||
assert entry["new_bytes"] == NEW_BYTES
|
||||
assert entry["expect_next_ok"] is True
|
||||
assert entry["size_delta"] == 0
|
||||
|
||||
def test_report_confirms_the_post_edit_instruction_boundary(self, sample, tmp_path):
|
||||
_result, _out, report = patch(sample, tmp_path)
|
||||
entry = json.loads(report.read_text(encoding="utf-8"))[0]
|
||||
assert entry["post_decode_clean"] is True
|
||||
assert entry["verifier_edges"] == 0
|
||||
|
||||
def test_stdout_states_the_polarity_check_passed(self, sample, tmp_path):
|
||||
result, _out, _report = patch(sample, tmp_path)
|
||||
assert "polarity: expect_next satisfied" in result.stdout
|
||||
|
||||
def test_dry_run_reports_without_writing(self, sample, tmp_path):
|
||||
out = tmp_path / "should-not-exist.dex"
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(sample["dex"]), "--spec", str(sample["spec"]),
|
||||
"-o", str(out), "--dry-run"])
|
||||
assert result.returncode == 0, result.output
|
||||
assert "dry run: nothing written" in result.stdout
|
||||
assert not out.exists()
|
||||
|
||||
def test_default_output_name_is_derived_from_the_input(self, sample, tmp_path):
|
||||
"""Run on a copy: the bench tree is evidence and is never written by this suite."""
|
||||
local = tmp_path / "classes.dex"
|
||||
local.write_bytes(sample["dex"].read_bytes())
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(local), "--spec", str(sample["spec"])])
|
||||
assert result.returncode == 0, result.output
|
||||
derived = tmp_path / "classes.patched.dex"
|
||||
assert derived.exists(), result.output
|
||||
assert hashlib.sha256(derived.read_bytes()).hexdigest() == \
|
||||
hashlib.sha256(sample["expected"].read_bytes()).hexdigest()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# refusal paths: a wrong spec must fail loudly, not write a plausible file
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestRefusals:
|
||||
def test_a_dry_run_against_a_missing_match_fails(self, sample, tmp_path):
|
||||
spec = tmp_path / "bad.spec.json"
|
||||
spec.write_text(json.dumps([{
|
||||
"name": "no such site",
|
||||
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
|
||||
"method": "onCreate",
|
||||
"desc": "(Landroid/os/Bundle;)V",
|
||||
"match": {"kind": "if-testz", "bytes": "deadbeef"},
|
||||
"replace": {"kind": "nops"},
|
||||
}]), encoding="utf-8")
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(sample["dex"]), "--spec", str(spec), "--dry-run"])
|
||||
assert result.returncode == 1, result.output
|
||||
assert "no instruction matched" in result.output
|
||||
|
||||
def test_a_wrong_polarity_pin_is_refused(self, sample, tmp_path):
|
||||
"""expect_next exists to make the wrong side of a branch impossible to patch."""
|
||||
spec = tmp_path / "bad-polarity.spec.json"
|
||||
spec.write_text(json.dumps([{
|
||||
"name": "wrong expected neighbour",
|
||||
"class": "Lsg/vantagepoint/uncrackable1/MainActivity;",
|
||||
"method": "onCreate",
|
||||
"desc": "(Landroid/os/Bundle;)V",
|
||||
"match": {"kind": "if-testz", "bytes": OLD_BYTES},
|
||||
"expect_next": {"kind": "return"},
|
||||
"replace": {"kind": "nops"},
|
||||
}]), encoding="utf-8")
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(sample["dex"]), "--spec", str(spec), "--dry-run"])
|
||||
assert result.returncode == 1, result.output
|
||||
assert "expect_next not satisfied" in result.output
|
||||
|
||||
def test_a_missing_dex_is_a_usage_error(self, tmp_path, sample):
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(tmp_path / "nope.dex"), "--spec", str(sample["spec"])])
|
||||
assert result.returncode == 2, result.output
|
||||
assert "no such file" in result.output
|
||||
|
||||
def test_a_missing_spec_argument_is_a_usage_error(self, sample):
|
||||
result = run_script("dex_patch_bytes.py", [str(sample["dex"])])
|
||||
assert result.returncode == 2, result.output
|
||||
|
||||
def test_a_non_dex_input_is_refused_with_code_1(self, tmp_path, sample):
|
||||
"""Structurally invalid input is a target-side failure (1), not a usage error (2)."""
|
||||
bogus = tmp_path / "bogus.dex"
|
||||
bogus.write_bytes(b"not a dex at all" * 8)
|
||||
result = run_script("dex_patch_bytes.py",
|
||||
[str(bogus), "--spec", str(sample["spec"]), "--dry-run"])
|
||||
assert result.returncode == 1, result.output
|
||||
assert "not a dex" in result.output
|
||||
@@ -0,0 +1,393 @@
|
||||
"""Unit tests for skills/apk-reverse/scripts/dexutil.py.
|
||||
|
||||
The dex format table is the one place where a wrong entry silently shifts every
|
||||
offset after it, so widths are asserted opcode by opcode against the cases the
|
||||
script's own docstring calls out. No device and no sample file are needed for
|
||||
the table itself; the sample-backed assertions skip when the git-ignored bench
|
||||
tree is absent.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import zlib
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script, require_path
|
||||
|
||||
dexutil = load_script("dexutil.py")
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# little-endian scalar readers
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestScalarReaders:
|
||||
def test_u16_and_u32_are_little_endian(self):
|
||||
assert dexutil.u16(b"\x01\x02", 0) == 0x0201
|
||||
assert dexutil.u32(b"\x01\x02\x03\x04", 0) == 0x04030201
|
||||
|
||||
def test_u16_and_u32_read_from_an_offset(self):
|
||||
buf = b"\xff\xff\x34\x12\x78\x56\x34\x12"
|
||||
assert dexutil.u16(buf, 2) == 0x1234
|
||||
assert dexutil.u32(buf, 4) == 0x12345678
|
||||
|
||||
@pytest.mark.parametrize("value,expected", [
|
||||
(0x0000, 0),
|
||||
(0x7FFF, 0x7FFF),
|
||||
(0x8000, -0x8000),
|
||||
(0xFFFF, -1),
|
||||
])
|
||||
def test_s16_boundaries(self, value, expected):
|
||||
assert dexutil.s16(value) == expected
|
||||
|
||||
def test_read_uleb_single_byte(self):
|
||||
assert dexutil.read_uleb(b"\x7f", 0) == (0x7F, 1)
|
||||
|
||||
def test_read_uleb_two_bytes_and_offset(self):
|
||||
value, off = dexutil.read_uleb(b"\x00\x80\x01", 1)
|
||||
assert value == 128
|
||||
assert off == 3
|
||||
|
||||
def test_read_uleb_multi_byte(self):
|
||||
# 0xE5 0x8E 0x26 -> 624485, the example from the format specification
|
||||
value, off = dexutil.read_uleb(b"\xe5\x8e\x26", 0)
|
||||
assert value == 624485
|
||||
assert off == 3
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# instruction width table
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Each case is a trap the module docstring names explicitly.
|
||||
WIDTH_CASES = [
|
||||
(0x00, 1, "plain nop is one unit"),
|
||||
(0x02, 2, "move/from16"),
|
||||
(0x03, 3, "move/16"),
|
||||
(0x13, 2, "const/16"),
|
||||
(0x14, 3, "const"),
|
||||
(0x16, 2, "const-wide/16"),
|
||||
(0x17, 3, "const-wide/32"),
|
||||
(0x18, 5, "const-wide"),
|
||||
(0x1A, 2, "const-string (21c) carries a 16-bit index"),
|
||||
(0x1B, 3, "const-string/jumbo (31c) carries a 32-bit index"),
|
||||
(0x1C, 2, "const-class"),
|
||||
(0x1E, 1, "monitor-exit breaks the same-family reading"),
|
||||
(0x20, 2, "instance-of"),
|
||||
(0x21, 1, "array-length breaks the same-family reading"),
|
||||
(0x27, 1, "throw"),
|
||||
(0x28, 1, "goto (10t) is one unit"),
|
||||
(0x29, 2, "goto/16 (20t)"),
|
||||
(0x2A, 3, "goto/32 (30t)"),
|
||||
(0x32, 2, "if-test (22t) is two units, not one"),
|
||||
(0x37, 2, "last if-test"),
|
||||
(0x38, 2, "if-testz (21t) is two units, not one"),
|
||||
(0x3D, 2, "last if-testz"),
|
||||
(0x6E, 3, "invoke-kind (35c)"),
|
||||
(0x74, 3, "invoke-kind/range (3rc)"),
|
||||
(0x7B, 1, "neg-int"),
|
||||
(0x90, 2, "add-int (23x)"),
|
||||
(0xD0, 2, "add-int/lit16 (22s)"),
|
||||
(0xFA, 4, "invoke-polymorphic (45cc)"),
|
||||
(0xFC, 3, "invoke-custom (35c)"),
|
||||
]
|
||||
|
||||
|
||||
class TestInstructionWidths:
|
||||
@pytest.mark.parametrize("op,units,why", WIDTH_CASES,
|
||||
ids=["%02x" % c[0] for c in WIDTH_CASES])
|
||||
def test_width(self, op, units, why):
|
||||
assert dexutil.insn_units(op, b"\x00" * 32, 0, 32) == units, why
|
||||
|
||||
def test_table_and_accessor_agree_for_every_listed_opcode(self):
|
||||
"""insn_units must not diverge from OP_UNITS for any allocated opcode."""
|
||||
buf = b"\x00" * 32
|
||||
drifted = []
|
||||
for op, units in sorted(dexutil.OP_UNITS.items()):
|
||||
got = dexutil.insn_units(op, buf, 0, len(buf))
|
||||
if got != units:
|
||||
drifted.append((hex(op), units, got))
|
||||
assert drifted == []
|
||||
|
||||
@pytest.mark.parametrize("op", [0x3E, 0x43, 0x73, 0x79, 0x7A, 0xE3, 0xF9])
|
||||
def test_unallocated_opcodes_still_terminate(self, op):
|
||||
"""An unlisted opcode must advance by one unit, not run off the end."""
|
||||
assert dexutil.insn_units(op, b"\x00" * 8, 0, 8) == 1
|
||||
|
||||
def test_packed_switch_payload_width(self):
|
||||
# 00 01 | size=3 -> 4 + 3*2 = 10 units
|
||||
buf = bytes([0x00, 0x01]) + (3).to_bytes(2, "little") + b"\x00" * 12
|
||||
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 10
|
||||
|
||||
def test_sparse_switch_payload_width(self):
|
||||
# 00 02 | size=3 -> 2 + 3*4 = 14 units
|
||||
buf = bytes([0x00, 0x02]) + (3).to_bytes(2, "little") + b"\x00" * 24
|
||||
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 14
|
||||
|
||||
def test_fill_array_data_payload_width(self):
|
||||
# 00 03 | element_width=4 | size=30 -> 4 + ceil(30*4/2) = 64 units.
|
||||
# The measured case named in the docstring: counting this as 5 units is
|
||||
# what desynchronises the rest of the walk.
|
||||
buf = bytes([0x00, 0x03]) + (4).to_bytes(2, "little") + (30).to_bytes(4, "little")
|
||||
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 64
|
||||
|
||||
def test_fill_array_data_payload_pads_to_an_even_byte_count(self):
|
||||
# element_width=1, size=3 -> 3 bytes pads to 4 -> 4 + 2 = 6 units
|
||||
buf = bytes([0x00, 0x03]) + (1).to_bytes(2, "little") + (3).to_bytes(4, "little")
|
||||
assert dexutil.insn_units(0x00, buf, 0, len(buf)) == 6
|
||||
|
||||
def test_plain_nop_is_not_treated_as_a_payload(self):
|
||||
assert dexutil.insn_units(0x00, b"\x00\x00\x00\x00", 0, 4) == 1
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# header parsing and the structural check
|
||||
# --------------------------------------------------------------------------- #
|
||||
# Verified dex header offsets, as declared by dexutil.OFFSETS. Asserted rather
|
||||
# than assumed: these are the values a hand-written reader most often gets wrong.
|
||||
HEADER_OFFSETS = {
|
||||
"file_size": 0x20, "header_size": 0x24, "endian_tag": 0x28,
|
||||
"map_off": 0x34, "string_ids_off": 0x3C, "type_ids_off": 0x44,
|
||||
"proto_ids_off": 0x4C, "field_ids_off": 0x54, "method_ids_off": 0x5C,
|
||||
"class_defs_off": 0x64,
|
||||
}
|
||||
|
||||
# The six table pointers dex.check() requires to be inside the file.
|
||||
TABLE_POINTERS = ("string_ids_off", "type_ids_off", "proto_ids_off",
|
||||
"field_ids_off", "method_ids_off", "class_defs_off")
|
||||
|
||||
|
||||
def synthetic_dex(size=512):
|
||||
"""A dex-shaped buffer whose header is internally consistent.
|
||||
|
||||
Enough for the structural reader: magic, a matching file_size, and the six
|
||||
table pointers inside the file. It is deliberately not a runnable dex, so
|
||||
nothing here claims parse-level validity of a real image.
|
||||
"""
|
||||
buf = bytearray(size)
|
||||
|
||||
def put(offset, value, width=4):
|
||||
buf[offset:offset + width] = value.to_bytes(width, "little")
|
||||
|
||||
buf[0:8] = b"dex\n035\x00"
|
||||
put(HEADER_OFFSETS["file_size"], size)
|
||||
put(HEADER_OFFSETS["header_size"], 0x70)
|
||||
put(HEADER_OFFSETS["endian_tag"], 0x12345678)
|
||||
put(HEADER_OFFSETS["map_off"], 0x70)
|
||||
for key in TABLE_POINTERS:
|
||||
# size 0, pointer 0x70: every pointer lands inside the file.
|
||||
put(HEADER_OFFSETS[key] - 4, 0)
|
||||
put(HEADER_OFFSETS[key], 0x70)
|
||||
return buf
|
||||
|
||||
|
||||
class TestHeaderParsing:
|
||||
def test_the_offset_table_matches_the_format_specification(self):
|
||||
for key, expected in HEADER_OFFSETS.items():
|
||||
assert dexutil.OFFSETS[key] == expected, key
|
||||
|
||||
def test_header_fields_are_parsed_out_of_the_buffer(self):
|
||||
dex = dexutil.Dex(bytes(synthetic_dex(512)))
|
||||
assert dex.header["file_size"] == 512
|
||||
assert dex.header["header_size"] == 0x70
|
||||
assert dex.header["string_ids_off"] == 0x70
|
||||
|
||||
def test_magic_is_parsed(self):
|
||||
dex = dexutil.Dex(bytes(synthetic_dex(256)))
|
||||
assert dex.d[:4] == b"dex\n"
|
||||
|
||||
def test_consistent_header_reports_no_problems(self):
|
||||
dex = dexutil.Dex(bytes(synthetic_dex(512)))
|
||||
assert dex.check() == []
|
||||
|
||||
def test_bad_magic_is_reported(self):
|
||||
buf = synthetic_dex(512)
|
||||
buf[0:4] = b"zip\n"
|
||||
assert any("not a dex" in p for p in dexutil.Dex(bytes(buf)).check())
|
||||
|
||||
def test_file_size_mismatch_is_reported(self):
|
||||
buf = synthetic_dex(512)
|
||||
buf[HEADER_OFFSETS["file_size"]:HEADER_OFFSETS["file_size"] + 4] = \
|
||||
(999).to_bytes(4, "little")
|
||||
assert any("file_size" in p for p in dexutil.Dex(bytes(buf)).check())
|
||||
|
||||
def test_out_of_range_table_pointer_is_reported(self):
|
||||
buf = synthetic_dex(512)
|
||||
at = HEADER_OFFSETS["string_ids_off"]
|
||||
buf[at:at + 4] = (0).to_bytes(4, "little")
|
||||
assert any("string_ids_off" in p for p in dexutil.Dex(bytes(buf)).check())
|
||||
|
||||
def test_string_safe_never_raises_on_a_junk_index(self):
|
||||
"""A desynced decode passes junk indices; the reader must not throw."""
|
||||
dex = dexutil.Dex(bytes(synthetic_dex(512)))
|
||||
assert dex.string_safe(10**6).startswith("<string_idx")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# branch targets
|
||||
# --------------------------------------------------------------------------- #
|
||||
def fake_dex(buf):
|
||||
"""A Dex instance over raw bytes, without header parsing.
|
||||
|
||||
branch_target needs only self.d plus the instruction dict, so this keeps the
|
||||
displacement arithmetic testable without a whole valid dex image.
|
||||
"""
|
||||
dex = dexutil.Dex.__new__(dexutil.Dex)
|
||||
dex.d = bytes(buf)
|
||||
dex.name = "<synthetic>"
|
||||
return dex
|
||||
|
||||
|
||||
def insn_at_zero(buf, op, raw_len):
|
||||
return {"off": 0, "op": op, "raw": bytes(buf[:raw_len]),
|
||||
"units": raw_len // 2, "name": "synthetic"}
|
||||
|
||||
|
||||
class TestBranchTargets:
|
||||
def test_if_testz_negative_displacement(self):
|
||||
buf = bytes([0x38, 0x00]) + (-1).to_bytes(2, "little", signed=True)
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x38, 4)) == -2
|
||||
|
||||
def test_if_testz_positive_displacement(self):
|
||||
buf = bytes([0x3D, 0x01]) + (2).to_bytes(2, "little", signed=True)
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x3D, 4)) == 4
|
||||
|
||||
def test_if_test_two_registers(self):
|
||||
buf = bytes([0x32, 0x10]) + (-4).to_bytes(2, "little", signed=True)
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x32, 4)) == -8
|
||||
|
||||
def test_goto_signed_byte(self):
|
||||
# 10t: op, int8. 0xFE -> -2 units -> -4 bytes from the instruction.
|
||||
buf = bytes([0x28, 0xFE])
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x28, 2)) == -4
|
||||
|
||||
def test_goto16(self):
|
||||
buf = bytes([0x29, 0x00]) + (3).to_bytes(2, "little", signed=True)
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x29, 4)) == 6
|
||||
|
||||
def test_goto32_negative(self):
|
||||
buf = bytes([0x2A, 0x00]) + (-1).to_bytes(4, "little", signed=True)
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x2A, 6)) == -2
|
||||
|
||||
def test_non_branch_returns_none(self):
|
||||
buf = bytes([0x0E, 0x00])
|
||||
assert fake_dex(buf).branch_target(insn_at_zero(buf, 0x0E, 2)) is None
|
||||
|
||||
def test_branch_regs_if_testz_reads_one_register(self):
|
||||
buf = bytes([0x38, 0x07]) + (-1).to_bytes(2, "little", signed=True)
|
||||
regs, kind = fake_dex(buf).branch_regs(insn_at_zero(buf, 0x38, 4))
|
||||
assert regs == [7]
|
||||
assert kind == "if-testz"
|
||||
|
||||
def test_branch_regs_if_test_reads_two_registers(self):
|
||||
buf = bytes([0x32, 0x21]) + (-1).to_bytes(2, "little", signed=True)
|
||||
regs, kind = fake_dex(buf).branch_regs(insn_at_zero(buf, 0x32, 4))
|
||||
assert regs == [2, 1]
|
||||
assert kind == "if-test"
|
||||
|
||||
def test_branch_regs_is_empty_for_a_non_branch(self):
|
||||
buf = bytes([0x0E, 0x00])
|
||||
assert fake_dex(buf).branch_regs(insn_at_zero(buf, 0x0E, 2)) == ([], None)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# header integrity: signature before checksum
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestHeaderIntegrity:
|
||||
def test_fix_then_verify_round_trips(self):
|
||||
data = bytearray(synthetic_dex(512))
|
||||
dexutil.fix_dex_header(data)
|
||||
assert dexutil.verify_dex_header(data) == (True, True)
|
||||
|
||||
def test_signature_covers_the_bytes_after_the_signature_field(self):
|
||||
data = bytearray(synthetic_dex(512))
|
||||
dexutil.fix_dex_header(data)
|
||||
assert bytes(data[12:32]) == hashlib.sha1(bytes(data[32:])).digest()
|
||||
|
||||
def test_checksum_covers_the_signature_field(self):
|
||||
"""The order requirement: adler32 over d[12:] includes the written signature."""
|
||||
data = bytearray(synthetic_dex(512))
|
||||
dexutil.fix_dex_header(data)
|
||||
stored = int.from_bytes(data[8:12], "little")
|
||||
assert stored == zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF
|
||||
|
||||
def test_checksum_taken_over_a_zeroed_signature_does_not_verify(self):
|
||||
"""The failure mode the module docstring names: the header never verifies."""
|
||||
data = bytearray(synthetic_dex(512))
|
||||
data[12:32] = b"\x00" * 20
|
||||
data[8:12] = (zlib.adler32(bytes(data[12:])) & 0xFFFFFFFF).to_bytes(4, "little")
|
||||
data[12:32] = hashlib.sha1(bytes(data[32:])).digest()
|
||||
checksum_ok, _signature_ok = dexutil.verify_dex_header(data)
|
||||
assert checksum_ok is False
|
||||
|
||||
def test_a_tampered_body_fails_verification(self):
|
||||
data = bytearray(synthetic_dex(512))
|
||||
dexutil.fix_dex_header(data)
|
||||
data[-1] ^= 0xFF
|
||||
assert dexutil.verify_dex_header(data) == (False, False)
|
||||
|
||||
def test_fix_reports_before_and_after_values(self):
|
||||
data = bytearray(synthetic_dex(512))
|
||||
report = dexutil.fix_dex_header(data)
|
||||
assert set(report) == {"before_checksum", "after_checksum",
|
||||
"before_signature", "after_signature"}
|
||||
assert report["before_checksum"] != report["after_checksum"]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# sample-backed assertions (skip when the git-ignored bench tree is absent)
|
||||
# --------------------------------------------------------------------------- #
|
||||
SAMPLE_CLASS = "Lsg/vantagepoint/uncrackable1/MainActivity;"
|
||||
SAMPLE_METHOD = ("onCreate", "(Landroid/os/Bundle;)V")
|
||||
SAMPLE_SITE = 0x9D0
|
||||
SAMPLE_SITE_BYTES = "39000e00"
|
||||
|
||||
|
||||
class TestAgainstTheRecordedSample:
|
||||
"""The l1l3 fixture is git-ignored, so these skip in CI by design."""
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _need_sample(self, bench_l1l3):
|
||||
self.dex_path = require_path(
|
||||
bench_l1l3 / "dex" / "classes.dex",
|
||||
"the l1l3 sample is a download into the git-ignored bench tree")
|
||||
|
||||
def _method(self):
|
||||
dex, _entry = dexutil.load_dex(str(self.dex_path))
|
||||
found = dex.find_method(SAMPLE_CLASS, *SAMPLE_METHOD)
|
||||
assert found, "the recorded fixture must still contain MainActivity.onCreate"
|
||||
_section, _idx, code_off = found
|
||||
return dex, code_off
|
||||
|
||||
def test_sample_dex_parses_cleanly(self):
|
||||
dex, _entry = dexutil.load_dex(str(self.dex_path))
|
||||
assert dex.check() == []
|
||||
|
||||
def test_decode_walks_exactly_insns_size_units(self):
|
||||
dex, code_off = self._method()
|
||||
insns, clean, expected = dex.decode_all(code_off)
|
||||
assert clean, "decode did not land on 0x%x" % expected
|
||||
assert insns
|
||||
|
||||
def test_the_recorded_root_check_branch_is_at_0x9d0(self):
|
||||
dex, code_off = self._method()
|
||||
target = dex.insn_at(code_off, SAMPLE_SITE)
|
||||
assert target is not None, "instruction boundary at 0x%x disappeared" % SAMPLE_SITE
|
||||
assert target["raw"].hex() == SAMPLE_SITE_BYTES
|
||||
assert target["op"] in dexutil.IF_TESTZ
|
||||
|
||||
def test_branch_target_resolves_to_an_instruction_boundary(self):
|
||||
dex, code_off = self._method()
|
||||
target = dex.insn_at(code_off, SAMPLE_SITE)
|
||||
dest = dex.branch_target(target)
|
||||
assert dest is not None
|
||||
assert dex.insn_at(code_off, dest) is not None, (
|
||||
"branch target 0x%x is not an instruction boundary" % dest)
|
||||
|
||||
def test_every_branch_target_is_an_instruction_boundary(self):
|
||||
"""A decode desync shows up here first: a branch into the middle of an insn."""
|
||||
dex, code_off = self._method()
|
||||
boundaries = {i["off"] for i in dex.decode(code_off)}
|
||||
stray = sorted(t for t in dex.all_targets(code_off) if t not in boundaries)
|
||||
assert stray == [], "branch targets outside the instruction stream: %r" % stray
|
||||
@@ -0,0 +1,192 @@
|
||||
"""Unit tests for skills/apk-reverse/scripts/protobuf_decode_raw.py.
|
||||
|
||||
Two properties are pinned here, because both are claims the reference material
|
||||
makes and neither is visible from reading the decoder:
|
||||
|
||||
* the built-in known-answer fixture set passes in full (26/26), and
|
||||
* a decode -> re-encode round trip is byte-identical for every fixture except
|
||||
the one fixture that is deliberately non-canonical, which must instead
|
||||
report *why* it differs rather than silently normalising.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script, run_script
|
||||
|
||||
pb = load_script("protobuf_decode_raw.py")
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
# 12 fixtures x (fixture check + round trip) + 2 wire-format trap checks.
|
||||
EXPECTED_FIXTURE_COUNT = 12
|
||||
EXPECTED_TOTAL_CHECKS = 26
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def fixtures():
|
||||
return pb._fixtures()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# varint primitives
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestVarintPrimitives:
|
||||
def test_canonical_example(self):
|
||||
assert pb.encode_varint(150) == bytes.fromhex("9601")
|
||||
|
||||
def test_read_varint_returns_info_next_index_and_no_error(self):
|
||||
info, nxt, err = pb.read_varint(bytes.fromhex("9601"), 0, 2)
|
||||
assert err is None
|
||||
assert info["value"] == 150
|
||||
assert info["bytes"] == 2
|
||||
assert nxt == 2
|
||||
|
||||
def test_a_truncated_varint_is_an_error_not_a_value(self):
|
||||
info, nxt, err = pb.read_varint(bytes.fromhex("96"), 0, 1)
|
||||
assert info is None
|
||||
assert err and "truncated" in err
|
||||
assert nxt == 0
|
||||
|
||||
def test_a_non_canonical_encoding_is_flagged(self):
|
||||
"""A padded varint must be reported: it is why re-encode cannot be assumed equal."""
|
||||
info, _nxt, _err = pb.read_varint(bytes.fromhex("8000"), 0, 2)
|
||||
assert info["value"] == 0
|
||||
assert info["non_canonical"] is True
|
||||
assert info["canonical_bytes"] == 1
|
||||
|
||||
def test_varint_len_matches_the_encoding(self):
|
||||
for value in (0, 1, 127, 128, 300, 2 ** 31, 2 ** 63 - 1):
|
||||
assert pb.varint_len(value) == len(pb.encode_varint(value))
|
||||
|
||||
def test_zigzag(self):
|
||||
assert pb.zigzag_decode(0) == 0
|
||||
assert pb.zigzag_decode(1) == -1
|
||||
assert pb.zigzag_decode(2) == 1
|
||||
assert pb.zigzag_decode(4294967294) == 2147483647
|
||||
|
||||
def test_int_views_wrap(self):
|
||||
assert pb.int32_view(0xFFFFFFFF) == -1
|
||||
assert pb.int64_view(0xFFFFFFFFFFFFFFFF) == -1
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the built-in fixture set
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestFixtureSet:
|
||||
def test_the_selftest_function_passes_end_to_end(self, capsys):
|
||||
assert pb.selftest(None) == 0
|
||||
out = capsys.readouterr().out
|
||||
assert "selftest: %d/%d PASS" % (EXPECTED_TOTAL_CHECKS, EXPECTED_TOTAL_CHECKS) in out
|
||||
assert "FAIL" not in out
|
||||
|
||||
def test_fixture_count_is_stable(self, fixtures):
|
||||
"""The 26/26 headline is 12 fixtures x 2 + 2; a silent drop would hide."""
|
||||
assert len(fixtures) == EXPECTED_FIXTURE_COUNT
|
||||
|
||||
def test_every_fixture_has_a_name_payload_and_expectations(self, fixtures):
|
||||
for name, payload, expects, _rt in fixtures:
|
||||
assert isinstance(name, str) and name
|
||||
assert isinstance(payload, (bytes, bytearray)) and payload
|
||||
assert expects, "fixture %s has no assertions" % name
|
||||
|
||||
def test_each_declared_expectation_holds(self, fixtures):
|
||||
failures = []
|
||||
for name, payload, expects, _rt in fixtures:
|
||||
ctx = pb.Ctx(max_depth=6)
|
||||
frames = pb._fixture_frames(name, payload)
|
||||
fields = []
|
||||
for start, end, _p in frames:
|
||||
found, _n, _s = pb.walk(payload, start, end, 0, ctx, "f", analyse=True)
|
||||
fields.extend(found)
|
||||
for label, num, expected in expects:
|
||||
got = pb._selftest_lookup(fields, frames, payload, label, num)
|
||||
if got != expected:
|
||||
failures.append((name, label, expected, got))
|
||||
assert failures == []
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# round trip
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestRoundTrip:
|
||||
def _reencode(self, name, payload):
|
||||
frames = pb._fixture_frames(name, payload)
|
||||
tree = {"frames": [{"fields": pb.walk(payload, s, e, 0, pb.Ctx(6), "f",
|
||||
analyse=True)[0]}
|
||||
for s, e, _p in frames]}
|
||||
report = {"fields": 0, "identical": 0, "mismatched": []}
|
||||
out = pb.reencode_tree(tree, report)
|
||||
expected = b"".join(payload[s:e] for s, e, _p in frames)
|
||||
return out, expected, report
|
||||
|
||||
def test_canonical_fixtures_reencode_byte_identically(self, fixtures):
|
||||
differing = []
|
||||
for name, payload, _expects, rt in fixtures:
|
||||
if rt == "non-canonical":
|
||||
continue
|
||||
out, expected, _report = self._reencode(name, payload)
|
||||
if out != expected:
|
||||
differing.append((name, expected.hex(), out.hex()))
|
||||
assert differing == []
|
||||
|
||||
def test_the_non_canonical_fixture_reports_why_it_differs(self, fixtures):
|
||||
"""A normalising re-encode must say so, not return different bytes quietly."""
|
||||
targets = [f for f in fixtures if f[3] == "non-canonical"]
|
||||
assert targets, "the non-canonical fixture disappeared from the set"
|
||||
for name, payload, _expects, _rt in targets:
|
||||
out, expected, report = self._reencode(name, payload)
|
||||
assert out != expected
|
||||
assert out.hex() == "089601"
|
||||
assert report["mismatched"], "the mismatch must be recorded"
|
||||
assert "non-canonical" in report["mismatched"][0]["reason"]
|
||||
|
||||
def test_reencode_accounts_for_every_field(self, fixtures):
|
||||
for name, payload, _expects, rt in fixtures:
|
||||
_out, _expected, report = self._reencode(name, payload)
|
||||
assert report["fields"] > 0, name
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the two wire-format traps, asserted rather than narrated
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestWireFormatTraps:
|
||||
def test_explicit_zero_is_not_the_same_bytes_as_absent(self):
|
||||
zero = pb.tag_0(5, 0) + pb.encode_varint(0)
|
||||
assert zero != b""
|
||||
|
||||
def test_explicit_zero_decodes_to_value_zero(self):
|
||||
zero = pb.tag_0(5, 0) + pb.encode_varint(0)
|
||||
fields = pb.walk(zero, 0, len(zero), 0, pb.Ctx(6), "f")[0]
|
||||
assert fields[0]["value"] == 0
|
||||
|
||||
def test_absent_field_decodes_to_no_fields(self):
|
||||
assert pb.walk(b"", 0, 0, 0, pb.Ctx(6), "f")[0] == []
|
||||
|
||||
def test_a_packed_payload_offers_both_readings(self):
|
||||
"""Three varints are also a valid nested message; only a schema decides."""
|
||||
packed = b"".join(pb.encode_varint(v) for v in (3, 270, 86942))
|
||||
cands, view = pb.analyse_payload(packed, 0, pb.Ctx(6), "f", 0)
|
||||
live = sorted(c["kind"] for c in cands if c["confidence"] > 0.0)
|
||||
all_kinds = sorted(c["kind"] for c in cands)
|
||||
assert "packed_varint" in live
|
||||
assert "nested_message" in all_kinds
|
||||
assert view == "packed_varint"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the subprocess contract
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestSelftestCli:
|
||||
def test_selftest_subcommand_exits_zero_and_prints_the_tally(self):
|
||||
result = run_script("protobuf_decode_raw.py", ["--selftest"], timeout=180)
|
||||
assert not result.timed_out
|
||||
assert not result.has_traceback, result.output
|
||||
assert result.returncode == 0, result.output
|
||||
assert "selftest: %d/%d PASS" % (EXPECTED_TOTAL_CHECKS, EXPECTED_TOTAL_CHECKS) \
|
||||
in result.stdout
|
||||
|
||||
def test_help_lists_the_selftest_flag(self):
|
||||
result = run_script("protobuf_decode_raw.py", ["--help"])
|
||||
assert result.returncode == 0
|
||||
assert "--selftest" in result.stdout
|
||||
@@ -0,0 +1,198 @@
|
||||
"""Unit tests for skills/apk-reverse/scripts/scan_leaks.py.
|
||||
|
||||
Decoys are built in a temporary directory at run time. No real target identity
|
||||
is committed here, and the token-shaped decoys are assembled from fragments so
|
||||
that this test file itself cannot become a finding if the scanner is ever
|
||||
pointed at the tests tree.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
from conftest import load_script, run_script
|
||||
|
||||
scan_leaks = load_script("scan_leaks.py")
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
# Synthetic identifiers: reverse-domain-shaped, not derived from any real target.
|
||||
DECOY_PACKAGE = "com." + "acme" + ".targetapp"
|
||||
DECOY_TOKEN = "AbCdEf" + "0123456789" + "XyZ"
|
||||
DECOY_USER = "mkessler"
|
||||
|
||||
|
||||
def write(root, name, body):
|
||||
path = root / name
|
||||
path.write_text(body, encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def scan(root, *extra, timeout=120):
|
||||
"""Run the scanner over a temporary root and parse its JSON report."""
|
||||
result = run_script("scan_leaks.py",
|
||||
["--root", str(root), "--format", "json"] + list(extra),
|
||||
timeout=timeout)
|
||||
assert not result.timed_out, "scan_leaks.py did not finish: %r" % result
|
||||
assert not result.has_traceback, result.output
|
||||
# The JSON document is printed first, then the RESULT token line.
|
||||
lines = result.stdout.splitlines()
|
||||
token = lines[-1] if lines else ""
|
||||
doc = json.loads(result.stdout[:result.stdout.rindex("}") + 1])
|
||||
return result, doc, token
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the exemption list as data
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestExemptions:
|
||||
@pytest.mark.parametrize("value", [
|
||||
"com.android.internal",
|
||||
"androidx.core.app",
|
||||
"com.google.firebase",
|
||||
"org.example.demo",
|
||||
"<pkg>",
|
||||
"${PKG}",
|
||||
"sg.vantagepoint.uncrackable1",
|
||||
])
|
||||
def test_benign_values_carry_an_exemption_reason(self, value):
|
||||
assert scan_leaks._benign_package(value) != "", value
|
||||
|
||||
@pytest.mark.parametrize("value", [
|
||||
DECOY_PACKAGE,
|
||||
"com.unknownvendor.someapp",
|
||||
])
|
||||
def test_unknown_reverse_domain_values_are_reportable(self, value):
|
||||
assert scan_leaks._benign_package(value) == "", value
|
||||
|
||||
def test_loopback_and_documentation_addresses_are_distinguished(self):
|
||||
"""Loopback is exempt; an RFC 5737 documentation address is reported on purpose."""
|
||||
assert scan_leaks._benign_ip("127.0.0.1:8080") != ""
|
||||
assert scan_leaks._benign_ip("192.0.2.10:8080") == ""
|
||||
|
||||
def test_placeholder_user_paths_are_exempt(self):
|
||||
assert scan_leaks._benign_path("C:\\Users\\<user>\\work") != ""
|
||||
assert scan_leaks._benign_path("/home/" + DECOY_USER + "/work") == ""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# decoys that must be reported
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestDecoyDetection:
|
||||
def test_strong_decoys_are_reported(self, tmp_path):
|
||||
write(tmp_path, "notes.md", "\n".join([
|
||||
'package="%s"' % DECOY_PACKAGE,
|
||||
"Authorization: Bearer " + DECOY_TOKEN,
|
||||
"/home/%s/work/build/classes.dex" % DECOY_USER,
|
||||
]) + "\n")
|
||||
|
||||
result, doc, token = scan(tmp_path)
|
||||
|
||||
rules = sorted({f["rule"] for f in doc["findings"]})
|
||||
assert "bundle_pkg_attr" in rules
|
||||
assert "token_authorization_header" in rules
|
||||
assert "user_path_posix" in rules
|
||||
assert all(f["strength"] in ("strong", "certain") for f in doc["findings"])
|
||||
assert token == "RESULT=leaks_found"
|
||||
assert result.returncode == 1, result.output
|
||||
|
||||
def test_findings_carry_context_not_just_a_line_number(self, tmp_path):
|
||||
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
|
||||
_result, doc, _token = scan(tmp_path)
|
||||
finding = doc["findings"][0]
|
||||
assert finding["context"].strip()
|
||||
assert finding["match"] == DECOY_PACKAGE
|
||||
assert finding["line"] == 1
|
||||
assert finding["file"] == "notes.md"
|
||||
|
||||
def test_device_serial_shaped_token_is_reported(self, tmp_path):
|
||||
# 16 uppercase alphanumerics with at least one letter, next to a device word.
|
||||
serial = "R58M" + "12ABCDEF" + "9K7Q"
|
||||
assert len(serial) == 16
|
||||
write(tmp_path, "adb.txt", "device %s\n" % serial)
|
||||
_result, doc, token = scan(tmp_path)
|
||||
assert any(f["rule"].startswith("device_serial") for f in doc["findings"]), doc
|
||||
assert token == "RESULT=leaks_found"
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# decoys that must not be reported
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestExemptDecoysStaySilent:
|
||||
def test_benign_identifiers_produce_no_findings(self, tmp_path):
|
||||
write(tmp_path, "notes.md", "\n".join([
|
||||
'package="com.android.internal"',
|
||||
'package="com.google.firebase.messaging"',
|
||||
'package="<pkg>"',
|
||||
"endpoint 127.0.0.1:8080",
|
||||
"path C:\\Users\\<user>\\work\\x.dex",
|
||||
"public crackme sg.vantagepoint.uncrackable1",
|
||||
]) + "\n")
|
||||
|
||||
result, doc, token = scan(tmp_path)
|
||||
|
||||
assert doc["findings"] == [], doc["findings"]
|
||||
assert token == "RESULT=clean"
|
||||
assert result.returncode == 0
|
||||
|
||||
def test_exempted_hits_are_kept_and_auditable_with_show_exempt(self, tmp_path):
|
||||
"""The exemption list must be inspectable, not just silent."""
|
||||
write(tmp_path, "notes.md", 'package="com.android.internal"\n')
|
||||
_result, doc, _token = scan(tmp_path, "--show-exempt")
|
||||
assert doc["findings"] == []
|
||||
assert doc["exempted"], "an exempted hit must still be recorded"
|
||||
assert doc["exempted"][0]["exempt_reason"]
|
||||
|
||||
def test_a_weak_only_hit_keeps_the_default_gate_green(self, tmp_path):
|
||||
"""Documented behaviour: an RFC 5737 address is reported but does not fail --fail-on strong.
|
||||
|
||||
Note the measured detail: `ip:port` satisfies both endpoint rules, so one
|
||||
address is reported twice (once as the `ip:port` form, once as the bare
|
||||
literal). Recorded here rather than smoothed over -- it is a report-volume
|
||||
question, not a missed or invented finding.
|
||||
"""
|
||||
write(tmp_path, "notes.md", "endpoint 192.0.2.10:8080\n")
|
||||
|
||||
result, doc, token = scan(tmp_path)
|
||||
assert {f["strength"] for f in doc["findings"]} == {"weak"}
|
||||
assert {f["rule"] for f in doc["findings"]} == {"plain_addr", "plain_addr_ipliteral"}
|
||||
assert token == "RESULT=leaks_found_strong_only"
|
||||
assert result.returncode == 0
|
||||
|
||||
strict, _doc2, strict_token = scan(tmp_path, "--fail-on", "any")
|
||||
assert strict_token == "RESULT=leaks_found"
|
||||
assert strict.returncode == 1
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# the machine-readable contract
|
||||
# --------------------------------------------------------------------------- #
|
||||
class TestReportContract:
|
||||
def test_json_report_has_the_documented_keys(self, tmp_path):
|
||||
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
|
||||
_result, doc, _token = scan(tmp_path)
|
||||
assert set(doc) >= {"root", "files_scanned", "findings", "exempted",
|
||||
"errors", "result", "fail_on"}
|
||||
assert doc["result"] == "leaks_found"
|
||||
assert doc["fail_on"] == "strong"
|
||||
|
||||
def test_result_token_is_always_the_last_line(self, tmp_path):
|
||||
write(tmp_path, "notes.md", 'package="%s"\n' % DECOY_PACKAGE)
|
||||
result, _doc, token = scan(tmp_path)
|
||||
assert result.stdout.splitlines()[-1] == token
|
||||
assert token.startswith("RESULT=")
|
||||
|
||||
def test_list_rules_prints_the_table_and_exits_zero(self):
|
||||
result = run_script("scan_leaks.py", ["--list-rules"])
|
||||
assert result.returncode == 0, result.output
|
||||
assert not result.has_traceback
|
||||
for rule_id in ("bundle_pkg_attr", "device_serial", "token_github_pat",
|
||||
"appkey_assignment", "user_path_posix"):
|
||||
assert rule_id in result.stdout
|
||||
|
||||
def test_unknown_category_is_a_usage_error(self, tmp_path):
|
||||
result = run_script("scan_leaks.py",
|
||||
["--root", str(tmp_path), "--only", "nosuchcategory"])
|
||||
assert result.returncode == 2, result.output
|
||||
Reference in New Issue
Block a user