This commit is contained in:
inliver233
2026-10-24 01:50:24 +08:00
committed by dpkg123
commit 863fe6b1bf
98 changed files with 28849 additions and 0 deletions
+135
View File
@@ -0,0 +1,135 @@
# tests/ — executable regression suite for the exe-reverse skill
The suite that proves the shipped scripts still do what their docstrings say, offline, with only
the standard library and pytest.
## When to load this
Read it before changing anything under `skills/exe-reverse/scripts/`, and read it before quoting
a coverage claim about this skill. It is the boundary between "the script was written to do X"
and "the script was measured doing X on this host".
## What it runs
| File | What it asserts |
| --- | --- |
| `conftest.py` | Shared fixtures: builds the synthetic PE images at test time, loads a script by path, runs a script as a subprocess with a timeout, and the two synthetic scan corpora. |
| `unit/test_peutil_reader.py` | Header fields against the fixture's declared metadata, RVA to file-offset round trips including the virtual-only tail, the PE32 versus PE32+ optional-header difference, the checksum against a second implementation, the resource tree, named and ordinal imports, the TLS callback, the RSDS/PDB record, relocation blocks, entropy and flag decoding, and the parser's rejections. |
| `unit/test_pepatch_patcher.py` | Pattern parsing and matching, the ambiguity refusal, the expect-mismatch refusal, the length-unproven and length-change cases, the resolver's refusals, the re-read verification, and the CLI-level facts that a dry run writes nothing and a real run leaves its input byte-identical. |
| `unit/test_aob_scan.py` | The verdict boundaries at both sides of each threshold, hit reporting with offsets, RVAs and sections, section filtering, context windows, and every accepted shape of a pattern file. |
| `unit/test_scan_leaks.py` | Every rule firing on a shaped sample, every exemption class staying silent, the three measured deviations from the exemption list, scan position reporting, and the file walk's skips. |
| `unit/test_capabilities_registry.py` | The capability table: every atom's kind, basis and probe name, the transitive closure against an independent walk, the dependency graph's acyclicity, `resolve` on missing atoms and on `file:` atoms that are and are not on disk, the declaration parser including the documentation template it must reject, and the `declared` / `mapped` / `unknown` classification. |
| `unit/test_doctor.py` | The static dependency pass (stdlib, siblings and a third-party name separated without executing the file), `discover_scripts` scanning rather than listing, the per-script table with an unclassified script still counted, and the environment facts with their keys and severity labels. |
| `unit/test_dart_snapshot_strings.py` | The ELF reader against a generated container, both literal encodings recovered at the offsets the fixture placed them at, the length and limit gates, and the pool counts -- zero on an image carrying neither idiom and non-zero on hand-built bytes carrying each one. |
| `cli/test_cli_surface.py` | Help and usage exit codes for every script, the documented exit codes for each script's error paths, and the negative case that inspect-only runs write nothing into the working directory. |
| `cli/test_cli_json.py` | Each script's machine-readable output: keys, values, and the conventions that are not what a reader would assume. |
| `cli/test_cli_registry_surface.py` | The registry and doctor subprocess contracts: `--list` / `--json` / single-capability exit codes and `RESULT=` tokens, the doctor's three blocks and its 0-versus-3 verdict, and the view flags that print one block only. |
| `cli/test_cli_dart_snapshot.py` | The snapshot reader's CLI: the refusals and their exit codes, `--find` case-insensitivity and the exit code a miss produces, `--at` by file offset and by virtual address, and the `--json` document. |
| `integration/test_pipeline_end_to_end.py` | Build, locate, patch, re-read: the patched bytes, the untouched control site, the recomputed checksum, and a control run in which nothing changes. |
| `integration/test_across_builds.py` | Pattern hit counts compared between two builds, including the case where a patch removes the very byte the pattern was located with. |
| `run_tests.py` | The one-command entry point; prints the install line and exits 3 when pytest is missing. |
| `benchmark.md` | The regression matrix: dimension, target, scripts, measured result, strength label. |
## How to run it
```
python -m pytest -q # from the repository root: the whole suite
python -m pytest -q -m unit # one layer: unit, cli or integration
python tests/run_tests.py # same thing, with a clear message if pytest is missing
python tests/run_tests.py -q -m unit # arguments are forwarded to pytest
```
If pytest is not installed: `python -m pip install pytest`.
## What it deliberately does not do
It does not attach a debugger, load a driver, open a socket, read a device, or execute a target.
Those routes are conditional on host hardware and policy, and a suite that asserts them is either
fragile or dishonest. The recorded experiments for them live in `benchmark.md` with the strength
label that says whether they were reproduced here.
It does not commit or download a sample. Every PE input is generated into a temporary directory
by `make_fixture_pe.py` on the way in and removed with the session, so the suite carries no
binary, no licence question and no dependency on a target that changes underneath it.
It does not assert on timing. The only time in the suite is a generous per-process timeout, which
exists to turn a hang into a failure rather than to measure throughput. The timings in
`benchmark.md` were taken by hand and are recorded as measurements, not enforced as limits.
It does not test the reference documents' prose. Cross-reference, budget and routing checks belong
to the repository's own gates.
## Fixtures
`conftest.py` builds four images per session, addressing `make_fixture_pe.py`'s own recipes rather
than duplicating their shapes:
| Name | Recipe | Why it is in the suite |
| --- | --- | --- |
| `pe64` | `default` | PE32+ with every structure the builder can emit, five sections and the planted pair. |
| `pe32` | `pe32` | The 32-bit optional-header layout, whose field offsets differ. |
| `bare` | `bare` | No exports, resources, TLS or debug directory, so the absent cases are representable. |
| `fatalign` | `fat-align` | A 4 KiB file alignment, large enough that `.rdata` has no virtual-only tail. |
The builder's `no-virtual-tail` recipe is deliberately not used: with a zero virtual pad its own
guard rejects the layout it names, so the recipe exits 1 and writes nothing. `fatalign` covers the
same property. `cli/test_cli_surface.py` pins that failure so the recipe cannot quietly stay
broken.
`conftest.build_synthetic_elf` builds a second kind of input, because `dart_snapshot_strings.py`
reads an ELF shared object rather than a PE and no sample of any kind may be committed: a valid
ELF64 header, a `.text` of sixteen bytes carrying neither idiom the pool observer counts, a
`.rodata` holding one one-byte (ASCII) literal and one two-byte (UTF-16LE) literal at known
offsets, and a section-name string table. It is a container to hold the reader against, not a
Flutter snapshot: the counts the script's docstring records came from a real 26.8 MB `app.so` and
cannot be reproduced from a generated file, which is why the counting logic is asserted in both
directions -- zero on the fixture, non-zero on hand-built bytes -- and the real measurement stays a
recorded observation.
## Conventions
Marker is the layer: `unit` for pure functions and parsers, `cli` for subprocess contracts,
`integration` for the no-debugger flows. All three are declared in `pytest.ini`, and
`--strict-markers` is on, so an undeclared marker is an error rather than a warning. There is no
slow or deselected tier and therefore no `--runslow` switch: nothing in the suite needs one, and
declaring a marker that no test carries would be decoration.
A missing prerequisite skips with a printed reason: an interpreter older than the 3.9 the scripts
declare, or a local tool a check needs. A wrong result always fails. Nothing is skipped silently,
and no test was weakened to pass.
Untrusted sample text is assembled from fragments in `conftest.py` rather than quoted whole. The
suite has to contain strings shaped like pasted credentials and profile paths in order to prove
the scanner catches them; written whole they would also be caught by the repository's own
pre-publish scan. The runtime values are identical, and the source text matches nothing.
## Deviations the suite pins
Each of these is a measured behaviour that differs from what the corresponding docstring implies.
They are asserted as observations so a fix shows up as a test change instead of as a quietly
gained or lost result, and each is also recorded in `benchmark.md`.
| Where | Measured behaviour |
| --- | --- |
| `make_fixture_pe.py` | The `no-virtual-tail` recipe contradicts its own guard and cannot be built. |
| `peutil.py`, `pepatch.py` | A file truncated mid-header raises `struct.error` and prints a traceback instead of using the documented exit code; a fully non-PE input does use it. |
| `pepatch.py` | A missing `--spec` file reaches an unguarded `open()` and prints a traceback. |
| `aob_scan.py` | A bare JSON array of patterns raises `AttributeError`; only the object form is accepted. |
| `scan_leaks.py` | Exit code 3 is documented for an empty scan surface and is unreachable: an empty root reports `RESULT=clean` and exits 0. |
| `scan_leaks.py` | The bare-GUID exemption matches inside the machine-guid rule's own match text, so that rule cannot produce a finding. |
| `scan_leaks.py` | The `.NET` entry matches any `.net` TLD, so a hit inside such a host is suppressed as a platform term. |
| `scan_leaks.py` | The `<[a-z_]+>` placeholder alternative is wrapped in `\b`, so it cannot match bracketed prose. |
| `hwprobe.py` | Two route rows answer with a sentence instead of one of the three documented states. |
| `pepatch.py` | `--allow-resize` writes in place, so the file does not grow and the bytes after the site are overwritten rather than shifted. |
## Limits
Coverage is bounded by the fixture, not by the reader. A synthetic image exercises every structure
the builder emits and nothing it does not; a route that needs a device, a debugger, a driver or a
network is out of scope here and is listed as not run in `benchmark.md`. The suite also cannot say
whether a pattern will survive a build it has never seen, which is why the comparisons it does make
are between two images it built itself.
The scripts under `skills/exe-reverse/scripts/` were being edited while this suite was written, and
several assertions moved with them. Re-run the suite after any change to those files rather than
trusting a count recorded earlier.
+104
View File
@@ -0,0 +1,104 @@
# Regression matrix — exe-reverse
The measured record behind `tests/`: one row per dimension, with the target, the scripts it
exercised, the result this host produced, and how strong that result is.
## When to load this
Read it when you need to know what was actually run rather than what the suite covers, or when a
coverage claim needs a target and a number behind it. `tests/README.md` describes the suite;
this file records its output.
## How a row is filled in
Every row below was produced by running the command in it on this host, except where the row says
otherwise in the result column. Rows that were not run say so and are not filled with an estimate.
Strength labels follow the repository convention: `observed` means the command and its output
exist, `inferred` means the result is reasoned from an observation, `unverified` means it is
reported but was not reproduced here.
Targets are public and legally usable: the generated fixture, and platform binaries that ship with
the operating system. No third-party product is a target, and no sample is stored in the
repository — the images are built at test time into a temporary directory.
Environment: Windows 11 build 26100, Python 3.14.0, pytest 8.4.1.
## The matrix
| Dimension | Target | Scripts exercised | Measured result | Strength |
| --- | --- | --- | --- | --- |
| PE32+ header surface | generated fixture, `default` recipe | `peutil.py` | 4096 bytes, 5 sections, magic 0x20B, entry RVA 0x1000, image base 0x140000000, declared checksum 0x10bad, zero warnings | observed |
| PE32 header surface | generated fixture, `pe32` recipe | `peutil.py` | magic 0x10B, machine 0x014C, image base 0x400000, same five sections | observed |
| Optional-header layout difference | both fixtures | `peutil.py` | number-of-directories lives at optional+92 (PE32) or optional+108 (PE32+), directories at +96 or +112; reading the other layout yields 40 and 0 respectively, neither being the real count 16 | observed |
| RVA to file offset, both directions | generated fixture | `peutil.py` | 0x1010 to 0x410 and back; header RVAs identity below SizeOfHeaders 0x400; an RVA in no section maps to nothing | observed |
| Virtual-only tail | generated fixture | `peutil.py` | `.rdata` has 1408 virtual and 512 raw bytes, so 896 bytes have no file image; RVA 0x2200 resolves to nothing while 0x21FF resolves to 0x9FF | observed |
| PE checksum | four generated recipes | `peutil.py` | declared equals computed equals a second, independent implementation of the published algorithm | observed |
| PE checksum on real images | platform binaries under `%SystemRoot%\System32` | `peutil.py` | `notepad.exe` (360448 bytes) declared 0x5e673 equals computed; `kernel32.dll` and `ntdll.dll` parse with matching checksums | observed |
| Imports | generated fixture | `peutil.py` | one DLL with one named import (hint 7) and one import by ordinal 0x2A, descriptor terminator respected, no delay imports | observed |
| Exports, resources, TLS, debug, relocations | generated fixture | `peutil.py` | one named export; the resource tree is exactly VERSION / 1 / 1033 with a 4-byte leaf; TLS callback RVA 0x1000; RSDS record with the fixture's PDB path and age 1; one relocation block with two entries | observed |
| Pattern locating, three verdicts | generated fixture | `aob_scan.py` | the planted pair reports 2 hits as `ambiguous` and exits 1; the PDB string reports 1 hit as `unique` and exits 0; an absent pattern reports 0 as `no-match` and exits 1 | observed |
| Verdict boundaries | generated fixture | `aob_scan.py` | 0 to `no-match`, 1 to `unique`, 2 through 7 to `ambiguous`, 8 and above to `pervasive` | observed |
| Equal-length patch and verification | generated fixture | `pepatch.py`, `peutil.py` | one patch at 0x410 pinned with `--expect`; the written file re-reads with no verification problems and a recomputed checksum of 0x10c24; exactly 3 bytes differ from the input, the patched byte and 2 checksum bytes | observed |
| Ambiguity refusal | generated fixture | `pepatch.py` | exit 1, `matched 2 sites`, no output file created, input hash unchanged | observed |
| Expect-mismatch refusal | generated fixture | `pepatch.py` | exit 1 with `the site moved or the wrong binary is open` | observed |
| Length discipline | generated fixture | `pepatch.py` | no `--expect` and no `--original-length` reports `[length unproven: pass expect or original-length]`; a 1-into-3 byte patch is refused without `--allow-resize`; with it, the run is marked structural and the file length is unchanged because the write is in place | observed |
| Dry run | generated fixture | `pepatch.py` | exit 0, no file created, input hash unchanged | observed |
| Control run, no-op patch | generated fixture | `pepatch.py` | output SHA-256 equals input SHA-256: the file is byte-identical, checksum included | observed |
| Cross-build comparison | two independently built fixtures | `aob_scan.py` | same recipe 1 versus 1, exit 0; `bare` recipe 1 versus 0, exit 1; patched build 2 versus 1, exit 1; a pattern with 2 hits on both sides still exits 1 because the count agreeing is not the same as the pattern being usable | observed |
| Identity scan, rules | synthetic corpus in a temporary root | `scan_leaks.py` | all 18 rules fire on their shaped samples; the corpus reports 13 strong findings and 4 weak, exit 1 | observed |
| Identity scan, exemptions | same corpus | `scan_leaks.py` | tool, protector, format, platform, placeholder, documentation-address and well-known-SID samples produce no finding; 19 hits suppressed | observed |
| Identity scan, clean surface | temporary root with no identity in it | `scan_leaks.py` | `RESULT=clean`, exit 0 | observed |
| Identity scan, this suite's own surface | `tests/` | `scan_leaks.py` | `RESULT=clean` with 0 strong, 0 weak and 19 suppressed hits | observed |
| Identity scan, the skill's own default surface | `skills/` (the root the script resolves with no arguments) | `scan_leaks.py` | 1 strong finding — a standard platform name read as a product identity bound to a version — with 18 suppressed, exit 1 | observed |
| Host capability probe | this machine | `hwprobe.py` | 8 routes, 3 of them `available`, exit 0; 2 rows answer with a sentence instead of one of the three documented states | observed |
| Suite runtime | this machine | `pytest` | 269 tests in 23.2 s: unit 1.5 s, cli 19.7 s, integration 4.6 s | observed |
| Reader throughput | 360448-byte platform image | `peutil.py` | `--summary` took 159 / 162 / 157 ms over three runs | observed |
| Feature-probe cost | this machine | `hwprobe.py` | `--json` took 1834 / 1821 ms over two runs | observed |
| Fixture reproducibility | generated fixture | `make_fixture_pe.py` | the same recipe produces the same SHA-256 and the same bytes on repeated runs; the `no-virtual-tail` recipe exits 1 and writes nothing | observed |
| Truncated-image handling | fixture cut off mid-header | `peutil.py`, `pepatch.py` | both print a traceback and exit non-zero instead of using their documented exit codes; a fully non-PE input does use them | observed |
| Missing spec file | `pepatch.py` with an absent `--spec` | `pepatch.py` | traceback rather than the documented usage error | observed |
| Bare JSON pattern array | `aob_scan.py` with a top-level array | `aob_scan.py` | `AttributeError`; only the object form is accepted | observed |
| Documented empty-surface exit code | `scan_leaks.py` over an empty directory | `scan_leaks.py` | exit 0 with `RESULT=clean`; the documented exit 3 has no code path | observed |
| Patched image launch | a copy of a platform binary patched at the entry point | `pepatch.py` | not run by this suite. Recorded from the project's measurement pass: one byte changed 48 to CC, exactly two bytes differed from the original (the patched byte and the checksum field), and the process terminated with exit code 0x80000003 about 4.0 s after launch while the unpatched copy showed no surviving process | unverified here |
| Device, debugger, network and driver routes | none | none | not run. This suite has no device, no debugger, no network and no kernel component; those routes are conditional host facts and are reported by `hwprobe.py` instead | unknown |
## Measurements obtained in this run
Reader throughput on a 360448-byte platform image, `peutil.py --summary`, three consecutive runs:
159 ms, 162 ms, 157 ms. Host capability probe, `hwprobe.py --json`, two runs: 1834 ms and 1821 ms.
Both sit within a few percent of the values recorded during the project's earlier measurement pass
(158/166/167 ms and 1858 ms), which is the expected spread for process startup on a busy desktop.
Suite runtime, `python -m pytest -q` from the repository root: 269 passed in 23.2 s. The cli layer
dominates because it starts a fresh interpreter per assertion, twice over for the feature probe.
Exact byte-level effect of a two-byte patch at file offset 0x410: three bytes differ from the
input, at 0x410, 0xD8 and 0xD9. The checksum field is four bytes wide, so a patch that changes two
bytes of image data can leave part of the old checksum value already correct.
## Rows not run
No row in the matrix covers a debugger attach, a hardware breakpoint, a trace capture, a driver
load, a network endpoint, a device, or a live target process. That is the suite's stated boundary,
not an oversight: none of those can be asserted deterministically on a bare checkout, and a row
that claims them would be describing a different machine on a different day.
Two further dimensions were considered and left out. Mass-scanning a large directory of real
images was left out because the time it takes depends on the machine's disk rather than on the
scripts. Comparing two builds of the same third-party product was left out because no such pair is
publicly usable here; the two-build comparison in the matrix uses two images this repository
builds itself.
## Notes for a reader of the results
The one strong finding on the skill's own default surface is also the only failing repository gate
at the time of writing: the tracked-surface scan reads the standard platform name `TPM 2.0` in a
shipped reference as a product bound to a version. Nothing in `tests/` contributes to it — this
suite's own surface reports 0 strong and 0 weak.
The scripts under `skills/exe-reverse/scripts/` were being edited while this matrix was being
produced, and two of its rows moved with them: the public-address rule gained an exclusion that
stops it reading a four-part version number as an address, and a standard-name exemption that had
suppressed every URL finding was corrected. Both the suite and this file record the state after
those corrections. Re-run the matrix after any further change to those scripts rather than
trusting these numbers.
+264
View File
@@ -0,0 +1,264 @@
"""Subprocess contract tests for skills/exe-reverse/scripts/dart_snapshot_strings.py.
The input is the generated ELF64 from `conftest.build_synthetic_elf`, never a sample: a valid
container with one one-byte literal, one two-byte literal and a `.text` that carries neither of the
idioms the pool observer counts.
Two conventions are pinned because they are not what the docstring's exit-code table suggests:
* `--json` returns 0 whatever the filter did, while the text mode returns 1 for "nothing matched";
* `--at` widens a match by `offset + len(text)`, and for a two-byte literal that is half the bytes
the literal occupies, so an offset inside the literal's second half is reported as a miss.
"""
import json
import pytest
pytestmark = pytest.mark.cli
def plain_lines(text):
return [line for line in text.splitlines() if line.strip()]
def reported_count(text):
"""The `N literal(s) reported.` line the text view ends with."""
for line in reversed(plain_lines(text)):
if "literal(s) reported." in line:
return int(line.split()[0])
raise AssertionError("no 'literal(s) reported.' line in:\n%s" % text)
# --- refusals ----------------------------------------------------------------
def test_a_file_that_is_not_an_elf_is_refused_with_the_documented_code(run_script, tmp_path):
path = tmp_path / "notes.txt"
path.write_text("this is not a snapshot\n", encoding="utf-8")
result = run_script("dart_snapshot_strings.py", path)
assert result.returncode == 3
assert "not an ELF image" in result.stderr
assert "Traceback" not in result.stderr
assert result.stdout == ""
def test_a_truncated_image_is_refused_without_a_traceback(run_script, synthetic_elf, tmp_path):
path = tmp_path / "truncated.so"
path.write_bytes(synthetic_elf.data[:0x30])
result = run_script("dart_snapshot_strings.py", path)
assert result.returncode == 3
assert "not an ELF image" in result.stderr
assert "Traceback" not in result.stderr
def test_a_missing_file_exits_two_with_a_reason(run_script, tmp_path):
result = run_script("dart_snapshot_strings.py", tmp_path / "absent.so")
assert result.returncode == 2
assert "error:" in result.stderr
assert "Traceback" not in result.stderr
def test_an_elf_with_no_code_section_is_refused(run_script, synthetic_elf_without_text):
"""A shared object without `.text` is not an AOT code image, and the reader says so."""
result = run_script("dart_snapshot_strings.py", synthetic_elf_without_text.path)
assert result.returncode == 3
assert "no .text section" in result.stderr
assert "Traceback" not in result.stderr
# --- recovery ----------------------------------------------------------------
def test_the_default_run_reports_every_literal_it_recovered(run_script, synthetic_elf):
result = run_script("dart_snapshot_strings.py", synthetic_elf.path)
assert result.returncode == 0, result.stderr
assert "x86-64, shared-object" in result.stdout
for entry in synthetic_elf.literals:
assert hex(entry["vaddr"]) in result.stdout
assert entry["text"] in result.stdout
assert "literals recovered : 3" in result.stdout
assert reported_count(result.stdout) == 3
def test_min_length_filters_the_short_literals_out(run_script, synthetic_elf):
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--min-length", "12")
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
assert "premium_license" in result.stdout
assert "trial_mode" not in result.stdout
assert ">= 12 chars" in result.stdout
def test_limit_caps_how_many_are_reported(run_script, synthetic_elf):
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--limit", "1")
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
def test_section_selection_is_honoured(run_script, synthetic_elf):
only_rodata = run_script("dart_snapshot_strings.py", synthetic_elf.path,
"--sections", ".rodata")
assert only_rodata.returncode == 0, only_rodata.stderr
assert reported_count(only_rodata.stdout) == 3
only_text = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--sections", ".text")
assert only_text.returncode == 1
assert reported_count(only_text.stdout) == 0
absent = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--sections", ".absent")
assert absent.returncode == 1
assert reported_count(absent.stdout) == 0
# --- keyword filter ----------------------------------------------------------
def test_find_matches_case_insensitively(run_script, synthetic_elf):
lower = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "premium")
upper = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "PREMIUM")
assert lower.returncode == 0, lower.stderr
assert upper.returncode == 0, upper.stderr
assert reported_count(lower.stdout) == reported_count(upper.stdout) == 1
assert "premium_license" in lower.stdout
assert "trial_mode" not in lower.stdout
def test_find_accepts_a_comma_separated_list(run_script, synthetic_elf):
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "trial, VIP")
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 2
assert "trial_mode" in result.stdout
assert "vip_pass" in result.stdout
assert "premium_license" not in result.stdout
def test_find_can_reach_a_two_byte_literal(run_script, synthetic_elf):
"""A keyword search that only read ASCII would miss this one, which is why both are recovered."""
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "vip")
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
assert "[two-byte]" in result.stdout
def test_a_keyword_that_matches_nothing_changes_the_exit_code(run_script, synthetic_elf):
"""A miss is a negative finding, not an empty success."""
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_keyword")
assert result.returncode == 1
assert reported_count(result.stdout) == 0
assert "Traceback" not in result.stderr
# --- --at --------------------------------------------------------------------
def test_at_resolves_a_literal_by_file_offset(run_script, synthetic_elf):
entry = synthetic_elf.literal("premium_license")
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(entry["offset"]))
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
assert "premium_license" in result.stdout
def test_at_resolves_a_two_byte_literal_by_file_offset(run_script, synthetic_elf):
entry = synthetic_elf.literal("vip_pass")
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(entry["offset"]))
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
assert "vip_pass" in result.stdout
def test_at_resolves_a_literal_by_virtual_address(run_script, synthetic_elf):
"""The vaddr is far above the file size, which is how the reader tells the two apart."""
entry = synthetic_elf.literal("vip_pass")
assert entry["vaddr"] > synthetic_elf.size
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", hex(entry["vaddr"]))
assert result.returncode == 0, result.stderr
assert reported_count(result.stdout) == 1
assert "vip_pass" in result.stdout
assert hex(entry["vaddr"]) in result.stdout
def test_at_inside_the_code_section_matches_nothing(run_script, synthetic_elf):
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at",
str(synthetic_elf.text["offset"] + 1))
assert result.returncode == 1
assert reported_count(result.stdout) == 0
def test_at_finds_every_byte_of_a_two_byte_literal(load_script, run_script, synthetic_elf):
"""`--at` measures its window in bytes, so **every** byte offset inside the literal matches.
The window used to be `offset + len(text)`. For a two-byte literal the offsets are bytes and the
text is characters, so the window covered only the first half of it and a real byte offset in
the second half came back as a miss. Note which byte is which: in UTF-16LE an ASCII character is
a printable byte followed by a zero byte, so the last meaningful offset in the literal is the
high byte of its final character, which is `0x00` and is a perfectly valid offset to probe.
"""
entry = synthetic_elf.literal("vip_pass")
assert entry["encoding"] == "two-byte"
span = 2 * len(entry["text"])
lo = entry["offset"]
last_low_byte = lo + span - 2
assert synthetic_elf.data[last_low_byte] in range(0x20, 0x7F)
assert synthetic_elf.data[lo + span - 1] == 0, "the final byte is the high byte of the character"
for probe in (lo, lo + 2, last_low_byte, lo + span - 1):
found = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(probe))
assert found.returncode == 0, (probe, found.stdout, found.stderr)
assert reported_count(found.stdout) == 1, (probe, found.stdout)
# The module computes the same window the CLI does.
recovered = load_script("dart_snapshot_strings.py").recover_strings(
synthetic_elf.data, {"offset": synthetic_elf.rodata["offset"],
"vaddr": synthetic_elf.rodata["vaddr"],
"size": synthetic_elf.rodata["size"]})
matched = [row[2] for row in recovered
if row[0] <= lo + span - 1 < row[0] + 2 * len(row[2])]
assert matched == ["vip_pass"]
def test_at_can_be_combined_with_find(run_script, synthetic_elf):
entry = synthetic_elf.literal("trial_mode")
hit = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "trial",
"--at", str(entry["offset"]))
assert hit.returncode == 0, hit.stderr
assert reported_count(hit.stdout) == 1
miss = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "premium",
"--at", str(entry["offset"]))
assert miss.returncode == 1
assert reported_count(miss.stdout) == 0
# --- json --------------------------------------------------------------------
def test_json_returns_one_when_the_filter_matched_nothing(run_script, synthetic_elf):
"""The exit code means the same thing in both output modes.
It used to be 1 from the text path and 0 from the JSON path for the identical filter, so a
caller that asked for JSON could not tell a match from a miss by reading the exit code.
"""
text_mode = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_word")
json_mode = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_word",
"--json")
assert text_mode.returncode == 1
assert json_mode.returncode == 1
payload = json.loads(json_mode.stdout)
assert payload["matches"] == []
assert payload["literals_recovered"] == 0
# An unfiltered run is a listing, not a search, and a listing that finds literals still exits 0.
unfiltered = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--json")
assert unfiltered.returncode == 0
assert json.loads(unfiltered.stdout)["matches"]
def test_an_unknown_flag_is_a_usage_error(run_script):
result = run_script("dart_snapshot_strings.py", "--definitely-not-a-flag")
assert result.returncode == 2
assert "usage:" in result.stderr.lower()
assert "Traceback" not in result.stderr
def test_help_exits_zero(run_script):
result = run_script("dart_snapshot_strings.py", "--help")
assert result.returncode == 0
assert "usage:" in result.stdout.lower()
assert "app.so" in result.stdout
+397
View File
@@ -0,0 +1,397 @@
"""Subprocess contract tests for the machine-readable output of every script.
One convention is not what a reader would assume, so it is asserted explicitly rather than
smoothed over: ``scan_leaks.py`` writes a JSON document and then a ``RESULT=`` line, so its
stdout as a whole is not valid JSON. The document parses from the front; the verdict line is
read from the end. A test pins that shape so a future change to either half is deliberate.
"""
import hashlib
import json
import pytest
pytestmark = pytest.mark.cli
# The reader's own view names, produced by the flags that populate them.
PEUTIL_VIEWS = {
"--summary": "summary",
"--sections": "section_detail",
"--imports": "imports",
"--exports": "exports",
"--resources": "resources",
"--rich": "rich_header",
"--tls": "tls",
"--debug": "debug",
"--relocs": "base_relocations",
"--entropy": "entropy",
"--check-sum": "checksum",
}
# Always present, whatever the flags.
PEUTIL_ALWAYS = ("security_directory", "load_config")
def json_document(text):
return json.JSONDecoder().raw_decode(text)[0]
def last_line(text):
lines = [line for line in text.splitlines() if line.strip()]
return lines[-1] if lines else ""
# --- peutil ------------------------------------------------------------------
def test_reader_json_carries_the_summary_the_text_view_shows(run_script, fixtures):
result = run_script("peutil.py", fixtures.pe64.path, "--json")
assert result.returncode == 0
payload = json.loads(result.stdout)
assert sorted(payload) == ["load_config", "security_directory", "summary"]
summary = payload["summary"]
assert summary["checksum_matches"] is True
assert summary["checksum_declared"] == fixtures.pe64.meta["checksum"]
assert summary["kind"] == "exe"
assert summary["arch"] == "amd64"
assert summary["file_size"] == fixtures.pe64.size
assert summary["sections"] and len(summary["sections"]) == 5
assert payload["security_directory"] is None
assert payload["load_config"] is None
@pytest.mark.parametrize("flag,key", sorted(PEUTIL_VIEWS.items()))
def test_each_reader_view_lands_under_its_own_key(run_script, fixtures, flag, key):
result = run_script("peutil.py", fixtures.pe64.path, flag, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert key in payload
def test_reader_json_with_every_view_present(run_script, fixtures):
flags = [flag for flag in sorted(PEUTIL_VIEWS)]
result = run_script("peutil.py", fixtures.pe64.path, *flags, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
for key in list(PEUTIL_VIEWS.values()) + list(PEUTIL_ALWAYS):
assert key in payload, key
assert payload["resources"][0]["path"] == ["VERSION", "1", "1033"]
assert payload["tls"]["callback_rvas"] == ["0x1000"]
assert payload["debug"][0]["format"] == "RSDS"
assert payload["checksum"]["matches"] is True
assert payload["imports"][0]["dll"] == "fixture-import.dll"
assert payload["exports"]["dll_name"] == "fixture-export.dll"
assert payload["rich_header"] is None # the fixture has none, and says so
assert payload["delay_imports"] == []
assert payload["base_relocations"]["blocks"] == 1
assert payload["section_detail"][0]["raw_tail"] == 0
assert len(payload["entropy"]) == 5
def test_reader_json_of_a_bare_recipe_reports_the_absent_structures(run_script, fixtures):
result = run_script("peutil.py", fixtures.bare.path, "--exports", "--resources", "--tls",
"--debug", "--json")
payload = json.loads(result.stdout)
assert result.returncode == 0
assert payload["exports"] is None
assert payload["resources"] == []
assert payload["tls"] is None
assert payload["debug"] == []
def test_reader_json_is_decodable_ascii(run_script, fixtures):
result = run_script("peutil.py", fixtures.pe64.path, "--json")
assert result.stdout.isascii()
# --- pepatch -----------------------------------------------------------------
def test_patcher_dry_run_json_reports_the_plan_and_says_it_wrote_nothing(run_script, fixtures,
tmp_path):
out = tmp_path / "never.exe"
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
"--bytes", "CC CC", "--original-length", "2", "--dry-run", "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["dry_run"] is True
assert payload["patches_requested"] == 1
assert payload["patches_applied"] == 1
assert payload["errors"] == []
assert payload["checksum_recomputed"] is False
applied = payload["applied"][0]
assert applied["offset"] == 0x410
assert applied["rva"] == "0x1010"
assert applied["before"] == "48 8B"
assert applied["after"] == "CC CC"
assert applied["equal_length"] is True
assert applied["length_basis"] == "original-length"
assert "output" not in payload
assert not out.exists()
def test_patcher_real_run_json_reports_verification_and_the_new_checksum(run_script, fixtures,
tmp_path):
out = tmp_path / "patched.exe"
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
"--expect", "48 8B", "--bytes", "CC 8B", "--out", out, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["output"] == str(out)
assert payload["output_size"] == payload["input_size"] == fixtures.pe64.size
assert payload["verify_problems"] == []
assert payload["checksum_after"]["matches"] is True
assert payload["checksum_recomputed"] is True
assert payload["signed_input"] is False
assert payload["signature_note"] is None
assert payload["structural"] == []
assert payload["length_unproven"] == []
assert out.read_bytes()[0x410:0x412] == b"\xCC\x8B"
def test_patcher_json_names_a_length_unproven_patch(load_script, run_script, fixtures, tmp_path):
out = tmp_path / "unproven.exe"
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
"--bytes", "CC", "--out", out, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["applied"][0]["equal_length"] is None
assert payload["length_unproven"] == ["cli"]
assert payload["structural"] == []
def test_patcher_json_reports_a_refusal_without_an_output(run_script, fixtures, tmp_path):
out = tmp_path / "never.exe"
result = run_script("pepatch.py", fixtures.pe64.path, "--pattern", fixtures.pe64.plant,
"--bytes", "90 " * 12, "--out", out, "--json")
assert result.returncode == 1
assert not out.exists()
assert "matched 2 sites" in result.stderr
assert result.stdout.strip() == ""
# --- aob_scan ----------------------------------------------------------------
def test_locator_json_reports_a_unique_pattern_as_ok(run_script, fixtures, hexpat):
pdb = hexpat(b"fixture.pdb")
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", pdb, "--json")
assert result.returncode == 0, result.stdout
payload = json.loads(result.stdout)
assert payload["result"] == "ok"
assert payload["target_sections"] == [".text", ".rdata", ".data", ".rsrc", ".reloc"]
assert payload["section_filter"] is None
entry = payload["patterns"][0]
assert entry["hits"] == 1
assert entry["verdict"] == "unique"
assert entry["wildcards"] == 0
assert entry["sites"][0]["section"] == ".rdata"
def test_locator_json_reports_the_planted_pair_as_ambiguous(run_script, fixtures):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
"--json")
assert result.returncode == 1, "an ambiguous pattern is the documented non-zero outcome"
payload = json.loads(result.stdout)
entry = payload["patterns"][0]
assert entry["hits"] == 2
assert entry["verdict"] == "ambiguous"
assert payload["result"] == "needs-attention"
offsets = [int(site["offset_hex"], 16) for site in entry["sites"]]
assert offsets == list(fixtures.pe64.plant_offsets)
assert [site["rva_hex"] for site in entry["sites"]] == \
[fixtures.pe64.meta["plant"]["first_rva"], fixtures.pe64.meta["plant"]["second_rva"]]
def test_locator_json_reports_an_absent_pattern_as_a_no_match(run_script, fixtures):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "DE AD BE EF CA FE",
"--json")
assert result.returncode == 1
payload = json.loads(result.stdout)
assert payload["patterns"][0]["hits"] == 0
assert payload["patterns"][0]["verdict"] == "no-match"
assert payload["patterns"][0]["sites"] == []
def test_locator_json_counts_wildcards_and_section_filtered_hits(run_script, fixtures):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns",
"48 8B 05 ?? ?? ?? 00", "--section", ".text", "--json")
payload = json.loads(result.stdout)
entry = payload["patterns"][0]
assert entry["wildcards"] == 3
assert entry["hits"] == 2
assert payload["section_filter"] == ".text"
assert {site["section"] for site in entry["sites"]} == {".text"}
def test_locator_json_from_a_pattern_file_keeps_the_names(run_script, fixtures, tmp_path, hexpat):
sigs = tmp_path / "sigs.txt"
sigs.write_text("# comment\n%s\nnamed-site\t%s\n" % (fixtures.pe64.plant, hexpat(b"FIXT")),
encoding="utf-8")
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", sigs, "--json")
payload = json.loads(result.stdout)
names = [entry["name"] for entry in payload["patterns"]]
assert names == ["sigs.txt:2", "named-site"]
assert payload["patterns"][1]["hits"] >= 1
# --- scan_leaks --------------------------------------------------------------
def test_scanner_json_is_valid_json_ahead_of_the_result_line(run_script, synthetic_leak_root):
"""The document parses from the front; the RESULT line is not part of it."""
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
assert result.returncode == 1
payload = json_document(result.stdout)
assert sorted(payload) == ["exempted", "files_scanned", "root", "strong", "verdict", "weak"]
assert payload["verdict"] == "findings"
assert payload["files_scanned"] >= 3
assert last_line(result.stdout) == "RESULT=findings"
def test_scanner_json_stdout_is_not_valid_as_a_whole(run_script, synthetic_leak_root):
"""A negative outcome: nothing may treat this stdout as one JSON document."""
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
with pytest.raises(json.JSONDecodeError):
json.loads(result.stdout)
def test_scanner_json_reports_exactly_the_expected_rules(run_script, synthetic_leak_root):
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json",
"--show-exempt")
payload = json_document(result.stdout)
assert {hit["rule"] for hit in payload["strong"]} == {
"private-key-block", "github-token", "aws-access-key", "slack-token",
"generic-secret-assignment", "licence-blob-literal", "windows-user-profile",
"posix-home", "windows-sid-literal", "volume-serial-literal",
"literal-endpoint-with-creds", "public-ipv4-literal", "product-name-with-version",
}
assert {hit["rule"] for hit in payload["weak"]} == {
"exe-name-with-version", "literal-host-with-path", "email-address",
"vendor-copyright-line",
}
for hit in payload["strong"]:
assert hit["strength"] == "strong"
assert hit["file"].endswith("notes.md")
assert hit["line"] > 0
assert hit["context"]
assert "machine-guid" not in {hit["rule"] for hit in payload["strong"]}
def test_scanner_json_show_exempt_records_the_suppressed_hits(run_script, synthetic_leak_root):
"""Measured: the machine-guid hit is suppressed by the bare-GUID exemption."""
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json",
"--show-exempt")
payload = json_document(result.stdout)
suppressed = {hit["rule"]: hit.get("exempt") for hit in payload["exempted"]}
assert isinstance(payload["exempted"], list)
assert suppressed.get("machine-guid") == \
"a bare GUID is not machine identity without MachineGuid context"
reasons = {hit["exempt"] for hit in payload["exempted"]
if hit["rule"] == "product-name-with-version"}
assert reasons and reasons <= {"tool or library name",
"platform or vendor-of-the-platform term",
"public protector/packer/managed tooling name"}
def test_scanner_json_without_show_exempt_reports_only_a_count(run_script, synthetic_leak_root):
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
payload = json_document(result.stdout)
assert isinstance(payload["exempted"], int)
assert payload["exempted"] > 0
def test_scanner_json_on_a_clean_root_says_clean(run_script, clean_scan_root):
result = run_script("scan_leaks.py", "--root", clean_scan_root, "--format", "json")
assert result.returncode == 0
payload = json_document(result.stdout)
assert payload["verdict"] == "clean"
assert payload["strong"] == []
assert payload["weak"] == []
# --- make_fixture_pe ---------------------------------------------------------
def test_fixture_builder_json_describes_what_it_wrote(run_script, tmp_path):
out = tmp_path / "described.exe"
result = run_script("make_fixture_pe.py", "--out", out, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["path"] == str(out)
assert payload["size"] == out.stat().st_size
assert payload["sha256"] == hashlib.sha256(out.read_bytes()).hexdigest()
assert sorted(payload["sections"]) == [".data", ".rdata", ".reloc", ".rsrc", ".text"]
plant = payload["plant"]
assert plant["first_file_offset"] != plant["second_file_offset"]
assert "planted twice" in plant["note"]
assert payload["checksum"].startswith("0x")
assert payload["virtual_only_tail_bytes"] > 0
assert payload["executable_and_writable_code"] is True
def test_fixture_builder_is_byte_reproducible(run_script, tmp_path):
first = tmp_path / "one.exe"
second = tmp_path / "two.exe"
a = run_script("make_fixture_pe.py", "--out", first, "--json")
b = run_script("make_fixture_pe.py", "--out", second, "--json")
assert a.returncode == b.returncode == 0
assert json.loads(a.stdout)["sha256"] == json.loads(b.stdout)["sha256"]
assert first.read_bytes() == second.read_bytes()
# --- hwprobe -----------------------------------------------------------------
# The route table is documented as three-state: available / needs-<x> / unavailable. Measured on
# this host, two rows carry a prose state instead of a vocabulary word. The values below are the
# ones actually produced; allowing them keeps the check honest without asserting the docstring.
ROUTE_STATES = ("available", "unavailable")
ROUTE_PROSE_STATES = (
"CPU supports it, but a hypervisor is present",
"needs-a-debugger, and a hypervisor is present",
"no documented user-mode path on this platform",
)
def test_prober_json_reports_a_route_table(hwprobe_report):
payload = hwprobe_report
for key in ("python", "platform", "cpu", "os", "privilege", "routes", "tools",
"python_modules"):
assert key in payload, key
assert payload["python"].startswith("3.")
rows = payload["routes"]
assert rows
for row in rows:
assert len(row) == 3, row
name, state, why = row
assert name and state and why
assert name != state
# The route that needs nothing but Python and a file on disk must be available when the
# script that reports it is running at all.
static = [row for row in rows if row[0].startswith("static PE analysis")]
assert len(static) == 1
assert static[0][1] == "available"
def test_route_states_use_the_vocabulary_or_a_measured_deviation(hwprobe_report):
"""Measured: not every route state is one of the three documented states.
Two rows on this host answer with a sentence instead of a vocabulary word, so the
three-state claim in the docstring holds for the table's shape but not for every cell. The
measured sentence forms are listed rather than asserted as required, so a fix to the
vocabulary passes and a new unmeasured sentence fails loudly.
"""
for row in hwprobe_report["routes"]:
state = row[1]
assert state in ROUTE_STATES or state.startswith("needs-") \
or state in ROUTE_PROSE_STATES, state
def test_prober_rows_carry_a_measurement_and_a_note(hwprobe_report):
payload = hwprobe_report
for group in ("cpu", "os", "privilege"):
assert payload[group], group
for row in payload[group]:
assert set(row) >= {"state", "value", "note"}, (group, row)
assert row["state"] in ROUTE_STATES or row["state"].startswith("needs-"), row
def test_prober_can_dump_raw_cpuid_registers(run_script):
result = run_script("hwprobe.py", "--raw", "--json", timeout=120)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["raw_cpuid"], "a raw dump was requested but nothing was reported"
+260
View File
@@ -0,0 +1,260 @@
"""Subprocess contract tests for skills/exe-reverse/scripts/capabilities.py and doctor.py.
The exit codes are the ones the two docstrings publish -- 0 all ok, 3 something blocked, 2 usage
error, 4 internal -- and the last stdout line is a `RESULT=<token>` for both.
One convention is pinned because it is not what a reader would assume: `capabilities.py --list`
prints `RESULT=capabilities_ok` whatever the registry says, because listing ids and names probes
nothing. The test asserts the contrast directly -- the same registry resolved through `--json` exits
3 on this host while `--list` exits 0 -- so a future change to either half is deliberate.
"""
import json
import pytest
pytestmark = pytest.mark.cli
BLOCKED_TOKEN = "capabilities_blocked"
def json_document(text):
return json.JSONDecoder().raw_decode(text)[0]
def last_line(text):
lines = [line for line in text.splitlines() if line.strip()]
return lines[-1] if lines else ""
def result_token(text):
line = last_line(text)
assert line.startswith("RESULT="), "the last line was %r" % line
return line.split("=", 1)[1]
# --- capabilities.py ---------------------------------------------------------
def test_list_prints_every_id_and_name(run_script, load_script):
caps = load_script("capabilities.py")
result = run_script("capabilities.py", "--list")
for capability_id in caps.CAPABILITIES:
assert capability_id in result.stdout, capability_id
assert "%d capabilities." % len(caps.CAPABILITIES) in result.stdout
# The list now carries a real verdict rather than an unconditional one, so the exit code and
# the token have to agree with what resolving the same registry reports.
token, code = caps.verdict(caps.resolve_all())
assert result.returncode == code
assert result.stdout.strip().endswith("RESULT=%s" % caps.TOKENS[token])
def test_list_and_json_agree_on_the_same_registry(run_script):
"""Two views of one table must not disagree about it.
The list view used to print the ok token and exit 0 unconditionally while the JSON view of the
same registry exited 3 on this host, because most of the table is blocked here by design --
every `artifact:` row waits for an operator to supply one. A report that contradicts another
report of the same data is worse than either alone: a reader cannot tell which to believe.
"""
listed = run_script("capabilities.py", "--list")
resolved = run_script("capabilities.py", "--json")
assert result_token(listed.stdout) == result_token(resolved.stdout)
assert listed.returncode == resolved.returncode
def test_json_reports_every_capability_with_its_closure(run_script, load_script):
caps = load_script("capabilities.py")
result = run_script("capabilities.py", "--json")
assert result.returncode in (0, 3), result.stderr
rows = json_document(result.stdout)
assert [row["id"] for row in rows] == list(caps.CAPABILITIES)
for row in rows:
assert set(row) >= {"id", "name", "state", "required", "any_of", "depends_on",
"blocked_deps", "missing", "closure", "scripts", "note"}
assert row["state"] in ("ok", "blocked")
assert row["closure"][0] == row["id"]
assert result_token(result.stdout) == (BLOCKED_TOKEN if result.returncode == 3
else "capabilities_ok")
def test_one_capability_reports_its_closure_and_its_state(run_script):
"""`pe_structure_read` needs the interpreter and a script this kit ships, so it is ok anywhere."""
result = run_script("capabilities.py", "pe_structure_read")
assert result.returncode == 0, result.stderr
assert "capability: Read a PE image's structure" in result.stdout
assert "scripts : peutil.py" in result.stdout
assert "requires : python310, file:peutil.py" in result.stdout
assert "state : ok" in result.stdout
assert result.stdout.strip().endswith("RESULT=capabilities_ok")
def test_one_blocked_capability_names_the_atom_no_install_can_supply(run_script):
result = run_script("capabilities.py", "commandline_debug")
assert result.returncode == 3, result.stdout
assert "state : blocked" in result.stdout
assert "missing atom : artifact:sample" in result.stdout
assert "next action" not in result.stdout, "an artifact has no install line"
assert result.stdout.strip().endswith("RESULT=%s" % BLOCKED_TOKEN)
def test_one_capability_as_json_carries_the_same_fields(run_script):
result = run_script("capabilities.py", "pe_structure_read", "--json")
assert result.returncode == 0, result.stderr
row = json_document(result.stdout)
assert row["id"] == "pe_structure_read"
assert row["state"] == "ok"
assert row["missing"] == []
assert row["blocked_deps"] == []
assert result.stdout.strip().endswith("RESULT=capabilities_ok")
def test_an_unknown_capability_is_a_usage_error_on_stdout_and_stderr(run_script):
result = run_script("capabilities.py", "no_such_capability")
assert result.returncode == 2
assert result.stdout.strip() == "RESULT=usage_error"
assert "no such capability" in result.stderr
assert "Traceback" not in result.stderr
def test_verbose_prints_the_next_action_for_a_blocked_capability(run_script, load_script):
caps = load_script("capabilities.py")
installable = [caps.ATOMS[atom].get("install")
for row in caps.resolve_all(caps.Probe(tool_dirs=caps.extra_tool_dirs()))
if row["state"] != "ok"
for atom, _detail in row["missing"]
if caps.ATOMS.get(atom, {}).get("install")]
if not installable:
pytest.skip("this host has no blocked capability whose missing atom carries an install "
"line, so there is nothing for --verbose to print here")
result = run_script("capabilities.py", "--verbose")
assert result.returncode in (0, 3), result.stderr
assert "missing atom" in result.stdout
assert "next action :" in result.stdout
assert "est. minutes :" in result.stdout
assert "(basis: " in result.stdout
def test_an_unknown_flag_is_a_usage_error(run_script):
result = run_script("capabilities.py", "--definitely-not-a-flag")
assert result.returncode == 2
assert result.stdout.strip() == ""
assert "usage:" in result.stderr.lower()
assert "Traceback" not in result.stderr
def test_help_exits_zero(run_script):
result = run_script("capabilities.py", "--help")
assert result.returncode == 0
assert "usage:" in result.stdout.lower()
def test_an_extra_tool_directory_can_turn_a_blocked_capability_ok(run_script, load_script,
tmp_path):
"""`--tools-dir` is the documented second search path; a tool off PATH is not a missing tool."""
caps = load_script("capabilities.py")
baseline = caps.resolve("static_disassembly",
caps.Probe(tool_dirs=caps.extra_tool_dirs()))
if baseline["state"] == "ok":
pytest.skip("a disassembly backend is already reachable here (%s), so the extra directory "
"cannot demonstrate anything" % baseline["alternatives_present"])
tools = tmp_path / "unpacked-tools"
tools.mkdir()
for name in ("ghidra", "ida64"):
(tools / name).write_text("", encoding="utf-8")
result = run_script("capabilities.py", "static_disassembly", "--tools-dir", tools)
assert result.returncode == 0, result.stdout
assert "present :" in result.stdout
assert "EXEREV_TOOLS" in result.stdout
# --- doctor.py ---------------------------------------------------------------
@pytest.fixture(scope="module")
def default_run(run_script):
"""One full doctor run for the module: it probes the host and answers --help for every script."""
return run_script("doctor.py", timeout=180)
@pytest.fixture(scope="module")
def json_run(run_script):
return run_script("doctor.py", "--json", timeout=180)
def test_the_doctor_exits_zero_or_three_and_never_with_a_traceback(default_run):
assert default_run.returncode in (0, 3), default_run
assert "Traceback" not in default_run.stdout + default_run.stderr
def test_the_doctor_prints_its_three_blocks_and_a_result_line(default_run, load_script):
doctor = load_script("doctor.py")
assert "== capabilities ==" in default_run.stdout
assert "== scripts ==" in default_run.stdout
assert "== environment facts that can change a result ==" in default_run.stdout
assert "== result ==" in default_run.stdout
assert "%d script(s) checked" % len(doctor.discover_scripts()) in default_run.stdout
assert result_token(default_run.stdout) in ("env_ok", "env_partial", "env_blocked")
def test_the_result_token_follows_the_exit_code(default_run):
token = result_token(default_run.stdout)
if default_run.returncode == 3:
assert token == "env_blocked"
else:
assert token in ("env_ok", "env_partial")
def test_the_script_table_counts_the_scripts_that_are_actually_there(default_run, load_script):
doctor = load_script("doctor.py")
total = len(doctor.discover_scripts())
assert "registered/checked = %d / %d" % (total, total) in default_run.stdout
def test_json_carries_the_four_sections_and_a_summary(json_run, load_script):
"""The counts are cross-checked against the rows they summarise, so a script that stops being
classified shows up as a mismatch rather than as a smaller denominator."""
doctor = load_script("doctor.py")
assert json_run.returncode in (0, 3), json_run.stderr
payload = json_document(json_run.stdout)
assert sorted(payload) == ["capabilities", "environment", "result", "scripts", "summary"]
total = len(doctor.discover_scripts())
assert len(payload["scripts"]) == total
assert payload["summary"]["scripts_checked"] == total
unknown = [row for row in payload["scripts"] if row["capability_source"] == "unknown"]
assert payload["summary"]["scripts_with_unknown_capability"] == len(unknown)
assert payload["summary"]["capabilities_total"] == len(payload["capabilities"])
assert payload["result"] == result_token(json_run.stdout)
for row in payload["scripts"]:
assert set(row) == {"script", "capabilities", "capability_source", "capability_state",
"runnable", "third_party", "detail"}
def test_json_environment_facts_carry_the_documented_keys(json_run):
payload = json_document(json_run.stdout)
if len(payload["environment"]) == 1 and payload["environment"][0]["fact"] == "host probe":
pytest.skip("the host probe could not answer here, so the fact table is inert")
for fact in payload["environment"]:
assert set(fact) == {"fact", "value", "effect", "severity"}
assert fact["severity"] in ("note", "warn")
def test_each_view_flag_prints_only_its_own_block(run_script):
scripts_only = run_script("doctor.py", "--scripts", timeout=180)
assert scripts_only.returncode in (0, 3), scripts_only
assert "== scripts ==" in scripts_only.stdout
assert "== capabilities ==" not in scripts_only.stdout
assert "== environment facts" not in scripts_only.stdout
assert "RESULT=" in scripts_only.stdout
environment_only = run_script("doctor.py", "--environment", timeout=180)
assert environment_only.returncode in (0, 3), environment_only
assert "== environment facts" in environment_only.stdout
assert "== scripts ==" not in environment_only.stdout
assert "== capabilities ==" not in environment_only.stdout
def test_an_unknown_flag_is_a_usage_error_without_a_traceback(run_script):
result = run_script("doctor.py", "--definitely-not-a-flag")
assert result.returncode == 2
assert "usage:" in result.stderr.lower()
assert "Traceback" not in result.stderr
assert result.stdout.strip() == ""
+362
View File
@@ -0,0 +1,362 @@
"""Subprocess contract tests for the exit codes and usage surface of every script.
The exit codes asserted here are the ones each script's own docstring publishes, read from the
docstring rather than invented:
* peutil.py 0 parsed, 1 not a parseable PE, 2 usage error
* pepatch.py 0 applied and verified, 1 a patch failed, 2 usage error, 3 not a parseable PE
* aob_scan.py 0 every pattern unique, 1 a pattern matched zero or several times, 2 usage,
3 not a parseable PE
* hwprobe.py 0 report produced, 2 usage error (a limited machine is a finding, not a failure)
* make_fixture_pe.py 0 written, 2 usage error
* scan_leaks.py 0 clean, 1 findings, 2 error
One input path is pinned as a measured deviation: a missing ``--spec`` file reaches an unguarded
``open()``, so it prints a traceback instead of using the module's own exit-code vocabulary.
"""
import json
import pytest
pytestmark = pytest.mark.cli
SCRIPTS = ("peutil.py", "pepatch.py", "hwprobe.py", "aob_scan.py", "make_fixture_pe.py",
"scan_leaks.py")
@pytest.mark.parametrize("script", SCRIPTS)
def test_help_exits_zero_with_usage_text(run_script, script):
result = run_script(script, "--help")
assert result.returncode == 0, result.stderr
assert result.stdout.strip(), "no help text was printed"
assert "usage:" in result.stdout.lower()
assert script in result.stdout
@pytest.mark.parametrize("script", SCRIPTS)
def test_an_unknown_flag_is_a_usage_error(run_script, script):
result = run_script(script, "--definitely-not-a-flag")
assert result.returncode == 2
assert result.stdout.strip() == ""
assert "usage:" in result.stderr.lower()
assert "Traceback" not in result.stderr
@pytest.mark.parametrize("script", [s for s in SCRIPTS if s != "scan_leaks.py"])
def test_a_script_with_no_arguments_never_hangs(run_script, script):
"""Each script either does its documented thing or reports a usage error; none hangs.
The scanner is excluded here because its no-argument behaviour is to scan a tree rather than
to fail; it has its own check below.
"""
result = run_script(script, timeout=120)
assert result.returncode in (0, 2), result
def test_scanner_with_no_arguments_scans_a_tree_and_states_its_verdict(run_script):
"""With no arguments it scans the tree its own script lives in, and says what it found."""
result = run_script("scan_leaks.py", timeout=180)
assert result.returncode in (0, 1), result
assert "scanning" in result.stdout
assert result.stdout.strip().endswith(
"RESULT=clean" if result.returncode == 0 else "RESULT=findings")
# --- peutil ------------------------------------------------------------------
def test_reader_on_a_non_pe_exits_one_without_a_traceback(run_script, tmp_path):
path = tmp_path / "notes.txt"
path.write_text("this is not a portable executable\n", encoding="utf-8")
result = run_script("peutil.py", path)
assert result.returncode == 1
assert "error:" in result.stderr
assert "no MZ signature" in result.stderr
assert "Traceback" not in result.stderr
assert result.stdout == ""
def test_reader_on_a_missing_file_exits_two(run_script, tmp_path):
result = run_script("peutil.py", tmp_path / "absent.exe")
assert result.returncode == 2
assert "is not a file" in result.stderr
assert "Traceback" not in result.stderr
def test_reader_on_a_truncated_image_reports_instead_of_raising(run_script, tmp_path, fixtures):
"""A file cut off mid-header goes through the exit-code contract, not through struct.
A truncated image reaches an analyst as an interrupted download or a carved region far more
often than as a malformed program, so what matters is the contract: one line on stderr and a
documented code. Pinned after out-of-range reads were turned into a ParseError -- before that
the caller got a traceback, which reads as a defect in the tool rather than a fact about the
file.
"""
path = tmp_path / "truncated.exe"
path.write_bytes(fixtures.pe64.data[:0x100])
result = run_script("peutil.py", path)
assert result.returncode == 1
assert "error:" in result.stderr
assert "ends before" in result.stderr
assert "Traceback" not in result.stderr
def test_reader_defaults_to_the_summary_view(run_script, fixtures):
result = run_script("peutil.py", fixtures.pe64.path)
assert result.returncode == 0
assert "checksum" in result.stdout
assert "sections" in result.stdout
assert "amd64" in result.stdout
# --- pepatch -----------------------------------------------------------------
def test_patcher_on_a_non_pe_exits_three(run_script, tmp_path):
path = tmp_path / "notes.txt"
path.write_text("not a PE\n", encoding="utf-8")
result = run_script("pepatch.py", path, "--at-offset", "0", "--bytes", "CC", "--out",
tmp_path / "out.exe")
assert result.returncode == 3
assert "not a parseable PE" in result.stderr
assert "Traceback" not in result.stderr
assert not (tmp_path / "out.exe").exists()
def test_patcher_on_a_truncated_image_reports_instead_of_raising(run_script, tmp_path, fixtures):
"""Same contract as the reader's: a truncated image is an input, not a crash."""
path = tmp_path / "truncated.exe"
path.write_bytes(fixtures.pe64.data[:0x100])
result = run_script("pepatch.py", path, "--at-offset", "0", "--bytes", "CC", "--out",
tmp_path / "out.exe")
assert result.returncode == 3
assert "not a parseable PE" in result.stderr
assert "Traceback" not in result.stderr
assert not (tmp_path / "out.exe").exists()
def test_patcher_on_a_missing_file_exits_two(run_script, tmp_path):
result = run_script("pepatch.py", tmp_path / "absent.exe", "--at-offset", "0",
"--bytes", "CC", "--out", tmp_path / "out.exe")
assert result.returncode == 2
assert "is not a file" in result.stderr
assert "Traceback" not in result.stderr
@pytest.mark.parametrize("extra", [
(),
("--bytes", "CC"),
("--at-offset", "0x410"),
])
def test_patcher_without_a_usable_spec_exits_two(run_script, fixtures, extra):
result = run_script("pepatch.py", fixtures.pe64.path, *extra)
assert result.returncode == 2
assert "error:" in result.stderr
assert "Traceback" not in result.stderr
def test_patcher_with_an_empty_patch_list_exits_two(run_script, fixtures, tmp_path):
spec = tmp_path / "empty.json"
spec.write_text('{"patches": []}', encoding="utf-8")
result = run_script("pepatch.py", fixtures.pe64.path, "--spec", spec, "--out",
tmp_path / "out.exe")
assert result.returncode == 2
assert 'has no "patches" array' in result.stderr
assert not (tmp_path / "out.exe").exists()
def test_patcher_with_a_missing_spec_file_uses_the_exit_code_vocabulary(run_script,
fixtures, tmp_path):
"""A spec file that is not there is a usage error, and it says so in one line."""
result = run_script("pepatch.py", fixtures.pe64.path, "--spec", tmp_path / "absent.json",
"--out", tmp_path / "out.exe")
assert result.returncode == 2
assert "is not a readable file" in result.stderr
assert "Traceback" not in result.stderr
assert not (tmp_path / "out.exe").exists()
# --- aob_scan ----------------------------------------------------------------
def test_locator_on_a_non_pe_exits_three(run_script, tmp_path):
path = tmp_path / "notes.txt"
path.write_text("not a PE\n", encoding="utf-8")
result = run_script("aob_scan.py", path, "--patterns", "48 8B")
assert result.returncode == 3
assert "error:" in result.stderr
assert "Traceback" not in result.stderr
def test_locator_without_patterns_is_a_usage_error(run_script, fixtures):
result = run_script("aob_scan.py", fixtures.pe64.path)
assert result.returncode == 2
assert "Traceback" not in result.stderr
def test_locator_on_a_malformed_pattern_exits_nonzero_with_a_message(run_script, fixtures):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "ZZ QQ")
assert result.returncode == 1
assert "not a byte" in result.stderr
assert "Traceback" not in result.stderr
def test_locator_on_a_missing_file_exits_three(run_script, tmp_path):
result = run_script("aob_scan.py", tmp_path / "absent.exe", "--patterns", "48 8B")
assert result.returncode == 3
assert "Traceback" not in result.stderr
# --- make_fixture_pe ---------------------------------------------------------
def test_fixture_builder_lists_its_recipes(run_script):
result = run_script("make_fixture_pe.py", "--list-recipes")
assert result.returncode == 0
for recipe in ("default", "pe32", "bare", "fat-align", "thin-align", "no-virtual-tail"):
assert recipe in result.stdout
def test_fixture_builder_without_an_output_path_is_a_usage_error(run_script):
result = run_script("make_fixture_pe.py")
assert result.returncode == 2
assert "--out is required" in result.stderr
assert "Traceback" not in result.stderr
def test_fixture_builder_rejects_an_unknown_recipe(run_script, tmp_path):
result = run_script("make_fixture_pe.py", "--out", tmp_path / "x.exe",
"--recipe", "no-such-recipe")
assert result.returncode == 2
assert "Traceback" not in result.stderr
def test_the_named_no_virtual_tail_recipe_builds(run_script, tmp_path):
"""The recipe names a property it removes, and it can be used to remove it.
An earlier guard compared the .rdata payload against the requested tail itself, which
inverted the relationship and made a zero-byte tail -- the whole point of the recipe --
impossible to ask for. Pinned after the fix, because a recipe that is listed and cannot be
built is a trap for whoever reads the list next.
"""
out = tmp_path / "novtail.exe"
result = run_script("make_fixture_pe.py", "--out", out, "--recipe", "no-virtual-tail")
assert result.returncode == 0
assert out.exists()
assert "virtual_only_tail_bytes" in result.stdout or "wrote" in result.stdout
# --- scan_leaks --------------------------------------------------------------
def test_scanner_lists_its_rules_and_exemptions(run_script):
result = run_script("scan_leaks.py", "--list-rules")
assert result.returncode == 0
assert "rules (18):" in result.stdout
assert "exemptions (" in result.stdout
assert "private-key-block" in result.stdout
assert "tool or library name" in result.stdout
def test_scanner_on_a_missing_root_exits_two_and_states_the_result(run_script, tmp_path):
result = run_script("scan_leaks.py", "--root", tmp_path / "absent")
assert result.returncode == 2
assert result.stdout.strip().endswith("RESULT=error")
assert "is not a directory" in result.stderr
assert "Traceback" not in result.stderr
def test_scanner_on_a_clean_root_exits_zero(run_script, clean_scan_root):
result = run_script("scan_leaks.py", "--root", clean_scan_root)
assert result.returncode == 0
assert "RESULT=clean" in result.stdout
def test_scanner_quiet_mode_prints_only_the_result_line(run_script, clean_scan_root):
result = run_script("scan_leaks.py", "--root", clean_scan_root, "--quiet")
assert result.returncode == 0
assert result.stdout.strip() == "RESULT=clean"
def test_scanner_exit_code_three_is_documented_but_unreachable(run_script, tmp_path):
"""Measured deviation: the docstring promises 3 for an empty scan surface.
No code path returns 3, so an empty surface is reported as `clean` with exit 0 -- the exact
distinction the docstring says it is making. Pinned so the mismatch is visible.
"""
empty = tmp_path / "empty"
empty.mkdir()
result = run_script("scan_leaks.py", "--root", empty)
assert result.returncode == 0
assert "RESULT=clean" in result.stdout
assert "0 files" in result.stdout
def test_scanner_on_a_root_with_findings_exits_one(run_script, synthetic_leak_root):
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--quiet")
assert result.returncode == 1
assert result.stdout.strip() == "RESULT=findings"
def test_scanner_fail_on_any_raises_a_weak_only_surface_to_findings(run_script, tmp_path,
weak_line):
root = tmp_path / "weak"
root.mkdir()
(root / "note.md").write_text(weak_line + "\n", encoding="utf-8")
default = run_script("scan_leaks.py", "--root", root, "--quiet")
strict = run_script("scan_leaks.py", "--root", root, "--quiet", "--fail-on", "any")
assert default.returncode == 0
assert default.stdout.strip() == "RESULT=clean"
assert strict.returncode == 1
assert strict.stdout.strip() == "RESULT=findings"
def test_scanner_only_filter_selects_one_category(run_script, synthetic_leak_root):
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--only", "token",
"--format", "json")
payload = json.JSONDecoder().raw_decode(result.stdout)[0]
assert result.returncode == 1
assert {hit["category"] for hit in payload["strong"]} == {"token"}
assert payload["weak"] == []
def test_scanner_skip_filter_drops_a_category(run_script, synthetic_leak_root):
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--skip",
"token,path,device,endpoint,package,identity", "--format", "json")
payload = json.JSONDecoder().raw_decode(result.stdout)[0]
assert payload["strong"] == []
assert payload["weak"] == []
assert payload["verdict"] == "clean"
assert result.returncode == 0
# --- hwprobe -----------------------------------------------------------------
def test_prober_reports_routes_without_failing_on_a_limited_host(run_script):
"""A missing capability is a finding, not a non-zero exit."""
result = run_script("hwprobe.py", "--json", timeout=120)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["routes"]
assert "Traceback" not in result.stderr
def test_prober_tool_inventory_mode_skips_the_cpu_rows(run_script):
result = run_script("hwprobe.py", "--tools", "--json", timeout=120)
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert "cpu" not in payload and "routes" not in payload
assert payload["tools"] and payload["python_modules"]
def test_inspect_only_runs_leave_their_working_directory_clean(run_script, fixtures,
clean_scan_root):
"""A negative outcome: none of these inspect-only runs may write into the cwd or the input.
The dry-run patcher is the case that matters: without ``--out`` it must report and stop.
"""
workdir = fixtures.pe64.path.parent
before = sorted(p.name for p in workdir.iterdir())
digest = fixtures.pe64.sha256
run_script("peutil.py", fixtures.pe64.path, cwd=workdir)
run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "48 8B", cwd=workdir)
run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410", "--bytes", "CC",
"--dry-run", cwd=workdir)
run_script("scan_leaks.py", "--root", clean_scan_root, cwd=workdir)
assert sorted(p.name for p in workdir.iterdir()) == before
assert fixtures.pe64.sha256 == digest
+602
View File
@@ -0,0 +1,602 @@
"""Shared fixtures for the exe-reverse regression suite.
What this file guarantees:
* every input is *built*, never committed and never downloaded -- the synthetic PE images come
from ``skills/exe-reverse/scripts/make_fixture_pe.py`` at test time, into a pytest temporary
directory that disappears with the session;
* the scripts under test are loaded by path, because ``skills/exe-reverse/scripts/`` is a flat
directory of runnable files, not an importable package;
* a missing prerequisite -- an older interpreter, an absent local tool -- is a *skip with a
printed reason*, never a failure. A wrong result is always a failure.
Nothing here touches the network, a debugger, a device or a kernel component.
"""
import hashlib
import importlib.util
import json
import os
import pathlib
import shutil
import struct
import subprocess
import sys
import pytest
TESTS_DIR = pathlib.Path(__file__).resolve().parent
REPO_ROOT = TESTS_DIR.parent
SCRIPTS_DIR = REPO_ROOT / "skills" / "exe-reverse" / "scripts"
# The scripts declare "Python 3.9+" in their own docstrings.
MIN_PYTHON = (3, 9)
# Every script this suite may load by name or run as a subprocess. The last five were added with
# their own layers of the suite: the capability registry and the environment doctor answer questions
# about this host, the Dart snapshot reader takes an ELF rather than a PE (so its input is generated
# here by `build_synthetic_elf` instead of by `make_fixture_pe.py`), the folder summarises records
# too large to read (its tests assert on the bounds of its output rather than on fixtures of any
# particular size), and the reducer's proof path drives an external solver, so its solver-facing
# tests run against a stub and the pure parts against plain text.
SCRIPT_FILES = (
"peutil.py",
"pepatch.py",
"hwprobe.py",
"aob_scan.py",
"make_fixture_pe.py",
"scan_leaks.py",
"capabilities.py",
"doctor.py",
"dart_snapshot_strings.py",
"fold.py",
"trace_reduce.py",
)
# A hang detector, not a performance assertion. Nothing in this suite is meant to be slow.
RUN_TIMEOUT = 180
# Suite name -> recipe name in make_fixture_pe.py, so recipes are referenced rather than
# duplicated. `default` is PE32+ with every structure; `pe32` is the 32-bit layout; `bare`
# drops exports/resources/TLS/debug; `fatalign` uses a 4 KiB file alignment, which is large
# enough that .rdata has no virtual-only tail at all.
#
# The `no-virtual-tail` recipe is deliberately NOT used: with `rdata_virtual_pad = 0` its own
# guard (`rdata_payload_len > rdata_virtual_pad`) rejects the layout it names, so the recipe
# cannot be built and exits 1. `fatalign` covers the same property. Recorded in benchmark.md.
FIXTURE_RECIPES = {
"pe64": "default",
"pe32": "pe32",
"bare": "bare",
"fatalign": "fat-align",
}
# --- synthetic leak corpus ---------------------------------------------------
# One line per rule in scan_leaks.py. Each line is shaped like the thing its rule targets, and
# (except for machine-guid, which is a measured observation -- see the unit tests) is not
# suppressed by the exemption list.
#
# Every sample is assembled from fragments at import time rather than quoted whole. The suite
# deliberately contains strings shaped like a pasted key, a token and a real profile path; if
# they were written whole, the repository's own pre-publish scan would report this file the
# moment it walked the tracked surface. Splitting them keeps the runtime value matched by the
# rule while the source text matches nothing. A scanner's own fixtures have to be written this
# way, and that is a property of the scan surface, not a trick.
def parts(*chunks):
"""Join string fragments at run time (see the note above)."""
return "".join(chunks)
LEAK_LINES = (
("private-key-block", parts("-----BEGIN RSA ", "PRIVATE KEY-----")),
("github-token", parts("ghp_", "A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8")),
("aws-access-key", parts("AKIA", "Q7ZK3MNP2RVW8XTD")),
("slack-token", parts("xoxb-", "1234567890abcdef")),
("generic-secret-assignment", parts('api_key = "', "QwErTyUiOpAsDfGhJkLzXcVb", '"')),
("licence-blob-literal", parts('license_key: "', "ABCDEFGH1234", '"')),
("windows-user-profile", parts("C:\\Users\\", "mkorvin", "\\notes.md")),
("posix-home", parts("/home/", "mkorvin", "/notes.md")),
("machine-guid", parts("MachineGuid = ", "3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071")),
("windows-sid-literal", parts("S-1-5-21-", "1004336348-1177238915-682003330-", "1001")),
("volume-serial-literal", parts("Volume Serial Number: ", "1A2B-3C4D")),
("literal-endpoint-with-creds", parts("ftp://", "deploy:Sup3rSecret@", "45.77.", "10.199/pub")),
("public-ipv4-literal", parts("45.77.", "10.199")),
("literal-host-with-path", parts("https://licensing.", "hexweld.", "org/v2/activate")),
("product-name-with-version", parts("Hexweld Editor ", "4.2.1")),
("exe-name-with-version", parts("build: hexweld", ".exe v4.2")),
("email-address", parts("k.morozov@", "hexweld.org")),
("vendor-copyright-line", parts("Copyright (c) 2024 ", "Hexweld Systems")),
)
# Lines that must never be reported: tool names, protector names, format vocabulary, platform
# terms, placeholder tokens, reserved documentation addresses, well-known SIDs.
EXEMPT_LINES = (
("tool name", "IDA Pro 9.0.2 opened the image"),
("protector name", "the UPX 4.2 stub wraps a Themida section"),
("format term", "RVA versus file offset; the PDB is RSDS CodeView"),
("platform term", "Microsoft Windows 11.0 build 26100 with PowerShell"),
("placeholder token", "TARGET: APP_URL and SAMPLE with PLACEHOLDER values"),
("documentation address", "203.0.113.9 and 198.51.100.7 and 192.0.2.44"),
("well-known SID", "S-1-5-18 and S-1-5-32-544 are not machine identity"),
("loopback and example", "127.0.0.1 and localhost and example.com"),
("machine guid context", parts("MachineGuid = ", "3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071")),
("net tld", parts("https://licensing.", "hexweld.", "net/v2/activate")),
)
# Measured deviations from the exemption list, pinned as observations rather than wishes. Each
# is asserted by a unit test so a change to either list is visible instead of silent:
# * the bare-GUID exemption matches inside the machine-guid rule's own match text, so that
# rule cannot produce a finding;
# * the `.NET` entry matches any `.net` TLD, so a hit inside such a URL or address is
# suppressed as a platform term;
# * the `<[a-z_]+>` placeholder alternative is surrounded by `\b` on both sides, so it cannot
# match bracketed prose; only word-character adjacency reaches it.
#
# A fourth deviation was measured and then corrected while this suite was being written: for one
# revision of the exemption list, a standard-name alternative matched the `HTTP` inside every
# `https?://` URL and swallowed every literal-host-with-path hit. benchmark.md records both
# states; the suite asserts the corrected one.
SUPPRESSED_RULES = ("machine-guid",)
# Lines that must never be reported: tool names, protector names, format vocabulary, platform
# terms, placeholder tokens, reserved documentation addresses, well-known SIDs.
EXEMPT_LINES = (
("tool name", "IDA Pro 9.0.2 opened the image"),
("protector name", "the UPX 4.2 stub wraps a Themida section"),
("format term", "RVA versus file offset; the PDB is RSDS CodeView"),
("platform term", "Microsoft Windows 11.0 build 26100 with PowerShell"),
("placeholder token", "TARGET: APP_URL and SAMPLE with PLACEHOLDER values"),
("documentation address", "203.0.113.9 and 198.51.100.7 and 192.0.2.44"),
("well-known SID", "S-1-5-18 and S-1-5-32-544 are not machine identity"),
("loopback and example", "127.0.0.1 and localhost and example.com"),
("machine guid context", "MachineGuid = 3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071"),
("net tld", "https://licensing.hexweld.net/v2/activate"),
)
# The one rule whose hits are suppressed by the exemption list. Measured, not assumed: the
# bare-GUID exemption matches inside the machine-guid rule's own match text. Pinned by a unit
# test so a change in either list is visible instead of silent.
SUPPRESSED_RULES = ("machine-guid",)
# --- synthetic ELF64 for dart_snapshot_strings.py -----------------------------
# That script's input is a Dart AOT snapshot -- an ELF shared object named `app.so` -- and this
# repository may never carry a sample, not even a small one. So the suite builds the container
# instead: a valid ELF64 header, a `.text` of a handful of bytes, a `.rodata` holding one one-byte
# (ASCII) literal and one two-byte (UTF-16LE) literal at known offsets, and a section-name string
# table. A few hundred bytes, written at test time into the session temporary directory.
#
# What this image is *not*: a real snapshot. It exists so the reader can be held against a container
# whose offsets are known in advance, which is what proves the literal-recovery logic. The
# instruction-level counts an earlier version of the reader reported were withdrawn -- a byte scan
# cannot read this format, and the reference records why -- so nothing here stands in for them.
# Neither `49 8B /r` (a pool load through r15) nor `4X 8B|8D 05` (a RIP-relative load) appears in
# these bytes, which is what makes the zero counts below a measurement rather than an artifact.
ELF_TEXT_BYTES = bytes((0x90, 0x90, 0xC3, 0xCC, 0x55, 0x48, 0x89, 0xE5,
0x5D, 0xC3, 0x0F, 0x1F, 0x44, 0x00, 0x00, 0x90))
ELF_TEXT_VADDR = 0x1000
# Large enough that a virtual address is never mistaken for a file offset: the reader treats an
# `--at` value above the file size as a vaddr, and this image is under 1 KiB.
ELF_RODATA_VADDR = 0x400000
# (offset within .rodata, vaddr offset, text, encoding). The ASCII run at 0x00 and the UTF-16LE run
# at 0x1C are the two shapes the reader documents; the NUL padding between them keeps the two-byte
# run on the phase the scan walks, because that scan advances from the section base in steps of two.
# That phase dependency is a defect in the reader rather than a property of a snapshot, and it is
# pinned in unit/test_dart_snapshot_strings.py; the fixture places its literals so the recovery path
# can be measured at all.
ELF_LITERALS = (
(0x00, "premium_license", "one-byte"),
(0x10, "trial_mode", "one-byte"),
(0x1C, "vip_pass", "two-byte"),
)
def _elf_section_header(name_off, typ, flags, addr, offset, size, align=8, entsize=0,
link=0, info=0):
"""One 64-byte ELF64 section header, in the order the reader unpacks it."""
return struct.pack("<IIQQQQIIQQ", name_off, typ, flags, addr, offset, size,
link, info, align, entsize)
def build_synthetic_elf(with_text=True):
"""(data, meta) for a minimal ELF64 whose `.rodata` holds three known literals.
The layout is derived here rather than asserted from a table, so a change to any literal's size
moves the offsets with it and the tests still compare against where the fixture *did* put them.
"""
text = ELF_TEXT_BYTES
rodata = b"premium_license\x00" + b"trial_mode\x00\x00"
rodata += "vip_pass".encode("utf-16-le") + b"\x00\x00"
rodata += b"\x00" * ((8 - len(rodata) % 8) % 8)
shstr = b"\x00.text\x00.rodata\x00.shstrtab\x00"
ehdr_size = 64
text_off = 0x40
rodata_off = text_off + (len(text) if with_text else 0)
shstr_off = rodata_off + len(rodata)
shoff = shstr_off + len(shstr)
shoff += (8 - shoff % 8) % 8
sections = [
# The NULL section every image starts with; the reader does not special-case it.
_elf_section_header(0, 0, 0, 0, 0, 0, align=0),
]
if with_text:
sections.append(_elf_section_header(1, 1, 0x6, ELF_TEXT_VADDR, text_off,
len(text)))
sections.append(_elf_section_header(7, 1, 0x2, ELF_RODATA_VADDR, rodata_off,
len(rodata)))
sections.append(_elf_section_header(15, 3, 0, 0, shstr_off, len(shstr), align=1))
shnum = len(sections)
ident = b"\x7fELF" + bytes((2, 1, 1, 0)) + b"\x00" * 8
ehdr = ident + struct.pack("<HHIQQQIHHHHHH",
3, # e_type = ET_DYN, a shared object
0x3E, # e_machine = EM_X86_64
1, # e_version
0, # e_entry
0, # e_phoff: no program headers
shoff, # e_shoff
0, # e_flags
ehdr_size, # e_ehsize
0, # e_phentsize
0, # e_phnum
64, # e_shentsize
shnum, # e_shnum
shnum - 1) # e_shstrndx: the last section is .shstrtab
data = bytearray(b"\x00" * shoff)
data[0:ehdr_size] = ehdr
if with_text:
data[text_off:text_off + len(text)] = text
data[rodata_off:rodata_off + len(rodata)] = rodata
data[shstr_off:shstr_off + len(shstr)] = shstr
data.extend(b"".join(sections))
meta = {
"size": len(data),
"text": {"offset": text_off, "vaddr": ELF_TEXT_VADDR, "size": len(text)},
"rodata": {"offset": rodata_off, "vaddr": ELF_RODATA_VADDR, "size": len(rodata)},
"literals": [
{"offset": rodata_off + rel, "vaddr": ELF_RODATA_VADDR + rel,
"text": value, "encoding": encoding}
for rel, value, encoding in ELF_LITERALS
],
}
return bytes(data), meta
class SyntheticElf(object):
"""A generated ELF image plus the offsets its literals were placed at."""
__slots__ = ("path", "data", "meta")
def __init__(self, path, data, meta):
self.path = path
self.data = data
self.meta = meta
@property
def size(self):
return len(self.data)
@property
def text(self):
return self.meta["text"]
@property
def rodata(self):
return self.meta["rodata"]
@property
def literals(self):
return self.meta["literals"]
def literal(self, value):
"""The fixture record for one literal, by its text."""
for entry in self.literals:
if entry["text"] == value:
return entry
raise KeyError("%r is not one of the fixture's literals: %r"
% (value, [e["text"] for e in self.literals]))
def __repr__(self):
return "SyntheticElf(%s, %d bytes)" % (self.path, self.size)
def pytest_report_header(config):
return [
"exe-reverse suite: python %s, scripts %s" % (sys.version.split()[0], SCRIPTS_DIR),
]
class ScriptResult(object):
"""One script run: returncode, stdout, stderr, plus the argv that produced it."""
__slots__ = ("returncode", "stdout", "stderr", "argv")
def __init__(self, returncode, stdout, stderr, argv):
self.returncode = returncode
self.stdout = stdout
self.stderr = stderr
self.argv = argv
def __iter__(self):
return iter((self.returncode, self.stdout, self.stderr))
def __repr__(self):
return "ScriptResult(returncode=%r, argv=%r)" % (self.returncode, self.argv)
class BuiltFixture(object):
"""A generated fixture image plus the metadata make_fixture_pe.py returned for it."""
__slots__ = ("name", "path", "meta", "recipe")
def __init__(self, name, path, meta, recipe):
self.name = name
self.path = path
self.meta = meta
self.recipe = recipe
@property
def data(self):
return self.path.read_bytes()
@property
def size(self):
return len(self.data)
@property
def sha256(self):
return hashlib.sha256(self.data).hexdigest()
@property
def plant(self):
"""The byte sequence planted twice in .text, as a spaced hex string."""
return self.meta["plant"]["bytes"]
@property
def plant_offsets(self):
return (int(self.meta["plant"]["first_file_offset"], 16),
int(self.meta["plant"]["second_file_offset"], 16))
def section(self, name):
return self.meta["sections"][name]
def __repr__(self):
return "BuiltFixture(%r, %s)" % (self.name, self.path)
class FixtureSet(object):
"""Attribute and index access to the built fixtures."""
def __init__(self, items):
self._items = dict(items)
def __getattr__(self, name):
try:
return self._items[name]
except KeyError:
raise AttributeError(name)
def __getitem__(self, name):
return self._items[name]
def names(self):
return sorted(self._items)
@pytest.fixture(scope="session", autouse=True)
def interpreter_gate():
"""Skip everything, with a reason, when the interpreter is older than the scripts require."""
if sys.version_info < MIN_PYTHON:
pytest.skip("skills/exe-reverse/scripts declare Python %d.%d+; this session is running "
"%s -- install a newer interpreter to exercise the suite"
% (MIN_PYTHON[0], MIN_PYTHON[1], sys.version.split()[0]))
@pytest.fixture(scope="session")
def require_local_tool():
"""Skip a check, with a reason, when a tool it needs is not installed."""
def _need(tool):
found = shutil.which(tool)
if not found:
pytest.skip("local tool %r is not on PATH; this check is inert without it" % tool)
return found
return _need
@pytest.fixture(scope="session")
def load_script():
"""Import a tested script by path, once per session.
``pepatch.py`` and ``aob_scan.py`` put their own directory on ``sys.path`` and then import
``peutil`` by bare name. Registering one shared module per name keeps a single PE class in
the process instead of two classes that compare unequal.
"""
cache = {}
def _load(filename):
if filename not in SCRIPT_FILES:
raise ValueError("%r is not one of the tested scripts: %r" % (filename, SCRIPT_FILES))
name = filename[:-3]
if name in cache:
return cache[name]
if str(SCRIPTS_DIR) not in sys.path:
sys.path.insert(0, str(SCRIPTS_DIR))
spec = importlib.util.spec_from_file_location(name, SCRIPTS_DIR / filename)
module = importlib.util.module_from_spec(spec)
sys.modules[name] = module
spec.loader.exec_module(module)
cache[name] = module
return module
return _load
@pytest.fixture(scope="session")
def run_script():
"""Run a tested script as a subprocess and return (returncode, stdout, stderr)."""
def _run(filename, *args, **kwargs):
if filename not in SCRIPT_FILES:
raise ValueError("%r is not one of the tested scripts: %r" % (filename, SCRIPT_FILES))
timeout = kwargs.pop("timeout", RUN_TIMEOUT)
cwd = kwargs.pop("cwd", None)
if kwargs:
raise TypeError("unexpected keyword arguments: %r" % sorted(kwargs))
argv = [sys.executable, str(SCRIPTS_DIR / filename)] + [str(a) for a in args]
try:
proc = subprocess.run(argv, capture_output=True, text=True, timeout=timeout,
cwd=cwd, encoding="utf-8", errors="replace")
except subprocess.TimeoutExpired:
pytest.fail("%s did not finish within %ds: %r" % (filename, timeout, argv))
return ScriptResult(proc.returncode, proc.stdout, proc.stderr, argv)
return _run
@pytest.fixture(scope="session")
def fixture_dir(tmp_path_factory):
return tmp_path_factory.mktemp("exe-reverse-fixtures")
@pytest.fixture(scope="session")
def build_fixture(fixture_dir, load_script, run_script):
"""Build one named fixture through the real CLI and cross-check it against build_pe()."""
maker = load_script("make_fixture_pe.py")
cache = {}
def _build(name):
if name in cache:
return cache[name]
if name not in FIXTURE_RECIPES:
raise ValueError("unknown fixture %r; known: %r" % (name, sorted(FIXTURE_RECIPES)))
recipe = FIXTURE_RECIPES[name]
dest = fixture_dir / (name + ".exe")
result = run_script("make_fixture_pe.py", "--out", dest, "--recipe", recipe)
if result.returncode != 0:
pytest.fail("make_fixture_pe.py --recipe %s failed with %d: %s"
% (recipe, result.returncode, result.stderr))
data, meta = maker.build_pe(**maker.RECIPES[recipe])
if dest.read_bytes() != data:
pytest.fail("make_fixture_pe.py wrote bytes that differ from its own build_pe() "
"output -- the CLI and the library disagree")
built = BuiltFixture(name, dest, meta, recipe)
cache[name] = built
return built
return _build
@pytest.fixture(scope="session")
def fixtures(build_fixture):
"""The four generated images, addressed as ``fixtures.pe64`` / ``fixtures['pe32']``."""
return FixtureSet({name: build_fixture(name) for name in FIXTURE_RECIPES})
@pytest.fixture(scope="session")
def synthetic_elf(fixture_dir):
"""The generated ELF64, written once for the whole session."""
data, meta = build_synthetic_elf()
path = fixture_dir / "app.so"
path.write_bytes(data)
return SyntheticElf(path, data, meta)
@pytest.fixture(scope="session")
def synthetic_elf_without_text(fixture_dir):
"""The same image with no `.text`, for the reader's "not an AOT code image" path."""
data, meta = build_synthetic_elf(with_text=False)
path = fixture_dir / "no-text.so"
path.write_bytes(data)
return SyntheticElf(path, data, meta)
@pytest.fixture(scope="session")
def hwprobe_report(run_script):
"""One host capability report for the whole session.
The probe executes a CPUID stub, so re-running it per assertion would buy nothing and cost
seconds. It is a host fact, not a per-test one.
"""
result = run_script("hwprobe.py", "--json")
if result.returncode != 0:
pytest.fail("hwprobe.py --json exited %d: %s" % (result.returncode, result.stderr))
return json.loads(result.stdout)
@pytest.fixture(scope="session")
def hexpat():
"""Render bytes as the spaced hex form the scripts accept."""
def _hexpat(obj):
if isinstance(obj, str):
return " ".join("%02X" % b for b in obj.encode("latin-1"))
return " ".join("%02X" % b for b in bytearray(obj))
return _hexpat
@pytest.fixture(scope="session")
def leak_lines():
return LEAK_LINES
@pytest.fixture(scope="session")
def exempt_lines():
return EXEMPT_LINES
@pytest.fixture(scope="session")
def weak_line(leak_lines):
"""A sample that produces exactly one weak finding and no strong one."""
return dict(leak_lines)["vendor-copyright-line"]
@pytest.fixture(scope="session")
def strong_line(leak_lines):
"""A sample that produces a strong finding."""
return dict(leak_lines)["windows-user-profile"]
@pytest.fixture
def synthetic_leak_root(tmp_path):
"""A scan surface holding one shaped leak per rule, plus exemption lines."""
root = tmp_path / "leakroot"
root.mkdir()
body = ["# synthetic analysis notes"] + [line for _rule, line in LEAK_LINES]
(root / "notes.md").write_text("\n".join(body) + "\n", encoding="utf-8")
(root / "exempt.md").write_text("\n".join(line for _cls, line in EXEMPT_LINES) + "\n",
encoding="utf-8")
nested = root / "skills" / "exe-reverse" / "references"
nested.mkdir(parents=True)
(nested / "deep.md").write_text("no identity here\n", encoding="utf-8")
return root
@pytest.fixture
def clean_scan_root(tmp_path):
"""A scan surface with nothing to report."""
root = tmp_path / "clean"
root.mkdir()
(root / "notes.md").write_text(
"Static PE analysis of the generated fixture: sections, imports, checksum.\n",
encoding="utf-8")
return root
@pytest.fixture
def platform_binary(require_local_tool):
"""A real platform image to parse, or a skip when this host has none."""
root = os.environ.get("SystemRoot", r"C:\Windows")
for rel in (r"System32\notepad.exe", r"System32\kernel32.dll", r"System32\ntdll.dll"):
candidate = pathlib.Path(root) / rel
if candidate.is_file():
return candidate
pytest.skip("no platform image under %s (System32 not present); this check is inert here"
% root)
+138
View File
@@ -0,0 +1,138 @@
"""Locator comparison between two builds of the same fixture.
`--across` answers one question: does the pattern still match the same number of sites on the
other build? A count that changes is a statement about the signature, not about the target, and
the point of these tests is that each of the three answers (same, changed, unusable either way)
is distinguishable in the output and in the exit code.
"""
import json
import shutil
import pytest
pytestmark = pytest.mark.integration
@pytest.fixture
def rebuilt(run_script, tmp_path, fixtures):
"""A second, independently built copy of the same recipe."""
out = tmp_path / "rebuilt.exe"
result = run_script("make_fixture_pe.py", "--out", out, "--recipe", fixtures.pe64.recipe)
assert result.returncode == 0, result.stderr
return out
def test_same_recipe_reports_an_unchanged_hit_count(run_script, fixtures, rebuilt, hexpat):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
"--across", rebuilt, "--json")
assert result.returncode == 0, result.stdout
payload = json.loads(result.stdout)
entry = payload["patterns"][0]
assert entry["hits"] == entry["across"]["hits"] == 1
assert entry["across"]["same_count"] is True
assert entry["across"]["verdict"] == "unique"
assert payload["result"] == "ok"
def test_a_recipe_without_the_debug_directory_changes_the_hit_count(run_script, fixtures,
hexpat):
"""The `bare` recipe has no debug directory, so the PDB pattern has nothing to match."""
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
"--across", fixtures.bare.path, "--json")
assert result.returncode == 1
payload = json.loads(result.stdout)
entry = payload["patterns"][0]
assert entry["hits"] == 1
assert entry["across"]["hits"] == 0
assert entry["across"]["same_count"] is False
assert entry["across"]["verdict"] == "no-match"
assert entry["across"]["sites"] == []
assert payload["result"] == "needs-attention"
def test_an_equal_hit_count_is_not_the_same_as_a_usable_locator(run_script, fixtures):
"""Two hits on both sides: the count agrees and the pattern is still a coin flip."""
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
"--across", fixtures.bare.path, "--json")
assert result.returncode == 1
payload = json.loads(result.stdout)
entry = payload["patterns"][0]
assert entry["hits"] == entry["across"]["hits"] == 2
assert entry["across"]["same_count"] is True
assert entry["verdict"] == "ambiguous"
assert payload["result"] == "needs-attention"
def test_the_text_view_explains_a_changed_count(run_script, fixtures, hexpat):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
"--across", fixtures.bare.path)
assert result.returncode == 1
assert "hit count changed between builds" in result.stdout
assert "across" in result.stdout
def test_the_text_view_explains_an_absent_pattern(run_script, fixtures, hexpat):
result = run_script("aob_scan.py", fixtures.bare.path, "--patterns", hexpat(b"fixture.pdb"))
assert result.returncode == 1
assert "does not exist in this build" in result.stdout
def test_a_patch_invalidates_the_signature_it_was_located_with(run_script, fixtures, tmp_path,
hexpat):
"""The complete lesson of the two tools together, in one flow.
The pattern that located the site includes the byte that gets patched, so after the patch the
pattern no longer matches that site. `--across` between the original and the patched copy is
what turns that from a silent hazard into a reported count change.
"""
patched = tmp_path / "patched.exe"
shutil.copyfile(str(fixtures.pe64.path), str(patched))
applied = run_script("pepatch.py", patched, "--pattern", fixtures.pe64.plant,
"--at-hit", "0", "--expect", fixtures.pe64.plant,
"--bytes", " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"]),
"--out", patched)
assert applied.returncode == 0, applied.stderr
located_first = run_script("aob_scan.py", patched, "--patterns", fixtures.pe64.plant, "--json")
assert located_first.returncode == 0
after = json.loads(located_first.stdout)["patterns"][0]
assert after["hits"] == 1, "the patched site no longer matches the pattern it was found with"
assert after["verdict"] == "unique"
assert after["sites"][0]["offset_hex"] == fixtures.pe64.meta["plant"]["second_file_offset"]
# The hazard, made concrete: the same pattern with the same index now selects a different
# site than it did before the patch, without any error from either tool.
dry = run_script("pepatch.py", patched, "--pattern", fixtures.pe64.plant, "--at-hit", "0",
"--expect", fixtures.pe64.plant,
"--bytes", " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"]),
"--dry-run", "--json")
assert dry.returncode == 0
assert json.loads(dry.stdout)["applied"][0]["offset"] == \
int(fixtures.pe64.meta["plant"]["second_file_offset"], 16)
# Unpatched copy on the left, patched copy on the right: the counts disagree, and that is
# reported rather than smoothed over.
comparison = run_script("aob_scan.py", fixtures.pe64.path, "--patterns",
fixtures.pe64.plant, "--across", patched, "--json")
assert comparison.returncode == 1
entry = json.loads(comparison.stdout)["patterns"][0]
assert entry["hits"] == 2
assert entry["across"]["hits"] == 1
assert entry["across"]["same_count"] is False
def test_comparing_against_a_non_pe_build_exits_three(run_script, fixtures, tmp_path):
other = tmp_path / "not-a-pe.txt"
other.write_text("this is not a portable executable\n", encoding="utf-8")
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
"--across", other)
assert result.returncode == 3
assert "--across" in result.stderr
assert "Traceback" not in result.stderr
def test_comparing_against_a_missing_build_exits_three(run_script, fixtures, tmp_path):
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
"--across", tmp_path / "absent.exe")
assert result.returncode == 3
assert "Traceback" not in result.stderr
@@ -0,0 +1,220 @@
"""End-to-end flows over the generated fixture: locate, patch, re-read, verify.
No debugger, no driver, no network and no live target takes part. Every step is a script call,
and the assertions are on what the next script in the chain observes -- which is the property
that matters, because these tools are meant to agree with each other about one image.
"""
import hashlib
import json
import shutil
import pytest
pytestmark = pytest.mark.integration
def sha256(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
@pytest.fixture
def work(tmp_path, fixtures):
"""An isolated copy of the PE32+ fixture, so a flow can never disturb another test."""
target = tmp_path / "sample.exe"
shutil.copyfile(str(fixtures.pe64.path), str(target))
return target
def test_locate_patch_reread_round_trip(run_script, fixtures, work, tmp_path, hexpat):
"""The full chain, with each tool checked against what the next one reports."""
patched = tmp_path / "patched.exe"
first, second = fixtures.pe64.plant_offsets
# 1. locate: the planted sequence is deliberately ambiguous, and the locator says so.
located = run_script("aob_scan.py", work, "--patterns", fixtures.pe64.plant, "--json")
assert located.returncode == 1
plan = json.loads(located.stdout)
assert plan["patterns"][0]["verdict"] == "ambiguous"
site = plan["patterns"][0]["sites"][0]
# 2. patch the first site only, keeping the replacement the same length as the site.
replacement = " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"])
applied = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant,
"--at-hit", "0", "--expect", fixtures.pe64.plant,
"--bytes", replacement, "--out", patched, "--json")
assert applied.returncode == 0, applied.stderr
result = json.loads(applied.stdout)
claim = result["applied"][0]
assert result["patches_applied"] == 1
assert result["verify_problems"] == []
# 3. the locator and the patcher must be talking about the same bytes.
assert int(site["offset_hex"], 16) == claim["offset"] == first
assert int(site["rva_hex"], 16) == int(claim["rva"], 16)
assert claim["before"] == fixtures.pe64.plant
assert claim["after"] == replacement
assert claim["equal_length"] is True
# 4. re-read the written file: the patch landed, the control site did not move, and the
# checksum was recomputed rather than left stale.
reread = run_script("peutil.py", patched, "--summary", "--check-sum", "--json")
assert reread.returncode == 0, reread.stderr
report = json.loads(reread.stdout)
assert report["summary"]["checksum_matches"] is True
assert report["summary"]["checksum_declared"] == report["summary"]["checksum_computed"]
assert report["checksum"]["matches"] is True
assert report["summary"]["file_size"] == fixtures.pe64.size
body = patched.read_bytes()
assert body[first:first + 12] == bytes(int(b, 16) for b in replacement.split())
assert body[second:second + 12] == fixtures.pe64.data[second:second + 12]
# 5. nothing outside the site and the checksum field moved.
checksum_field = int(fixtures.pe64.meta["checksum_offset"], 16)
diff = {i for i, (a, b) in enumerate(zip(fixtures.pe64.data, body)) if a != b}
assert diff - {first + 10} <= set(range(checksum_field, checksum_field + 4))
assert first + 10 in diff
def test_control_run_that_changes_nothing_leaves_the_file_identical(run_script, fixtures, work,
tmp_path):
"""The control for the flow above: a patch whose bytes equal the site's own bytes.
This is the only zero-change run the pipeline has: a spec with an empty patch array is
refused as a usage error, so a no-op patch is what "the pipeline ran and changed nothing"
looks like. The output must be byte-identical to the input, checksum included.
"""
out = tmp_path / "noop.exe"
before = sha256(work)
result = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
"--expect", fixtures.pe64.plant, "--bytes", fixtures.pe64.plant,
"--out", out, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["checksum_recomputed"] is True
assert payload["checksum_after"]["matches"] is True
assert payload["output_sha256"] == payload["input_sha256"] == before
assert sha256(out) == before
assert sha256(work) == before
def test_an_empty_patch_array_is_refused_rather_than_writing_a_copy(run_script, fixtures, work,
tmp_path):
spec = tmp_path / "empty.json"
spec.write_text('{"patches": []}', encoding="utf-8")
out = tmp_path / "copy.exe"
result = run_script("pepatch.py", work, "--spec", spec, "--out", out)
assert result.returncode == 2
assert not out.exists()
assert sha256(work) == sha256(fixtures.pe64.path)
def test_the_flow_stops_before_writing_when_the_pattern_is_ambiguous(run_script, fixtures, work,
tmp_path):
out = tmp_path / "never.exe"
before = sha256(work)
result = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant,
"--bytes", "90 " * 12, "--out", out)
assert result.returncode == 1
assert "matched 2 sites" in result.stderr
assert not out.exists()
assert sha256(work) == before
def test_dry_run_reports_the_plan_and_writes_nothing(run_script, fixtures, work, tmp_path):
out = tmp_path / "would-be.exe"
before = sha256(work)
result = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant, "--at-hit", "1",
"--expect", fixtures.pe64.plant, "--bytes", "CC " * 12,
"--dry-run", "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["dry_run"] is True
assert payload["patches_applied"] == 1
assert payload["applied"][0]["offset"] == fixtures.pe64.plant_offsets[1]
assert not out.exists()
assert sha256(work) == before
def test_a_spec_file_can_reach_a_branch_inside_a_longer_pattern(run_script, fixtures, work,
tmp_path):
"""The pattern-offset flow: anchor on a unique longer sequence, then patch the byte inside."""
out = tmp_path / "branch.exe"
site = fixtures.pe64.plant_offsets[0]
spec = tmp_path / "spec.json"
spec.write_text(json.dumps({"patches": [{
"name": "gate",
"pattern": fixtures.pe64.plant + " CC",
"pattern-offset": 10,
"expect": "74",
"bytes": "EB",
"why": "invert the conditional branch, keeping the taken distance",
}]}), encoding="utf-8")
result = run_script("pepatch.py", work, "--spec", spec, "--out", out, "--json")
assert result.returncode == 0, result.stderr
payload = json.loads(result.stdout)
assert payload["patches_applied"] == 1
record = payload["applied"][0]
assert record["offset"] == site + 10
assert record["before"] == "74"
assert record["after"] == "EB"
assert record["length_basis"] == "expect"
assert record["why"].startswith("invert")
reread = run_script("peutil.py", out, "--summary", "--check-sum", "--json")
assert reread.returncode == 0
assert json.loads(reread.stdout)["checksum"]["matches"] is True
assert out.read_bytes()[site + 10] == 0xEB
assert out.read_bytes()[site + 11] == 0x05
def test_a_length_change_is_refused_before_anything_is_written(run_script, fixtures, work,
tmp_path):
out = tmp_path / "resized.exe"
result = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
"--expect", "48", "--bytes", "CC CC CC", "--out", out)
assert result.returncode == 1
assert "--allow-resize" in result.stderr
assert not out.exists()
allowed = tmp_path / "resized-allowed.exe"
forced = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
"--expect", "48", "--bytes", "CC CC CC", "--allow-resize",
"--out", allowed, "--json")
assert forced.returncode == 0, forced.stderr
payload = json.loads(forced.stdout)
assert payload["structural"] == ["cli"]
# One byte replaced by three: the file grows by two, and that is the structural edit the
# caller asked for by passing --allow-resize.
assert payload["output_size"] == payload["input_size"] + 2
def test_the_patched_output_still_parses_as_an_image(run_script, fixtures, work, tmp_path,
hexpat):
"""A patch that lands in the file must not damage a single header the reader trusts."""
out = tmp_path / "patched.exe"
result = run_script("pepatch.py", work, "--at-offset", "0x410", "--expect", "48 8B",
"--bytes", "CC 8B", "--out", out)
assert result.returncode == 0, result.stderr
full = run_script("peutil.py", out, "--summary", "--sections", "--imports", "--exports",
"--resources", "--tls", "--debug", "--relocs", "--check-sum", "--json")
assert full.returncode == 0, full.stderr
report = json.loads(full.stdout)
assert report["summary"]["sha256"] == sha256(out)
assert report["checksum"]["matches"] is True
assert [s["name"] for s in report["summary"]["sections"]] == \
[".text", ".rdata", ".data", ".rsrc", ".reloc"]
assert report["resources"][0]["path"] == ["VERSION", "1", "1033"]
assert report["tls"]["callback_rvas"] == ["0x1000"]
assert report["debug"][0]["format"] == "RSDS"
assert report["imports"][0]["functions"][0]["name"] == "ImportedFunction"
def test_a_real_platform_image_reproduces_its_declared_checksum(run_script, platform_binary):
"""The fixture is synthetic; this is the same claim measured on a real system image."""
result = run_script("peutil.py", platform_binary, "--summary", "--check-sum", "--json")
assert result.returncode == 0, result.stderr
report = json.loads(result.stdout)
assert report["summary"]["checksum_matches"] is True
assert report["checksum"]["declared"] == report["checksum"]["computed"]
assert report["summary"]["file_size"] == platform_binary.stat().st_size
assert report["summary"]["sections"], "a platform image must have a section table"
+68
View File
@@ -0,0 +1,68 @@
#!/usr/bin/env python3
"""Run the regression suite with one command, and fail usefully when it cannot run.
Why this exists next to `python -m pytest`: the interesting failure is not a failing test, it is
a suite that cannot start. "python -m pytest" answers a missing pytest with
``No module named pytest``, which reads like a bug in the suite rather than a missing
prerequisite. This runner states the prerequisite, prints the install line the repository
documents, and exits 3 -- the repository's environment-capability failure code -- instead of
letting a toolchain gap look like a test result.
Usage:
python tests/run_tests.py # the whole suite
python tests/run_tests.py -q -m unit # anything pytest accepts is forwarded
Exit codes: pytest's own code (0 = passed, 1 = failures, 2 = interrupted, 4 = usage error,
5 = nothing collected), or 3 = the environment cannot run the suite at all. Note that pytest
also uses 3 for its own internal error; the printed message distinguishes the two.
Requirements: Python 3.9+ and pytest. No other third-party package, no debugger, no driver, no
network and no committed binary sample -- the PE inputs are generated into a temporary directory
at test time.
"""
import importlib.util
import os
import sys
MIN_PYTHON = (3, 9)
EXIT_ENVIRONMENT = 3
INSTALL_INSTRUCTION = "python -m pip install pytest"
TESTS_DIR = os.path.dirname(os.path.abspath(__file__))
REPO_ROOT = os.path.dirname(TESTS_DIR)
def _message(lines):
for line in lines:
sys.stderr.write("%s\n" % line)
def main(argv):
if sys.version_info < MIN_PYTHON:
_message([
"This suite needs Python %d.%d or newer; this interpreter is %s."
% (MIN_PYTHON[0], MIN_PYTHON[1], sys.version.split()[0]),
"The tested scripts declare the same requirement in their own docstrings.",
"Install a newer interpreter, then run this command again.",
])
return EXIT_ENVIRONMENT
if importlib.util.find_spec("pytest") is None:
_message([
"pytest is not installed, so the regression suite cannot run.",
"",
" install: %s" % INSTALL_INSTRUCTION,
"",
"or run the suite after installing it: python tests/run_tests.py",
])
return EXIT_ENVIRONMENT
# pytest reads pytest.ini relative to the rootdir, so run from the repository root whatever
# directory the caller used.
os.chdir(REPO_ROOT)
import pytest
return pytest.main(list(argv))
if __name__ == "__main__":
sys.exit(main(sys.argv[1:]))
+161
View File
@@ -0,0 +1,161 @@
"""Unit tests for skills/exe-reverse/scripts/aob_scan.py.
The tool's real output is the *verdict*: a pattern that matches twice is a coin flip, and the
whole point is that the caller is told so. The boundary table is asserted at both sides of each
threshold, and the pattern-file parser is tested in all four of its accepted shapes.
"""
import types
import pytest
pytestmark = pytest.mark.unit
def ns(**kwargs):
base = {"patterns": []}
base.update(kwargs)
return types.SimpleNamespace(**base)
# --- verdict boundaries ------------------------------------------------------
@pytest.mark.parametrize("count,expected", [
(0, "no-match"),
(1, "unique"),
(2, "ambiguous"),
(3, "ambiguous"),
(7, "ambiguous"),
(8, "pervasive"),
(64, "pervasive"),
])
def test_verdict_boundaries(load_script, count, expected):
assert load_script("aob_scan.py").verdict(count, count) == expected
def test_verdict_thresholds_are_the_documented_ones(load_script):
aob = load_script("aob_scan.py")
assert aob.NOTICEABLE_AT == 8
assert aob.AMBIGUOUS_AT == 2
assert aob.verdict(aob.NOTICEABLE_AT - 1, 0) == "ambiguous"
assert aob.verdict(aob.NOTICEABLE_AT, 0) == "pervasive"
def test_verdict_ignores_the_unused_total_argument(load_script):
aob = load_script("aob_scan.py")
assert aob.verdict(1, 999) == aob.verdict(1, 0) == "unique"
# --- locate ------------------------------------------------------------------
def test_locate_reports_offset_rva_and_section(load_script, fixtures):
aob = load_script("aob_scan.py")
peutil = load_script("peutil.py")
pepatch = load_script("pepatch.py")
pe = peutil.PE(fixtures.pe64.data, None)
pat = pepatch.parse_bytes(fixtures.pe64.plant)[0]
rows = aob.locate(pe, fixtures.pe64.data, pat)
assert [r["offset"] for r in rows] == list(fixtures.pe64.plant_offsets)
assert [r["rva_hex"] for r in rows] == [fixtures.pe64.meta["plant"]["first_rva"],
fixtures.pe64.meta["plant"]["second_rva"]]
assert [r["section"] for r in rows] == [".text", ".text"]
assert rows[0]["offset_hex"] == fixtures.pe64.meta["plant"]["first_file_offset"]
def test_locate_section_filter_drops_hits_elsewhere(load_script, fixtures):
aob = load_script("aob_scan.py")
peutil = load_script("peutil.py")
pat = [0x66, 0x69, 0x78, 0x74, 0x75, 0x72, 0x65, 0x2E, 0x70, 0x64, 0x62] # "fixture.pdb"
pe = peutil.PE(fixtures.pe64.data, None)
assert [r["section"] for r in aob.locate(pe, fixtures.pe64.data, pat)] == [".rdata"]
assert aob.locate(pe, fixtures.pe64.data, pat, section_filter=".text") == []
assert len(aob.locate(pe, fixtures.pe64.data, pat, section_filter=".rdata")) == 1
def test_locate_context_is_centred_on_the_match(load_script, fixtures):
aob = load_script("aob_scan.py")
peutil = load_script("peutil.py")
pepatch = load_script("pepatch.py")
pe = peutil.PE(fixtures.pe64.data, None)
pat = pepatch.parse_bytes(fixtures.pe64.plant)[0]
rows = aob.locate(pe, fixtures.pe64.data, pat, context=8)
assert "context" in rows[0]
assert rows[0]["context_marks"] == "pre 8 bytes, match 12, post 8"
assert len(pepatch.parse_bytes(rows[0]["context"])[0]) == len(pat) + 16
def test_locate_returns_no_rows_when_the_pattern_is_absent(load_script, fixtures):
"""A negative outcome: an absent pattern yields an empty list, not a nearest match."""
aob = load_script("aob_scan.py")
peutil = load_script("peutil.py")
pe = peutil.PE(fixtures.bare.data, None)
pat = [0x66, 0x69, 0x78, 0x74, 0x75, 0x72, 0x65, 0x2E, 0x70, 0x64, 0x62]
assert aob.locate(pe, fixtures.bare.data, pat) == []
# --- load_patterns -----------------------------------------------------------
def test_load_patterns_accepts_an_inline_pattern(load_script):
aob = load_script("aob_scan.py")
assert aob.load_patterns(ns(patterns=["48 8B ?? 05"])) == [
("inline", [0x48, 0x8B, None, 0x05], "48 8B ?? 05")]
def test_load_patterns_without_patterns_is_empty(load_script):
assert load_script("aob_scan.py").load_patterns(ns()) == []
assert load_script("aob_scan.py").load_patterns(ns(patterns=[])) == []
def test_load_patterns_parses_a_text_file_with_comments_and_names(load_script, tmp_path):
aob = load_script("aob_scan.py")
path = tmp_path / "sigs.txt"
path.write_text("# a comment\n\n48 8B 05\nuse-site\t90 90 90\n", encoding="utf-8")
got = aob.load_patterns(ns(patterns=[str(path)]))
assert got == [("sigs.txt:3", [0x48, 0x8B, 0x05], "48 8B 05"),
("use-site", [0x90, 0x90, 0x90], "90 90 90")]
def test_load_patterns_parses_a_json_object(load_script, tmp_path):
aob = load_script("aob_scan.py")
obj = tmp_path / "named.json"
obj.write_text('{"patterns": [{"name": "gate", "pattern": "74 05"}, "EB 05"]}',
encoding="utf-8")
assert aob.load_patterns(ns(patterns=[str(obj)])) == [
("gate", [0x74, 0x05], "74 05"),
("named.json[1]", [0xEB, 0x05], "EB 05")]
def test_a_bare_json_list_is_accepted(load_script, tmp_path):
"""A top-level array is as reasonable a shape as an object with a "patterns" key.
The reader used to call ``.get`` on whatever ``json.load`` returned, so a top-level array
raised AttributeError instead of the SystemExit this module uses for every other bad input.
Both shapes are read now, and anything else is refused with a message naming what was found.
"""
aob = load_script("aob_scan.py")
bare = tmp_path / "bare.json"
bare.write_text('["CC CC"]', encoding="utf-8")
assert aob.load_patterns(ns(patterns=[str(bare)])) == [("bare.json[0]", [0xCC, 0xCC], "CC CC")]
def test_a_json_scalar_is_refused_with_a_message(load_script, tmp_path):
"""The shape that is still not a pattern file says so instead of raising."""
aob = load_script("aob_scan.py")
scalar = tmp_path / "scalar.json"
scalar.write_text('42', encoding="utf-8")
with pytest.raises(SystemExit) as excinfo:
aob.load_patterns(ns(patterns=[str(scalar)]))
assert "must hold an array or an object" in str(excinfo.value)
@pytest.mark.parametrize("text,expected", [
("ZZ QQ", "not a byte"),
("?? ??", "empty or all wildcards"),
])
def test_load_patterns_exits_on_a_pattern_it_cannot_parse(load_script, tmp_path, text,
expected):
aob = load_script("aob_scan.py")
path = tmp_path / "bad.txt"
path.write_text(text + "\n", encoding="utf-8")
with pytest.raises(SystemExit) as excinfo:
aob.load_patterns(ns(patterns=[str(path)]))
assert expected in str(excinfo.value)
+509
View File
@@ -0,0 +1,509 @@
"""Unit tests for skills/exe-reverse/scripts/capabilities.py, the capability registry.
The registry's value is that a verdict is *computed* from a probed closure rather than read off a
name, so what is asserted here is the table it computes over: every atom a capability names, every
atom's kind and basis, and the transitive closure the resolution walks.
One part of the table is not well formed, and the tests pin the exact extent rather than asserting
the invariant that fails. `Probe.atom` answers "atom is not defined" for an id with no `ATOMS` row,
so a capability naming a *capability* in `required` or `any_of` has an atom nothing can ever
satisfy. Two rows do that today (`dump_capture`, `hypervisor_observe`); they are named in
`UNTABLED_REFERENCES` so a third one fails the suite instead of arriving silently. Reported as a
defect in the digest: these tests do not repair the script.
`declared_capability` gets its own group, because the convention it reads is documented inside this
file's own docstring. A looser pattern once read that documentation as a declaration and reported
the registry itself as declaring `capability: <capability_id>[, <capability_id>]`; the template is
pinned as *not* a declaration below.
"""
import os
import pytest
pytestmark = pytest.mark.unit
# Measured, not assumed: the `required` / `any_of` entries that name a capability id instead of an
# atom id. Both are answered by `Probe.atom` with the literal detail "atom is not defined", which is
# pinned separately -- the mechanism is what makes the reference unrepairable from outside the file.
UNTABLED_REFERENCES = frozenset({
("dump_capture", "any_of", "commandline_debug"),
("hypervisor_observe", "required", "kernel_driver_load"),
})
# The kinds `Probe.atom` dispatches on, and the field each one is probed through.
PROBE_HANDLED_KINDS = {
"python": None,
"module": "module",
"tool": "tool",
"file": "script",
"host": "probe",
"artifact": None,
}
# The probe names `_host_flag` understands. Not decoration: a `host` atom whose probe name is not in
# this set falls through to `None`, which reads as "not available on this host" forever.
HOST_PROBE_NAMES = frozenset({
"elevation", "hypervisor", "intel_pt", "breakpoints", "test_signing", "etw",
})
# Both are the documented strength labels of the *measurement*, not of the route.
BASIS_LABELS = frozenset({"observed", "unverified"})
def probe(caps):
"""A fresh probe with no extra tool directories, so PATH alone decides."""
return caps.Probe(tool_dirs=[])
def untabled_references(caps):
out = set()
for capability_id, spec in caps.CAPABILITIES.items():
for key in ("required", "any_of"):
for atom in spec.get(key, []):
if atom not in caps.ATOMS:
out.add((capability_id, key, atom))
return out
def reference_closure(caps, capability_id):
"""An independent breadth-first walk to compare the module's ordering against."""
out, queue = [], [capability_id]
while queue:
current = queue.pop(0)
if current in out:
continue
out.append(current)
queue.extend(caps.CAPABILITIES.get(current, {}).get("depends_on", []))
return out
# --- the atom table ----------------------------------------------------------
def test_the_registry_is_not_empty(load_script):
caps = load_script("capabilities.py")
assert len(caps.ATOMS) > 1
assert len(caps.CAPABILITIES) > 1
def test_every_capability_row_carries_a_name_and_a_required_list(load_script):
caps = load_script("capabilities.py")
for capability_id, spec in caps.CAPABILITIES.items():
assert spec.get("name"), "%s has no name" % capability_id
required = spec.get("required")
assert isinstance(required, list), "%s has no required list" % capability_id
# An empty `required` is legitimate when the row's prerequisites are other capabilities:
# `hypervisor_observe` needs no atom of its own, it needs the capability below it. What is
# not legitimate is a row that names nothing at all, because then no probe can decide it.
assert required or spec.get("any_of") or spec.get("depends_on"), \
"%s names nothing at all" % capability_id
def test_capability_rows_use_only_the_documented_keys(load_script):
caps = load_script("capabilities.py")
allowed = {"name", "required", "any_of", "depends_on", "scripts", "note"}
for capability_id, spec in caps.CAPABILITIES.items():
assert set(spec) <= allowed, "%s carries %r" % (capability_id, sorted(set(spec) - allowed))
def test_no_unrecorded_atom_reference_is_outside_the_atom_table(load_script):
"""Every name in a `required` or `any_of` list has a row in the atom table.
This was false for two entries: a **capability** id had been written into an atom slot, so the
probe answered `atom is not defined` and the branch could never be satisfied -- and one of them,
`hypervisor_observe`, was therefore permanently blocked with no next action to print. A slot
whose vocabulary is wrong is silent, which is why it is asserted as a property now rather than
as the measured extent of the breakage.
"""
caps = load_script("capabilities.py")
assert untabled_references(caps) == set()
def test_a_capability_named_where_an_atom_belongs_is_answered_as_undefined(load_script):
"""The mechanism behind the pinned deviation, so the two rows are not merely suspicious names."""
caps = load_script("capabilities.py")
for _capability_id, _key, named in sorted(UNTABLED_REFERENCES):
assert named in caps.CAPABILITIES, "%s should be a capability id" % named
result = probe(caps).atom(named)
assert result["ok"] is False
assert result["detail"] == "atom is not defined"
def test_dependencies_name_capabilities_rather_than_atoms(load_script):
"""`depends_on` is a capability edge. An atom id here would be silently unresolvable."""
caps = load_script("capabilities.py")
for capability_id, spec in caps.CAPABILITIES.items():
for dep in spec.get("depends_on", []):
assert dep in caps.CAPABILITIES, \
"%s depends on %r, which is not a capability" % (capability_id, dep)
assert dep not in caps.ATOMS, \
"%s depends on %r, which is an atom: closure() would treat it as a capability" \
% (capability_id, dep)
assert dep != capability_id, "%s depends on itself" % capability_id
def test_the_script_map_points_only_at_capabilities_that_exist(load_script):
caps = load_script("capabilities.py")
assert caps.SCRIPT_CAPABILITIES
for script, ids in caps.SCRIPT_CAPABILITIES.items():
assert ids, "%s is mapped to an empty list, which reads as a claim" % script
for capability_id in ids:
assert capability_id in caps.CAPABILITIES, \
"%s is mapped to unknown capability %r" % (script, capability_id)
def test_the_dependency_graph_is_acyclic(load_script):
"""A cycle would make `resolve` recurse forever rather than report a blocked capability."""
caps = load_script("capabilities.py")
state = {}
def walk(capability_id):
stack = state.get(capability_id, 0)
assert stack != 1, "dependency cycle through %s" % capability_id
if stack == 2:
return
state[capability_id] = 1
for dep in caps.CAPABILITIES.get(capability_id, {}).get("depends_on", []):
walk(dep)
state[capability_id] = 2
for capability_id in caps.CAPABILITIES:
walk(capability_id)
def test_every_atom_carries_the_basis_label_the_registry_documents(load_script):
caps = load_script("capabilities.py")
for atom_id, spec in caps.ATOMS.items():
assert spec.get("basis") in BASIS_LABELS, \
"%s has basis %r, which is neither %s" % (atom_id, spec.get("basis"),
" nor ".join(sorted(BASIS_LABELS)))
def test_every_atom_kind_is_one_the_probe_handles(load_script):
caps = load_script("capabilities.py")
for atom_id, spec in caps.ATOMS.items():
assert spec.get("kind") in PROBE_HANDLED_KINDS, \
"%s has kind %r, which Probe.atom does not dispatch on" % (atom_id, spec.get("kind"))
def test_every_atom_carries_the_field_its_kind_is_probed_through(load_script):
"""`Probe.atom` subscripts these keys, so a missing one is a KeyError, not a verdict."""
caps = load_script("capabilities.py")
for atom_id, spec in caps.ATOMS.items():
field = PROBE_HANDLED_KINDS[spec["kind"]]
if field is not None:
assert spec.get(field), "%s (%s) has no %r" % (atom_id, spec["kind"], field)
def test_every_host_atom_names_a_probe_the_host_reader_implements(load_script):
caps = load_script("capabilities.py")
for atom_id, spec in caps.ATOMS.items():
if spec["kind"] == "host":
assert spec["probe"] in HOST_PROBE_NAMES, \
"%s probes %r, which _host_flag does not answer (it would be false forever)" \
% (atom_id, spec["probe"])
def test_atom_ids_are_prefixed_with_their_kind(load_script):
caps = load_script("capabilities.py")
for atom_id, spec in caps.ATOMS.items():
if spec["kind"] == "python":
continue
assert atom_id.startswith(spec["kind"] + ":"), \
"%s does not name its kind (%s)" % (atom_id, spec["kind"])
def test_every_atom_answers_something_other_than_undefined(load_script):
"""Each row is reachable through the probe: no row falls through to the undefined branch."""
caps = load_script("capabilities.py")
p = probe(caps)
for atom_id in caps.ATOMS:
result = p.atom(atom_id)
assert isinstance(result.get("ok"), bool), "%s answered %r" % (atom_id, result)
assert result["detail"] != "atom is not defined", \
"%s has a row but the probe still calls it undefined" % atom_id
def test_which_falls_back_to_an_extra_directory(load_script, tmp_path):
"""A tool that is installed off PATH is not a missing tool, which is the whole point of the
second search path. The file is never executed -- only its presence is checked."""
caps = load_script("capabilities.py")
tools = tmp_path / "unpacked"
tools.mkdir()
(tools / "ghidra_off_path_probe").write_text("", encoding="utf-8")
p = caps.Probe(tool_dirs=[str(tools)])
found, via = p.which("ghidra_off_path_probe")
assert found == str(tools / "ghidra_off_path_probe")
assert via == "EXEREV_TOOLS"
assert p.which("no_such_tool_anywhere_xyz") == (None, None)
def test_extra_tool_dirs_reads_both_environment_variable_names(load_script, monkeypatch, tmp_path):
caps = load_script("capabilities.py")
monkeypatch.setenv("EXEREV_TOOLS", str(tmp_path / "one"))
monkeypatch.setenv("EXE_REVERSE_TOOLS", str(tmp_path / "two") + os.pathsep +
str(tmp_path / "three"))
assert caps.extra_tool_dirs() == [str(tmp_path / "one"), str(tmp_path / "two"),
str(tmp_path / "three")]
monkeypatch.delenv("EXEREV_TOOLS")
monkeypatch.delenv("EXE_REVERSE_TOOLS")
assert caps.extra_tool_dirs() == []
# --- closure -----------------------------------------------------------------
def test_closure_of_a_capability_with_no_dependencies_is_its_only_member(load_script):
caps = load_script("capabilities.py")
assert caps.closure("pe_structure_read") == ["pe_structure_read"]
def test_closure_matches_an_independent_walk_over_the_real_table(load_script):
caps = load_script("capabilities.py")
for capability_id in caps.CAPABILITIES:
expected = reference_closure(caps, capability_id)
assert caps.closure(capability_id) == expected, capability_id
def test_closure_terminates_and_lists_each_capability_once(load_script):
caps = load_script("capabilities.py")
for capability_id in caps.CAPABILITIES:
chain = caps.closure(capability_id)
assert chain[0] == capability_id
assert len(chain) == len(set(chain)), "%s repeats a member: %r" % (capability_id, chain)
assert set(chain) <= set(caps.CAPABILITIES), \
"%s reaches something that is not a capability: %r" % (capability_id, chain)
def test_closure_follows_more_than_one_hop_and_deduplicates_a_diamond(load_script, monkeypatch):
"""Synthetic: the real table is only one hop deep, so a diamond is built to reach the walk."""
caps = load_script("capabilities.py")
for name, deps in (("synthetic_top", ["synthetic_left", "synthetic_right"]),
("synthetic_left", ["synthetic_bottom"]),
("synthetic_right", ["synthetic_bottom"]),
("synthetic_bottom", [])):
monkeypatch.setitem(caps.CAPABILITIES, name,
{"name": name, "required": ["python310"], "depends_on": deps})
assert caps.closure("synthetic_top") == ["synthetic_top", "synthetic_left",
"synthetic_right", "synthetic_bottom"]
def test_closure_of_an_id_that_is_not_a_capability_is_the_id_alone(load_script):
"""Documented behaviour: no error, no members. Pinned so a caller can see what it gets."""
assert load_script("capabilities.py").closure("no_such_capability") == ["no_such_capability"]
# --- resolve -----------------------------------------------------------------
def synthetic_capability(monkeypatch, caps, capability_id, **spec):
body = {"name": "Synthetic " + capability_id, "required": ["python310"]}
body.update(spec)
monkeypatch.setitem(caps.CAPABILITIES, capability_id, body)
def test_resolve_reports_blocked_and_names_the_missing_atom(load_script, monkeypatch):
caps = load_script("capabilities.py")
synthetic_capability(monkeypatch, caps, "synthetic_missing",
required=["python310", "artifact:sample"])
result = caps.resolve("synthetic_missing", probe(caps))
assert result["state"] == "blocked"
assert [atom for atom, _detail in result["missing"]] == ["artifact:sample"]
def test_resolve_of_a_real_capability_names_the_artifact_it_cannot_probe(load_script):
"""`artifact:sample` is operator-supplied by design, so this verdict is stable on any host."""
caps = load_script("capabilities.py")
result = caps.resolve("commandline_debug", probe(caps))
assert result["state"] == "blocked"
assert "artifact:sample" in [atom for atom, _detail in result["missing"]]
def test_resolve_reports_a_blocked_dependency_by_name(load_script):
caps = load_script("capabilities.py")
result = caps.resolve("hardware_breakpoint", probe(caps))
assert result["state"] == "blocked"
assert "commandline_debug" in [dep for dep, _state in result["blocked_deps"]]
def test_resolve_on_a_file_atom_follows_the_disk(load_script):
"""A shipped script is present, and the verdict says which path was looked at."""
caps = load_script("capabilities.py")
result = caps.resolve("pe_structure_read", probe(caps))
assert result["state"] == "ok"
assert result["missing"] == []
file_atom = probe(caps).atom("file:peutil.py")
assert file_atom["ok"] is True
assert file_atom["detail"] == os.path.join(caps.HERE, "peutil.py")
def test_resolve_on_a_fabricated_file_atom_is_missing_and_names_its_path(load_script,
monkeypatch):
caps = load_script("capabilities.py")
monkeypatch.setitem(caps.ATOMS, "file:no_such_script_xyz.py",
{"name": "Fabricated", "kind": "file", "basis": "observed",
"script": "no_such_script_xyz.py", "est_minutes": 0})
synthetic_capability(monkeypatch, caps, "synthetic_file",
required=["file:no_such_script_xyz.py"])
result = caps.resolve("synthetic_file", probe(caps))
assert result["state"] == "blocked"
atom, detail = result["missing"][0]
assert atom == "file:no_such_script_xyz.py"
assert detail.endswith("no_such_script_xyz.py")
def test_resolve_reports_the_alternatives_it_found(load_script, monkeypatch):
caps = load_script("capabilities.py")
synthetic_capability(monkeypatch, caps, "synthetic_any",
required=[], any_of=["file:peutil.py", "artifact:sample"])
result = caps.resolve("synthetic_any", probe(caps))
assert result["alternatives_present"] == ["file:peutil.py"]
assert result["state"] == "ok"
assert result["missing"] == []
def test_resolve_of_an_unknown_capability_is_none(load_script):
caps = load_script("capabilities.py")
assert caps.resolve("no_such_capability", probe(caps)) is None
def test_resolve_all_covers_every_capability_and_agrees_with_resolve(load_script):
caps = load_script("capabilities.py")
p = probe(caps)
rows = caps.resolve_all(p)
assert [row["id"] for row in rows] == list(caps.CAPABILITIES)
for row in rows:
assert row == caps.resolve(row["id"], p)
def test_verdict_is_ok_only_when_every_row_is_ok(load_script):
caps = load_script("capabilities.py")
assert caps.verdict([]) == ("ok", 0)
assert caps.verdict([{"state": "ok"}, {"state": "ok"}]) == ("ok", 0)
assert caps.verdict([{"state": "ok"}, {"state": "blocked"}]) == ("blocked", 3)
def test_human_report_marks_every_row_and_explains_a_blocked_one(load_script):
caps = load_script("capabilities.py")
rows = [
{"id": "ok_one", "name": "Okay", "state": "ok", "missing": [], "blocked_deps": []},
{"id": "blocked_one", "name": "Blocked", "state": "blocked",
"missing": [("artifact:sample", "operator supplied; not detectable from here")],
"blocked_deps": [("other_cap", "blocked")]},
]
quiet = caps.human_report(rows, verbose=False)
assert any("ok_one" in line and line.startswith(" ok ") for line in quiet)
assert any("blocked_one" in line and line.startswith(" BLK") for line in quiet)
loud = caps.human_report(rows, verbose=True)
assert any("missing atom : artifact:sample" in line for line in loud)
assert any("blocked by : other_cap (blocked)" in line for line in loud)
# --- declaration parsing -----------------------------------------------------
def write_script(tmp_path, name, body):
path = tmp_path / name
path.write_text(body, encoding="utf-8")
return path
def test_declared_capability_reads_a_single_id(load_script, tmp_path):
caps = load_script("capabilities.py")
path = write_script(tmp_path, "one.py",
'"""capability: byte_patch\n\nA script that declares its capability.\n"""\n')
assert caps.declared_capability(str(path)) == ["byte_patch"]
def test_declared_capability_reads_a_comma_separated_list(load_script, tmp_path):
caps = load_script("capabilities.py")
path = write_script(tmp_path, "two.py",
'"""capability: byte_patch, pe_structure_read\n\nTwo of them.\n"""\n')
assert caps.declared_capability(str(path)) == ["byte_patch", "pe_structure_read"]
def test_declared_capability_rejects_the_documentation_template(load_script, tmp_path):
"""The exact line this registry's own docstring uses to describe the convention.
It was once read as a declaration, which reported the registry as declaring a capability named
`<capability_id>`. A template is not a declaration, and the pin is on that exact string.
"""
caps = load_script("capabilities.py")
template = " capability: <capability_id>[, <capability_id>]"
path = write_script(tmp_path, "template.py", '"""%s\n\nProse.\n"""\n' % template)
assert caps.declared_capability(str(path)) == []
def test_the_template_really_does_live_in_the_registrys_own_docstring(load_script):
"""The pin above is only meaningful if the string it names is really in this file."""
caps = load_script("capabilities.py")
source = open(os.path.join(caps.HERE, "capabilities.py"), encoding="utf-8").read()
assert "capability: <capability_id>[, <capability_id>]" in source
assert caps.declared_capability(os.path.join(caps.HERE, "capabilities.py")) == []
def test_declared_capability_reads_a_real_declaring_script(load_script):
caps = load_script("capabilities.py")
assert caps.declared_capability(os.path.join(caps.HERE, "dart_snapshot_strings.py")) == \
["static_disassembly"]
@pytest.mark.parametrize("name,body,expected", [
("prose.py", '"""No declaration here, only prose.\n\nMore prose.\n"""\n', []),
("no-docstring.py", "import os\n", []),
("template-only.py", '"""\ncapability: <capability_id>\n"""\n', []),
("not-a-list.py", '"""capability: one two\n"""\n', []),
("trailing-junk.py", '"""capability: one # comment\n"""\n', []),
("hyphenated-id.py", '"""capability: byte-patch\n"""\n', []),
("space-before-colon.py", '"""capability : byte_patch\n"""\n', []),
("list-with-a-hyphen.py", '"""capability: byte_patch, pe-structure\n"""\n', []),
("trailing-space-and-comma.py", '"""capability: byte_patch,\n"""\n', []),
])
def test_declared_capability_refuses_a_shape_that_is_not_a_declaration(load_script, tmp_path,
name, body, expected):
caps = load_script("capabilities.py")
assert caps.declared_capability(str(write_script(tmp_path, name, body))) == expected
def test_declared_capability_matches_case_insensitively(load_script, tmp_path):
"""Pinned, because it is wider than the ids it can name: the pattern is compiled `(?i)`, so a
mixed-case declaration is returned verbatim and then resolves to nothing -- every id in both
tables is lowercase. Recorded as an observation rather than asserted as desirable."""
caps = load_script("capabilities.py")
path = write_script(tmp_path, "mixed-case.py", '"""capability: BytePatch\n"""\n')
assert caps.declared_capability(str(path)) == ["BytePatch"]
assert "BytePatch" not in caps.CAPABILITIES
def test_declared_capability_is_silent_about_a_file_it_cannot_read(load_script, tmp_path):
caps = load_script("capabilities.py")
assert caps.declared_capability(str(tmp_path / "absent.py")) == []
broken = write_script(tmp_path, "broken.py", '"""capability: byte_patch\n"""\nif True print)\n')
assert caps.declared_capability(str(broken)) == []
# --- script -> capability ----------------------------------------------------
def test_capabilities_for_script_reports_a_declaration(load_script):
caps = load_script("capabilities.py")
path = os.path.join(caps.HERE, "dart_snapshot_strings.py")
assert caps.capabilities_for_script("dart_snapshot_strings.py", path) == \
(["static_disassembly"], "declared")
def test_capabilities_for_script_falls_back_to_the_map(load_script):
caps = load_script("capabilities.py")
assert caps.capabilities_for_script("aob_scan.py") == (["aob_locate"], "mapped")
def test_capabilities_for_script_prefers_the_declaration_over_the_map(load_script, tmp_path):
"""A script may not be silently reclassified by the map once it declares its own capability."""
caps = load_script("capabilities.py")
path = write_script(tmp_path, "aob_scan.py",
'"""capability: pe_structure_read\n\nDeclares what it is.\n"""\n')
assert caps.capabilities_for_script("aob_scan.py", str(path)) == \
(["pe_structure_read"], "declared")
def test_a_script_in_neither_table_is_unknown_rather_than_empty_and_claimed(load_script, tmp_path):
caps = load_script("capabilities.py")
path = write_script(tmp_path, "stranger.py", '"""Nothing is claimed here.\n"""\n')
ids, source = caps.capabilities_for_script("stranger.py", str(path))
assert (ids, source) == ([], "unknown")
assert source != "mapped", "an unknown script must not be reported as mapped to nothing"
+226
View File
@@ -0,0 +1,226 @@
"""Unit tests for skills/exe-reverse/scripts/dart_snapshot_strings.py, the Dart AOT string reader.
The input is an ELF shared object and no sample may be committed, so the container is generated by
`conftest.build_synthetic_elf`: a valid ELF64 header, a `.text` of sixteen bytes, a `.rodata` with
one one-byte (ASCII) literal and one two-byte (UTF-16LE) literal at known offsets, and a
section-name string table. Offsets are read from the fixture's own record of where it put each
literal, never hardcoded here.
What the synthetic image cannot do is be a real snapshot. Literal recovery is what these tests hold:
a one-byte and a two-byte literal at known offsets, a case-insensitive filter, and an offset probe
that resolves anywhere a literal occupies. The instruction-level observations this file once carried
are **withdrawn** -- a byte scan cannot read this format, and the reference records why -- so nothing
here asserts anything about how a reference is encoded.
"""
import pytest
pytestmark = pytest.mark.unit
ONE_BYTE = "one-byte"
TWO_BYTE = "two-byte"
# The two idioms the observer counts, assembled here so the positive cases are built rather than
# asserted from a table. `49 8B /r` is a load through r15 -- Dart's pool pointer; `48 8B 05` is a
# RIP-relative load, which the docstring records as near-absent in a real image.
POOL_LOAD = bytes((0x49, 0x8B, 0x87, 0x10, 0x00, 0x00, 0x00))
FILLER = bytes((0x90,)) * 8
def text_section(data, vaddr=0x1000, offset=0):
return {"offset": offset, "vaddr": vaddr, "size": len(data)}
def hex_bytes(*values):
return bytes(values)
# --- the container -----------------------------------------------------------
def test_the_fixture_is_an_elf_shared_object_the_reader_accepts(load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
elf = dss.Elf(synthetic_elf.data, str(synthetic_elf.path))
assert elf.machine == 0x3E
assert elf.etype == 3
described = elf.describe()
assert described["machine"] == "x86-64"
assert described["type"] == "shared-object"
assert [s["name"] for s in described["sections"]] == [".text", ".rodata", ".shstrtab"]
def test_the_fixture_is_the_size_the_builder_promised(synthetic_elf):
assert len(synthetic_elf.data) == synthetic_elf.size
assert synthetic_elf.size < 0x1000, "a virtual address above this cannot be a file offset"
def test_sections_are_found_by_name(load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
elf = dss.Elf(synthetic_elf.data)
text = elf.section(".text")
rodata = elf.section(".rodata")
assert text["offset"] == synthetic_elf.text["offset"]
assert text["vaddr"] == synthetic_elf.text["vaddr"]
assert text["size"] == synthetic_elf.text["size"]
assert rodata["offset"] == synthetic_elf.rodata["offset"]
assert rodata["vaddr"] == synthetic_elf.rodata["vaddr"]
assert rodata["size"] == synthetic_elf.rodata["size"]
assert elf.section(".absent") is None
@pytest.mark.parametrize("mutation,message", [
(lambda d: bytearray(b"not an elf image at all" + bytes(64)), "not an ELF image"),
(lambda d: bytearray(b"\x7fELF" + d[4:]), None), # a valid image
(lambda d: bytearray(d[:4] + bytes((1,)) + d[5:]), "not a 64-bit ELF"),
(lambda d: bytearray(d[:5] + bytes((2,)) + d[6:]), "not little-endian"),
(lambda d: bytearray(d[:0x30]), "not an ELF image"), # truncated header
])
def test_the_reader_refuses_a_container_it_cannot_read(load_script, synthetic_elf, mutation,
message):
"""A refusal is a ValueError with a reason; the CLI turns it into the documented exit code."""
dss = load_script("dart_snapshot_strings.py")
data = mutation(bytearray(synthetic_elf.data))
if message is None:
assert dss.Elf(bytes(data)).etype == 3
return
with pytest.raises(ValueError) as excinfo:
dss.Elf(bytes(data))
assert message in str(excinfo.value)
def test_the_address_maps_round_trip(load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
elf = dss.Elf(synthetic_elf.data)
for entry in synthetic_elf.literals:
assert elf.vaddr_to_offset(entry["vaddr"]) == entry["offset"]
assert elf.offset_to_vaddr(entry["offset"]) == entry["vaddr"]
assert elf.vaddr_to_offset(0xDEADBEEF) is None
assert elf.offset_to_vaddr(0xFFFFFF) is None
# --- literal recovery --------------------------------------------------------
def test_every_literal_in_the_fixture_is_recovered_at_the_offset_it_was_placed(
load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
elf = dss.Elf(synthetic_elf.data)
recovered = dss.recover_strings(synthetic_elf.data, elf.section(".rodata"))
assert [(off, va, text, enc) for off, va, text, enc in recovered] == [
(entry["offset"], entry["vaddr"], entry["text"], entry["encoding"])
for entry in synthetic_elf.literals]
def test_both_encodings_are_recovered(load_script, synthetic_elf):
"""A keyword search that only reads ASCII misses half of a localised application's literals."""
dss = load_script("dart_snapshot_strings.py")
elf = dss.Elf(synthetic_elf.data)
recovered = dss.recover_strings(synthetic_elf.data, elf.section(".rodata"))
by_encoding = {}
for _off, _va, text, encoding in recovered:
by_encoding.setdefault(encoding, set()).add(text)
assert by_encoding[ONE_BYTE] == {"premium_license", "trial_mode"}
assert by_encoding[TWO_BYTE] == {"vip_pass"}
def test_a_two_byte_literal_reports_byte_offsets_and_character_counts(load_script, synthetic_elf):
"""Pinned, because the two units differ by a factor of two for this encoding.
The offsets are bytes; the text is characters. `--at` filters on `offset <= at < offset +
len(text)`, so for a two-byte literal it accepts only the first half of the bytes the literal
occupies. Measured here as a unit fact; the CLI layer pins the effect.
"""
dss = load_script("dart_snapshot_strings.py")
entry = synthetic_elf.literal("vip_pass")
recovered = dss.recover_strings(synthetic_elf.data,
dss.Elf(synthetic_elf.data).section(".rodata"))
offset, vaddr, text, encoding = [row for row in recovered if row[0] == entry["offset"]][0]
assert (offset, vaddr, encoding) == (entry["offset"], entry["vaddr"], TWO_BYTE)
assert text == "vip_pass"
assert len(text) * 2 == 16, "sixteen bytes on disk, eight characters reported"
def test_a_two_byte_run_does_not_glue_the_tail_of_one_literal_to_the_next(load_script):
"""A run is reported from its own first byte, never from the tail of the literal before it.
This used to be a false positive. The scan accepted *any* printable byte followed by a NUL --
which is exactly the shape of the last character of an ASCII literal plus its alignment byte --
so a two-byte literal beginning right after one was reported as a literal that does not exist,
at an offset inside a different one. Both directions are asserted here, because the cheap fix
for the glue error (skip anything preceded by a character-shaped pair) reintroduces the opposite
one: the real run then starts with its own bytes looking like an interior position.
"""
dss = load_script("dart_snapshot_strings.py")
ascii_part = b"odd_phase_literal\x00"
assert len(ascii_part) % 2 == 0, "the two-byte run needs to start at an even offset"
data = ascii_part + "vip_pass".encode("utf-16-le") + b"\x00\x00"
recovered = dss.recover_strings(data, {"offset": 0, "vaddr": 0x1000, "size": len(data)})
texts = [row[2] for row in recovered]
assert "odd_phase_literal" in texts
assert "vip_pass" in texts, texts
assert "lvip_pass" not in texts, "the tail of the ASCII literal was glued onto the next run"
assert texts.count("vip_pass") == 1
def test_a_two_byte_literal_landing_at_an_odd_offset_is_recovered(load_script):
"""A two-byte run at an odd offset is found, and the run before it does not manufacture one.
Two errors lived here. The scan stepped through the two-byte form from the section base, so a
run shifted to an odd offset -- which one odd-length literal is enough to do -- was never
examined and was missed whole. Advancing one byte at a time fixes that, and the second half of
this test guards the error it introduces: the tail of an even-length one-byte literal is a
printable byte followed by alignment, which reads as the start of a two-byte run and yields a
literal that does not exist.
"""
dss = load_script("dart_snapshot_strings.py")
ascii_part = b"even_phase_liter\x00"
assert len(ascii_part) % 2 == 1, "the two-byte run needs to start at an odd offset"
data = ascii_part + "vip_pass".encode("utf-16-le") + b"\x00\x00"
recovered = dss.recover_strings(data, {"offset": 0, "vaddr": 0x1000, "size": len(data)})
texts = [row[2] for row in recovered]
assert "vip_pass" in texts, texts
assert "even_phase_liter" in texts, texts
# No phantom literal: a run is reported once, from its own first byte.
assert texts.count("vip_pass") == 1
two_byte = [row for row in recovered if row[3] == "two-byte"]
assert len(two_byte) == 1, two_byte
# An even-length one-byte literal before a two-byte run must not start a phantom run.
data2 = b"even_phase_lite\x00" + "vip_pass".encode("utf-16-le") + b"\x00\x00"
got2 = [row[2] for row in dss.recover_strings(
data2, {"offset": 0, "vaddr": 0x1000, "size": len(data2)})]
assert "vip_pass" in got2, got2
assert "vip_pass"[0] not in [t for t in got2 if t != "vip_pass"], got2
def test_min_length_drops_the_shorter_literals(load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
rodata = dss.Elf(synthetic_elf.data).section(".rodata")
long_only = dss.recover_strings(synthetic_elf.data, rodata, min_length=12)
assert [row[2] for row in long_only] == ["premium_license"]
everything = dss.recover_strings(synthetic_elf.data, rodata, min_length=6)
assert {row[2] for row in everything} == {"premium_license", "trial_mode", "vip_pass"}
def test_the_limit_caps_how_many_literals_come_back(load_script, synthetic_elf):
dss = load_script("dart_snapshot_strings.py")
rodata = dss.Elf(synthetic_elf.data).section(".rodata")
assert len(dss.recover_strings(synthetic_elf.data, rodata, limit=1)) == 1
assert len(dss.recover_strings(synthetic_elf.data, rodata, limit=2)) == 2
def test_a_section_claiming_more_bytes_than_the_file_holds_is_clamped(load_script, synthetic_elf):
"""A truncated image must not read past the end; the reader clamps to the data it has."""
dss = load_script("dart_snapshot_strings.py")
truncated = synthetic_elf.data[:synthetic_elf.rodata["offset"] + 0x10]
rodata = dict(synthetic_elf.rodata, size=synthetic_elf.rodata["size"] + 0x1000)
recovered = dss.recover_strings(truncated, rodata)
assert [row[2] for row in recovered] == ["premium_license"]
def test_the_code_section_yields_no_literals(load_script, synthetic_elf):
"""The fixture's `.text` is sixteen bytes of instructions, so nothing survives the length gate."""
dss = load_script("dart_snapshot_strings.py")
text = dss.Elf(synthetic_elf.data).section(".text")
assert dss.recover_strings(synthetic_elf.data, text) == []
# --- pool observation --------------------------------------------------------
+306
View File
@@ -0,0 +1,306 @@
"""Unit tests for skills/exe-reverse/scripts/doctor.py, the environment doctor.
Three properties of the doctor are what make its report a measurement, and each gets a test rather
than a description:
* its per-script dependency pass is static, so a file whose *execution* would raise is inspected
without ever being imported -- proved with a script that writes a sentinel and imports a module
that does not exist;
* `discover_scripts` scans the directory instead of holding a list, proved by pointing the doctor at
a synthetic directory and checking that what it reports is what is there;
* a script with no capability is reported `unknown` and *counted*, because a denominator that
shrinks quietly is how a report stops being a measurement.
The host facts come from a real probe, so a host where the probe cannot answer skips the fact tests
with a printed reason instead of failing them.
"""
import os
import sys
import pytest
pytestmark = pytest.mark.unit
def write(root, name, body):
path = root / name
path.write_text(body, encoding="utf-8")
return path
def synthetic_scripts_dir(tmp_path, sentinel=None):
"""A directory of the six shapes `scan_scripts` has to classify."""
root = tmp_path / "scriptdir"
root.mkdir()
write(root, "declaring.py",
'"""capability: pe_structure_read\n\nDeclares its own capability.\n"""\n'
"import json\n\nprint('usage: declaring')\n")
write(root, "aob_scan.py",
'"""A script the map knows and that declares nothing.\n"""\n'
"import os\n\nprint('usage: aob_scan')\n")
write(root, "needs-tool.py",
'"""Imports a module that is not installed, safely.\n"""\n'
"try:\n import zebra_kit\nexcept ImportError:\n zebra_kit = None\n\n"
"print('usage: needs-tool')\n")
write(root, "broken.py",
'"""Raises as soon as it is executed.\n"""\n'
"raise RuntimeError('this script cannot start')\n")
write(root, "stranger.py",
'"""Neither declares nor is mapped.\n"""\n'
"import sibling_helper\n\nprint('usage: stranger')\n")
write(root, "helper.js", "// not a python script\n")
write(root, "sibling_helper.py",
'"""A file this directory ships, so it is not a third-party dependency.\n"""\n')
if sentinel is not None:
write(root, "sentinel_target.py",
'"""Would write a sentinel the moment it were executed.\n"""\n'
"import pathlib\n\n"
"pathlib.Path(%r).write_text('executed')\n"
"import zebra_kit_that_does_not_exist\n" % str(sentinel))
return root
# --- static dependency pass --------------------------------------------------
def test_static_deps_separates_stdlib_siblings_and_third_party(load_script, tmp_path, monkeypatch):
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
assert doctor.static_python_deps(str(root / "declaring.py")) == (set(), None)
assert doctor.static_python_deps(str(root / "aob_scan.py")) == (set(), None)
assert doctor.static_python_deps(str(root / "stranger.py")) == (set(), None)
deps, reason = doctor.static_python_deps(str(root / "needs-tool.py"))
assert (deps, reason) == ({"zebra_kit"}, None)
def test_static_deps_sees_an_import_inside_a_try_block(load_script, tmp_path, monkeypatch):
"""The pass reads the AST, so an import that is guarded at run time is still a dependency."""
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
deps, _reason = doctor.static_python_deps(str(root / "needs-tool.py"))
assert deps == {"zebra_kit"}
def test_static_deps_does_not_import_or_execute_the_file_it_inspects(load_script, tmp_path,
monkeypatch):
"""The sentinel file is the proof: if the file were imported, it would exist -- or the import of
the deliberately missing module would raise out of `static_python_deps` entirely."""
doctor = load_script("doctor.py")
sentinel = tmp_path / "sentinel.txt"
root = synthetic_scripts_dir(tmp_path, sentinel=sentinel)
monkeypatch.setattr(doctor, "HERE", str(root))
deps, reason = doctor.static_python_deps(str(root / "sentinel_target.py"))
assert deps == {"zebra_kit_that_does_not_exist"}
assert reason is None
assert not sentinel.exists(), "the inspected script was executed"
assert "zebra_kit_that_does_not_exist" not in sys.modules
def test_static_deps_reports_a_syntax_error_instead_of_raising(load_script, tmp_path):
doctor = load_script("doctor.py")
write(tmp_path, "broken.py", "def f(:\n")
deps, reason = doctor.static_python_deps(str(tmp_path / "broken.py"))
assert deps == set()
assert reason
def test_static_deps_reports_an_unreadable_path_instead_of_raising(load_script, tmp_path):
doctor = load_script("doctor.py")
deps, reason = doctor.static_python_deps(str(tmp_path / "absent.py"))
assert deps == set()
assert "absent.py" in reason
def test_every_shipped_script_is_standard_library_only(load_script):
"""The claim the skill's own README makes, measured on the scripts in the tree."""
doctor = load_script("doctor.py")
for name in doctor.discover_scripts():
if not name.endswith(".py"):
continue
deps, reason = doctor.static_python_deps(os.path.join(doctor.HERE, name))
assert reason is None, "%s could not be read: %s" % (name, reason)
assert deps == set(), "%s needs %r, which the kit does not ship" % (name, sorted(deps))
# --- discovery ---------------------------------------------------------------
def test_discover_scripts_reports_whether_files_are_really_there(load_script):
doctor = load_script("doctor.py")
present = sorted(n for n in os.listdir(doctor.HERE) if n.endswith((".py", ".js")))
assert doctor.discover_scripts() == present
for name in ("capabilities.py", "doctor.py", "dart_snapshot_strings.py"):
assert name in present
def test_discover_scripts_scans_a_directory_rather_than_a_fixed_list(load_script, tmp_path,
monkeypatch):
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
found = doctor.discover_scripts()
assert found == sorted(n for n in os.listdir(str(root)) if n.endswith((".py", ".js")))
assert set(found) >= {"declaring.py", "aob_scan.py", "needs-tool.py", "broken.py",
"stranger.py", "helper.js", "sibling_helper.py"}
assert "notes.txt" not in found
write(root, "late_arrival.py", '"""Added after the scan.\n"""\n')
assert "late_arrival.py" in doctor.discover_scripts()
def test_discover_scripts_ignores_files_that_are_not_scripts(load_script, tmp_path, monkeypatch):
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
write(root, "notes.txt", "not a script\n")
write(root, "Makefile", "all:\n\ttrue\n")
monkeypatch.setattr(doctor, "HERE", str(root))
found = doctor.discover_scripts()
assert "notes.txt" not in found
assert "Makefile" not in found
# --- per-script table --------------------------------------------------------
@pytest.fixture(scope="module")
def capability_results(load_script):
caps = load_script("capabilities.py")
return caps.resolve_all(caps.Probe(tool_dirs=[]))
def test_scan_scripts_classifies_every_script_it_finds(load_script, tmp_path, monkeypatch,
capability_results):
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
rows = doctor.scan_scripts(capability_results)
by_name = {row["script"]: row for row in rows}
assert len(rows) == len(doctor.discover_scripts()) == 7
assert by_name["declaring.py"]["capability_source"] == "declared"
assert by_name["declaring.py"]["capabilities"] == ["pe_structure_read"]
assert by_name["aob_scan.py"]["capability_source"] == "mapped"
assert by_name["aob_scan.py"]["capabilities"] == ["aob_locate"]
assert by_name["needs-tool.py"]["capability_source"] == "unknown"
assert by_name["needs-tool.py"]["third_party"] == ["zebra_kit"]
assert by_name["helper.js"]["runnable"] is True
assert "not a python script" in by_name["helper.js"]["detail"]
def test_a_script_with_no_capability_is_unknown_counted_and_never_dropped(
load_script, tmp_path, monkeypatch, capability_results):
"""The denominator is the number of scripts in the directory, unknown ones included."""
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
rows = doctor.scan_scripts(capability_results)
unknown = [row for row in rows if row["capability_source"] == "unknown"]
assert len(rows) == len(doctor.discover_scripts())
assert {row["script"] for row in unknown} == {"needs-tool.py", "broken.py", "stranger.py",
"helper.js", "sibling_helper.py"}
assert len(unknown) == 5, "the unknown rows stay in the table; only two scripts claim one"
assert {row["script"] for row in rows if row["capability_source"] != "unknown"} == \
{"declaring.py", "aob_scan.py"}
for row in unknown:
assert row["capabilities"] == []
assert row["capability_state"] == "unknown"
assert row["script"] in {r["script"] for r in rows}
def test_scan_scripts_marks_a_script_that_cannot_start(load_script, tmp_path, monkeypatch,
capability_results):
doctor = load_script("doctor.py")
root = synthetic_scripts_dir(tmp_path)
monkeypatch.setattr(doctor, "HERE", str(root))
row = {r["script"]: r for r in doctor.scan_scripts(capability_results)}["broken.py"]
assert row["runnable"] is False
assert "RuntimeError" in row["detail"]
assert "Traceback" not in row["detail"]
def test_scan_scripts_agrees_with_the_registry_about_every_real_script(load_script,
capability_results):
"""On the shipped directory, every script is classified and the states come from the registry."""
doctor = load_script("doctor.py")
caps = load_script("capabilities.py")
rows = doctor.scan_scripts(capability_results)
assert [row["script"] for row in rows] == doctor.discover_scripts()
states = {result["id"]: result["state"] for result in capability_results}
for row in rows:
declared, source = caps.capabilities_for_script(row["script"],
os.path.join(doctor.HERE, row["script"]))
assert row["capabilities"] == declared
assert row["capability_source"] == source
if not declared:
assert row["capability_state"] == "unknown"
continue
assert row["capability_state"] == ("ok" if all(states[c] == "ok" for c in declared)
else "blocked")
# --- environment facts -------------------------------------------------------
@pytest.fixture(scope="module")
def host_facts(load_script):
"""One environment report for the module: it runs a probe and shells out to logman."""
doctor = load_script("doctor.py")
report = doctor.environment_report()
if len(report) == 1 and report[0].get("fact") == "host probe":
pytest.skip("the host probe could not answer here (%s), so the fact table is inert"
% report[0].get("effect"))
return report
def test_environment_report_is_a_list_of_facts_with_the_documented_keys(host_facts):
assert isinstance(host_facts, list) and host_facts
for fact in host_facts:
assert set(fact) == {"fact", "value", "effect", "severity"}, sorted(fact)
assert fact["fact"] and fact["value"] is not None and fact["effect"]
def test_every_environment_fact_is_labelled_note_or_warn(host_facts):
"""The label is the whole point: `warn` means it can change a result, and the doctor's exit
token turns on that count."""
for fact in host_facts:
assert fact["severity"] in ("note", "warn"), fact
def test_environment_report_always_states_the_working_directory(host_facts):
"""Unconditional, so the fact table can never be empty on a host where the probe answers."""
facts = {fact["fact"]: fact for fact in host_facts}
assert "working directory" in facts
assert facts["working directory"]["value"] == os.getcwd()
assert facts["working directory"]["severity"] == "note"
def test_severity_summary_counts_only_warnings(load_script):
doctor = load_script("doctor.py")
assert doctor._severity_summary([]) == 0
assert doctor._severity_summary([{"severity": "note"}, {"severity": "note"}]) == 0
assert doctor._severity_summary([{"severity": "note"}, {"severity": "warn"},
{"severity": "warn"}]) == 2
# --- primitives --------------------------------------------------------------
def test_run_reports_a_failure_instead_of_raising(load_script):
doctor = load_script("doctor.py")
code, output = doctor.run(["definitely-not-a-real-command-xyz"])
assert code == -1
assert output
def test_run_returns_the_code_and_combined_output(load_script):
doctor = load_script("doctor.py")
code, output = doctor.run([sys.executable, "-B", "-c", "print('probe')"])
assert code == 0
assert "probe" in output
def test_the_documented_exit_codes_and_tokens_are_the_ones_used(load_script):
doctor = load_script("doctor.py")
assert (doctor.EXIT_OK, doctor.EXIT_USE, doctor.EXIT_ENV, doctor.EXIT_INTERNAL) == (0, 2, 3, 4)
assert doctor.TOKENS["ok"] == "env_ok"
assert doctor.TOKENS["partial"] == "env_partial"
assert doctor.TOKENS["blocked"] == "env_blocked"
assert doctor.TOKENS["use"] == "usage_error"
assert doctor.TOKENS["internal"] == "internal_error"
+138
View File
@@ -0,0 +1,138 @@
"""Unit tests for skills/exe-reverse/scripts/fold.py, the bounded summariser.
What these tests hold is the promise the tool makes about itself: the summary is bounded, it says
what it withheld, and the numbers it prints are true of the file's actual shape.
The shape tests exist because the first version read a pretty-printed 32 MB JSON document line by
line -- its shape decision was the first byte plus a size cutoff, and both a document and a record
file begin with `{` -- and summarised it as 16,278 records with 1.7 million unparseable lines. Every
count in that summary was honestly computed and the summary described a file that did not exist.
The depth scan that replaced it is what the first three tests hold, including the case that breaks
naive counters: braces and escaped quotes inside string values.
"""
import json
import sys
import pytest
pytestmark = pytest.mark.unit
def run_fold(load_script, monkeypatch, capsys, tmp_path, name, content, extra=()):
"""Write the input, run main() as if from the command line, and return (code, stdout)."""
path = tmp_path / name
if isinstance(content, bytes):
path.write_bytes(content)
else:
path.write_text(content, encoding="utf-8")
fold = load_script("fold.py")
monkeypatch.setattr(sys, "argv", ["fold.py", str(path), *extra])
code = fold.main()
out = capsys.readouterr().out
return code, out
# --- the shape is decided by the bytes, not the name or the size ----------------
def test_two_records_on_two_lines_are_records(load_script, monkeypatch, capsys, tmp_path):
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "a.jsonl",
'{"a": 1}\n{"a": 2}\n')
assert code == 0
assert "records : 2" in out
assert "RESULT=fold_ok" in out
def test_a_pretty_printed_document_is_one_value_not_many(load_script, monkeypatch, capsys,
tmp_path):
# The string value carries braces, brackets and escaped quotes of its own: a naive depth count
# is fooled by exactly this, the scanner must not be.
doc = {"outer": {"s": 'a "{[quoted]}" body', "n": 3}, "arr": [1, 2, 3]}
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "graph.json",
json.dumps(doc, indent=2))
assert code == 0
assert "one JSON document" in out
assert "records" not in out
def test_content_after_the_first_value_closes_means_records(load_script, monkeypatch, capsys,
tmp_path):
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "b.jsonl",
'{"a": 1}\n{"a": 2}\n{"a": 3}\n')
assert code == 0
assert "records : 3" in out
def test_a_single_document_past_the_walk_limit_is_left_unwalked(load_script, monkeypatch, capsys,
tmp_path):
fold = load_script("fold.py")
path = tmp_path / "huge.json"
path.write_text('{"a": "%s"}' % ("x" * 40), encoding="utf-8")
monkeypatch.setattr(fold, "JSON_WALK_LIMIT", 8)
monkeypatch.setattr(sys, "argv", ["fold.py", str(path)])
assert fold.main() == 0
out = capsys.readouterr().out
assert "was not loaded" in out
assert "RESULT=fold_ok" in out
# --- the summary admits its bounds ----------------------------------------------
def test_a_key_with_few_values_counts_them_exactly(load_script, monkeypatch, capsys, tmp_path):
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "c.jsonl",
'{"kind": "call"}\n{"kind": "call"}\n{"kind": "ret"}\n',
("--key", "kind", "--limit", "4"))
assert code == 0
assert 'key "kind": 2 distinct value(s) seen (showing 2)' in out
assert "x2" in out
def test_a_key_at_the_tracking_cap_is_reported_as_a_floor(load_script, monkeypatch, capsys,
tmp_path):
fold = load_script("fold.py")
body = "".join(json.dumps({"v": i}) + "\n" for i in range(fold.VALUE_TRACK_CAP + 5))
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "wide.jsonl", body,
("--key", "v", "--limit", "3"))
assert code == 0
assert "at least" in out
assert "floor" in out
# The cap must not leak as an exact count: "at least" only ever prefixes the capped number.
assert "2005 distinct" not in out
def test_grep_reports_how_many_matches_were_withheld(load_script, monkeypatch, capsys, tmp_path):
text = "\n".join("hit %d premium" % i if i % 2 else "miss %d plain" % i
for i in range(10)) + "\n"
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "log.txt", text,
("--grep", "premium", "--limit", "1"))
assert code == 0
assert "matches: 5 (showing 1)" in out
assert "4 more match(es) withheld" in out
def test_text_input_never_touches_the_json_paths(load_script, monkeypatch, capsys, tmp_path):
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "notes.txt",
"first\nsecond\nthird\n", ("--limit", "2"))
assert code == 0
assert "shape : text" in out
assert "lines : 3" in out
# --- the record itself is never the cost ----------------------------------------
def test_the_input_is_not_modified(load_script, monkeypatch, capsys, tmp_path):
body = '{"a": 1}\n{"a": 2}\n'
path = tmp_path / "keep.jsonl"
path.write_text(body, encoding="utf-8")
run_fold(load_script, monkeypatch, capsys, tmp_path, "keep.jsonl", body)
assert path.read_text(encoding="utf-8") == body
def test_a_missing_file_is_a_usage_error_not_a_summary(load_script, monkeypatch, capsys,
tmp_path):
fold = load_script("fold.py")
monkeypatch.setattr(sys, "argv", ["fold.py", str(tmp_path / "absent.jsonl")])
assert fold.main() == 2
assert "RESULT=usage_error" in capsys.readouterr().out
+377
View File
@@ -0,0 +1,377 @@
"""Unit tests for skills/exe-reverse/scripts/pepatch.py.
The refusals matter more than the success path: a patcher that silently picks the wrong
occurrence, or accepts a length change, destroys the baseline it was supposed to preserve. Each
refusal below is asserted against the real exception text, and the CLI-level tests drive
``pepatch.main()`` in-process so no subprocess is required to prove that a dry run writes
nothing and that a real run leaves its input byte-identical.
"""
import hashlib
import shutil
import sys
import pytest
pytestmark = pytest.mark.unit
@pytest.fixture
def pillow(tmp_path, fixtures):
"""A privately owned copy of the fixture, so a test can never disturb another test."""
dest = tmp_path / "input.exe"
shutil.copyfile(str(fixtures.pe64.path), str(dest))
return dest
@pytest.fixture
def plant(fixtures):
"""The byte sequence the fixture plants twice, as a spaced hex string."""
return fixtures.pe64.plant
def sha256(path):
return hashlib.sha256(path.read_bytes()).hexdigest()
def run_main(module, monkeypatch, argv):
monkeypatch.setattr(sys, "argv", [getattr(module, "__file__", "pepatch.py")] + list(argv))
return module.main()
# --- parse_bytes -------------------------------------------------------------
@pytest.mark.parametrize("text,expected", [
("48 8B 05", [0x48, 0x8B, 0x05]),
("488B05", [0x48, 0x8B, 0x05]),
("48,8B,05", [0x48, 0x8B, 0x05]),
("0x48 0x8b", [0x48, 0x8B]),
("48 8B ??", [0x48, 0x8B, None]),
("?", [None]),
("**", [None]),
("xx", [None]),
(" 48 8B ", [0x48, 0x8B]),
("48", [0x48]),
])
def test_parse_bytes_accepts_the_documented_forms(load_script, text, expected):
body, tokens = load_script("pepatch.py").parse_bytes(text)
assert body == expected
assert tokens
@pytest.mark.parametrize("text", ["", None, " "])
def test_parse_bytes_of_nothing_is_empty(load_script, text):
assert load_script("pepatch.py").parse_bytes(text) == ([], [])
@pytest.mark.parametrize("token", ["ZZ", "4", "ABC", "GG HH", "0x1", "48 8G"])
def test_parse_bytes_rejects_a_bad_token(load_script, token):
with pytest.raises(ValueError, match="not a byte"):
load_script("pepatch.py").parse_bytes(token)
def test_parse_bytes_returns_the_original_tokens(load_script):
body, tokens = load_script("pepatch.py").parse_bytes("48 8B ??")
assert tokens == ["48", "8B", "??"]
assert body == [0x48, 0x8B, None]
# --- find_pattern ------------------------------------------------------------
def test_find_pattern_returns_every_occurrence(load_script, fixtures):
"""The fixture plants the same sequence twice, on purpose."""
pepatch = load_script("pepatch.py")
data = fixtures.pe64.data
pattern = pepatch.parse_bytes(fixtures.pe64.plant)[0]
first, second = fixtures.pe64.plant_offsets
assert pepatch.find_pattern(data, pattern) == [first, second]
assert data[first:first + len(pattern)] == bytes(pattern)
assert data[second:second + len(pattern)] == bytes(pattern)
def test_find_pattern_honours_wildcards(load_script):
pepatch = load_script("pepatch.py")
data = bytes([0x00, 0x10, 0x99, 0x10, 0x00])
assert pepatch.find_pattern(data, [None, 0x10]) == [0, 2]
assert pepatch.find_pattern(data, [0x10, None]) == [1, 3]
assert pepatch.find_pattern(data, [None, 0x10, None]) == [0, 2]
def test_find_pattern_start_and_limit(load_script, fixtures):
pepatch = load_script("pepatch.py")
data = fixtures.pe64.data
pattern = pepatch.parse_bytes(fixtures.pe64.plant)[0]
first, second = fixtures.pe64.plant_offsets
assert pepatch.find_pattern(data, pattern, start=first + 1) == [second]
assert pepatch.find_pattern(data, pattern, limit=1) == [first]
def test_find_pattern_reports_no_hits_rather_than_guessing(load_script, fixtures):
"""A negative outcome: a pattern that is not there must not match somewhere else."""
pepatch = load_script("pepatch.py")
assert pepatch.find_pattern(fixtures.pe64.data, [0xDE, 0xAD, 0xBE, 0xEF]) == []
assert pepatch.find_pattern(fixtures.pe64.data,
pepatch.parse_bytes("63 6F 6D 6D 6F 6E")[0]) == []
@pytest.mark.parametrize("pattern", [[], [None], [None, None]])
def test_find_pattern_refuses_an_empty_or_all_wildcard_pattern(load_script, pattern):
with pytest.raises(ValueError, match="empty or entirely wildcards"):
load_script("pepatch.py").find_pattern(b"\x00\x01\x02", pattern)
# --- length_mark -------------------------------------------------------------
def test_length_mark_distinguishes_three_states(load_script):
pepatch = load_script("pepatch.py")
assert pepatch.length_mark({"equal_length": True}) == ""
assert "LENGTH CHANGE" in pepatch.length_mark({"equal_length": False})
assert "length unproven" in pepatch.length_mark({"equal_length": None})
# --- apply_one ---------------------------------------------------------------
def test_ambiguous_pattern_is_refused(load_script, fixtures):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "bytes": "90 " * 12}
with pytest.raises(pepatch.PatchError, match="matched 2 sites"):
pepatch.apply_one(spec, pe, fixtures.pe64.data)
def test_explicit_index_selects_one_site(load_script, fixtures):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
first, second = fixtures.pe64.plant_offsets
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "index": 1,
"bytes": "90 " * 12, "expect": fixtures.pe64.plant}
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data)
assert applied["offset"] == second
assert applied["offset"] != first
assert applied["equal_length"] is True
assert applied["length_basis"] == "expect"
def test_allow_many_takes_the_first_hit(load_script, fixtures):
"""The documented escape hatch: it is not silent, and it still reports one site."""
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "allow-many": True,
"bytes": "90 " * 12, "expect": fixtures.pe64.plant}
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data)
assert applied["offset"] == fixtures.pe64.plant_offsets[0]
def test_expect_mismatch_is_refused(load_script, fixtures):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
with pytest.raises(pepatch.PatchError, match="the site moved or the wrong binary is open"):
pepatch.apply_one({"name": "gate", "offset": "0x410", "expect": "90 90",
"bytes": "CC CC"}, pe, fixtures.pe64.data)
def test_length_is_unproven_without_expect_or_original_length(load_script, fixtures):
"""No disassembler, no honest claim: the marker must say the length is unknown."""
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
applied = pepatch.apply_one({"name": "site", "offset": "0x410", "bytes": "CC"},
pe, fixtures.pe64.data)
assert applied["equal_length"] is None
assert applied["length_basis"] == "unknown (no expect/original-length given)"
assert "length unproven" in pepatch.length_mark(applied)
def test_length_change_is_refused_without_allow_resize(load_script, fixtures):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
spec = {"name": "gate", "offset": "0x410", "expect": "48 8B", "bytes": "CC CC CC"}
with pytest.raises(pepatch.PatchError, match="--allow-resize"):
pepatch.apply_one(spec, pe, fixtures.pe64.data)
def test_length_change_is_marked_structural_with_allow_resize(load_script, fixtures):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
spec = {"name": "gate", "offset": "0x410", "expect": "48 8B", "bytes": "CC CC CC"}
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data, allow_resize=True)
assert applied["equal_length"] is False
assert "LENGTH CHANGE" in pepatch.length_mark(applied)
@pytest.mark.parametrize("spec,expected", [
({"name": "wild", "offset": "0x410", "bytes": "CC ??"}, "no concrete replacement bytes"),
({"name": "tail", "rva": "0x2200", "bytes": "CC"}, "maps to no file offset"),
({"name": "eof", "offset": "0xFF8", "original-length": 16, "bytes": "CC"},
"runs past end of file"),
({"name": "neg", "offset": -1, "bytes": "CC"}, "must not be negative"),
({"name": "nowhere", "bytes": "CC"}, "none of pattern/rva/offset"),
({"name": "oob", "pattern": "48 8B 05 11 22 33 00 48 85 C0 74 05", "index": 5,
"bytes": "CC"}, "index 5 out of range"),
({"name": "none", "pattern": "DE AD BE EF", "bytes": "CC"}, "matched nothing"),
])
def test_resolve_refuses_specs_it_cannot_honour(load_script, fixtures, spec, expected):
pepatch = load_script("pepatch.py")
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
with pytest.raises(pepatch.PatchError, match=expected):
pepatch.resolve(spec, pe, fixtures.pe64.data)
# --- verify ------------------------------------------------------------------
def test_verify_reports_bytes_that_are_not_there(load_script, fixtures):
"""A negative outcome: the re-read step must fail when the patch did not land."""
pepatch = load_script("pepatch.py")
_pe, problems = pepatch.verify(str(fixtures.pe64.path),
[{"name": "ghost", "offset": 0x410, "after": "CC"}])
assert len(problems) == 1
assert "ghost" in problems[0] and "expected CC" in problems[0]
def test_verify_is_satisfied_when_the_bytes_are_present(load_script, fixtures):
pepatch = load_script("pepatch.py")
first, _second = fixtures.pe64.plant_offsets
present = pepatch.fmt_bytes(fixtures.pe64.data[first:first + 4])
_pe, problems = pepatch.verify(str(fixtures.pe64.path),
[{"name": "real", "offset": first, "after": present}])
assert problems == []
# --- main() ------------------------------------------------------------------
def test_dry_run_writes_nothing(load_script, pillow, monkeypatch, capsys):
pepatch = load_script("pepatch.py")
before = sha256(pillow)
code = run_main(pepatch, monkeypatch,
[str(pillow), "--at-offset", "0x410", "--bytes", "CC", "--dry-run"])
out = capsys.readouterr()
assert code == 0
assert "dry run" in out.out
assert sha256(pillow) == before
assert list(pillow.parent.iterdir()) == [pillow], "a dry run created a file"
def test_dry_run_reports_a_refusal_without_writing(load_script, pillow, monkeypatch, capsys):
pepatch = load_script("pepatch.py")
code = run_main(pepatch, monkeypatch,
[str(pillow), "--at-offset", "0x410", "--expect", "90 90",
"--bytes", "CC CC", "--dry-run"])
out = capsys.readouterr()
assert code == 1
assert "FAIL" in out.out
assert list(pillow.parent.iterdir()) == [pillow]
def test_out_is_required_to_write_a_patched_file(load_script, pillow, monkeypatch, capsys):
pepatch = load_script("pepatch.py")
before = sha256(pillow)
code = run_main(pepatch, monkeypatch, [str(pillow), "--at-offset", "0x410",
"--bytes", "CC"])
err = capsys.readouterr().err
assert code == 2
assert "--out is required" in err
assert sha256(pillow) == before
assert list(pillow.parent.iterdir()) == [pillow]
def test_real_run_preserves_the_input_and_verifies_the_output(load_script, pillow, fixtures,
monkeypatch, capsys):
pepatch = load_script("pepatch.py")
peutil = load_script("peutil.py")
before = sha256(pillow)
first, _second = fixtures.pe64.plant_offsets
replacement = "48 8B 05 11 22 33 00 48 85 C0 EB 05"
out = pillow.parent / "patched.exe"
code = run_main(pepatch, monkeypatch,
[str(pillow), "--pattern", fixtures.pe64.plant, "--at-hit", "0",
"--expect", fixtures.pe64.plant, "--bytes", replacement, "--out", str(out)])
text = capsys.readouterr().out
assert code == 0, text
assert "verified" in text
assert sha256(pillow) == before, "the input file was modified"
assert out.exists()
pe_out = peutil.PE(out.read_bytes(), str(out))
assert pe_out.file_size == fixtures.pe64.size
assert out.read_bytes()[first:first + 12] == bytes(pepatch.parse_bytes(replacement)[0])
# The second planted copy must be untouched: only the selected site moved.
assert out.read_bytes()[_second:_second + 12] == bytes(
pepatch.parse_bytes(fixtures.pe64.plant)[0])
assert pe_out.checksum_ok() is True
assert pe_out.checksum != 0
def test_input_and_output_differ_only_where_expected(load_script, pillow, fixtures,
monkeypatch, capsys):
pepatch = load_script("pepatch.py")
first, _second = fixtures.pe64.plant_offsets
checksum_field = int(fixtures.pe64.meta["checksum_offset"], 16)
out = pillow.parent / "one-byte.exe"
code = run_main(pepatch, monkeypatch,
[str(pillow), "--at-offset", hex(first), "--expect", "48 8B",
"--bytes", "CC 8B", "--out", str(out)])
capsys.readouterr()
assert code == 0
diff = [i for i, (a, b) in enumerate(zip(pillow.read_bytes(), out.read_bytes())) if a != b]
# The patched byte plus at most the four checksum bytes.
assert first in diff
assert len(diff) <= 5
assert set(diff) - {first} <= set(range(checksum_field, checksum_field + 4))
def test_no_checksum_leaves_the_declared_field_stale(load_script, pillow, monkeypatch, capsys):
pepatch = load_script("pepatch.py")
peutil = load_script("peutil.py")
out = pillow.parent / "stale.exe"
code = run_main(pepatch, monkeypatch,
[str(pillow), "--at-offset", "0x410", "--expect", "48 8B",
"--bytes", "90 90", "--no-checksum", "--out", str(out), "--json"])
captured = capsys.readouterr()
assert code == 0, captured.err
import json
payload = json.loads(captured.out)
assert payload["checksum_recomputed"] is False
assert payload["checksum_after"]["matches"] is False
assert peutil.PE(out.read_bytes(), None).checksum_ok() is False
def test_allow_resize_shifts_the_following_bytes(load_script, pillow, fixtures,
monkeypatch, capsys):
"""A resize changes the file's length, and everything after the site moves with it.
This is the behaviour the flag promises. An earlier version wrote exactly as many bytes as
it read, so the file never grew while the documented contract said it would -- the flag's
only real effect was to defeat the equal-length guard. The assertions below pin the genuine
consequence, shift included, because a shifted file is a structural edit and the caller is
entitled to see that stated rather than inferred.
"""
pepatch = load_script("pepatch.py")
first, _second = fixtures.pe64.plant_offsets
original = pillow.read_bytes()
out = pillow.parent / "resized.exe"
code = run_main(pepatch, monkeypatch,
[str(pillow), "--at-offset", hex(first), "--expect", "48",
"--bytes", "CC CC CC", "--allow-resize", "--out", str(out), "--json"])
import json
payload = json.loads(capsys.readouterr().out)
assert code == 0
assert payload["structural"] == ["cli"]
assert payload["output_size"] == payload["input_size"] + 2
written = out.read_bytes()
assert len(written) == len(original) + 2
assert written[first:first + 3] == b"\xCC\xCC\xCC"
# One byte was replaced by three, so the tail moved two bytes along.
assert written[first + 3:first + 14] == original[first + 1:first + 12]
def test_failed_patches_stop_the_write(load_script, pillow, plant, monkeypatch, capsys):
"""An ambiguous pattern must fail the run instead of patching the first match."""
pepatch = load_script("pepatch.py")
before = sha256(pillow)
out = pillow.parent / "never.exe"
code = run_main(pepatch, monkeypatch,
[str(pillow), "--pattern", plant, "--bytes", "90 " * 12, "--out", str(out)])
err = capsys.readouterr().err
assert code == 1
assert "refusing to write an output with failed patches" in err
assert "matched 2 sites" in err
assert not out.exists()
assert sha256(pillow) == before
+417
View File
@@ -0,0 +1,417 @@
"""Unit tests for skills/exe-reverse/scripts/peutil.py.
Everything here runs against the generated fixtures, whose declared metadata (section RVAs,
plant offsets, checksum) is asserted against the parser instead of being re-derived from it, so
a parser change cannot make its own test pass.
The checksum check is deliberately a second implementation: if the test called the parser's own
function to produce the expected value, a wrong algorithm would agree with itself.
"""
import hashlib
import struct
import pytest
pytestmark = pytest.mark.unit
def ms_checksum(data, field_offset):
"""The published Microsoft PE checksum, written independently of peutil.py."""
buf = bytearray(data)
buf[field_offset:field_offset + 4] = b"\x00\x00\x00\x00"
total = 0
for i in range(0, len(buf) - (len(buf) % 2), 2):
total += buf[i] | (buf[i + 1] << 8)
total = (total & 0xFFFF) + (total >> 16)
if len(buf) % 2:
total += buf[-1]
total = (total & 0xFFFF) + (total >> 16)
total = (total & 0xFFFF) + (total >> 16)
return (total + len(buf)) & 0xFFFFFFFF
# --- header fields -----------------------------------------------------------
def test_pe32plus_fixture_header_fields_match_the_recipe(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, str(fixtures.pe64.path))
meta = fixtures.pe64.meta
assert pe.is_pe32plus is True
assert pe.magic == 0x20B
assert pe.machine == 0x8664
assert pe.arch == "amd64"
assert pe.number_of_sections == 5
assert pe.number_of_rva_and_sizes == 16
assert [s.name for s in pe.sections] == [".text", ".rdata", ".data", ".rsrc", ".reloc"]
assert pe.address_of_entrypoint == int(meta["entry_rva"], 16) == 0x1000
assert pe.image_base == int(meta["image_base"], 16) == 0x140000000
assert pe.size_of_image == meta["size_of_image"]
assert pe.size_of_headers == meta["size_of_headers"]
assert pe.section_alignment == 0x1000
assert pe.file_alignment == 0x200
assert pe.subsystem == 3
assert pe.timestamp == 0x5F5E1000
assert pe.file_size == meta["size"] == 4096
assert pe.e_lfanew == 0x80
def test_pe32_fixture_reports_the_32bit_layout(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe32.data, str(fixtures.pe32.path))
assert pe.is_pe32plus is False
assert pe.magic == 0x10B
assert pe.machine == 0x014C
assert pe.arch == "i386"
assert pe.image_base == 0x400000
assert pe.number_of_sections == 5
@pytest.mark.parametrize("name,plus", [("pe64", True), ("pe32", False)])
def test_optional_header_field_offsets_follow_the_magic(load_script, fixtures, name, plus):
"""The PE32 and PE32+ optional headers are different layouts, not different sizes."""
pe = load_script("peutil.py").PE(fixtures[name].data, None)
oh = pe.optional_offset
assert pe._num_dir_off - oh == (108 if plus else 92)
assert pe._dir_off - oh == (112 if plus else 96)
assert pe.checksum == struct.unpack_from("<I", pe.d, oh + 64)[0]
if plus:
assert pe.image_base == struct.unpack_from("<Q", pe.d, oh + 24)[0]
else:
assert pe.image_base == struct.unpack_from("<I", pe.d, oh + 28)[0]
@pytest.mark.parametrize("name", ["pe64", "pe32"])
def test_reading_the_other_layout_yields_a_wrong_directory_count(load_script, fixtures, name):
"""The trap the fixture exists to expose: the wrong layout parses without error."""
pe = load_script("peutil.py").PE(fixtures[name].data, None)
wrong = pe._num_dir_off - pe.optional_offset
other = 92 if wrong == 108 else 108
misread = struct.unpack_from("<I", pe.d, pe.optional_offset + other)[0]
assert pe.number_of_rva_and_sizes == 16
assert misread != 16, "the two layouts are not distinguishable on this fixture"
def test_characteristics_and_kind_derivations(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
assert pe.is_dll is False
assert pe.is_driver is False
assert pe.is_dotnet is False
assert pe.is_signed is False
assert pe.security_directory() is None
assert "executable_image" in pe.summary()["file_characteristics"]
assert "large_address_aware" in pe.summary()["file_characteristics"]
def test_fixture_is_well_formed_so_no_warnings_are_raised(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
assert pe.warnings == []
# --- RVA <-> file offset -----------------------------------------------------
@pytest.mark.parametrize("section", [".text", ".rdata", ".data", ".rsrc", ".reloc"])
def test_rva_to_file_offset_round_trips_every_section(load_script, fixtures, section):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
decl = fixtures.pe64.section(section)
rva = int(decl["rva"], 16)
raw = int(decl["raw_pointer"], 16)
assert pe.rva_to_offset(rva) == raw
assert pe.offset_to_rva(raw) == rva
for delta in (1, decl["raw_size"] // 2, decl["raw_size"] - 1):
assert pe.rva_to_offset(rva + delta) == raw + delta
assert pe.offset_to_rva(raw + delta) == rva + delta
assert pe.section_of_rva(rva).name == section
def test_header_rvas_map_identity_below_size_of_headers(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
for rva in (0, 0x40, 0x98, pe.size_of_headers - 1):
assert pe.rva_to_offset(rva) == rva
assert pe.offset_to_rva(rva) == rva
def test_virtual_only_tail_has_no_bytes_on_disk(load_script, fixtures):
"""The .rdata tail is allocated in memory and zero-filled at load, so it has no file bytes."""
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
decl = fixtures.pe64.section(".rdata")
rva = int(decl["rva"], 16)
raw_size = decl["raw_size"]
# Measured shape of this fixture: a 384-byte payload aligned up to a 512-byte raw span,
# plus the recipe's 1024-byte virtual pad, gives 1408 virtual bytes -- so 896 of them have
# no file image at all.
assert decl["virtual_size"] - raw_size == 896
assert fixtures.pe64.meta["virtual_only_tail_bytes"] == 1024
# Last byte with a file image, then the first byte without one.
assert pe.rva_to_offset(rva + raw_size - 1) == int(decl["raw_pointer"], 16) + raw_size - 1
assert pe.rva_to_offset(rva + raw_size) is None
assert pe.rva_to_offset(rva + decl["virtual_size"] - 1) is None
assert pe.rva_bytes(rva + raw_size, 4) == b""
assert pe.section_of_rva(rva + raw_size).name == ".rdata"
# The RVA still exists, it just does not resolve to a file offset.
assert pe.rva_to_offset(rva) is not None
def test_recipe_without_a_virtual_tail_has_none(load_script, fixtures):
"""A 4 KiB file alignment is large enough that every virtual byte has a file image."""
pe = load_script("peutil.py").PE(fixtures.fatalign.data, None)
decl = fixtures.fatalign.section(".rdata")
rva = int(decl["rva"], 16)
assert decl["virtual_size"] <= decl["raw_size"]
assert pe.rva_to_offset(rva + decl["virtual_size"] - 1) is not None
assert pe.rva_to_offset(rva + decl["raw_size"]) is None
def test_offset_to_rva_returns_none_outside_every_mapping(load_script, fixtures):
"""A negative outcome: an offset with no mapping must not be reported as RVA 0."""
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
assert pe.offset_to_rva(pe.file_size) is None
assert pe.offset_to_rva(pe.file_size + 0x1000) is None
def test_rva_to_offset_rejects_unmapped_rvas(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
assert pe.rva_to_offset(0x900000) is None
assert pe.section_of_rva(0x900000) is None
assert pe.read_cstring(None) == ""
assert pe.read_cstring(pe.file_size + 1) == ""
# --- checksum ----------------------------------------------------------------
@pytest.mark.parametrize("name", ["pe64", "pe32", "bare", "fatalign"])
def test_checksum_reproduces_the_declared_value(load_script, fixtures, name):
built = fixtures[name]
pe = load_script("peutil.py").PE(built.data, str(built.path))
declared = int(built.meta["checksum"], 16)
assert pe.checksum == declared
assert pe.computed_checksum() == declared
assert pe.checksum_ok() is True
# Independent implementation of the same published algorithm.
assert ms_checksum(built.data, int(built.meta["checksum_offset"], 16)) == declared
def test_checksum_mismatch_is_reported_not_hidden(load_script, fixtures):
data = bytearray(fixtures.pe64.data)
data[int(fixtures.pe64.meta["checksum_offset"], 16)] ^= 0xFF # corrupt one byte
pe = load_script("peutil.py").PE(bytes(data), None)
assert pe.checksum_ok() is False
assert pe.computed_checksum() != pe.checksum
# --- structures --------------------------------------------------------------
def test_resource_walk_returns_exactly_the_declared_entries(load_script, fixtures):
"""type(16=VERSION) -> name(1) -> language(1033) -> one data entry, and nothing else.
A 32-bit read of either directory count would swallow the next entry and inflate this to a
much larger number, which is why the exact count is asserted rather than "at least one".
"""
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
entries = pe.resources()
assert len(entries) == 1
entry = entries[0]
assert entry["path"] == ["VERSION", "1", "1033"]
assert entry["size"] == 4
assert entry["codepage"] == 1200
assert int(entry["data_rva"], 16) == int(fixtures.pe64.section(".rsrc")["rva"], 16) + 0x90
assert pe.rva_bytes(int(entry["data_rva"], 16), 4) == b"FIXT"
def test_recipe_without_resources_reports_an_empty_tree(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
assert pe.resources() == []
def test_imports_include_one_named_import_and_one_by_ordinal(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
imports = pe.imports()
assert len(imports) == 1
desc = imports[0]
assert desc["dll"] == fixtures.pe64.meta["import_dll"] == "fixture-import.dll"
assert desc["descriptor_rva"] == int(fixtures.pe64.meta["import_descriptor_rva"], 16)
funcs = desc["functions"]
assert len(funcs) == 2, "the descriptor terminator must stop the walk"
named = funcs[0]
assert named["name"] == fixtures.pe64.meta["import_named"] == "ImportedFunction"
assert named["hint"] == 7
assert "ordinal" not in named
by_ordinal = funcs[1]
assert by_ordinal["ordinal"] == int(fixtures.pe64.meta["import_ordinal"], 16) == 0x2A
assert "name" not in by_ordinal
assert funcs[1]["iat_rva"] > funcs[0]["iat_rva"]
assert pe.delay_imports() == []
@pytest.mark.parametrize("name", ["pe64", "pe32"])
def test_export_directory_has_one_named_function(load_script, fixtures, name):
pe = load_script("peutil.py").PE(fixtures[name].data, None)
exports = pe.exports()
assert exports["dll_name"] == "fixture-export.dll"
assert exports["ordinal_base"] == 1
assert exports["number_of_functions"] == 1
assert exports["number_of_names"] == 1
funcs = exports["functions"]
assert len(funcs) == 1
assert funcs[0]["name"] == "ExportedFunction"
assert funcs[0]["ordinal"] == 1
assert funcs[0]["rva"] == "0x1000"
assert "forwarder" not in funcs[0]
def test_recipe_without_exports_returns_none(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
assert pe.exports() is None
def test_tls_callback_rva_is_reachable(load_script, fixtures):
"""The callback pointer is stored as a VA, so the reported RVA needs image-base subtraction."""
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
tls = pe.tls()
assert tls is not None
assert tls["callback_rvas"] == [fixtures.pe64.meta["entry_rva"]] == ["0x1000"]
assert tls["callbacks_rva"] == fixtures.pe64.meta["tls_callback_rva"]
assert tls["note"], "a callback that runs before the entry point must be flagged"
assert pe.imports() and tls["callbacks_rva"] != fixtures.pe64.meta["import_descriptor_rva"]
def test_recipe_without_tls_reports_none(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
assert pe.tls() is None
def test_debug_record_carries_the_rsds_pdb_path(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
records = pe.debug_directory()
assert len(records) == 1
rec = records[0]
assert rec["type"] == "codeview" and rec["type_id"] == 2
assert rec["format"] == "RSDS"
assert rec["pdb"] == fixtures.pe64.meta["pdb_path"] == "C:\\build\\fixture.pdb"
assert rec["age"] == 1
assert rec["guid"] == hashlib.md5(bytes(range(16))).hexdigest()
# PointerToRawData is a file offset while AddressOfRawData is an RVA: prove the pair agrees.
rsds_rva = int(fixtures.pe64.meta["rdata_structure_rvas"]["dbg_rsds"], 16)
assert pe.rva_to_offset(rsds_rva) == int(rec["pointer_to_raw_data"], 16)
assert pe.d[pe.rva_to_offset(rsds_rva):pe.rva_to_offset(rsds_rva) + 4] == b"RSDS"
def test_recipe_without_debug_directory_reports_no_records(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
assert pe.debug_directory() == []
def test_base_relocations_block_is_parsed_by_type(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
relocs = pe.base_relocations()
assert relocs["blocks"] == 1
assert relocs["entries"] == 2
assert relocs["truncated"] is False
assert relocs["types"] == {"3": 1, "10": 1}
def test_absent_load_config_is_reported_as_none(load_script, fixtures):
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
assert pe.load_config() is None
assert pe.rich_header() is None, "the fixture has no Rich header; none may be invented"
def test_summary_is_json_serialisable_and_names_the_documented_keys(load_script, fixtures):
import json
pe = load_script("peutil.py").PE(fixtures.pe64.data, str(fixtures.pe64.path))
summary = pe.summary()
text = json.dumps(summary)
assert '"checksum_matches": true' in text
for key in ("path", "file_size", "sha256", "arch", "kind", "sections", "directories",
"checksum_declared", "checksum_computed", "checksum_matches", "warnings"):
assert key in summary
assert summary["sha256"] == hashlib.sha256(fixtures.pe64.data).hexdigest()
# --- pure helpers ------------------------------------------------------------
def test_entropy_bounds(load_script):
peutil = load_script("peutil.py")
assert peutil.entropy(b"") == 0.0
assert peutil.entropy(b"\x00" * 256) == 0.0
assert peutil.entropy(bytes(range(256))) == pytest.approx(8.0, abs=1e-9)
assert 0.0 <= peutil.entropy(bytes(range(128)) * 2) <= 8.0
def test_section_align_is_an_index_not_a_set_of_bits(load_script):
"""Bits [23:20] hold an alignment index, so a normal image reports no alignment name."""
peutil = load_script("peutil.py")
assert peutil.section_align(0x00600000) == "align_32"
assert peutil.section_align(0xE0000020) is None # the fixture's .text
assert peutil.section_align(0x60000020) is None # a normal compiler's .text
def test_flag_decoding_is_ordered_and_complete(load_script):
peutil = load_script("peutil.py")
assert peutil.bits(0x0022, peutil.FILE_CHARACTERISTICS) == ["executable_image",
"large_address_aware"]
assert peutil.bits(0, peutil.FILE_CHARACTERISTICS) == []
assert "mem_execute" in peutil.bits(0xE0000020, peutil.SECTION_CHARACTERISTICS)
assert "mem_write" in peutil.bits(0xE0000020, peutil.SECTION_CHARACTERISTICS)
def test_integer_readers_are_little_endian(load_script):
peutil = load_script("peutil.py")
blob = b"\x01\x02\x03\x04\x05\x06\x07\x08"
assert peutil.u8(blob, 0) == 0x01
assert peutil.u16(blob, 0) == 0x0201
assert peutil.u32(blob, 0) == 0x04030201
assert peutil.u64(blob, 0) == 0x0807060504030201
assert peutil.s32(b"\xFF\xFF\xFF\xFF", 0) == -1
# --- rejection ---------------------------------------------------------------
@pytest.mark.parametrize("blob,expected", [
(b"", "no MZ signature"),
(b"MZ", "no MZ signature"),
(b"this is not a PE file at all, just text" * 4, "no MZ signature"),
])
def test_parse_error_on_non_pe_input(load_script, blob, expected):
peutil = load_script("peutil.py")
with pytest.raises(peutil.ParseError, match=expected):
peutil.PE(blob, None)
def test_parse_error_when_e_lfanew_points_past_end_of_file(load_script):
peutil = load_script("peutil.py")
blob = b"MZ" + b"\x00" * 0x3A + struct.pack("<I", 0xFFFFFF) + b"\x00" * 8
with pytest.raises(peutil.ParseError, match="points past end of file"):
peutil.PE(blob, None)
def test_parse_error_without_the_pe_signature(load_script):
peutil = load_script("peutil.py")
blob = b"MZ" + b"\x00" * 0x3A + struct.pack("<I", 0x40) + b"\x00" * 0x40
with pytest.raises(peutil.ParseError, match=r"no PE\\0\\0 signature"):
peutil.PE(blob, None)
def test_parse_error_on_an_unexpected_optional_header_magic(load_script, fixtures):
peutil = load_script("peutil.py")
data = bytearray(fixtures.pe64.data)
struct.pack_into("<H", data, 0x98, 0x999)
with pytest.raises(peutil.ParseError, match="unexpected optional-header magic 0x0999"):
peutil.PE(bytes(data), None)
def test_parse_error_on_an_object_file_with_no_optional_header(load_script, fixtures):
peutil = load_script("peutil.py")
data = bytearray(fixtures.pe64.data)
struct.pack_into("<H", data, 0x80 + 4 + 16, 0)
with pytest.raises(peutil.ParseError, match="object file, not an image"):
peutil.PE(bytes(data), None)
def test_load_rejects_a_missing_path(tmp_path, load_script):
peutil = load_script("peutil.py")
with pytest.raises(OSError):
peutil.load(str(tmp_path / "absent.exe"))
def test_load_sets_the_path_on_the_parsed_image(load_script, fixtures):
pe = load_script("peutil.py").load(str(fixtures.pe64.path))
assert pe.path == str(fixtures.pe64.path)
assert pe.file_size == fixtures.pe64.size
+271
View File
@@ -0,0 +1,271 @@
"""Unit tests for skills/exe-reverse/scripts/scan_leaks.py.
Two failure modes are tested at once, because only one of them is obvious. A scanner that misses
a pasted key is dangerous; a scanner that flags `x64dbg`, `PE32+` or a reserved documentation
address gets switched off, which is equally dangerous and much easier to do. So every rule is
asserted to fire on a shaped sample, and every exemption class is asserted to stay quiet.
Three measured observations are pinned here rather than hidden, because each is a false negative
or a dead branch in the rule table:
* the bare-GUID exemption matches inside the machine-guid rule's own match text, so that rule
cannot produce a finding;
* the `.NET` entry in the platform list matches any `.net` TLD, so a hit inside such a URL or
address is suppressed as a platform term;
* the ``<[a-z_]+>`` placeholder alternative is wrapped in ``\\b`` on both sides, so it cannot
match bracketed prose.
The shaped samples come from the corpus in conftest.py, which assembles them from fragments so
that this suite does not itself trip the scan it tests.
"""
import pathlib
import pytest
# The rules the exemption list structurally suppresses: a hit exists, and an exemption claims
# it, so the rule cannot produce a finding. Measured, and re-asserted below so a fix shows up as
# a test change rather than as a quietly gained finding.
SUPPRESSED_RULES = ("machine-guid",)
pytestmark = pytest.mark.unit
def findings_in(scan_leaks, text):
"""What main() would report: hits that no exemption suppresses."""
out = []
for hit in scan_leaks.scan_text(text):
if scan_leaks.exempt_reason(hit["match"]) or scan_leaks.exempt_reason(hit["context"]):
continue
out.append(hit)
return out
@pytest.fixture
def corpus(leak_lines):
return dict(leak_lines)
@pytest.fixture
def exemption_corpus(exempt_lines):
return dict(exempt_lines)
# --- every rule fires --------------------------------------------------------
def test_every_rule_fires_on_its_shaped_sample(load_script, leak_lines):
scan = load_script("scan_leaks.py")
missing = []
for rule, line in leak_lines:
fired = {hit["rule"] for hit in scan.scan_text(line)}
if rule not in fired:
missing.append((rule, sorted(fired)))
assert missing == []
def test_every_rule_except_the_suppressed_ones_is_reportable(load_script, leak_lines):
scan = load_script("scan_leaks.py")
suppressed = set(SUPPRESSED_RULES)
unexpected = []
for rule, line in leak_lines:
reported = {hit["rule"] for hit in findings_in(scan, line)}
if rule in suppressed:
if rule in reported:
unexpected.append((rule, "expected to be suppressed by the exemption list"))
elif rule not in reported:
unexpected.append((rule, "not reported"))
assert unexpected == []
assert suppressed == {"machine-guid"}
def test_the_rule_table_is_well_formed(load_script):
scan = load_script("scan_leaks.py")
names = [name for _c, _s, name, _p, _w in scan.RULES]
assert len(names) == len(set(names)) == len(scan.RULES) == 18
assert {strength for _c, strength, _n, _p, _w in scan.RULES} == {"strong", "weak"}
for category, strength, name, pattern, why in scan.RULES:
assert category and name and why, (category, name)
assert strength in ("strong", "weak")
assert pattern.pattern, name
# --- the three measured deviations ------------------------------------------
def test_machine_guid_rule_fires(load_script, corpus):
scan = load_script("scan_leaks.py")
hits = list(scan.scan_text(corpus["machine-guid"]))
assert [h["rule"] for h in hits] == ["machine-guid"]
def test_machine_guid_finding_is_suppressed_by_the_bare_guid_exemption(load_script, corpus):
"""Measured: the bare-GUID exemption matches inside this rule's own match text.
Recorded here so that a change to either list shows up as a test change instead of as a
silently gained or lost finding.
"""
scan = load_script("scan_leaks.py")
line = corpus["machine-guid"]
hit = list(scan.scan_text(line))[0]
assert "bare GUID is not machine identity" in scan.exempt_reason(hit["match"])
assert findings_in(scan, line) == []
def test_a_url_finding_is_not_swallowed_by_an_exemption(load_script, corpus):
"""A concrete host must still be reported, and only as the weak finding the rule declares.
Measured, twice: while this suite was being written, a standard-name exemption alternative
matched the `HTTP` inside every `https?://` URL and swallowed the rule's every hit. The
assertion below is the state after that was corrected -- a URL hit is reported, at weak
strength, with no exemption claiming it. `benchmark.md` records both states.
"""
scan = load_script("scan_leaks.py")
line = corpus["literal-host-with-path"]
hits = list(scan.scan_text(line))
assert [h["rule"] for h in hits] == ["literal-host-with-path"]
assert hits[0]["strength"] == "weak"
assert scan.exempt_reason(hits[0]["match"]) is None
reported = findings_in(scan, line)
assert [hit["rule"] for hit in reported] == ["literal-host-with-path"]
def test_a_net_domain_hides_a_hit_behind_the_platform_exemption(load_script, corpus,
exemption_corpus):
"""Measured deviation: `.NET` in the platform list matches any `.net` TLD.
A bare domain under `.net` is therefore read as a platform term while the same domain under
another TLD is not, which is a false negative rather than a rule. Pinned, not endorsed.
"""
scan = load_script("scan_leaks.py")
net_url = exemption_corpus["net tld"]
org_url = corpus["literal-host-with-path"]
assert net_url.endswith(".net/v2/activate") and org_url.endswith(".org/v2/activate")
assert "platform or vendor" in scan.exempt_reason(net_url)
# Isolate the cause to the TLD: the same names under another TLD claim no platform exemption.
assert scan.exempt_reason("licensing.hexweld.net") is not None
assert scan.exempt_reason("licensing.hexweld.org") is None
def test_the_bracketed_placeholder_form_is_unreachable_in_prose(load_script):
"""Measured: `<[a-z_]+>` sits inside `\\b...\\b`, so bracketed prose never matches."""
scan = load_script("scan_leaks.py")
assert scan.exempt_reason("<user>") is None
assert scan.exempt_reason("a <user> b") is None
assert scan.PLACEHOLDER_HINT.search("a <user> b") is None
# Only word-character adjacency reaches it, which is why a bare TARGET is the form that works.
assert scan.PLACEHOLDER_HINT.search("x<user>x") is not None
assert scan.exempt_reason("TARGET") == "placeholder token"
# --- exemptions --------------------------------------------------------------
@pytest.mark.parametrize("sample,reason", [
("x64dbg", "tool or library name"),
("WinDbg", "tool or library name"),
("IDA", "tool or library name"),
("CFF Explorer", "tool or library name"),
("the copy is packed with UPX", "public protector/packer/managed tooling name"),
("Themida", "public protector/packer/managed tooling name"),
("IMAGE_NT_HEADERS", "format, standard or platform term"),
("CWE-119", "format, standard or platform term"),
("Authenticode", "format, standard or platform term"),
("Windows", "platform or vendor-of-the-platform term"),
("PowerShell", "platform or vendor-of-the-platform term"),
("TARGET", "placeholder token"),
("YOURAPP", "placeholder token"),
("REDACTED", "placeholder token"),
("x<user>x", "placeholder token"),
("192.0.2.44", "reserved documentation address"),
("198.51.100.7", "reserved documentation address"),
("203.0.113.9", "reserved documentation address"),
("127.0.0.1", "reserved documentation address"),
("S-1-5-18", "well-known SID"),
("3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071", "bare GUID is not machine identity"),
])
def test_exempt_reason_classifies_the_classes_that_must_stay(load_script, sample, reason):
"""The reason is asserted by the class it names, not by its exact prose.
The wording of a reason is a message, and messages get reworded; which exemption class
claims a sample is the behaviour under test.
"""
assert reason in load_script("scan_leaks.py").exempt_reason(sample)
def test_exempt_reason_does_not_excuse_a_real_finding(load_script, corpus):
"""A negative outcome: the exemption list must not swallow target identity."""
scan = load_script("scan_leaks.py")
for rule in ("private-key-block", "github-token", "windows-user-profile", "posix-home",
"windows-sid-literal", "volume-serial-literal", "product-name-with-version"):
assert scan.exempt_reason(corpus[rule]) is None, rule
@pytest.mark.parametrize("label", [
"tool name", "protector name", "format term", "platform term", "placeholder token",
"documentation address", "well-known SID", "loopback and example", "machine guid context",
"net tld",
])
def test_exemption_classes_are_never_reported(load_script, exemption_corpus, label):
assert findings_in(load_script("scan_leaks.py"), exemption_corpus[label]) == [], label
# --- scan_text ---------------------------------------------------------------
def test_scan_text_reports_position_and_strength(load_script, corpus):
scan = load_script("scan_leaks.py")
text = "line one is clean\nsecond line: " + corpus["private-key-block"] + "\n"
hits = list(scan.scan_text(text))
assert len(hits) == 1
hit = hits[0]
assert hit["line"] == 2
assert hit["col"] == 14
assert hit["strength"] == "strong"
assert hit["category"] == "token"
assert hit["rule"] == "private-key-block"
assert hit["why"]
assert corpus["private-key-block"] in hit["match"]
assert hit["context"].startswith("second line:")
def test_clean_text_produces_no_hits_at_all(load_script):
"""A negative outcome: a clean note must not produce a single hit of any strength."""
scan = load_script("scan_leaks.py")
text = ("## Triage\n\nRead the section table with peutil.py and printed the entropy.\n"
"The checksum matched the declared value, so the image was not hand-edited.\n")
assert list(scan.scan_text(text)) == []
def test_scan_text_does_not_report_the_same_line_as_a_different_rule(load_script, corpus):
scan = load_script("scan_leaks.py")
hits = scan.scan_text(corpus["generic-secret-assignment"])
assert [h["rule"] for h in hits] == ["generic-secret-assignment"]
# --- file selection ----------------------------------------------------------
def test_iter_files_skips_binaries_and_cache_directories(load_script, tmp_path, corpus):
"""A negative outcome: the scanner must not walk binaries, caches or VCS metadata."""
scan = load_script("scan_leaks.py")
root = tmp_path / "repo"
(root / "__pycache__").mkdir(parents=True)
(root / ".git").mkdir()
(root / "notes.md").write_text("clean\n", encoding="utf-8")
(root / "trap.exe").write_bytes(corpus["private-key-block"].encode("ascii") + b"\n")
(root / "trap.png").write_bytes(corpus["github-token"].encode("ascii"))
(root / "__pycache__" / "cached.md").write_text(corpus["private-key-block"] + "\n",
encoding="utf-8")
(root / ".git" / "config.md").write_text(corpus["private-key-block"] + "\n",
encoding="utf-8")
found = sorted(pathlib.Path(p).relative_to(root).as_posix()
for p in scan.iter_files(str(root)))
assert found == ["notes.md"]
def test_iter_files_honours_an_explicit_file_list(load_script, tmp_path):
scan = load_script("scan_leaks.py")
root = tmp_path / "repo"
root.mkdir()
(root / "a.md").write_text("a\n", encoding="utf-8")
(root / "b.md").write_text("b\n", encoding="utf-8")
listing = tmp_path / "listed.txt"
listing.write_text("a.md\nmissing.md\n\n", encoding="utf-8")
found = sorted(pathlib.Path(p).relative_to(root).as_posix()
for p in scan.iter_files(str(root), str(listing)))
assert found == ["a.md"]
+185
View File
@@ -0,0 +1,185 @@
"""Unit tests for skills/exe-reverse/scripts/trace_reduce.py, the bit-vector reducer.
Three layers, each held separately because they make different promises:
* the rewriting core -- the classic mixed-boolean-arithmetic substitutions reduce to `x + y` and
survive a sampling check, and a genuinely different pair is caught with a counterexample;
* the SMT emitter -- pure text, checked exactly, because the solver will execute it verbatim and a
proof about wrongly-encoded semantics is worse than no proof. The shift-count mask is the case
that matters: `evaluate()` masks a count to six bits, and the encoding must do the same or the
solver proves a cleaner function than the one the samples ran on;
* the proof path -- driven against a stub solver (one line of output) and against a missing path,
never against a real solver: CI has none, and a test that needs one is a test that skips.
"""
import sys
import pytest
pytestmark = pytest.mark.unit
def parse(tr, text):
return tr.Parser(tr.tokenize(text)).parse()
# --- the rewriting core ---------------------------------------------------------
def test_the_two_classic_substitutions_reduce_to_add(load_script):
tr = load_script("trace_reduce.py")
for obfuscated in ("(x | y) + (x & y)", "(x ^ y) + 2 * (x & y)"):
tree = parse(tr, obfuscated)
reduced, steps = tr.reduce_all(tree, tr.MASK[32])
assert reduced.text() == "x + y"
assert steps, "a reduction with no recorded steps changed nothing"
equal, counter, _ = tr.verify(tree, reduced, tr.MASK[32], 64)
assert equal, "counterexample: %r" % (counter,)
def test_a_genuinely_different_pair_is_refuted_with_a_witness(load_script):
tr = load_script("trace_reduce.py")
original, other = parse(tr, "x + y"), parse(tr, "x | y")
equal, counter, checked = tr.verify(original, other, tr.MASK[32], 64)
assert checked > 0
assert not equal
# The witness is re-checked here rather than trusted: the point of a counterexample is that it
# can be substituted by hand.
env = counter["env"]
assert tr.evaluate(original, env, tr.MASK[32]) != tr.evaluate(other, env, tr.MASK[32])
# --- the SMT emitter ------------------------------------------------------------
def test_a_constant_is_emitted_at_the_full_word_width(load_script):
tr = load_script("trace_reduce.py")
term = parse(tr, "0x2a")
assert tr.smt2_term(term, 32) == "#x0000002a"
assert tr.smt2_term(term, 8) == "#x2a"
def test_every_operator_maps_to_its_bit_vector_counterpart(load_script):
tr = load_script("trace_reduce.py")
assert tr.smt2_term(parse(tr, "x + y"), 32) == "(bvadd x y)"
assert tr.smt2_term(parse(tr, "x - y"), 32) == "(bvsub x y)"
assert tr.smt2_term(parse(tr, "x * y"), 32) == "(bvmul x y)"
assert tr.smt2_term(parse(tr, "x & y"), 32) == "(bvand x y)"
assert tr.smt2_term(parse(tr, "x | y"), 32) == "(bvor x y)"
assert tr.smt2_term(parse(tr, "x ^ y"), 32) == "(bvxor x y)"
assert tr.smt2_term(parse(tr, "~x"), 32) == "(bvnot x)"
assert tr.smt2_term(parse(tr, "-x"), 32) == "(bvneg x)"
def test_a_shift_count_is_masked_exactly_as_evaluate_masks_it(load_script):
tr = load_script("trace_reduce.py")
# evaluate() shifts by `count & 63`; the encoding must assert the same thing, or the solver
# proves a different function than the one the samples ran on.
assert tr.smt2_term(parse(tr, "x << y"), 32) == "(bvshl x (bvand y #x0000003f))"
assert tr.smt2_term(parse(tr, "x >> y"), 64) == "(bvlshr x (bvand y #x000000000000003f))"
def test_the_script_declares_variables_asserts_a_difference_and_asks(load_script):
tr = load_script("trace_reduce.py")
script = tr.smt2_for(parse(tr, "y + x"), parse(tr, "x + y"), 32)
lines = script.strip().splitlines()
assert lines[0] == "(set-logic QF_BV)"
assert "(declare-fun x () (_ BitVec 32))" in lines
assert "(declare-fun y () (_ BitVec 32))" in lines
assert any(line.startswith("(assert (distinct") for line in lines)
assert "(check-sat)" in lines
assert "(get-model)" in lines
def test_a_model_is_read_into_an_environment_and_masked(load_script):
tr = load_script("trace_reduce.py")
model = """
sat
(
(define-fun x () (_ BitVec 32) #x00000001)
(define-fun y () (_ BitVec 32) #x00000003)
(define-fun ignored () Int 7)
)
"""
assert tr.model_env(model, tr.MASK[32]) == {"x": 1, "y": 3}
assert tr.model_env("unsat", tr.MASK[32]) == {}
# --- the proof path, against stubs ----------------------------------------------
@pytest.fixture
def unsat_solver(tmp_path):
"""A stand-in solver that answers `unsat`: one line, no matter the question."""
if sys.platform == "win32":
path = tmp_path / "unsat.bat"
path.write_text("@echo unsat\r\n", encoding="ascii")
else:
path = tmp_path / "unsat.sh"
path.write_text("#!/bin/sh\necho unsat\n", encoding="ascii")
path.chmod(0o755)
return str(path)
@pytest.fixture
def lying_sat_solver(tmp_path):
"""A stand-in that says `sat` and shows a witness that does not distinguish the forms.
The reducer's own rules are identities, so a real sat never appears in a run of this tool; a
lying one is the case that must not be trusted on arrival, and this stub is how that is held.
"""
body = ("sat\n(\n(define-fun x () (_ BitVec 32) #x00000000)\n"
"(define-fun y () (_ BitVec 32) #x00000000)\n)\n")
if sys.platform == "win32":
path = tmp_path / "sat.bat"
path.write_text("@echo sat\r\n@echo (\r\n@echo (define-fun x () (_ BitVec 32) "
"#x00000000)\r\n@echo (define-fun y () (_ BitVec 32) #x00000000)\r\n"
"@echo )\r\n", encoding="ascii")
else:
path = tmp_path / "sat.sh"
path.write_text("#!/bin/sh\ncat <<'EOF'\n%s\nEOF\n" % body, encoding="ascii")
path.chmod(0o755)
return str(path)
def test_an_unsat_answer_upgrades_the_verdict_to_proved(load_script, unsat_solver):
tr = load_script("trace_reduce.py")
original, reduced = parse(tr, "(x ^ y) + 2 * (x & y)"), parse(tr, "x + y")
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32, unsat_solver, 10)
assert verdict == "proved"
def test_a_witness_that_does_not_distinguish_is_not_believed(load_script, lying_sat_solver):
tr = load_script("trace_reduce.py")
original, reduced = parse(tr, "(x ^ y) + 2 * (x & y)"), parse(tr, "x + y")
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32, lying_sat_solver, 10)
assert verdict == "no-verdict"
assert "does not distinguish" in info["reason"]
def test_a_missing_solver_path_is_reported_not_crashed(load_script, tmp_path):
tr = load_script("trace_reduce.py")
original, reduced = parse(tr, "x + y"), parse(tr, "x + y")
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32,
str(tmp_path / "not-there.exe"), 10)
assert verdict == "no-verdict"
assert "does not exist" in info["reason"]
def test_prove_with_a_missing_path_is_a_usage_error_at_the_cli(load_script, monkeypatch, capsys,
tmp_path):
tr = load_script("trace_reduce.py")
monkeypatch.setattr(sys, "argv", ["trace_reduce.py", "x + 0",
"--prove", str(tmp_path / "absent.exe")])
assert tr.main() == 2
out = capsys.readouterr()
assert "RESULT=usage_error" in out.out
def test_a_proved_run_reports_the_proved_token(load_script, monkeypatch, capsys, unsat_solver):
tr = load_script("trace_reduce.py")
monkeypatch.setattr(sys, "argv", ["trace_reduce.py", "(x ^ y) + 2 * (x & y)",
"--prove", unsat_solver])
assert tr.main() == 0
out = capsys.readouterr().out
assert "RESULT=reduce_proved" in out
assert "every input" in out