mirror of
https://codeberg.org/reverse-skill-lab/exe-reverse
synced 2026-11-17 04:54:20 +08:00
Init
This commit is contained in:
+135
@@ -0,0 +1,135 @@
|
||||
# tests/ — executable regression suite for the exe-reverse skill
|
||||
|
||||
The suite that proves the shipped scripts still do what their docstrings say, offline, with only
|
||||
the standard library and pytest.
|
||||
|
||||
## When to load this
|
||||
|
||||
Read it before changing anything under `skills/exe-reverse/scripts/`, and read it before quoting
|
||||
a coverage claim about this skill. It is the boundary between "the script was written to do X"
|
||||
and "the script was measured doing X on this host".
|
||||
|
||||
## What it runs
|
||||
|
||||
| File | What it asserts |
|
||||
| --- | --- |
|
||||
| `conftest.py` | Shared fixtures: builds the synthetic PE images at test time, loads a script by path, runs a script as a subprocess with a timeout, and the two synthetic scan corpora. |
|
||||
| `unit/test_peutil_reader.py` | Header fields against the fixture's declared metadata, RVA to file-offset round trips including the virtual-only tail, the PE32 versus PE32+ optional-header difference, the checksum against a second implementation, the resource tree, named and ordinal imports, the TLS callback, the RSDS/PDB record, relocation blocks, entropy and flag decoding, and the parser's rejections. |
|
||||
| `unit/test_pepatch_patcher.py` | Pattern parsing and matching, the ambiguity refusal, the expect-mismatch refusal, the length-unproven and length-change cases, the resolver's refusals, the re-read verification, and the CLI-level facts that a dry run writes nothing and a real run leaves its input byte-identical. |
|
||||
| `unit/test_aob_scan.py` | The verdict boundaries at both sides of each threshold, hit reporting with offsets, RVAs and sections, section filtering, context windows, and every accepted shape of a pattern file. |
|
||||
| `unit/test_scan_leaks.py` | Every rule firing on a shaped sample, every exemption class staying silent, the three measured deviations from the exemption list, scan position reporting, and the file walk's skips. |
|
||||
| `unit/test_capabilities_registry.py` | The capability table: every atom's kind, basis and probe name, the transitive closure against an independent walk, the dependency graph's acyclicity, `resolve` on missing atoms and on `file:` atoms that are and are not on disk, the declaration parser including the documentation template it must reject, and the `declared` / `mapped` / `unknown` classification. |
|
||||
| `unit/test_doctor.py` | The static dependency pass (stdlib, siblings and a third-party name separated without executing the file), `discover_scripts` scanning rather than listing, the per-script table with an unclassified script still counted, and the environment facts with their keys and severity labels. |
|
||||
| `unit/test_dart_snapshot_strings.py` | The ELF reader against a generated container, both literal encodings recovered at the offsets the fixture placed them at, the length and limit gates, and the pool counts -- zero on an image carrying neither idiom and non-zero on hand-built bytes carrying each one. |
|
||||
| `cli/test_cli_surface.py` | Help and usage exit codes for every script, the documented exit codes for each script's error paths, and the negative case that inspect-only runs write nothing into the working directory. |
|
||||
| `cli/test_cli_json.py` | Each script's machine-readable output: keys, values, and the conventions that are not what a reader would assume. |
|
||||
| `cli/test_cli_registry_surface.py` | The registry and doctor subprocess contracts: `--list` / `--json` / single-capability exit codes and `RESULT=` tokens, the doctor's three blocks and its 0-versus-3 verdict, and the view flags that print one block only. |
|
||||
| `cli/test_cli_dart_snapshot.py` | The snapshot reader's CLI: the refusals and their exit codes, `--find` case-insensitivity and the exit code a miss produces, `--at` by file offset and by virtual address, and the `--json` document. |
|
||||
| `integration/test_pipeline_end_to_end.py` | Build, locate, patch, re-read: the patched bytes, the untouched control site, the recomputed checksum, and a control run in which nothing changes. |
|
||||
| `integration/test_across_builds.py` | Pattern hit counts compared between two builds, including the case where a patch removes the very byte the pattern was located with. |
|
||||
| `run_tests.py` | The one-command entry point; prints the install line and exits 3 when pytest is missing. |
|
||||
| `benchmark.md` | The regression matrix: dimension, target, scripts, measured result, strength label. |
|
||||
|
||||
## How to run it
|
||||
|
||||
```
|
||||
python -m pytest -q # from the repository root: the whole suite
|
||||
python -m pytest -q -m unit # one layer: unit, cli or integration
|
||||
python tests/run_tests.py # same thing, with a clear message if pytest is missing
|
||||
python tests/run_tests.py -q -m unit # arguments are forwarded to pytest
|
||||
```
|
||||
|
||||
If pytest is not installed: `python -m pip install pytest`.
|
||||
|
||||
## What it deliberately does not do
|
||||
|
||||
It does not attach a debugger, load a driver, open a socket, read a device, or execute a target.
|
||||
Those routes are conditional on host hardware and policy, and a suite that asserts them is either
|
||||
fragile or dishonest. The recorded experiments for them live in `benchmark.md` with the strength
|
||||
label that says whether they were reproduced here.
|
||||
|
||||
It does not commit or download a sample. Every PE input is generated into a temporary directory
|
||||
by `make_fixture_pe.py` on the way in and removed with the session, so the suite carries no
|
||||
binary, no licence question and no dependency on a target that changes underneath it.
|
||||
|
||||
It does not assert on timing. The only time in the suite is a generous per-process timeout, which
|
||||
exists to turn a hang into a failure rather than to measure throughput. The timings in
|
||||
`benchmark.md` were taken by hand and are recorded as measurements, not enforced as limits.
|
||||
|
||||
It does not test the reference documents' prose. Cross-reference, budget and routing checks belong
|
||||
to the repository's own gates.
|
||||
|
||||
## Fixtures
|
||||
|
||||
`conftest.py` builds four images per session, addressing `make_fixture_pe.py`'s own recipes rather
|
||||
than duplicating their shapes:
|
||||
|
||||
| Name | Recipe | Why it is in the suite |
|
||||
| --- | --- | --- |
|
||||
| `pe64` | `default` | PE32+ with every structure the builder can emit, five sections and the planted pair. |
|
||||
| `pe32` | `pe32` | The 32-bit optional-header layout, whose field offsets differ. |
|
||||
| `bare` | `bare` | No exports, resources, TLS or debug directory, so the absent cases are representable. |
|
||||
| `fatalign` | `fat-align` | A 4 KiB file alignment, large enough that `.rdata` has no virtual-only tail. |
|
||||
|
||||
The builder's `no-virtual-tail` recipe is deliberately not used: with a zero virtual pad its own
|
||||
guard rejects the layout it names, so the recipe exits 1 and writes nothing. `fatalign` covers the
|
||||
same property. `cli/test_cli_surface.py` pins that failure so the recipe cannot quietly stay
|
||||
broken.
|
||||
|
||||
`conftest.build_synthetic_elf` builds a second kind of input, because `dart_snapshot_strings.py`
|
||||
reads an ELF shared object rather than a PE and no sample of any kind may be committed: a valid
|
||||
ELF64 header, a `.text` of sixteen bytes carrying neither idiom the pool observer counts, a
|
||||
`.rodata` holding one one-byte (ASCII) literal and one two-byte (UTF-16LE) literal at known
|
||||
offsets, and a section-name string table. It is a container to hold the reader against, not a
|
||||
Flutter snapshot: the counts the script's docstring records came from a real 26.8 MB `app.so` and
|
||||
cannot be reproduced from a generated file, which is why the counting logic is asserted in both
|
||||
directions -- zero on the fixture, non-zero on hand-built bytes -- and the real measurement stays a
|
||||
recorded observation.
|
||||
|
||||
## Conventions
|
||||
|
||||
Marker is the layer: `unit` for pure functions and parsers, `cli` for subprocess contracts,
|
||||
`integration` for the no-debugger flows. All three are declared in `pytest.ini`, and
|
||||
`--strict-markers` is on, so an undeclared marker is an error rather than a warning. There is no
|
||||
slow or deselected tier and therefore no `--runslow` switch: nothing in the suite needs one, and
|
||||
declaring a marker that no test carries would be decoration.
|
||||
|
||||
A missing prerequisite skips with a printed reason: an interpreter older than the 3.9 the scripts
|
||||
declare, or a local tool a check needs. A wrong result always fails. Nothing is skipped silently,
|
||||
and no test was weakened to pass.
|
||||
|
||||
Untrusted sample text is assembled from fragments in `conftest.py` rather than quoted whole. The
|
||||
suite has to contain strings shaped like pasted credentials and profile paths in order to prove
|
||||
the scanner catches them; written whole they would also be caught by the repository's own
|
||||
pre-publish scan. The runtime values are identical, and the source text matches nothing.
|
||||
|
||||
## Deviations the suite pins
|
||||
|
||||
Each of these is a measured behaviour that differs from what the corresponding docstring implies.
|
||||
They are asserted as observations so a fix shows up as a test change instead of as a quietly
|
||||
gained or lost result, and each is also recorded in `benchmark.md`.
|
||||
|
||||
| Where | Measured behaviour |
|
||||
| --- | --- |
|
||||
| `make_fixture_pe.py` | The `no-virtual-tail` recipe contradicts its own guard and cannot be built. |
|
||||
| `peutil.py`, `pepatch.py` | A file truncated mid-header raises `struct.error` and prints a traceback instead of using the documented exit code; a fully non-PE input does use it. |
|
||||
| `pepatch.py` | A missing `--spec` file reaches an unguarded `open()` and prints a traceback. |
|
||||
| `aob_scan.py` | A bare JSON array of patterns raises `AttributeError`; only the object form is accepted. |
|
||||
| `scan_leaks.py` | Exit code 3 is documented for an empty scan surface and is unreachable: an empty root reports `RESULT=clean` and exits 0. |
|
||||
| `scan_leaks.py` | The bare-GUID exemption matches inside the machine-guid rule's own match text, so that rule cannot produce a finding. |
|
||||
| `scan_leaks.py` | The `.NET` entry matches any `.net` TLD, so a hit inside such a host is suppressed as a platform term. |
|
||||
| `scan_leaks.py` | The `<[a-z_]+>` placeholder alternative is wrapped in `\b`, so it cannot match bracketed prose. |
|
||||
| `hwprobe.py` | Two route rows answer with a sentence instead of one of the three documented states. |
|
||||
| `pepatch.py` | `--allow-resize` writes in place, so the file does not grow and the bytes after the site are overwritten rather than shifted. |
|
||||
|
||||
## Limits
|
||||
|
||||
Coverage is bounded by the fixture, not by the reader. A synthetic image exercises every structure
|
||||
the builder emits and nothing it does not; a route that needs a device, a debugger, a driver or a
|
||||
network is out of scope here and is listed as not run in `benchmark.md`. The suite also cannot say
|
||||
whether a pattern will survive a build it has never seen, which is why the comparisons it does make
|
||||
are between two images it built itself.
|
||||
|
||||
The scripts under `skills/exe-reverse/scripts/` were being edited while this suite was written, and
|
||||
several assertions moved with them. Re-run the suite after any change to those files rather than
|
||||
trusting a count recorded earlier.
|
||||
@@ -0,0 +1,104 @@
|
||||
# Regression matrix — exe-reverse
|
||||
|
||||
The measured record behind `tests/`: one row per dimension, with the target, the scripts it
|
||||
exercised, the result this host produced, and how strong that result is.
|
||||
|
||||
## When to load this
|
||||
|
||||
Read it when you need to know what was actually run rather than what the suite covers, or when a
|
||||
coverage claim needs a target and a number behind it. `tests/README.md` describes the suite;
|
||||
this file records its output.
|
||||
|
||||
## How a row is filled in
|
||||
|
||||
Every row below was produced by running the command in it on this host, except where the row says
|
||||
otherwise in the result column. Rows that were not run say so and are not filled with an estimate.
|
||||
Strength labels follow the repository convention: `observed` means the command and its output
|
||||
exist, `inferred` means the result is reasoned from an observation, `unverified` means it is
|
||||
reported but was not reproduced here.
|
||||
|
||||
Targets are public and legally usable: the generated fixture, and platform binaries that ship with
|
||||
the operating system. No third-party product is a target, and no sample is stored in the
|
||||
repository — the images are built at test time into a temporary directory.
|
||||
|
||||
Environment: Windows 11 build 26100, Python 3.14.0, pytest 8.4.1.
|
||||
|
||||
## The matrix
|
||||
|
||||
| Dimension | Target | Scripts exercised | Measured result | Strength |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| PE32+ header surface | generated fixture, `default` recipe | `peutil.py` | 4096 bytes, 5 sections, magic 0x20B, entry RVA 0x1000, image base 0x140000000, declared checksum 0x10bad, zero warnings | observed |
|
||||
| PE32 header surface | generated fixture, `pe32` recipe | `peutil.py` | magic 0x10B, machine 0x014C, image base 0x400000, same five sections | observed |
|
||||
| Optional-header layout difference | both fixtures | `peutil.py` | number-of-directories lives at optional+92 (PE32) or optional+108 (PE32+), directories at +96 or +112; reading the other layout yields 40 and 0 respectively, neither being the real count 16 | observed |
|
||||
| RVA to file offset, both directions | generated fixture | `peutil.py` | 0x1010 to 0x410 and back; header RVAs identity below SizeOfHeaders 0x400; an RVA in no section maps to nothing | observed |
|
||||
| Virtual-only tail | generated fixture | `peutil.py` | `.rdata` has 1408 virtual and 512 raw bytes, so 896 bytes have no file image; RVA 0x2200 resolves to nothing while 0x21FF resolves to 0x9FF | observed |
|
||||
| PE checksum | four generated recipes | `peutil.py` | declared equals computed equals a second, independent implementation of the published algorithm | observed |
|
||||
| PE checksum on real images | platform binaries under `%SystemRoot%\System32` | `peutil.py` | `notepad.exe` (360448 bytes) declared 0x5e673 equals computed; `kernel32.dll` and `ntdll.dll` parse with matching checksums | observed |
|
||||
| Imports | generated fixture | `peutil.py` | one DLL with one named import (hint 7) and one import by ordinal 0x2A, descriptor terminator respected, no delay imports | observed |
|
||||
| Exports, resources, TLS, debug, relocations | generated fixture | `peutil.py` | one named export; the resource tree is exactly VERSION / 1 / 1033 with a 4-byte leaf; TLS callback RVA 0x1000; RSDS record with the fixture's PDB path and age 1; one relocation block with two entries | observed |
|
||||
| Pattern locating, three verdicts | generated fixture | `aob_scan.py` | the planted pair reports 2 hits as `ambiguous` and exits 1; the PDB string reports 1 hit as `unique` and exits 0; an absent pattern reports 0 as `no-match` and exits 1 | observed |
|
||||
| Verdict boundaries | generated fixture | `aob_scan.py` | 0 to `no-match`, 1 to `unique`, 2 through 7 to `ambiguous`, 8 and above to `pervasive` | observed |
|
||||
| Equal-length patch and verification | generated fixture | `pepatch.py`, `peutil.py` | one patch at 0x410 pinned with `--expect`; the written file re-reads with no verification problems and a recomputed checksum of 0x10c24; exactly 3 bytes differ from the input, the patched byte and 2 checksum bytes | observed |
|
||||
| Ambiguity refusal | generated fixture | `pepatch.py` | exit 1, `matched 2 sites`, no output file created, input hash unchanged | observed |
|
||||
| Expect-mismatch refusal | generated fixture | `pepatch.py` | exit 1 with `the site moved or the wrong binary is open` | observed |
|
||||
| Length discipline | generated fixture | `pepatch.py` | no `--expect` and no `--original-length` reports `[length unproven: pass expect or original-length]`; a 1-into-3 byte patch is refused without `--allow-resize`; with it, the run is marked structural and the file length is unchanged because the write is in place | observed |
|
||||
| Dry run | generated fixture | `pepatch.py` | exit 0, no file created, input hash unchanged | observed |
|
||||
| Control run, no-op patch | generated fixture | `pepatch.py` | output SHA-256 equals input SHA-256: the file is byte-identical, checksum included | observed |
|
||||
| Cross-build comparison | two independently built fixtures | `aob_scan.py` | same recipe 1 versus 1, exit 0; `bare` recipe 1 versus 0, exit 1; patched build 2 versus 1, exit 1; a pattern with 2 hits on both sides still exits 1 because the count agreeing is not the same as the pattern being usable | observed |
|
||||
| Identity scan, rules | synthetic corpus in a temporary root | `scan_leaks.py` | all 18 rules fire on their shaped samples; the corpus reports 13 strong findings and 4 weak, exit 1 | observed |
|
||||
| Identity scan, exemptions | same corpus | `scan_leaks.py` | tool, protector, format, platform, placeholder, documentation-address and well-known-SID samples produce no finding; 19 hits suppressed | observed |
|
||||
| Identity scan, clean surface | temporary root with no identity in it | `scan_leaks.py` | `RESULT=clean`, exit 0 | observed |
|
||||
| Identity scan, this suite's own surface | `tests/` | `scan_leaks.py` | `RESULT=clean` with 0 strong, 0 weak and 19 suppressed hits | observed |
|
||||
| Identity scan, the skill's own default surface | `skills/` (the root the script resolves with no arguments) | `scan_leaks.py` | 1 strong finding — a standard platform name read as a product identity bound to a version — with 18 suppressed, exit 1 | observed |
|
||||
| Host capability probe | this machine | `hwprobe.py` | 8 routes, 3 of them `available`, exit 0; 2 rows answer with a sentence instead of one of the three documented states | observed |
|
||||
| Suite runtime | this machine | `pytest` | 269 tests in 23.2 s: unit 1.5 s, cli 19.7 s, integration 4.6 s | observed |
|
||||
| Reader throughput | 360448-byte platform image | `peutil.py` | `--summary` took 159 / 162 / 157 ms over three runs | observed |
|
||||
| Feature-probe cost | this machine | `hwprobe.py` | `--json` took 1834 / 1821 ms over two runs | observed |
|
||||
| Fixture reproducibility | generated fixture | `make_fixture_pe.py` | the same recipe produces the same SHA-256 and the same bytes on repeated runs; the `no-virtual-tail` recipe exits 1 and writes nothing | observed |
|
||||
| Truncated-image handling | fixture cut off mid-header | `peutil.py`, `pepatch.py` | both print a traceback and exit non-zero instead of using their documented exit codes; a fully non-PE input does use them | observed |
|
||||
| Missing spec file | `pepatch.py` with an absent `--spec` | `pepatch.py` | traceback rather than the documented usage error | observed |
|
||||
| Bare JSON pattern array | `aob_scan.py` with a top-level array | `aob_scan.py` | `AttributeError`; only the object form is accepted | observed |
|
||||
| Documented empty-surface exit code | `scan_leaks.py` over an empty directory | `scan_leaks.py` | exit 0 with `RESULT=clean`; the documented exit 3 has no code path | observed |
|
||||
| Patched image launch | a copy of a platform binary patched at the entry point | `pepatch.py` | not run by this suite. Recorded from the project's measurement pass: one byte changed 48 to CC, exactly two bytes differed from the original (the patched byte and the checksum field), and the process terminated with exit code 0x80000003 about 4.0 s after launch while the unpatched copy showed no surviving process | unverified here |
|
||||
| Device, debugger, network and driver routes | none | none | not run. This suite has no device, no debugger, no network and no kernel component; those routes are conditional host facts and are reported by `hwprobe.py` instead | unknown |
|
||||
|
||||
## Measurements obtained in this run
|
||||
|
||||
Reader throughput on a 360448-byte platform image, `peutil.py --summary`, three consecutive runs:
|
||||
159 ms, 162 ms, 157 ms. Host capability probe, `hwprobe.py --json`, two runs: 1834 ms and 1821 ms.
|
||||
Both sit within a few percent of the values recorded during the project's earlier measurement pass
|
||||
(158/166/167 ms and 1858 ms), which is the expected spread for process startup on a busy desktop.
|
||||
|
||||
Suite runtime, `python -m pytest -q` from the repository root: 269 passed in 23.2 s. The cli layer
|
||||
dominates because it starts a fresh interpreter per assertion, twice over for the feature probe.
|
||||
|
||||
Exact byte-level effect of a two-byte patch at file offset 0x410: three bytes differ from the
|
||||
input, at 0x410, 0xD8 and 0xD9. The checksum field is four bytes wide, so a patch that changes two
|
||||
bytes of image data can leave part of the old checksum value already correct.
|
||||
|
||||
## Rows not run
|
||||
|
||||
No row in the matrix covers a debugger attach, a hardware breakpoint, a trace capture, a driver
|
||||
load, a network endpoint, a device, or a live target process. That is the suite's stated boundary,
|
||||
not an oversight: none of those can be asserted deterministically on a bare checkout, and a row
|
||||
that claims them would be describing a different machine on a different day.
|
||||
|
||||
Two further dimensions were considered and left out. Mass-scanning a large directory of real
|
||||
images was left out because the time it takes depends on the machine's disk rather than on the
|
||||
scripts. Comparing two builds of the same third-party product was left out because no such pair is
|
||||
publicly usable here; the two-build comparison in the matrix uses two images this repository
|
||||
builds itself.
|
||||
|
||||
## Notes for a reader of the results
|
||||
|
||||
The one strong finding on the skill's own default surface is also the only failing repository gate
|
||||
at the time of writing: the tracked-surface scan reads the standard platform name `TPM 2.0` in a
|
||||
shipped reference as a product bound to a version. Nothing in `tests/` contributes to it — this
|
||||
suite's own surface reports 0 strong and 0 weak.
|
||||
|
||||
The scripts under `skills/exe-reverse/scripts/` were being edited while this matrix was being
|
||||
produced, and two of its rows moved with them: the public-address rule gained an exclusion that
|
||||
stops it reading a four-part version number as an address, and a standard-name exemption that had
|
||||
suppressed every URL finding was corrected. Both the suite and this file record the state after
|
||||
those corrections. Re-run the matrix after any further change to those scripts rather than
|
||||
trusting these numbers.
|
||||
@@ -0,0 +1,264 @@
|
||||
"""Subprocess contract tests for skills/exe-reverse/scripts/dart_snapshot_strings.py.
|
||||
|
||||
The input is the generated ELF64 from `conftest.build_synthetic_elf`, never a sample: a valid
|
||||
container with one one-byte literal, one two-byte literal and a `.text` that carries neither of the
|
||||
idioms the pool observer counts.
|
||||
|
||||
Two conventions are pinned because they are not what the docstring's exit-code table suggests:
|
||||
|
||||
* `--json` returns 0 whatever the filter did, while the text mode returns 1 for "nothing matched";
|
||||
* `--at` widens a match by `offset + len(text)`, and for a two-byte literal that is half the bytes
|
||||
the literal occupies, so an offset inside the literal's second half is reported as a miss.
|
||||
"""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.cli
|
||||
|
||||
|
||||
def plain_lines(text):
|
||||
return [line for line in text.splitlines() if line.strip()]
|
||||
|
||||
|
||||
def reported_count(text):
|
||||
"""The `N literal(s) reported.` line the text view ends with."""
|
||||
for line in reversed(plain_lines(text)):
|
||||
if "literal(s) reported." in line:
|
||||
return int(line.split()[0])
|
||||
raise AssertionError("no 'literal(s) reported.' line in:\n%s" % text)
|
||||
|
||||
|
||||
# --- refusals ----------------------------------------------------------------
|
||||
|
||||
def test_a_file_that_is_not_an_elf_is_refused_with_the_documented_code(run_script, tmp_path):
|
||||
path = tmp_path / "notes.txt"
|
||||
path.write_text("this is not a snapshot\n", encoding="utf-8")
|
||||
result = run_script("dart_snapshot_strings.py", path)
|
||||
assert result.returncode == 3
|
||||
assert "not an ELF image" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
assert result.stdout == ""
|
||||
|
||||
|
||||
def test_a_truncated_image_is_refused_without_a_traceback(run_script, synthetic_elf, tmp_path):
|
||||
path = tmp_path / "truncated.so"
|
||||
path.write_bytes(synthetic_elf.data[:0x30])
|
||||
result = run_script("dart_snapshot_strings.py", path)
|
||||
assert result.returncode == 3
|
||||
assert "not an ELF image" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_a_missing_file_exits_two_with_a_reason(run_script, tmp_path):
|
||||
result = run_script("dart_snapshot_strings.py", tmp_path / "absent.so")
|
||||
assert result.returncode == 2
|
||||
assert "error:" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_an_elf_with_no_code_section_is_refused(run_script, synthetic_elf_without_text):
|
||||
"""A shared object without `.text` is not an AOT code image, and the reader says so."""
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf_without_text.path)
|
||||
assert result.returncode == 3
|
||||
assert "no .text section" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
# --- recovery ----------------------------------------------------------------
|
||||
|
||||
def test_the_default_run_reports_every_literal_it_recovered(run_script, synthetic_elf):
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path)
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "x86-64, shared-object" in result.stdout
|
||||
for entry in synthetic_elf.literals:
|
||||
assert hex(entry["vaddr"]) in result.stdout
|
||||
assert entry["text"] in result.stdout
|
||||
assert "literals recovered : 3" in result.stdout
|
||||
assert reported_count(result.stdout) == 3
|
||||
|
||||
|
||||
def test_min_length_filters_the_short_literals_out(run_script, synthetic_elf):
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--min-length", "12")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
assert "premium_license" in result.stdout
|
||||
assert "trial_mode" not in result.stdout
|
||||
assert ">= 12 chars" in result.stdout
|
||||
|
||||
|
||||
def test_limit_caps_how_many_are_reported(run_script, synthetic_elf):
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--limit", "1")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
|
||||
|
||||
def test_section_selection_is_honoured(run_script, synthetic_elf):
|
||||
only_rodata = run_script("dart_snapshot_strings.py", synthetic_elf.path,
|
||||
"--sections", ".rodata")
|
||||
assert only_rodata.returncode == 0, only_rodata.stderr
|
||||
assert reported_count(only_rodata.stdout) == 3
|
||||
|
||||
only_text = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--sections", ".text")
|
||||
assert only_text.returncode == 1
|
||||
assert reported_count(only_text.stdout) == 0
|
||||
|
||||
absent = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--sections", ".absent")
|
||||
assert absent.returncode == 1
|
||||
assert reported_count(absent.stdout) == 0
|
||||
|
||||
|
||||
# --- keyword filter ----------------------------------------------------------
|
||||
|
||||
def test_find_matches_case_insensitively(run_script, synthetic_elf):
|
||||
lower = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "premium")
|
||||
upper = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "PREMIUM")
|
||||
assert lower.returncode == 0, lower.stderr
|
||||
assert upper.returncode == 0, upper.stderr
|
||||
assert reported_count(lower.stdout) == reported_count(upper.stdout) == 1
|
||||
assert "premium_license" in lower.stdout
|
||||
assert "trial_mode" not in lower.stdout
|
||||
|
||||
|
||||
def test_find_accepts_a_comma_separated_list(run_script, synthetic_elf):
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "trial, VIP")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 2
|
||||
assert "trial_mode" in result.stdout
|
||||
assert "vip_pass" in result.stdout
|
||||
assert "premium_license" not in result.stdout
|
||||
|
||||
|
||||
def test_find_can_reach_a_two_byte_literal(run_script, synthetic_elf):
|
||||
"""A keyword search that only read ASCII would miss this one, which is why both are recovered."""
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "vip")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
assert "[two-byte]" in result.stdout
|
||||
|
||||
|
||||
def test_a_keyword_that_matches_nothing_changes_the_exit_code(run_script, synthetic_elf):
|
||||
"""A miss is a negative finding, not an empty success."""
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_keyword")
|
||||
assert result.returncode == 1
|
||||
assert reported_count(result.stdout) == 0
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
# --- --at --------------------------------------------------------------------
|
||||
|
||||
def test_at_resolves_a_literal_by_file_offset(run_script, synthetic_elf):
|
||||
entry = synthetic_elf.literal("premium_license")
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(entry["offset"]))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
assert "premium_license" in result.stdout
|
||||
|
||||
|
||||
def test_at_resolves_a_two_byte_literal_by_file_offset(run_script, synthetic_elf):
|
||||
entry = synthetic_elf.literal("vip_pass")
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(entry["offset"]))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
assert "vip_pass" in result.stdout
|
||||
|
||||
|
||||
def test_at_resolves_a_literal_by_virtual_address(run_script, synthetic_elf):
|
||||
"""The vaddr is far above the file size, which is how the reader tells the two apart."""
|
||||
entry = synthetic_elf.literal("vip_pass")
|
||||
assert entry["vaddr"] > synthetic_elf.size
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", hex(entry["vaddr"]))
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert reported_count(result.stdout) == 1
|
||||
assert "vip_pass" in result.stdout
|
||||
assert hex(entry["vaddr"]) in result.stdout
|
||||
|
||||
|
||||
def test_at_inside_the_code_section_matches_nothing(run_script, synthetic_elf):
|
||||
result = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at",
|
||||
str(synthetic_elf.text["offset"] + 1))
|
||||
assert result.returncode == 1
|
||||
assert reported_count(result.stdout) == 0
|
||||
|
||||
|
||||
def test_at_finds_every_byte_of_a_two_byte_literal(load_script, run_script, synthetic_elf):
|
||||
"""`--at` measures its window in bytes, so **every** byte offset inside the literal matches.
|
||||
|
||||
The window used to be `offset + len(text)`. For a two-byte literal the offsets are bytes and the
|
||||
text is characters, so the window covered only the first half of it and a real byte offset in
|
||||
the second half came back as a miss. Note which byte is which: in UTF-16LE an ASCII character is
|
||||
a printable byte followed by a zero byte, so the last meaningful offset in the literal is the
|
||||
high byte of its final character, which is `0x00` and is a perfectly valid offset to probe.
|
||||
"""
|
||||
entry = synthetic_elf.literal("vip_pass")
|
||||
assert entry["encoding"] == "two-byte"
|
||||
span = 2 * len(entry["text"])
|
||||
lo = entry["offset"]
|
||||
last_low_byte = lo + span - 2
|
||||
|
||||
assert synthetic_elf.data[last_low_byte] in range(0x20, 0x7F)
|
||||
assert synthetic_elf.data[lo + span - 1] == 0, "the final byte is the high byte of the character"
|
||||
|
||||
for probe in (lo, lo + 2, last_low_byte, lo + span - 1):
|
||||
found = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--at", str(probe))
|
||||
assert found.returncode == 0, (probe, found.stdout, found.stderr)
|
||||
assert reported_count(found.stdout) == 1, (probe, found.stdout)
|
||||
|
||||
# The module computes the same window the CLI does.
|
||||
recovered = load_script("dart_snapshot_strings.py").recover_strings(
|
||||
synthetic_elf.data, {"offset": synthetic_elf.rodata["offset"],
|
||||
"vaddr": synthetic_elf.rodata["vaddr"],
|
||||
"size": synthetic_elf.rodata["size"]})
|
||||
matched = [row[2] for row in recovered
|
||||
if row[0] <= lo + span - 1 < row[0] + 2 * len(row[2])]
|
||||
assert matched == ["vip_pass"]
|
||||
|
||||
|
||||
def test_at_can_be_combined_with_find(run_script, synthetic_elf):
|
||||
entry = synthetic_elf.literal("trial_mode")
|
||||
hit = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "trial",
|
||||
"--at", str(entry["offset"]))
|
||||
assert hit.returncode == 0, hit.stderr
|
||||
assert reported_count(hit.stdout) == 1
|
||||
|
||||
miss = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "premium",
|
||||
"--at", str(entry["offset"]))
|
||||
assert miss.returncode == 1
|
||||
assert reported_count(miss.stdout) == 0
|
||||
|
||||
|
||||
# --- json --------------------------------------------------------------------
|
||||
|
||||
def test_json_returns_one_when_the_filter_matched_nothing(run_script, synthetic_elf):
|
||||
"""The exit code means the same thing in both output modes.
|
||||
|
||||
It used to be 1 from the text path and 0 from the JSON path for the identical filter, so a
|
||||
caller that asked for JSON could not tell a match from a miss by reading the exit code.
|
||||
"""
|
||||
text_mode = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_word")
|
||||
json_mode = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--find", "no_such_word",
|
||||
"--json")
|
||||
assert text_mode.returncode == 1
|
||||
assert json_mode.returncode == 1
|
||||
payload = json.loads(json_mode.stdout)
|
||||
assert payload["matches"] == []
|
||||
assert payload["literals_recovered"] == 0
|
||||
|
||||
# An unfiltered run is a listing, not a search, and a listing that finds literals still exits 0.
|
||||
unfiltered = run_script("dart_snapshot_strings.py", synthetic_elf.path, "--json")
|
||||
assert unfiltered.returncode == 0
|
||||
assert json.loads(unfiltered.stdout)["matches"]
|
||||
|
||||
|
||||
def test_an_unknown_flag_is_a_usage_error(run_script):
|
||||
result = run_script("dart_snapshot_strings.py", "--definitely-not-a-flag")
|
||||
assert result.returncode == 2
|
||||
assert "usage:" in result.stderr.lower()
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_help_exits_zero(run_script):
|
||||
result = run_script("dart_snapshot_strings.py", "--help")
|
||||
assert result.returncode == 0
|
||||
assert "usage:" in result.stdout.lower()
|
||||
assert "app.so" in result.stdout
|
||||
@@ -0,0 +1,397 @@
|
||||
"""Subprocess contract tests for the machine-readable output of every script.
|
||||
|
||||
One convention is not what a reader would assume, so it is asserted explicitly rather than
|
||||
smoothed over: ``scan_leaks.py`` writes a JSON document and then a ``RESULT=`` line, so its
|
||||
stdout as a whole is not valid JSON. The document parses from the front; the verdict line is
|
||||
read from the end. A test pins that shape so a future change to either half is deliberate.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.cli
|
||||
|
||||
# The reader's own view names, produced by the flags that populate them.
|
||||
PEUTIL_VIEWS = {
|
||||
"--summary": "summary",
|
||||
"--sections": "section_detail",
|
||||
"--imports": "imports",
|
||||
"--exports": "exports",
|
||||
"--resources": "resources",
|
||||
"--rich": "rich_header",
|
||||
"--tls": "tls",
|
||||
"--debug": "debug",
|
||||
"--relocs": "base_relocations",
|
||||
"--entropy": "entropy",
|
||||
"--check-sum": "checksum",
|
||||
}
|
||||
|
||||
# Always present, whatever the flags.
|
||||
PEUTIL_ALWAYS = ("security_directory", "load_config")
|
||||
|
||||
|
||||
def json_document(text):
|
||||
return json.JSONDecoder().raw_decode(text)[0]
|
||||
|
||||
|
||||
def last_line(text):
|
||||
lines = [line for line in text.splitlines() if line.strip()]
|
||||
return lines[-1] if lines else ""
|
||||
|
||||
|
||||
# --- peutil ------------------------------------------------------------------
|
||||
|
||||
def test_reader_json_carries_the_summary_the_text_view_shows(run_script, fixtures):
|
||||
result = run_script("peutil.py", fixtures.pe64.path, "--json")
|
||||
assert result.returncode == 0
|
||||
payload = json.loads(result.stdout)
|
||||
assert sorted(payload) == ["load_config", "security_directory", "summary"]
|
||||
summary = payload["summary"]
|
||||
assert summary["checksum_matches"] is True
|
||||
assert summary["checksum_declared"] == fixtures.pe64.meta["checksum"]
|
||||
assert summary["kind"] == "exe"
|
||||
assert summary["arch"] == "amd64"
|
||||
assert summary["file_size"] == fixtures.pe64.size
|
||||
assert summary["sections"] and len(summary["sections"]) == 5
|
||||
assert payload["security_directory"] is None
|
||||
assert payload["load_config"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("flag,key", sorted(PEUTIL_VIEWS.items()))
|
||||
def test_each_reader_view_lands_under_its_own_key(run_script, fixtures, flag, key):
|
||||
result = run_script("peutil.py", fixtures.pe64.path, flag, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert key in payload
|
||||
|
||||
|
||||
def test_reader_json_with_every_view_present(run_script, fixtures):
|
||||
flags = [flag for flag in sorted(PEUTIL_VIEWS)]
|
||||
result = run_script("peutil.py", fixtures.pe64.path, *flags, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
for key in list(PEUTIL_VIEWS.values()) + list(PEUTIL_ALWAYS):
|
||||
assert key in payload, key
|
||||
assert payload["resources"][0]["path"] == ["VERSION", "1", "1033"]
|
||||
assert payload["tls"]["callback_rvas"] == ["0x1000"]
|
||||
assert payload["debug"][0]["format"] == "RSDS"
|
||||
assert payload["checksum"]["matches"] is True
|
||||
assert payload["imports"][0]["dll"] == "fixture-import.dll"
|
||||
assert payload["exports"]["dll_name"] == "fixture-export.dll"
|
||||
assert payload["rich_header"] is None # the fixture has none, and says so
|
||||
assert payload["delay_imports"] == []
|
||||
assert payload["base_relocations"]["blocks"] == 1
|
||||
assert payload["section_detail"][0]["raw_tail"] == 0
|
||||
assert len(payload["entropy"]) == 5
|
||||
|
||||
|
||||
def test_reader_json_of_a_bare_recipe_reports_the_absent_structures(run_script, fixtures):
|
||||
result = run_script("peutil.py", fixtures.bare.path, "--exports", "--resources", "--tls",
|
||||
"--debug", "--json")
|
||||
payload = json.loads(result.stdout)
|
||||
assert result.returncode == 0
|
||||
assert payload["exports"] is None
|
||||
assert payload["resources"] == []
|
||||
assert payload["tls"] is None
|
||||
assert payload["debug"] == []
|
||||
|
||||
|
||||
def test_reader_json_is_decodable_ascii(run_script, fixtures):
|
||||
result = run_script("peutil.py", fixtures.pe64.path, "--json")
|
||||
assert result.stdout.isascii()
|
||||
|
||||
|
||||
# --- pepatch -----------------------------------------------------------------
|
||||
|
||||
def test_patcher_dry_run_json_reports_the_plan_and_says_it_wrote_nothing(run_script, fixtures,
|
||||
tmp_path):
|
||||
out = tmp_path / "never.exe"
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
|
||||
"--bytes", "CC CC", "--original-length", "2", "--dry-run", "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["dry_run"] is True
|
||||
assert payload["patches_requested"] == 1
|
||||
assert payload["patches_applied"] == 1
|
||||
assert payload["errors"] == []
|
||||
assert payload["checksum_recomputed"] is False
|
||||
applied = payload["applied"][0]
|
||||
assert applied["offset"] == 0x410
|
||||
assert applied["rva"] == "0x1010"
|
||||
assert applied["before"] == "48 8B"
|
||||
assert applied["after"] == "CC CC"
|
||||
assert applied["equal_length"] is True
|
||||
assert applied["length_basis"] == "original-length"
|
||||
assert "output" not in payload
|
||||
assert not out.exists()
|
||||
|
||||
|
||||
def test_patcher_real_run_json_reports_verification_and_the_new_checksum(run_script, fixtures,
|
||||
tmp_path):
|
||||
out = tmp_path / "patched.exe"
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
|
||||
"--expect", "48 8B", "--bytes", "CC 8B", "--out", out, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["output"] == str(out)
|
||||
assert payload["output_size"] == payload["input_size"] == fixtures.pe64.size
|
||||
assert payload["verify_problems"] == []
|
||||
assert payload["checksum_after"]["matches"] is True
|
||||
assert payload["checksum_recomputed"] is True
|
||||
assert payload["signed_input"] is False
|
||||
assert payload["signature_note"] is None
|
||||
assert payload["structural"] == []
|
||||
assert payload["length_unproven"] == []
|
||||
assert out.read_bytes()[0x410:0x412] == b"\xCC\x8B"
|
||||
|
||||
|
||||
def test_patcher_json_names_a_length_unproven_patch(load_script, run_script, fixtures, tmp_path):
|
||||
out = tmp_path / "unproven.exe"
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410",
|
||||
"--bytes", "CC", "--out", out, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["applied"][0]["equal_length"] is None
|
||||
assert payload["length_unproven"] == ["cli"]
|
||||
assert payload["structural"] == []
|
||||
|
||||
|
||||
def test_patcher_json_reports_a_refusal_without_an_output(run_script, fixtures, tmp_path):
|
||||
out = tmp_path / "never.exe"
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--pattern", fixtures.pe64.plant,
|
||||
"--bytes", "90 " * 12, "--out", out, "--json")
|
||||
assert result.returncode == 1
|
||||
assert not out.exists()
|
||||
assert "matched 2 sites" in result.stderr
|
||||
assert result.stdout.strip() == ""
|
||||
|
||||
|
||||
# --- aob_scan ----------------------------------------------------------------
|
||||
|
||||
def test_locator_json_reports_a_unique_pattern_as_ok(run_script, fixtures, hexpat):
|
||||
pdb = hexpat(b"fixture.pdb")
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", pdb, "--json")
|
||||
assert result.returncode == 0, result.stdout
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["result"] == "ok"
|
||||
assert payload["target_sections"] == [".text", ".rdata", ".data", ".rsrc", ".reloc"]
|
||||
assert payload["section_filter"] is None
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["hits"] == 1
|
||||
assert entry["verdict"] == "unique"
|
||||
assert entry["wildcards"] == 0
|
||||
assert entry["sites"][0]["section"] == ".rdata"
|
||||
|
||||
|
||||
def test_locator_json_reports_the_planted_pair_as_ambiguous(run_script, fixtures):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
|
||||
"--json")
|
||||
assert result.returncode == 1, "an ambiguous pattern is the documented non-zero outcome"
|
||||
payload = json.loads(result.stdout)
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["hits"] == 2
|
||||
assert entry["verdict"] == "ambiguous"
|
||||
assert payload["result"] == "needs-attention"
|
||||
offsets = [int(site["offset_hex"], 16) for site in entry["sites"]]
|
||||
assert offsets == list(fixtures.pe64.plant_offsets)
|
||||
assert [site["rva_hex"] for site in entry["sites"]] == \
|
||||
[fixtures.pe64.meta["plant"]["first_rva"], fixtures.pe64.meta["plant"]["second_rva"]]
|
||||
|
||||
|
||||
def test_locator_json_reports_an_absent_pattern_as_a_no_match(run_script, fixtures):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "DE AD BE EF CA FE",
|
||||
"--json")
|
||||
assert result.returncode == 1
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["patterns"][0]["hits"] == 0
|
||||
assert payload["patterns"][0]["verdict"] == "no-match"
|
||||
assert payload["patterns"][0]["sites"] == []
|
||||
|
||||
|
||||
def test_locator_json_counts_wildcards_and_section_filtered_hits(run_script, fixtures):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns",
|
||||
"48 8B 05 ?? ?? ?? 00", "--section", ".text", "--json")
|
||||
payload = json.loads(result.stdout)
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["wildcards"] == 3
|
||||
assert entry["hits"] == 2
|
||||
assert payload["section_filter"] == ".text"
|
||||
assert {site["section"] for site in entry["sites"]} == {".text"}
|
||||
|
||||
|
||||
def test_locator_json_from_a_pattern_file_keeps_the_names(run_script, fixtures, tmp_path, hexpat):
|
||||
sigs = tmp_path / "sigs.txt"
|
||||
sigs.write_text("# comment\n%s\nnamed-site\t%s\n" % (fixtures.pe64.plant, hexpat(b"FIXT")),
|
||||
encoding="utf-8")
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", sigs, "--json")
|
||||
payload = json.loads(result.stdout)
|
||||
names = [entry["name"] for entry in payload["patterns"]]
|
||||
assert names == ["sigs.txt:2", "named-site"]
|
||||
assert payload["patterns"][1]["hits"] >= 1
|
||||
|
||||
|
||||
# --- scan_leaks --------------------------------------------------------------
|
||||
|
||||
def test_scanner_json_is_valid_json_ahead_of_the_result_line(run_script, synthetic_leak_root):
|
||||
"""The document parses from the front; the RESULT line is not part of it."""
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
|
||||
assert result.returncode == 1
|
||||
payload = json_document(result.stdout)
|
||||
assert sorted(payload) == ["exempted", "files_scanned", "root", "strong", "verdict", "weak"]
|
||||
assert payload["verdict"] == "findings"
|
||||
assert payload["files_scanned"] >= 3
|
||||
assert last_line(result.stdout) == "RESULT=findings"
|
||||
|
||||
|
||||
def test_scanner_json_stdout_is_not_valid_as_a_whole(run_script, synthetic_leak_root):
|
||||
"""A negative outcome: nothing may treat this stdout as one JSON document."""
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
|
||||
with pytest.raises(json.JSONDecodeError):
|
||||
json.loads(result.stdout)
|
||||
|
||||
|
||||
def test_scanner_json_reports_exactly_the_expected_rules(run_script, synthetic_leak_root):
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json",
|
||||
"--show-exempt")
|
||||
payload = json_document(result.stdout)
|
||||
assert {hit["rule"] for hit in payload["strong"]} == {
|
||||
"private-key-block", "github-token", "aws-access-key", "slack-token",
|
||||
"generic-secret-assignment", "licence-blob-literal", "windows-user-profile",
|
||||
"posix-home", "windows-sid-literal", "volume-serial-literal",
|
||||
"literal-endpoint-with-creds", "public-ipv4-literal", "product-name-with-version",
|
||||
}
|
||||
assert {hit["rule"] for hit in payload["weak"]} == {
|
||||
"exe-name-with-version", "literal-host-with-path", "email-address",
|
||||
"vendor-copyright-line",
|
||||
}
|
||||
for hit in payload["strong"]:
|
||||
assert hit["strength"] == "strong"
|
||||
assert hit["file"].endswith("notes.md")
|
||||
assert hit["line"] > 0
|
||||
assert hit["context"]
|
||||
assert "machine-guid" not in {hit["rule"] for hit in payload["strong"]}
|
||||
|
||||
|
||||
def test_scanner_json_show_exempt_records_the_suppressed_hits(run_script, synthetic_leak_root):
|
||||
"""Measured: the machine-guid hit is suppressed by the bare-GUID exemption."""
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json",
|
||||
"--show-exempt")
|
||||
payload = json_document(result.stdout)
|
||||
suppressed = {hit["rule"]: hit.get("exempt") for hit in payload["exempted"]}
|
||||
assert isinstance(payload["exempted"], list)
|
||||
assert suppressed.get("machine-guid") == \
|
||||
"a bare GUID is not machine identity without MachineGuid context"
|
||||
reasons = {hit["exempt"] for hit in payload["exempted"]
|
||||
if hit["rule"] == "product-name-with-version"}
|
||||
assert reasons and reasons <= {"tool or library name",
|
||||
"platform or vendor-of-the-platform term",
|
||||
"public protector/packer/managed tooling name"}
|
||||
|
||||
|
||||
def test_scanner_json_without_show_exempt_reports_only_a_count(run_script, synthetic_leak_root):
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--format", "json")
|
||||
payload = json_document(result.stdout)
|
||||
assert isinstance(payload["exempted"], int)
|
||||
assert payload["exempted"] > 0
|
||||
|
||||
|
||||
def test_scanner_json_on_a_clean_root_says_clean(run_script, clean_scan_root):
|
||||
result = run_script("scan_leaks.py", "--root", clean_scan_root, "--format", "json")
|
||||
assert result.returncode == 0
|
||||
payload = json_document(result.stdout)
|
||||
assert payload["verdict"] == "clean"
|
||||
assert payload["strong"] == []
|
||||
assert payload["weak"] == []
|
||||
|
||||
|
||||
# --- make_fixture_pe ---------------------------------------------------------
|
||||
|
||||
def test_fixture_builder_json_describes_what_it_wrote(run_script, tmp_path):
|
||||
out = tmp_path / "described.exe"
|
||||
result = run_script("make_fixture_pe.py", "--out", out, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["path"] == str(out)
|
||||
assert payload["size"] == out.stat().st_size
|
||||
assert payload["sha256"] == hashlib.sha256(out.read_bytes()).hexdigest()
|
||||
assert sorted(payload["sections"]) == [".data", ".rdata", ".reloc", ".rsrc", ".text"]
|
||||
plant = payload["plant"]
|
||||
assert plant["first_file_offset"] != plant["second_file_offset"]
|
||||
assert "planted twice" in plant["note"]
|
||||
assert payload["checksum"].startswith("0x")
|
||||
assert payload["virtual_only_tail_bytes"] > 0
|
||||
assert payload["executable_and_writable_code"] is True
|
||||
|
||||
|
||||
def test_fixture_builder_is_byte_reproducible(run_script, tmp_path):
|
||||
first = tmp_path / "one.exe"
|
||||
second = tmp_path / "two.exe"
|
||||
a = run_script("make_fixture_pe.py", "--out", first, "--json")
|
||||
b = run_script("make_fixture_pe.py", "--out", second, "--json")
|
||||
assert a.returncode == b.returncode == 0
|
||||
assert json.loads(a.stdout)["sha256"] == json.loads(b.stdout)["sha256"]
|
||||
assert first.read_bytes() == second.read_bytes()
|
||||
|
||||
|
||||
# --- hwprobe -----------------------------------------------------------------
|
||||
|
||||
# The route table is documented as three-state: available / needs-<x> / unavailable. Measured on
|
||||
# this host, two rows carry a prose state instead of a vocabulary word. The values below are the
|
||||
# ones actually produced; allowing them keeps the check honest without asserting the docstring.
|
||||
ROUTE_STATES = ("available", "unavailable")
|
||||
ROUTE_PROSE_STATES = (
|
||||
"CPU supports it, but a hypervisor is present",
|
||||
"needs-a-debugger, and a hypervisor is present",
|
||||
"no documented user-mode path on this platform",
|
||||
)
|
||||
|
||||
|
||||
def test_prober_json_reports_a_route_table(hwprobe_report):
|
||||
payload = hwprobe_report
|
||||
for key in ("python", "platform", "cpu", "os", "privilege", "routes", "tools",
|
||||
"python_modules"):
|
||||
assert key in payload, key
|
||||
assert payload["python"].startswith("3.")
|
||||
rows = payload["routes"]
|
||||
assert rows
|
||||
for row in rows:
|
||||
assert len(row) == 3, row
|
||||
name, state, why = row
|
||||
assert name and state and why
|
||||
assert name != state
|
||||
# The route that needs nothing but Python and a file on disk must be available when the
|
||||
# script that reports it is running at all.
|
||||
static = [row for row in rows if row[0].startswith("static PE analysis")]
|
||||
assert len(static) == 1
|
||||
assert static[0][1] == "available"
|
||||
|
||||
|
||||
def test_route_states_use_the_vocabulary_or_a_measured_deviation(hwprobe_report):
|
||||
"""Measured: not every route state is one of the three documented states.
|
||||
|
||||
Two rows on this host answer with a sentence instead of a vocabulary word, so the
|
||||
three-state claim in the docstring holds for the table's shape but not for every cell. The
|
||||
measured sentence forms are listed rather than asserted as required, so a fix to the
|
||||
vocabulary passes and a new unmeasured sentence fails loudly.
|
||||
"""
|
||||
for row in hwprobe_report["routes"]:
|
||||
state = row[1]
|
||||
assert state in ROUTE_STATES or state.startswith("needs-") \
|
||||
or state in ROUTE_PROSE_STATES, state
|
||||
|
||||
|
||||
def test_prober_rows_carry_a_measurement_and_a_note(hwprobe_report):
|
||||
payload = hwprobe_report
|
||||
for group in ("cpu", "os", "privilege"):
|
||||
assert payload[group], group
|
||||
for row in payload[group]:
|
||||
assert set(row) >= {"state", "value", "note"}, (group, row)
|
||||
assert row["state"] in ROUTE_STATES or row["state"].startswith("needs-"), row
|
||||
|
||||
|
||||
def test_prober_can_dump_raw_cpuid_registers(run_script):
|
||||
result = run_script("hwprobe.py", "--raw", "--json", timeout=120)
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["raw_cpuid"], "a raw dump was requested but nothing was reported"
|
||||
@@ -0,0 +1,260 @@
|
||||
"""Subprocess contract tests for skills/exe-reverse/scripts/capabilities.py and doctor.py.
|
||||
|
||||
The exit codes are the ones the two docstrings publish -- 0 all ok, 3 something blocked, 2 usage
|
||||
error, 4 internal -- and the last stdout line is a `RESULT=<token>` for both.
|
||||
|
||||
One convention is pinned because it is not what a reader would assume: `capabilities.py --list`
|
||||
prints `RESULT=capabilities_ok` whatever the registry says, because listing ids and names probes
|
||||
nothing. The test asserts the contrast directly -- the same registry resolved through `--json` exits
|
||||
3 on this host while `--list` exits 0 -- so a future change to either half is deliberate.
|
||||
"""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.cli
|
||||
|
||||
BLOCKED_TOKEN = "capabilities_blocked"
|
||||
|
||||
|
||||
def json_document(text):
|
||||
return json.JSONDecoder().raw_decode(text)[0]
|
||||
|
||||
|
||||
def last_line(text):
|
||||
lines = [line for line in text.splitlines() if line.strip()]
|
||||
return lines[-1] if lines else ""
|
||||
|
||||
|
||||
def result_token(text):
|
||||
line = last_line(text)
|
||||
assert line.startswith("RESULT="), "the last line was %r" % line
|
||||
return line.split("=", 1)[1]
|
||||
|
||||
|
||||
# --- capabilities.py ---------------------------------------------------------
|
||||
|
||||
def test_list_prints_every_id_and_name(run_script, load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
result = run_script("capabilities.py", "--list")
|
||||
for capability_id in caps.CAPABILITIES:
|
||||
assert capability_id in result.stdout, capability_id
|
||||
assert "%d capabilities." % len(caps.CAPABILITIES) in result.stdout
|
||||
# The list now carries a real verdict rather than an unconditional one, so the exit code and
|
||||
# the token have to agree with what resolving the same registry reports.
|
||||
token, code = caps.verdict(caps.resolve_all())
|
||||
assert result.returncode == code
|
||||
assert result.stdout.strip().endswith("RESULT=%s" % caps.TOKENS[token])
|
||||
|
||||
|
||||
def test_list_and_json_agree_on_the_same_registry(run_script):
|
||||
"""Two views of one table must not disagree about it.
|
||||
|
||||
The list view used to print the ok token and exit 0 unconditionally while the JSON view of the
|
||||
same registry exited 3 on this host, because most of the table is blocked here by design --
|
||||
every `artifact:` row waits for an operator to supply one. A report that contradicts another
|
||||
report of the same data is worse than either alone: a reader cannot tell which to believe.
|
||||
"""
|
||||
listed = run_script("capabilities.py", "--list")
|
||||
resolved = run_script("capabilities.py", "--json")
|
||||
assert result_token(listed.stdout) == result_token(resolved.stdout)
|
||||
assert listed.returncode == resolved.returncode
|
||||
|
||||
|
||||
def test_json_reports_every_capability_with_its_closure(run_script, load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
result = run_script("capabilities.py", "--json")
|
||||
assert result.returncode in (0, 3), result.stderr
|
||||
rows = json_document(result.stdout)
|
||||
assert [row["id"] for row in rows] == list(caps.CAPABILITIES)
|
||||
for row in rows:
|
||||
assert set(row) >= {"id", "name", "state", "required", "any_of", "depends_on",
|
||||
"blocked_deps", "missing", "closure", "scripts", "note"}
|
||||
assert row["state"] in ("ok", "blocked")
|
||||
assert row["closure"][0] == row["id"]
|
||||
assert result_token(result.stdout) == (BLOCKED_TOKEN if result.returncode == 3
|
||||
else "capabilities_ok")
|
||||
|
||||
|
||||
def test_one_capability_reports_its_closure_and_its_state(run_script):
|
||||
"""`pe_structure_read` needs the interpreter and a script this kit ships, so it is ok anywhere."""
|
||||
result = run_script("capabilities.py", "pe_structure_read")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert "capability: Read a PE image's structure" in result.stdout
|
||||
assert "scripts : peutil.py" in result.stdout
|
||||
assert "requires : python310, file:peutil.py" in result.stdout
|
||||
assert "state : ok" in result.stdout
|
||||
assert result.stdout.strip().endswith("RESULT=capabilities_ok")
|
||||
|
||||
|
||||
def test_one_blocked_capability_names_the_atom_no_install_can_supply(run_script):
|
||||
result = run_script("capabilities.py", "commandline_debug")
|
||||
assert result.returncode == 3, result.stdout
|
||||
assert "state : blocked" in result.stdout
|
||||
assert "missing atom : artifact:sample" in result.stdout
|
||||
assert "next action" not in result.stdout, "an artifact has no install line"
|
||||
assert result.stdout.strip().endswith("RESULT=%s" % BLOCKED_TOKEN)
|
||||
|
||||
|
||||
def test_one_capability_as_json_carries_the_same_fields(run_script):
|
||||
result = run_script("capabilities.py", "pe_structure_read", "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
row = json_document(result.stdout)
|
||||
assert row["id"] == "pe_structure_read"
|
||||
assert row["state"] == "ok"
|
||||
assert row["missing"] == []
|
||||
assert row["blocked_deps"] == []
|
||||
assert result.stdout.strip().endswith("RESULT=capabilities_ok")
|
||||
|
||||
|
||||
def test_an_unknown_capability_is_a_usage_error_on_stdout_and_stderr(run_script):
|
||||
result = run_script("capabilities.py", "no_such_capability")
|
||||
assert result.returncode == 2
|
||||
assert result.stdout.strip() == "RESULT=usage_error"
|
||||
assert "no such capability" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_verbose_prints_the_next_action_for_a_blocked_capability(run_script, load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
installable = [caps.ATOMS[atom].get("install")
|
||||
for row in caps.resolve_all(caps.Probe(tool_dirs=caps.extra_tool_dirs()))
|
||||
if row["state"] != "ok"
|
||||
for atom, _detail in row["missing"]
|
||||
if caps.ATOMS.get(atom, {}).get("install")]
|
||||
if not installable:
|
||||
pytest.skip("this host has no blocked capability whose missing atom carries an install "
|
||||
"line, so there is nothing for --verbose to print here")
|
||||
result = run_script("capabilities.py", "--verbose")
|
||||
assert result.returncode in (0, 3), result.stderr
|
||||
assert "missing atom" in result.stdout
|
||||
assert "next action :" in result.stdout
|
||||
assert "est. minutes :" in result.stdout
|
||||
assert "(basis: " in result.stdout
|
||||
|
||||
|
||||
def test_an_unknown_flag_is_a_usage_error(run_script):
|
||||
result = run_script("capabilities.py", "--definitely-not-a-flag")
|
||||
assert result.returncode == 2
|
||||
assert result.stdout.strip() == ""
|
||||
assert "usage:" in result.stderr.lower()
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_help_exits_zero(run_script):
|
||||
result = run_script("capabilities.py", "--help")
|
||||
assert result.returncode == 0
|
||||
assert "usage:" in result.stdout.lower()
|
||||
|
||||
|
||||
def test_an_extra_tool_directory_can_turn_a_blocked_capability_ok(run_script, load_script,
|
||||
tmp_path):
|
||||
"""`--tools-dir` is the documented second search path; a tool off PATH is not a missing tool."""
|
||||
caps = load_script("capabilities.py")
|
||||
baseline = caps.resolve("static_disassembly",
|
||||
caps.Probe(tool_dirs=caps.extra_tool_dirs()))
|
||||
if baseline["state"] == "ok":
|
||||
pytest.skip("a disassembly backend is already reachable here (%s), so the extra directory "
|
||||
"cannot demonstrate anything" % baseline["alternatives_present"])
|
||||
|
||||
tools = tmp_path / "unpacked-tools"
|
||||
tools.mkdir()
|
||||
for name in ("ghidra", "ida64"):
|
||||
(tools / name).write_text("", encoding="utf-8")
|
||||
result = run_script("capabilities.py", "static_disassembly", "--tools-dir", tools)
|
||||
assert result.returncode == 0, result.stdout
|
||||
assert "present :" in result.stdout
|
||||
assert "EXEREV_TOOLS" in result.stdout
|
||||
|
||||
|
||||
# --- doctor.py ---------------------------------------------------------------
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def default_run(run_script):
|
||||
"""One full doctor run for the module: it probes the host and answers --help for every script."""
|
||||
return run_script("doctor.py", timeout=180)
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def json_run(run_script):
|
||||
return run_script("doctor.py", "--json", timeout=180)
|
||||
|
||||
|
||||
def test_the_doctor_exits_zero_or_three_and_never_with_a_traceback(default_run):
|
||||
assert default_run.returncode in (0, 3), default_run
|
||||
assert "Traceback" not in default_run.stdout + default_run.stderr
|
||||
|
||||
|
||||
def test_the_doctor_prints_its_three_blocks_and_a_result_line(default_run, load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
assert "== capabilities ==" in default_run.stdout
|
||||
assert "== scripts ==" in default_run.stdout
|
||||
assert "== environment facts that can change a result ==" in default_run.stdout
|
||||
assert "== result ==" in default_run.stdout
|
||||
assert "%d script(s) checked" % len(doctor.discover_scripts()) in default_run.stdout
|
||||
assert result_token(default_run.stdout) in ("env_ok", "env_partial", "env_blocked")
|
||||
|
||||
|
||||
def test_the_result_token_follows_the_exit_code(default_run):
|
||||
token = result_token(default_run.stdout)
|
||||
if default_run.returncode == 3:
|
||||
assert token == "env_blocked"
|
||||
else:
|
||||
assert token in ("env_ok", "env_partial")
|
||||
|
||||
|
||||
def test_the_script_table_counts_the_scripts_that_are_actually_there(default_run, load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
total = len(doctor.discover_scripts())
|
||||
assert "registered/checked = %d / %d" % (total, total) in default_run.stdout
|
||||
|
||||
|
||||
def test_json_carries_the_four_sections_and_a_summary(json_run, load_script):
|
||||
"""The counts are cross-checked against the rows they summarise, so a script that stops being
|
||||
classified shows up as a mismatch rather than as a smaller denominator."""
|
||||
doctor = load_script("doctor.py")
|
||||
assert json_run.returncode in (0, 3), json_run.stderr
|
||||
payload = json_document(json_run.stdout)
|
||||
assert sorted(payload) == ["capabilities", "environment", "result", "scripts", "summary"]
|
||||
total = len(doctor.discover_scripts())
|
||||
assert len(payload["scripts"]) == total
|
||||
assert payload["summary"]["scripts_checked"] == total
|
||||
unknown = [row for row in payload["scripts"] if row["capability_source"] == "unknown"]
|
||||
assert payload["summary"]["scripts_with_unknown_capability"] == len(unknown)
|
||||
assert payload["summary"]["capabilities_total"] == len(payload["capabilities"])
|
||||
assert payload["result"] == result_token(json_run.stdout)
|
||||
for row in payload["scripts"]:
|
||||
assert set(row) == {"script", "capabilities", "capability_source", "capability_state",
|
||||
"runnable", "third_party", "detail"}
|
||||
|
||||
|
||||
def test_json_environment_facts_carry_the_documented_keys(json_run):
|
||||
payload = json_document(json_run.stdout)
|
||||
if len(payload["environment"]) == 1 and payload["environment"][0]["fact"] == "host probe":
|
||||
pytest.skip("the host probe could not answer here, so the fact table is inert")
|
||||
for fact in payload["environment"]:
|
||||
assert set(fact) == {"fact", "value", "effect", "severity"}
|
||||
assert fact["severity"] in ("note", "warn")
|
||||
|
||||
|
||||
def test_each_view_flag_prints_only_its_own_block(run_script):
|
||||
scripts_only = run_script("doctor.py", "--scripts", timeout=180)
|
||||
assert scripts_only.returncode in (0, 3), scripts_only
|
||||
assert "== scripts ==" in scripts_only.stdout
|
||||
assert "== capabilities ==" not in scripts_only.stdout
|
||||
assert "== environment facts" not in scripts_only.stdout
|
||||
assert "RESULT=" in scripts_only.stdout
|
||||
|
||||
environment_only = run_script("doctor.py", "--environment", timeout=180)
|
||||
assert environment_only.returncode in (0, 3), environment_only
|
||||
assert "== environment facts" in environment_only.stdout
|
||||
assert "== scripts ==" not in environment_only.stdout
|
||||
assert "== capabilities ==" not in environment_only.stdout
|
||||
|
||||
|
||||
def test_an_unknown_flag_is_a_usage_error_without_a_traceback(run_script):
|
||||
result = run_script("doctor.py", "--definitely-not-a-flag")
|
||||
assert result.returncode == 2
|
||||
assert "usage:" in result.stderr.lower()
|
||||
assert "Traceback" not in result.stderr
|
||||
assert result.stdout.strip() == ""
|
||||
@@ -0,0 +1,362 @@
|
||||
"""Subprocess contract tests for the exit codes and usage surface of every script.
|
||||
|
||||
The exit codes asserted here are the ones each script's own docstring publishes, read from the
|
||||
docstring rather than invented:
|
||||
|
||||
* peutil.py 0 parsed, 1 not a parseable PE, 2 usage error
|
||||
* pepatch.py 0 applied and verified, 1 a patch failed, 2 usage error, 3 not a parseable PE
|
||||
* aob_scan.py 0 every pattern unique, 1 a pattern matched zero or several times, 2 usage,
|
||||
3 not a parseable PE
|
||||
* hwprobe.py 0 report produced, 2 usage error (a limited machine is a finding, not a failure)
|
||||
* make_fixture_pe.py 0 written, 2 usage error
|
||||
* scan_leaks.py 0 clean, 1 findings, 2 error
|
||||
|
||||
One input path is pinned as a measured deviation: a missing ``--spec`` file reaches an unguarded
|
||||
``open()``, so it prints a traceback instead of using the module's own exit-code vocabulary.
|
||||
"""
|
||||
import json
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.cli
|
||||
|
||||
SCRIPTS = ("peutil.py", "pepatch.py", "hwprobe.py", "aob_scan.py", "make_fixture_pe.py",
|
||||
"scan_leaks.py")
|
||||
|
||||
|
||||
@pytest.mark.parametrize("script", SCRIPTS)
|
||||
def test_help_exits_zero_with_usage_text(run_script, script):
|
||||
result = run_script(script, "--help")
|
||||
assert result.returncode == 0, result.stderr
|
||||
assert result.stdout.strip(), "no help text was printed"
|
||||
assert "usage:" in result.stdout.lower()
|
||||
assert script in result.stdout
|
||||
|
||||
|
||||
@pytest.mark.parametrize("script", SCRIPTS)
|
||||
def test_an_unknown_flag_is_a_usage_error(run_script, script):
|
||||
result = run_script(script, "--definitely-not-a-flag")
|
||||
assert result.returncode == 2
|
||||
assert result.stdout.strip() == ""
|
||||
assert "usage:" in result.stderr.lower()
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
@pytest.mark.parametrize("script", [s for s in SCRIPTS if s != "scan_leaks.py"])
|
||||
def test_a_script_with_no_arguments_never_hangs(run_script, script):
|
||||
"""Each script either does its documented thing or reports a usage error; none hangs.
|
||||
|
||||
The scanner is excluded here because its no-argument behaviour is to scan a tree rather than
|
||||
to fail; it has its own check below.
|
||||
"""
|
||||
result = run_script(script, timeout=120)
|
||||
assert result.returncode in (0, 2), result
|
||||
|
||||
|
||||
def test_scanner_with_no_arguments_scans_a_tree_and_states_its_verdict(run_script):
|
||||
"""With no arguments it scans the tree its own script lives in, and says what it found."""
|
||||
result = run_script("scan_leaks.py", timeout=180)
|
||||
assert result.returncode in (0, 1), result
|
||||
assert "scanning" in result.stdout
|
||||
assert result.stdout.strip().endswith(
|
||||
"RESULT=clean" if result.returncode == 0 else "RESULT=findings")
|
||||
|
||||
|
||||
# --- peutil ------------------------------------------------------------------
|
||||
|
||||
def test_reader_on_a_non_pe_exits_one_without_a_traceback(run_script, tmp_path):
|
||||
path = tmp_path / "notes.txt"
|
||||
path.write_text("this is not a portable executable\n", encoding="utf-8")
|
||||
result = run_script("peutil.py", path)
|
||||
assert result.returncode == 1
|
||||
assert "error:" in result.stderr
|
||||
assert "no MZ signature" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
assert result.stdout == ""
|
||||
|
||||
|
||||
def test_reader_on_a_missing_file_exits_two(run_script, tmp_path):
|
||||
result = run_script("peutil.py", tmp_path / "absent.exe")
|
||||
assert result.returncode == 2
|
||||
assert "is not a file" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_reader_on_a_truncated_image_reports_instead_of_raising(run_script, tmp_path, fixtures):
|
||||
"""A file cut off mid-header goes through the exit-code contract, not through struct.
|
||||
|
||||
A truncated image reaches an analyst as an interrupted download or a carved region far more
|
||||
often than as a malformed program, so what matters is the contract: one line on stderr and a
|
||||
documented code. Pinned after out-of-range reads were turned into a ParseError -- before that
|
||||
the caller got a traceback, which reads as a defect in the tool rather than a fact about the
|
||||
file.
|
||||
"""
|
||||
path = tmp_path / "truncated.exe"
|
||||
path.write_bytes(fixtures.pe64.data[:0x100])
|
||||
result = run_script("peutil.py", path)
|
||||
assert result.returncode == 1
|
||||
assert "error:" in result.stderr
|
||||
assert "ends before" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_reader_defaults_to_the_summary_view(run_script, fixtures):
|
||||
result = run_script("peutil.py", fixtures.pe64.path)
|
||||
assert result.returncode == 0
|
||||
assert "checksum" in result.stdout
|
||||
assert "sections" in result.stdout
|
||||
assert "amd64" in result.stdout
|
||||
|
||||
|
||||
# --- pepatch -----------------------------------------------------------------
|
||||
|
||||
def test_patcher_on_a_non_pe_exits_three(run_script, tmp_path):
|
||||
path = tmp_path / "notes.txt"
|
||||
path.write_text("not a PE\n", encoding="utf-8")
|
||||
result = run_script("pepatch.py", path, "--at-offset", "0", "--bytes", "CC", "--out",
|
||||
tmp_path / "out.exe")
|
||||
assert result.returncode == 3
|
||||
assert "not a parseable PE" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
assert not (tmp_path / "out.exe").exists()
|
||||
|
||||
|
||||
def test_patcher_on_a_truncated_image_reports_instead_of_raising(run_script, tmp_path, fixtures):
|
||||
"""Same contract as the reader's: a truncated image is an input, not a crash."""
|
||||
path = tmp_path / "truncated.exe"
|
||||
path.write_bytes(fixtures.pe64.data[:0x100])
|
||||
result = run_script("pepatch.py", path, "--at-offset", "0", "--bytes", "CC", "--out",
|
||||
tmp_path / "out.exe")
|
||||
assert result.returncode == 3
|
||||
assert "not a parseable PE" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
assert not (tmp_path / "out.exe").exists()
|
||||
|
||||
|
||||
def test_patcher_on_a_missing_file_exits_two(run_script, tmp_path):
|
||||
result = run_script("pepatch.py", tmp_path / "absent.exe", "--at-offset", "0",
|
||||
"--bytes", "CC", "--out", tmp_path / "out.exe")
|
||||
assert result.returncode == 2
|
||||
assert "is not a file" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
@pytest.mark.parametrize("extra", [
|
||||
(),
|
||||
("--bytes", "CC"),
|
||||
("--at-offset", "0x410"),
|
||||
])
|
||||
def test_patcher_without_a_usable_spec_exits_two(run_script, fixtures, extra):
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, *extra)
|
||||
assert result.returncode == 2
|
||||
assert "error:" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_patcher_with_an_empty_patch_list_exits_two(run_script, fixtures, tmp_path):
|
||||
spec = tmp_path / "empty.json"
|
||||
spec.write_text('{"patches": []}', encoding="utf-8")
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--spec", spec, "--out",
|
||||
tmp_path / "out.exe")
|
||||
assert result.returncode == 2
|
||||
assert 'has no "patches" array' in result.stderr
|
||||
assert not (tmp_path / "out.exe").exists()
|
||||
|
||||
|
||||
def test_patcher_with_a_missing_spec_file_uses_the_exit_code_vocabulary(run_script,
|
||||
fixtures, tmp_path):
|
||||
"""A spec file that is not there is a usage error, and it says so in one line."""
|
||||
result = run_script("pepatch.py", fixtures.pe64.path, "--spec", tmp_path / "absent.json",
|
||||
"--out", tmp_path / "out.exe")
|
||||
assert result.returncode == 2
|
||||
assert "is not a readable file" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
assert not (tmp_path / "out.exe").exists()
|
||||
|
||||
|
||||
# --- aob_scan ----------------------------------------------------------------
|
||||
|
||||
def test_locator_on_a_non_pe_exits_three(run_script, tmp_path):
|
||||
path = tmp_path / "notes.txt"
|
||||
path.write_text("not a PE\n", encoding="utf-8")
|
||||
result = run_script("aob_scan.py", path, "--patterns", "48 8B")
|
||||
assert result.returncode == 3
|
||||
assert "error:" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_locator_without_patterns_is_a_usage_error(run_script, fixtures):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path)
|
||||
assert result.returncode == 2
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_locator_on_a_malformed_pattern_exits_nonzero_with_a_message(run_script, fixtures):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "ZZ QQ")
|
||||
assert result.returncode == 1
|
||||
assert "not a byte" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_locator_on_a_missing_file_exits_three(run_script, tmp_path):
|
||||
result = run_script("aob_scan.py", tmp_path / "absent.exe", "--patterns", "48 8B")
|
||||
assert result.returncode == 3
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
# --- make_fixture_pe ---------------------------------------------------------
|
||||
|
||||
def test_fixture_builder_lists_its_recipes(run_script):
|
||||
result = run_script("make_fixture_pe.py", "--list-recipes")
|
||||
assert result.returncode == 0
|
||||
for recipe in ("default", "pe32", "bare", "fat-align", "thin-align", "no-virtual-tail"):
|
||||
assert recipe in result.stdout
|
||||
|
||||
|
||||
def test_fixture_builder_without_an_output_path_is_a_usage_error(run_script):
|
||||
result = run_script("make_fixture_pe.py")
|
||||
assert result.returncode == 2
|
||||
assert "--out is required" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_fixture_builder_rejects_an_unknown_recipe(run_script, tmp_path):
|
||||
result = run_script("make_fixture_pe.py", "--out", tmp_path / "x.exe",
|
||||
"--recipe", "no-such-recipe")
|
||||
assert result.returncode == 2
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_the_named_no_virtual_tail_recipe_builds(run_script, tmp_path):
|
||||
"""The recipe names a property it removes, and it can be used to remove it.
|
||||
|
||||
An earlier guard compared the .rdata payload against the requested tail itself, which
|
||||
inverted the relationship and made a zero-byte tail -- the whole point of the recipe --
|
||||
impossible to ask for. Pinned after the fix, because a recipe that is listed and cannot be
|
||||
built is a trap for whoever reads the list next.
|
||||
"""
|
||||
out = tmp_path / "novtail.exe"
|
||||
result = run_script("make_fixture_pe.py", "--out", out, "--recipe", "no-virtual-tail")
|
||||
assert result.returncode == 0
|
||||
assert out.exists()
|
||||
assert "virtual_only_tail_bytes" in result.stdout or "wrote" in result.stdout
|
||||
|
||||
|
||||
# --- scan_leaks --------------------------------------------------------------
|
||||
|
||||
def test_scanner_lists_its_rules_and_exemptions(run_script):
|
||||
result = run_script("scan_leaks.py", "--list-rules")
|
||||
assert result.returncode == 0
|
||||
assert "rules (18):" in result.stdout
|
||||
assert "exemptions (" in result.stdout
|
||||
assert "private-key-block" in result.stdout
|
||||
assert "tool or library name" in result.stdout
|
||||
|
||||
|
||||
def test_scanner_on_a_missing_root_exits_two_and_states_the_result(run_script, tmp_path):
|
||||
result = run_script("scan_leaks.py", "--root", tmp_path / "absent")
|
||||
assert result.returncode == 2
|
||||
assert result.stdout.strip().endswith("RESULT=error")
|
||||
assert "is not a directory" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_scanner_on_a_clean_root_exits_zero(run_script, clean_scan_root):
|
||||
result = run_script("scan_leaks.py", "--root", clean_scan_root)
|
||||
assert result.returncode == 0
|
||||
assert "RESULT=clean" in result.stdout
|
||||
|
||||
|
||||
def test_scanner_quiet_mode_prints_only_the_result_line(run_script, clean_scan_root):
|
||||
result = run_script("scan_leaks.py", "--root", clean_scan_root, "--quiet")
|
||||
assert result.returncode == 0
|
||||
assert result.stdout.strip() == "RESULT=clean"
|
||||
|
||||
|
||||
def test_scanner_exit_code_three_is_documented_but_unreachable(run_script, tmp_path):
|
||||
"""Measured deviation: the docstring promises 3 for an empty scan surface.
|
||||
|
||||
No code path returns 3, so an empty surface is reported as `clean` with exit 0 -- the exact
|
||||
distinction the docstring says it is making. Pinned so the mismatch is visible.
|
||||
"""
|
||||
empty = tmp_path / "empty"
|
||||
empty.mkdir()
|
||||
result = run_script("scan_leaks.py", "--root", empty)
|
||||
assert result.returncode == 0
|
||||
assert "RESULT=clean" in result.stdout
|
||||
assert "0 files" in result.stdout
|
||||
|
||||
|
||||
def test_scanner_on_a_root_with_findings_exits_one(run_script, synthetic_leak_root):
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--quiet")
|
||||
assert result.returncode == 1
|
||||
assert result.stdout.strip() == "RESULT=findings"
|
||||
|
||||
|
||||
def test_scanner_fail_on_any_raises_a_weak_only_surface_to_findings(run_script, tmp_path,
|
||||
weak_line):
|
||||
root = tmp_path / "weak"
|
||||
root.mkdir()
|
||||
(root / "note.md").write_text(weak_line + "\n", encoding="utf-8")
|
||||
default = run_script("scan_leaks.py", "--root", root, "--quiet")
|
||||
strict = run_script("scan_leaks.py", "--root", root, "--quiet", "--fail-on", "any")
|
||||
assert default.returncode == 0
|
||||
assert default.stdout.strip() == "RESULT=clean"
|
||||
assert strict.returncode == 1
|
||||
assert strict.stdout.strip() == "RESULT=findings"
|
||||
|
||||
|
||||
def test_scanner_only_filter_selects_one_category(run_script, synthetic_leak_root):
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--only", "token",
|
||||
"--format", "json")
|
||||
payload = json.JSONDecoder().raw_decode(result.stdout)[0]
|
||||
assert result.returncode == 1
|
||||
assert {hit["category"] for hit in payload["strong"]} == {"token"}
|
||||
assert payload["weak"] == []
|
||||
|
||||
|
||||
def test_scanner_skip_filter_drops_a_category(run_script, synthetic_leak_root):
|
||||
result = run_script("scan_leaks.py", "--root", synthetic_leak_root, "--skip",
|
||||
"token,path,device,endpoint,package,identity", "--format", "json")
|
||||
payload = json.JSONDecoder().raw_decode(result.stdout)[0]
|
||||
assert payload["strong"] == []
|
||||
assert payload["weak"] == []
|
||||
assert payload["verdict"] == "clean"
|
||||
assert result.returncode == 0
|
||||
|
||||
|
||||
# --- hwprobe -----------------------------------------------------------------
|
||||
|
||||
def test_prober_reports_routes_without_failing_on_a_limited_host(run_script):
|
||||
"""A missing capability is a finding, not a non-zero exit."""
|
||||
result = run_script("hwprobe.py", "--json", timeout=120)
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["routes"]
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_prober_tool_inventory_mode_skips_the_cpu_rows(run_script):
|
||||
result = run_script("hwprobe.py", "--tools", "--json", timeout=120)
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert "cpu" not in payload and "routes" not in payload
|
||||
assert payload["tools"] and payload["python_modules"]
|
||||
|
||||
|
||||
def test_inspect_only_runs_leave_their_working_directory_clean(run_script, fixtures,
|
||||
clean_scan_root):
|
||||
"""A negative outcome: none of these inspect-only runs may write into the cwd or the input.
|
||||
|
||||
The dry-run patcher is the case that matters: without ``--out`` it must report and stop.
|
||||
"""
|
||||
workdir = fixtures.pe64.path.parent
|
||||
before = sorted(p.name for p in workdir.iterdir())
|
||||
digest = fixtures.pe64.sha256
|
||||
run_script("peutil.py", fixtures.pe64.path, cwd=workdir)
|
||||
run_script("aob_scan.py", fixtures.pe64.path, "--patterns", "48 8B", cwd=workdir)
|
||||
run_script("pepatch.py", fixtures.pe64.path, "--at-offset", "0x410", "--bytes", "CC",
|
||||
"--dry-run", cwd=workdir)
|
||||
run_script("scan_leaks.py", "--root", clean_scan_root, cwd=workdir)
|
||||
assert sorted(p.name for p in workdir.iterdir()) == before
|
||||
assert fixtures.pe64.sha256 == digest
|
||||
@@ -0,0 +1,602 @@
|
||||
"""Shared fixtures for the exe-reverse regression suite.
|
||||
|
||||
What this file guarantees:
|
||||
|
||||
* every input is *built*, never committed and never downloaded -- the synthetic PE images come
|
||||
from ``skills/exe-reverse/scripts/make_fixture_pe.py`` at test time, into a pytest temporary
|
||||
directory that disappears with the session;
|
||||
* the scripts under test are loaded by path, because ``skills/exe-reverse/scripts/`` is a flat
|
||||
directory of runnable files, not an importable package;
|
||||
* a missing prerequisite -- an older interpreter, an absent local tool -- is a *skip with a
|
||||
printed reason*, never a failure. A wrong result is always a failure.
|
||||
|
||||
Nothing here touches the network, a debugger, a device or a kernel component.
|
||||
"""
|
||||
import hashlib
|
||||
import importlib.util
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import shutil
|
||||
import struct
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
TESTS_DIR = pathlib.Path(__file__).resolve().parent
|
||||
REPO_ROOT = TESTS_DIR.parent
|
||||
SCRIPTS_DIR = REPO_ROOT / "skills" / "exe-reverse" / "scripts"
|
||||
|
||||
# The scripts declare "Python 3.9+" in their own docstrings.
|
||||
MIN_PYTHON = (3, 9)
|
||||
|
||||
# Every script this suite may load by name or run as a subprocess. The last five were added with
|
||||
# their own layers of the suite: the capability registry and the environment doctor answer questions
|
||||
# about this host, the Dart snapshot reader takes an ELF rather than a PE (so its input is generated
|
||||
# here by `build_synthetic_elf` instead of by `make_fixture_pe.py`), the folder summarises records
|
||||
# too large to read (its tests assert on the bounds of its output rather than on fixtures of any
|
||||
# particular size), and the reducer's proof path drives an external solver, so its solver-facing
|
||||
# tests run against a stub and the pure parts against plain text.
|
||||
SCRIPT_FILES = (
|
||||
"peutil.py",
|
||||
"pepatch.py",
|
||||
"hwprobe.py",
|
||||
"aob_scan.py",
|
||||
"make_fixture_pe.py",
|
||||
"scan_leaks.py",
|
||||
"capabilities.py",
|
||||
"doctor.py",
|
||||
"dart_snapshot_strings.py",
|
||||
"fold.py",
|
||||
"trace_reduce.py",
|
||||
)
|
||||
|
||||
# A hang detector, not a performance assertion. Nothing in this suite is meant to be slow.
|
||||
RUN_TIMEOUT = 180
|
||||
|
||||
# Suite name -> recipe name in make_fixture_pe.py, so recipes are referenced rather than
|
||||
# duplicated. `default` is PE32+ with every structure; `pe32` is the 32-bit layout; `bare`
|
||||
# drops exports/resources/TLS/debug; `fatalign` uses a 4 KiB file alignment, which is large
|
||||
# enough that .rdata has no virtual-only tail at all.
|
||||
#
|
||||
# The `no-virtual-tail` recipe is deliberately NOT used: with `rdata_virtual_pad = 0` its own
|
||||
# guard (`rdata_payload_len > rdata_virtual_pad`) rejects the layout it names, so the recipe
|
||||
# cannot be built and exits 1. `fatalign` covers the same property. Recorded in benchmark.md.
|
||||
FIXTURE_RECIPES = {
|
||||
"pe64": "default",
|
||||
"pe32": "pe32",
|
||||
"bare": "bare",
|
||||
"fatalign": "fat-align",
|
||||
}
|
||||
|
||||
# --- synthetic leak corpus ---------------------------------------------------
|
||||
# One line per rule in scan_leaks.py. Each line is shaped like the thing its rule targets, and
|
||||
# (except for machine-guid, which is a measured observation -- see the unit tests) is not
|
||||
# suppressed by the exemption list.
|
||||
#
|
||||
# Every sample is assembled from fragments at import time rather than quoted whole. The suite
|
||||
# deliberately contains strings shaped like a pasted key, a token and a real profile path; if
|
||||
# they were written whole, the repository's own pre-publish scan would report this file the
|
||||
# moment it walked the tracked surface. Splitting them keeps the runtime value matched by the
|
||||
# rule while the source text matches nothing. A scanner's own fixtures have to be written this
|
||||
# way, and that is a property of the scan surface, not a trick.
|
||||
def parts(*chunks):
|
||||
"""Join string fragments at run time (see the note above)."""
|
||||
return "".join(chunks)
|
||||
|
||||
|
||||
LEAK_LINES = (
|
||||
("private-key-block", parts("-----BEGIN RSA ", "PRIVATE KEY-----")),
|
||||
("github-token", parts("ghp_", "A1b2C3d4E5f6G7h8I9j0K1l2M3n4O5p6Q7r8")),
|
||||
("aws-access-key", parts("AKIA", "Q7ZK3MNP2RVW8XTD")),
|
||||
("slack-token", parts("xoxb-", "1234567890abcdef")),
|
||||
("generic-secret-assignment", parts('api_key = "', "QwErTyUiOpAsDfGhJkLzXcVb", '"')),
|
||||
("licence-blob-literal", parts('license_key: "', "ABCDEFGH1234", '"')),
|
||||
("windows-user-profile", parts("C:\\Users\\", "mkorvin", "\\notes.md")),
|
||||
("posix-home", parts("/home/", "mkorvin", "/notes.md")),
|
||||
("machine-guid", parts("MachineGuid = ", "3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071")),
|
||||
("windows-sid-literal", parts("S-1-5-21-", "1004336348-1177238915-682003330-", "1001")),
|
||||
("volume-serial-literal", parts("Volume Serial Number: ", "1A2B-3C4D")),
|
||||
("literal-endpoint-with-creds", parts("ftp://", "deploy:Sup3rSecret@", "45.77.", "10.199/pub")),
|
||||
("public-ipv4-literal", parts("45.77.", "10.199")),
|
||||
("literal-host-with-path", parts("https://licensing.", "hexweld.", "org/v2/activate")),
|
||||
("product-name-with-version", parts("Hexweld Editor ", "4.2.1")),
|
||||
("exe-name-with-version", parts("build: hexweld", ".exe v4.2")),
|
||||
("email-address", parts("k.morozov@", "hexweld.org")),
|
||||
("vendor-copyright-line", parts("Copyright (c) 2024 ", "Hexweld Systems")),
|
||||
)
|
||||
|
||||
# Lines that must never be reported: tool names, protector names, format vocabulary, platform
|
||||
# terms, placeholder tokens, reserved documentation addresses, well-known SIDs.
|
||||
EXEMPT_LINES = (
|
||||
("tool name", "IDA Pro 9.0.2 opened the image"),
|
||||
("protector name", "the UPX 4.2 stub wraps a Themida section"),
|
||||
("format term", "RVA versus file offset; the PDB is RSDS CodeView"),
|
||||
("platform term", "Microsoft Windows 11.0 build 26100 with PowerShell"),
|
||||
("placeholder token", "TARGET: APP_URL and SAMPLE with PLACEHOLDER values"),
|
||||
("documentation address", "203.0.113.9 and 198.51.100.7 and 192.0.2.44"),
|
||||
("well-known SID", "S-1-5-18 and S-1-5-32-544 are not machine identity"),
|
||||
("loopback and example", "127.0.0.1 and localhost and example.com"),
|
||||
("machine guid context", parts("MachineGuid = ", "3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071")),
|
||||
("net tld", parts("https://licensing.", "hexweld.", "net/v2/activate")),
|
||||
)
|
||||
|
||||
# Measured deviations from the exemption list, pinned as observations rather than wishes. Each
|
||||
# is asserted by a unit test so a change to either list is visible instead of silent:
|
||||
# * the bare-GUID exemption matches inside the machine-guid rule's own match text, so that
|
||||
# rule cannot produce a finding;
|
||||
# * the `.NET` entry matches any `.net` TLD, so a hit inside such a URL or address is
|
||||
# suppressed as a platform term;
|
||||
# * the `<[a-z_]+>` placeholder alternative is surrounded by `\b` on both sides, so it cannot
|
||||
# match bracketed prose; only word-character adjacency reaches it.
|
||||
#
|
||||
# A fourth deviation was measured and then corrected while this suite was being written: for one
|
||||
# revision of the exemption list, a standard-name alternative matched the `HTTP` inside every
|
||||
# `https?://` URL and swallowed every literal-host-with-path hit. benchmark.md records both
|
||||
# states; the suite asserts the corrected one.
|
||||
SUPPRESSED_RULES = ("machine-guid",)
|
||||
|
||||
# Lines that must never be reported: tool names, protector names, format vocabulary, platform
|
||||
# terms, placeholder tokens, reserved documentation addresses, well-known SIDs.
|
||||
EXEMPT_LINES = (
|
||||
("tool name", "IDA Pro 9.0.2 opened the image"),
|
||||
("protector name", "the UPX 4.2 stub wraps a Themida section"),
|
||||
("format term", "RVA versus file offset; the PDB is RSDS CodeView"),
|
||||
("platform term", "Microsoft Windows 11.0 build 26100 with PowerShell"),
|
||||
("placeholder token", "TARGET: APP_URL and SAMPLE with PLACEHOLDER values"),
|
||||
("documentation address", "203.0.113.9 and 198.51.100.7 and 192.0.2.44"),
|
||||
("well-known SID", "S-1-5-18 and S-1-5-32-544 are not machine identity"),
|
||||
("loopback and example", "127.0.0.1 and localhost and example.com"),
|
||||
("machine guid context", "MachineGuid = 3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071"),
|
||||
("net tld", "https://licensing.hexweld.net/v2/activate"),
|
||||
)
|
||||
|
||||
# The one rule whose hits are suppressed by the exemption list. Measured, not assumed: the
|
||||
# bare-GUID exemption matches inside the machine-guid rule's own match text. Pinned by a unit
|
||||
# test so a change in either list is visible instead of silent.
|
||||
SUPPRESSED_RULES = ("machine-guid",)
|
||||
|
||||
|
||||
# --- synthetic ELF64 for dart_snapshot_strings.py -----------------------------
|
||||
# That script's input is a Dart AOT snapshot -- an ELF shared object named `app.so` -- and this
|
||||
# repository may never carry a sample, not even a small one. So the suite builds the container
|
||||
# instead: a valid ELF64 header, a `.text` of a handful of bytes, a `.rodata` holding one one-byte
|
||||
# (ASCII) literal and one two-byte (UTF-16LE) literal at known offsets, and a section-name string
|
||||
# table. A few hundred bytes, written at test time into the session temporary directory.
|
||||
#
|
||||
# What this image is *not*: a real snapshot. It exists so the reader can be held against a container
|
||||
# whose offsets are known in advance, which is what proves the literal-recovery logic. The
|
||||
# instruction-level counts an earlier version of the reader reported were withdrawn -- a byte scan
|
||||
# cannot read this format, and the reference records why -- so nothing here stands in for them.
|
||||
|
||||
# Neither `49 8B /r` (a pool load through r15) nor `4X 8B|8D 05` (a RIP-relative load) appears in
|
||||
# these bytes, which is what makes the zero counts below a measurement rather than an artifact.
|
||||
ELF_TEXT_BYTES = bytes((0x90, 0x90, 0xC3, 0xCC, 0x55, 0x48, 0x89, 0xE5,
|
||||
0x5D, 0xC3, 0x0F, 0x1F, 0x44, 0x00, 0x00, 0x90))
|
||||
ELF_TEXT_VADDR = 0x1000
|
||||
# Large enough that a virtual address is never mistaken for a file offset: the reader treats an
|
||||
# `--at` value above the file size as a vaddr, and this image is under 1 KiB.
|
||||
ELF_RODATA_VADDR = 0x400000
|
||||
|
||||
# (offset within .rodata, vaddr offset, text, encoding). The ASCII run at 0x00 and the UTF-16LE run
|
||||
# at 0x1C are the two shapes the reader documents; the NUL padding between them keeps the two-byte
|
||||
# run on the phase the scan walks, because that scan advances from the section base in steps of two.
|
||||
# That phase dependency is a defect in the reader rather than a property of a snapshot, and it is
|
||||
# pinned in unit/test_dart_snapshot_strings.py; the fixture places its literals so the recovery path
|
||||
# can be measured at all.
|
||||
ELF_LITERALS = (
|
||||
(0x00, "premium_license", "one-byte"),
|
||||
(0x10, "trial_mode", "one-byte"),
|
||||
(0x1C, "vip_pass", "two-byte"),
|
||||
)
|
||||
|
||||
|
||||
def _elf_section_header(name_off, typ, flags, addr, offset, size, align=8, entsize=0,
|
||||
link=0, info=0):
|
||||
"""One 64-byte ELF64 section header, in the order the reader unpacks it."""
|
||||
return struct.pack("<IIQQQQIIQQ", name_off, typ, flags, addr, offset, size,
|
||||
link, info, align, entsize)
|
||||
|
||||
|
||||
def build_synthetic_elf(with_text=True):
|
||||
"""(data, meta) for a minimal ELF64 whose `.rodata` holds three known literals.
|
||||
|
||||
The layout is derived here rather than asserted from a table, so a change to any literal's size
|
||||
moves the offsets with it and the tests still compare against where the fixture *did* put them.
|
||||
"""
|
||||
text = ELF_TEXT_BYTES
|
||||
rodata = b"premium_license\x00" + b"trial_mode\x00\x00"
|
||||
rodata += "vip_pass".encode("utf-16-le") + b"\x00\x00"
|
||||
rodata += b"\x00" * ((8 - len(rodata) % 8) % 8)
|
||||
shstr = b"\x00.text\x00.rodata\x00.shstrtab\x00"
|
||||
|
||||
ehdr_size = 64
|
||||
text_off = 0x40
|
||||
rodata_off = text_off + (len(text) if with_text else 0)
|
||||
shstr_off = rodata_off + len(rodata)
|
||||
shoff = shstr_off + len(shstr)
|
||||
shoff += (8 - shoff % 8) % 8
|
||||
|
||||
sections = [
|
||||
# The NULL section every image starts with; the reader does not special-case it.
|
||||
_elf_section_header(0, 0, 0, 0, 0, 0, align=0),
|
||||
]
|
||||
if with_text:
|
||||
sections.append(_elf_section_header(1, 1, 0x6, ELF_TEXT_VADDR, text_off,
|
||||
len(text)))
|
||||
sections.append(_elf_section_header(7, 1, 0x2, ELF_RODATA_VADDR, rodata_off,
|
||||
len(rodata)))
|
||||
sections.append(_elf_section_header(15, 3, 0, 0, shstr_off, len(shstr), align=1))
|
||||
shnum = len(sections)
|
||||
|
||||
ident = b"\x7fELF" + bytes((2, 1, 1, 0)) + b"\x00" * 8
|
||||
ehdr = ident + struct.pack("<HHIQQQIHHHHHH",
|
||||
3, # e_type = ET_DYN, a shared object
|
||||
0x3E, # e_machine = EM_X86_64
|
||||
1, # e_version
|
||||
0, # e_entry
|
||||
0, # e_phoff: no program headers
|
||||
shoff, # e_shoff
|
||||
0, # e_flags
|
||||
ehdr_size, # e_ehsize
|
||||
0, # e_phentsize
|
||||
0, # e_phnum
|
||||
64, # e_shentsize
|
||||
shnum, # e_shnum
|
||||
shnum - 1) # e_shstrndx: the last section is .shstrtab
|
||||
|
||||
data = bytearray(b"\x00" * shoff)
|
||||
data[0:ehdr_size] = ehdr
|
||||
if with_text:
|
||||
data[text_off:text_off + len(text)] = text
|
||||
data[rodata_off:rodata_off + len(rodata)] = rodata
|
||||
data[shstr_off:shstr_off + len(shstr)] = shstr
|
||||
data.extend(b"".join(sections))
|
||||
|
||||
meta = {
|
||||
"size": len(data),
|
||||
"text": {"offset": text_off, "vaddr": ELF_TEXT_VADDR, "size": len(text)},
|
||||
"rodata": {"offset": rodata_off, "vaddr": ELF_RODATA_VADDR, "size": len(rodata)},
|
||||
"literals": [
|
||||
{"offset": rodata_off + rel, "vaddr": ELF_RODATA_VADDR + rel,
|
||||
"text": value, "encoding": encoding}
|
||||
for rel, value, encoding in ELF_LITERALS
|
||||
],
|
||||
}
|
||||
return bytes(data), meta
|
||||
|
||||
|
||||
class SyntheticElf(object):
|
||||
"""A generated ELF image plus the offsets its literals were placed at."""
|
||||
|
||||
__slots__ = ("path", "data", "meta")
|
||||
|
||||
def __init__(self, path, data, meta):
|
||||
self.path = path
|
||||
self.data = data
|
||||
self.meta = meta
|
||||
|
||||
@property
|
||||
def size(self):
|
||||
return len(self.data)
|
||||
|
||||
@property
|
||||
def text(self):
|
||||
return self.meta["text"]
|
||||
|
||||
@property
|
||||
def rodata(self):
|
||||
return self.meta["rodata"]
|
||||
|
||||
@property
|
||||
def literals(self):
|
||||
return self.meta["literals"]
|
||||
|
||||
def literal(self, value):
|
||||
"""The fixture record for one literal, by its text."""
|
||||
for entry in self.literals:
|
||||
if entry["text"] == value:
|
||||
return entry
|
||||
raise KeyError("%r is not one of the fixture's literals: %r"
|
||||
% (value, [e["text"] for e in self.literals]))
|
||||
|
||||
def __repr__(self):
|
||||
return "SyntheticElf(%s, %d bytes)" % (self.path, self.size)
|
||||
|
||||
|
||||
def pytest_report_header(config):
|
||||
return [
|
||||
"exe-reverse suite: python %s, scripts %s" % (sys.version.split()[0], SCRIPTS_DIR),
|
||||
]
|
||||
|
||||
|
||||
class ScriptResult(object):
|
||||
"""One script run: returncode, stdout, stderr, plus the argv that produced it."""
|
||||
|
||||
__slots__ = ("returncode", "stdout", "stderr", "argv")
|
||||
|
||||
def __init__(self, returncode, stdout, stderr, argv):
|
||||
self.returncode = returncode
|
||||
self.stdout = stdout
|
||||
self.stderr = stderr
|
||||
self.argv = argv
|
||||
|
||||
def __iter__(self):
|
||||
return iter((self.returncode, self.stdout, self.stderr))
|
||||
|
||||
def __repr__(self):
|
||||
return "ScriptResult(returncode=%r, argv=%r)" % (self.returncode, self.argv)
|
||||
|
||||
|
||||
class BuiltFixture(object):
|
||||
"""A generated fixture image plus the metadata make_fixture_pe.py returned for it."""
|
||||
|
||||
__slots__ = ("name", "path", "meta", "recipe")
|
||||
|
||||
def __init__(self, name, path, meta, recipe):
|
||||
self.name = name
|
||||
self.path = path
|
||||
self.meta = meta
|
||||
self.recipe = recipe
|
||||
|
||||
@property
|
||||
def data(self):
|
||||
return self.path.read_bytes()
|
||||
|
||||
@property
|
||||
def size(self):
|
||||
return len(self.data)
|
||||
|
||||
@property
|
||||
def sha256(self):
|
||||
return hashlib.sha256(self.data).hexdigest()
|
||||
|
||||
@property
|
||||
def plant(self):
|
||||
"""The byte sequence planted twice in .text, as a spaced hex string."""
|
||||
return self.meta["plant"]["bytes"]
|
||||
|
||||
@property
|
||||
def plant_offsets(self):
|
||||
return (int(self.meta["plant"]["first_file_offset"], 16),
|
||||
int(self.meta["plant"]["second_file_offset"], 16))
|
||||
|
||||
def section(self, name):
|
||||
return self.meta["sections"][name]
|
||||
|
||||
def __repr__(self):
|
||||
return "BuiltFixture(%r, %s)" % (self.name, self.path)
|
||||
|
||||
|
||||
class FixtureSet(object):
|
||||
"""Attribute and index access to the built fixtures."""
|
||||
|
||||
def __init__(self, items):
|
||||
self._items = dict(items)
|
||||
|
||||
def __getattr__(self, name):
|
||||
try:
|
||||
return self._items[name]
|
||||
except KeyError:
|
||||
raise AttributeError(name)
|
||||
|
||||
def __getitem__(self, name):
|
||||
return self._items[name]
|
||||
|
||||
def names(self):
|
||||
return sorted(self._items)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session", autouse=True)
|
||||
def interpreter_gate():
|
||||
"""Skip everything, with a reason, when the interpreter is older than the scripts require."""
|
||||
if sys.version_info < MIN_PYTHON:
|
||||
pytest.skip("skills/exe-reverse/scripts declare Python %d.%d+; this session is running "
|
||||
"%s -- install a newer interpreter to exercise the suite"
|
||||
% (MIN_PYTHON[0], MIN_PYTHON[1], sys.version.split()[0]))
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def require_local_tool():
|
||||
"""Skip a check, with a reason, when a tool it needs is not installed."""
|
||||
|
||||
def _need(tool):
|
||||
found = shutil.which(tool)
|
||||
if not found:
|
||||
pytest.skip("local tool %r is not on PATH; this check is inert without it" % tool)
|
||||
return found
|
||||
|
||||
return _need
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def load_script():
|
||||
"""Import a tested script by path, once per session.
|
||||
|
||||
``pepatch.py`` and ``aob_scan.py`` put their own directory on ``sys.path`` and then import
|
||||
``peutil`` by bare name. Registering one shared module per name keeps a single PE class in
|
||||
the process instead of two classes that compare unequal.
|
||||
"""
|
||||
cache = {}
|
||||
|
||||
def _load(filename):
|
||||
if filename not in SCRIPT_FILES:
|
||||
raise ValueError("%r is not one of the tested scripts: %r" % (filename, SCRIPT_FILES))
|
||||
name = filename[:-3]
|
||||
if name in cache:
|
||||
return cache[name]
|
||||
if str(SCRIPTS_DIR) not in sys.path:
|
||||
sys.path.insert(0, str(SCRIPTS_DIR))
|
||||
spec = importlib.util.spec_from_file_location(name, SCRIPTS_DIR / filename)
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[name] = module
|
||||
spec.loader.exec_module(module)
|
||||
cache[name] = module
|
||||
return module
|
||||
|
||||
return _load
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def run_script():
|
||||
"""Run a tested script as a subprocess and return (returncode, stdout, stderr)."""
|
||||
|
||||
def _run(filename, *args, **kwargs):
|
||||
if filename not in SCRIPT_FILES:
|
||||
raise ValueError("%r is not one of the tested scripts: %r" % (filename, SCRIPT_FILES))
|
||||
timeout = kwargs.pop("timeout", RUN_TIMEOUT)
|
||||
cwd = kwargs.pop("cwd", None)
|
||||
if kwargs:
|
||||
raise TypeError("unexpected keyword arguments: %r" % sorted(kwargs))
|
||||
argv = [sys.executable, str(SCRIPTS_DIR / filename)] + [str(a) for a in args]
|
||||
try:
|
||||
proc = subprocess.run(argv, capture_output=True, text=True, timeout=timeout,
|
||||
cwd=cwd, encoding="utf-8", errors="replace")
|
||||
except subprocess.TimeoutExpired:
|
||||
pytest.fail("%s did not finish within %ds: %r" % (filename, timeout, argv))
|
||||
return ScriptResult(proc.returncode, proc.stdout, proc.stderr, argv)
|
||||
|
||||
return _run
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def fixture_dir(tmp_path_factory):
|
||||
return tmp_path_factory.mktemp("exe-reverse-fixtures")
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def build_fixture(fixture_dir, load_script, run_script):
|
||||
"""Build one named fixture through the real CLI and cross-check it against build_pe()."""
|
||||
maker = load_script("make_fixture_pe.py")
|
||||
cache = {}
|
||||
|
||||
def _build(name):
|
||||
if name in cache:
|
||||
return cache[name]
|
||||
if name not in FIXTURE_RECIPES:
|
||||
raise ValueError("unknown fixture %r; known: %r" % (name, sorted(FIXTURE_RECIPES)))
|
||||
recipe = FIXTURE_RECIPES[name]
|
||||
dest = fixture_dir / (name + ".exe")
|
||||
result = run_script("make_fixture_pe.py", "--out", dest, "--recipe", recipe)
|
||||
if result.returncode != 0:
|
||||
pytest.fail("make_fixture_pe.py --recipe %s failed with %d: %s"
|
||||
% (recipe, result.returncode, result.stderr))
|
||||
data, meta = maker.build_pe(**maker.RECIPES[recipe])
|
||||
if dest.read_bytes() != data:
|
||||
pytest.fail("make_fixture_pe.py wrote bytes that differ from its own build_pe() "
|
||||
"output -- the CLI and the library disagree")
|
||||
built = BuiltFixture(name, dest, meta, recipe)
|
||||
cache[name] = built
|
||||
return built
|
||||
|
||||
return _build
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def fixtures(build_fixture):
|
||||
"""The four generated images, addressed as ``fixtures.pe64`` / ``fixtures['pe32']``."""
|
||||
return FixtureSet({name: build_fixture(name) for name in FIXTURE_RECIPES})
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def synthetic_elf(fixture_dir):
|
||||
"""The generated ELF64, written once for the whole session."""
|
||||
data, meta = build_synthetic_elf()
|
||||
path = fixture_dir / "app.so"
|
||||
path.write_bytes(data)
|
||||
return SyntheticElf(path, data, meta)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def synthetic_elf_without_text(fixture_dir):
|
||||
"""The same image with no `.text`, for the reader's "not an AOT code image" path."""
|
||||
data, meta = build_synthetic_elf(with_text=False)
|
||||
path = fixture_dir / "no-text.so"
|
||||
path.write_bytes(data)
|
||||
return SyntheticElf(path, data, meta)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def hwprobe_report(run_script):
|
||||
"""One host capability report for the whole session.
|
||||
|
||||
The probe executes a CPUID stub, so re-running it per assertion would buy nothing and cost
|
||||
seconds. It is a host fact, not a per-test one.
|
||||
"""
|
||||
result = run_script("hwprobe.py", "--json")
|
||||
if result.returncode != 0:
|
||||
pytest.fail("hwprobe.py --json exited %d: %s" % (result.returncode, result.stderr))
|
||||
return json.loads(result.stdout)
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def hexpat():
|
||||
"""Render bytes as the spaced hex form the scripts accept."""
|
||||
|
||||
def _hexpat(obj):
|
||||
if isinstance(obj, str):
|
||||
return " ".join("%02X" % b for b in obj.encode("latin-1"))
|
||||
return " ".join("%02X" % b for b in bytearray(obj))
|
||||
|
||||
return _hexpat
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def leak_lines():
|
||||
return LEAK_LINES
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def exempt_lines():
|
||||
return EXEMPT_LINES
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def weak_line(leak_lines):
|
||||
"""A sample that produces exactly one weak finding and no strong one."""
|
||||
return dict(leak_lines)["vendor-copyright-line"]
|
||||
|
||||
|
||||
@pytest.fixture(scope="session")
|
||||
def strong_line(leak_lines):
|
||||
"""A sample that produces a strong finding."""
|
||||
return dict(leak_lines)["windows-user-profile"]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def synthetic_leak_root(tmp_path):
|
||||
"""A scan surface holding one shaped leak per rule, plus exemption lines."""
|
||||
root = tmp_path / "leakroot"
|
||||
root.mkdir()
|
||||
body = ["# synthetic analysis notes"] + [line for _rule, line in LEAK_LINES]
|
||||
(root / "notes.md").write_text("\n".join(body) + "\n", encoding="utf-8")
|
||||
(root / "exempt.md").write_text("\n".join(line for _cls, line in EXEMPT_LINES) + "\n",
|
||||
encoding="utf-8")
|
||||
nested = root / "skills" / "exe-reverse" / "references"
|
||||
nested.mkdir(parents=True)
|
||||
(nested / "deep.md").write_text("no identity here\n", encoding="utf-8")
|
||||
return root
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def clean_scan_root(tmp_path):
|
||||
"""A scan surface with nothing to report."""
|
||||
root = tmp_path / "clean"
|
||||
root.mkdir()
|
||||
(root / "notes.md").write_text(
|
||||
"Static PE analysis of the generated fixture: sections, imports, checksum.\n",
|
||||
encoding="utf-8")
|
||||
return root
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def platform_binary(require_local_tool):
|
||||
"""A real platform image to parse, or a skip when this host has none."""
|
||||
root = os.environ.get("SystemRoot", r"C:\Windows")
|
||||
for rel in (r"System32\notepad.exe", r"System32\kernel32.dll", r"System32\ntdll.dll"):
|
||||
candidate = pathlib.Path(root) / rel
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
pytest.skip("no platform image under %s (System32 not present); this check is inert here"
|
||||
% root)
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Locator comparison between two builds of the same fixture.
|
||||
|
||||
`--across` answers one question: does the pattern still match the same number of sites on the
|
||||
other build? A count that changes is a statement about the signature, not about the target, and
|
||||
the point of these tests is that each of the three answers (same, changed, unusable either way)
|
||||
is distinguishable in the output and in the exit code.
|
||||
"""
|
||||
import json
|
||||
import shutil
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def rebuilt(run_script, tmp_path, fixtures):
|
||||
"""A second, independently built copy of the same recipe."""
|
||||
out = tmp_path / "rebuilt.exe"
|
||||
result = run_script("make_fixture_pe.py", "--out", out, "--recipe", fixtures.pe64.recipe)
|
||||
assert result.returncode == 0, result.stderr
|
||||
return out
|
||||
|
||||
|
||||
def test_same_recipe_reports_an_unchanged_hit_count(run_script, fixtures, rebuilt, hexpat):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
|
||||
"--across", rebuilt, "--json")
|
||||
assert result.returncode == 0, result.stdout
|
||||
payload = json.loads(result.stdout)
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["hits"] == entry["across"]["hits"] == 1
|
||||
assert entry["across"]["same_count"] is True
|
||||
assert entry["across"]["verdict"] == "unique"
|
||||
assert payload["result"] == "ok"
|
||||
|
||||
|
||||
def test_a_recipe_without_the_debug_directory_changes_the_hit_count(run_script, fixtures,
|
||||
hexpat):
|
||||
"""The `bare` recipe has no debug directory, so the PDB pattern has nothing to match."""
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
|
||||
"--across", fixtures.bare.path, "--json")
|
||||
assert result.returncode == 1
|
||||
payload = json.loads(result.stdout)
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["hits"] == 1
|
||||
assert entry["across"]["hits"] == 0
|
||||
assert entry["across"]["same_count"] is False
|
||||
assert entry["across"]["verdict"] == "no-match"
|
||||
assert entry["across"]["sites"] == []
|
||||
assert payload["result"] == "needs-attention"
|
||||
|
||||
|
||||
def test_an_equal_hit_count_is_not_the_same_as_a_usable_locator(run_script, fixtures):
|
||||
"""Two hits on both sides: the count agrees and the pattern is still a coin flip."""
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
|
||||
"--across", fixtures.bare.path, "--json")
|
||||
assert result.returncode == 1
|
||||
payload = json.loads(result.stdout)
|
||||
entry = payload["patterns"][0]
|
||||
assert entry["hits"] == entry["across"]["hits"] == 2
|
||||
assert entry["across"]["same_count"] is True
|
||||
assert entry["verdict"] == "ambiguous"
|
||||
assert payload["result"] == "needs-attention"
|
||||
|
||||
|
||||
def test_the_text_view_explains_a_changed_count(run_script, fixtures, hexpat):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", hexpat(b"fixture.pdb"),
|
||||
"--across", fixtures.bare.path)
|
||||
assert result.returncode == 1
|
||||
assert "hit count changed between builds" in result.stdout
|
||||
assert "across" in result.stdout
|
||||
|
||||
|
||||
def test_the_text_view_explains_an_absent_pattern(run_script, fixtures, hexpat):
|
||||
result = run_script("aob_scan.py", fixtures.bare.path, "--patterns", hexpat(b"fixture.pdb"))
|
||||
assert result.returncode == 1
|
||||
assert "does not exist in this build" in result.stdout
|
||||
|
||||
|
||||
def test_a_patch_invalidates_the_signature_it_was_located_with(run_script, fixtures, tmp_path,
|
||||
hexpat):
|
||||
"""The complete lesson of the two tools together, in one flow.
|
||||
|
||||
The pattern that located the site includes the byte that gets patched, so after the patch the
|
||||
pattern no longer matches that site. `--across` between the original and the patched copy is
|
||||
what turns that from a silent hazard into a reported count change.
|
||||
"""
|
||||
patched = tmp_path / "patched.exe"
|
||||
shutil.copyfile(str(fixtures.pe64.path), str(patched))
|
||||
applied = run_script("pepatch.py", patched, "--pattern", fixtures.pe64.plant,
|
||||
"--at-hit", "0", "--expect", fixtures.pe64.plant,
|
||||
"--bytes", " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"]),
|
||||
"--out", patched)
|
||||
assert applied.returncode == 0, applied.stderr
|
||||
|
||||
located_first = run_script("aob_scan.py", patched, "--patterns", fixtures.pe64.plant, "--json")
|
||||
assert located_first.returncode == 0
|
||||
after = json.loads(located_first.stdout)["patterns"][0]
|
||||
assert after["hits"] == 1, "the patched site no longer matches the pattern it was found with"
|
||||
assert after["verdict"] == "unique"
|
||||
assert after["sites"][0]["offset_hex"] == fixtures.pe64.meta["plant"]["second_file_offset"]
|
||||
|
||||
# The hazard, made concrete: the same pattern with the same index now selects a different
|
||||
# site than it did before the patch, without any error from either tool.
|
||||
dry = run_script("pepatch.py", patched, "--pattern", fixtures.pe64.plant, "--at-hit", "0",
|
||||
"--expect", fixtures.pe64.plant,
|
||||
"--bytes", " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"]),
|
||||
"--dry-run", "--json")
|
||||
assert dry.returncode == 0
|
||||
assert json.loads(dry.stdout)["applied"][0]["offset"] == \
|
||||
int(fixtures.pe64.meta["plant"]["second_file_offset"], 16)
|
||||
|
||||
# Unpatched copy on the left, patched copy on the right: the counts disagree, and that is
|
||||
# reported rather than smoothed over.
|
||||
comparison = run_script("aob_scan.py", fixtures.pe64.path, "--patterns",
|
||||
fixtures.pe64.plant, "--across", patched, "--json")
|
||||
assert comparison.returncode == 1
|
||||
entry = json.loads(comparison.stdout)["patterns"][0]
|
||||
assert entry["hits"] == 2
|
||||
assert entry["across"]["hits"] == 1
|
||||
assert entry["across"]["same_count"] is False
|
||||
|
||||
|
||||
def test_comparing_against_a_non_pe_build_exits_three(run_script, fixtures, tmp_path):
|
||||
other = tmp_path / "not-a-pe.txt"
|
||||
other.write_text("this is not a portable executable\n", encoding="utf-8")
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
|
||||
"--across", other)
|
||||
assert result.returncode == 3
|
||||
assert "--across" in result.stderr
|
||||
assert "Traceback" not in result.stderr
|
||||
|
||||
|
||||
def test_comparing_against_a_missing_build_exits_three(run_script, fixtures, tmp_path):
|
||||
result = run_script("aob_scan.py", fixtures.pe64.path, "--patterns", fixtures.pe64.plant,
|
||||
"--across", tmp_path / "absent.exe")
|
||||
assert result.returncode == 3
|
||||
assert "Traceback" not in result.stderr
|
||||
@@ -0,0 +1,220 @@
|
||||
"""End-to-end flows over the generated fixture: locate, patch, re-read, verify.
|
||||
|
||||
No debugger, no driver, no network and no live target takes part. Every step is a script call,
|
||||
and the assertions are on what the next script in the chain observes -- which is the property
|
||||
that matters, because these tools are meant to agree with each other about one image.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import shutil
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.integration
|
||||
|
||||
|
||||
def sha256(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def work(tmp_path, fixtures):
|
||||
"""An isolated copy of the PE32+ fixture, so a flow can never disturb another test."""
|
||||
target = tmp_path / "sample.exe"
|
||||
shutil.copyfile(str(fixtures.pe64.path), str(target))
|
||||
return target
|
||||
|
||||
|
||||
def test_locate_patch_reread_round_trip(run_script, fixtures, work, tmp_path, hexpat):
|
||||
"""The full chain, with each tool checked against what the next one reports."""
|
||||
patched = tmp_path / "patched.exe"
|
||||
first, second = fixtures.pe64.plant_offsets
|
||||
|
||||
# 1. locate: the planted sequence is deliberately ambiguous, and the locator says so.
|
||||
located = run_script("aob_scan.py", work, "--patterns", fixtures.pe64.plant, "--json")
|
||||
assert located.returncode == 1
|
||||
plan = json.loads(located.stdout)
|
||||
assert plan["patterns"][0]["verdict"] == "ambiguous"
|
||||
site = plan["patterns"][0]["sites"][0]
|
||||
|
||||
# 2. patch the first site only, keeping the replacement the same length as the site.
|
||||
replacement = " ".join(fixtures.pe64.plant.split()[:-2] + ["EB", "05"])
|
||||
applied = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant,
|
||||
"--at-hit", "0", "--expect", fixtures.pe64.plant,
|
||||
"--bytes", replacement, "--out", patched, "--json")
|
||||
assert applied.returncode == 0, applied.stderr
|
||||
result = json.loads(applied.stdout)
|
||||
claim = result["applied"][0]
|
||||
assert result["patches_applied"] == 1
|
||||
assert result["verify_problems"] == []
|
||||
|
||||
# 3. the locator and the patcher must be talking about the same bytes.
|
||||
assert int(site["offset_hex"], 16) == claim["offset"] == first
|
||||
assert int(site["rva_hex"], 16) == int(claim["rva"], 16)
|
||||
assert claim["before"] == fixtures.pe64.plant
|
||||
assert claim["after"] == replacement
|
||||
assert claim["equal_length"] is True
|
||||
|
||||
# 4. re-read the written file: the patch landed, the control site did not move, and the
|
||||
# checksum was recomputed rather than left stale.
|
||||
reread = run_script("peutil.py", patched, "--summary", "--check-sum", "--json")
|
||||
assert reread.returncode == 0, reread.stderr
|
||||
report = json.loads(reread.stdout)
|
||||
assert report["summary"]["checksum_matches"] is True
|
||||
assert report["summary"]["checksum_declared"] == report["summary"]["checksum_computed"]
|
||||
assert report["checksum"]["matches"] is True
|
||||
assert report["summary"]["file_size"] == fixtures.pe64.size
|
||||
body = patched.read_bytes()
|
||||
assert body[first:first + 12] == bytes(int(b, 16) for b in replacement.split())
|
||||
assert body[second:second + 12] == fixtures.pe64.data[second:second + 12]
|
||||
|
||||
# 5. nothing outside the site and the checksum field moved.
|
||||
checksum_field = int(fixtures.pe64.meta["checksum_offset"], 16)
|
||||
diff = {i for i, (a, b) in enumerate(zip(fixtures.pe64.data, body)) if a != b}
|
||||
assert diff - {first + 10} <= set(range(checksum_field, checksum_field + 4))
|
||||
assert first + 10 in diff
|
||||
|
||||
|
||||
def test_control_run_that_changes_nothing_leaves_the_file_identical(run_script, fixtures, work,
|
||||
tmp_path):
|
||||
"""The control for the flow above: a patch whose bytes equal the site's own bytes.
|
||||
|
||||
This is the only zero-change run the pipeline has: a spec with an empty patch array is
|
||||
refused as a usage error, so a no-op patch is what "the pipeline ran and changed nothing"
|
||||
looks like. The output must be byte-identical to the input, checksum included.
|
||||
"""
|
||||
out = tmp_path / "noop.exe"
|
||||
before = sha256(work)
|
||||
result = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
|
||||
"--expect", fixtures.pe64.plant, "--bytes", fixtures.pe64.plant,
|
||||
"--out", out, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["checksum_recomputed"] is True
|
||||
assert payload["checksum_after"]["matches"] is True
|
||||
assert payload["output_sha256"] == payload["input_sha256"] == before
|
||||
assert sha256(out) == before
|
||||
assert sha256(work) == before
|
||||
|
||||
|
||||
def test_an_empty_patch_array_is_refused_rather_than_writing_a_copy(run_script, fixtures, work,
|
||||
tmp_path):
|
||||
spec = tmp_path / "empty.json"
|
||||
spec.write_text('{"patches": []}', encoding="utf-8")
|
||||
out = tmp_path / "copy.exe"
|
||||
result = run_script("pepatch.py", work, "--spec", spec, "--out", out)
|
||||
assert result.returncode == 2
|
||||
assert not out.exists()
|
||||
assert sha256(work) == sha256(fixtures.pe64.path)
|
||||
|
||||
|
||||
def test_the_flow_stops_before_writing_when_the_pattern_is_ambiguous(run_script, fixtures, work,
|
||||
tmp_path):
|
||||
out = tmp_path / "never.exe"
|
||||
before = sha256(work)
|
||||
result = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant,
|
||||
"--bytes", "90 " * 12, "--out", out)
|
||||
assert result.returncode == 1
|
||||
assert "matched 2 sites" in result.stderr
|
||||
assert not out.exists()
|
||||
assert sha256(work) == before
|
||||
|
||||
|
||||
def test_dry_run_reports_the_plan_and_writes_nothing(run_script, fixtures, work, tmp_path):
|
||||
out = tmp_path / "would-be.exe"
|
||||
before = sha256(work)
|
||||
result = run_script("pepatch.py", work, "--pattern", fixtures.pe64.plant, "--at-hit", "1",
|
||||
"--expect", fixtures.pe64.plant, "--bytes", "CC " * 12,
|
||||
"--dry-run", "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["dry_run"] is True
|
||||
assert payload["patches_applied"] == 1
|
||||
assert payload["applied"][0]["offset"] == fixtures.pe64.plant_offsets[1]
|
||||
assert not out.exists()
|
||||
assert sha256(work) == before
|
||||
|
||||
|
||||
def test_a_spec_file_can_reach_a_branch_inside_a_longer_pattern(run_script, fixtures, work,
|
||||
tmp_path):
|
||||
"""The pattern-offset flow: anchor on a unique longer sequence, then patch the byte inside."""
|
||||
out = tmp_path / "branch.exe"
|
||||
site = fixtures.pe64.plant_offsets[0]
|
||||
spec = tmp_path / "spec.json"
|
||||
spec.write_text(json.dumps({"patches": [{
|
||||
"name": "gate",
|
||||
"pattern": fixtures.pe64.plant + " CC",
|
||||
"pattern-offset": 10,
|
||||
"expect": "74",
|
||||
"bytes": "EB",
|
||||
"why": "invert the conditional branch, keeping the taken distance",
|
||||
}]}), encoding="utf-8")
|
||||
result = run_script("pepatch.py", work, "--spec", spec, "--out", out, "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
payload = json.loads(result.stdout)
|
||||
assert payload["patches_applied"] == 1
|
||||
record = payload["applied"][0]
|
||||
assert record["offset"] == site + 10
|
||||
assert record["before"] == "74"
|
||||
assert record["after"] == "EB"
|
||||
assert record["length_basis"] == "expect"
|
||||
assert record["why"].startswith("invert")
|
||||
|
||||
reread = run_script("peutil.py", out, "--summary", "--check-sum", "--json")
|
||||
assert reread.returncode == 0
|
||||
assert json.loads(reread.stdout)["checksum"]["matches"] is True
|
||||
assert out.read_bytes()[site + 10] == 0xEB
|
||||
assert out.read_bytes()[site + 11] == 0x05
|
||||
|
||||
|
||||
def test_a_length_change_is_refused_before_anything_is_written(run_script, fixtures, work,
|
||||
tmp_path):
|
||||
out = tmp_path / "resized.exe"
|
||||
result = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
|
||||
"--expect", "48", "--bytes", "CC CC CC", "--out", out)
|
||||
assert result.returncode == 1
|
||||
assert "--allow-resize" in result.stderr
|
||||
assert not out.exists()
|
||||
|
||||
allowed = tmp_path / "resized-allowed.exe"
|
||||
forced = run_script("pepatch.py", work, "--at-offset", hex(fixtures.pe64.plant_offsets[0]),
|
||||
"--expect", "48", "--bytes", "CC CC CC", "--allow-resize",
|
||||
"--out", allowed, "--json")
|
||||
assert forced.returncode == 0, forced.stderr
|
||||
payload = json.loads(forced.stdout)
|
||||
assert payload["structural"] == ["cli"]
|
||||
# One byte replaced by three: the file grows by two, and that is the structural edit the
|
||||
# caller asked for by passing --allow-resize.
|
||||
assert payload["output_size"] == payload["input_size"] + 2
|
||||
|
||||
|
||||
def test_the_patched_output_still_parses_as_an_image(run_script, fixtures, work, tmp_path,
|
||||
hexpat):
|
||||
"""A patch that lands in the file must not damage a single header the reader trusts."""
|
||||
out = tmp_path / "patched.exe"
|
||||
result = run_script("pepatch.py", work, "--at-offset", "0x410", "--expect", "48 8B",
|
||||
"--bytes", "CC 8B", "--out", out)
|
||||
assert result.returncode == 0, result.stderr
|
||||
full = run_script("peutil.py", out, "--summary", "--sections", "--imports", "--exports",
|
||||
"--resources", "--tls", "--debug", "--relocs", "--check-sum", "--json")
|
||||
assert full.returncode == 0, full.stderr
|
||||
report = json.loads(full.stdout)
|
||||
assert report["summary"]["sha256"] == sha256(out)
|
||||
assert report["checksum"]["matches"] is True
|
||||
assert [s["name"] for s in report["summary"]["sections"]] == \
|
||||
[".text", ".rdata", ".data", ".rsrc", ".reloc"]
|
||||
assert report["resources"][0]["path"] == ["VERSION", "1", "1033"]
|
||||
assert report["tls"]["callback_rvas"] == ["0x1000"]
|
||||
assert report["debug"][0]["format"] == "RSDS"
|
||||
assert report["imports"][0]["functions"][0]["name"] == "ImportedFunction"
|
||||
|
||||
|
||||
def test_a_real_platform_image_reproduces_its_declared_checksum(run_script, platform_binary):
|
||||
"""The fixture is synthetic; this is the same claim measured on a real system image."""
|
||||
result = run_script("peutil.py", platform_binary, "--summary", "--check-sum", "--json")
|
||||
assert result.returncode == 0, result.stderr
|
||||
report = json.loads(result.stdout)
|
||||
assert report["summary"]["checksum_matches"] is True
|
||||
assert report["checksum"]["declared"] == report["checksum"]["computed"]
|
||||
assert report["summary"]["file_size"] == platform_binary.stat().st_size
|
||||
assert report["summary"]["sections"], "a platform image must have a section table"
|
||||
@@ -0,0 +1,68 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Run the regression suite with one command, and fail usefully when it cannot run.
|
||||
|
||||
Why this exists next to `python -m pytest`: the interesting failure is not a failing test, it is
|
||||
a suite that cannot start. "python -m pytest" answers a missing pytest with
|
||||
``No module named pytest``, which reads like a bug in the suite rather than a missing
|
||||
prerequisite. This runner states the prerequisite, prints the install line the repository
|
||||
documents, and exits 3 -- the repository's environment-capability failure code -- instead of
|
||||
letting a toolchain gap look like a test result.
|
||||
|
||||
Usage:
|
||||
python tests/run_tests.py # the whole suite
|
||||
python tests/run_tests.py -q -m unit # anything pytest accepts is forwarded
|
||||
|
||||
Exit codes: pytest's own code (0 = passed, 1 = failures, 2 = interrupted, 4 = usage error,
|
||||
5 = nothing collected), or 3 = the environment cannot run the suite at all. Note that pytest
|
||||
also uses 3 for its own internal error; the printed message distinguishes the two.
|
||||
|
||||
Requirements: Python 3.9+ and pytest. No other third-party package, no debugger, no driver, no
|
||||
network and no committed binary sample -- the PE inputs are generated into a temporary directory
|
||||
at test time.
|
||||
"""
|
||||
import importlib.util
|
||||
import os
|
||||
import sys
|
||||
|
||||
MIN_PYTHON = (3, 9)
|
||||
EXIT_ENVIRONMENT = 3
|
||||
INSTALL_INSTRUCTION = "python -m pip install pytest"
|
||||
|
||||
TESTS_DIR = os.path.dirname(os.path.abspath(__file__))
|
||||
REPO_ROOT = os.path.dirname(TESTS_DIR)
|
||||
|
||||
|
||||
def _message(lines):
|
||||
for line in lines:
|
||||
sys.stderr.write("%s\n" % line)
|
||||
|
||||
|
||||
def main(argv):
|
||||
if sys.version_info < MIN_PYTHON:
|
||||
_message([
|
||||
"This suite needs Python %d.%d or newer; this interpreter is %s."
|
||||
% (MIN_PYTHON[0], MIN_PYTHON[1], sys.version.split()[0]),
|
||||
"The tested scripts declare the same requirement in their own docstrings.",
|
||||
"Install a newer interpreter, then run this command again.",
|
||||
])
|
||||
return EXIT_ENVIRONMENT
|
||||
|
||||
if importlib.util.find_spec("pytest") is None:
|
||||
_message([
|
||||
"pytest is not installed, so the regression suite cannot run.",
|
||||
"",
|
||||
" install: %s" % INSTALL_INSTRUCTION,
|
||||
"",
|
||||
"or run the suite after installing it: python tests/run_tests.py",
|
||||
])
|
||||
return EXIT_ENVIRONMENT
|
||||
|
||||
# pytest reads pytest.ini relative to the rootdir, so run from the repository root whatever
|
||||
# directory the caller used.
|
||||
os.chdir(REPO_ROOT)
|
||||
import pytest
|
||||
return pytest.main(list(argv))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main(sys.argv[1:]))
|
||||
@@ -0,0 +1,161 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/aob_scan.py.
|
||||
|
||||
The tool's real output is the *verdict*: a pattern that matches twice is a coin flip, and the
|
||||
whole point is that the caller is told so. The boundary table is asserted at both sides of each
|
||||
threshold, and the pattern-file parser is tested in all four of its accepted shapes.
|
||||
"""
|
||||
import types
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def ns(**kwargs):
|
||||
base = {"patterns": []}
|
||||
base.update(kwargs)
|
||||
return types.SimpleNamespace(**base)
|
||||
|
||||
|
||||
# --- verdict boundaries ------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("count,expected", [
|
||||
(0, "no-match"),
|
||||
(1, "unique"),
|
||||
(2, "ambiguous"),
|
||||
(3, "ambiguous"),
|
||||
(7, "ambiguous"),
|
||||
(8, "pervasive"),
|
||||
(64, "pervasive"),
|
||||
])
|
||||
def test_verdict_boundaries(load_script, count, expected):
|
||||
assert load_script("aob_scan.py").verdict(count, count) == expected
|
||||
|
||||
|
||||
def test_verdict_thresholds_are_the_documented_ones(load_script):
|
||||
aob = load_script("aob_scan.py")
|
||||
assert aob.NOTICEABLE_AT == 8
|
||||
assert aob.AMBIGUOUS_AT == 2
|
||||
assert aob.verdict(aob.NOTICEABLE_AT - 1, 0) == "ambiguous"
|
||||
assert aob.verdict(aob.NOTICEABLE_AT, 0) == "pervasive"
|
||||
|
||||
|
||||
def test_verdict_ignores_the_unused_total_argument(load_script):
|
||||
aob = load_script("aob_scan.py")
|
||||
assert aob.verdict(1, 999) == aob.verdict(1, 0) == "unique"
|
||||
|
||||
|
||||
# --- locate ------------------------------------------------------------------
|
||||
|
||||
def test_locate_reports_offset_rva_and_section(load_script, fixtures):
|
||||
aob = load_script("aob_scan.py")
|
||||
peutil = load_script("peutil.py")
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = peutil.PE(fixtures.pe64.data, None)
|
||||
pat = pepatch.parse_bytes(fixtures.pe64.plant)[0]
|
||||
rows = aob.locate(pe, fixtures.pe64.data, pat)
|
||||
assert [r["offset"] for r in rows] == list(fixtures.pe64.plant_offsets)
|
||||
assert [r["rva_hex"] for r in rows] == [fixtures.pe64.meta["plant"]["first_rva"],
|
||||
fixtures.pe64.meta["plant"]["second_rva"]]
|
||||
assert [r["section"] for r in rows] == [".text", ".text"]
|
||||
assert rows[0]["offset_hex"] == fixtures.pe64.meta["plant"]["first_file_offset"]
|
||||
|
||||
|
||||
def test_locate_section_filter_drops_hits_elsewhere(load_script, fixtures):
|
||||
aob = load_script("aob_scan.py")
|
||||
peutil = load_script("peutil.py")
|
||||
pat = [0x66, 0x69, 0x78, 0x74, 0x75, 0x72, 0x65, 0x2E, 0x70, 0x64, 0x62] # "fixture.pdb"
|
||||
pe = peutil.PE(fixtures.pe64.data, None)
|
||||
assert [r["section"] for r in aob.locate(pe, fixtures.pe64.data, pat)] == [".rdata"]
|
||||
assert aob.locate(pe, fixtures.pe64.data, pat, section_filter=".text") == []
|
||||
assert len(aob.locate(pe, fixtures.pe64.data, pat, section_filter=".rdata")) == 1
|
||||
|
||||
|
||||
def test_locate_context_is_centred_on_the_match(load_script, fixtures):
|
||||
aob = load_script("aob_scan.py")
|
||||
peutil = load_script("peutil.py")
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = peutil.PE(fixtures.pe64.data, None)
|
||||
pat = pepatch.parse_bytes(fixtures.pe64.plant)[0]
|
||||
rows = aob.locate(pe, fixtures.pe64.data, pat, context=8)
|
||||
assert "context" in rows[0]
|
||||
assert rows[0]["context_marks"] == "pre 8 bytes, match 12, post 8"
|
||||
assert len(pepatch.parse_bytes(rows[0]["context"])[0]) == len(pat) + 16
|
||||
|
||||
|
||||
def test_locate_returns_no_rows_when_the_pattern_is_absent(load_script, fixtures):
|
||||
"""A negative outcome: an absent pattern yields an empty list, not a nearest match."""
|
||||
aob = load_script("aob_scan.py")
|
||||
peutil = load_script("peutil.py")
|
||||
pe = peutil.PE(fixtures.bare.data, None)
|
||||
pat = [0x66, 0x69, 0x78, 0x74, 0x75, 0x72, 0x65, 0x2E, 0x70, 0x64, 0x62]
|
||||
assert aob.locate(pe, fixtures.bare.data, pat) == []
|
||||
|
||||
|
||||
# --- load_patterns -----------------------------------------------------------
|
||||
|
||||
def test_load_patterns_accepts_an_inline_pattern(load_script):
|
||||
aob = load_script("aob_scan.py")
|
||||
assert aob.load_patterns(ns(patterns=["48 8B ?? 05"])) == [
|
||||
("inline", [0x48, 0x8B, None, 0x05], "48 8B ?? 05")]
|
||||
|
||||
|
||||
def test_load_patterns_without_patterns_is_empty(load_script):
|
||||
assert load_script("aob_scan.py").load_patterns(ns()) == []
|
||||
assert load_script("aob_scan.py").load_patterns(ns(patterns=[])) == []
|
||||
|
||||
|
||||
def test_load_patterns_parses_a_text_file_with_comments_and_names(load_script, tmp_path):
|
||||
aob = load_script("aob_scan.py")
|
||||
path = tmp_path / "sigs.txt"
|
||||
path.write_text("# a comment\n\n48 8B 05\nuse-site\t90 90 90\n", encoding="utf-8")
|
||||
got = aob.load_patterns(ns(patterns=[str(path)]))
|
||||
assert got == [("sigs.txt:3", [0x48, 0x8B, 0x05], "48 8B 05"),
|
||||
("use-site", [0x90, 0x90, 0x90], "90 90 90")]
|
||||
|
||||
|
||||
def test_load_patterns_parses_a_json_object(load_script, tmp_path):
|
||||
aob = load_script("aob_scan.py")
|
||||
obj = tmp_path / "named.json"
|
||||
obj.write_text('{"patterns": [{"name": "gate", "pattern": "74 05"}, "EB 05"]}',
|
||||
encoding="utf-8")
|
||||
assert aob.load_patterns(ns(patterns=[str(obj)])) == [
|
||||
("gate", [0x74, 0x05], "74 05"),
|
||||
("named.json[1]", [0xEB, 0x05], "EB 05")]
|
||||
|
||||
|
||||
def test_a_bare_json_list_is_accepted(load_script, tmp_path):
|
||||
"""A top-level array is as reasonable a shape as an object with a "patterns" key.
|
||||
|
||||
The reader used to call ``.get`` on whatever ``json.load`` returned, so a top-level array
|
||||
raised AttributeError instead of the SystemExit this module uses for every other bad input.
|
||||
Both shapes are read now, and anything else is refused with a message naming what was found.
|
||||
"""
|
||||
aob = load_script("aob_scan.py")
|
||||
bare = tmp_path / "bare.json"
|
||||
bare.write_text('["CC CC"]', encoding="utf-8")
|
||||
assert aob.load_patterns(ns(patterns=[str(bare)])) == [("bare.json[0]", [0xCC, 0xCC], "CC CC")]
|
||||
|
||||
|
||||
def test_a_json_scalar_is_refused_with_a_message(load_script, tmp_path):
|
||||
"""The shape that is still not a pattern file says so instead of raising."""
|
||||
aob = load_script("aob_scan.py")
|
||||
scalar = tmp_path / "scalar.json"
|
||||
scalar.write_text('42', encoding="utf-8")
|
||||
with pytest.raises(SystemExit) as excinfo:
|
||||
aob.load_patterns(ns(patterns=[str(scalar)]))
|
||||
assert "must hold an array or an object" in str(excinfo.value)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text,expected", [
|
||||
("ZZ QQ", "not a byte"),
|
||||
("?? ??", "empty or all wildcards"),
|
||||
])
|
||||
def test_load_patterns_exits_on_a_pattern_it_cannot_parse(load_script, tmp_path, text,
|
||||
expected):
|
||||
aob = load_script("aob_scan.py")
|
||||
path = tmp_path / "bad.txt"
|
||||
path.write_text(text + "\n", encoding="utf-8")
|
||||
with pytest.raises(SystemExit) as excinfo:
|
||||
aob.load_patterns(ns(patterns=[str(path)]))
|
||||
assert expected in str(excinfo.value)
|
||||
@@ -0,0 +1,509 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/capabilities.py, the capability registry.
|
||||
|
||||
The registry's value is that a verdict is *computed* from a probed closure rather than read off a
|
||||
name, so what is asserted here is the table it computes over: every atom a capability names, every
|
||||
atom's kind and basis, and the transitive closure the resolution walks.
|
||||
|
||||
One part of the table is not well formed, and the tests pin the exact extent rather than asserting
|
||||
the invariant that fails. `Probe.atom` answers "atom is not defined" for an id with no `ATOMS` row,
|
||||
so a capability naming a *capability* in `required` or `any_of` has an atom nothing can ever
|
||||
satisfy. Two rows do that today (`dump_capture`, `hypervisor_observe`); they are named in
|
||||
`UNTABLED_REFERENCES` so a third one fails the suite instead of arriving silently. Reported as a
|
||||
defect in the digest: these tests do not repair the script.
|
||||
|
||||
`declared_capability` gets its own group, because the convention it reads is documented inside this
|
||||
file's own docstring. A looser pattern once read that documentation as a declaration and reported
|
||||
the registry itself as declaring `capability: <capability_id>[, <capability_id>]`; the template is
|
||||
pinned as *not* a declaration below.
|
||||
"""
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
# Measured, not assumed: the `required` / `any_of` entries that name a capability id instead of an
|
||||
# atom id. Both are answered by `Probe.atom` with the literal detail "atom is not defined", which is
|
||||
# pinned separately -- the mechanism is what makes the reference unrepairable from outside the file.
|
||||
UNTABLED_REFERENCES = frozenset({
|
||||
("dump_capture", "any_of", "commandline_debug"),
|
||||
("hypervisor_observe", "required", "kernel_driver_load"),
|
||||
})
|
||||
|
||||
# The kinds `Probe.atom` dispatches on, and the field each one is probed through.
|
||||
PROBE_HANDLED_KINDS = {
|
||||
"python": None,
|
||||
"module": "module",
|
||||
"tool": "tool",
|
||||
"file": "script",
|
||||
"host": "probe",
|
||||
"artifact": None,
|
||||
}
|
||||
|
||||
# The probe names `_host_flag` understands. Not decoration: a `host` atom whose probe name is not in
|
||||
# this set falls through to `None`, which reads as "not available on this host" forever.
|
||||
HOST_PROBE_NAMES = frozenset({
|
||||
"elevation", "hypervisor", "intel_pt", "breakpoints", "test_signing", "etw",
|
||||
})
|
||||
|
||||
# Both are the documented strength labels of the *measurement*, not of the route.
|
||||
BASIS_LABELS = frozenset({"observed", "unverified"})
|
||||
|
||||
|
||||
def probe(caps):
|
||||
"""A fresh probe with no extra tool directories, so PATH alone decides."""
|
||||
return caps.Probe(tool_dirs=[])
|
||||
|
||||
|
||||
def untabled_references(caps):
|
||||
out = set()
|
||||
for capability_id, spec in caps.CAPABILITIES.items():
|
||||
for key in ("required", "any_of"):
|
||||
for atom in spec.get(key, []):
|
||||
if atom not in caps.ATOMS:
|
||||
out.add((capability_id, key, atom))
|
||||
return out
|
||||
|
||||
|
||||
def reference_closure(caps, capability_id):
|
||||
"""An independent breadth-first walk to compare the module's ordering against."""
|
||||
out, queue = [], [capability_id]
|
||||
while queue:
|
||||
current = queue.pop(0)
|
||||
if current in out:
|
||||
continue
|
||||
out.append(current)
|
||||
queue.extend(caps.CAPABILITIES.get(current, {}).get("depends_on", []))
|
||||
return out
|
||||
|
||||
|
||||
# --- the atom table ----------------------------------------------------------
|
||||
|
||||
def test_the_registry_is_not_empty(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert len(caps.ATOMS) > 1
|
||||
assert len(caps.CAPABILITIES) > 1
|
||||
|
||||
|
||||
def test_every_capability_row_carries_a_name_and_a_required_list(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for capability_id, spec in caps.CAPABILITIES.items():
|
||||
assert spec.get("name"), "%s has no name" % capability_id
|
||||
required = spec.get("required")
|
||||
assert isinstance(required, list), "%s has no required list" % capability_id
|
||||
# An empty `required` is legitimate when the row's prerequisites are other capabilities:
|
||||
# `hypervisor_observe` needs no atom of its own, it needs the capability below it. What is
|
||||
# not legitimate is a row that names nothing at all, because then no probe can decide it.
|
||||
assert required or spec.get("any_of") or spec.get("depends_on"), \
|
||||
"%s names nothing at all" % capability_id
|
||||
|
||||
|
||||
def test_capability_rows_use_only_the_documented_keys(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
allowed = {"name", "required", "any_of", "depends_on", "scripts", "note"}
|
||||
for capability_id, spec in caps.CAPABILITIES.items():
|
||||
assert set(spec) <= allowed, "%s carries %r" % (capability_id, sorted(set(spec) - allowed))
|
||||
|
||||
|
||||
def test_no_unrecorded_atom_reference_is_outside_the_atom_table(load_script):
|
||||
"""Every name in a `required` or `any_of` list has a row in the atom table.
|
||||
|
||||
This was false for two entries: a **capability** id had been written into an atom slot, so the
|
||||
probe answered `atom is not defined` and the branch could never be satisfied -- and one of them,
|
||||
`hypervisor_observe`, was therefore permanently blocked with no next action to print. A slot
|
||||
whose vocabulary is wrong is silent, which is why it is asserted as a property now rather than
|
||||
as the measured extent of the breakage.
|
||||
"""
|
||||
caps = load_script("capabilities.py")
|
||||
assert untabled_references(caps) == set()
|
||||
|
||||
|
||||
def test_a_capability_named_where_an_atom_belongs_is_answered_as_undefined(load_script):
|
||||
"""The mechanism behind the pinned deviation, so the two rows are not merely suspicious names."""
|
||||
caps = load_script("capabilities.py")
|
||||
for _capability_id, _key, named in sorted(UNTABLED_REFERENCES):
|
||||
assert named in caps.CAPABILITIES, "%s should be a capability id" % named
|
||||
result = probe(caps).atom(named)
|
||||
assert result["ok"] is False
|
||||
assert result["detail"] == "atom is not defined"
|
||||
|
||||
|
||||
def test_dependencies_name_capabilities_rather_than_atoms(load_script):
|
||||
"""`depends_on` is a capability edge. An atom id here would be silently unresolvable."""
|
||||
caps = load_script("capabilities.py")
|
||||
for capability_id, spec in caps.CAPABILITIES.items():
|
||||
for dep in spec.get("depends_on", []):
|
||||
assert dep in caps.CAPABILITIES, \
|
||||
"%s depends on %r, which is not a capability" % (capability_id, dep)
|
||||
assert dep not in caps.ATOMS, \
|
||||
"%s depends on %r, which is an atom: closure() would treat it as a capability" \
|
||||
% (capability_id, dep)
|
||||
assert dep != capability_id, "%s depends on itself" % capability_id
|
||||
|
||||
|
||||
def test_the_script_map_points_only_at_capabilities_that_exist(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.SCRIPT_CAPABILITIES
|
||||
for script, ids in caps.SCRIPT_CAPABILITIES.items():
|
||||
assert ids, "%s is mapped to an empty list, which reads as a claim" % script
|
||||
for capability_id in ids:
|
||||
assert capability_id in caps.CAPABILITIES, \
|
||||
"%s is mapped to unknown capability %r" % (script, capability_id)
|
||||
|
||||
|
||||
def test_the_dependency_graph_is_acyclic(load_script):
|
||||
"""A cycle would make `resolve` recurse forever rather than report a blocked capability."""
|
||||
caps = load_script("capabilities.py")
|
||||
state = {}
|
||||
|
||||
def walk(capability_id):
|
||||
stack = state.get(capability_id, 0)
|
||||
assert stack != 1, "dependency cycle through %s" % capability_id
|
||||
if stack == 2:
|
||||
return
|
||||
state[capability_id] = 1
|
||||
for dep in caps.CAPABILITIES.get(capability_id, {}).get("depends_on", []):
|
||||
walk(dep)
|
||||
state[capability_id] = 2
|
||||
|
||||
for capability_id in caps.CAPABILITIES:
|
||||
walk(capability_id)
|
||||
|
||||
|
||||
def test_every_atom_carries_the_basis_label_the_registry_documents(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for atom_id, spec in caps.ATOMS.items():
|
||||
assert spec.get("basis") in BASIS_LABELS, \
|
||||
"%s has basis %r, which is neither %s" % (atom_id, spec.get("basis"),
|
||||
" nor ".join(sorted(BASIS_LABELS)))
|
||||
|
||||
|
||||
def test_every_atom_kind_is_one_the_probe_handles(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for atom_id, spec in caps.ATOMS.items():
|
||||
assert spec.get("kind") in PROBE_HANDLED_KINDS, \
|
||||
"%s has kind %r, which Probe.atom does not dispatch on" % (atom_id, spec.get("kind"))
|
||||
|
||||
|
||||
def test_every_atom_carries_the_field_its_kind_is_probed_through(load_script):
|
||||
"""`Probe.atom` subscripts these keys, so a missing one is a KeyError, not a verdict."""
|
||||
caps = load_script("capabilities.py")
|
||||
for atom_id, spec in caps.ATOMS.items():
|
||||
field = PROBE_HANDLED_KINDS[spec["kind"]]
|
||||
if field is not None:
|
||||
assert spec.get(field), "%s (%s) has no %r" % (atom_id, spec["kind"], field)
|
||||
|
||||
|
||||
def test_every_host_atom_names_a_probe_the_host_reader_implements(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for atom_id, spec in caps.ATOMS.items():
|
||||
if spec["kind"] == "host":
|
||||
assert spec["probe"] in HOST_PROBE_NAMES, \
|
||||
"%s probes %r, which _host_flag does not answer (it would be false forever)" \
|
||||
% (atom_id, spec["probe"])
|
||||
|
||||
|
||||
def test_atom_ids_are_prefixed_with_their_kind(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for atom_id, spec in caps.ATOMS.items():
|
||||
if spec["kind"] == "python":
|
||||
continue
|
||||
assert atom_id.startswith(spec["kind"] + ":"), \
|
||||
"%s does not name its kind (%s)" % (atom_id, spec["kind"])
|
||||
|
||||
|
||||
def test_every_atom_answers_something_other_than_undefined(load_script):
|
||||
"""Each row is reachable through the probe: no row falls through to the undefined branch."""
|
||||
caps = load_script("capabilities.py")
|
||||
p = probe(caps)
|
||||
for atom_id in caps.ATOMS:
|
||||
result = p.atom(atom_id)
|
||||
assert isinstance(result.get("ok"), bool), "%s answered %r" % (atom_id, result)
|
||||
assert result["detail"] != "atom is not defined", \
|
||||
"%s has a row but the probe still calls it undefined" % atom_id
|
||||
|
||||
|
||||
def test_which_falls_back_to_an_extra_directory(load_script, tmp_path):
|
||||
"""A tool that is installed off PATH is not a missing tool, which is the whole point of the
|
||||
second search path. The file is never executed -- only its presence is checked."""
|
||||
caps = load_script("capabilities.py")
|
||||
tools = tmp_path / "unpacked"
|
||||
tools.mkdir()
|
||||
(tools / "ghidra_off_path_probe").write_text("", encoding="utf-8")
|
||||
p = caps.Probe(tool_dirs=[str(tools)])
|
||||
found, via = p.which("ghidra_off_path_probe")
|
||||
assert found == str(tools / "ghidra_off_path_probe")
|
||||
assert via == "EXEREV_TOOLS"
|
||||
assert p.which("no_such_tool_anywhere_xyz") == (None, None)
|
||||
|
||||
|
||||
def test_extra_tool_dirs_reads_both_environment_variable_names(load_script, monkeypatch, tmp_path):
|
||||
caps = load_script("capabilities.py")
|
||||
monkeypatch.setenv("EXEREV_TOOLS", str(tmp_path / "one"))
|
||||
monkeypatch.setenv("EXE_REVERSE_TOOLS", str(tmp_path / "two") + os.pathsep +
|
||||
str(tmp_path / "three"))
|
||||
assert caps.extra_tool_dirs() == [str(tmp_path / "one"), str(tmp_path / "two"),
|
||||
str(tmp_path / "three")]
|
||||
monkeypatch.delenv("EXEREV_TOOLS")
|
||||
monkeypatch.delenv("EXE_REVERSE_TOOLS")
|
||||
assert caps.extra_tool_dirs() == []
|
||||
|
||||
|
||||
# --- closure -----------------------------------------------------------------
|
||||
|
||||
def test_closure_of_a_capability_with_no_dependencies_is_its_only_member(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.closure("pe_structure_read") == ["pe_structure_read"]
|
||||
|
||||
|
||||
def test_closure_matches_an_independent_walk_over_the_real_table(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for capability_id in caps.CAPABILITIES:
|
||||
expected = reference_closure(caps, capability_id)
|
||||
assert caps.closure(capability_id) == expected, capability_id
|
||||
|
||||
|
||||
def test_closure_terminates_and_lists_each_capability_once(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
for capability_id in caps.CAPABILITIES:
|
||||
chain = caps.closure(capability_id)
|
||||
assert chain[0] == capability_id
|
||||
assert len(chain) == len(set(chain)), "%s repeats a member: %r" % (capability_id, chain)
|
||||
assert set(chain) <= set(caps.CAPABILITIES), \
|
||||
"%s reaches something that is not a capability: %r" % (capability_id, chain)
|
||||
|
||||
|
||||
def test_closure_follows_more_than_one_hop_and_deduplicates_a_diamond(load_script, monkeypatch):
|
||||
"""Synthetic: the real table is only one hop deep, so a diamond is built to reach the walk."""
|
||||
caps = load_script("capabilities.py")
|
||||
for name, deps in (("synthetic_top", ["synthetic_left", "synthetic_right"]),
|
||||
("synthetic_left", ["synthetic_bottom"]),
|
||||
("synthetic_right", ["synthetic_bottom"]),
|
||||
("synthetic_bottom", [])):
|
||||
monkeypatch.setitem(caps.CAPABILITIES, name,
|
||||
{"name": name, "required": ["python310"], "depends_on": deps})
|
||||
assert caps.closure("synthetic_top") == ["synthetic_top", "synthetic_left",
|
||||
"synthetic_right", "synthetic_bottom"]
|
||||
|
||||
|
||||
def test_closure_of_an_id_that_is_not_a_capability_is_the_id_alone(load_script):
|
||||
"""Documented behaviour: no error, no members. Pinned so a caller can see what it gets."""
|
||||
assert load_script("capabilities.py").closure("no_such_capability") == ["no_such_capability"]
|
||||
|
||||
|
||||
# --- resolve -----------------------------------------------------------------
|
||||
|
||||
def synthetic_capability(monkeypatch, caps, capability_id, **spec):
|
||||
body = {"name": "Synthetic " + capability_id, "required": ["python310"]}
|
||||
body.update(spec)
|
||||
monkeypatch.setitem(caps.CAPABILITIES, capability_id, body)
|
||||
|
||||
|
||||
def test_resolve_reports_blocked_and_names_the_missing_atom(load_script, monkeypatch):
|
||||
caps = load_script("capabilities.py")
|
||||
synthetic_capability(monkeypatch, caps, "synthetic_missing",
|
||||
required=["python310", "artifact:sample"])
|
||||
result = caps.resolve("synthetic_missing", probe(caps))
|
||||
assert result["state"] == "blocked"
|
||||
assert [atom for atom, _detail in result["missing"]] == ["artifact:sample"]
|
||||
|
||||
|
||||
def test_resolve_of_a_real_capability_names_the_artifact_it_cannot_probe(load_script):
|
||||
"""`artifact:sample` is operator-supplied by design, so this verdict is stable on any host."""
|
||||
caps = load_script("capabilities.py")
|
||||
result = caps.resolve("commandline_debug", probe(caps))
|
||||
assert result["state"] == "blocked"
|
||||
assert "artifact:sample" in [atom for atom, _detail in result["missing"]]
|
||||
|
||||
|
||||
def test_resolve_reports_a_blocked_dependency_by_name(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
result = caps.resolve("hardware_breakpoint", probe(caps))
|
||||
assert result["state"] == "blocked"
|
||||
assert "commandline_debug" in [dep for dep, _state in result["blocked_deps"]]
|
||||
|
||||
|
||||
def test_resolve_on_a_file_atom_follows_the_disk(load_script):
|
||||
"""A shipped script is present, and the verdict says which path was looked at."""
|
||||
caps = load_script("capabilities.py")
|
||||
result = caps.resolve("pe_structure_read", probe(caps))
|
||||
assert result["state"] == "ok"
|
||||
assert result["missing"] == []
|
||||
file_atom = probe(caps).atom("file:peutil.py")
|
||||
assert file_atom["ok"] is True
|
||||
assert file_atom["detail"] == os.path.join(caps.HERE, "peutil.py")
|
||||
|
||||
|
||||
def test_resolve_on_a_fabricated_file_atom_is_missing_and_names_its_path(load_script,
|
||||
monkeypatch):
|
||||
caps = load_script("capabilities.py")
|
||||
monkeypatch.setitem(caps.ATOMS, "file:no_such_script_xyz.py",
|
||||
{"name": "Fabricated", "kind": "file", "basis": "observed",
|
||||
"script": "no_such_script_xyz.py", "est_minutes": 0})
|
||||
synthetic_capability(monkeypatch, caps, "synthetic_file",
|
||||
required=["file:no_such_script_xyz.py"])
|
||||
result = caps.resolve("synthetic_file", probe(caps))
|
||||
assert result["state"] == "blocked"
|
||||
atom, detail = result["missing"][0]
|
||||
assert atom == "file:no_such_script_xyz.py"
|
||||
assert detail.endswith("no_such_script_xyz.py")
|
||||
|
||||
|
||||
def test_resolve_reports_the_alternatives_it_found(load_script, monkeypatch):
|
||||
caps = load_script("capabilities.py")
|
||||
synthetic_capability(monkeypatch, caps, "synthetic_any",
|
||||
required=[], any_of=["file:peutil.py", "artifact:sample"])
|
||||
result = caps.resolve("synthetic_any", probe(caps))
|
||||
assert result["alternatives_present"] == ["file:peutil.py"]
|
||||
assert result["state"] == "ok"
|
||||
assert result["missing"] == []
|
||||
|
||||
|
||||
def test_resolve_of_an_unknown_capability_is_none(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.resolve("no_such_capability", probe(caps)) is None
|
||||
|
||||
|
||||
def test_resolve_all_covers_every_capability_and_agrees_with_resolve(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
p = probe(caps)
|
||||
rows = caps.resolve_all(p)
|
||||
assert [row["id"] for row in rows] == list(caps.CAPABILITIES)
|
||||
for row in rows:
|
||||
assert row == caps.resolve(row["id"], p)
|
||||
|
||||
|
||||
def test_verdict_is_ok_only_when_every_row_is_ok(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.verdict([]) == ("ok", 0)
|
||||
assert caps.verdict([{"state": "ok"}, {"state": "ok"}]) == ("ok", 0)
|
||||
assert caps.verdict([{"state": "ok"}, {"state": "blocked"}]) == ("blocked", 3)
|
||||
|
||||
|
||||
def test_human_report_marks_every_row_and_explains_a_blocked_one(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
rows = [
|
||||
{"id": "ok_one", "name": "Okay", "state": "ok", "missing": [], "blocked_deps": []},
|
||||
{"id": "blocked_one", "name": "Blocked", "state": "blocked",
|
||||
"missing": [("artifact:sample", "operator supplied; not detectable from here")],
|
||||
"blocked_deps": [("other_cap", "blocked")]},
|
||||
]
|
||||
quiet = caps.human_report(rows, verbose=False)
|
||||
assert any("ok_one" in line and line.startswith(" ok ") for line in quiet)
|
||||
assert any("blocked_one" in line and line.startswith(" BLK") for line in quiet)
|
||||
loud = caps.human_report(rows, verbose=True)
|
||||
assert any("missing atom : artifact:sample" in line for line in loud)
|
||||
assert any("blocked by : other_cap (blocked)" in line for line in loud)
|
||||
|
||||
|
||||
# --- declaration parsing -----------------------------------------------------
|
||||
|
||||
def write_script(tmp_path, name, body):
|
||||
path = tmp_path / name
|
||||
path.write_text(body, encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def test_declared_capability_reads_a_single_id(load_script, tmp_path):
|
||||
caps = load_script("capabilities.py")
|
||||
path = write_script(tmp_path, "one.py",
|
||||
'"""capability: byte_patch\n\nA script that declares its capability.\n"""\n')
|
||||
assert caps.declared_capability(str(path)) == ["byte_patch"]
|
||||
|
||||
|
||||
def test_declared_capability_reads_a_comma_separated_list(load_script, tmp_path):
|
||||
caps = load_script("capabilities.py")
|
||||
path = write_script(tmp_path, "two.py",
|
||||
'"""capability: byte_patch, pe_structure_read\n\nTwo of them.\n"""\n')
|
||||
assert caps.declared_capability(str(path)) == ["byte_patch", "pe_structure_read"]
|
||||
|
||||
|
||||
def test_declared_capability_rejects_the_documentation_template(load_script, tmp_path):
|
||||
"""The exact line this registry's own docstring uses to describe the convention.
|
||||
|
||||
It was once read as a declaration, which reported the registry as declaring a capability named
|
||||
`<capability_id>`. A template is not a declaration, and the pin is on that exact string.
|
||||
"""
|
||||
caps = load_script("capabilities.py")
|
||||
template = " capability: <capability_id>[, <capability_id>]"
|
||||
path = write_script(tmp_path, "template.py", '"""%s\n\nProse.\n"""\n' % template)
|
||||
assert caps.declared_capability(str(path)) == []
|
||||
|
||||
|
||||
def test_the_template_really_does_live_in_the_registrys_own_docstring(load_script):
|
||||
"""The pin above is only meaningful if the string it names is really in this file."""
|
||||
caps = load_script("capabilities.py")
|
||||
source = open(os.path.join(caps.HERE, "capabilities.py"), encoding="utf-8").read()
|
||||
assert "capability: <capability_id>[, <capability_id>]" in source
|
||||
assert caps.declared_capability(os.path.join(caps.HERE, "capabilities.py")) == []
|
||||
|
||||
|
||||
def test_declared_capability_reads_a_real_declaring_script(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.declared_capability(os.path.join(caps.HERE, "dart_snapshot_strings.py")) == \
|
||||
["static_disassembly"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,body,expected", [
|
||||
("prose.py", '"""No declaration here, only prose.\n\nMore prose.\n"""\n', []),
|
||||
("no-docstring.py", "import os\n", []),
|
||||
("template-only.py", '"""\ncapability: <capability_id>\n"""\n', []),
|
||||
("not-a-list.py", '"""capability: one two\n"""\n', []),
|
||||
("trailing-junk.py", '"""capability: one # comment\n"""\n', []),
|
||||
("hyphenated-id.py", '"""capability: byte-patch\n"""\n', []),
|
||||
("space-before-colon.py", '"""capability : byte_patch\n"""\n', []),
|
||||
("list-with-a-hyphen.py", '"""capability: byte_patch, pe-structure\n"""\n', []),
|
||||
("trailing-space-and-comma.py", '"""capability: byte_patch,\n"""\n', []),
|
||||
])
|
||||
def test_declared_capability_refuses_a_shape_that_is_not_a_declaration(load_script, tmp_path,
|
||||
name, body, expected):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.declared_capability(str(write_script(tmp_path, name, body))) == expected
|
||||
|
||||
|
||||
def test_declared_capability_matches_case_insensitively(load_script, tmp_path):
|
||||
"""Pinned, because it is wider than the ids it can name: the pattern is compiled `(?i)`, so a
|
||||
mixed-case declaration is returned verbatim and then resolves to nothing -- every id in both
|
||||
tables is lowercase. Recorded as an observation rather than asserted as desirable."""
|
||||
caps = load_script("capabilities.py")
|
||||
path = write_script(tmp_path, "mixed-case.py", '"""capability: BytePatch\n"""\n')
|
||||
assert caps.declared_capability(str(path)) == ["BytePatch"]
|
||||
assert "BytePatch" not in caps.CAPABILITIES
|
||||
|
||||
|
||||
def test_declared_capability_is_silent_about_a_file_it_cannot_read(load_script, tmp_path):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.declared_capability(str(tmp_path / "absent.py")) == []
|
||||
broken = write_script(tmp_path, "broken.py", '"""capability: byte_patch\n"""\nif True print)\n')
|
||||
assert caps.declared_capability(str(broken)) == []
|
||||
|
||||
|
||||
# --- script -> capability ----------------------------------------------------
|
||||
|
||||
def test_capabilities_for_script_reports_a_declaration(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
path = os.path.join(caps.HERE, "dart_snapshot_strings.py")
|
||||
assert caps.capabilities_for_script("dart_snapshot_strings.py", path) == \
|
||||
(["static_disassembly"], "declared")
|
||||
|
||||
|
||||
def test_capabilities_for_script_falls_back_to_the_map(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
assert caps.capabilities_for_script("aob_scan.py") == (["aob_locate"], "mapped")
|
||||
|
||||
|
||||
def test_capabilities_for_script_prefers_the_declaration_over_the_map(load_script, tmp_path):
|
||||
"""A script may not be silently reclassified by the map once it declares its own capability."""
|
||||
caps = load_script("capabilities.py")
|
||||
path = write_script(tmp_path, "aob_scan.py",
|
||||
'"""capability: pe_structure_read\n\nDeclares what it is.\n"""\n')
|
||||
assert caps.capabilities_for_script("aob_scan.py", str(path)) == \
|
||||
(["pe_structure_read"], "declared")
|
||||
|
||||
|
||||
def test_a_script_in_neither_table_is_unknown_rather_than_empty_and_claimed(load_script, tmp_path):
|
||||
caps = load_script("capabilities.py")
|
||||
path = write_script(tmp_path, "stranger.py", '"""Nothing is claimed here.\n"""\n')
|
||||
ids, source = caps.capabilities_for_script("stranger.py", str(path))
|
||||
assert (ids, source) == ([], "unknown")
|
||||
assert source != "mapped", "an unknown script must not be reported as mapped to nothing"
|
||||
@@ -0,0 +1,226 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/dart_snapshot_strings.py, the Dart AOT string reader.
|
||||
|
||||
The input is an ELF shared object and no sample may be committed, so the container is generated by
|
||||
`conftest.build_synthetic_elf`: a valid ELF64 header, a `.text` of sixteen bytes, a `.rodata` with
|
||||
one one-byte (ASCII) literal and one two-byte (UTF-16LE) literal at known offsets, and a
|
||||
section-name string table. Offsets are read from the fixture's own record of where it put each
|
||||
literal, never hardcoded here.
|
||||
|
||||
What the synthetic image cannot do is be a real snapshot. Literal recovery is what these tests hold:
|
||||
a one-byte and a two-byte literal at known offsets, a case-insensitive filter, and an offset probe
|
||||
that resolves anywhere a literal occupies. The instruction-level observations this file once carried
|
||||
are **withdrawn** -- a byte scan cannot read this format, and the reference records why -- so nothing
|
||||
here asserts anything about how a reference is encoded.
|
||||
"""
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
ONE_BYTE = "one-byte"
|
||||
TWO_BYTE = "two-byte"
|
||||
|
||||
# The two idioms the observer counts, assembled here so the positive cases are built rather than
|
||||
# asserted from a table. `49 8B /r` is a load through r15 -- Dart's pool pointer; `48 8B 05` is a
|
||||
# RIP-relative load, which the docstring records as near-absent in a real image.
|
||||
POOL_LOAD = bytes((0x49, 0x8B, 0x87, 0x10, 0x00, 0x00, 0x00))
|
||||
FILLER = bytes((0x90,)) * 8
|
||||
|
||||
|
||||
def text_section(data, vaddr=0x1000, offset=0):
|
||||
return {"offset": offset, "vaddr": vaddr, "size": len(data)}
|
||||
|
||||
|
||||
def hex_bytes(*values):
|
||||
return bytes(values)
|
||||
|
||||
|
||||
# --- the container -----------------------------------------------------------
|
||||
|
||||
def test_the_fixture_is_an_elf_shared_object_the_reader_accepts(load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
elf = dss.Elf(synthetic_elf.data, str(synthetic_elf.path))
|
||||
assert elf.machine == 0x3E
|
||||
assert elf.etype == 3
|
||||
described = elf.describe()
|
||||
assert described["machine"] == "x86-64"
|
||||
assert described["type"] == "shared-object"
|
||||
assert [s["name"] for s in described["sections"]] == [".text", ".rodata", ".shstrtab"]
|
||||
|
||||
|
||||
def test_the_fixture_is_the_size_the_builder_promised(synthetic_elf):
|
||||
assert len(synthetic_elf.data) == synthetic_elf.size
|
||||
assert synthetic_elf.size < 0x1000, "a virtual address above this cannot be a file offset"
|
||||
|
||||
|
||||
def test_sections_are_found_by_name(load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
elf = dss.Elf(synthetic_elf.data)
|
||||
text = elf.section(".text")
|
||||
rodata = elf.section(".rodata")
|
||||
assert text["offset"] == synthetic_elf.text["offset"]
|
||||
assert text["vaddr"] == synthetic_elf.text["vaddr"]
|
||||
assert text["size"] == synthetic_elf.text["size"]
|
||||
assert rodata["offset"] == synthetic_elf.rodata["offset"]
|
||||
assert rodata["vaddr"] == synthetic_elf.rodata["vaddr"]
|
||||
assert rodata["size"] == synthetic_elf.rodata["size"]
|
||||
assert elf.section(".absent") is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("mutation,message", [
|
||||
(lambda d: bytearray(b"not an elf image at all" + bytes(64)), "not an ELF image"),
|
||||
(lambda d: bytearray(b"\x7fELF" + d[4:]), None), # a valid image
|
||||
(lambda d: bytearray(d[:4] + bytes((1,)) + d[5:]), "not a 64-bit ELF"),
|
||||
(lambda d: bytearray(d[:5] + bytes((2,)) + d[6:]), "not little-endian"),
|
||||
(lambda d: bytearray(d[:0x30]), "not an ELF image"), # truncated header
|
||||
])
|
||||
def test_the_reader_refuses_a_container_it_cannot_read(load_script, synthetic_elf, mutation,
|
||||
message):
|
||||
"""A refusal is a ValueError with a reason; the CLI turns it into the documented exit code."""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
data = mutation(bytearray(synthetic_elf.data))
|
||||
if message is None:
|
||||
assert dss.Elf(bytes(data)).etype == 3
|
||||
return
|
||||
with pytest.raises(ValueError) as excinfo:
|
||||
dss.Elf(bytes(data))
|
||||
assert message in str(excinfo.value)
|
||||
|
||||
|
||||
def test_the_address_maps_round_trip(load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
elf = dss.Elf(synthetic_elf.data)
|
||||
for entry in synthetic_elf.literals:
|
||||
assert elf.vaddr_to_offset(entry["vaddr"]) == entry["offset"]
|
||||
assert elf.offset_to_vaddr(entry["offset"]) == entry["vaddr"]
|
||||
assert elf.vaddr_to_offset(0xDEADBEEF) is None
|
||||
assert elf.offset_to_vaddr(0xFFFFFF) is None
|
||||
|
||||
|
||||
# --- literal recovery --------------------------------------------------------
|
||||
|
||||
def test_every_literal_in_the_fixture_is_recovered_at_the_offset_it_was_placed(
|
||||
load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
elf = dss.Elf(synthetic_elf.data)
|
||||
recovered = dss.recover_strings(synthetic_elf.data, elf.section(".rodata"))
|
||||
assert [(off, va, text, enc) for off, va, text, enc in recovered] == [
|
||||
(entry["offset"], entry["vaddr"], entry["text"], entry["encoding"])
|
||||
for entry in synthetic_elf.literals]
|
||||
|
||||
|
||||
def test_both_encodings_are_recovered(load_script, synthetic_elf):
|
||||
"""A keyword search that only reads ASCII misses half of a localised application's literals."""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
elf = dss.Elf(synthetic_elf.data)
|
||||
recovered = dss.recover_strings(synthetic_elf.data, elf.section(".rodata"))
|
||||
by_encoding = {}
|
||||
for _off, _va, text, encoding in recovered:
|
||||
by_encoding.setdefault(encoding, set()).add(text)
|
||||
assert by_encoding[ONE_BYTE] == {"premium_license", "trial_mode"}
|
||||
assert by_encoding[TWO_BYTE] == {"vip_pass"}
|
||||
|
||||
|
||||
def test_a_two_byte_literal_reports_byte_offsets_and_character_counts(load_script, synthetic_elf):
|
||||
"""Pinned, because the two units differ by a factor of two for this encoding.
|
||||
|
||||
The offsets are bytes; the text is characters. `--at` filters on `offset <= at < offset +
|
||||
len(text)`, so for a two-byte literal it accepts only the first half of the bytes the literal
|
||||
occupies. Measured here as a unit fact; the CLI layer pins the effect.
|
||||
"""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
entry = synthetic_elf.literal("vip_pass")
|
||||
recovered = dss.recover_strings(synthetic_elf.data,
|
||||
dss.Elf(synthetic_elf.data).section(".rodata"))
|
||||
offset, vaddr, text, encoding = [row for row in recovered if row[0] == entry["offset"]][0]
|
||||
assert (offset, vaddr, encoding) == (entry["offset"], entry["vaddr"], TWO_BYTE)
|
||||
assert text == "vip_pass"
|
||||
assert len(text) * 2 == 16, "sixteen bytes on disk, eight characters reported"
|
||||
|
||||
|
||||
def test_a_two_byte_run_does_not_glue_the_tail_of_one_literal_to_the_next(load_script):
|
||||
"""A run is reported from its own first byte, never from the tail of the literal before it.
|
||||
|
||||
This used to be a false positive. The scan accepted *any* printable byte followed by a NUL --
|
||||
which is exactly the shape of the last character of an ASCII literal plus its alignment byte --
|
||||
so a two-byte literal beginning right after one was reported as a literal that does not exist,
|
||||
at an offset inside a different one. Both directions are asserted here, because the cheap fix
|
||||
for the glue error (skip anything preceded by a character-shaped pair) reintroduces the opposite
|
||||
one: the real run then starts with its own bytes looking like an interior position.
|
||||
"""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
ascii_part = b"odd_phase_literal\x00"
|
||||
assert len(ascii_part) % 2 == 0, "the two-byte run needs to start at an even offset"
|
||||
data = ascii_part + "vip_pass".encode("utf-16-le") + b"\x00\x00"
|
||||
recovered = dss.recover_strings(data, {"offset": 0, "vaddr": 0x1000, "size": len(data)})
|
||||
texts = [row[2] for row in recovered]
|
||||
assert "odd_phase_literal" in texts
|
||||
assert "vip_pass" in texts, texts
|
||||
assert "lvip_pass" not in texts, "the tail of the ASCII literal was glued onto the next run"
|
||||
assert texts.count("vip_pass") == 1
|
||||
|
||||
|
||||
def test_a_two_byte_literal_landing_at_an_odd_offset_is_recovered(load_script):
|
||||
"""A two-byte run at an odd offset is found, and the run before it does not manufacture one.
|
||||
|
||||
Two errors lived here. The scan stepped through the two-byte form from the section base, so a
|
||||
run shifted to an odd offset -- which one odd-length literal is enough to do -- was never
|
||||
examined and was missed whole. Advancing one byte at a time fixes that, and the second half of
|
||||
this test guards the error it introduces: the tail of an even-length one-byte literal is a
|
||||
printable byte followed by alignment, which reads as the start of a two-byte run and yields a
|
||||
literal that does not exist.
|
||||
"""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
ascii_part = b"even_phase_liter\x00"
|
||||
assert len(ascii_part) % 2 == 1, "the two-byte run needs to start at an odd offset"
|
||||
data = ascii_part + "vip_pass".encode("utf-16-le") + b"\x00\x00"
|
||||
recovered = dss.recover_strings(data, {"offset": 0, "vaddr": 0x1000, "size": len(data)})
|
||||
texts = [row[2] for row in recovered]
|
||||
assert "vip_pass" in texts, texts
|
||||
assert "even_phase_liter" in texts, texts
|
||||
|
||||
# No phantom literal: a run is reported once, from its own first byte.
|
||||
assert texts.count("vip_pass") == 1
|
||||
two_byte = [row for row in recovered if row[3] == "two-byte"]
|
||||
assert len(two_byte) == 1, two_byte
|
||||
|
||||
# An even-length one-byte literal before a two-byte run must not start a phantom run.
|
||||
data2 = b"even_phase_lite\x00" + "vip_pass".encode("utf-16-le") + b"\x00\x00"
|
||||
got2 = [row[2] for row in dss.recover_strings(
|
||||
data2, {"offset": 0, "vaddr": 0x1000, "size": len(data2)})]
|
||||
assert "vip_pass" in got2, got2
|
||||
assert "vip_pass"[0] not in [t for t in got2 if t != "vip_pass"], got2
|
||||
|
||||
|
||||
def test_min_length_drops_the_shorter_literals(load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
rodata = dss.Elf(synthetic_elf.data).section(".rodata")
|
||||
long_only = dss.recover_strings(synthetic_elf.data, rodata, min_length=12)
|
||||
assert [row[2] for row in long_only] == ["premium_license"]
|
||||
everything = dss.recover_strings(synthetic_elf.data, rodata, min_length=6)
|
||||
assert {row[2] for row in everything} == {"premium_license", "trial_mode", "vip_pass"}
|
||||
|
||||
|
||||
def test_the_limit_caps_how_many_literals_come_back(load_script, synthetic_elf):
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
rodata = dss.Elf(synthetic_elf.data).section(".rodata")
|
||||
assert len(dss.recover_strings(synthetic_elf.data, rodata, limit=1)) == 1
|
||||
assert len(dss.recover_strings(synthetic_elf.data, rodata, limit=2)) == 2
|
||||
|
||||
|
||||
def test_a_section_claiming_more_bytes_than_the_file_holds_is_clamped(load_script, synthetic_elf):
|
||||
"""A truncated image must not read past the end; the reader clamps to the data it has."""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
truncated = synthetic_elf.data[:synthetic_elf.rodata["offset"] + 0x10]
|
||||
rodata = dict(synthetic_elf.rodata, size=synthetic_elf.rodata["size"] + 0x1000)
|
||||
recovered = dss.recover_strings(truncated, rodata)
|
||||
assert [row[2] for row in recovered] == ["premium_license"]
|
||||
|
||||
|
||||
def test_the_code_section_yields_no_literals(load_script, synthetic_elf):
|
||||
"""The fixture's `.text` is sixteen bytes of instructions, so nothing survives the length gate."""
|
||||
dss = load_script("dart_snapshot_strings.py")
|
||||
text = dss.Elf(synthetic_elf.data).section(".text")
|
||||
assert dss.recover_strings(synthetic_elf.data, text) == []
|
||||
|
||||
|
||||
# --- pool observation --------------------------------------------------------
|
||||
@@ -0,0 +1,306 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/doctor.py, the environment doctor.
|
||||
|
||||
Three properties of the doctor are what make its report a measurement, and each gets a test rather
|
||||
than a description:
|
||||
|
||||
* its per-script dependency pass is static, so a file whose *execution* would raise is inspected
|
||||
without ever being imported -- proved with a script that writes a sentinel and imports a module
|
||||
that does not exist;
|
||||
* `discover_scripts` scans the directory instead of holding a list, proved by pointing the doctor at
|
||||
a synthetic directory and checking that what it reports is what is there;
|
||||
* a script with no capability is reported `unknown` and *counted*, because a denominator that
|
||||
shrinks quietly is how a report stops being a measurement.
|
||||
|
||||
The host facts come from a real probe, so a host where the probe cannot answer skips the fact tests
|
||||
with a printed reason instead of failing them.
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def write(root, name, body):
|
||||
path = root / name
|
||||
path.write_text(body, encoding="utf-8")
|
||||
return path
|
||||
|
||||
|
||||
def synthetic_scripts_dir(tmp_path, sentinel=None):
|
||||
"""A directory of the six shapes `scan_scripts` has to classify."""
|
||||
root = tmp_path / "scriptdir"
|
||||
root.mkdir()
|
||||
write(root, "declaring.py",
|
||||
'"""capability: pe_structure_read\n\nDeclares its own capability.\n"""\n'
|
||||
"import json\n\nprint('usage: declaring')\n")
|
||||
write(root, "aob_scan.py",
|
||||
'"""A script the map knows and that declares nothing.\n"""\n'
|
||||
"import os\n\nprint('usage: aob_scan')\n")
|
||||
write(root, "needs-tool.py",
|
||||
'"""Imports a module that is not installed, safely.\n"""\n'
|
||||
"try:\n import zebra_kit\nexcept ImportError:\n zebra_kit = None\n\n"
|
||||
"print('usage: needs-tool')\n")
|
||||
write(root, "broken.py",
|
||||
'"""Raises as soon as it is executed.\n"""\n'
|
||||
"raise RuntimeError('this script cannot start')\n")
|
||||
write(root, "stranger.py",
|
||||
'"""Neither declares nor is mapped.\n"""\n'
|
||||
"import sibling_helper\n\nprint('usage: stranger')\n")
|
||||
write(root, "helper.js", "// not a python script\n")
|
||||
write(root, "sibling_helper.py",
|
||||
'"""A file this directory ships, so it is not a third-party dependency.\n"""\n')
|
||||
if sentinel is not None:
|
||||
write(root, "sentinel_target.py",
|
||||
'"""Would write a sentinel the moment it were executed.\n"""\n'
|
||||
"import pathlib\n\n"
|
||||
"pathlib.Path(%r).write_text('executed')\n"
|
||||
"import zebra_kit_that_does_not_exist\n" % str(sentinel))
|
||||
return root
|
||||
|
||||
|
||||
# --- static dependency pass --------------------------------------------------
|
||||
|
||||
def test_static_deps_separates_stdlib_siblings_and_third_party(load_script, tmp_path, monkeypatch):
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
assert doctor.static_python_deps(str(root / "declaring.py")) == (set(), None)
|
||||
assert doctor.static_python_deps(str(root / "aob_scan.py")) == (set(), None)
|
||||
assert doctor.static_python_deps(str(root / "stranger.py")) == (set(), None)
|
||||
deps, reason = doctor.static_python_deps(str(root / "needs-tool.py"))
|
||||
assert (deps, reason) == ({"zebra_kit"}, None)
|
||||
|
||||
|
||||
def test_static_deps_sees_an_import_inside_a_try_block(load_script, tmp_path, monkeypatch):
|
||||
"""The pass reads the AST, so an import that is guarded at run time is still a dependency."""
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
deps, _reason = doctor.static_python_deps(str(root / "needs-tool.py"))
|
||||
assert deps == {"zebra_kit"}
|
||||
|
||||
|
||||
def test_static_deps_does_not_import_or_execute_the_file_it_inspects(load_script, tmp_path,
|
||||
monkeypatch):
|
||||
"""The sentinel file is the proof: if the file were imported, it would exist -- or the import of
|
||||
the deliberately missing module would raise out of `static_python_deps` entirely."""
|
||||
doctor = load_script("doctor.py")
|
||||
sentinel = tmp_path / "sentinel.txt"
|
||||
root = synthetic_scripts_dir(tmp_path, sentinel=sentinel)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
deps, reason = doctor.static_python_deps(str(root / "sentinel_target.py"))
|
||||
assert deps == {"zebra_kit_that_does_not_exist"}
|
||||
assert reason is None
|
||||
assert not sentinel.exists(), "the inspected script was executed"
|
||||
assert "zebra_kit_that_does_not_exist" not in sys.modules
|
||||
|
||||
|
||||
def test_static_deps_reports_a_syntax_error_instead_of_raising(load_script, tmp_path):
|
||||
doctor = load_script("doctor.py")
|
||||
write(tmp_path, "broken.py", "def f(:\n")
|
||||
deps, reason = doctor.static_python_deps(str(tmp_path / "broken.py"))
|
||||
assert deps == set()
|
||||
assert reason
|
||||
|
||||
|
||||
def test_static_deps_reports_an_unreadable_path_instead_of_raising(load_script, tmp_path):
|
||||
doctor = load_script("doctor.py")
|
||||
deps, reason = doctor.static_python_deps(str(tmp_path / "absent.py"))
|
||||
assert deps == set()
|
||||
assert "absent.py" in reason
|
||||
|
||||
|
||||
def test_every_shipped_script_is_standard_library_only(load_script):
|
||||
"""The claim the skill's own README makes, measured on the scripts in the tree."""
|
||||
doctor = load_script("doctor.py")
|
||||
for name in doctor.discover_scripts():
|
||||
if not name.endswith(".py"):
|
||||
continue
|
||||
deps, reason = doctor.static_python_deps(os.path.join(doctor.HERE, name))
|
||||
assert reason is None, "%s could not be read: %s" % (name, reason)
|
||||
assert deps == set(), "%s needs %r, which the kit does not ship" % (name, sorted(deps))
|
||||
|
||||
|
||||
# --- discovery ---------------------------------------------------------------
|
||||
|
||||
def test_discover_scripts_reports_whether_files_are_really_there(load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
present = sorted(n for n in os.listdir(doctor.HERE) if n.endswith((".py", ".js")))
|
||||
assert doctor.discover_scripts() == present
|
||||
for name in ("capabilities.py", "doctor.py", "dart_snapshot_strings.py"):
|
||||
assert name in present
|
||||
|
||||
|
||||
def test_discover_scripts_scans_a_directory_rather_than_a_fixed_list(load_script, tmp_path,
|
||||
monkeypatch):
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
found = doctor.discover_scripts()
|
||||
assert found == sorted(n for n in os.listdir(str(root)) if n.endswith((".py", ".js")))
|
||||
assert set(found) >= {"declaring.py", "aob_scan.py", "needs-tool.py", "broken.py",
|
||||
"stranger.py", "helper.js", "sibling_helper.py"}
|
||||
assert "notes.txt" not in found
|
||||
write(root, "late_arrival.py", '"""Added after the scan.\n"""\n')
|
||||
assert "late_arrival.py" in doctor.discover_scripts()
|
||||
|
||||
|
||||
def test_discover_scripts_ignores_files_that_are_not_scripts(load_script, tmp_path, monkeypatch):
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
write(root, "notes.txt", "not a script\n")
|
||||
write(root, "Makefile", "all:\n\ttrue\n")
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
found = doctor.discover_scripts()
|
||||
assert "notes.txt" not in found
|
||||
assert "Makefile" not in found
|
||||
|
||||
|
||||
# --- per-script table --------------------------------------------------------
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def capability_results(load_script):
|
||||
caps = load_script("capabilities.py")
|
||||
return caps.resolve_all(caps.Probe(tool_dirs=[]))
|
||||
|
||||
|
||||
def test_scan_scripts_classifies_every_script_it_finds(load_script, tmp_path, monkeypatch,
|
||||
capability_results):
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
rows = doctor.scan_scripts(capability_results)
|
||||
by_name = {row["script"]: row for row in rows}
|
||||
|
||||
assert len(rows) == len(doctor.discover_scripts()) == 7
|
||||
assert by_name["declaring.py"]["capability_source"] == "declared"
|
||||
assert by_name["declaring.py"]["capabilities"] == ["pe_structure_read"]
|
||||
assert by_name["aob_scan.py"]["capability_source"] == "mapped"
|
||||
assert by_name["aob_scan.py"]["capabilities"] == ["aob_locate"]
|
||||
assert by_name["needs-tool.py"]["capability_source"] == "unknown"
|
||||
assert by_name["needs-tool.py"]["third_party"] == ["zebra_kit"]
|
||||
assert by_name["helper.js"]["runnable"] is True
|
||||
assert "not a python script" in by_name["helper.js"]["detail"]
|
||||
|
||||
|
||||
def test_a_script_with_no_capability_is_unknown_counted_and_never_dropped(
|
||||
load_script, tmp_path, monkeypatch, capability_results):
|
||||
"""The denominator is the number of scripts in the directory, unknown ones included."""
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
rows = doctor.scan_scripts(capability_results)
|
||||
|
||||
unknown = [row for row in rows if row["capability_source"] == "unknown"]
|
||||
assert len(rows) == len(doctor.discover_scripts())
|
||||
assert {row["script"] for row in unknown} == {"needs-tool.py", "broken.py", "stranger.py",
|
||||
"helper.js", "sibling_helper.py"}
|
||||
assert len(unknown) == 5, "the unknown rows stay in the table; only two scripts claim one"
|
||||
assert {row["script"] for row in rows if row["capability_source"] != "unknown"} == \
|
||||
{"declaring.py", "aob_scan.py"}
|
||||
for row in unknown:
|
||||
assert row["capabilities"] == []
|
||||
assert row["capability_state"] == "unknown"
|
||||
assert row["script"] in {r["script"] for r in rows}
|
||||
|
||||
|
||||
def test_scan_scripts_marks_a_script_that_cannot_start(load_script, tmp_path, monkeypatch,
|
||||
capability_results):
|
||||
doctor = load_script("doctor.py")
|
||||
root = synthetic_scripts_dir(tmp_path)
|
||||
monkeypatch.setattr(doctor, "HERE", str(root))
|
||||
row = {r["script"]: r for r in doctor.scan_scripts(capability_results)}["broken.py"]
|
||||
assert row["runnable"] is False
|
||||
assert "RuntimeError" in row["detail"]
|
||||
assert "Traceback" not in row["detail"]
|
||||
|
||||
|
||||
def test_scan_scripts_agrees_with_the_registry_about_every_real_script(load_script,
|
||||
capability_results):
|
||||
"""On the shipped directory, every script is classified and the states come from the registry."""
|
||||
doctor = load_script("doctor.py")
|
||||
caps = load_script("capabilities.py")
|
||||
rows = doctor.scan_scripts(capability_results)
|
||||
assert [row["script"] for row in rows] == doctor.discover_scripts()
|
||||
states = {result["id"]: result["state"] for result in capability_results}
|
||||
for row in rows:
|
||||
declared, source = caps.capabilities_for_script(row["script"],
|
||||
os.path.join(doctor.HERE, row["script"]))
|
||||
assert row["capabilities"] == declared
|
||||
assert row["capability_source"] == source
|
||||
if not declared:
|
||||
assert row["capability_state"] == "unknown"
|
||||
continue
|
||||
assert row["capability_state"] == ("ok" if all(states[c] == "ok" for c in declared)
|
||||
else "blocked")
|
||||
|
||||
|
||||
# --- environment facts -------------------------------------------------------
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def host_facts(load_script):
|
||||
"""One environment report for the module: it runs a probe and shells out to logman."""
|
||||
doctor = load_script("doctor.py")
|
||||
report = doctor.environment_report()
|
||||
if len(report) == 1 and report[0].get("fact") == "host probe":
|
||||
pytest.skip("the host probe could not answer here (%s), so the fact table is inert"
|
||||
% report[0].get("effect"))
|
||||
return report
|
||||
|
||||
|
||||
def test_environment_report_is_a_list_of_facts_with_the_documented_keys(host_facts):
|
||||
assert isinstance(host_facts, list) and host_facts
|
||||
for fact in host_facts:
|
||||
assert set(fact) == {"fact", "value", "effect", "severity"}, sorted(fact)
|
||||
assert fact["fact"] and fact["value"] is not None and fact["effect"]
|
||||
|
||||
|
||||
def test_every_environment_fact_is_labelled_note_or_warn(host_facts):
|
||||
"""The label is the whole point: `warn` means it can change a result, and the doctor's exit
|
||||
token turns on that count."""
|
||||
for fact in host_facts:
|
||||
assert fact["severity"] in ("note", "warn"), fact
|
||||
|
||||
|
||||
def test_environment_report_always_states_the_working_directory(host_facts):
|
||||
"""Unconditional, so the fact table can never be empty on a host where the probe answers."""
|
||||
facts = {fact["fact"]: fact for fact in host_facts}
|
||||
assert "working directory" in facts
|
||||
assert facts["working directory"]["value"] == os.getcwd()
|
||||
assert facts["working directory"]["severity"] == "note"
|
||||
|
||||
|
||||
def test_severity_summary_counts_only_warnings(load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
assert doctor._severity_summary([]) == 0
|
||||
assert doctor._severity_summary([{"severity": "note"}, {"severity": "note"}]) == 0
|
||||
assert doctor._severity_summary([{"severity": "note"}, {"severity": "warn"},
|
||||
{"severity": "warn"}]) == 2
|
||||
|
||||
|
||||
# --- primitives --------------------------------------------------------------
|
||||
|
||||
def test_run_reports_a_failure_instead_of_raising(load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
code, output = doctor.run(["definitely-not-a-real-command-xyz"])
|
||||
assert code == -1
|
||||
assert output
|
||||
|
||||
|
||||
def test_run_returns_the_code_and_combined_output(load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
code, output = doctor.run([sys.executable, "-B", "-c", "print('probe')"])
|
||||
assert code == 0
|
||||
assert "probe" in output
|
||||
|
||||
|
||||
def test_the_documented_exit_codes_and_tokens_are_the_ones_used(load_script):
|
||||
doctor = load_script("doctor.py")
|
||||
assert (doctor.EXIT_OK, doctor.EXIT_USE, doctor.EXIT_ENV, doctor.EXIT_INTERNAL) == (0, 2, 3, 4)
|
||||
assert doctor.TOKENS["ok"] == "env_ok"
|
||||
assert doctor.TOKENS["partial"] == "env_partial"
|
||||
assert doctor.TOKENS["blocked"] == "env_blocked"
|
||||
assert doctor.TOKENS["use"] == "usage_error"
|
||||
assert doctor.TOKENS["internal"] == "internal_error"
|
||||
@@ -0,0 +1,138 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/fold.py, the bounded summariser.
|
||||
|
||||
What these tests hold is the promise the tool makes about itself: the summary is bounded, it says
|
||||
what it withheld, and the numbers it prints are true of the file's actual shape.
|
||||
|
||||
The shape tests exist because the first version read a pretty-printed 32 MB JSON document line by
|
||||
line -- its shape decision was the first byte plus a size cutoff, and both a document and a record
|
||||
file begin with `{` -- and summarised it as 16,278 records with 1.7 million unparseable lines. Every
|
||||
count in that summary was honestly computed and the summary described a file that did not exist.
|
||||
The depth scan that replaced it is what the first three tests hold, including the case that breaks
|
||||
naive counters: braces and escaped quotes inside string values.
|
||||
"""
|
||||
import json
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def run_fold(load_script, monkeypatch, capsys, tmp_path, name, content, extra=()):
|
||||
"""Write the input, run main() as if from the command line, and return (code, stdout)."""
|
||||
path = tmp_path / name
|
||||
if isinstance(content, bytes):
|
||||
path.write_bytes(content)
|
||||
else:
|
||||
path.write_text(content, encoding="utf-8")
|
||||
fold = load_script("fold.py")
|
||||
monkeypatch.setattr(sys, "argv", ["fold.py", str(path), *extra])
|
||||
code = fold.main()
|
||||
out = capsys.readouterr().out
|
||||
return code, out
|
||||
|
||||
|
||||
# --- the shape is decided by the bytes, not the name or the size ----------------
|
||||
|
||||
|
||||
def test_two_records_on_two_lines_are_records(load_script, monkeypatch, capsys, tmp_path):
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "a.jsonl",
|
||||
'{"a": 1}\n{"a": 2}\n')
|
||||
assert code == 0
|
||||
assert "records : 2" in out
|
||||
assert "RESULT=fold_ok" in out
|
||||
|
||||
|
||||
def test_a_pretty_printed_document_is_one_value_not_many(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
# The string value carries braces, brackets and escaped quotes of its own: a naive depth count
|
||||
# is fooled by exactly this, the scanner must not be.
|
||||
doc = {"outer": {"s": 'a "{[quoted]}" body', "n": 3}, "arr": [1, 2, 3]}
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "graph.json",
|
||||
json.dumps(doc, indent=2))
|
||||
assert code == 0
|
||||
assert "one JSON document" in out
|
||||
assert "records" not in out
|
||||
|
||||
|
||||
def test_content_after_the_first_value_closes_means_records(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "b.jsonl",
|
||||
'{"a": 1}\n{"a": 2}\n{"a": 3}\n')
|
||||
assert code == 0
|
||||
assert "records : 3" in out
|
||||
|
||||
|
||||
def test_a_single_document_past_the_walk_limit_is_left_unwalked(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
fold = load_script("fold.py")
|
||||
path = tmp_path / "huge.json"
|
||||
path.write_text('{"a": "%s"}' % ("x" * 40), encoding="utf-8")
|
||||
monkeypatch.setattr(fold, "JSON_WALK_LIMIT", 8)
|
||||
monkeypatch.setattr(sys, "argv", ["fold.py", str(path)])
|
||||
assert fold.main() == 0
|
||||
out = capsys.readouterr().out
|
||||
assert "was not loaded" in out
|
||||
assert "RESULT=fold_ok" in out
|
||||
|
||||
|
||||
# --- the summary admits its bounds ----------------------------------------------
|
||||
|
||||
|
||||
def test_a_key_with_few_values_counts_them_exactly(load_script, monkeypatch, capsys, tmp_path):
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "c.jsonl",
|
||||
'{"kind": "call"}\n{"kind": "call"}\n{"kind": "ret"}\n',
|
||||
("--key", "kind", "--limit", "4"))
|
||||
assert code == 0
|
||||
assert 'key "kind": 2 distinct value(s) seen (showing 2)' in out
|
||||
assert "x2" in out
|
||||
|
||||
|
||||
def test_a_key_at_the_tracking_cap_is_reported_as_a_floor(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
fold = load_script("fold.py")
|
||||
body = "".join(json.dumps({"v": i}) + "\n" for i in range(fold.VALUE_TRACK_CAP + 5))
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "wide.jsonl", body,
|
||||
("--key", "v", "--limit", "3"))
|
||||
assert code == 0
|
||||
assert "at least" in out
|
||||
assert "floor" in out
|
||||
# The cap must not leak as an exact count: "at least" only ever prefixes the capped number.
|
||||
assert "2005 distinct" not in out
|
||||
|
||||
|
||||
def test_grep_reports_how_many_matches_were_withheld(load_script, monkeypatch, capsys, tmp_path):
|
||||
text = "\n".join("hit %d premium" % i if i % 2 else "miss %d plain" % i
|
||||
for i in range(10)) + "\n"
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "log.txt", text,
|
||||
("--grep", "premium", "--limit", "1"))
|
||||
assert code == 0
|
||||
assert "matches: 5 (showing 1)" in out
|
||||
assert "4 more match(es) withheld" in out
|
||||
|
||||
|
||||
def test_text_input_never_touches_the_json_paths(load_script, monkeypatch, capsys, tmp_path):
|
||||
code, out = run_fold(load_script, monkeypatch, capsys, tmp_path, "notes.txt",
|
||||
"first\nsecond\nthird\n", ("--limit", "2"))
|
||||
assert code == 0
|
||||
assert "shape : text" in out
|
||||
assert "lines : 3" in out
|
||||
|
||||
|
||||
# --- the record itself is never the cost ----------------------------------------
|
||||
|
||||
|
||||
def test_the_input_is_not_modified(load_script, monkeypatch, capsys, tmp_path):
|
||||
body = '{"a": 1}\n{"a": 2}\n'
|
||||
path = tmp_path / "keep.jsonl"
|
||||
path.write_text(body, encoding="utf-8")
|
||||
run_fold(load_script, monkeypatch, capsys, tmp_path, "keep.jsonl", body)
|
||||
assert path.read_text(encoding="utf-8") == body
|
||||
|
||||
|
||||
def test_a_missing_file_is_a_usage_error_not_a_summary(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
fold = load_script("fold.py")
|
||||
monkeypatch.setattr(sys, "argv", ["fold.py", str(tmp_path / "absent.jsonl")])
|
||||
assert fold.main() == 2
|
||||
assert "RESULT=usage_error" in capsys.readouterr().out
|
||||
@@ -0,0 +1,377 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/pepatch.py.
|
||||
|
||||
The refusals matter more than the success path: a patcher that silently picks the wrong
|
||||
occurrence, or accepts a length change, destroys the baseline it was supposed to preserve. Each
|
||||
refusal below is asserted against the real exception text, and the CLI-level tests drive
|
||||
``pepatch.main()`` in-process so no subprocess is required to prove that a dry run writes
|
||||
nothing and that a real run leaves its input byte-identical.
|
||||
"""
|
||||
import hashlib
|
||||
import shutil
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def pillow(tmp_path, fixtures):
|
||||
"""A privately owned copy of the fixture, so a test can never disturb another test."""
|
||||
dest = tmp_path / "input.exe"
|
||||
shutil.copyfile(str(fixtures.pe64.path), str(dest))
|
||||
return dest
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def plant(fixtures):
|
||||
"""The byte sequence the fixture plants twice, as a spaced hex string."""
|
||||
return fixtures.pe64.plant
|
||||
|
||||
|
||||
def sha256(path):
|
||||
return hashlib.sha256(path.read_bytes()).hexdigest()
|
||||
|
||||
|
||||
def run_main(module, monkeypatch, argv):
|
||||
monkeypatch.setattr(sys, "argv", [getattr(module, "__file__", "pepatch.py")] + list(argv))
|
||||
return module.main()
|
||||
|
||||
|
||||
# --- parse_bytes -------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("text,expected", [
|
||||
("48 8B 05", [0x48, 0x8B, 0x05]),
|
||||
("488B05", [0x48, 0x8B, 0x05]),
|
||||
("48,8B,05", [0x48, 0x8B, 0x05]),
|
||||
("0x48 0x8b", [0x48, 0x8B]),
|
||||
("48 8B ??", [0x48, 0x8B, None]),
|
||||
("?", [None]),
|
||||
("**", [None]),
|
||||
("xx", [None]),
|
||||
(" 48 8B ", [0x48, 0x8B]),
|
||||
("48", [0x48]),
|
||||
])
|
||||
def test_parse_bytes_accepts_the_documented_forms(load_script, text, expected):
|
||||
body, tokens = load_script("pepatch.py").parse_bytes(text)
|
||||
assert body == expected
|
||||
assert tokens
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text", ["", None, " "])
|
||||
def test_parse_bytes_of_nothing_is_empty(load_script, text):
|
||||
assert load_script("pepatch.py").parse_bytes(text) == ([], [])
|
||||
|
||||
|
||||
@pytest.mark.parametrize("token", ["ZZ", "4", "ABC", "GG HH", "0x1", "48 8G"])
|
||||
def test_parse_bytes_rejects_a_bad_token(load_script, token):
|
||||
with pytest.raises(ValueError, match="not a byte"):
|
||||
load_script("pepatch.py").parse_bytes(token)
|
||||
|
||||
|
||||
def test_parse_bytes_returns_the_original_tokens(load_script):
|
||||
body, tokens = load_script("pepatch.py").parse_bytes("48 8B ??")
|
||||
assert tokens == ["48", "8B", "??"]
|
||||
assert body == [0x48, 0x8B, None]
|
||||
|
||||
|
||||
# --- find_pattern ------------------------------------------------------------
|
||||
|
||||
def test_find_pattern_returns_every_occurrence(load_script, fixtures):
|
||||
"""The fixture plants the same sequence twice, on purpose."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
data = fixtures.pe64.data
|
||||
pattern = pepatch.parse_bytes(fixtures.pe64.plant)[0]
|
||||
first, second = fixtures.pe64.plant_offsets
|
||||
assert pepatch.find_pattern(data, pattern) == [first, second]
|
||||
assert data[first:first + len(pattern)] == bytes(pattern)
|
||||
assert data[second:second + len(pattern)] == bytes(pattern)
|
||||
|
||||
|
||||
def test_find_pattern_honours_wildcards(load_script):
|
||||
pepatch = load_script("pepatch.py")
|
||||
data = bytes([0x00, 0x10, 0x99, 0x10, 0x00])
|
||||
assert pepatch.find_pattern(data, [None, 0x10]) == [0, 2]
|
||||
assert pepatch.find_pattern(data, [0x10, None]) == [1, 3]
|
||||
assert pepatch.find_pattern(data, [None, 0x10, None]) == [0, 2]
|
||||
|
||||
|
||||
def test_find_pattern_start_and_limit(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
data = fixtures.pe64.data
|
||||
pattern = pepatch.parse_bytes(fixtures.pe64.plant)[0]
|
||||
first, second = fixtures.pe64.plant_offsets
|
||||
assert pepatch.find_pattern(data, pattern, start=first + 1) == [second]
|
||||
assert pepatch.find_pattern(data, pattern, limit=1) == [first]
|
||||
|
||||
|
||||
def test_find_pattern_reports_no_hits_rather_than_guessing(load_script, fixtures):
|
||||
"""A negative outcome: a pattern that is not there must not match somewhere else."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
assert pepatch.find_pattern(fixtures.pe64.data, [0xDE, 0xAD, 0xBE, 0xEF]) == []
|
||||
assert pepatch.find_pattern(fixtures.pe64.data,
|
||||
pepatch.parse_bytes("63 6F 6D 6D 6F 6E")[0]) == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("pattern", [[], [None], [None, None]])
|
||||
def test_find_pattern_refuses_an_empty_or_all_wildcard_pattern(load_script, pattern):
|
||||
with pytest.raises(ValueError, match="empty or entirely wildcards"):
|
||||
load_script("pepatch.py").find_pattern(b"\x00\x01\x02", pattern)
|
||||
|
||||
|
||||
# --- length_mark -------------------------------------------------------------
|
||||
|
||||
def test_length_mark_distinguishes_three_states(load_script):
|
||||
pepatch = load_script("pepatch.py")
|
||||
assert pepatch.length_mark({"equal_length": True}) == ""
|
||||
assert "LENGTH CHANGE" in pepatch.length_mark({"equal_length": False})
|
||||
assert "length unproven" in pepatch.length_mark({"equal_length": None})
|
||||
|
||||
|
||||
# --- apply_one ---------------------------------------------------------------
|
||||
|
||||
def test_ambiguous_pattern_is_refused(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "bytes": "90 " * 12}
|
||||
with pytest.raises(pepatch.PatchError, match="matched 2 sites"):
|
||||
pepatch.apply_one(spec, pe, fixtures.pe64.data)
|
||||
|
||||
|
||||
def test_explicit_index_selects_one_site(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
first, second = fixtures.pe64.plant_offsets
|
||||
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "index": 1,
|
||||
"bytes": "90 " * 12, "expect": fixtures.pe64.plant}
|
||||
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data)
|
||||
assert applied["offset"] == second
|
||||
assert applied["offset"] != first
|
||||
assert applied["equal_length"] is True
|
||||
assert applied["length_basis"] == "expect"
|
||||
|
||||
|
||||
def test_allow_many_takes_the_first_hit(load_script, fixtures):
|
||||
"""The documented escape hatch: it is not silent, and it still reports one site."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
spec = {"name": "gate", "pattern": fixtures.pe64.plant, "allow-many": True,
|
||||
"bytes": "90 " * 12, "expect": fixtures.pe64.plant}
|
||||
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data)
|
||||
assert applied["offset"] == fixtures.pe64.plant_offsets[0]
|
||||
|
||||
|
||||
def test_expect_mismatch_is_refused(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
with pytest.raises(pepatch.PatchError, match="the site moved or the wrong binary is open"):
|
||||
pepatch.apply_one({"name": "gate", "offset": "0x410", "expect": "90 90",
|
||||
"bytes": "CC CC"}, pe, fixtures.pe64.data)
|
||||
|
||||
|
||||
def test_length_is_unproven_without_expect_or_original_length(load_script, fixtures):
|
||||
"""No disassembler, no honest claim: the marker must say the length is unknown."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
applied = pepatch.apply_one({"name": "site", "offset": "0x410", "bytes": "CC"},
|
||||
pe, fixtures.pe64.data)
|
||||
assert applied["equal_length"] is None
|
||||
assert applied["length_basis"] == "unknown (no expect/original-length given)"
|
||||
assert "length unproven" in pepatch.length_mark(applied)
|
||||
|
||||
|
||||
def test_length_change_is_refused_without_allow_resize(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
spec = {"name": "gate", "offset": "0x410", "expect": "48 8B", "bytes": "CC CC CC"}
|
||||
with pytest.raises(pepatch.PatchError, match="--allow-resize"):
|
||||
pepatch.apply_one(spec, pe, fixtures.pe64.data)
|
||||
|
||||
|
||||
def test_length_change_is_marked_structural_with_allow_resize(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
spec = {"name": "gate", "offset": "0x410", "expect": "48 8B", "bytes": "CC CC CC"}
|
||||
applied = pepatch.apply_one(spec, pe, fixtures.pe64.data, allow_resize=True)
|
||||
assert applied["equal_length"] is False
|
||||
assert "LENGTH CHANGE" in pepatch.length_mark(applied)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("spec,expected", [
|
||||
({"name": "wild", "offset": "0x410", "bytes": "CC ??"}, "no concrete replacement bytes"),
|
||||
({"name": "tail", "rva": "0x2200", "bytes": "CC"}, "maps to no file offset"),
|
||||
({"name": "eof", "offset": "0xFF8", "original-length": 16, "bytes": "CC"},
|
||||
"runs past end of file"),
|
||||
({"name": "neg", "offset": -1, "bytes": "CC"}, "must not be negative"),
|
||||
({"name": "nowhere", "bytes": "CC"}, "none of pattern/rva/offset"),
|
||||
({"name": "oob", "pattern": "48 8B 05 11 22 33 00 48 85 C0 74 05", "index": 5,
|
||||
"bytes": "CC"}, "index 5 out of range"),
|
||||
({"name": "none", "pattern": "DE AD BE EF", "bytes": "CC"}, "matched nothing"),
|
||||
])
|
||||
def test_resolve_refuses_specs_it_cannot_honour(load_script, fixtures, spec, expected):
|
||||
pepatch = load_script("pepatch.py")
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
with pytest.raises(pepatch.PatchError, match=expected):
|
||||
pepatch.resolve(spec, pe, fixtures.pe64.data)
|
||||
|
||||
|
||||
# --- verify ------------------------------------------------------------------
|
||||
|
||||
def test_verify_reports_bytes_that_are_not_there(load_script, fixtures):
|
||||
"""A negative outcome: the re-read step must fail when the patch did not land."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
_pe, problems = pepatch.verify(str(fixtures.pe64.path),
|
||||
[{"name": "ghost", "offset": 0x410, "after": "CC"}])
|
||||
assert len(problems) == 1
|
||||
assert "ghost" in problems[0] and "expected CC" in problems[0]
|
||||
|
||||
|
||||
def test_verify_is_satisfied_when_the_bytes_are_present(load_script, fixtures):
|
||||
pepatch = load_script("pepatch.py")
|
||||
first, _second = fixtures.pe64.plant_offsets
|
||||
present = pepatch.fmt_bytes(fixtures.pe64.data[first:first + 4])
|
||||
_pe, problems = pepatch.verify(str(fixtures.pe64.path),
|
||||
[{"name": "real", "offset": first, "after": present}])
|
||||
assert problems == []
|
||||
|
||||
|
||||
# --- main() ------------------------------------------------------------------
|
||||
|
||||
def test_dry_run_writes_nothing(load_script, pillow, monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
before = sha256(pillow)
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--at-offset", "0x410", "--bytes", "CC", "--dry-run"])
|
||||
out = capsys.readouterr()
|
||||
assert code == 0
|
||||
assert "dry run" in out.out
|
||||
assert sha256(pillow) == before
|
||||
assert list(pillow.parent.iterdir()) == [pillow], "a dry run created a file"
|
||||
|
||||
|
||||
def test_dry_run_reports_a_refusal_without_writing(load_script, pillow, monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--at-offset", "0x410", "--expect", "90 90",
|
||||
"--bytes", "CC CC", "--dry-run"])
|
||||
out = capsys.readouterr()
|
||||
assert code == 1
|
||||
assert "FAIL" in out.out
|
||||
assert list(pillow.parent.iterdir()) == [pillow]
|
||||
|
||||
|
||||
def test_out_is_required_to_write_a_patched_file(load_script, pillow, monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
before = sha256(pillow)
|
||||
code = run_main(pepatch, monkeypatch, [str(pillow), "--at-offset", "0x410",
|
||||
"--bytes", "CC"])
|
||||
err = capsys.readouterr().err
|
||||
assert code == 2
|
||||
assert "--out is required" in err
|
||||
assert sha256(pillow) == before
|
||||
assert list(pillow.parent.iterdir()) == [pillow]
|
||||
|
||||
|
||||
def test_real_run_preserves_the_input_and_verifies_the_output(load_script, pillow, fixtures,
|
||||
monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
peutil = load_script("peutil.py")
|
||||
before = sha256(pillow)
|
||||
first, _second = fixtures.pe64.plant_offsets
|
||||
replacement = "48 8B 05 11 22 33 00 48 85 C0 EB 05"
|
||||
out = pillow.parent / "patched.exe"
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--pattern", fixtures.pe64.plant, "--at-hit", "0",
|
||||
"--expect", fixtures.pe64.plant, "--bytes", replacement, "--out", str(out)])
|
||||
text = capsys.readouterr().out
|
||||
assert code == 0, text
|
||||
assert "verified" in text
|
||||
assert sha256(pillow) == before, "the input file was modified"
|
||||
assert out.exists()
|
||||
|
||||
pe_out = peutil.PE(out.read_bytes(), str(out))
|
||||
assert pe_out.file_size == fixtures.pe64.size
|
||||
assert out.read_bytes()[first:first + 12] == bytes(pepatch.parse_bytes(replacement)[0])
|
||||
# The second planted copy must be untouched: only the selected site moved.
|
||||
assert out.read_bytes()[_second:_second + 12] == bytes(
|
||||
pepatch.parse_bytes(fixtures.pe64.plant)[0])
|
||||
assert pe_out.checksum_ok() is True
|
||||
assert pe_out.checksum != 0
|
||||
|
||||
|
||||
def test_input_and_output_differ_only_where_expected(load_script, pillow, fixtures,
|
||||
monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
first, _second = fixtures.pe64.plant_offsets
|
||||
checksum_field = int(fixtures.pe64.meta["checksum_offset"], 16)
|
||||
out = pillow.parent / "one-byte.exe"
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--at-offset", hex(first), "--expect", "48 8B",
|
||||
"--bytes", "CC 8B", "--out", str(out)])
|
||||
capsys.readouterr()
|
||||
assert code == 0
|
||||
diff = [i for i, (a, b) in enumerate(zip(pillow.read_bytes(), out.read_bytes())) if a != b]
|
||||
# The patched byte plus at most the four checksum bytes.
|
||||
assert first in diff
|
||||
assert len(diff) <= 5
|
||||
assert set(diff) - {first} <= set(range(checksum_field, checksum_field + 4))
|
||||
|
||||
|
||||
def test_no_checksum_leaves_the_declared_field_stale(load_script, pillow, monkeypatch, capsys):
|
||||
pepatch = load_script("pepatch.py")
|
||||
peutil = load_script("peutil.py")
|
||||
out = pillow.parent / "stale.exe"
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--at-offset", "0x410", "--expect", "48 8B",
|
||||
"--bytes", "90 90", "--no-checksum", "--out", str(out), "--json"])
|
||||
captured = capsys.readouterr()
|
||||
assert code == 0, captured.err
|
||||
import json
|
||||
payload = json.loads(captured.out)
|
||||
assert payload["checksum_recomputed"] is False
|
||||
assert payload["checksum_after"]["matches"] is False
|
||||
assert peutil.PE(out.read_bytes(), None).checksum_ok() is False
|
||||
|
||||
|
||||
def test_allow_resize_shifts_the_following_bytes(load_script, pillow, fixtures,
|
||||
monkeypatch, capsys):
|
||||
"""A resize changes the file's length, and everything after the site moves with it.
|
||||
|
||||
This is the behaviour the flag promises. An earlier version wrote exactly as many bytes as
|
||||
it read, so the file never grew while the documented contract said it would -- the flag's
|
||||
only real effect was to defeat the equal-length guard. The assertions below pin the genuine
|
||||
consequence, shift included, because a shifted file is a structural edit and the caller is
|
||||
entitled to see that stated rather than inferred.
|
||||
"""
|
||||
pepatch = load_script("pepatch.py")
|
||||
first, _second = fixtures.pe64.plant_offsets
|
||||
original = pillow.read_bytes()
|
||||
out = pillow.parent / "resized.exe"
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--at-offset", hex(first), "--expect", "48",
|
||||
"--bytes", "CC CC CC", "--allow-resize", "--out", str(out), "--json"])
|
||||
import json
|
||||
payload = json.loads(capsys.readouterr().out)
|
||||
assert code == 0
|
||||
assert payload["structural"] == ["cli"]
|
||||
assert payload["output_size"] == payload["input_size"] + 2
|
||||
written = out.read_bytes()
|
||||
assert len(written) == len(original) + 2
|
||||
assert written[first:first + 3] == b"\xCC\xCC\xCC"
|
||||
# One byte was replaced by three, so the tail moved two bytes along.
|
||||
assert written[first + 3:first + 14] == original[first + 1:first + 12]
|
||||
|
||||
|
||||
def test_failed_patches_stop_the_write(load_script, pillow, plant, monkeypatch, capsys):
|
||||
"""An ambiguous pattern must fail the run instead of patching the first match."""
|
||||
pepatch = load_script("pepatch.py")
|
||||
before = sha256(pillow)
|
||||
out = pillow.parent / "never.exe"
|
||||
code = run_main(pepatch, monkeypatch,
|
||||
[str(pillow), "--pattern", plant, "--bytes", "90 " * 12, "--out", str(out)])
|
||||
err = capsys.readouterr().err
|
||||
assert code == 1
|
||||
assert "refusing to write an output with failed patches" in err
|
||||
assert "matched 2 sites" in err
|
||||
assert not out.exists()
|
||||
assert sha256(pillow) == before
|
||||
@@ -0,0 +1,417 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/peutil.py.
|
||||
|
||||
Everything here runs against the generated fixtures, whose declared metadata (section RVAs,
|
||||
plant offsets, checksum) is asserted against the parser instead of being re-derived from it, so
|
||||
a parser change cannot make its own test pass.
|
||||
|
||||
The checksum check is deliberately a second implementation: if the test called the parser's own
|
||||
function to produce the expected value, a wrong algorithm would agree with itself.
|
||||
"""
|
||||
import hashlib
|
||||
import struct
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def ms_checksum(data, field_offset):
|
||||
"""The published Microsoft PE checksum, written independently of peutil.py."""
|
||||
buf = bytearray(data)
|
||||
buf[field_offset:field_offset + 4] = b"\x00\x00\x00\x00"
|
||||
total = 0
|
||||
for i in range(0, len(buf) - (len(buf) % 2), 2):
|
||||
total += buf[i] | (buf[i + 1] << 8)
|
||||
total = (total & 0xFFFF) + (total >> 16)
|
||||
if len(buf) % 2:
|
||||
total += buf[-1]
|
||||
total = (total & 0xFFFF) + (total >> 16)
|
||||
total = (total & 0xFFFF) + (total >> 16)
|
||||
return (total + len(buf)) & 0xFFFFFFFF
|
||||
|
||||
|
||||
# --- header fields -----------------------------------------------------------
|
||||
|
||||
def test_pe32plus_fixture_header_fields_match_the_recipe(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, str(fixtures.pe64.path))
|
||||
meta = fixtures.pe64.meta
|
||||
assert pe.is_pe32plus is True
|
||||
assert pe.magic == 0x20B
|
||||
assert pe.machine == 0x8664
|
||||
assert pe.arch == "amd64"
|
||||
assert pe.number_of_sections == 5
|
||||
assert pe.number_of_rva_and_sizes == 16
|
||||
assert [s.name for s in pe.sections] == [".text", ".rdata", ".data", ".rsrc", ".reloc"]
|
||||
assert pe.address_of_entrypoint == int(meta["entry_rva"], 16) == 0x1000
|
||||
assert pe.image_base == int(meta["image_base"], 16) == 0x140000000
|
||||
assert pe.size_of_image == meta["size_of_image"]
|
||||
assert pe.size_of_headers == meta["size_of_headers"]
|
||||
assert pe.section_alignment == 0x1000
|
||||
assert pe.file_alignment == 0x200
|
||||
assert pe.subsystem == 3
|
||||
assert pe.timestamp == 0x5F5E1000
|
||||
assert pe.file_size == meta["size"] == 4096
|
||||
assert pe.e_lfanew == 0x80
|
||||
|
||||
|
||||
def test_pe32_fixture_reports_the_32bit_layout(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe32.data, str(fixtures.pe32.path))
|
||||
assert pe.is_pe32plus is False
|
||||
assert pe.magic == 0x10B
|
||||
assert pe.machine == 0x014C
|
||||
assert pe.arch == "i386"
|
||||
assert pe.image_base == 0x400000
|
||||
assert pe.number_of_sections == 5
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name,plus", [("pe64", True), ("pe32", False)])
|
||||
def test_optional_header_field_offsets_follow_the_magic(load_script, fixtures, name, plus):
|
||||
"""The PE32 and PE32+ optional headers are different layouts, not different sizes."""
|
||||
pe = load_script("peutil.py").PE(fixtures[name].data, None)
|
||||
oh = pe.optional_offset
|
||||
assert pe._num_dir_off - oh == (108 if plus else 92)
|
||||
assert pe._dir_off - oh == (112 if plus else 96)
|
||||
assert pe.checksum == struct.unpack_from("<I", pe.d, oh + 64)[0]
|
||||
if plus:
|
||||
assert pe.image_base == struct.unpack_from("<Q", pe.d, oh + 24)[0]
|
||||
else:
|
||||
assert pe.image_base == struct.unpack_from("<I", pe.d, oh + 28)[0]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["pe64", "pe32"])
|
||||
def test_reading_the_other_layout_yields_a_wrong_directory_count(load_script, fixtures, name):
|
||||
"""The trap the fixture exists to expose: the wrong layout parses without error."""
|
||||
pe = load_script("peutil.py").PE(fixtures[name].data, None)
|
||||
wrong = pe._num_dir_off - pe.optional_offset
|
||||
other = 92 if wrong == 108 else 108
|
||||
misread = struct.unpack_from("<I", pe.d, pe.optional_offset + other)[0]
|
||||
assert pe.number_of_rva_and_sizes == 16
|
||||
assert misread != 16, "the two layouts are not distinguishable on this fixture"
|
||||
|
||||
|
||||
def test_characteristics_and_kind_derivations(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
assert pe.is_dll is False
|
||||
assert pe.is_driver is False
|
||||
assert pe.is_dotnet is False
|
||||
assert pe.is_signed is False
|
||||
assert pe.security_directory() is None
|
||||
assert "executable_image" in pe.summary()["file_characteristics"]
|
||||
assert "large_address_aware" in pe.summary()["file_characteristics"]
|
||||
|
||||
|
||||
def test_fixture_is_well_formed_so_no_warnings_are_raised(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
assert pe.warnings == []
|
||||
|
||||
|
||||
# --- RVA <-> file offset -----------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("section", [".text", ".rdata", ".data", ".rsrc", ".reloc"])
|
||||
def test_rva_to_file_offset_round_trips_every_section(load_script, fixtures, section):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
decl = fixtures.pe64.section(section)
|
||||
rva = int(decl["rva"], 16)
|
||||
raw = int(decl["raw_pointer"], 16)
|
||||
assert pe.rva_to_offset(rva) == raw
|
||||
assert pe.offset_to_rva(raw) == rva
|
||||
for delta in (1, decl["raw_size"] // 2, decl["raw_size"] - 1):
|
||||
assert pe.rva_to_offset(rva + delta) == raw + delta
|
||||
assert pe.offset_to_rva(raw + delta) == rva + delta
|
||||
assert pe.section_of_rva(rva).name == section
|
||||
|
||||
|
||||
def test_header_rvas_map_identity_below_size_of_headers(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
for rva in (0, 0x40, 0x98, pe.size_of_headers - 1):
|
||||
assert pe.rva_to_offset(rva) == rva
|
||||
assert pe.offset_to_rva(rva) == rva
|
||||
|
||||
|
||||
def test_virtual_only_tail_has_no_bytes_on_disk(load_script, fixtures):
|
||||
"""The .rdata tail is allocated in memory and zero-filled at load, so it has no file bytes."""
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
decl = fixtures.pe64.section(".rdata")
|
||||
rva = int(decl["rva"], 16)
|
||||
raw_size = decl["raw_size"]
|
||||
# Measured shape of this fixture: a 384-byte payload aligned up to a 512-byte raw span,
|
||||
# plus the recipe's 1024-byte virtual pad, gives 1408 virtual bytes -- so 896 of them have
|
||||
# no file image at all.
|
||||
assert decl["virtual_size"] - raw_size == 896
|
||||
assert fixtures.pe64.meta["virtual_only_tail_bytes"] == 1024
|
||||
# Last byte with a file image, then the first byte without one.
|
||||
assert pe.rva_to_offset(rva + raw_size - 1) == int(decl["raw_pointer"], 16) + raw_size - 1
|
||||
assert pe.rva_to_offset(rva + raw_size) is None
|
||||
assert pe.rva_to_offset(rva + decl["virtual_size"] - 1) is None
|
||||
assert pe.rva_bytes(rva + raw_size, 4) == b""
|
||||
assert pe.section_of_rva(rva + raw_size).name == ".rdata"
|
||||
# The RVA still exists, it just does not resolve to a file offset.
|
||||
assert pe.rva_to_offset(rva) is not None
|
||||
|
||||
|
||||
def test_recipe_without_a_virtual_tail_has_none(load_script, fixtures):
|
||||
"""A 4 KiB file alignment is large enough that every virtual byte has a file image."""
|
||||
pe = load_script("peutil.py").PE(fixtures.fatalign.data, None)
|
||||
decl = fixtures.fatalign.section(".rdata")
|
||||
rva = int(decl["rva"], 16)
|
||||
assert decl["virtual_size"] <= decl["raw_size"]
|
||||
assert pe.rva_to_offset(rva + decl["virtual_size"] - 1) is not None
|
||||
assert pe.rva_to_offset(rva + decl["raw_size"]) is None
|
||||
|
||||
|
||||
def test_offset_to_rva_returns_none_outside_every_mapping(load_script, fixtures):
|
||||
"""A negative outcome: an offset with no mapping must not be reported as RVA 0."""
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
assert pe.offset_to_rva(pe.file_size) is None
|
||||
assert pe.offset_to_rva(pe.file_size + 0x1000) is None
|
||||
|
||||
|
||||
def test_rva_to_offset_rejects_unmapped_rvas(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
assert pe.rva_to_offset(0x900000) is None
|
||||
assert pe.section_of_rva(0x900000) is None
|
||||
assert pe.read_cstring(None) == ""
|
||||
assert pe.read_cstring(pe.file_size + 1) == ""
|
||||
|
||||
|
||||
# --- checksum ----------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("name", ["pe64", "pe32", "bare", "fatalign"])
|
||||
def test_checksum_reproduces_the_declared_value(load_script, fixtures, name):
|
||||
built = fixtures[name]
|
||||
pe = load_script("peutil.py").PE(built.data, str(built.path))
|
||||
declared = int(built.meta["checksum"], 16)
|
||||
assert pe.checksum == declared
|
||||
assert pe.computed_checksum() == declared
|
||||
assert pe.checksum_ok() is True
|
||||
# Independent implementation of the same published algorithm.
|
||||
assert ms_checksum(built.data, int(built.meta["checksum_offset"], 16)) == declared
|
||||
|
||||
|
||||
def test_checksum_mismatch_is_reported_not_hidden(load_script, fixtures):
|
||||
data = bytearray(fixtures.pe64.data)
|
||||
data[int(fixtures.pe64.meta["checksum_offset"], 16)] ^= 0xFF # corrupt one byte
|
||||
pe = load_script("peutil.py").PE(bytes(data), None)
|
||||
assert pe.checksum_ok() is False
|
||||
assert pe.computed_checksum() != pe.checksum
|
||||
|
||||
|
||||
# --- structures --------------------------------------------------------------
|
||||
|
||||
def test_resource_walk_returns_exactly_the_declared_entries(load_script, fixtures):
|
||||
"""type(16=VERSION) -> name(1) -> language(1033) -> one data entry, and nothing else.
|
||||
|
||||
A 32-bit read of either directory count would swallow the next entry and inflate this to a
|
||||
much larger number, which is why the exact count is asserted rather than "at least one".
|
||||
"""
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
entries = pe.resources()
|
||||
assert len(entries) == 1
|
||||
entry = entries[0]
|
||||
assert entry["path"] == ["VERSION", "1", "1033"]
|
||||
assert entry["size"] == 4
|
||||
assert entry["codepage"] == 1200
|
||||
assert int(entry["data_rva"], 16) == int(fixtures.pe64.section(".rsrc")["rva"], 16) + 0x90
|
||||
assert pe.rva_bytes(int(entry["data_rva"], 16), 4) == b"FIXT"
|
||||
|
||||
|
||||
def test_recipe_without_resources_reports_an_empty_tree(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
|
||||
assert pe.resources() == []
|
||||
|
||||
|
||||
def test_imports_include_one_named_import_and_one_by_ordinal(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
imports = pe.imports()
|
||||
assert len(imports) == 1
|
||||
desc = imports[0]
|
||||
assert desc["dll"] == fixtures.pe64.meta["import_dll"] == "fixture-import.dll"
|
||||
assert desc["descriptor_rva"] == int(fixtures.pe64.meta["import_descriptor_rva"], 16)
|
||||
funcs = desc["functions"]
|
||||
assert len(funcs) == 2, "the descriptor terminator must stop the walk"
|
||||
named = funcs[0]
|
||||
assert named["name"] == fixtures.pe64.meta["import_named"] == "ImportedFunction"
|
||||
assert named["hint"] == 7
|
||||
assert "ordinal" not in named
|
||||
by_ordinal = funcs[1]
|
||||
assert by_ordinal["ordinal"] == int(fixtures.pe64.meta["import_ordinal"], 16) == 0x2A
|
||||
assert "name" not in by_ordinal
|
||||
assert funcs[1]["iat_rva"] > funcs[0]["iat_rva"]
|
||||
assert pe.delay_imports() == []
|
||||
|
||||
|
||||
@pytest.mark.parametrize("name", ["pe64", "pe32"])
|
||||
def test_export_directory_has_one_named_function(load_script, fixtures, name):
|
||||
pe = load_script("peutil.py").PE(fixtures[name].data, None)
|
||||
exports = pe.exports()
|
||||
assert exports["dll_name"] == "fixture-export.dll"
|
||||
assert exports["ordinal_base"] == 1
|
||||
assert exports["number_of_functions"] == 1
|
||||
assert exports["number_of_names"] == 1
|
||||
funcs = exports["functions"]
|
||||
assert len(funcs) == 1
|
||||
assert funcs[0]["name"] == "ExportedFunction"
|
||||
assert funcs[0]["ordinal"] == 1
|
||||
assert funcs[0]["rva"] == "0x1000"
|
||||
assert "forwarder" not in funcs[0]
|
||||
|
||||
|
||||
def test_recipe_without_exports_returns_none(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
|
||||
assert pe.exports() is None
|
||||
|
||||
|
||||
def test_tls_callback_rva_is_reachable(load_script, fixtures):
|
||||
"""The callback pointer is stored as a VA, so the reported RVA needs image-base subtraction."""
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
tls = pe.tls()
|
||||
assert tls is not None
|
||||
assert tls["callback_rvas"] == [fixtures.pe64.meta["entry_rva"]] == ["0x1000"]
|
||||
assert tls["callbacks_rva"] == fixtures.pe64.meta["tls_callback_rva"]
|
||||
assert tls["note"], "a callback that runs before the entry point must be flagged"
|
||||
assert pe.imports() and tls["callbacks_rva"] != fixtures.pe64.meta["import_descriptor_rva"]
|
||||
|
||||
|
||||
def test_recipe_without_tls_reports_none(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
|
||||
assert pe.tls() is None
|
||||
|
||||
|
||||
def test_debug_record_carries_the_rsds_pdb_path(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
records = pe.debug_directory()
|
||||
assert len(records) == 1
|
||||
rec = records[0]
|
||||
assert rec["type"] == "codeview" and rec["type_id"] == 2
|
||||
assert rec["format"] == "RSDS"
|
||||
assert rec["pdb"] == fixtures.pe64.meta["pdb_path"] == "C:\\build\\fixture.pdb"
|
||||
assert rec["age"] == 1
|
||||
assert rec["guid"] == hashlib.md5(bytes(range(16))).hexdigest()
|
||||
# PointerToRawData is a file offset while AddressOfRawData is an RVA: prove the pair agrees.
|
||||
rsds_rva = int(fixtures.pe64.meta["rdata_structure_rvas"]["dbg_rsds"], 16)
|
||||
assert pe.rva_to_offset(rsds_rva) == int(rec["pointer_to_raw_data"], 16)
|
||||
assert pe.d[pe.rva_to_offset(rsds_rva):pe.rva_to_offset(rsds_rva) + 4] == b"RSDS"
|
||||
|
||||
|
||||
def test_recipe_without_debug_directory_reports_no_records(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.bare.data, None)
|
||||
assert pe.debug_directory() == []
|
||||
|
||||
|
||||
def test_base_relocations_block_is_parsed_by_type(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
relocs = pe.base_relocations()
|
||||
assert relocs["blocks"] == 1
|
||||
assert relocs["entries"] == 2
|
||||
assert relocs["truncated"] is False
|
||||
assert relocs["types"] == {"3": 1, "10": 1}
|
||||
|
||||
|
||||
def test_absent_load_config_is_reported_as_none(load_script, fixtures):
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, None)
|
||||
assert pe.load_config() is None
|
||||
assert pe.rich_header() is None, "the fixture has no Rich header; none may be invented"
|
||||
|
||||
|
||||
def test_summary_is_json_serialisable_and_names_the_documented_keys(load_script, fixtures):
|
||||
import json
|
||||
pe = load_script("peutil.py").PE(fixtures.pe64.data, str(fixtures.pe64.path))
|
||||
summary = pe.summary()
|
||||
text = json.dumps(summary)
|
||||
assert '"checksum_matches": true' in text
|
||||
for key in ("path", "file_size", "sha256", "arch", "kind", "sections", "directories",
|
||||
"checksum_declared", "checksum_computed", "checksum_matches", "warnings"):
|
||||
assert key in summary
|
||||
assert summary["sha256"] == hashlib.sha256(fixtures.pe64.data).hexdigest()
|
||||
|
||||
|
||||
# --- pure helpers ------------------------------------------------------------
|
||||
|
||||
def test_entropy_bounds(load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
assert peutil.entropy(b"") == 0.0
|
||||
assert peutil.entropy(b"\x00" * 256) == 0.0
|
||||
assert peutil.entropy(bytes(range(256))) == pytest.approx(8.0, abs=1e-9)
|
||||
assert 0.0 <= peutil.entropy(bytes(range(128)) * 2) <= 8.0
|
||||
|
||||
|
||||
def test_section_align_is_an_index_not_a_set_of_bits(load_script):
|
||||
"""Bits [23:20] hold an alignment index, so a normal image reports no alignment name."""
|
||||
peutil = load_script("peutil.py")
|
||||
assert peutil.section_align(0x00600000) == "align_32"
|
||||
assert peutil.section_align(0xE0000020) is None # the fixture's .text
|
||||
assert peutil.section_align(0x60000020) is None # a normal compiler's .text
|
||||
|
||||
|
||||
def test_flag_decoding_is_ordered_and_complete(load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
assert peutil.bits(0x0022, peutil.FILE_CHARACTERISTICS) == ["executable_image",
|
||||
"large_address_aware"]
|
||||
assert peutil.bits(0, peutil.FILE_CHARACTERISTICS) == []
|
||||
assert "mem_execute" in peutil.bits(0xE0000020, peutil.SECTION_CHARACTERISTICS)
|
||||
assert "mem_write" in peutil.bits(0xE0000020, peutil.SECTION_CHARACTERISTICS)
|
||||
|
||||
|
||||
def test_integer_readers_are_little_endian(load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
blob = b"\x01\x02\x03\x04\x05\x06\x07\x08"
|
||||
assert peutil.u8(blob, 0) == 0x01
|
||||
assert peutil.u16(blob, 0) == 0x0201
|
||||
assert peutil.u32(blob, 0) == 0x04030201
|
||||
assert peutil.u64(blob, 0) == 0x0807060504030201
|
||||
assert peutil.s32(b"\xFF\xFF\xFF\xFF", 0) == -1
|
||||
|
||||
|
||||
# --- rejection ---------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("blob,expected", [
|
||||
(b"", "no MZ signature"),
|
||||
(b"MZ", "no MZ signature"),
|
||||
(b"this is not a PE file at all, just text" * 4, "no MZ signature"),
|
||||
])
|
||||
def test_parse_error_on_non_pe_input(load_script, blob, expected):
|
||||
peutil = load_script("peutil.py")
|
||||
with pytest.raises(peutil.ParseError, match=expected):
|
||||
peutil.PE(blob, None)
|
||||
|
||||
|
||||
def test_parse_error_when_e_lfanew_points_past_end_of_file(load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
blob = b"MZ" + b"\x00" * 0x3A + struct.pack("<I", 0xFFFFFF) + b"\x00" * 8
|
||||
with pytest.raises(peutil.ParseError, match="points past end of file"):
|
||||
peutil.PE(blob, None)
|
||||
|
||||
|
||||
def test_parse_error_without_the_pe_signature(load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
blob = b"MZ" + b"\x00" * 0x3A + struct.pack("<I", 0x40) + b"\x00" * 0x40
|
||||
with pytest.raises(peutil.ParseError, match=r"no PE\\0\\0 signature"):
|
||||
peutil.PE(blob, None)
|
||||
|
||||
|
||||
def test_parse_error_on_an_unexpected_optional_header_magic(load_script, fixtures):
|
||||
peutil = load_script("peutil.py")
|
||||
data = bytearray(fixtures.pe64.data)
|
||||
struct.pack_into("<H", data, 0x98, 0x999)
|
||||
with pytest.raises(peutil.ParseError, match="unexpected optional-header magic 0x0999"):
|
||||
peutil.PE(bytes(data), None)
|
||||
|
||||
|
||||
def test_parse_error_on_an_object_file_with_no_optional_header(load_script, fixtures):
|
||||
peutil = load_script("peutil.py")
|
||||
data = bytearray(fixtures.pe64.data)
|
||||
struct.pack_into("<H", data, 0x80 + 4 + 16, 0)
|
||||
with pytest.raises(peutil.ParseError, match="object file, not an image"):
|
||||
peutil.PE(bytes(data), None)
|
||||
|
||||
|
||||
def test_load_rejects_a_missing_path(tmp_path, load_script):
|
||||
peutil = load_script("peutil.py")
|
||||
with pytest.raises(OSError):
|
||||
peutil.load(str(tmp_path / "absent.exe"))
|
||||
|
||||
|
||||
def test_load_sets_the_path_on_the_parsed_image(load_script, fixtures):
|
||||
pe = load_script("peutil.py").load(str(fixtures.pe64.path))
|
||||
assert pe.path == str(fixtures.pe64.path)
|
||||
assert pe.file_size == fixtures.pe64.size
|
||||
@@ -0,0 +1,271 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/scan_leaks.py.
|
||||
|
||||
Two failure modes are tested at once, because only one of them is obvious. A scanner that misses
|
||||
a pasted key is dangerous; a scanner that flags `x64dbg`, `PE32+` or a reserved documentation
|
||||
address gets switched off, which is equally dangerous and much easier to do. So every rule is
|
||||
asserted to fire on a shaped sample, and every exemption class is asserted to stay quiet.
|
||||
|
||||
Three measured observations are pinned here rather than hidden, because each is a false negative
|
||||
or a dead branch in the rule table:
|
||||
|
||||
* the bare-GUID exemption matches inside the machine-guid rule's own match text, so that rule
|
||||
cannot produce a finding;
|
||||
* the `.NET` entry in the platform list matches any `.net` TLD, so a hit inside such a URL or
|
||||
address is suppressed as a platform term;
|
||||
* the ``<[a-z_]+>`` placeholder alternative is wrapped in ``\\b`` on both sides, so it cannot
|
||||
match bracketed prose.
|
||||
|
||||
The shaped samples come from the corpus in conftest.py, which assembles them from fragments so
|
||||
that this suite does not itself trip the scan it tests.
|
||||
"""
|
||||
import pathlib
|
||||
|
||||
import pytest
|
||||
|
||||
# The rules the exemption list structurally suppresses: a hit exists, and an exemption claims
|
||||
# it, so the rule cannot produce a finding. Measured, and re-asserted below so a fix shows up as
|
||||
# a test change rather than as a quietly gained finding.
|
||||
SUPPRESSED_RULES = ("machine-guid",)
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def findings_in(scan_leaks, text):
|
||||
"""What main() would report: hits that no exemption suppresses."""
|
||||
out = []
|
||||
for hit in scan_leaks.scan_text(text):
|
||||
if scan_leaks.exempt_reason(hit["match"]) or scan_leaks.exempt_reason(hit["context"]):
|
||||
continue
|
||||
out.append(hit)
|
||||
return out
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def corpus(leak_lines):
|
||||
return dict(leak_lines)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def exemption_corpus(exempt_lines):
|
||||
return dict(exempt_lines)
|
||||
|
||||
|
||||
# --- every rule fires --------------------------------------------------------
|
||||
|
||||
def test_every_rule_fires_on_its_shaped_sample(load_script, leak_lines):
|
||||
scan = load_script("scan_leaks.py")
|
||||
missing = []
|
||||
for rule, line in leak_lines:
|
||||
fired = {hit["rule"] for hit in scan.scan_text(line)}
|
||||
if rule not in fired:
|
||||
missing.append((rule, sorted(fired)))
|
||||
assert missing == []
|
||||
|
||||
|
||||
def test_every_rule_except_the_suppressed_ones_is_reportable(load_script, leak_lines):
|
||||
scan = load_script("scan_leaks.py")
|
||||
suppressed = set(SUPPRESSED_RULES)
|
||||
unexpected = []
|
||||
for rule, line in leak_lines:
|
||||
reported = {hit["rule"] for hit in findings_in(scan, line)}
|
||||
if rule in suppressed:
|
||||
if rule in reported:
|
||||
unexpected.append((rule, "expected to be suppressed by the exemption list"))
|
||||
elif rule not in reported:
|
||||
unexpected.append((rule, "not reported"))
|
||||
assert unexpected == []
|
||||
assert suppressed == {"machine-guid"}
|
||||
|
||||
|
||||
def test_the_rule_table_is_well_formed(load_script):
|
||||
scan = load_script("scan_leaks.py")
|
||||
names = [name for _c, _s, name, _p, _w in scan.RULES]
|
||||
assert len(names) == len(set(names)) == len(scan.RULES) == 18
|
||||
assert {strength for _c, strength, _n, _p, _w in scan.RULES} == {"strong", "weak"}
|
||||
for category, strength, name, pattern, why in scan.RULES:
|
||||
assert category and name and why, (category, name)
|
||||
assert strength in ("strong", "weak")
|
||||
assert pattern.pattern, name
|
||||
|
||||
|
||||
# --- the three measured deviations ------------------------------------------
|
||||
|
||||
def test_machine_guid_rule_fires(load_script, corpus):
|
||||
scan = load_script("scan_leaks.py")
|
||||
hits = list(scan.scan_text(corpus["machine-guid"]))
|
||||
assert [h["rule"] for h in hits] == ["machine-guid"]
|
||||
|
||||
|
||||
def test_machine_guid_finding_is_suppressed_by_the_bare_guid_exemption(load_script, corpus):
|
||||
"""Measured: the bare-GUID exemption matches inside this rule's own match text.
|
||||
|
||||
Recorded here so that a change to either list shows up as a test change instead of as a
|
||||
silently gained or lost finding.
|
||||
"""
|
||||
scan = load_script("scan_leaks.py")
|
||||
line = corpus["machine-guid"]
|
||||
hit = list(scan.scan_text(line))[0]
|
||||
assert "bare GUID is not machine identity" in scan.exempt_reason(hit["match"])
|
||||
assert findings_in(scan, line) == []
|
||||
|
||||
|
||||
def test_a_url_finding_is_not_swallowed_by_an_exemption(load_script, corpus):
|
||||
"""A concrete host must still be reported, and only as the weak finding the rule declares.
|
||||
|
||||
Measured, twice: while this suite was being written, a standard-name exemption alternative
|
||||
matched the `HTTP` inside every `https?://` URL and swallowed the rule's every hit. The
|
||||
assertion below is the state after that was corrected -- a URL hit is reported, at weak
|
||||
strength, with no exemption claiming it. `benchmark.md` records both states.
|
||||
"""
|
||||
scan = load_script("scan_leaks.py")
|
||||
line = corpus["literal-host-with-path"]
|
||||
hits = list(scan.scan_text(line))
|
||||
assert [h["rule"] for h in hits] == ["literal-host-with-path"]
|
||||
assert hits[0]["strength"] == "weak"
|
||||
assert scan.exempt_reason(hits[0]["match"]) is None
|
||||
reported = findings_in(scan, line)
|
||||
assert [hit["rule"] for hit in reported] == ["literal-host-with-path"]
|
||||
|
||||
|
||||
def test_a_net_domain_hides_a_hit_behind_the_platform_exemption(load_script, corpus,
|
||||
exemption_corpus):
|
||||
"""Measured deviation: `.NET` in the platform list matches any `.net` TLD.
|
||||
|
||||
A bare domain under `.net` is therefore read as a platform term while the same domain under
|
||||
another TLD is not, which is a false negative rather than a rule. Pinned, not endorsed.
|
||||
"""
|
||||
scan = load_script("scan_leaks.py")
|
||||
net_url = exemption_corpus["net tld"]
|
||||
org_url = corpus["literal-host-with-path"]
|
||||
assert net_url.endswith(".net/v2/activate") and org_url.endswith(".org/v2/activate")
|
||||
assert "platform or vendor" in scan.exempt_reason(net_url)
|
||||
# Isolate the cause to the TLD: the same names under another TLD claim no platform exemption.
|
||||
assert scan.exempt_reason("licensing.hexweld.net") is not None
|
||||
assert scan.exempt_reason("licensing.hexweld.org") is None
|
||||
|
||||
|
||||
def test_the_bracketed_placeholder_form_is_unreachable_in_prose(load_script):
|
||||
"""Measured: `<[a-z_]+>` sits inside `\\b...\\b`, so bracketed prose never matches."""
|
||||
scan = load_script("scan_leaks.py")
|
||||
assert scan.exempt_reason("<user>") is None
|
||||
assert scan.exempt_reason("a <user> b") is None
|
||||
assert scan.PLACEHOLDER_HINT.search("a <user> b") is None
|
||||
# Only word-character adjacency reaches it, which is why a bare TARGET is the form that works.
|
||||
assert scan.PLACEHOLDER_HINT.search("x<user>x") is not None
|
||||
assert scan.exempt_reason("TARGET") == "placeholder token"
|
||||
|
||||
|
||||
# --- exemptions --------------------------------------------------------------
|
||||
|
||||
@pytest.mark.parametrize("sample,reason", [
|
||||
("x64dbg", "tool or library name"),
|
||||
("WinDbg", "tool or library name"),
|
||||
("IDA", "tool or library name"),
|
||||
("CFF Explorer", "tool or library name"),
|
||||
("the copy is packed with UPX", "public protector/packer/managed tooling name"),
|
||||
("Themida", "public protector/packer/managed tooling name"),
|
||||
("IMAGE_NT_HEADERS", "format, standard or platform term"),
|
||||
("CWE-119", "format, standard or platform term"),
|
||||
("Authenticode", "format, standard or platform term"),
|
||||
("Windows", "platform or vendor-of-the-platform term"),
|
||||
("PowerShell", "platform or vendor-of-the-platform term"),
|
||||
("TARGET", "placeholder token"),
|
||||
("YOURAPP", "placeholder token"),
|
||||
("REDACTED", "placeholder token"),
|
||||
("x<user>x", "placeholder token"),
|
||||
("192.0.2.44", "reserved documentation address"),
|
||||
("198.51.100.7", "reserved documentation address"),
|
||||
("203.0.113.9", "reserved documentation address"),
|
||||
("127.0.0.1", "reserved documentation address"),
|
||||
("S-1-5-18", "well-known SID"),
|
||||
("3f2a1b4c-5d6e-7f80-9a1b-2c3d4e5f6071", "bare GUID is not machine identity"),
|
||||
])
|
||||
def test_exempt_reason_classifies_the_classes_that_must_stay(load_script, sample, reason):
|
||||
"""The reason is asserted by the class it names, not by its exact prose.
|
||||
|
||||
The wording of a reason is a message, and messages get reworded; which exemption class
|
||||
claims a sample is the behaviour under test.
|
||||
"""
|
||||
assert reason in load_script("scan_leaks.py").exempt_reason(sample)
|
||||
|
||||
|
||||
def test_exempt_reason_does_not_excuse_a_real_finding(load_script, corpus):
|
||||
"""A negative outcome: the exemption list must not swallow target identity."""
|
||||
scan = load_script("scan_leaks.py")
|
||||
for rule in ("private-key-block", "github-token", "windows-user-profile", "posix-home",
|
||||
"windows-sid-literal", "volume-serial-literal", "product-name-with-version"):
|
||||
assert scan.exempt_reason(corpus[rule]) is None, rule
|
||||
|
||||
|
||||
@pytest.mark.parametrize("label", [
|
||||
"tool name", "protector name", "format term", "platform term", "placeholder token",
|
||||
"documentation address", "well-known SID", "loopback and example", "machine guid context",
|
||||
"net tld",
|
||||
])
|
||||
def test_exemption_classes_are_never_reported(load_script, exemption_corpus, label):
|
||||
assert findings_in(load_script("scan_leaks.py"), exemption_corpus[label]) == [], label
|
||||
|
||||
|
||||
# --- scan_text ---------------------------------------------------------------
|
||||
|
||||
def test_scan_text_reports_position_and_strength(load_script, corpus):
|
||||
scan = load_script("scan_leaks.py")
|
||||
text = "line one is clean\nsecond line: " + corpus["private-key-block"] + "\n"
|
||||
hits = list(scan.scan_text(text))
|
||||
assert len(hits) == 1
|
||||
hit = hits[0]
|
||||
assert hit["line"] == 2
|
||||
assert hit["col"] == 14
|
||||
assert hit["strength"] == "strong"
|
||||
assert hit["category"] == "token"
|
||||
assert hit["rule"] == "private-key-block"
|
||||
assert hit["why"]
|
||||
assert corpus["private-key-block"] in hit["match"]
|
||||
assert hit["context"].startswith("second line:")
|
||||
|
||||
|
||||
def test_clean_text_produces_no_hits_at_all(load_script):
|
||||
"""A negative outcome: a clean note must not produce a single hit of any strength."""
|
||||
scan = load_script("scan_leaks.py")
|
||||
text = ("## Triage\n\nRead the section table with peutil.py and printed the entropy.\n"
|
||||
"The checksum matched the declared value, so the image was not hand-edited.\n")
|
||||
assert list(scan.scan_text(text)) == []
|
||||
|
||||
|
||||
def test_scan_text_does_not_report_the_same_line_as_a_different_rule(load_script, corpus):
|
||||
scan = load_script("scan_leaks.py")
|
||||
hits = scan.scan_text(corpus["generic-secret-assignment"])
|
||||
assert [h["rule"] for h in hits] == ["generic-secret-assignment"]
|
||||
|
||||
|
||||
# --- file selection ----------------------------------------------------------
|
||||
|
||||
def test_iter_files_skips_binaries_and_cache_directories(load_script, tmp_path, corpus):
|
||||
"""A negative outcome: the scanner must not walk binaries, caches or VCS metadata."""
|
||||
scan = load_script("scan_leaks.py")
|
||||
root = tmp_path / "repo"
|
||||
(root / "__pycache__").mkdir(parents=True)
|
||||
(root / ".git").mkdir()
|
||||
(root / "notes.md").write_text("clean\n", encoding="utf-8")
|
||||
(root / "trap.exe").write_bytes(corpus["private-key-block"].encode("ascii") + b"\n")
|
||||
(root / "trap.png").write_bytes(corpus["github-token"].encode("ascii"))
|
||||
(root / "__pycache__" / "cached.md").write_text(corpus["private-key-block"] + "\n",
|
||||
encoding="utf-8")
|
||||
(root / ".git" / "config.md").write_text(corpus["private-key-block"] + "\n",
|
||||
encoding="utf-8")
|
||||
found = sorted(pathlib.Path(p).relative_to(root).as_posix()
|
||||
for p in scan.iter_files(str(root)))
|
||||
assert found == ["notes.md"]
|
||||
|
||||
|
||||
def test_iter_files_honours_an_explicit_file_list(load_script, tmp_path):
|
||||
scan = load_script("scan_leaks.py")
|
||||
root = tmp_path / "repo"
|
||||
root.mkdir()
|
||||
(root / "a.md").write_text("a\n", encoding="utf-8")
|
||||
(root / "b.md").write_text("b\n", encoding="utf-8")
|
||||
listing = tmp_path / "listed.txt"
|
||||
listing.write_text("a.md\nmissing.md\n\n", encoding="utf-8")
|
||||
found = sorted(pathlib.Path(p).relative_to(root).as_posix()
|
||||
for p in scan.iter_files(str(root), str(listing)))
|
||||
assert found == ["a.md"]
|
||||
@@ -0,0 +1,185 @@
|
||||
"""Unit tests for skills/exe-reverse/scripts/trace_reduce.py, the bit-vector reducer.
|
||||
|
||||
Three layers, each held separately because they make different promises:
|
||||
|
||||
* the rewriting core -- the classic mixed-boolean-arithmetic substitutions reduce to `x + y` and
|
||||
survive a sampling check, and a genuinely different pair is caught with a counterexample;
|
||||
* the SMT emitter -- pure text, checked exactly, because the solver will execute it verbatim and a
|
||||
proof about wrongly-encoded semantics is worse than no proof. The shift-count mask is the case
|
||||
that matters: `evaluate()` masks a count to six bits, and the encoding must do the same or the
|
||||
solver proves a cleaner function than the one the samples ran on;
|
||||
* the proof path -- driven against a stub solver (one line of output) and against a missing path,
|
||||
never against a real solver: CI has none, and a test that needs one is a test that skips.
|
||||
"""
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
pytestmark = pytest.mark.unit
|
||||
|
||||
|
||||
def parse(tr, text):
|
||||
return tr.Parser(tr.tokenize(text)).parse()
|
||||
|
||||
|
||||
# --- the rewriting core ---------------------------------------------------------
|
||||
|
||||
|
||||
def test_the_two_classic_substitutions_reduce_to_add(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
for obfuscated in ("(x | y) + (x & y)", "(x ^ y) + 2 * (x & y)"):
|
||||
tree = parse(tr, obfuscated)
|
||||
reduced, steps = tr.reduce_all(tree, tr.MASK[32])
|
||||
assert reduced.text() == "x + y"
|
||||
assert steps, "a reduction with no recorded steps changed nothing"
|
||||
equal, counter, _ = tr.verify(tree, reduced, tr.MASK[32], 64)
|
||||
assert equal, "counterexample: %r" % (counter,)
|
||||
|
||||
|
||||
def test_a_genuinely_different_pair_is_refuted_with_a_witness(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
original, other = parse(tr, "x + y"), parse(tr, "x | y")
|
||||
equal, counter, checked = tr.verify(original, other, tr.MASK[32], 64)
|
||||
assert checked > 0
|
||||
assert not equal
|
||||
# The witness is re-checked here rather than trusted: the point of a counterexample is that it
|
||||
# can be substituted by hand.
|
||||
env = counter["env"]
|
||||
assert tr.evaluate(original, env, tr.MASK[32]) != tr.evaluate(other, env, tr.MASK[32])
|
||||
|
||||
|
||||
# --- the SMT emitter ------------------------------------------------------------
|
||||
|
||||
|
||||
def test_a_constant_is_emitted_at_the_full_word_width(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
term = parse(tr, "0x2a")
|
||||
assert tr.smt2_term(term, 32) == "#x0000002a"
|
||||
assert tr.smt2_term(term, 8) == "#x2a"
|
||||
|
||||
|
||||
def test_every_operator_maps_to_its_bit_vector_counterpart(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
assert tr.smt2_term(parse(tr, "x + y"), 32) == "(bvadd x y)"
|
||||
assert tr.smt2_term(parse(tr, "x - y"), 32) == "(bvsub x y)"
|
||||
assert tr.smt2_term(parse(tr, "x * y"), 32) == "(bvmul x y)"
|
||||
assert tr.smt2_term(parse(tr, "x & y"), 32) == "(bvand x y)"
|
||||
assert tr.smt2_term(parse(tr, "x | y"), 32) == "(bvor x y)"
|
||||
assert tr.smt2_term(parse(tr, "x ^ y"), 32) == "(bvxor x y)"
|
||||
assert tr.smt2_term(parse(tr, "~x"), 32) == "(bvnot x)"
|
||||
assert tr.smt2_term(parse(tr, "-x"), 32) == "(bvneg x)"
|
||||
|
||||
|
||||
def test_a_shift_count_is_masked_exactly_as_evaluate_masks_it(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
# evaluate() shifts by `count & 63`; the encoding must assert the same thing, or the solver
|
||||
# proves a different function than the one the samples ran on.
|
||||
assert tr.smt2_term(parse(tr, "x << y"), 32) == "(bvshl x (bvand y #x0000003f))"
|
||||
assert tr.smt2_term(parse(tr, "x >> y"), 64) == "(bvlshr x (bvand y #x000000000000003f))"
|
||||
|
||||
|
||||
def test_the_script_declares_variables_asserts_a_difference_and_asks(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
script = tr.smt2_for(parse(tr, "y + x"), parse(tr, "x + y"), 32)
|
||||
lines = script.strip().splitlines()
|
||||
assert lines[0] == "(set-logic QF_BV)"
|
||||
assert "(declare-fun x () (_ BitVec 32))" in lines
|
||||
assert "(declare-fun y () (_ BitVec 32))" in lines
|
||||
assert any(line.startswith("(assert (distinct") for line in lines)
|
||||
assert "(check-sat)" in lines
|
||||
assert "(get-model)" in lines
|
||||
|
||||
|
||||
def test_a_model_is_read_into_an_environment_and_masked(load_script):
|
||||
tr = load_script("trace_reduce.py")
|
||||
model = """
|
||||
sat
|
||||
(
|
||||
(define-fun x () (_ BitVec 32) #x00000001)
|
||||
(define-fun y () (_ BitVec 32) #x00000003)
|
||||
(define-fun ignored () Int 7)
|
||||
)
|
||||
"""
|
||||
assert tr.model_env(model, tr.MASK[32]) == {"x": 1, "y": 3}
|
||||
assert tr.model_env("unsat", tr.MASK[32]) == {}
|
||||
|
||||
|
||||
# --- the proof path, against stubs ----------------------------------------------
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def unsat_solver(tmp_path):
|
||||
"""A stand-in solver that answers `unsat`: one line, no matter the question."""
|
||||
if sys.platform == "win32":
|
||||
path = tmp_path / "unsat.bat"
|
||||
path.write_text("@echo unsat\r\n", encoding="ascii")
|
||||
else:
|
||||
path = tmp_path / "unsat.sh"
|
||||
path.write_text("#!/bin/sh\necho unsat\n", encoding="ascii")
|
||||
path.chmod(0o755)
|
||||
return str(path)
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def lying_sat_solver(tmp_path):
|
||||
"""A stand-in that says `sat` and shows a witness that does not distinguish the forms.
|
||||
|
||||
The reducer's own rules are identities, so a real sat never appears in a run of this tool; a
|
||||
lying one is the case that must not be trusted on arrival, and this stub is how that is held.
|
||||
"""
|
||||
body = ("sat\n(\n(define-fun x () (_ BitVec 32) #x00000000)\n"
|
||||
"(define-fun y () (_ BitVec 32) #x00000000)\n)\n")
|
||||
if sys.platform == "win32":
|
||||
path = tmp_path / "sat.bat"
|
||||
path.write_text("@echo sat\r\n@echo (\r\n@echo (define-fun x () (_ BitVec 32) "
|
||||
"#x00000000)\r\n@echo (define-fun y () (_ BitVec 32) #x00000000)\r\n"
|
||||
"@echo )\r\n", encoding="ascii")
|
||||
else:
|
||||
path = tmp_path / "sat.sh"
|
||||
path.write_text("#!/bin/sh\ncat <<'EOF'\n%s\nEOF\n" % body, encoding="ascii")
|
||||
path.chmod(0o755)
|
||||
return str(path)
|
||||
|
||||
|
||||
def test_an_unsat_answer_upgrades_the_verdict_to_proved(load_script, unsat_solver):
|
||||
tr = load_script("trace_reduce.py")
|
||||
original, reduced = parse(tr, "(x ^ y) + 2 * (x & y)"), parse(tr, "x + y")
|
||||
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32, unsat_solver, 10)
|
||||
assert verdict == "proved"
|
||||
|
||||
|
||||
def test_a_witness_that_does_not_distinguish_is_not_believed(load_script, lying_sat_solver):
|
||||
tr = load_script("trace_reduce.py")
|
||||
original, reduced = parse(tr, "(x ^ y) + 2 * (x & y)"), parse(tr, "x + y")
|
||||
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32, lying_sat_solver, 10)
|
||||
assert verdict == "no-verdict"
|
||||
assert "does not distinguish" in info["reason"]
|
||||
|
||||
|
||||
def test_a_missing_solver_path_is_reported_not_crashed(load_script, tmp_path):
|
||||
tr = load_script("trace_reduce.py")
|
||||
original, reduced = parse(tr, "x + y"), parse(tr, "x + y")
|
||||
verdict, info = tr.prove(original, reduced, tr.MASK[32], 32,
|
||||
str(tmp_path / "not-there.exe"), 10)
|
||||
assert verdict == "no-verdict"
|
||||
assert "does not exist" in info["reason"]
|
||||
|
||||
|
||||
def test_prove_with_a_missing_path_is_a_usage_error_at_the_cli(load_script, monkeypatch, capsys,
|
||||
tmp_path):
|
||||
tr = load_script("trace_reduce.py")
|
||||
monkeypatch.setattr(sys, "argv", ["trace_reduce.py", "x + 0",
|
||||
"--prove", str(tmp_path / "absent.exe")])
|
||||
assert tr.main() == 2
|
||||
out = capsys.readouterr()
|
||||
assert "RESULT=usage_error" in out.out
|
||||
|
||||
|
||||
def test_a_proved_run_reports_the_proved_token(load_script, monkeypatch, capsys, unsat_solver):
|
||||
tr = load_script("trace_reduce.py")
|
||||
monkeypatch.setattr(sys, "argv", ["trace_reduce.py", "(x ^ y) + 2 * (x & y)",
|
||||
"--prove", unsat_solver])
|
||||
assert tr.main() == 0
|
||||
out = capsys.readouterr().out
|
||||
assert "RESULT=reduce_proved" in out
|
||||
assert "every input" in out
|
||||
Reference in New Issue
Block a user