Audit Who Writes Your Agent’s Rules
A coding agent’s explanation is not the approval record for a new dependency. Check who supplied the rules and who approved the package.
The PackHallu preprint, published on 7 October 2026, tested poisoned instruction files that redirected dependency choices. Its controlled setup used a local PyPI mirror, a harmless marker and disabled manual confirmations.
The defensive question begins before installation. Who supplied the instructions that the agent treats as project policy, and who is allowed to change a dependency name, source or lockfile?
A rule file may look like documentation, but an agent can use it to shape code and tool decisions. Once the workflow can resolve and install packages, prose has reached a software supply-chain boundary.
The percentage needs a denominator
Across five models running OpenHands on BigCodeBench, the study reports averages of 79.29% syntactic and 67.68% deployable success. These are not eight-framework, thirteen-model or production-task rates.
Syntactic success meant importing the substituted package; deployable success also required execution and invocation. Their eligible baselines required, respectively, the intended import and successful benign execution. Deployable results fell to 45.00% on DS-1000 and 16.28% on RefactorBench.
The decline matters. A strong result in a file-level coding benchmark does not become a universal success rate for complex repositories, every agent stack or supervised work. The test shows a credible mechanism and where it weakened.
A detector cannot make the approval decision
The authors paired 100 benign rule files with 100 malicious versions to test detectors. That same-author corpus is not an independent broad benchmark.
That makes the detector results useful as a warning rather than a procurement scorecard. A scanner can prioritise review, but it should not be able to approve a new dependency by itself. The final gate should check a concrete package name, registry, publisher and resolved version.
Put controls after the prompt
BlackTree recommends:
- Version the rules. Keep agent instruction files under the same review and ownership controls as build and workflow configuration.
- Approve dependency changes. Require a person to review new package names, registry changes and lockfile differences before installation.
- Constrain resolution. Use approved registries, namespace policy and package provenance checks that the rule file cannot override.
- Reduce execution reach. Run untrusted coding work without production credentials and with limited filesystem access and outbound traffic.
- Join the audit trail. Correlate rule-file changes with dependency-manifest changes and package installation events.
The important boundary is deterministic. Agent reasoning may help explain a change, but identity, provenance and policy should decide whether the selected package can enter the environment.
What the study does not establish
BlackTree has not reproduced the preprint, and this review established no independent reproduction.
No production exploit campaign or real malicious package is documented. Success requires an adopted poisoned rule and available chosen package. No BlackTree control was tested.
Those limits keep the conclusion useful. Teams have evidence for a trust-boundary review without treating a controlled benchmark as proof of current compromise.


