Small independently executed evaluation · October 2026
Safety evaluation under an attack-family shift
How much can an apparently strong prompt-injection detector score hide about evaluation coverage?
TL;DR
A simple word-level classifier reached 97.9% recall on held-out samples from a small published prompt-injection benchmark, and 95.6% recall on two attack families excluded from training. A character-level classifier reached 97.8% recall in the latter condition. But a high score here does not prove an agent is safe: the benchmark has only 303 curated examples and no live attacks were executed.
Not a real-agent red-team evaluation
1. Experiment
I trained word and character TF-IDF + logistic-regression detectors on the stef41 public labeled benchmark. The data contains 200 labeled injection and 103 benign prompts. I held out two attack categories entirely from training: encoding bypass and delimiter injection. The shifted test reuses benign controls from the in-distribution split; it is not an independent external dataset.
All numbers below come from executing a Python script on the published data using a deterministic split (seed 42), not a mock or hand-entered test set. The upstream benchmark itself includes authored/curated examples.
2. Measured results
Attack recall at a fixed 0.5 threshold. Change the displayed subset below to compare the two actually measured conditions.
Within benchmark: both models identified 46 out of 47 positive examples, with 0 false positives on 31 benign controls.
| Model | Original holdout recall | Held-out family recall |
|---|---|---|
| Word TF-IDF | 46/47 (97.9%) | 43/45 (95.6%) |
| Character TF-IDF | 46/47 (97.9%) | 44/45 (97.8%) |
Observation: Both lexical baselines score very highly on this small benchmark. This could reflect repeated language patterns or simplistic examples; treating it as robust prompt-injection prevention would be unjustified.
3. Limitations
The project uses a real third-party dataset, but that dataset is curated and partly hand-authored. The experiment does not run a model, execute malicious tool outputs, or measure agent compromise. The held-out categories are not an independently collected source and the benign test examples are reused. Only one seed and no semantic near-duplicate audit were performed. A larger intended cross-dataset evaluation was not completed, and is not claimed here.
4. Reproduce and inspect
Python project and method · Recorded measured results · Upstream data.
Independent engineering experiment. Not affiliated with Trajectory Labs.