Trying to bring good engineering practices to AI Safety Evaluations :)

• Measuring Prefill Awareness in transcript-based evals. Building an Inspect-based eval to audit existing benchmarks for Prefill Awareness as a confounding factor: prefill-awareness-audit.
• Eval tooling and methodology contributions to UK AISI's Inspect AI and Inspect Evals.

Inspect AI Ecosystem:

• inspect_ai#3709 — Add vLLM chat template controls for base-model evals.

• inspect_ai#3969 - Add pass_k reducer metric for reliability in agentic evals.

• inspect_evals#1503 — Fix mean_of on_missing="skip" to also skip None-valued samples.

• inspect_evals#1501 — cyberseceval_4: tolerate fenced and prose-wrapped judge JSON.

• inspect_evals#1429 — Fix CodeIPI exfiltration scorer to check tool result messages.

• inspect_scout#455 - Fix events_data parser to fit new Petri eval schema.

• Email: joesposito8@gmail.com
• LinkedIn: joseph-esposito8