#2139·garak

Migrate owasp:llmNN probe tags to year-qualified owasp:YYYY:llmNN format

Author: lavkeshdwivediCreated Sep 1, 2026Updated Sep 18, 2026

Spun out of #1919 and #1920 per @jmartin-tech's review.

Background

agent_breaker.AgentBreaker is currently the only probe tagged owasp:llm07 / owasp:llm08, while its doc_uri points at LLM06 (Excessive Agency) in the 2025 list. #1920 attempted to retag just that one probe, but as noted in review that would leave main inconsistent: the owasp:llm06 tag in garak was defined against the 2023-24 OWASP Top 10, tag-based grouping selects and groups by prefix, and changing a single probe conflates taxonomy versions.

Proposal

Migrate every owasp:llmNN tag in the probe catalog (78 entries) to a year-qualified form, e.g. for AgentBreaker:

owasp:2023:llm07
owasp:2023:llm08
owasp:2025:llm06
owasp:2026:llm03
  • At minimum, add the 2023 year marker to all existing entries so their meaning is unchanged.
  • Add newer-list tags (2025, 2026) where a probe maps onto them.
  • Update tag selection and grouping logic that keys on the owasp:llm prefix so owasp:2023:*, owasp:2025:* etc. group correctly.
  • Update docs that reference the prefix scheme.

Open questions

  • Preferred spelling for the year-qualified prefix (owasp:2023:llm07 vs. an alternative).
  • Whether newer-list tags (2025 / 2026) land in the same PR or a follow-up.

Happy to own this PR if the team wants to proceed.