From Evaluation to Guardrails: What We Brought to ACM FAccT 2026
At ACM FAccT 2026, we argued that AI guardrails need the same scrutiny as the models they govern. By evaluating refugee scenarios across multiple languages, we found that static policies fail without context. Our hands-on session demonstrated that agentic guardrails equipped with tools like web search are vital for reliable, real-world deployment, moving beyond simple text filtering to dynamic, fact-checked safety.
Evaluating guardrails is as important as evaluating the LLMs they protect, and context- and language-specific evaluation results should inform guardrails that move beyond static taxonomies of harm toward dynamic policies.