CIRCLE: A framework for evaluating AI from a real-world lens.

Abstract

Most AI evaluations rely on static benchmarks that measure model outputs in isolation, offering little evidence about how systems behave once embedded in real-world workflows. As a result, decision- makers lack systematic evidence about downstream effects, operational risks, and long-term impacts that matter for deployment, governance, and procurement. We introduce CIRCLE, a six-stage lifecycle-based framework that links stakeholder concerns to context-sensitive evaluation methods, longitudinal measurement, and ongoing monitoring of deployed AI systems. The framework integrates evaluation methods such as A/B testing, field testing, red teaming, and longitudinal studies into a coordinated evaluation pipeline rather than treating them as isolated activities. Together, these methods support more contextualized, iterative, and decision-relevant assessments of AI systems. By aligning constructs, methods, and metrics with real deployment contexts, CIRCLE supports more actionable, iterative, and governance-relevant evaluation of AI systems and their secondary and tertiary effects.