Structural, not discursive, redesign of assessments is essential to preserve validity in the era of generative AI.

Objective:
The main purpose of “Talk is cheap: why structural assessment changes are needed for a time of GenAI” is to critically examine how current institutional responses to generative AI (GenAI)—such as traffic-light policies, AI assessment scales, and AI-use declarations—fail to ensure assessment validity. The authors aim to introduce a new conceptual framework distinguishing discursive versus structural assessment changes, arguing that only structural redesign can meaningfully protect assessment integrity when AI can complete tasks undetectably.

Methods:
This article employs a conceptual and critical analysis approach rather than empirical experimentation. The authors analyze dominant institutional and scholarly frameworks (traffic-light systems, AI Assessment Scales, mandatory AI declarations) and review research on AI detection tools, student compliance, and integrity challenges. Drawing on theoretical scholarship in assessment validity and integrity, the authors develop and articulate a new distinction—discursive vs. structural changes—and apply it systematically to existing models to demonstrate their limitations. Case examples (e.g., essays, multiple-choice quizzes, lab reports) are used to illustrate how discursive versus structural changes differ in practice.

Key Findings:

• Current AI frameworks overwhelmingly rely on discursive changes, which are rule-based, instruction-based, and unenforceable. These frameworks assume student compliance with directives such as “AI is not allowed” or “AI may be used only for editing,” but offer no mechanism to verify adherence.

• Discursive approaches create an “enforcement illusion.” By using language borrowed from structurally enforced real-world systems (e.g., traffic lights), institutions mistakenly believe they have established control when no enforcement mechanism exists.

• AI detection tools are unreliable, plagued by false positives and false negatives, making prohibition-based approaches especially untenable.

• Structural changes are the only sustainable path to assessment validity. Structural changes modify the task mechanics—for example, supervised timed assessments, process-based evaluation, real-time demonstrations, authenticated checkpoints, or connected multi-stage assessments.

• Validity cannot depend on voluntary compliance in an environment where AI assistance is ubiquitous, undetectable, and highly capable.

Implications:
The article’s conceptual framework contributes significantly to AI-in-education discourse by reframing the core challenge: assessment security cannot be achieved through policy language alone. It calls for a paradigm shift—away from rule-making and toward redesigned assessment architectures. This perspective equips institutions, policymakers, and educators with a more realistic understanding of what is required to protect validity in higher education. It also aligns with emerging accreditation and quality-assurance concerns globally as AI rapidly accelerates.

Limitations:
Because the work is conceptual, not empirical, it does not test specific structural models or measure their effectiveness in real courses. Additionally, while it presents examples across disciplines, it acknowledges that precise structural redesigns must be discipline-specific and cannot be prescribed generically. The argument focuses primarily on assessment of learning and does not address broader pedagogical or learning-oriented uses of AI.

Future Directions:
The authors call for:

• Empirical research validating specific structural redesign strategies.

• Discipline-specific frameworks for structural assessment transformation.

• Longitudinal research tracking how structural changes affect student learning, equity, workload, and integrity.

• Institutional models that view assessment validity at the program level (not task level), establishing chains of evidence across multiple assessment points.

• Continued exploration of dual-lane assessment systems (secure vs. open assessments) with structural, not discursive, underpinnings.

Title and Authors:
“Talk is cheap: why structural assessment changes are needed for a time of GenAI”
By Thomas Corbin, Phillip Dawson, and Danny Liu.

Talk is cheap why structural a…

Published On:
Published online May 15, 2025.

Talk is cheap why structural a…

Published By:
Assessment & Evaluation in Higher Education (Taylor & Francis / Informa UK Limited).