AI didn’t break your AI policy. It broke your proxy.

For as long as there have been schools, we have inferred what a student can do from what a student hands in: the artifact. The essay stands in for the ability to argue. The lab report stands in for the ability to reason from evidence. The problem set stands in for the ability to choose a method and defend it.

That inference underwrites nearly every grade, transcript, and placement decision a district makes. And it broke everywhere at once.

What most districts did

Most districts responded the way the moment seemed to require: they wrote a policy. Acceptable use. Permitted tools. Disclosure requirements. Consequences.

That was a reasonable first move. It was fast, it was defensible, and a board could see it. But look at what it governs. A rule about how a student may produce work says nothing about what the work proves. The policy governs the tool. The proxy is what broke. A lot of the energy since has gone into catching students. But the student who hands in an AI-written essay didn’t break anything. They exposed an assumption we never tested.

The artifact was always an approximation of the skill. Most of us never checked how good an approximation it was.

The proxy was always a proxy

This is the part that gets lost in the AI conversation. The link between an artifact and a skill was never as tight as we assumed. When I was a data specialist with Millstone Township School District, we ran a calibration on Dibels. 10 teachers scored the same student work, blind, with the same rubric. On the first run, they had some gaps that needed to be talked through. That was 2015, years before anyone had heard of ChatGPT.

AI didn’t create that gap. It just made it impossible to ignore. Teachers know their students. What differs is how that knowledge turns into a grade, and that part has rarely been checked.

The better question

The question in front of district leaders is older than AI policy:

• What should students be able to do now?
• What should they be able to do without the tool, and what with it?
• How would we know whether they can?
• Would we know if our teaching got better at it?

Those are curriculum, assessment, and measurement questions. They sit in the instructional core, not in the policy perimeter. And they have answers, but not ones you can write into a handbook.

Where to start

Start where the break shows first: assessment validity. Pick one course and one common assessment. Ask whether the way you score it would tell the difference between a student who did the thinking and one who didn’t. Then have three teachers score the same five pieces of work, blind.

Count how often they agree exactly. If it’s below 80%, the fix isn’t a new rule. It’s a task where the thinking has to show up in the work. Five pieces and three raters won’t settle anything, but they will tell you whether the question is worth asking.

The number you get back is the beginning of an honest conversation, and it doesn’t need the word “AI” anywhere in it.