American schools deployed AI tools at an unprecedented pace after 2022 and most of them did it without a governance framework in place. By 2024, more than 60 percent of U.S. school districts reported using at least one AI-powered platform, yet fewer than a third had formal policies covering how those tools accessed or retained student data. That gap is not a minor compliance oversight. Under the Family Educational Rights and Privacy Act, it exposes institutions to enforcement action, erodes the trust of students and parents, and creates liabilities that no IT team can quietly manage away.
This piece examines exactly how FERPA applies to AI systems, why the rules work differently across K-12 and higher education, and what a functional AI governance framework looks like in each context.
Why FERPA Is the Starting Point for Any Education AI Policy
The Family Educational Rights and Privacy Act has governed student records since 1974. Its core provision is straightforward: educational institutions receiving federal funding cannot disclose education records or personally identifiable information without written parental consent (or student consent once the student turns 18 or enters postsecondary education). What makes FERPA complicated in the AI context is the broad scope of what qualifies as an education record.
Grades and transcripts are the obvious examples. Less obvious: behavioral logs, learning management system activity, cafeteria purchases, library checkouts, and in some interpretations, keystroke patterns collected by proctoring software. Each of these categories is now a data point that AI systems can process, store, and act on. The moment a school contracts with an AI vendor that ingests these data points, FERPA school official exception is triggered and that exception comes with specific conditions most vendors do not automatically satisfy.
The School Official Exception and What It Actually Requires
Under 34 C.F.R. 99.31(a)(1), schools can share student data with a third-party vendor without parental consent only if that vendor qualifies as a “school official” with a “legitimate educational interest.” The Department of Education has clarified that this applies to contractors, consultants, and technology providers but only when four conditions are met:
- The vendor performs a service the institution would otherwise use its own employees to perform.
- The vendor is under the institution’s direct control regarding the use and maintenance of the education records.
- The vendor uses the data only for the purpose for which the disclosure was made.
- The vendor does not re-disclose the data without authorization.
Condition two is where most AI deployments fail scrutiny. A large language model fine-tuned on student writing samples, a predictive analytics system that retains behavioral histories, or a chatbot that logs queries to improve its own performance none of these are automatically under “direct control” of the school in the way a human employee is. Governance frameworks must address this explicitly.
K-12 vs. Higher Education: A Structural Difference in How Governance Works
Both levels of education fall under FERPA, but the practical governance challenges are meaningfully different. Treating them as identical leads to policies that fail on both ends.
K-12: Minors, Parental Rights, and a Denser Regulatory Web
In K-12 settings, FERPA operates alongside the Children’s Online Privacy Protection Act (COPPA) and, for many states, the Student Online Personal Information Protection Act (SOPIPA) or state equivalents. The cumulative effect: schools bear a much higher burden of accountability for any vendor that touches student data. Children under 13 cannot provide meaningful consent under COPPA, so the school’s consent-by-proxy role is non-negotiable.
This creates a practical governance requirement that many districts underestimate. Every AI tool deployed in a K-12 classroom whether it’s an adaptive math platform, an AI writing assistant, or an attendance prediction system must go through a vendor data review before deployment. That review needs to confirm what data is collected, where it is stored, how long it is retained, whether it is used to train models beyond the school’s instance, and whether it is ever shared with advertising networks.
The state layer adds further complexity. California’s AB 1584 and SOPIPA, New York’s Education Law Section 2-d, and similar statutes in over 40 states have layered additional obligations onto FERPA’s baseline. A K-12 AI governance framework that only references federal law is already incomplete.
Education institutions, especially public schools and universities, sit close to the government accountability layer. FERPA, vendor contracts, student rights, and public transparency all connect to the larger question of how agencies govern automated decisions. For that reason, education leaders should also understand AI governance in the U.S. public sector when building AI policies around student data.
Higher Education: Student Rights, Research Purposes, and the Consent Paradox
At the postsecondary level, FERPA rights transfer from parents to students. A university deploying an AI-powered academic advising system must ensure that students know what data feeds into the system, what recommendations it generates, and how they can access or correct the underlying records. This is more manageable in theory adults can review and exercise their own rights but practically complicated by scale.
Universities also operate research environments where student data frequently flows through IRB-approved studies, creating legitimate but complex data pathways that AI systems can inadvertently access. A machine learning model trained on historical enrollment and dropout data, for example, might incorporate identifiable student records from research datasets unless governance policies explicitly wall off those data sources.
The other higher education challenge is institutional heterogeneity. A single university may operate dozens of AI tools independently procured by different schools, departments, or individual faculty members. Central governance is harder to enforce, which makes policy architecture rather than point-in-time audits the more sustainable approach.
What Student Data AI Systems Actually Collect (And Why It Matters for Governance)
Most governance failures in education AI are not the result of bad intentions. They result from institutions not fully understanding the data surface area of the tools they deploy. This matters because FERPA liability follows data, not intent.
Consider what a typical AI-powered learning management system integration might collect:
- Login timestamps and session duration
- Content interaction patterns (which resources students opened, how long they spent, where they dropped off)
- Assessment responses and revision histories
- Communication metadata (who messaged whom, when, and how frequently)
- Behavioral inferences generated by the AI (engagement scores, dropout risk flags, predicted grades)
That final category AI-generated inferences is an emerging governance frontier. The Department of Education’s 2023 guidance on AI and student privacy noted that inferences derived from education records are themselves education records, and are therefore subject to FERPA. This has significant implications for algorithmic accountability: a student flagged as a dropout risk by a predictive model has the right to access that flag in their education record and to challenge its accuracy.
The re-identification risk is also underappreciated. Even “anonymized” behavioral data from a small class cohort can be re-identified with relatively modest external information. Any AI governance policy must account for this, particularly when student data is shared with vendor systems for model improvement.
Building an AI Governance Framework for Educational Institutions
Governance is not a compliance checklist. It is a set of structural decisions that determine how AI is procured, deployed, monitored, and retired within an institution. The following framework addresses the distinct needs of both K-12 and higher education while building on a shared foundation.
1. Governance Structure and Ownership
Every institution needs a named decision-making body for AI procurement. In K-12 districts, this typically sits with the technology director in partnership with the superintendent’s office. In universities, a Chief Privacy Officer or similar role should have formal authority over AI vendor contracts, with representation from academic affairs, legal, and IT security.
Without clear ownership, AI tool adoption defaults to individual teachers or faculty members making decisions that carry institutional liability. A distributed adoption model is fine for sandboxed experiments; it is not acceptable for tools that access student records at scale.
2. Vendor Due Diligence and Data Processing Agreements
No AI tool should be deployed in a student-facing context without a signed Data Processing Agreement (DPA) that explicitly addresses:
- The legal basis for data processing under FERPA (school official exception or signed consent)
- Data retention and deletion timelines, with written confirmation that student data is purged upon contract termination
- Prohibition on using student data to train or improve AI models beyond the institution’s contracted instance
- Prohibition on sharing student data with advertising networks, data brokers, or third parties without explicit authorization
- Incident notification obligations (breach notification within 72 hours is now standard best practice, though FERPA does not mandate a specific timeframe)
The most important clause is often the least negotiated: the data use prohibition for model training. Many large AI platforms include clauses permitting them to use user inputs to improve their systems. In a K-12 context, that clause is almost certainly a FERPA violation. Governance teams must identify and remove or explicitly restrict it.
3. Algorithmic Accountability and Student Rights
When AI systems generate inferences about students risk scores, engagement metrics, behavioral predictions those outputs carry real consequences. Students denied access to resources based on a low risk score, or counseled toward particular academic pathways because of a model’s prediction, deserve transparency about how those decisions were made.
At minimum, an institutional AI governance framework should require:
- Human review before any AI-generated recommendation is acted upon in a high-stakes context (placement, academic intervention, disciplinary action)
- Documentation of the model’s logic in plain language accessible to students and parents
- A formal process for students to dispute AI-generated assessments of their records
This last point has particular legal weight. FERPA’s amendment rights extend to AI-generated inferences stored in student records. Institutions that cannot explain or contest those inferences are exposed.
4. Staff Training and Procurement Policy
Governance documents are only as useful as the people implementing them. K-12 teachers need practical training on what data their classroom AI tools collect and what they cannot share or input into commercial AI systems (student names, IDs, assessment data, behavioral concerns). University faculty need parallel guidance, with particular attention to AI use in research contexts where IRB protections may not align automatically with FERPA obligations.
Procurement policy should require that any AI tool touching student data go through the governance review process before purchase approval. This means procurement controls must precede, not follow, departmental adoption. The common pattern of faculty adopting tools and notifying IT afterward is not governable.
State Laws That Change the Calculation
FERPA sets a federal floor. State laws frequently set a higher ceiling, and in 2025, most U.S. states have at least one student data privacy law that intersects with AI governance.
State/Law | Key Provision | Applies To | Notable AI Implication |
California (AB 1584, SOPIPA) | Prohibits operators from using student data to build user profiles; requires deletion on request | K-12 | AI adaptive learning platforms that build learner profiles may be prohibited without explicit policy alignment |
New York (Ed. Law §2-d) | Requires contracts with vendors to include specific data security protections; parent Bill of Rights for Data Privacy | K-12 and Higher Ed | All AI vendor contracts need explicit §2-d compliant data processing agreements |
Illinois (Student Online Personal Protection Act) | Prohibits targeted advertising and sale of student data; extends to covered information used by operators | K-12 | Any AI tool with behavioral targeting features must be reviewed for SOPPA compliance |
Colorado (Student Data Transparency and Security Act) | Requires annual review of student data practices; public disclosure of all third-party vendors | K-12 | AI vendor inventory must be publicly listed and reviewed annually |
Higher education institutions face an additional layer of complexity: state laws designed primarily for K-12 sometimes have ambiguous applicability to universities, while other states have separate higher education data privacy provisions. Legal counsel familiar with both FERPA and applicable state law is not optional in this environment it is a governance prerequisite.
Algorithmic Bias and Equity: The Dimension Most Governance Frameworks Miss
FERPA compliance is necessary but not sufficient. A governance framework that only asks “is this legal?” will consistently miss the equity dimension of AI in education and that dimension carries both ethical and legal weight under Title VI, Title IX, and Section 504 of the Rehabilitation Act.
Predictive analytics systems used in K-12 education have a documented history of producing racially stratified outputs. A 2021 analysis by the Center for Democracy and Technology found that automated decision systems in education disproportionately flagged Black and Hispanic students as behavior risks and assigned them lower engagement scores outputs derived from historical data that itself reflected systemic inequities. An AI system trained on this data does not neutralize the bias. It operationalizes it.
Governance frameworks must require demographic disparity analysis before any predictive model is deployed at scale. This means:
- Disaggregating model outputs by race, gender, disability status, and socioeconomic background before adoption
- Establishing acceptable thresholds for disparate impact and halting deployment if those thresholds are exceeded
- Repeating the analysis periodically as the model continues to learn from new data
For higher education specifically, AI tools used in admissions, financial aid, or academic advising carry heightened scrutiny under anti-discrimination law. The Supreme Court’s 2023 decision in Students for Fair Admissions v. Harvard clarified that race-conscious admissions are prohibited a decision that has prompted renewed legal examination of any institutional algorithm that incorporates demographic proxies, even inadvertently.
What Good AI Governance Actually Looks Like in Practice
Theory is useful. Examples are more useful. The following illustrates what mature AI governance looks like in both contexts.
K-12: Unified School District (Composite Example)
A mid-sized district in California adopted an AI tutoring platform in 2023. Before signing the contract, the district’s technology director ran the vendor through a pre-procurement review that included a FERPA compliance questionnaire, a SOPIPA compliance checklist, and a legal review of the data processing agreement. The DPA was revised three times to remove a clause permitting the vendor to use aggregated student interaction data for product improvement.
The district established a Student Data Governance Committee with representatives from legal, IT, curriculum, and the school board. The committee meets quarterly to review active AI vendors, audit new deployment requests, and update the district’s AI use policy. All classroom teachers completed a 90-minute privacy training module before using any district-approved AI tool. The district’s vendor list is published on the district website and updated annually a practice aligned with Colorado’s requirements and increasingly adopted voluntarily elsewhere.
Higher Education: Research University (Composite Example)
A large research university in the Northeast centralized AI governance under its Chief Privacy Officer in 2024 after three separate departments independently adopted AI tools with conflicting data processing terms. The CPO’s office created a streamlined review process: any AI tool touching student data requires a 15-day review before procurement. Tools that only use anonymized, aggregate data at the department level go through a lighter-touch review.
The university published an AI Transparency Registry listing every AI system in student-facing use, its data inputs, its decision outputs, and the name of the institutional owner responsible for it. Students can submit data correction requests through a dedicated portal. The registry is updated each semester and shared with the student government.
Neither of these examples is perfect, but both share the structural features of genuine governance: clear ownership, documented processes, meaningful transparency, and accountability mechanisms that exist before something goes wrong rather than in response to a crisis.
The Road Ahead: Federal Guidance and Emerging Standards
The regulatory environment for AI in education is not static. The Department of Education’s 2023 report “Artificial Intelligence and the Future of Teaching and Learning” signaled that more specific AI guidance is coming, likely including clarifications on how FERPA applies to AI-generated inferences and vendor model training. Institutions that build governance frameworks now, rather than waiting for mandates, will be significantly better positioned.
At the federal level, the White House’s executive actions on AI in 2023 and 2024 directed federal agencies to develop sector-specific guidance. For education, this is likely to produce FERPA-aligned AI contracting standards and potentially a federal student data privacy statute that closes some of FERPA’s well-documented gaps (FERPA does not require breach notification; it does not prohibit data sale explicitly; it does not address AI inferences directly).
Several professional bodies are also developing standards relevant to this space. The National Institute of Standards and Technology’s AI Risk Management Framework (NIST AI RMF) provides a sector-agnostic structure that maps reasonably well onto educational contexts, particularly its GOVERN and MEASURE functions. The International Society for Technology in Education (ISTE) has published AI competency frameworks for educators that complement governance policy with professional development.
Institutions that align their internal frameworks with NIST AI RMF now will have an easier transition when sector-specific federal standards arrive and will have demonstrably better governance infrastructure than peers who waited.
Frequently Asked Questions
Does FERPA apply to AI tools that only use anonymized student data?
Not automatically. FERPA applies to education records containing personally identifiable information. Genuinely anonymized data where re-identification is not reasonably possible falls outside FERPA’s scope. However, “anonymized” is often a weaker protection than institutions assume. Behavioral data from small class cohorts, or data sets combined with external information, frequently enable re-identification. The burden is on the institution to confirm that data has been de-identified in a way that meets FERPA’s de-identification standard, not simply that names have been removed.
Can a university use student essays to train its own internal AI writing assistant?
With limitations. Using student-submitted work to fine-tune an AI model constitutes processing of education records. Unless students have consented or the activity qualifies under a legitimate educational interest exception, this likely requires student notification and opt-out rights at minimum. Institutions considering this approach should seek legal guidance and consider whether an opt-in consent model is more defensible than a notification-and-opt-out approach.
What is the difference between FERPA and COPPA for K-12 AI governance?
FERPA governs what schools can do with education records they hold. COPPA governs what commercial operators can do with data they collect directly from children under 13 online. In K-12 AI deployments, both apply simultaneously. FERPA requires the school to ensure its vendors handle student records appropriately. COPPA requires those same vendors to have verifiable parental consent before collecting personal information from children, or to obtain that consent through the school operator exception. A complete K-12 AI governance framework must satisfy both.
Are AI-generated risk scores stored as education records under FERPA?
Yes, per the Department of Education’s 2023 guidance. Inferences, predictions, and risk scores derived from education records and maintained in a student’s file are themselves education records. This means students have the right to inspect them, request amendments if they believe the scores are inaccurate or misleading, and receive notice when they are disclosed to third parties.
What should institutions do if a vendor refuses to modify its data processing agreement?
Walk away from the tool, or escalate to legal counsel to assess whether the vendor’s standard terms create acceptable risk. Most large AI vendors have education-specific DPA templates that include stronger protections than their default consumer terms but institutions need to ask for them. If a vendor is unwilling to sign a FERPA-compliant DPA, deploying their tool creates liability that cannot be contractually shifted back to the vendor.
How often should an institution audit its AI vendor relationships?
At minimum annually, and following any significant update to a vendor’s platform, privacy policy, or terms of service. Annual audits should confirm that the DPA is still current, that the vendor’s actual data practices match what the DPA commits to, and that the tool’s risk profile has not changed materially. Some institutions conduct lightweight quarterly reviews for high-risk tools (those with broad data access or AI-generated inference outputs) and annual reviews for lower-risk tools.
Conclusion
The core governance challenge for AI in education is not technical. The technology is available to build systems that respect privacy, operate transparently, and account for the equity implications of automated decision-making. The challenge is institutional: building the processes, authorities, and accountabilities that ensure those capabilities are actually used.
FERPA provides the legal foundation. State laws add specificity. But neither translates automatically into a governance framework. That requires deliberate structural choices: who owns AI procurement decisions, what a compliant vendor contract looks like, how students access and challenge AI-generated assessments of their records, and how institutions identify and address the bias that historical data injects into predictive models.
Institutions that treat this as a compliance exercise will find themselves perpetually behind the regulatory curve. Those that treat it as an institutional responsibility one owed to the students whose data and futures they hold will build something more durable.
GAICC offers ISO/IEC 42001-aligned certification programs that equip governance professionals to build and lead AI management systems in regulated sectors, including education. Explore our Lead Implementer and AI Law & Compliance Professional programs to understand how international governance standards apply to the education context.
