AI data privacy: the questions a Pakistani school should ask before adopting any AI tool

A teacher pastes a chapter summary into an AI tool at 10pm and gets a lesson plan back in seconds. Nothing about that moment feels like a data transfer, which is exactly why it deserves a second look. The text just travelled to a company's servers, was processed by a model, and may or may not be stored, logged, or reused. For a school, the question isn't whether that's alarming. It's whether anyone at the school actually asked where it went before the tool was adopted.

The legal backdrop in Pakistan makes those questions more important, not less. According to DLA Piper's data protection guide for Pakistan, Pakistan has not enacted a general data protection statute; a draft Personal Data Protection Bill, introduced by the Ministry of Information Technology and Telecommunication, still needs consultation, approval by both houses of Parliament, and presidential assent. In the meantime, the Prevention of Electronic Crimes Act, 2016 and its 2025 amendment address data misuse mainly through penal sanctions. Drafts can change and this landscape may move, so treat that guide as a starting point and check the current status rather than relying on this paragraph. The practical consequence is simple: there is no general rulebook a school can point to and say the vendor must comply with it, so a school's protection largely comes from what it asks, and what the vendor commits to in writing.

Internationally, the direction of travel is clear. UNESCO's guidance on generative AI in schools calls on governments to adopt data protection and privacy standards, to train teachers, and to set an age limit of 13 for using AI tools in the classroom. You don't need to wait for a national law to borrow that mindset: treat data protection as a condition of adoption, and treat unsupervised use by young children as something to avoid.

Here are the questions worth putting to any AI vendor, in roughly the order they matter. First: where does what we type actually go, and which company processes it? Many classroom AI products are thin layers on top of a large model provider, and the provider is the party that really handles your content. A trustworthy vendor names it. Second: is our content used to train or improve models? This answer often depends on the plan, not just the product. Google's own Gemini API terms are a useful illustration: they say content submitted to unpaid services may be used to improve Google's products and warn against submitting sensitive or personal information there, while for paid services Google says it doesn't use prompts or responses to improve its products. The lesson isn't about one company. It's that "does it train on our data" has different answers on different tiers, so ask which tier applies to you.

Third: what personal information does the tool ask for, and does it need any about students? A tool built for teachers to generate lesson plans, worksheets, and quizzes has no reason to receive student names, roll numbers, or marks. If a product asks for them, ask why. Fourth: how long is content kept, and can we delete it? Look for a plain answer about retention and a real route to deletion, not a promise to "take privacy seriously." Fifth: who else touches the data? Hosting providers, analytics tools, and support platforms all count. A short, honest list is a good sign; silence is not.

Sixth: is the tool meant for children at all? This one cuts both ways. The Gemini API terms mentioned above say the services must not be used in applications directed at people under 18, which is one more reason a classroom tool should be teacher-facing rather than something students log into. Seventh: what happens when we leave? Can you export what you created, and is your data removed afterwards? Eighth: what does the vendor say when something goes wrong? A named contact and a stated process beat a generic support form.

A school-side habit matters as much as any vendor answer: never put student-identifying information into a general AI tool. Names, admission numbers, marks, health or family details, and photographs don't belong in prompts. Nearly every useful classroom request works fine with the class, subject, chapter, and learning outcome, none of which identify a child. Make that a written rule for staff, and repeat it when new teachers join.

Here is how Muallim answers these questions, so you can hold it to the same standard. Muallim is built by DIGIT Pakistan for teachers and school admins, not students; children don't have accounts. Our Privacy Policy names Google's Gemini API as the AI system used to generate lesson plans, worksheets, and quizzes, and says the details you enter for generation are sent to Google for processing. It also states what we collect and why, including that payment proof and support messages are collected when you upgrade or contact us. What it can't do is make a decision for you: read it, ask us follow-up questions at salam@muallim.org, and compare our answers to the list above. The Lesson Planner and the Worksheet Builder only need a class, subject, and topic to do their job, so the exposure is small by design, but that design depends on your staff following the no-student-data rule.

One practical way to run this is a one-page checklist that the principal or coordinator fills in for each tool, with the vendor's answer, the date, and where the answer was found. Review it once a year and whenever a vendor changes its terms. It takes an afternoon to set up and turns a vague sense of unease into a record you can show a parent or an inspector. And if a vendor can't or won't answer a plain question in writing, that is itself an answer.