Chatbots and language models seem to now be a default setting for children, introduced quietly, without public discussion or informed consent. Just as politicians and the public are finally waking up to the harms caused by social media, Big Tech is rolling out its next wave of commercial – potentially damaging – products. This moment demands scrutiny and resistance when it comes to kids.
“It is on my children’s Chromebooks. It is integrated in Chromebooks for children age 6 and up. The children are free to use it, and they do not receive instruction on how to use it. The teachers do not really know about it. The school administrators are unaware of it, and therefore have no policy or guidelines for its use.”
This is a LinkedIn post from a father to kids who study in a basic school in Helsingør, north of Copenhagen. Yes, it is the father, Jesper Graugaard, who tried to fight Chromebooks in basic schools in Denmark. He is now realising that his kids have obtained free access to Gemini via the school computer. He is surprised that this has happend and that no one in Denmark is discussing it. Neither the traditional press nor politicians and authorities.
In his post on LinkedIn he asks parents, organisations, and politicians if they think kids are ready to use AI tools without guidance? What this question doesn’t encompass is that it is the companies behind the technology that use the children – not the other way around. And moreover, it is not only about guidance, it is about at lack of clear judicial rules, enforcement and ethical principles.
So, our answer is to Jesper’s question is NO – and here are some of the reasons why.
The main reason is learning or a lack of it. If a machine constantly nudges you to help you write, edit or rewrite something, how will you learn to do it yourself? If you are a 3rd grader learning to read and write, an AI tool can take away the incentive and capability to learn. Google Gemini itself explains what it can do for pupils in schools:
“Gemini can function as a tool to help write better and support various stages of schoolwork:
- Drafting: Assisting in generating suggestions for texts, emails, or presentation slides (often integrated directly into Google Docs, Slides, and Sheets).
- Editing and Feedback: Reviewing ongoing work and providing suggestions for improvements that help students understand where to focus their efforts.
- Idea Generation: Serving as a sparring partner to explore issues, find inspiration for new projects, and generate creative ideas.
- Differentiated Learning: Creating customized learning experiences, for example, by generating practice quizzes based on notes or specific topics.
- Language Support: Assisting with translations and providing insight into the meaning and context of expressions.”
In school children need to learn many basic skills such as reading and writing and much more. School is about learning these things, but Gemini makes it easy for children to skip this hard process: To not want to learn, because they might as well use AI instead.
Furthermore, all generative AI models hallucinate, therefore it is important that the kids learn to fact check every word coming out of the AI models, and this can be even more time consuming than doing the work by yourself from scratch – and learning.
Google in Danish schools is tailor-made. Thus, services like YouTube are not part of the school’s offer to kids. Obviously, Gemini has been eased in without any scrutiny.
We’ve reached out to Jesper Vangkilde, communications lead in Google Denmark, and he has promised to get back to us on that. He confirms that kids have access to Google’s AI via the browser, but it is with safeguards, he writes. We’ve asked him to elaborate, and we will update this story as soon as we have his answer.
Gemini As Your Kid’s Mentor?
Gemini is not only a writing tool. Gemini itself explains in the following:
There is also the possibility to chat with Gemini. Gemini functions as an AI assistant or a chatbot (e.g., via gemini.google.com or the mobile app) that students and teachers can interact with.
- Direct Dialogue: Students can ask questions, both simple and complex, and receive answers that can serve as a starting point for further learning.
- Mentor Function: The AI can be used as a kind of digital mentor to elaborate on subjects or provide additional context.
And here is another call out. Would you like your kid to consult a machine designed to optimise their use of it (because that equals profit) as a mentor?
Kids should not talk to chatbots where there are no proper age appropriate safeguards or guardrails as to what is being said. They are machines predicting the next word in a sentence, and nobody knows what comes out of them. What has been documented is that they are sycophantic – and this is not good, as one of the writers of this article explains in this podcast.
Further, the use of chatbots have, unfortunately in some cases, escalated into use of the machine for therapeutic reasons. Humans typically start talking to the chatbots, and then slowly develop a trusted relationship, where they will confide in the machines. Most AI generators will most likely send the user in the direction of professional human help, but many will also offer to further discuss the problems. These machines are deliberately designed in a very human-like way to make humans use them longer. But it can have catastrophic consequences; some users have ended up in a delusional spiral, others have killedthemselves after becoming emotionally attached to a chatbot.
Age-Verification on Social Media Should Also Include Generative AI
Danish decision-makers have finally realised that social media is built in a highly unethical way, manipulating our kids (and grown-ups) and forcing values upon them that is not in sync with the values of a European society. Therefore, there is a lot of talk about age-verification on social media these days. Politicians and childrens’ rights organisations have announced that Denmark is now establishing an age limit on social media of 15 years. Up to now the age limit was 13 in Denmark, as opposed to many other EU countries who followed the EU recommendation of 16 years. The age limit and demands of age-verification is focused on social media and its commercial and manipulative algorithms. What it does not encompass is language models/chatbots.
According to a source in a childrens’ organisation, the new Danish law on age-limits will cover hyper-commercial social media with +45 million users, those called VLOPs in the Digital Service Act. So, if a service is ‘just’ a chatbot, it is not covered by the Danish demand of age-verification. This might be different in other European countries. UNESCO has called for an age-limit of 13 years. Countries like Germany and France have no specific age-limits on the use of GenAI but relies on the GDPR-age limit of 16 years.
There is no doubt that it is a good idea to sue Meta for systematic neglect when it comes to protecting children on social media. But it is also a class action suit that comes too late – after harm has been done. And the harms of social media and the “old” digital environment should not distract us – schools included – from recognizing that these dangers are potentially even bigger.
Should we wait for the harm to be done to our children with generative AI, just as we did with social media? We don’t think so!
We have used Google’s Geminia to generate both text and illustrations for this article. We don’t usually used generative AI for artificial illustrations. If we use text generation we use European services such as Lumo or Mistral.
