AI Generated Abuse: Deepfakes

Those who have a responsibility to safeguard children must be aware of the risks that they face in modern life. Whilst some safeguarding risks are a risk to children regardless of the generation into which they are born, new technologies have brought with them unique risks for the children of today. Generative AI is one of the largest technological advancements that we have seen in the modern age, however this advancement also brings with it new and unexpected risks. AI generated abuse in the form of deepfakes are becoming increasingly prevalent. It is vital that those who work with children and young people have an understanding of what deepfakes are and how they can become a safeguarding issue. In this article we will outline everything you need to know about deepfakes and provide you with tips on how to tackle this safeguarding issue. 

---

What are Deepfakes? 

Deepfakes are fake images, videos or audio content that has been made or manipulated by AI. Deepfakes often use real content, for example a real image of someone on holiday with their family, and alter it to make an exact likeness. This can make deepfakes indistinguishable from the real thing. Deepfakes aren’t always harmful nor are they inherently illegal. However, they are increasingly being used for malicious purposes which can be a criminal offence.

Technological advancements can bring numerous risks alongside their potential benefits. Deepfakes are just one form of AI generated abuse and the speed with which AI is developing can make it hard to keep track of potential dangers. A report by Internet Matters found that 25% of vulnerable children had had experiences with nude deepfakes. They also found that the majority of deepfakes are used for harmful purposes with nude deepfakes making up approximately 98% of all deepfakes; though it remains difficult to establish the true scale of them circulating online. However, despite this, almost two-thirds of children and almost half of parents reported not knowing or understanding the term ‘deepfake.’ When abuse is not understood it makes it difficult to recognise and teach children how to protect themselves and others. As such, a basic understanding of deepfakes and the safeguarding risks they can pose is essential to ensuring that children are kept safe and risks are identified and managed in a timely manner. 

Deepfakes and the Law

As mentioned, deepfakes are not inherently illegal however their use is illegal in some instances. This includes: 

---

Why Deepfakes are a Safeguarding Issue

As mentioned, deepfakes are not inherently illegal, however, when they are used for malicious purposes, such as those listed above, it can constitute a criminal offense. Children are not protected from these malicious acts simply because of their age and new Keeping Children Safe in Education (KCSIE) guidance specifically notes the potential for AI generated abuse, such as deepfakes, to cause serious harm to young people. The nature of deepfakes places children at risk of seeing age-inappropriate content of people they know or even themselves without their consent or the consent of others. Deepfakes violate an individual’s privacy, perpetuate harmful stereotypes and, at times, can be used to incite violence. This, coupled with the ease with which young people can access the technology to create deepfakes, can facilitate child-on-child abuse, encourage the adoption of harmful opinions and create an unsafe educational environment for many children. 

It’s important to note that the definition of child-on-child abuse now specifically includes the sharing of AI generated images and videos, including deepfakes. KCSIE guidance recognises that children can abuse other children both in-person and online. Online this form of abuse can include the non-consensual sharing of self-generated intimate images or videos, such as deepfakes. Deepfakes of abusive and/or pornographic images can be freely shared without an individual’s consent and to those who do not want to receive such content. Taking and sharing nude photographs of those aged under 18, including those generated using AI, is a criminal offense. Deepfakes can be used as part of online sexual harassment and the development and advancement of new technologies means that schools must consider sexual harrasment in broader terms. Child-on-child abuse is a safeguarding issue for both the perpetrator and the victim and if left unchallenged this behaviour can become normalised creating a culture of that minimises harm and slowly leads to sexual violence. 

New KCSIE guidance also highlights that schools and colleges play a crucial role in preventative education. This education should create a culture of zero tolerance for sexism, racism, misogyny, homophobia, sexual violence and harassment, derogatory behaviour or other forms of physical violence and conflict. In order to do this, schools must, at age-appropriate stages, support students to understand the harms that they could face. AI generated abuse is still relatively new and whilst one can hope that it will not become more prevalent, it is arguably unrealistic to not prepare both staff and students against the dangers. As such, new guidance reiterates the importance of schools updating their knowledge on AI generated abuse. Staff must be aware that technology is a significant component in many safeguarding and wellbeing issues and that children are at risk of abuse and other risks online as well as face-to-face. 

---

How to Tackle the Issue of Deepfakes

Tackling the issue of deepfakes can feel like a gargantuan task, especially given the seeming limitlessness of technology. When young people have instant access to technology that can create deepfakes it can be hard to monitor what they are doing and what it is being used for. The use of mobile phones in schools is a hotly debated topic, so too is banning social media for certain ages. However, whilst limiting access to certain platforms and tools can help to tackle the issue of deepfakes, one of the most effective ways to tackle the issue is through education. 

As mentioned, KCSIE guidance recognises the importance of school staff in providing preventative education. Preventative education is a proactive approach to tackling deepfakes as it helps to create a zero tolerance culture in which any form of bigotry, hatred or violence is unacceptable. This embeds these views into the core of a school’s identity and when students see that their school has a clear set of values and standards and upholds them in demonstrable ways, they are more likely to model that behaviour. This is particularly important as deepfake incidents are likely to happen outside of school and then be ‘bought into’ school via mobile phones. When young people have a strong sense of what constitutes bigotry or harmful content, they are less likely to engage with it and therefore less likely to bring it into school. 

Education is a vital way for schools to inform young people of the danger of deepfakes. Discussions around deepfakes must always be had in an age-appropriate manner and it is vital that all students, regardless of age, understand that they can be used to cause people serious harm. As deepfakes are not inherently harmful or illegal, some young people may struggle to understand why they are bad. However, manosphere culture relies on people engaging with seemingly innocuous content then pulling them further in, encouraging them to adopt right-wing, misogynistic views. As such, young people must be taught to be critical of what they see, questioning whether it is real and if it is being used to embarrass or humiliate someone. 

Given the potential for deepfakes to be used in a number of criminal ways it is also vital for children to understand the legal consequences of deepfakes. What might be played off as ‘a little bit of revenge’ could in fact be the sharing of indecent images of a minor and constitute the possession and sharing of child pornography. When young people are ignorant of the potential legal consequences of using deepfakes they are more likely to trivialise its use, thereby potentially normalising increasingly violent content. 

Preventative education can be done in numerous different ways. Specific subjects such as PSHE and RSE could be used to focus on the issue of deepfakes and the harm they can cause. Conversely, discussions around AI generated abuse could be introduced in numerous different subjects and in more exploratory ways depending on the age of the students. For example, GCSE students studying IT could be encouraged to think about the dangers of this kind of technology and what IT systems could be put in place to protect young people from deepfakes. Moreover, primary school children could be taught about consent in a simple way. This could then be expanded, in an age-appropriate way,  to include pictures that they didn’t want to be included in or have shared. As preventative education can be embedded in numerous places in the curriculum, schools have an element of freedom to adapt it according to the needs of their pupils and setting. Some schools may prefer more targeted approaches with specific lessons that focus on deepfakes, whilst other schools may want to embed AI safety and the threat of deepfakes in smaller amounts across several subjects. Whichever approach you choose, it must be tailored to suit the needs of your specific setting. 

Other ways in which schools can tackle the issues of deepfakes include:

• Training Staff – Investing in high quality training ensures that all school staff are knowledgeable about the risks associated with deepfakes and how to respond accordingly should an incident occur. Adequate training gives staff the confidence to recognise signs and act quickly, ensuring that all safeguarding responsibilities are met. 

• Engaging with News and Encouraging Open Discussion – Incidents of deepfakes are on the rise and both students and teachers have been victimised by them. AI generated abuse should be discussed in an age-appropriate and open manner so that young people are aware of the real world consequences of it. Engaging with the news shows students the harm that it causes individuals and the tragic impact that it can have. Whilst the goal should never be to scare children and any news shared must be done in a way that is suitable, when young people are unaware of the degree of harm that they may be causing it is significantly easier for them to continue engaging with it. However, when they see the news and hear from survivors it is more likely to encourage pause and reflection on what part their actions could be playing for one of their peers. This can also be used to facilitate open discussion in which students can ask questions without fear of judgement and with the goal to understand. This also enables staff to get a general temperature gauge on how students view AI generated abuse and to potentially recognise the signs that an incident has or may occur. 

• Write Clear Policies – New KCSIE guidance explicitly states deepfakes and the harm that they can cause. Reflecting this in school policies reaffirms a school’s dedication to their safeguarding responsibilities by recognising that risk can manifest in new and evolved ways. Schools that want to create a culture of zero tolerance should ensure that all policies are kept up-to-date and accurately reflect the risks that pupils face in modern life. 

How to Respond to Deepfake Incidents 

Whilst it may appear somewhat defeatist, it is arguably better to approach deepfakes incidents as a matter of when they occur, rather than if they occur. This ensures that all staff are prepared for such an event and time is not wasted due to shock. If a deepfake incident occurs schools should:

• Immediately contact the school Designated Safeguarding Lead (DSL). They will know what to do regarding contacting the police if necessary and contacting parents. Staff must not attempt to investigate the matter further themselves as viewing a deepfake may breach child abuse laws.
• Follow the school policy and ensure all safeguarding responsibilities are upheld. 
• Provide support to the victim of the deepfake. This may be pastoral care or contacting a school counsellor.  
• Review internal policies relating to safeguarding and deepfakes and update accordingly. 

---

Deepfakes pose a significant risk to young people. Preventative education is integral to keeping them safe and ensuring that they don’t engage with deepfakes and play their part in helping to keep people safe. All school staff must have an awareness of the risk that deepfakes pose and know what to do should a deepfake incident occur. 

---

Further Resources:

• What you Need to Know About Child-on-Child Abuse
• What You Need to Know About Sextortion
• Online Safety & Harms