Navigating the digital minefield: The impact of social media, AI, and deepfake pornography on kids

“A rising concern within Australian schools is the prevalence of deepfake pornography, a form of harassment occurring among peers and targeting teachers.”

By Dr Alexia Maddox, Senior Lecturer Pedagogy and Education Futures – La Trobe University

In the ever-evolving landscape of technology, particularly in the context of education, the integration of digital networked technologies has become an intrinsic part of a child’s life from a very early age. As we delve into the impact of social media, artificial intelligence (AI), and the rising concern of deepfake pornography on students and their learning experiences, it becomes apparent that while technology offers immense opportunities for growth and development, it also presents significant challenges that demand our attention.

The Positive Side of Tech in Education

Research indicates that children benefit most from technology when engaged actively, participating in groups, receiving frequent interaction and feedback, and connecting learning to real-world contexts. Technology serves as a powerful tool for developing critical thinking, analytical skills, and scientific inquiry. However, as children encounter technology in various facets of their lives, including social and family environments, informal learning, and social connections, it is crucial to acknowledge the potential negative impacts that can disrupt the learning process.

Digital Inclusion and Socio-Economic Disparities

In Australia, where we consider ourselves a digital society, access to and skills with technology remain uneven. The post-pandemic Digital Inclusion Index findings for 2023 highlight the persistent gaps and inequities. Schools are viewed as a critical intervention point to bridge these gaps and promote equitable and inclusive learning with technology. However, challenges persist, ranging from teacher capacity to varying school resources.

Social Media: A Double-Edged Sword

Among the identified concerns regarding technology’s negative impact on learning, distraction through smartphone usage and issues of digital literacy, especially in discerning information credibility, are at the forefront. Peer and family influence play a significant role in shaping a child’s perception of the world, and social media amplifies this influence. The virality, influence misuse, and distribution of misinformation in social media platforms pose a serious threat and highlight the need to develop critical thinking and digital literacy skills among students.

Deepfake Pornography: A Disturbing Trend

A rising concern within Australian schools is the prevalence of deepfake pornography, a form of harassment occurring among peers and targeting teachers. Deepfake technology, fuelled by algorithms trained to manipulate images, can be used for something as simple or fun as face-swap apps and filters on social media. However, it can also be used to dehumanise another. When used maliciously, these practices not only violate privacy but, along with other image manipulation technologies, can have negative implications on how children perceive their bodies, relationships, and life in general. The ability to doctor photographic, video or audio content in a hyperrealistic way for personalised attacks raises a critical concern about how this may impact the emotional and mental wellbeing of students, including those who may be generating or targeting others through this means. 

As highlighted by the eSafety Commission, the misuse of deepfakes for targeting and harming individuals isn’t solely a technological challenge. It stems from the manifestation of broader social, cultural, and behavioural issues in the online sphere. Therefore, addressing and preventing such incidents requires a comprehensive approach that combines both technological solutions and social interventions. 

Impact of Pornography on Children

Research highlights the pervasive exposure of children and young people to a sexualised digital media, flagging a link between pornography and shifts in attitudes and behaviours regarding sex amongst young people. The unintended consequences of such exposure can include increased risks of sexual violence, changes in moral values, and altered behaviour, especially among boys prone to antisocial conduct. The challenge for educators and caregivers lies in navigating the delicate balance between acknowledging children’s digital rights and young people’s healthy sexual expression and exploration while supporting their emotional and mental health.

Navigating the Digital Minefield

As a sociologist deeply invested in understanding the impact of technology on education, I emphasise the need for a nuanced approach. Technology is a powerful tool, but it is also experimental, with unintended consequences. The rapid pace of technological development and the lag in regulatory frameworks create an uncertain landscape that we must navigate together. The seedy underbelly of technology must be acknowledged and addressed to ensure a safe and stable environment for our children.

Protecting Young Minds: A Balancing Act

Educators and caregivers face the challenge of enabling children to navigate digital environments while ensuring their safety and wellbeing. The adoption of children’s digital rights by the UN and the eSafety commission emphasises the importance of a balanced approach. Being too heavy handed or overly paternalistic may hinder the development of well-adjusted individuals capable of engaging positively in digital contexts.

The Urgent Need for Change

The risks of not addressing the negative impacts of technology on children are evident in the growing challenges faced by today’s youth. Cyberbullying, exposure to online hate, sexual predation, and image-based abuse are real threats that can impact their overall wellbeing. Fostering self-confidence, security, and healthy socialisation becomes imperative to counter the potential repercussions of technology misuse.

Conclusion: Staying with the Trouble

To borrow a phrase from technology scholar, Donna Haraway, I advocate that we ‘stay with the trouble’. She encourages an active and engaged approach to complex and challenging situations and staying with problems, grappling with them, and working towards understanding and addressing them over time. 

In a world characterised by uncertainty, educators, caregivers, and researchers must engage in difficult conversations to understand the challenges children face in digital environments. The goal is not to demonise technology but to shape its responsible use. From the technology development side, the digital minefield can be navigated successfully with age-appropriate exposure, content moderation, and a focus on safety by design principles. Socially, we must engage in a collective effort that involves both shaping technology and guiding children through their digital journeys. Only by staying with the trouble can we pave the way for a future where technology aligns with our values and contributes positively to society and the wellbeing of our children and young people.

Dr Alexia Maddox is a Senior Lecturer in Pedagogy and Education Futures and Director of Digital Education in the School of Education at La Trobe University. She has a background in the sociology of technology and specialises in researching digital frontiers and socio-technical transformation.

Her most recent research draws together insights into emerging technology spaces with immersive environments/sensory experiences and technology trends surrounding artificial intelligence, machine learning, algorithmic processes, Web3, gamification, pleasure and play. Alexia combines a long-term interest in the social impacts of the internet with practical questions about human-technology encounters and how to research them.