School surveillance technology could hurt LGBTQ students, privacy advocates say

As conservatives stoke fears about books on LGBTQ subjects in libraries and school policies affirming transgender youth, another practice threatens LGBTQ students’ privacy and safety: Many school districts are monitoring students’ activity on devices such as cellphones and computers, viewing their email accounts, search histories, messaging, and chats, sometimes 24 hours a day.

Marketers of surveillance software for schools say it is a tool that can assist kids dealing with mental health issues and bullying, among other threats, but privacy experts say they worry that the use of surveillance software could result in harm to LGBTQ kids.

Elizabeth Laird, director of the Center for Democracy and Technology’s equity in civic technology project, said, “I think what’s important to know is that the vendors who are doing it — these are not neutral things. This is about someone who has opinions and values that is building algorithms to automate some of these things and the process that they’re using to do that.”

Laird said that the effect this technology has on LGBTQ students is unknown because there’s so much that these companies do not reveal about how they go about flagging certain LGBTQ-related terms.

While some political leaders and privacy experts ask questions about what this school surveillance means for marginalized students, including LGBTQ students, there has been a crackdown on all mentions of LGBTQ people and their experiences in education. In the past few months, conservative political leaders, parents, and right-wing groups have stepped up efforts to remove LGBTQ content from schools, including books and Pride flags. 

The technology companies Securly, GoGuardian, Bark Technologies, and Gaggle all provide schools with tools to use in the surveillance of students. 

Gaggle monitors over 5.2 million students in 1,500 districts across the nation, according to the company. It primarily focuses on students’ email accounts, Google Hangouts and Chat, and Microsoft Office use. “Gaggle monitors student activity on school-provided digital accounts and platforms 24 hours a day, 7 days a week, 365 days a year,” according to Gaggle. 

Gaggle flags LGBTQ terms such as “lesbian,” “queer,” and “gay” in its monitoring, as Gaggle CEO and founder Jeff Patterson acknowledged in an interview with CBS News. Patterson defended this practice by saying that the intent is to protect LGBTQ students from bullying.

However, such flagging can have other results: A high school student newspaper in Minneapolis reported that a student was allegedly outed to their family without their knowledge after the software flagged their activity.  

On Sept. 29, Sens. Richard Blumenthal (D-CT), Elizabeth Warren (D-MA), and Ed Markey (D-MA) sent letters to Securly, GoGuardian, Bark Technologies, and Gaggle demanding clarification of their products, how they work, and how they are used. 

“Artificial intelligence and algorithmic systems frequently mischaracterize students’ activity and flag harmless activity as a ‘threat.’ Students from minority or marginalized communities, including students of color and LGBTQ+ students, are far more likely to be flagged,” they wrote. They continued:

According to mental health advocates and experts, LGBTQ+ students are more likely to seek help online, and these tools frequently prevent them from accessing the health information they seek due to website filtering by student surveillance programs. The policing of students’ online activity may also be further discouraging students of color and LGBTQ+ students from reaching out to adults for help, leaving students without critical information and support.

Laird said that as schools provide kids with devices to do their schoolwork, “The rapid proliferation of devices has gone along with the rapid proliferation of the student activity monitoring software. And I don’t think some of the consequences, like outing students, were considered.”

According to a September survey from the Center for Democracy and Technology, 47% of teachers say the technology could have unintended consequences such as outing LGBTQ students, and 51% of parents say the same. Seventy percent of parents say they were aware that their child’s school had such software. 

Jason Kelley, associate director of digital strategy for the Electronic Frontier Foundation’s activism team, said the software comes at the expense of student liberty.

“Unfortunately, when it comes to software that scans for ‘potentially dangerous’ terms, it often overlooks sites, like some neo-Nazi websites, and overblocks and overreacts, assuming words like ‘gay’ are pejorative rather than simply descriptive. This ends up harming LGBTQ students in particular who have conversations online about their sexuality or are unable to use the web to learn more about LGTBQ issues.”

Gaggle said it requires all employees to undergo implicit bias training.

Patterson said in a statement, “The allegation that Gaggle ‘outed’ a child or is in any way targeting members of the LGBTQ+ community for discrimination or harm is patently false and is a gross misrepresentation of how our tool works.” Only in .0036% of cases does a flagged piece of content make it to a school district emergency contact, he said.

Patterson said that the company has resources for helping school administrators use the software: “We see there is a clear and urgent need to increase our training resources around helping struggling LGBTQ+ students, and this is now a top priority.”

The Center for Democracy and Technology report recommends steps school districts can take to limit the potential harms of surveillance. It says families should have full transparency from schools on how the data is used and collected; that schools should limit the amount of time that data is being collected and the activities the software monitors; and that schools should have restrictions on how they use the data for disciplinary purposes or for sharing it with police.

Published with permission of The American Independent Foundation.