An AI-powered gun detection system hooked up to a Baltimore County high school’s cameras mistook a bag of Doritos chips as a weapon — and called the cops on a 16-year-old student.
As local news station WBAL-TV 11 News reports, Taki Allen was enjoying the snack while sitting outside of Kenwood High School after football practice.
Twenty minutes later, he was visited by a small army of heavily-armed police officers.
“It was like eight cop cars that came pulling up for us,” he told WBAL-TV 11 News. “At first, I didn’t know where they were going until they started walking toward me with guns, talking about, ‘Get on the ground,’ and I was like, ‘What?’”
“They made me get on my knees, put my hands behind my back, and cuffed me,” Allen added. “Then, they searched me and they figured out I had nothing.”
“I was just holding a Doritos bag — it was two hands and one finger out, and they said it looked like a gun,” the student said.
The incident highlights glaring shortcomings with current gun detection systems, which are being rolled out at schools across the country. That’s not to mention the problematic privacy concerns of monitoring students with flawed AI tech or the outsized role law enforcement plays in public schools.
Besides false positives, gun identification software has proven unable to prevent deadly shootings, such as the one at Antioch High School in suburban Nashville earlier this year.
Other systems focused on gun detection have previously been accused of furthering racial biases, raising the possibility that Black students, like Allen, could be facing AI-facilitated discrimination while spending time at school.
The Baltimore County Public Schools system rolled out Virginia-based startup Omnilert’s gun detection tech last year. Once hooked up to public cameras, it can scan surveillance footage and alert police to potential weapons in real time.
Omnilert is only one of a whole host of US-based startups aiming to put an end to gun violence at schools, a demonstrably flawed alternative to gun control regulation.
According to the Baltimore Banner, Omnilert’s tech analyzes image frames from 7,000 school cameras for suspicious activity.
“Because the image closely resembled a gun being held, it was verified and forwarded to the Baltimore County Public Schools safety team within seconds for their assessment and decision-making,” Omnilert spokesperson Blake Mitchell told the Baltimore Banner.
“Even as we look at it now, with full awareness that it’s not a gun, it still looks like to most people like one,” he conceded.
According to FOX45 News, Omnilert later called the latest incident a “false positive” but maintained that it “functioned as intended: to prioritize safety and awareness through rapid human verification.”
Besides being scared for his life, Allen told FOX45 News that he had never received an apology from the school.
“They just told me it was protocol,” he said. “I was expecting at least somebody to talk to me about it.”
It’s a horrifying incident, highlighting how flawed tech is needlessly instilling fear in the hearts of innocent students.
“I don’t feel like going out there anymore,” Allen told FOX45. “If I eat another bag of chips or drink something, I feel like they’re going to come again.”
Allen’s relatives are understandably calling for more oversight.
“There was no threat for eight guns to be pointed at a 16-year-old,” his grandfather, Lamont Davis, told the Baltimore Banner.
More on gun detection: School’s $1 Million AI Gun Detection System Fails to Detect Weapon Before Fatal School Shooting