A Tucson high schooler got suspended when AI caught him typing — and then deleting — a joke

The debate over the role of artifical intelligence security tools in schools took a strange turn recently at Marana High School in Tucson.

A student, disappointed with his grade on an assignment, was sitting in his kitchen, talking to his mom. She suggested he should email his teacher about it — so he opened up his school-issued Chromebook laptop and began typing an email.

He had no idea that the email — which he never even sent — was about to upend his high school career.

Gary Grado wrote about the incident, involving a student referred to as NM, for the Arizona Agenda.

Full conversation

GARY GRADO: Being a, you know, a high school kid, you know, he felt awkward about talking to his teacher and making a request like that. And so the first thing he typed out was, “Mister, mister, I want to date your sister,” and he and his mom laughed. And then he deleted that, and he wrote, “Skibidi toilet, my grade is in the toilet,” and they laugh again.

So then the third thing he writes, “Gang gang, give me a better grade or I shoot up da school, homie,” and they're in all caps. And of course, his mom is not amused, and she tells him, you know, you better delete that before you accidentally send it. And so he did.

SAM DINGMAN: And to be clear, he says that this was just, you know, he was increasing the intensity of these jokes that he was making with his mom.

GRADO: Exactly, yeah, it was all a joke. And so he did not send the email. And these Chromebooks that schools issue to students, of course, they have all sorts of software that monitors their sites that they go to. They don't want the kids going to porn sites or how to make a bomb sites, those kinds of things, right? ... And so, the other thing that they do is they monitor their keystrokes. So —

DINGMAN: Yeah, well, so this, this is a very interesting part of your piece, and I didn't realize that software like this existed, to be honest. This is this piece of software, I believe it's called Gaggle, that tracks every keystroke that a student makes on a computer. And it's an AI tool, and it's trained to recognize, and, if necessary, report certain phrases.

GRADO: Correct, yes. And the principal actually got a, you know, a copy, a screenshot or something like that of the email. And, you know, within an hour, called the mother and they wanted to know where he was, first of all. You know, you could imagine it ... would be concerning. And Mom says, well, he's right here with me, and then she tells me about the email, and she's like, “Oh, you know, this is he was just joking. He never sent it, he never intended to send it.”

DINGMAN: I mean ... t wasn't even an email draft really, right? It was just a sentence he momentarily typed and then deleted.

GRADO: Yes, yeah, So they gave him a 45-day suspension. They ended up cutting it down to nine nine days, which, you know, is significant because anything over 10 days or more has to be approved by the school board — or he can appeal to the school board. I'm sorry.

DINGMAN: I see, I see. So it, it seems like it's possible that the school was trying to give the type of suspension that would make it impossible for NM and his lawyers to make a bigger deal out of this.

GRADO: Correct, correct.

Gary Grado

DINGMAN: So, one of the other factors here is that NM is is a very good student, correct?

GRADO: Apparently, yes. You know, he had the highest grade point average in his freshman class. He's in the academic decathlon, and then he does the things that high-achieving kids do outside of school. You know, he refs soccer, and he takes flight lessons on the weekends, and it part of the Civil Air Patrol and that sort of stuff. So, part of the concern is that this is gonna be a blemish on his record, and so his parents want to clear his name.

DINGMAN: Right, and I assume there were no other incidents on his record that suggested any other potential for violence.

GRADO: That's correct. That's correct. And, you know, his attorney made a point that during his suspension he was allowed back onto the campus to take the PSAT. He .... went there unsupervised. He sat around and waited for his mom to pick him up, you know, he sat alone, waiting for his mom to pick him up.

And when he returned to school, there was no — no one who followed up with him or questioned him about, you know, his mental health or how he was feeling. You know, did he have access to guns or anything like that.

DINGMAN: So, the Supreme Court has looked into this question of so-called student speech previously, right?

GRADO: Yeah, and the bedrock case that everything else follows is from 1969 where four students went into a school wearing armbands to protest the Vietnam War, and they were suspended. And the Supreme Court said, for a student to lose their First Amendment rights at school, they have to be materially disruptive or involve substantial disorder or invade the rights of others.

It wasn't until 2020 that the U.S. Supreme Court took up off-campus speech. [It] involved a girl who was on a Saturday at a convenience store with her friends. She's upset. She didn't make varsity cheer squad. She was a little upset about it, and, you know, kids nowadays, they have Snapchat. And instead of writing it on the bathroom wall or writing it in a notebook and closing your notebook, they post it.

And so she had a selfie of her raising her middle finger with a caption that had certain expletives. And so she got kicked off the cheer squad for a year. And the Supreme Court, what they said was, if it doesn't disrupt the school, you know, you can't regulate the speech. Although, they did say there are grounds for regulating off-campus speech, but they didn't give a hard rule about it, and so courts are gonna have to take these case by case. But —

DINGMAN: Well, and this was, this was tricky in NM's case, right? Because a month before NM briefly typed and then deleted this sentence, there had been a school shooting in Georgia. And then there had been a series of hoax threats all across the country, right?

GRADO: That's correct. And, you know, the school administrators, they got a tough job. You know, I wish Marana had spoken to me, and they could have explained a little bit about that in their own words.

DINGMAN: I mean, it makes me wonder how common it is that kids like NM are doing similar things like typing jokes or maybe, you know, trying to access websites they shouldn't be trying to access without realizing just how closely they are being tracked.

GRADO: Yeah, and NM was able to sue. He got the help of ... public interest law firm. I'm sure that kids get in trouble for this kind of stuff all the time, but they just don't have the help of a public interest law firm or, you know, their parents can't afford to get a lawyer. And you've probably got these cases where the punishment doesn't fit the crime.

DINGMAN: All right, well, I can imagine parents and schools will be watching the outcome of this case very closely. I've been speaking with Gary Grado, a journalist who covers politics and courts and wrote about NM's case for Arizona Agenda. Gary, thank you so much.

GRADO: All right, Sam, thank you.