On February 10, 2026, Australian author Lauren Ashley Matrosa was convicted of possessing and distributing child abuse material. However, no children were involved or harmed. The conviction was based on an erotic novel that she had written, in which the only sexual activity takes place between adults play-acting a “Daddy Dom/little girl” (DD/lg) fantasy. Nevertheless, the judge ruled that Australia’s definition of “child abuse material” (which in other jurisdictions is known as CSAM, CSAEM, or child pornography) is broad enough to include even fictional representations of such roleplay.

A kinky canary in the coal mine

It may seem paradoxical that representations of DD/lg (or ageplay, the broader term for age-based BDSM roleplay) should be illegal while the actual practice of ageplay remains legal. BDSM-related fantasies are widely practiced as a means of exploring power, identity, or even processing past experiences. Yet despite this, ageplay communities are frequently stigmatised as inherently abusive or as proxies for child sexual abuse (CSA), even though no minors are involved.

A prominent UK child safety advisor who supports banning DD/lg content has described it as the currency of predatory paedophiles, precipitating a ban of the topic from Facebook. Today, ageplay-related search terms are shadowbanned on major platforms or trigger deterrence messages warning users that they may be seeking illegal content.

My argument has always been that this is the wrong approach. It’s true that BDSM content is offensive to many and that people should not be exposed to it without their consent. But this is why “trigger warnings” in a book, or equivalent tags and filters for online content, are a better approach than blanket censorship or even criminalization. Matrosa’s book had a myriad of trigger warnings, but this was judged irrelevant by the court, who assessed the legality of the book not by the standards of the BDSM community that it was written for, but rather by the standards of a hypothetical offended observer.

UN moves to loosen the CSAM definition

The Matrosa case is not an anomaly. It is the domestic manifestation of a broader shift in how institutions define and regulate “child abuse material.” Just one week prior to Matrosa’s conviction, the United Nations agency for children, UNICEF, issued a statement urging states to loosen their legal definitions of CSAM to include AI-generated content “even without an identifiable victim”.

While this statement does not expressly address the case of novels, a 2019 UN draft proposal had recommended that states extend the definition of CSAM to include “written materials in print or online”. When this draft recommendation encountered significant opposition, including objections from over 17,000 signatories to a petition, the final document settled on a less specific recommendation that “representations of non-existing children or of persons appearing to be children” should be covered.

The problem is this: CSAM is a term that doesn’t get stronger the more you pack into it. Rather, it gets weaker. The term Child Sexual Abuse Material was expressly coined to replace the term child pornography because the latter term fails to infer that those depicted in such material are victims of child sexual abuse. The moral gravity of the term, and the justification for associating it with lengthy criminal penalties of imprisonment, are weakened when it is loosened to include victimless materials within its scope.

Deepfakes are a problem we can solve

With all that said, the UNICEF statement does call attention to a very real and legitimate concern: generative AI is being used to create sexually explicit deepfake images and videos of real victims. Across 11 countries studied by UNICEF, ECPAT, and Interpol, as many as 1.2 million children had their images manipulated into deepfakes in the past year, most commonly by their own peers. Despite the virtual nature of the images, this constitutes a form of image-based sexual abuse, causing direct and profound harm.

Clearly, something must be done. But loosening the definition of CSAM to include victimless content is neither necessary nor sufficient to address this problem. The harm of deepfakes arises from the non-consensual use of a real person’s image — not from the abstract existence of offensive synthetic imagery. Expanding criminal categories to cover all AI-generated content risks diverting attention and resources away from identifying victims, removing abusive material, and holding perpetrators accountable.

Here are three better solutions:

• Education: If, as UNICEF reports, most deepfakes are created by peers within schools, then our response must account for the reality that many perpetrators are themselves minors. Indiscriminate prosecution can exacerbate harm, entrench stigma, and disrupt rehabilitation among minors who may not fully appreciate the consequences. Instead, policy should prioritize education, digital literacy programs, age-appropriate interventions, and restorative justice approaches. Criminal penalties may be justified in egregious cases involving intent to harass or exploit, but proportionality must prevail: prevention and education outperform punishment in addressing youth-driven behavior.
• NCII laws: Targeted, victim-centered alternatives to CSAM laws already exist and work better in many contexts. Numerous jurisdictions have enacted laws on non-consensual intimate imagery (popularly called revenge porn, and a subset of tech-facilitated gender-based violence or TFGBV), some explicitly extending protections to AI-generated content featuring identifiable individuals. These frameworks offer practical tools: rapid content takedown mechanisms, civil damages for victims, and platform liability for failing to remove clearly unlawful material.
• Data protection laws: In other jurisdictions, particularly within Europe, existing data protection frameworks offer another more targeted solution to the problem of AI deepfakes. The EU’s General Data Protection Regulation (GDPR) treats biometric data — including facial recognition data and image embeddings used in AI systems — as a special category requiring a lawful basis for processing. The scraping, storage, and manipulation of children’s images to generate deepfakes may already violate these provisions. Rather than expanding criminal definitions of CSAM to encompass all synthetic imagery, enforcement efforts could focus on unlawful data processing and misappropriation of likeness.

Conclusion

When it comes to generated or artistic content, the decisive question is simple: does it exploit and harm a real, non-consenting person? If it does — as in the case of deepfakes of real children — it is a form of abuse and demands a firm, targeted response. If it does not, then however offensive it may be, it does not belong in the same criminal category.

Edge cases like AI-generated deepfakes have led some to argue for collapsing all depictions of minors, real or imagined, into the definition of CSAM. But conflating fiction with victimization weakens both enforcement and principle. Criminal law loses clarity. Resources are misdirected. And the moral gravity of the term “child sexual abuse material” is diluted.

The case of Lauren Matrosa illustrates where this path leads: criminal liability imposed not for harm, but for offense. A free society does not protect only inoffensive art. It protects art and literature precisely because criminal penalties must be necessary and proportionate, imposed only to prevent or punish conduct that causes real and identifiable harm. This is not a radical proposition; it is a cornerstone of international human rights law. Offensive art shouldn’t be distributed without safeguards, but it should be allowed to exist.

Real CSAM on the other hand — including deepfakes of real children — is not merely offensive. It is abusive. Our response to it demands precision, enforcement, and support for victims. Therefore the solution is to target the harm directly, through measures such as preventative education, data privacy frameworks, and targeted image abuse laws — not to expand the existing criminal category of CSAM until it loses its meaning.

This May, the Center for Online Safety and Liberty (COSL) will convene a session at RightsCon 2026 to examine these sensitive and complex issues directly. RightsCon is the world’s leading summit on human rights in the Internet age, attended by Internet companies, governmental bodies such as UNICEF, and civil society activists.

Our session will explore how policymakers can address AI-enabled abuse without collapsing fiction into criminality, how tools and frameworks for addressing the issue can be deployed more effectively, and how freedom of expression principles can be preserved while strengthening protection for real victims. The discussion will include lawyer and activist Mar Diez, Professor K S Park of Open Net Korea, Shambhawi Paudel of ILGA Asia, and Emma Shapiro of the Don’t Delete Art project- each bringing expertise on digital rights, platform governance, gender-based violence, and artistic censorship. We would welcome you to join us there.

Originally published at https://c4osl.org on February 18, 2026.

When Jonas (not his real name) posted a photo of himself in sports clothes to his own Instagram account, the last thing that he expected was for his account to be suspended for suspected child exploitation. Although Jonas is in his 20s, and his sports clothes do not sexualize him at all, evidently an AI image classifier used in automated content moderation had falsely flagged his upload as possible child sexual abuse material (CSAM). Jonas initially reacted with disbelief when this happened — but this was soon followed by a mounting sense of fear about what being under investigation for child exploitation might mean for him.

Adriana (not her real name) was also shocked when she was banned from a popular adult-only platform after its AI moderation system incorrectly flagged her use of common terms within BDSM communities. Despite the platform claiming to use human review, no one assessed her case before the ban was enforced, revealing a significant operational failure in how moderation systems are applied. She writes:

As a survivor of CSA, being banned for safe, sane, and consensual kink practices was deeply triggering. It feels hypocritical to punish adults for sexual expression while simultaneously failing to build stronger safeguards against child exploitation. Human review should be mandatory whenever content is flagged — both to protect children and prevent false positives.

Wrongful arrests and lost memories

Jonas and Adriana are both right to be concerned. When innocent content is reported as child exploitation, innocent lives can be ruined. In 2024, a grandmother was reported to Australian police by TikTok over a non-sexual massage video of her infant granddaughter, sent to the child’s mother; media coverage at the time falsely labeled her as a child abuser. In 2022, Google reported a father to police because he used Gmail to send medical photos of his toddler to the child’s doctor. Both were eventually cleared of wrongdoing, but have received no redress over being falsely accused.

There are many similar stories. Thousands of users not only of Instagram, but also Facebook and YouTube, are facing platform bans and the prospect of police investigation. Some victims of these bans talk of losing thousands of sentimental photos, while others have had online businesses ruined. For others, the bone-chilling fear of being reported to authorities weighs on them more heavily than losing their account. When a young gay man engaged in consensual sexual role-play with another adult was arrested by police on child exploitation charges in 2021, he couldn’t face the shame that the false charges brought upon him, and took his own life.

Internet platforms bear a heavy responsibility to keep their platforms free of real child sexual abuse, and some are not discharging this responsibility very well, or are even contributing to the problem. But shortcomings in online child-safety reporting systems cut both ways. While it is unacceptable when real CSAM remains online without being taken down, falsely accusing innocent people of serious crimes isn’t a trade-off that we should have to accept.

AI is worsening the problem

This problem isn’t going away, but rather it’s getting worse. One reason is increasing reliance on inaccurate AI classifiers, with even photos of family pets being flagged as child abuse, and apparently never undergoing human review. Platforms must report genuine CSAM, and the initial use of AI systems to help flag it in public uploads is justified. But this should never result in an immediate account ban without manual review by a human moderator.

Another factor contributing to the rise in innocent people being flagged for child abuse is that platforms are becoming increasingly risk-averse, under pressure from lawmakers and an electorate of concerned parents who aim to hold them responsible for online child sexual abuse. Laws both current and threatened are shifting increasing liability onto platforms for getting it wrong — with the predictable result that they are taking fewer chances, no matter the cost to users who are wrongly accused.

One option to address this problem is for victims of over-reporting to use the same tactic — hitting platforms in the hip pocket when they get it wrong. That’s what William Lawshe did, after being wrongly reported to authorities by Verizon over what were plainly 18+ erotic images, even bearing adult site watermarks. Lawshe sued Verizon and its CSAM-scanning service provider to seek compensation for the disgrace and health problems that he suffered after being carelessly and wrongly reported to authorities. A final decision is yet to be handed down, but the court has already allowed key claims to proceed.

How platforms can do better

Nobody should have to resort to a lawsuit simply to keep their name and their online presence clear of false child exploitation allegations: prevention is, as usual, better than cure. A large part of that responsibility falls on platforms simply to do a better job: to draw their child exploitation policies narrowly and precisely, and to involve humans before taking actions such as account bans or referrals to law enforcement.

For those that consistently fail to live up to their responsibilities towards innocent users, sunlight might be able to help. One of the first projects of the Center for Online Safety and Liberty (COSL) was the launch of the Harmful to Minors transparency archive, where false child exploitation takedowns are published for public critique.

Later in 2026, COSL will also be publishing a second edition of our Drawing the Line Watchlist. The first edition of the Watchlist evaluated ten countries around the world for how accurately their laws draw the line between personal expression and lived abuse. The second will extend this analysis to the policies and enforcement practices of Internet platforms, including how well they safeguard users against false positives, provide appeal mechanisms, and limit automated escalation.

Over the longer term, COSL is also pursuing a vision to foster and establish alternative platforms and tools that hold safety and liberty in better balance, allowing a diverse range of creative content and personal expression to flourish, without sacrificing safety. These projects include our privacy-first offshore hosting service Liberato, our upcoming fan community Fan Refuge, and our open source content warning system, Dead Dove.

Conclusion

The fight against child sexual abuse online is too important to be undermined by blunt, unaccountable systems that punish the innocent. When platforms treat false positives as an acceptable cost of doing business, they shift the burden of their own errors onto ordinary users, who are left to face fear, stigma, and lasting harm alone.

Child safety and civil liberties are not opposing values, and we should reject any approach that claims otherwise. The solution is not less vigilance, but better vigilance: narrower rules, human judgment, transparency, and accountability. Until platforms adopt those principles, innocent people will continue to pay the price for mistakes they did not make.

Originally published at https://c4osl.org on January 11, 2026.

When Danish AI artist “Barry Coty” was arrested in 2023, it came as a surprise. He had believed that the fantasy AI porn images he was creating were a harmless outlet that might reduce demand for real child abuse material. Instead, he found himself at the center of a global Interpol operation and facing criminal charges that would help reshape laws across multiple countries.

For prosecutors, the case was clear-cut: Coty had created and distributed sexual imagery depicting minors, regardless of whether those minors were real or AI-generated. His case became a catalyst for an upcoming 2025 law in Denmark banning AI-generated sexual content involving minors, part of a global wave of legislation that has swept through 39 U.S. states and the European Parliament, which approved a directive to criminalize AI systems used to generate such content. Globally, pressure is rising for countries to criminalize more virtual sex offenses.

This movement reflects a seemingly “common sense” principle: content depicting anyone under 18 sexually should be illegal, whether the subjects are real or virtual. But scratch beneath this consensus, and a more complex picture emerges-one that encompasses far more than AI-generated images. As researcher Aurélie Petit recently discussed with me, while AI deepfakes may grab the headlines, the end result of a zero-tolerance approach is that AI images are treated alongside fan fiction, art, memoirs, and more, principally from queer creators and women, all within the thought-terminating category of child sexual abuse material (CSAM).

This article explores the views of a growing number of experts, including lawyers and psychologists, who challenge this approach, arguing that it drives over-criminalization, stifles artistic expression, disproportionately harms marginalized communities like LGBTQ+ individuals, and even obstructs effective sex abuse prevention efforts. Through these insights, we examine whether the rush to criminalize AI-generated content (and more) oversimplifies a complex issue-and what’s at stake when nuance is ignored.

From protecting victims to policing fiction

In response to a proposal in the 2026 New York budget to redefine felony sex offenses to include AI-generated content, the New York City Bar Association, a 23,000-member organization of prosecutors, defense attorneys, and judges, voiced strong opposition, alongside other groups such as New York Legal Aid. The City Bar argued:

The purpose of criminalization is not to punish the distasteful proclivities of adults who view it, but rather to disincentivize recorded abuse and thereby prevent future victimization. Part L’s proposed amendments move away from this rationale of protecting children, as no actual children are involved-simply computer-generated representations-and embrace punishing people for their sexual fantasies. This would be a sea change in the justification for laws against child pornography.

While this shift toward criminalizing fantasies may seem novel in the U.S., other countries have embraced it for decades. In 1993, Canadian lawmakers confidently declared that “it is wrong to have these fantasies and wrong to write them down. Period.” Within a year of passing a law to criminalize such content, police were raiding art galleries and arresting artists and authors for works deemed child pornography. In Australia, where a parody Simpsons cartoon was prosecuted as “child abuse material”, author Lauren Tesolin-Mastrosa faces similar charges in 2025 over a fictional erotic novel depicting adults. Authorities there not only use the same laws against real and fictional crimes, but don’t even keep track of the difference in official statistics.

Now, the U.S. is following Canada’s lead, with police also literally raiding art galleries over works mislabeled as child pornography. Legal and human rights experts are pushing back, arguing that the distinction is critical. In the Australian Simpsons case, Justice Adams emphasized:

At the outset it is necessary to appreciate, as I think, that there is a fundamental difference in kind between a depiction of an actual human being and the depiction of an imaginary person. … There was a tendency in the arguments before me to suggest that the distinction is merely one of degree. This is quite wrong. Such an approach would trivialize pornography that utilized real children and make far too culpable the possession of representations that did not.

Challenging the link between fictional content and harm

This blurring of lines between fictional and real abuse is fueling a broader wave of U.S. laws that use similar logic to target sexual content more generally. The same unproven assumptions that drive AI-specific legislation-that consuming certain content leads to harmful behavior-now appear in laws targeting pornography broadly. California’s AB 1831 asserts, without evidence, that “pornography may increase sexually aggressive thoughts and behaviors” and that AI-generated content “normalizes and validates the sexual exploitation of children.” Age-verification laws like Texas’s HB 1181 apply this reasoning to restrict access to content deemed “harmful to minors,” sweeping in manga and anime alongside actual pornography. A proposed federal Interstate Obscenity Definition Act would further expand criminalization of sexual content based on these same theoretical harms. The common thread: laws justified by the claim that consuming sexual content promotes abuse-but does this “slippery slope” argument hold up?

The New York City Bar Association’s submission opposing the redefinition of felony sex offenses to include AI-generated content cites numerous studies that challenge that assumption. A 2010 study found sex crimes, including child sex offenses, declined during periods of unregulated pornography access. A 2023 meta-study on fictional sexual materials (FSM), including depictions of minors, found no link to sexual aggression and suggested FSM use might reduce harmful impulses in high-risk individuals through a “cathartic effect,” proposing it as a harm reduction outlet.

A 2008 study examined claims made during the passage of an earlier U.S. law, the 2003 PROTECT Act, which equated penalties for virtual image-based crimes with those involving real children. It found no evidence that FSM increases acceptance of child sexual abuse, directly rebutting the Act’s rationale. In 2012, a Danish sexological clinic also advised against banning FSM, citing no clear harm. Yet as the political winds changed, Denmark reversed course with its 2025 law criminalizing AI-generated content, spurred by an Interpol operation targeting synthetic CSAM networks. And this brings us back to Barry Coty.

An agenda to raise penalties for fictional sex crimes

Danish AI artist Barry Coty (a pseudonym) was arrested in 2023 for creating and distributing AI-generated depictions of minors in sexual contexts via a paid subscription platform, as part of an Interpol sting operation called Operation Cumberland, that also netted a string of other arrests around the world. Coty’s case, one of the first targeting an AI porn creator, highlights the global push to equate fictional content with real abuse. He contacted me unsolicited in order to tell his story.

In January 2025, Coty pled guilty to the charges and originally received a sentence of one year and three months, partly suspended, and 200 hours of community service. But the prosecution appealed the verdict, hoping to establish a stricter precedent around synthetic material. Today (June 12, 2025), they succeeded, increasing Coty’s sentence to eighteen months of actual jail time. Coty plans to appeal that decision to Denmark’s Supreme Court. He explains:

My intentions for making and distributing these images have always been to reduce the amount of real CSAM material being sought out and shared on the internet, with a less demoralizing replacement to the individual consuming them, and of course to reduce the suffering being continuously done to children that are victims of real CSAM by their abusive and non-consensual images being spread around the internet.

Policymakers face genuine challenges here. The rapid emergence of AI technology has outpaced existing legal frameworks, creating uncertainty about how to protect children while preserving legitimate rights. The visceral public reaction to any content involving minors-even fictional-creates enormous political pressure to act decisively.

There also seems to be no question-going purely from descriptions of them-that the images Coty produced would be perceived as confronting and offensive by most. There’s little question why authorities target these images as a starting point to broaden the criminalization of fictional content-it’s unsettling to consider their existence online, even in obscure sex forums, and to acknowledge that they fulfill a sexual interest for some. For many, criminalizing these images serves as a convenient stand-in for criminalizing that interest itself.

The wrong tool for a misunderstood problem

But the law is a blunt tool, and the wrong one, for managing the existence of paraphilic sexual interests within the community, especially among those, like Coty, who have gone out of their way to find harmless artistic outlets for them. Criminalization advocates have suggested that the distribution of such images in niche online sex forums carries a “terrifying potential to flood the internet with a tsunami of abuse imagery”, or even that this could trigger the “conversion” of those who unwittingly view them into pedophiles.

But both of these are far-fetched suggestions. The reality is that almost all online platforms strictly rule against AI-generated sexual content with characters resembling minors, and there are already effective tools to weed it out and to distinguish it from real abuse imagery. If such imagery were to be distributed on a mainstream platform, it would doubtless be reported to authorities, who have clearly affirmed that they already possess the legal authority to prosecute.

As to the possibility that unwitting viewers of these images could be transformed into pedophiles, this smacks even more strongly of fearmongering. It is widely accepted by experts that accessing a particular type of sexual content is a sign that a person already has an interest in it, rather than being the catalyst for them to develop a new sexual interest, especially not in something that they previously found disgusting. As Coty stated to me, “most people feel an innate repulsion towards such imagery”.

There are, and always will be, Internet users who reject child sexual abuse, while at the same time being drawn towards representations of underage sexuality. Their reasons for doing so, and the representations that they seek out, both exist on a spectrum. Within that spectrum are many legitimate artistic works, disproportionately created and consumed by sexual abuse survivors, LGBTQ+ people, young people, and women. Some may even escape misclassification as CSAM and enjoy critical acclaim. In a 2024 submission that I wrote to the Australian government urging reforms to its classification system, I wrote:

Mainstream TV series such as Euphoria (depicting characters represented as children having sex) and Game of Thrones (representing characters engaged in incest) are routinely passed with MA 15+ or R 18+ ratings… But while mainstream Hollywood TV and movies can be classified quite leniently, it is no exaggeration to say that if a person enters the country with Japanese cartoons that depict exactly the same subjects as Euphoria or Game of Thrones, they stand a very real risk of being arrested.

It will never be possible, nor would it be desirable, to erase all such representations from the Internet and to criminalize those who seek them out. There will always be those who create or consume representations of minors that make us uncomfortable, or who do so for reasons we find uncomfortable. In cases in which the consumption of such content crosses the line into fueling problematic behaviors, the approach that professionals recommend is one of harm reduction and prevention, not criminalization. So the real solution may simply be for us to make peace with this, and allow those professionals to do their jobs.

Fictional material is not CSAM

The conflation of fictional and real sexual abuse material represents a calculated political strategy decades in the making. Advocacy organizations and their allies in government have systematically engineered linguistic shifts to expand the scope of what constitutes sexual crimes, often in ways that serve interests beyond actual child protection.

For example, the 2016 Luxembourg Terminology Guidelines began to establish a new international norm that the term “child sexual abuse material” should be used in preference to “child pornography” when referring to “material that depicts and/or that documents acts that are sexually abusive and/or exploitative to a child.” While this makes sense, the devil in the details-made more explicit in a 2025 Revision-was to include fictional content also, a move completely at odds with the stated rationale that undermines the term’s gravity. Even the original term “child pornography” is a more accurate term for sexually arousing images that don’t record abuse, as FBI Special Agent Kenneth Lanning observes, writing:

The efforts to encourage use of this new term is a good example of well-intentioned people trying to solve a problem by emotionally exaggerating the problem… It is interesting to note some of those advocating for use of the term child-abuse images also advocate for criminalizing as child pornography visual images that do not even portray actual children. You cannot have it both ways.

Common justifications given for the criminalization of AI-generated images depicting minors are that they may be made in the image of real individuals, be generated using models that were trained on real CSAM, or be used in grooming children. Such cases involve specific abuses that should be prosecuted directly, not used to criminalize all fictional works. Barry Coty insists that none of those justifications apply in his case. No real CSAM was ever found in his possession, and he has no history of sex offending. While small volumes of illicit content have been inadvertently included in the training data of mainstream generative AI models, Coty insists he never used any unlawful content in editing his creations, and he described to me in some detail the technical process that he followed to achieve this.

In such cases, whether AI tools are involved or not, equating fictional content to photos and videos recorded at actual crime scenes trivializes the real suffering of the victims of those crimes. When policymakers obscure this distinction with linguistic sleight-of-hand, it should be called out as dishonest, and their real political motivations exposed: inflating “child abuse material” statistics, providing moral cover for broader censorship campaigns, and creating new categories of criminals to justify expanded enforcement budgets and powers. Coty writes:

Why are the lawmakers to keen to include fictional abuse into real abuse statistics? I think there is some incentive to do so in order to argue why the state should have more oversight into private messages through online surveillance. In this way, fictive child pornography becomes a catalyst/scapegoat to finally get better tools to go after drug dealers and terrorists.

Calling this out is fraught, as honest discourse on this topic is often ruthlessly punished. Yet, one cannot claim moral authority while blurring actual abuse with offensive fiction for broader political ends.

The fight back begins with Fan Refuge

Censorship of fictional sexual materials is an issue that sits at the very nexus of the four priority areas of the Center for Online Safety and Liberty (COSL): promoting safer hosting, supporting fans, combating cyberbullying and abuse, and engaging in legal advocacy. At the very core of our mission-and even our name-is the firm belief that it is neither acceptable nor necessary to sacrifice online liberty for the sake of safety.

That includes upholding the liberty for creators and fans to express themselves without fear of prosecution over fictional content, while at the same time ensuring that nobody is exposed to potentially offensive content, even if it is fictional, without their consent.

Here’s how we’re putting that into practice, starting right at home. First, this month we are launching a crowdfunding campaign for a new creator platform called Fan Refuge, which will launch as a testbed for some open source trust and safety tools that we’ve been developing. Fan Refuge won’t be an adult content platform, and it won’t allow AI generated content at all. But it will prioritize empowering its users to curate their own experiences, rather than imposing site-wide censorship on arbitrary moral grounds.

Justice for Real Survivors

Second, we’re launching a major new advocacy project titled Justice for Real Survivors, directed at the problem that politicians and policymakers are intentionally blurring the lines between fictional and non-fictional sex crimes. The project’s aim is to begin to reshape laws, policies, social norms, and language to prioritize real sex crimes with real victims, and to clearly distinguish them from crimes under obscenity or censorship laws.

To kick off the Justice for Real Survivors project, we are convening a diverse advisory board who will develop a statement of principles setting out the harms of conflating sex crimes with fictional, artistic, and educational texts. These principles will be opened for broader sign-on, and will create a framework for other activities under the project, including research, coalition-building, policy advisory, public campaigns, and strategic litigation support. (The opinions expressed in this article are my own, not those of the advisory board.)

Only miniscule funding is made available for resources for the prevention of sexual abuse. So we’re grateful to have already secured the interest of a philanthropic donor in supporting the project’s first research output. This will be a major legal review which will provide a comparative analysis of the treatment of fictional sexual materials across ten countries, and assess the compatibility of these legal regimes with human rights standards. We hope to begin this survey in the third quarter of 2025.

Conclusion

The rush to criminalize fictional content-from AI-generated images to erotic novels-promises safety but delivers censorship, punishing creators while diverting resources from real victims. Policymakers, swayed by unproven claims that niche fantasies will “flood” the mainstream or “convert” viewers into predators, employ linguistic shifts that equate offensive art with heinous crimes. Yet, as science shows, porn consumption reflects pre-existing interests, not new ones, and banning fictional outlets will only push consumers into darker corners and obstruct harm reduction.

The conflation of art and abuse isn’t just misguided-it’s harmful. With fewer than 4% of real sexual assaults leading to convictions and only 3.5% of CSAM reports investigated, survivors are sidelined as authorities pursue victimless prosecutions. Marginalized creators- often survivors themselves, along with LGBTQ+ artists, young people, and women-face censorship or prosecution for works that challenge norms, while honest discourse is stifled by stigmatizing attacks. The solution lies not in erasing uncomfortable content but in embracing nuance: prioritizing real victims, empowering creators, and trusting professionals to prevent abuse without sacrificing liberty.

Barry Coty’s case represents one end of the spectrum-his AI-generated content would be deeply offensive to most people, and few are likely to rush to his defense. But the legal principles established through his prosecution won’t stop with creators like him. The same frameworks now being used to criminalize his work will also expand to target fan fiction writers, manga artists, abuse survivors processing trauma through art, and LGBTQ+ creators exploring identity and sexuality. When we normalize prosecuting people for offensive but victimless content, we create precedents that reach far beyond the most unsympathetic cases.

Through Fan Refuge and Justice for Real Survivors, we’re forging a path forward-building platforms that respect user choice, reshaping laws to focus on actual harm, and amplifying survivors’ voices. But change demands courage. Will we confront the uncomfortable truth that fictional content isn’t abuse, or cling to moral panic at the cost of justice? The choice is ours, and the stakes are high.

Originally published at https://c4osl.org on June 12, 2025.

This is a transcript of a conversation between Jeremy Malcolm, Brandy Brightman and Aurélie Petit held on May 23, 2025 for the podcast Beyond the Filter, which is presented here edited for length and clarity.

Brandy: Hello and welcome to Beyond the Filter, a podcast about censorship. My name’s Brandy…

Jeremy: And my name’s Jeremy. In this episode, we’ll be talking with a special guest, Aurélie Petit, who is a social and visual researcher of animation and technology and a PhD candidate in the Film Studies Department at Concordia University Montreal. She’s just published an article in the Journal of Porn Studies titled “The Limits of Zero Tolerance Policies for Animated Pornographic Media.” Welcome to the podcast, Aurélie.

Aurélie: Hi, thank you for having me.

Jeremy: So congratulations on the article. … First, I wondered if you would relate the story that you open your article with about what happened on Twitch in 2023.

Aurélie: Yes, Twitch, you know, as a response to fans’ criticism that the platform was targeting sexuality where they felt it was unjustified-and a lot of queer people were being targeted, a lot of women, everybody who was showing too much nudity, like, if you could see your cleavage on Twitch, you would be considered basically doing porn-and so the platform started to really soften its policies over adult content. And especially when it was about animation, you know. And they were saying, if you were drawing, sculpting, or making animation on Twitch, and it was drawing naked people, it was fine. And in one day, so many people made porn, like animated characters in porn, that the day after, the platform decided to cancel its policies. In one day, which is crazy for like a platform like Twitch.

Jeremy: Well, I think that’s not the only example of something like that happening. Didn’t OnlyFans also turn their policies around in a couple of days or so? So, yeah, I think one of the themes of your paper is that these platforms maybe don’t have very well thought-through policies on animated content, versus real content that depicts the same thing. So, what did you discover in the platforms that you looked at? I think there were 30 platforms, right?

Aurélie: Yeah, so, you know, this example of Twitch, I used it to open the question of, as you say, that platforms don’t know what to do with animation. And it ended up having them write policies that are totally weird or go against common sense. And so, I was curious of this question of, how do you moderate animation? And especially how do you moderate animation on a pornographic platform that actually have to be like, is this “good sex” or is it “bad sex”? And so, I decided to find 30 pornographic websites and I took their policy documents and, you know, I was focusing on this question of age. So, I wanted to see, when they talk about age and, you know, child pornography, how do they include animation in this discussion?

Jeremy: Yeah, so I noticed it’s interesting that you used the term “animated child pornography” because child pornography as a term is not really used anymore, is it? We’ve started to move to child sexual abuse material. But speaking for myself, I always feel uncomfortable in using the term child sexual abuse material when there’s no child sexual abuse, right? When there’s no child. And in that way, even though child pornography is like an antiquated term, it’s more appropriate in some ways.

I also feel uncomfortable talking about child sexual abuse materials, you know, and talking about child exploitation, knowing that, of course, it’s a spectrum.

Aurélie: I also feel uncomfortable talking about child sexual abuse materials, you know, and talking about child exploitation, knowing that, of course, it’s a spectrum. Like, you know, computer-generated imagery, sometimes that is AI, where the child may be used as a model, you know, and it’s maybe part of an exploitation circle or trafficking, you know. But then you also have the Bart Simpsons of the world, as I call them, you know, and it’s like, it’s a spectrum. And what I was able to see in a lot of those policies is like, they don’t make a lot of space for this spectrum, you know? A very problematic, hyper-realistic animated image of a child in a sexual context was, at least on paper, put at the same level as a very cartoonish representation of a child.

Brandy: And I feel like animation is just fundamentally ambiguous because with live action, you’ve got an actor involved or a person or whatever, and they’ve got an unambiguous age. No matter what they look like, no matter what age they’re playing, they have a true age, which you can use to ultimately judge the content. But with animation, they don’t have a true age. They just have however old the creator decides to say they are on that day, and however old they look. And the age of the character may change on the creator’s whim, and you can’t really judge purely on appearance because in reality, you can’t really judge purely on appearance either. And the stylization of animation makes things even more ambiguous. A creator may have a character that they say is usually 15, but whenever they appear in pornography, they may just claim that they’re 18, even though they draw them exactly the same way.

Jeremy: Aging up.

Aurélie: Yeah. That’s why I’m more interested in talking about problematic representation, you know, than about age and gender. Like, we can have a conversation about, you know, representing young looking characters in pornography. And it’s a better conversation than trying to argue which age is a character. But, as a community we can wonder, do we want the imagery of pornography to be mostly young looking people. Like, do we want it to be the main representation of animation? Knowing that it’s not always the case, but it’s also often made by, you know, sexist animation studios, sexist creators. Like, it’s a more interesting conversation for me.

Jeremy: Isn’t there also a cultural history behind, particularly, as you mentioned, lolicon and shotacon? Those are particular genres that have a stylized look for their characters that isn’t directly related to age, right? And I think you found only two of the platforms that you looked at actually used that fandom terminology to talk about particular types of content that, by those names, you know, referring to them as shotacon and lolicon content.

Aurélie: Well, again, if we link it to this idea of problematic media representations, the history we have for the beginning of lolicon is that it was a genre that kind of always existed. You can trace it back to the golden age of manga, but then it really solidified where a group of men in Japan started to realize that like, yaoi [, gay male porn for women, was] taking a lot of space in the fandom, the conventions, so as an answer they decided to really popularize this representation of young girls. So this woman-exclusionary, queer-exclusionary story is very much part of it. So, for me, that’s kind of the history of it that I see. And then, because those representations match perfectly within the patriarchal society, they became the norm. But, at the origin, they were made in a sexual setting, just as a way to make women feel uncomfortable, which they succeeded at, you know. And loli has a complicated history now because, yes, it’s very present in pornography, but also this idea of the young, moey, cute, kawaii girl is also just very popular now. And of course, you also have lesbian and queer creators in Japan who are also making those kind of manga. And I don’t want to make them totally disappear from the conversation.

Jeremy: That’s part of the problem, isn’t it, with a zero tolerance approach, is that you can’t really draw those lines. And so, one of the main risks, I guess, of a zero tolerance approach is that you mentioned in your paper is that violators can actually be reported to authorities, right, over this sort of content. So, what consequences can platform users suffer if they’re reported to authorities over animated content?

Aurélie: Well, you know, the language that platforms use in their policies reveal a lot about the kind of moderation they want to perform for a reader or for online users. So, to say zero tolerance, it’s very stern legal language. When we’re showing we have zero tolerance over it, we are in compliance with the authorities, we even collaborate with them. It means that they can provide the authorities with all of the data they have about the user. And the problem comes when it’s animation or a cartoon, because they’re going to use the same language and they’re going to group animation with child sexual abuse materials.

Brandy: So I find it crazy to think that you could have someone posting a picture of a real child of being abused. And then on the other hand, you could have someone posting a racy picture of, like, Summer from Rick and Morty, and they could get treated exactly the same-that just sounds insane to me.

Aurélie: Yeah, I was talking with a content creator who used Patreon. So Patreon is actually interesting for adult content creators because their policies leave a lot of room for animation and pornography. You know, while they explicitly state “we do not want live action pornography,” they do allow animation, cartoons, and illustrations. But I was talking to this creator who had had his entire account deleted in one day without being able to negotiate, because suddenly they decided he was doing bestiality or it was incest or something, and he was like, “I was not doing anything that I had not done before.” But because they’re using those big terms, it’s very dangerous when it says bestiality, child sexual abuse materials. All of these are like paraphilic sexualities that we have a fair reason to believe should be banned and illegal [in real life]. But for animation, you know, he was doing 3D monster porn.

Jeremy: Yeah, I’ve heard that Patreon treats monster porn as bestiality. And I believe that actually comes not from Patreon themselves, but from their payment processor, which enforces that rule. And of course, that throws up a whole lot of questions about furry fandom and their right to express themselves in visual form.

Brandy: Yeah, I feel like we’re entering another spectrum of ambiguity. We’ve got the age spectrum of ambiguity. And then we’ve got the sentience spectrum of ambiguity, where you’ve got like animals on one side, human on the other, and then in the middle, you’ve got like these anthropomorphized animals and bestial aliens and things. Where do you draw the line there if you’re talking about bestiality?

Aurélie: Yeah, and that’s a problem for a platform. So that’s why I end up not even advocating for better tools, because I don’t think the answer is there, you know. The answer is not to have tools that are going to be more efficient at flagging, because it’s a cultural problem.

Jeremy: Yeah, there is something else that you refer to in your paper about more reliance being placed on AI for moderation. It’s not going to solve the problems, right? It’s just going to push them to a different level, where we have to assess are these tools going to do a better job at making the important differentiations between different sorts of content? Are they going to do a better job than humans?

Aurélie: Mm-hmm. Which even leave less room for negotiation. It’s exactly what happened with Twitch, you know, people who had hundreds of thousands of followers. You know, and it’s the same for the person I was talking to who was a Patreon content creator. That’s how he pays his rent. And he lost his platform in one day. And I feel like people are going to assume AI moderators are more objective when they’re really not.

Brandy: Like, I read an article the other day that an AI couldn’t even tell that like a stick bug wasn’t a stick where humans clearly could. I mean, it’s going to get better, but… I think we’ve brought up before that AI has its own biases. Was it in your article that you mentioned that it has a harder time recognizing like dark faces over pale faces?

If animation is always going to be wrongly flagged, then this myth of efficiency of those AI automated detection tools doesn’t exist anymore.

Aurélie: Yeah… But then if it doesn’t work, if animation is always going to be wrongly flagged, then this myth of efficiency of those AI automated detection tools doesn’t exist anymore.

Jeremy: So just to change the topic slightly, I want to ask you about what we should be doing to reduce the harms of content that maybe people don’t want to be exposed to other than reporting them to authorities, which is clearly disproportionate and harmful in itself. What else can we do about people being exposed to content that they don’t want to see?

Aurélie: Well, you know, there are some things that really surprised me when I started to think through this research. I went on Pornhub and I wrote “lolicon.” And then I got an automated pop-up that told me, “you know, “if you’re a pedophile and you need help, you can go on this website.” And I felt like I started to see the contour of the problem. Me, I was doing it as a researcher, but I was like, if I am a consumer and I see this, my first answer is going to be, “Pornhub doesn’t know what’s happening.” You know, “they don’t know what they’re talking about. They don’t understand that like, I’m just looking for cartoons. You know, I’m just looking for an image.” And it’s crazy. And like, I think I would feel very defensive. And there’s a very libertarian approach to, you know, animation and anime porn that has always existed in the anime fandom community, because for a long time, pornographic animation was unfairly targeted. And so people became very protective of it. And it created a lot of discourse that was dismissive of concern over age and gender that we can legitimately have. Again, it’s not saying, “let’s ban all animated pornography” if we just say, “it’s kind of crazy that all of those girls look so young.” It’s a conversation that can be nuanced and exist. So for me, it would be, first of all, for those platforms to start using those terms used by fans, like shotacon, lolicon, and actually understand and define them. Being able to explain in their policies where they’re using it, and why this content is actually banned. You know, instead of just saying, “we ban all child pornography (real, virtual, drawing, cartoons),” it’s to say, “we ban all like child sexual abuse materials. Within those child sexual abuse materials, we include animation. Here’s why.” I think it would be a beginning of an answer.

Jeremy: I think you’ve acknowledged that there is a spectrum, right? There is definitely material that is perhaps AI-generated, trained on real humans, and we don’t want to see it. And then there’s also really creative, artistic content that does reference characters who may be of indeterminate age. And the creators of that content may themselves be queer. They may have legitimate artistic points to make, and yet it’s all bundled together under the same category. Now, my fear is that if we do that, then people who have legitimate content are going to be forced to post that in dark spaces, in encrypted channels, maybe on the Tor network, places like that, where we also find real child sexual abuse material. So I feel like there has to be some middle ground where we can post content that fits within that part of the spectrum on the clear web, rather than pushing it into the dark web, which is only going to associate it more closely with real abuse content.

Aurélie: It’s actually interesting because today I translated another article I wrote for Porn Studies called “The Hentai Streaming Platform Wars,” which I can send to you if you want after. So it’s been that I was reading it again today, and it’s an article I published in December on the ecosystem of pornographic animation online, and all of those streaming platforms that exist and a lot of them are totally black boxes, you know, we don’t know where they come from, we don’t know who is running them, we don’t know how exactly they make money, but they have a lot of content. And a lot of the content is on the margin of legality and probably illegal in a lot of countries. So I think first we would need to have a pornographic platform that is dedicated to animation, because then this kind of platform would allow for much more nuance, because they would only be talking about animation. They would not have to even think about this question of, “but what do you do if it’s a real child?”, and I think it would solve a lot of problems.

Brandy: As someone who consumes a lot of fantasy content, it muddies the water even more. Like, I was watching an anime the other day and one of the characters looks like a child and is often mistaken for a child by a lot of the other characters, but they’re actually basically a hobbit, they’re like a middle-aged man with like a family of three. So it just made me start wondering, how would you classify the hentai if someone decided to make hentai of this character? He’s got the maturity and the chronological age of like a 35-year-old, but he looks about 11.

Jeremy: So how much of this is about media criticism and like media education, rather than making it about censorship as it is at present?

Aurélie: Zahra Stardust, she’s an Australian activist and tech writer and academic, I remember she stated this and she said it in her recent book Indy Porn that we insist so much on telling people that porn is not real, especially teenagers. But maybe we should start to tell people actually porn is real, you know, it is an industry. And it’s the same with those representations, like, you know, instead of saying like, oh, but it’s not real, is to say, actually, it is real, like someone is making it. It’s a question I used to tell my students in a class on porn, because I was teaching animation and we had a week on pornography. And I was like, are those characters sex workers? And of course they were all saying, no, they’re not. And I was like, okay, so where are the workers here? You know, where are the people? What do we think is behind this short animation? Someone made it. There was maybe a voice actor involved. To kind of make the industry appear, I think that’s part of education. And, once you start to think of the industry, you start to think also about the political economy of it, and the cultural economy. And like, what are the politics of the creators? What kind of content do you want to consume? And what kind of content do we want to distribute?

Brandy: Are you saying that the thing we should use to judge a piece of content is the character of the creator, who made it?

Aurélie: More to understand the context of the production. Because if you are born to understand that like, the Bart Simpson [porn], it’s a parody, it’s a commentary on how those comics are super family-friendly, and they’re actually not family-friendly. But they’re not supposed to be pornographic. It gives you some hints to understand them.

I think framing these as problematic rather than as illegal or even just not using terms that impute illegality when instead we should just be talking about them as being problematic is a lot more helpful.

Jeremy: Yeah, I think framing these as problematic rather than as illegal or even just not using terms that impute illegality when instead we should just be talking about them as being problematic is a lot more helpful. And really allows a lot more space for a conversation around these works rather than shutting down conversation as we otherwise would do by applying blanket categories of censorship to them.

Aurélie: Can I give you another example that is maybe more related to the question of age? In my thesis right now, I’m looking at this anime called Kite and it was distributed in the US in the 2000s. But the first time it was distributed, it was heavily edited, and some fans were mad because they were like, “oh, they took off all of the pornography part of it.” But among the pornography part of it, that the editors decided to take off during importation, there was a lot of scenes of rape against a child. It was animation, but you had a fan movement to bring back the unedited version. And again, it was a very libertarian attitude and super resistant to the idea of any kind of editing being done. But then you put it in perspective and you’re like, it was a very problematic media representation. Like, how do you think the younger woman in anime fandom felt to see fans being like, “it is actually super important for us to be able to see the rape of this child”? Obviously, a fictional child, but still. Sometimes it’s more important and about creative freedom of course, super important. But sometimes it’s like, let’s take a step back and actually wonder what are we fighting for. And, again, how does it make all the people feel when we do this?

Jeremy: So, you’ve mentioned your PhD research. Is there anything else in your research that you’re planning to publish articles about? Or is there any other research that is on the horizon for you?

Aurélie: I have another article that is about the metaphor that a lot of AI porn platforms use about, “oh, you can do whatever you want,” but actually there are policies and rules. A lot of those rules are good, but what does that mean when they’re pretending that there are no rules, and then how do you actually apply those rules to content that is not realistic? So I’m working on this and I’m hopefully finishing my thesis in the next couple of months.

Brandy: I feel like, yeah, AI is going to be a big fly in the ointment because we’ve got live action, and then we’ve got animated. But then AI is going to create this whole other subgenre of things that things that look live animated, but don’t actually involve people, which I think are going to have to have their own completely individual set of rules.

Jeremy: Yeah, I mean, one of the problems is that if there’s only two categories, which is real or fictional, then AI is always going to be the thin end of the wedge to regulate all fictional media. And so I think there is some merit in saying maybe there should be a third category of content in the way that we regulate it. And that is going to be incredibly contentious because as you may know, just just in the last few days, the US has put a moratorium on new state level regulation of AI. So we are kind of stuck in an unregulated state for a while.

Aurélie: If this is like the last part where I can give advice, it’s to look at how other communities who have always dealt with those questions have been dealing with them. Actually get interested in this history of the moderation of non-realistic content. Because animation has always existed. And animation has always been regulated. Look what worked, look what didn’t work. Talk to those people who are fans and who actually will be the consumers being impacted. And get curious. That’s why I’m defending also using terms like lolicon, which actually talk to this community, they understand what you mean.

To listen to the full, unedited version of this interview and read the episode notes with sources, search for “Beyond the Filter” in your favorite podcasting app.

Originally published at https://jere.my on May 29, 2025.

In 2020, Yulia Tsvetkov, a 26-year-old Russian theater director and artist, found herself under house arrest, facing up to six years in prison. Her crime? Sharing feminist and LGBTQ+-friendly artwork on social media, which authorities branded as “propaganda of non-traditional sexual relations among minors.” Just a year earlier, Michelle, a 53-year-old transgender woman, was sentenced to three years in a Russian men’s prison-where she faced potential violence and medical neglect-for posting erotic manga illustrations online. Prosecutors twisted her art into evidence of child abuse, ignoring her identity and the context of her work.

These aren’t isolated incidents. They’re stark reminders of how governments can wield vague laws to silence queer voices, turning self-expression into a criminal act. Russia’s crackdown on “non-traditional” content is a chilling blueprint-one that feels uncomfortably relevant as the United States veers toward conservative authoritarianism.

With cultural tides shifting, how confident can we be that LGBTQ+ individuals-and the platforms hosting their voices-won’t face similar criminalization here? History offers little comfort: from Joe Orton’s 1962 arrest in the UK for defacing library books to the censorship battles over Allen Ginsberg’s Howl in 1950s America, queer art has long been a target. Today, as book bans surge and political rhetoric sharpens, the question isn’t if this could happen-it’s when. More urgently, what can human rights activists and trust and safety professionals do to protect these freedoms before they’re lost to us all?

A swing against LGBTQ+ content online

Not long ago, Big Tech platforms waved rainbow flags and touted inclusivity. Now, hate speech targeting transgender people is openly allowed on platforms like X and Facebook, echoing the U.S.’s rollback of transgender rights. Are we nearing a day when hosting LGBTQ+ content becomes a crime in itself?

Though it sounded far-fetched a year ago, the pieces of a ban are falling into place. A slick rhetorical trick equates queer content-think inclusive sex ed, heartfelt memoirs, or a kids’ book about two penguins-with hardcore porn, obscenity, and even child sex abuse material (CSAM), all swept under the vague label of “content harmful to minors.” With this sleight of hand, laws that look reasonable at first glance-like age verification for adult sites or holding platforms accountable for harmful content-threaten to silence queer kids seeking community and trans adults sharing their truth online.

Below, I unpack three tactics that could underpin an LGBTQ+ content ban-two already in motion, one looming on the horizon. I’ll make the case that anyone who values internet freedom and LGBTQ+ rights must act now to counter these attacks before they lock us out of a free web. I’ll share how I’m fighting back, and I invite you to join me-whether by speaking out, coding solutions, or amplifying this fight.

Threat 1: Age verification laws

The first tactic is already rolling out: laws that cloak censorship as child protection, starting with age verification. The far-right Heritage Foundation’s Project 2025 agenda for Trump’s second term lays it bare:

Pornography should be outlawed. The people who produce and distribute it should be imprisoned. Educators and public librarians who purvey it should be classed as registered sex offenders. And telecommunications and technology firms that facilitate its spread should be shuttered.

This isn’t just about porn-it equates LGBTQ+ sex education, dismissed as “gender ideology,” with an existential threat to conservative values, blaming Big Tech for enabling both. Offline, this conflation first hit libraries. Across the U.S., lawmakers are proposing and passing bills to jail librarians for lending books deemed “harmful to minors”-a label slapped on LGBTQ+ works like the award-winning Gender Queer.

Now, this library crackdown is fueling a wave of state age verification laws online. These rules force platforms to verify users’ ages before granting access to anything vaguely “harmful to minors”-a net that could catch everything from memoirs to queer teen forums. Texas’s law is under Supreme Court review right now, and its fate could decide whether dozens of similar measures survive or collapse.

Threat 2: Section 230 repeal

If age verification falters, there’s another play in the works: unraveling the law that keeps platforms safe from lawsuits. Whatever the Supreme Court decides on age laws, Section 230 of the 1996 Communications Decency Act stands as a key shield for sites hosting user-generated LGBTQ+ content labeled “harmful to minors.” It’s the rule that protects platforms from liability for what users post while letting them moderate freely-basically, the internet’s free-speech backbone.

Gutting Section 230 could force platforms to preemptively scrub LGBTQ+ content-no age checks needed. Bipartisan bills like the EARN IT Act (discussed here and here) are pushing this, tying immunity to aggressive crackdowns on CSAM. The result? Over-censorship on steroids. After FOSTA-SESTA passed, platforms didn’t just target sex trafficking-they axed legal content overnight. Tumblr’s 2018 porn ban, meant to dodge liability, ended up nuking queer art and support forums too. This could wipe out trans creators’ posts or queer teen support groups in a heartbeat, all to avoid a lawsuit.

Here’s the twist: while the Trump-aligned Heritage Foundation pushes deregulation elsewhere, bipartisan calls to dismantle Section 230 keep growing. Some Democrats, oddly, lead the charge, ignoring its role as a free-speech lifeline. A second Trump term might not prioritize this-even Elon Musk, who’s tweaked X with 230 in mind, calls full repeal a “ disaster “ after leaning on it in court. Still, that’s cold comfort when queer voices hang in the balance, one policy shift from being silenced.

Threat 3: Quasi-official censorship

Beyond age laws and Section 230, a third threat looms-one that’s more speculative but already weaponizable. The Internet has a built-in takedown machine: the CSAM hash list run by the National Center for Missing and Exploited Children (NCMEC). This quasi-governmental group maintains a database of real child abuse images-hashed for platforms to scan and remove matches, often triggering criminal probes. It’s meant for a narrow purpose: stopping actual child pornography, as U.S. law defines it. So why isn’t it censoring queer content yet? Because it’s been tightly controlled-until now.

To date, NCMEC has assiduously limited its hash list to real child abuse images; a 2024 audit helped to ensure that this was so, backed by Supreme Court rulings tying the legal definition of CSAM to harm against actual kids. But pressure’s mounting from both within and outside NCMEC to stretch that definition. Generative AI art is the thin edge-uncontroversial to some, but opening the door to other artwork being included. In 2025, Texas police raided an art gallery over non-sexualized pieces misidentified as child porn, while lawmakers have slapped that same loaded term on Gender Queer. Redefining “CSAM” to include LGBTQ+ content isn’t a leap-it’s a step.

Look abroad, and the warning signs flash brighter. The Canadian Center for Child Protection has overreached its NCMEC-like blocklist, demanding takedowns of frames from a kids’ movie and ethnographic photos-once even reporting a teen for her blog art. Europe and the United Kingdom push AI to flag broad swaths of text and graphics, while Australia prosecutes over Simpsons parodies and erotic novels for women. These aren’t hypotheticals-they’re blueprints for abuse.

As a trust-and-safety professional, I’ve always said we should tackle distasteful art without diluting the horror of real CSAM-abuse of actual kids. But NCMEC’s already leaning right. Post-Trump’s reelection, it scrubbed transgender victims from its site. How much would it take to flip it into a censorship tool, hashing queer content as “illegal”? With lawmakers long smearing LGBTQ+ folks as predators, the groundwork’s there. This isn’t just crystal-ball gazing-it’s a threat we can’t ignore.

Fighting back: a new Center for Online Safety and Liberty

This is a collective effort. COSL acts as an incubator for independent projects tied to a core mission: empowering individuals and communities to thrive online by building safer spaces, fostering creativity, combating harm, and championing digital rights and freedom. We’re confronting threats like age verification, Section 230 rollbacks, encryption battles, and content-scanning overreach. We’re crafting free, open-source trust-and-safety tools-starting with my own Dead Dove and Modtools:Image, with more ambitious ones ahead. And we’re fostering safe, inclusive communities, beginning with fan spaces (yes, I’m letting my geek flag fly).

Our first highlighted project, Liberato, is a nonprofit hosting service where I’m Head of Trust and Safety, that sets out to serve marginalized communities who face the highest risks of censorship and surveillance. It scans content against NCMEC’s CSAM hash list and removes matches-no compromises there. But if anyone demands we axe artistic or LGBTQ+ content, we’ll log it in a transparency archive and push back hard.

Liberato is only the beginning. Each month, COSL will unveil new efforts-podcasts, fundraisers, petitions, software, social platforms-all driving our cause forward.

But COSL needs you to succeed. These threats strike deep-silencing queer voices isn’t a future risk, it’s happening. If that stirs you, join us; there are plenty of ways that you can:

However you contribute, COSL amplifies your impact. A freer, safer Internet starts with us-let’s build it now.

Originally published at https://jere.my on March 24, 2025.

Navigating legal challenges, whether business or personal, can feel overwhelming and time-consuming. It’s no wonder many people are now turning to generative AI chatbots such as ChatGPT to answer their legal questions, with greater convenience and far lower cost than traditional legal services.

But despite the advances made in generative AI over the past several years, relying on a chatbot’s advice on legal matters would be foolish. AI chatbots often return outdated, inaccurate, or even completely fictitious information. They’re also not great at identifying jurisdiction-specific nuances. Chatbots themselves will tell you that it’s a good idea to seek the advice of a real local lawyer before placing reliance on AI answers.

So today I’m launching a new legal advice website, AskLex.ai, that makes that easy. Like other chatbots, AskLex.ai’s chatbot Lex is trained on publicly available legal texts, court decisions, educational resources, and legal articles, with some modifications specifically designed to limit it from straying outside of its competence or using information from foreign jurisdictions.

But unlike other chatbots, AskLex.ai allows you to upgrade your chat to include a real lawyer, licensed to practice in your area. I have some experience with this. More than 20 years ago, I launched iLaw, Australia’s first full-service, fully on-line legal practice, which presented a unique offer: “You can ask a question on any topic related to Australian law, and for only $50 a qualified iLaw consultant in your locality will contact you with the answer.”

Through AskLex.ai, I’m bringing that same offer back today — even at the same price! — with a few AI improvements. This video shows how it works:

How accurate is Lex? Well, while developing AskLex.ai (and moonlighting on a legal advice line), I’ve spent about 400 hours testing it on topics as diverse as drafting a lease agreement, understanding your rights in a workplace dispute, licensing an invention, and negotiating a divorce. A few times, I was able to catch Lex out. But she caught me out a few times too. Overall, I would say that Lex and a real lawyer make a good team together.

One of the most exciting improvements that AskLex.ai offers over my previous online legal advice service iLaw, is that it also offers fixed price phone consultations (currently also just $50 during the alpha release phase), and simple legal document drafting (currently $100). Additionally, the site offers a $100 membership that provides clients with unlimited chat access both to Lex and to a real local lawyer.

AskLex.ai is currently in an alpha public release. During this phase, Lex’s advice is limited to U.S. and Australian law, since those are the jurisdictions that I am licensed in, and have been able to review and tune her accuracy. The availability of local legal advice from a real lawyer is currently to limited to New York State and Western Australia (the latter for pro bono cases only), however I plan to expand the service soon.

AI chatbots offer incredible convenience and efficiency, but they are not a substitute for the expertise of a licensed lawyer when it comes to serious legal matters. AskLex.ai bridges the gap by providing users with the best of both worlds: AI-driven insights and the option to upgrade to professional legal advice at an affordable price. Try AskLex.ai out yourself — I’d love to hear your comments!

In the first week of December 2024, controversial journalist Jesse Singal joined upstart social network Bluesky. Bluesky had been experiencing massive user growth since the election result had been called for Donald Trump in November, as many users of X (formerly Twitter) looked to jump ship for a vessel not helmed by Trump ally Elon Musk.

Due to Singal’s record of journalism critiquing the case for youth gender transition and aggressively clashing with his critics, his presence on Bluesky created immediate tension within Bluesky’s existing user base, which skewed transgender. Many trans users had adopted Bluesky early while Musk was already making them feel unwelcome on X, such as by labelling the term “cisgender” as hate speech.

Singal soon became Bluesky’s most blocked user — but many didn’t simply want him blocked, they wanted him gone, and were prepared to raise the stakes until they got this. One of the tactics employed was to throw additional mud, accusing Singal not only of misconduct in his reporting on trans issues, but also of being a pedophile.

On 13 December, Roderick’s team made a ruling. Ignoring the pedophilia slurs (which his more level-headed critics on Bluesky recognized as risable and false), and focusing on more credible allegations that Singal had infringed the privacy of Bluesky users, the team ultimately decided that despite the outcry, he had done nothing to violate the Bluesky terms of service and could stay. But was this the right decision?

The case against Singal

I am the father of a wonderful trans daughter, who began her medical transition while she was underage. The policies that Singal’s journalism advocates would have made it more difficult for her to begin her transition. While I cannot speak to the experiences of other trans children, I do know that any such delay to my daughter’s transition would not have been what she needed.

While I accept Singal’s right to conduct journalism that I disagree with, I ultimately supported those who called for Singal to be banned from Bluesky, on the ground that he had stepped over the line by violating a rule against promoting material from hate groups, when he posted a screenshot from the deplorable doxxing and cyberstalking community, Kiwifarms.

When Cloudflare banned Kiwifarms in 2022, it identified the website as an exceptional case in which censorship was justified due to its promotion of real-world violence. By allowing Singal to post screenshots from Kiwifarms, other Bluesky users are now being emboldened to do the same, which can only lead to more doxxing and harassment of trans people.

With that said, Singal and I also have a history. In 2021, I had spoken to Singal’s colleague Katie Herzog on background for an episode of their podcast, Blocked & Reported, in which the two journalists investigated one of my earliest trust and safety clients, MAP Support Club (MSC) — a support group for teenagers and adults who identified as experiencing sexual attraction towards younger children, which they had chosen never to act upon.

The sensitivity and the importance of this group was both immediately apparent to me when I first assumed its trust and safety role, and I took the role very seriously:

• I consulted child sexual abuse (CSA) prevention professionals about what safeguards would be needed, and raised funding from the Just Beginnings Collaborative to support those measures.
• I secured a partnership with survivor-led CSA prevention group Stop It Now, to ensure that its helpline operators were available in the group to provide regular guided group support sessions.
• I commissioned the development of software that would ensure that no illicit content was uploaded to the club’s chat forum, and collaborated with MSC’s administrators on the strengthening of the group’s safeguarding rules.
• I engaged an independent team from Nottingham-Trent University to conduct an evaluation of the group’s safety and effectiveness.

Singal and Herzog conducted their investigation of the group in the face of a fierce social media backlash against it and its fiscal sponsor from those wrongly convinced of a conspiracy theory that it was a front for a grooming operation.

But the journalists found otherwise, concluding that “They are genuinely trying to make life better, to reduce the likelihood of children being harmed. And they’re also trying to save the lives of people who have a… shitty lot in life.” Independent academic experts have since reached the same conclusion, and the group remains active today, in partnership with professional clinicians.

Beyond this, the only evidence offered against Singal to support the slurs against him were a series of articles that he wrote that his critics perceived as being too sympathetic to pedophiles, in that they recommended “treating people with pedophilic interests like human beings who can be reasoned and empathized with.” Such a sympathetic framing can be uncomfortable to read, but it does align with the approach taken by mainstream public health professionals.

The case against Rodericks

The movement to have Singal expelled from Bluesky didn’t stop with him, but extended to Bluesky Head of Trust & Safety, Aaron Rodericks, over his inaction on the matter and his perceived favouritism towards Singal. And in echoes of an attack previously levied against Twitter Head of Trust & Safety Yoel Roth by Elon Musk himself, this included extending false allegations that Rodericks too was a pedophile.

As far as I know, I have never met Rodericks personally. However although my Bluesky presence was then (and still is) pretty small, he and I had been mutual followers since about early 2024 due to our shared industry connection. At that time, another brouhaha was brewing on Bluesky over how it was enforcing its guidelines against child abuse. While Bluesky was banning users who promoted or excused abuse, many argued that it ought to also ban those who liked or shared suggestive-but-legal artwork of fictional characters resembling children or animals, and those who admitted to struggling with pedophilic impulses.

One user who argued that Bluesky had the balance right and shouldn’t be engaging in a broader crackdown was a queer artist named Terra Wilder, who wrote (from a now-deactivated account):

Ok, what alternative should they have as a space to be social because they’re still human beings. Your idea is entirely unreasonable, they’ll never stop trying to enter spaces. Or should we just kill them all. That’ll definitely stop them from emerging forever. … Nobody wants to handle this as a realistic problem it is just gut disgust and no solution.

This promoted a pile-on of abuse against her in which she herself was accused of pedophilia, with the ultimate outcome that she attempted to take her own life and was admitted to hospital. I expressed outrage of this in an exchange of my own directed to the prominent 35-thousand follower account that had been leading the pile-on. Referring to Terra and to another user Jamie who had been subjected to a similar pile-on, I wrote:

Jamie is right. There is no credible case to be made that either they or the person who was hospitalised were pro-abuse. You just don’t want queer people like them to have community. But they correctly call you on your bullshit: this isn’t about abuse. It’s about you taking offense and lashing out.

In retaliation for me making these comments, the user in question posted false smears against me based on an article written by Anna Slatz, a notorious transphobic far-right journalist and Kiwifarms user who had once published a Nazi manifesto. Her secondary source was a smear website from a disgruntled former volunteer colleague of mine at the Internet Corporation for Assigned Names and Numbers (ICANN), who was dismissed from that organisation after stalking and abusing several other colleagues.

Rodericks was then drawn into the dispute by association simply because he followed me. Rather than ignoring this bullying, or at least investigating the sources behind it as he should have done, Rodericks capitulated and unfollowed.

Predictably, this wasn’t enough to satisfy these extremists, who continued to push for a mass crackdown on accounts associated (in their minds) with pedophilia or zoophilia, which Bluesky eventually implemented in November. One message from a Bluesky moderator (or perhaps an AI) to a user banned over furry art provided a sweeping justification for art censorship:

Depicting sexual acts between humans and animals, even as art, is deeply problematic. Animals cannot consent, and such depictions promote the exploitation and abuse of animals. Art has a powerful influence and can normalize harmful behaviors, unintentionally endorsing or promoting these acts. Therefore, it’s crucial to avoid representing such.

But predictably again, this crackdown resulted in many accounts of innocent trans people being targeted, which Bluesky itself acknowledged, reversing many of the bans by late November. While attempting to placate one faction of users, Rodericks had outraged another. So it goes in this profession.

Six months after Terra Wilder’s hospitalisation, many remain convinced that Rodericks, myself, and Jesse Singal are all engaged in a joint conspiracy against them aimed at promoting pedophilia and undermining trans people. It would be laughable if only the real-world consequences of such misinformation and bullying campaigns weren’t so serious.

Lessons for Bluesky and other platforms

I have always insisted that platforms have a social as well as a legal responsibility to avoid hosting sexual abuse content or facilitating grooming. In other blog articles, I have outlined some of the practical approaches that they can implement for finding and reporting CSAM, and for making their platforms safe for younger users.

But at the same time, not everything that causes users to cry “pedophile” should be actioned. In fact, much of the time when this word is uttered, it is used purely for its rhetorical effect, by users who are themselves engaged in antisocial behaviours such as targeted harassment.

The users who engage in such pedojacketing abuse may honestly feel that they are justified in doing so. Jesse Singal has, undoubtedly, made it more difficult for transitioners, and his reporting on pedophilia, while scientifically accurate, may be legitimately triggering for abuse survivors, who may feel that he expresses greater sympathy for pedophiles than for them.

Associating professionals with stigmatised populations that they report on or work with is so common that there’s even a word for it; courtesy stigma. But that doesn’t make it OK. Too often, journalists and trust and safety professionals who are targeted are themselves marginalised. It is no coincidence that pile-on campaigns targeting these individuals attract participants from far-right groups such as Kiwifarms, even when they may have been initiated by progressives with good intentions.

As other platforms have also discovered, a small, very vocal faction of users who fling pedophilia smears can have an outsized negative influence over its moderation practices. Bluesky must learn to resist their influence. Its composable moderation architecture already provides effective mechanisms for users who are triggered by content that offends them. Bending further to those who use ugly false smears to get their way is not in the longer term interests of Bluesky as a healthy community.

Originally published at https://jere.my on December 16, 2024.

Australia is one of the few countries of its size, democratic or otherwise, that doesn’t protect human rights in its constitution. Although the High Court has judicially constructed a few constitutionally implied human rights, these are very narrow. In particular, the implied freedom of speech right recognised by the High Court only extends to political speech, leaving the vast majority of other speech acts unprotected.

Whether recognised in domestic law or not, Australia is still obliged to uphold the human rights recognised in the treaties to which it is a party, such as the International Covenant on Civil and Political Rights (ICCPR) and the Convention on the Rights of the Child (CRC). There is a Parliamentary Joint Committee on Human Rights (PJCHR) to review bills and legislative instruments for human rights compatibility based on Australia’s international obligations.

However the PJCHR does not have the power to block legislation, and the recommendations it does make are both politically influenced and time constrained. What this means is that if a person believes that their human rights are infringed by an Australian law and those rights aren’t reflected in domestic legislation, their practical options for obtaining recourse are very limited.

Efforts towards a Human Rights Act

In my recent submission on Modernising Australia’s National Classification Scheme, I referred to a rare case in which the United Nations Human Rights Committee (UNHRC) ruled directly in favour of a Australian whose human rights were infringed by a Tasmanian anti-homosexuality law, which led to the passage of the Human Rights (Sexual Conduct) Act of 1994 to ensure that those rights were reflected in Commonwealth law, so that they could be directly enforced.

This points in the direction of a possible compromise solution: if amending the constitution to add a Bill of Rights is improbable (the last attempt at enshrining new rights in the Constitution, the proposal to establish an indigenous voice in Parliament, failed spectacularly in 2023), why couldn’t the Parliament at least pass a Human Rights Act? While weaker than a constitutional instrument, this is, after all, the approach that New Zealand took with the passage of its New Zealand Bill of Rights Act 1990 (NZBORA).

Well, there have been proposals for Australia to do just this. Over 2008–2009, a National Human Rights Consultation recommended a federal Human Rights Act. But despite widespread public support, the proposal was never implemented due to lack of political will. Today, the passage of a federal Human Rights Act remains the subject of advocacy by various groups including Australian Lawyers for Human Rights and Amnesty Australia, but remains no closer to realisation.

The same is mostly true at a State level. In Western Australia, an independent consultation committee convened by the government in 2007 strongly recommended that it introduce a Human Rights Act. But no action was taken then, nor has been taken since. Yesterday, I attended a convening of the coalition Western Australia for a Human Rights Act who are still advocating for the adoption of a WA Human Rights Act, following the lead of Queensland, the Australian Capital Territory, and Victoria which have already adopted similar laws.

Exactly what this Act could do to help those who are subjected to government human rights violations will be considered towards the conclusion of this article. But to set some context, what evidence exists that such laws are needed in the first place? Are Australians actually more vulnerable to governmental overreach affecting their rights, compared with other countries that explicitly recognise human rights in domestic law? We don’t need to look any further than the current Federal Parliamentary session to find an answer to those questions.

New laws: misinformation versus freedom of expression

The absence of constitutional or even statutory protection for human rights has offered Australian governments the latitude to pass some of the world’s most invasive restrictions on Internet users, and to grant the most sweeping law enforcement powers.

The most effective lever for convincing the public that such authoritarian laws are justified is frequently by invoking child safety. Although experts recommend that child safety be treated as a public health issue, it is more expedient for politicians to “securitise” the issue, since this enables them to push through extreme measures of surveillance, censorship, and criminalisation, that the public otherwise wouldn’t accept.

This year alone, two federal initiatives fit that bill. The first was the Combatting Misinformation and Disinformation Bill 2024, which was promoted as being necessary to safeguard young people from harmful content, particularly misinformation related to health, safety, and social issues.

But the Australian Human Rights Commission, faith-based organisations, and opposition political parties were among those expressing apprehension about the government’s power to define and regulate “truth,” raising alarms about potential censorship and government overreach​. Following these widespread concerns, the Bill was ultimately abandoned in November 2024.

New laws: privacy versus age assurance

The second such measure is the Online Safety Amendment (Social Media Minimum Age) Bill, which is expected to pass this week, given its bipartisan support. Nevertheless this populist bill, which would require social media companies to enforce a minimum age limit of 16, has also been broadly criticised by experts for its impacts on the human rights of young people to engage in the online world.

During the scant 24 hours during which submissions were received in a rushed consultation, over 15,000 responses were received. In my own response, I wrote:

The bill undermines the crucial role of parents, who are better equipped than the government to assess the risks and benefits of their children’s participation in online society. By imposing a blanket ban, the bill incentivizes children and parents to circumvent the law, fostering disrespect for both legal authority and the institutions that enact such measures.

Moreover, the bill presents significant risks to privacy and freedom of expression. In their effort to comply, social media platforms may collect additional personal information, increasing the potential for misuse or data breaches. Requiring identification could also chill free speech, including political discourse, raising constitutional concerns and violating fundamental rights.

Many children are ready to engage with social media under parental supervision well before the age of 16. For marginalized groups or children with disabilities, social media offers essential support and connection. Denying access to these platforms could have profoundly negative effects, stunting their social development and leaving them ill-prepared to navigate adulthood when they eventually gain legal access.

Enforcement creep: free expression

It’s not only new laws that have the potential to encroach upon the human rights of Australian Internet users, but also the “enforcement creep” the occurs when regulators test the use of their powers under existing laws. The best example of that this year occurred during stand-off between Julie Inman-Grant, Australia’s eSafety Commissioner, and Elon Musk over demands for the global removal of a violent video from X (formerly Twitter) showing a stabbing attack on a Bishop during a livestreamed sermon in April 2024.

Inman-Grant issued a takedown order under Australia’s Online Safety Act, aiming to prevent the graphic content from being accessible to children. She declined to accept X’s compromise of blocking the video to Australian users only-arguing (in fairness, correctly), that a Virtual Private Network (VPN) could be used to bypass a national-level block. While other platforms complied, X refused, arguing the video did not violate its policies and raising concerns about global censorship implications. Inman-Grant took X to Australia’s Federal Court to enforce the removal but ultimately dropped the case in June 2024.

Elon Musk is a poor champion of freedom of expression, given his censorship of simple words like “cisgender” on X, and his penchant for using pedophilia smears against critics, competitors, and even his own former Head of Trust & Safety. Nonetheless, Musk was on the right side of this issue: nobody elected Julie Inman-Grant to any position at all, least of all to the position of global Internet censor.

Enforcement creep: encryption

A second example of Australian regulators foreshadowing their intention to exert their existing powers in ways that threaten human rights came in September 2024, when the head of the Australian Security Intelligence Organisation (ASIO) said that the organisation may start forcing technology companies to provide access to encrypted chats during certain security investigations.

The following month, the encrypted messenger company Session left Australia to reestablish itself in Switzerland, after police visited an employee’s residence and asked them questions about the app’s operation and sought details about a particular user.

The interception and decryption of online communications is not a new power of Australian regulators, but it has laid mostly latent since passage of the Assistance and Access Act in 2018. Then Prime Minister Malcolm Turnbull was rightly ridiculed the previous year for blustering that “The laws of mathematics are very commendable, but the only law that applies in Australia is the law of Australia.” But with executives of encrypted Internet platforms actually coming under police investigation, who is laughing now?

Violating children’s rights: privacy

It’s notable that Queensland is one of three states and territories, along with Victoria and the Australian Capital Territory, that have so far adopted human rights legislation designed to prevent the government from infringing human rights, because as I’ve previously described, it was also Queensland that hosted perhaps the most egregious violations of children’s rights ever committed in a law enforcement operation.

The operation, in which authorities directly uploaded abuse images of non-consenting minors for the consumption of online abusers, was subsequently declared a “clear violation of the UN children’s convention” by UNICEF, and also faced condemnation from Amnesty International. Forum shopping by cooperating law enforcement agencies had settled on Queensland as a base for the operation precisely because of it lacked the legal safeguards that would have made the operation unlawful elsewhere.

Violating children’s rights: real CSAM versus obscenity

Another area that infringes children’s rights, and one that I have devoted a lot of attention to on this blog due to its intersection with my professional work, is the way in which image-based abuse crimes against them are conflated with obscenity prosecutions over offensive art or literature. I have argued that treating real child abuse content as if it was nothing more than a risque anime cartoon is bad policy, and indeed a disgraceful manifestation of rape culture.

Furthermore, as I recently uncovered through FOIA applications to the Australian Federal Police (AFP) and the Commonwealth Director of Public Prosecutions (CDPP), neither agency even tracks the distinction between real abuse crimes, and fictional and fantasy sexual materials. Seeking to uncover more, I have made a further FOIA request to uncover the CDPP’s internal prosecution guidelines for such cases. The CDPP has refused to comply, and I have requested a review from the Office of the Australian Information Commissioner (OAIC), which remains pending at the date of this article.

Meanwhile, I am pursuing law reform through other avenues. In a letter on this topic to the Attorney-General, I recently wrote:

The current approach risks sending the message that federal criminal law is less about preventing harm to children and more about policing perceived deviant desires. This is a misstep, as it trivialises the horrific nature of real image-based abuse and conflates it with offenses better addressed through obscenity laws. It is vital to recognise that the exploitation of real children is a fundamentally different and far more serious issue than fictional or symbolic representations… A legislative adjustment to address this disparity would represent a significant step toward ensuring that the law’s primary focus remains on protecting real children from harm.

How far would a Human Rights Act go to help?

While what has been covered so far suffices to show that the human rights of Australians are under constant threat from overreaching and misguided government policies aimed at controlling our behaviour online, the biggest remaining open question is what, if anything, a Human Rights Act could do to fix that. In short, the answer is that while a Human Rights Act offers significant potential benefits, it is not a panacea.

One of its key limitations is that, unlike constitutional rights, statutory rights can be amended or overridden by subsequent legislation. This means that while the Act could provide robust protections on paper, it remains vulnerable to political shifts. For instance, a future government could weaken or dismantle its protections, especially without widespread public support. With that said, strong public support and alignment with international human rights norms can create a political cost for governments considering regressive changes.

An additional limitation is that courts under such an Act likely would not have the power to strike down incompatible laws. Instead, they could only issue declarations of incompatibility, leaving the final decision to Parliament. This structure maintains legislative supremacy but could limit the Act’s practical impact if political will is lacking. The experience of other countries shows that strong advocacy and public engagement are crucial for these frameworks to function effectively.

Finally, there is no immediate prospect of a federal Human Rights Act, and the benefits of a patchwork of State-based laws are limited. Yesterday’s seminar on a potential WA Human Rights Act included case studies of how Australians in States that do have a human rights law have benefited from access to a formalised process for asserting their human rights in response to government actions that violate them.

But there is little that can be done at a State level to protect the rights of those who are affected by Commonwealth legislation to regulate Internet usage nationwide. A federal Human Rights Act would not render State laws redundant; instead, it would create a baseline standard, allowing States to build on these protections with region-specific legislation. Both, in other words, are needed.

Potential benefits of implementing a Human Rights Act

Despite these inherent challenges, the experience of other countries demonstrates that the benefits of even a statutory Human Rights Act can be significant, especially when paired with active public engagement and advocacy. A Human Rights Act would provide a useful first step towards protecting civil liberties and promoting governmental accountability, by serving as a legal benchmark against which all new legislation is assessed, similar to frameworks in New Zealand and the United Kingdom.

Australians could also potentially directly invoke the Act in legal disputes, enhancing their ability to challenge governmental decisions. Currently, legal recourse depends largely on domestic laws aligning with international treaties, which often lack enforceability. Under a Human Rights Act, courts would be empowered to scrutinise government actions and legislation for human rights compliance. While it may not overturn laws, this process could compel Parliament to reconsider measures found incompatible with international human rights standards.

In practical terms, a Federal Human Rights Act could serve as a safeguard against overreach in areas like surveillance and online regulation. For example, debates around the Online Safety Act or proposed misinformation laws might have seen more balanced outcomes if guided by clear statutory human rights protections.

Conclusion

My own career of digital rights activism has taken me around the world, from the United Nations to Silicon Valley, but it began in Australia during the early 2000s, going head to head against the government on behalf of organisations such as the Internet Society of Australia, the WA Internet Association, and Electronic Frontiers Australia. This year that I’ve spent back in Australia has taken me full circle, bringing my advocacy back to its local roots.

That’s timely, because for a country of its size, Australia has increasingly become a world leader in bad Internet policy-from age assurance mandates, to demands for encryption backdoors, to global content takedowns, and criminalisation of speech. Australia’s lack of a constitutional, or even a statutory bill of rights plays a big part in it holding that dubious honour.

It should go without saying that the government should not be able to violate international human rights law, even when they claim child protection as the justification. Yet thanks to fearmongering and misinformation propagated from both right and left, the public has become all too credulous when government overreach is framed as being necessary to protect child safety. Human rights are one of the few safeguards that we have to stand firm against such encroachments.

Although I don’t harbour any illusions that a Human Rights Act would operate as a cure-all, any new weapon in the armoury of a civil rights defender is to be welcomed. I strongly support current Western Australian advocacy efforts to protect civil liberties and bring accountability and redress to victims of State government overreach and misfeasance.

But even more important in the context of Commonwealth laws affecting the Internet will be the passage of a Federal Human Rights Act, to extend human rights protections to all Australians on a uniform basis, and lay a stronger foundation for a more principled approach to Internet policymaking going forward.

Originally published at https://jere.my on November 28, 2024.

What should a social media platform do about content that is lawful, complies with the platform’s terms of service, but could be offensive to some of its users? Expecting users to deal with such content by proactively blocking it will burden and drive them away. On the other hand, burying such content leads to a milquetoast experience, especially for minorities who rely on these platforms to express their identities, share lived experiences, and challenge societal norms. Suppressing edgy content risks silencing these voices and reinforcing mainstream narratives that marginalise them.

A delicate balance is needed-one that empowers users to control their experience without erasing the diversity of thought and expression that makes social media vibrant and inclusive. The question, then, is how platforms can foster respectful engagement while maintaining space for controversial or challenging ideas. As someone passionate about free expression and safety online, the solution that I have always recommended has been the use of tags and content warnings to allow users to curate their own experience and avoid unwanted exposure to potentially offensive content online, without resorting to heavy-handed censorship.

Announcing Dead Dove

So today, I’m announcing my release of , a new content moderation plugin for WordPress that is designed to make this easy for the 43.5% of all websites globally that use this incredibly popular publishing platform. Simply by installing this plugin, administrators of WordPress-based sites that typically lack a dedicated engineering team, gain access to a flexible system that allows them to define a default set of tags that will gate potentially offensive content behind an informative warning-while also allowing users to override those defaults and establish their own content preferences.

It works by leveraging WordPress’s existing tags feature. To add a content warning, the administrator simply adds a tag to identify the type of content that requires a warning, including an extended description which will be shown in the warning box. Below is an example of the tags screen from one of my trust and safety clients that is developing a WordPress-based site. There is a separate preference screen (not shown here) where the administrator marks which of these tags should generate a warning by default.

(Oh, and if you’re wondering where “dead dove” phraseology comes from? It’s a term used in fandom circles to describe a content warning that can be given about potentially triggering content, without spoiling exactly what you’re about to encounter if you proceed. Check out this clip from Arrested Development to see where it came from.)

Once defined by the administrator, the tag can be applied to an entire post, a block such as a paragraph or image, or even just an excerpt of text such as a single sentence. When the end user views the content to which the tag has been applied, they will see it blurred out, and a dialogue box similar to the following will be displayed:

Customising the warnings

Note the “Modify your content warning settings” link at the bottom of the dialogue. This is provided so that the user can modify the default set of tags that generate content warnings, if they don’t agree with the choices made by the site administrator. Below is an example of the preferences screen that the user will see when they click that link.

Of course, the exact tags that the user sees will be those that their site administrator has defined. For the examples above, I’ve used the major warning tags from the long-running fandom website, Archive of Our Own, and added a general “Not Safe For Work” tag for 18+ adult content. Note that it’s possible both to deselect tags that are set by default, and to select tags that aren’t-which is especially useful for users to customise their own icks, triggers, and phobias.

The future

Of course, a content moderation system based around Dead Dove can only be effective if the site enforces users tagging their content appropriately. For now, although Dead Dove works with both the modern WordPress block editor (Gutenberg), as well as with the classic editor, tagging content does require access to the WordPress dashboard. Therefore, it’s currently most useful in an editorial environment, such as a private company or organisation website with a limited number of trained users.

But I’m also working on making it more useful for social websites that allow arbitrary numbers of end users to add their own blogs or images to the site. This will involve extending a future version of the plugin to support the simplified blog editor provided by BuddyBoss, a social community platform for WordPress.

Other enhancements planned for future versions include hiding content previews on category pages, which are currently not hidden from view, and allowing the smart application of content warnings based on the user’s physical location-specifically, there are categories of pornography, artwork, and even prose, that although legal in the United States, are illegal in some other countries. The plugin could provide a measure of additional safety to users from countries with such repressive laws.

A platform that intends to use Dead Dove at scale will need to educate its community about how to correctly tag their own content not only for content discovery, but also content warning. We have examples of how this can work in practice-such as Archive of Our Own, already mentioned above. But we’ve also seen what goes wrong when platforms unwisely entrust too much editorial power to untrained volunteers.

Enabling platforms to strike a better balance between user empowerment and safety is part of the service that I provide in my own trust and safety practice. For me, helping my clients develop communities that care about their own safety and freedom, is just as important as helping them develop or deploy software to protect their platforms.

Conclusion

Ultimately, the goal of Dead Dove is to empower both site administrators and users, fostering a safer, more inclusive online environment without compromising freedom of expression. By offering thoughtful content warnings and customisable preferences, this plugin ensures that platforms can accommodate diverse perspectives while respecting individual boundaries. As the Internet continues to evolve, tools like Dead Dove offer a practical way to strike the delicate balance between expression and sensitivity-without sacrificing either.

Try Dead Dove on your site today and let me know what you think! You can download it for free from the official WordPress plugins registry.

Originally published at https://jere.my on October 30, 2024.

Following five years working in trust and safety in the United States, this year I moved back to Australia. While here, I received a notification from Cloudflare’s excellent CSAM Scanning Tool (reviewed here) that a forum post uploaded to a website of one of my clients had been identified as suspected CSAM (child sexual abuse material). No problem right? Simply review the image, determine if it really is CSAM, and then complete the usual removal, archival, and reporting procedures.

Well, it turns out that that’s easier said than done…

Law Enforcement and Trust & Safety at Odds

It’s surprising how often police and trust and safety professionals are not on the same team. Some years ago a client of mine, an adult website, received a request from the police that we not ban certain accounts that we identified as apparently engaged in grooming or solicitation, because those were actually bait or honeypot accounts being run by the police. For the safety of our users, we refused the request and banned the abusive accounts.

If that wasn’t bad enough, in both the United States and Australia, individual trust and safety professionals have been targeted by police over possession offences. For example, in 2022 a school principal was arrested for maintaining possession of child abuse images shared from students for the purpose only of reporting and discipline — there was no allegation that the images were used or intended for any other purpose.

The problem of overzealous enforcement isn’t limited to trust and safety teams and professionals. As described in depth in my recent submission on the review of Australia’s content classification laws, other odd choices of target for prosecution have included a grandmother over innocent footage of her grandchild, and a man who shared a sexual Simpsons meme.

So when I receive a report of CSAM, I’m not only worried for the safety of whoever might be a victim of that material, I’m also worried about the safety of the person who reported it, and about my own safety. Simply put, I no longer trust Australian law enforcement to allow me to do my job without hindrance.

This article will outline the legal risks that trust and safety professionals face (mostly in Australia, though also comparing with U.S. law), the insufficient legal protections that they enjoy, and some of the cybersecurity precautions that they may be advised to take in order to protect themselves.

Insufficient Legal Protection

Nineteenth century legal philosopher John Austin maintained that laws do not necessarily have any moral basis, but rather simply express the will of the sovereign authority, backed by the threat of sanctions. There is no better illustration of this than the case of CSAM. It might be lawful (but is never morally right) for police to widely distribute child abuse images, while conversely it is morally right (but might always not be lawful) for trust and safety professionals to handle CSAM as part of their duties to remove and report it.

If an ambulance driver exceeds the legal speed limit, they could raise a legal defence of necessity if they were ever charged over doing so, because the law recognises that their otherwise illegal actions were justified. So too, there are some circumstances in which trust and safety professionals are able to raise a defence to a charge of CSAM possession.

The problem for the profession is that these circumstances are very narrow. In the United States, handling as few as three items of CSAM, even if the images in question were promptly deleted and/or reported to authorities, can land a trust and safety professional with a possession charge to which they can raise no defence. This limit is very few for professionals who may uncover large amounts of CSAM all at once, all from a single user of their platform.

Under Australian law there is no such two item amnesty, but there are defences for those engaged in enforcing, monitoring compliance with, or investigating a contravention of Australian or foreign law. Unless such a defence can be established, handling such content at all amounts to a strict liability crime, which depending on the circumstances could see charges brought under the Criminal Code, the Customs Act, and/or State law.

It’s also worth noting that this Australian defence isn’t available at all in response to charges brought under the Customs Act following a search at the border. In other words, trust and safety professionals who travel with work devices are liable to having these searched at the border without a warrant, and have no defence if sensitive content is found on that devices, perhaps even in cache or deleted space.

Another problem is that both the U.S. two item amnesty and the Australian Criminal Code defences are what are called affirmative defences. This means that you can still be charged with a crime and possibly imprisoned without bail, before having the opportunity to raise the defence. You are then effectively required to prove your own innocence at trial (possibly waiting years), or to cop a guilty plea.

While you might not think that this would be an issue for you, because you don’t store such content on your work device anyway, does your company have any moderation guides that include samples that might be illegal (hopefully not, and yet…)? Does your web browser cache contain images that passed through your platform’s moderation dashboard? How sure are you of your answer?

Also keep the destination’s law in mind. In Australia, illegal content includes much consensual 18+ pornography, artwork, fiction, and non-fiction that might be entirely permissible under your platform’s terms of service and the platform’s local law — posing an especially high risk for professionals who work for adult, LGBTQ+, or fan platforms and who might routinely deal with such content.

Threat Modelling

This being so, it remains incumbent upon trust and safety professionals to take care of their own safety by exercising sensible cybersecurity practices. This doesn’t mean that they should ever intentionally break the law — but it does mean that they should avoid ever being put in a situation where they risk being arrested by overzealous law enforcement authorities, and having to affirmatively prove their own innocence.

The starting point in threat modelling for cybersecurity involves asking four questions:

• Who are you? If your work includes receiving, triaging, investigating, or acting on reports of illicit platform content, then the channels through which you receive these reports are of interest to law enforcement. If you’re travelling with electronics, you’re also automatically placed under suspicion.
• Who is your adversary? While our ultimate adversaries are online abusers, as explained above it is unfortunately necessary to treat state, federal, and border law enforcement agencies as potential adversaries also.
• What do they want? Law enforcement’s priority is simply making arrests and convictions. To support these convictions, what is needed is evidence their target dealt with illicit material in some way, such as possession, importation, or sharing.
• How will they try to get it? There are three main ways:
• Reporting: Often charges begin with a report from a platform, and sometimes those reports can be false. For example, Google once reported a man to police over medical photos of his child that had been stored on Google's cloud, and it was TikTok who informed on the Australian grandmother.
• Surveillance: Under both U.S. and Australian law, telecommunications providers and online platforms have obligations to assist law enforcement, usually under warrant or similar order. These obligations are broader in Australia, where the communications regulator even possesses the power to compel providers to provide access to encrypted communications.
• Border search: Both U.S. and Australian border police can also search personal electronics at the border without a warrant. In the U.S., a suspect can be forced to use biometrics to unlock their device. In Australia, a suspect can be forced to divulge a PIN or password in some circumstances.

In short, law enforcement agencies may wrongly treat the channels through which trust and safety professionals receive reports as potential evidence of criminal activity, and may select them as targets for investigation. This can include engaging in upstream online surveillance, physical attacks, and legal coercion.

This doesn’t necessarily come with a polite request or warning. In 2021, a client of mine was reported to authorities by Google and its entire cloud account suspended without notice, simply because a single user had misused the platform for CSAM. I’ve had other clients whose sites have been taken down in similar circumstances, before anyone thought to talk with the client’s own trust and safety team.

In some cases, the authorities literally even come in guns blazing. In 2019 the owner of a website that published taboo sex stories was the subject of an over-the-top paramilitary style raid of his property that uncovered nothing (though the publisher was eventually sentenced to an astonishing 40 years imprisonment). Neighbours filmed explosions at the scene.

Law enforcement do not care about the difference between fantasy and reality, between art and abuse, between a family photo and a crime scene, to them it is all one and the same. Frankly, most members of the public are of pretty much the same view.

Today, supposedly progressive commentators, arm in arm with sex work abolitionists, baldly put forth the view that it is crazy and wrong to say that AI pornography shouldn’t be regulated by the same legal standards as CSAM. In this environment, law enforcement operates with significant impunity for overstepping, and few are willing to even talk about it.

Travelling with a Work Device

A comprehensive personal cybersecurity tutorial is beyond the scope of this article. For that, I recommend DigitalDefense.io. Instead, here I will focus on some measures that trust and safety professionals should consider in one particular situation in which they are the most vulnerable — when they are travelling. Many of these tips are also equally applicable to those working remotely from a home office.

To protect against physical attacks at the border, the simplest advice is simply never to travel with a device that you have used for accessing material that might be unlawful in any country that you are visiting or transiting through. There is no simple way for you to ensure that traces of that content do not remain on your device, possibly in forms that are invisible to you and that you cannot easily remove. If you can, keep separate devices for travel, and only use them to access known safe content while you are away.

If that isn’t possible and you do need to work while travelling, then there are a few next best options. The one that I would recommend is only to work from a temporary operating system such as Tails or Whonix, that won’t store anything to your device. Both options also automatically use the Tor network to avoid network based surveillance. The main difference between them is that Tails can run directly from a removable device such as a USB flash drive, DVD, or SD card, while Whonix requires virtualisation software such Virtualbox running on the host machine.

If your tolerance for risk is a little higher, you could travel and work with a Chromebook. This would require that before crossing a border, you perform a powerwash and then sign in again with a separate, second account that hasn’t been ever used for work, in case you are selected for a search. This isn’t quite as safe as using Tails or Whonix, as it is possible that advanced forensics techniques may still be able to recover traces of the previous account’s data from the device’s storage or from Google.

A Chromebook does not come with a built-in VPN, but it does support many third party VPN services and corporate VPNs. For power users, enabling Chrome OS’s support for Debian GNU/Linux makes a range of other security software, including the Tor browser, available.

Storage

I would not advise ever travelling with a device that has previously been used to access unsafe content that may have been cached or stored to a local filesystem. Even deleting such content that you are aware of is not guaranteed to render it irretrievable. Secure deletion utilities that were once relatively reliable are no longer effective when used with certain filesystems and storage technologies, including SSDs.

With that said, a situation could conceivably arise in which a trust and safety professional could be required to store CSAM while travelling, eg. under the REPORT Act mentioned below. In this case, the best options are to keep that content in an end-to-end encrypted file storage service or in a separate locally encrypted filesystem — never, it should go without saying, on Google Drive, OneDrive, or similar. Note that relying on your device’s full-disk encryption is not enough. Although important in case of your device being lost or stolen, full disk encryption is less useful if the device is seized at the border and you are forced to power up and unlock it.

Passwords

It is also important to store strong and unique passwords for each device and service in a password manager such as Bitwarden or 1Password, so that they won’t all be compromised if you are required to unlock a device. Consider also using unique usernames for sensitive online services such as VPNs, and storing these in the password manager too.

If you use a PIN as a quick unlock mechanism for your device or your password manager — generally a bad idea — be sure that it is not reused. Also ensure that your password manager does not auto-unlock on login, that it auto-locks when inactive, and (of course) that you commit the unlock password to memory rather than writing it down. Ideally, temporarily uninstall it from your device while travelling.

Do realise that if you use a phone or phone-based authenticator app as a second-factor authentication device for any online service and that phone is seized, you will lose your second factor — this is aside from the fact that SMS authentication is insecure anyway. A better option would be to discretely travel with a 2FA device such as a Yubikey.

Holding Law Enforcement Accountable

If you are disturbed that it has become necessary for trust and safety professionals to jump through such hoops simply to protect themselves from overzealous enforcement authorities, then you’re right to be concerned. In the longer term, law enforcement does need to be held accountable for the misuse of its powers. But this will be a long fight, that few have shown themselves willing to take on.

I have experienced first hand the manner in which law enforcement has colonised the establishment child protection sector, crowding out and seeking to discredit dissenting voices such as sex worker collectives and human rights advocates, or simply making them feel unwelcome by partnering with sex work abolitionist groups or surveillance tech vendors that threaten the rights and safety of minorities.

It’s fair to say that in the broader trust and safety field however the terrain remains contested. While law enforcement perspectives remain influential, their advocates share mindspace with those promoting public health based approaches, human rights assessments, and LGBTQ+ rights. While law enforcement-friendly viewpoints and vendors still dominate, sufficient noise is being made on the sidelines that the field as a whole has not yet been ceded to law enforcement, nor ever should it be.

In fact I’ve written before that it is incumbent upon trust and safety professionals who situate their work within a rights framework to push back against government proposals that flout human rights norms, rather than acquiescing to a dystopian future of blanket surveillance that imperils our communities and ourselves. This means opposing measures such as those that would require private communications to be vetted by AI robots, or that would further criminalise speech.

Not only should new authoritarian law enforcement powers be on our radar, but it’s just as important to continue applying scrutiny to the use of existing powers by agencies such as the eSafety Commissioner which has platformed hate groups, and Australian Border Force (ABF) which has a record of systematically misusing its coercive powers of search and arrest for corrupt and illegal purposes.

Fighting for Better Protections for the Profession

Finally, as a profession we ought to be actively advocating for new protections to be enacted for trust and safety professionals performing their duties in good faith. While working in the United States, I routinely dealt with lawful content that could nevertheless be judged obscene under Australia’s puritan censorship laws. Even doing that, outside the country, is technically an offence under Australian law.

The existing protections in both U.S. and Australian law are in urgent need of strengthening, so that trust and safety professionals acting reasonably and in good faith have immunity for dealings with content that they are required to access as a bona fide requirement of their profession.

To begin with, in Australia, the existing trust and safety defences under the Criminal Code should be extended to importation offences, and Australians should not be criminalised over web browsing that they lawfully do overseas. In the United States, there should be a higher threshold number of images before a defendant loses the possible defence that they were reporting, or had deleted, the images.

Unfortunately however lawmakers have shown little interest in supporting a safe enabling environment for trust and safety professionals. The Revising Existing Procedures On Reporting via Technology Act (or the REPORT Act), which passed into U.S. law in May 2024, does now authorise platforms to retain illicit content for longer after they have reported it to NCMEC — now for one year, rather than 90 days. But this change was made for the benefit of law enforcement, not the trust and safety profession.

Conclusion

Trust and safety professionals who receive reports of objectionable material already bear a heavy mental and emotional burden by being exposed to this content. They shouldn’t also have to worry about being arrested simply for doing their jobs. Yet these professionals frequently work at great personal risk, especially when they travel.

The current legal landscape in both Australia and the United States presents significant risks to professionals in this field that cannot be ignored. For as long as this remains the case, it will be crucial for such professionals to adopt stringent cybersecurity practices and to stay informed about the legal implications of their work.

Furthermore, there is a pressing need for legal reforms that recognise and protect the essential work of trust and safety professionals. This means pushing for amendments that offer comprehensive immunity for actions taken in good faith and expanding current defenses to cover all aspects of their duties. Only through such measures can we create a safer and more supportive environment for those on the front lines of online safety.