synonymous

I really wish we'd have any idea of what the number of actual CSEM is that has been found on AO3. There have been two incidents where PAC was emailed CSEM in late 2021 and the two email to parts of the volunteers in May 2022.

There is the image that Az describes they found as an embedded GIF to an underage story that was ambiguous (I assume because it clearly read as underage because of its context?). Was that removed and reported?

And this is not an attempt to minimize OTW as a clearly dysfunctional organization nor to suggest that working in PAC isn't emotionally and mentally draining. OTW clearly must find ways to reorganize and reform, and volunteers should clearly have the organizational framework of all the support that can make their difficult work easier.

But I do think that the size of the issue is something that would be useful to have in the conversation. We know that antis often put in abuse tickets for CSEM for stories, so how is the percentage for actual illegal material that needs be reported and removed in contrast to someone complaining about Harry/Draco?

azarias

I can't give you hard answers but I can maybe spitball?

CSAM, images of real children involved in sexual acts, was quite rare. Not counting the cyberattacks, the majority of cases I saw were either ambiguous or indirect, and were not a large number of cases overall. By ambiguous, I mean content like a real minor's head badly photoshopped into live action pornography, or gifs that were presented as illustrating a child being raped but where I could not determine whether or not the bodies involves were actually children, rather than petite adults with secondary sexual characteristics altered or obscured. By indirect, I mean instructions on finding CSAM elsewhere on the internet, or requests to share and trade CSAM, or to hold viewing parties, but without directly linking or embedding CSAM on AO3. I would also include users bragging about having raped children or expressing a wish to rape children IRL in this category. AI generated art was not the hot topic last year that it is this year, so I never had to deal with AI generated images that appeared to be CSAM. I have a horrible feeling that's something today's PAC does have to worry about.

CSEM, images of real children being used in ways that are sexual but don't memorialize real life sexual abuse, was much more common, though still infrequent compared to other report categories. US law is less clear on CSEM than on CSAM, and websites require more nuanced policies for identifying and dealing with it. I don't think AO3's policies were very good at all, nor were our tools adequate, nor did we have anyone with much relevant expertise on the matter. But this is a much greyer area and one in which reasonable people can disagree.

To be clear, I don't mean fiction. I want everyone to have the freedom to write all the nasty, raunchy, extreme fiction they want; I just won't read it. In every case, I mean real actions being done to real children. We certainly fielded far more complaints from antis complaining about fiction than we did reports of actual CSAM/CSEM. I'll go further and state that antis made it harder for us to deal with actionable reports, because it was difficult to quickly identify the real issues among the flood of merely offended users.

I would love for AO3 to make a good faith effort to quantify these problems.

...When I was there, I was often told not to do this - don't look for violations, we should just focus on the reports, etc. Is that a policy that changed?

I also remember, once, with plagiarism, that when we banned someone permanently via banhammer, you could chose to delete all the content of the account. But I think you said that wasn't possible. Did it change, or did you mean you couldn't do it because of a policy?

azarias

When I was there, I was often told not to do this - don't look for violations, we should just focus on the reports, etc. Is that a policy that changed?

Yes and no. In general, we weren't supposed to police the Archive, that is, we weren't to go looking for problems. There are real legal reasons for this that have to do with "safe harbor" provisions for websites that host user-generated content, which are dry and technical but the point is that "don't police the Archive" isn't something PAC or Legal made up. It's basic protocol for UGC sites.

That said, when a user is reported as violating the site's TOS, to my understanding there isn't a legal reason not to act on other violations you find while investigating the report. If a work gets reported for harassment in the tags, and when I look at the work I notice that there's also a link to the author's Patreon in the summary, I can ding them for both harassment and breaking the commerce rule. I don't have to pretend I don't see it.

Now, what about looking through the user's account to see if there's Patreon in all there works? That's more ambiguous. I don't recall us being told we couldn't do that, only that we don't have to. In practice, we mostly dealt with this by saying "Remove the Patreon link from [specific work] and from any other works or comments you've created. If you get reported again and we find more Patreon links, we'll give you a second strike and suspension, even if those links predate this message to you." We did it that way because going through all of someone's work to compile a list of what they need to change is a lot of work and it's easier to tell the user to do it themselves Or Else.

Now, what if I'm banning the user because they offered to distribute videos of toddlers being sexually abused? Obviously I'm going to remove what got reported. Am I going to cross my fingers and hope that they only did it once? Do I just pray they haven't embedded their Wickr contact info in all their other works so that other people can get in touch with them to share CSAM? Is that a good idea? I don't think it is.

I would have greatly preferred to remove or hide all of an account's contents once we caught that account distributing or trying to distribute CSAM. You're right that there's a tool in admin that can delete all works and comments for an account; the tool is intended for spam accounts, but works on anyone. I would have loved to just be able to use that tool. But that might have meant removing content that did not violate the TOS, so I wasn't allowed to do so. Therefore my choices were either to look through everything or to let potential CSAM sit. I decided I could sleep better at night if I looked.

To be clear, no one told me to do this. Everything in PAC is voluntary. I didn't have to take these tickets at all, and having taken them, I wasn't required to do more than the bare minimum. But, I mean ... that's bad, right? Having very good reason to believe an account is full of CSAM distribution info, having just banned the owner for that very thing, and deciding to let it sit because I don't want to look would be bad? At minimum, no one told me not to, and it seemed evident to me that I should. I thought a better solution would be to get rid of everything, but that was not allowed.