Ok before yall this suggestion should be rejected, well AI Moderation does have benefits, but yeah it does have downsides also. But let's take a look at the benefits and downsides of AI Moderation.
The Benefits of AI Moderation
1. People do not have to work as much with AI moderating Scratch
2. AI will be always active even during the night hours, so the ST doesn't have to.
3. It can detect anything that would break the Scratch rules and ban the users without ST having to do it and mistakenly ban someone for something they didn't do.
4. With AI the current human moderators could finally rest.
Though it's good and but like I said it does have its problems
5. Scratch wouldn't have to waste money on new moderators
The problems of AI Moderation
1. The current moderators would sadly lose their jobs due to AI taking their jobs
2. AI is bound to make mistakes, so some mistakes might slip in-between here and there
3. It is bound to be modified every new update
But those are the only benefits and problems I can think of, but there are many more benefits and problems but I think an AI Moderator would help Scratch a ton plus it could give the current ST members a break from moderating and could finally relax so I think it is a win-win or a win-lose depending on how you see it

AI is not advanced enough for this to be reliable and within the ST's budget.

The_Game_ wrote:

AI is not advanced enough for this to be reliable and within the ST's budget.

Yeah you do have a point but maybe whenever it becomes more advanced then it would

ExtraordinaryCow wrote:

The_Game_ wrote:

AI is not advanced enough for this to be reliable and within the ST's budget.

Yeah you do have a point but maybe whenever it become more advanced then it would

That is YEARS from now.
Edit: Hello to whoever is reading this in the far future by the way.

I honestly can't say anything else other than Scratch is nonprofit so they might struggle.
Also if Mods are replaced by a bot, how would Contact Us work?

Semi-Support.

EDIT: ninja'd

Semi-support. Discord Automod now has something similar, and it works wonders! Only problem is that an AI is outside of Scratch's budget.

Most people actually hate moderation due to being done by bots. And with it being nonprofit makes it even worse.
Count how many times people insulted Scratch team and accused them of using ai moderation. Too much to count.

No support, AI can have a LOT of false positives and ban users falsely. I think it's best to always rely on human moderators instead of robots to moderate the site.

haIfIife wrote:

No support, AI can have a LOT of false positives and ban users falsely. I think it's best to always rely on human moderators instead of robots to moderate the site.

Human moderators also false ban people (rip me). While LLMs will make mistakes, they will make less mistakes.

But still, it's too expensive to host or use an API for LLMS.

My brother in christ, if the scratch team could automate moderation, they would have done it ages ago.

Semi-Support

It's impossible for Scratch to add AI moderation. I agree with portal power, if they could, they would. Great idea though, maybe in a few decades.

Hi people from the future btw

Spentinium wrote:

Human moderators also false ban people (rip me). While LLMs will make mistakes, they will make less mistakes.

Wrong.





(only 696 digits???)












Even if it's in Romanian, you can tell that's not code.

Spentinium wrote:

Human moderators also false ban people (rip me). While LLMs will make mistakes, they will make less mistakes.

But still, it's too expensive to host or use an API for LLMS.

That is incredibly optimistic given that I would expect a human to understand context more than anything automated.

Crispydogs101 wrote:

Most people actually hate moderation due to being done by bots. And with it being nonprofit makes it even worse.
Count how many times people insulted Scratch team and accused them of using ai moderation. Too much to count.

Indeed — the only automated moderation that exists currently is getting muted from the filterbot and the automatic project-censoring system that only happens when a project is reported enough. There are many complaints about both of these — and they often stem into questions like “Are there even real humans moderating the website?” Even if automated moderation gets better, I'm still inclined to believe that many users would not trust it.

ExtraordinaryCow wrote:

1. People do not have to work as much with AI moderating Scratch

That's debatable — the AI would certainly need some updating based on Scratch website updates, and also based on Scratch rule changes. Perhaps this is less work, but there are some maintenance issues that an AI moderator would introduce that are not present with human moderators.

ExtraordinaryCow wrote:

2. AI will be always active even during the night hours, so the ST doesn't have to.

That's a pretty good benefit. But usually the only Scratchers active at night are those who are not in the United States, many of whom do not speak English, so the next question is whether or not such an AI moderator could handle foreign languages (thus adding to the aforementioned maintenance issues).

ExtraordinaryCow wrote:

3. It can detect anything that would break the Scratch rules and ban the users without ST having to do it and mistakenly ban someone for something they didn't do.

And who's to guarantee that an AI moderator would not mistakenly ban someone for something they didn't do?

ExtraordinaryCow wrote:

4. With AI the current human moderators could finally rest.

I mean … I don't think anyone would have become a Scratch moderator if they didn't want to. Any moderator can “finally relax” by leaving the Scratch Team. That's their choice.

ExtraordinaryCow wrote:

Though it's good and but like I said it does have its problems

So would AI moderation. There is no perfect solution.

ExtraordinaryCow wrote:

5. Scratch wouldn't have to waste money on new moderators

I wouldn't consider it “wasting” money to help maintain a positive community. Similarly to how you would not consider it “wasting” money to maintain an AI moderator.

ExtraordinaryCow wrote:

1. The current moderators would sadly lose their jobs due to AI taking their jobs

Why not have both of them work at once? That way the human moderators can also fix any issues that come up as a result of the AI moderator.

ExtraordinaryCow wrote:

2. AI is bound to make mistakes, so some mistakes might slip in-between here and there

Just like humans. Evidently an AI moderator is not a perfect solution, even if the Scratch Team has the funds for it.

ExtraordinaryCow wrote:

3. It is bound to be modified every new update

Yep.

ExtraordinaryCow wrote:

But those are the only benefits and problems I can think of,

Here's another problem: project moderation. Is AI moderation really advanced enough to where it could censor inappropriate projects? Remember that this is not just inappropriate images, but also gossip projects, cloud chat projects, “music-sharing” projects, … With an AI moderator, I would expect a lot of false positives when it comes to project moderation. Again, humans are much better at determining context and intent than I would expect an AI to be.

ExtraordinaryCow wrote:

but there are many more benefits and problems but I think an AI Moderator would help Scratch a ton plus it could give the current ST members a break from moderating and could finally relax so I think it is a win-win or a win-lose depending on how you see it

I see it as win-lose, if anything — the benefits that come with AI moderation also bring (not insignificant!) problems. For this reason, I don't think it would be a good idea to invest in AI moderation at this time.

Za-Chary wrote:

(#14)
And who's to guarantee that an AI moderator would not mistakenly ban someone for something they didn't do?

This is actually a great question.
How would it react if person-a reported person-b for something that never even happened?
Since AI is more limited than we are (currently, although at one point it may surpass us even).

What about replacing the fiterbot with a context-sensitive one? No bans, mutes or takedowns, but a LLM could lightly analyse your comment which is better for things like other languages.

Additionally, a stricter LLM could add the content to the report queue if it triggers it.

Spentinium wrote:

(#9)
Human moderators also false ban people (rip me). While LLMs will make mistakes, they will make less mistakes.

This is definitely not the case. I would be surprised if AIs managed to handle 30% of moderation actions correctly. And while ST does make mistakes, as they are human, you can just use contact us if you think they made a mistake.

-

Popping my head in here to briefly mention a limitation that has not been brought up yet: automation in general can cause website lag. We sometimes get this with some of the current automated moderation stuff in place (eg word filter, forum anti-SEO-spam bot); not saying all automation is bad; but it would be a good idea to think how much resources would be needed to support it.