AI-Powered Search

Context
The current plans for the CLA will probably if it is done like that and working out mean, that projects will be pretty much be crawled and indexed, including their content. This could in my opinion allow for way more advanced searching of projects.
February 03, 2026 (this might be wrong in the future, idk)

Suggestion:
I suggest that we use the fact that we already indexed all these things to expand our project searching capacities.
This would mean that it might be possible to search for projects that contain for example a music file, a costume, a sprite, a background et cetera.
Such abilities would be useful for Scratch moderation to remove bad projects faster by being able to detect them by specific content.
This suggestion proposes to extend it to possible to search projects and their content by description (aka find me a platformer game about birds) or find me projects that calculate pi, and same for scripts sprites et cetera. For these tools, it isn't possible to rely entirely on indexing, but this part might rely on AI.

How the searching works:
It should work like this that you describe the AI what you are looking for and then the AI has the ability to search existent projects for that and come up with projects, sprites, music, costumes et cetera.
The AI is mainly supposed to generate the search results. It is supposed to be able to crawl the indexed projects and pick out fitting projects out of your prompt (an actual full text prompt in natural language that describes the will of the user can be used) or fitting project content (sprites, music, costumes etc).
It is not supposed to generate any project content by itself.
The AI should bring out the results and be able to reason them in text and the user can give feedback and tell the AI to adjust it's search until fitting results come up.

You are still able to content search like described here (without AI just looking for what projects contain this sprite, sound, backdrop etc and use the old search. This is just an option you may use as well.

For the discussion, also read this (this is the inspiration and source of the idea and good point to discuss on):

Za-Chary wrote:

AI searching is a little different from Google searching, right? Sometimes I've seen AI used to try to find more information about a mathematical concept, for instance, and the AI will give correct explanations, but when providing the source sometimes the AI will give a link to something that doesn't exist. Is this something about AI that could be reasonably fixed? And are such concerns relevant for your suggestion?

If you would like to just be able to search projects by project content without AI, this is the original suggestion.

Is the regular search not enough? Why must we throw AI into everything?

support
this would be really useful for finding particular projects. I think I will be the only one tho because everyone iscomplete anti ai ,and I'm only anti ai to image//video generators

BitcoinFarmer wrote:

AI-Powered Search

Hmm. Like semantic search or literally having an LLM respond? for the former, Scratch's search service Elasticsearch already implements it

cambegonefr wrote:

Is the regular search not enough? Why must we throw AI into everything?

Can we please have more substantive critiques other than ai bad (but to answer anecdotally for the past 9 years almost, no, absolutely not)

ajskateboarder wrote:

BitcoinFarmer wrote:

AI-Powered Search

Hmm. Like semantic search or literally having an LLM respond? for the former, Scratch's search service Elasticsearch already implements it

cambegonefr wrote:

Is the regular search not enough? Why must we throw AI into everything?

Can we please have more substantive critiques other than ai bad (but to answer anecdotally for the past 9 years almost, no, absolutely not)

How I imagined this is that you can describe the AI what you are looking for and then the AI has the ability to search existent projects for that and come up with projects, sprites, music, costumes et cetera.
The AI is mainly supposed to generate the search results. It is supposed to be able to crawl the indexed projects and pick out fitting projects out of your prompt (an actual full text prompt in natural language that describes the will of the user can be used) or fitting project content (sprites, music, costumes etc).
The AI should bring out the results and be able to reason them in text and the user can give feedback and tell the AI to adjust it's search until fitting results come up.

ajskateboarder wrote:

cambegonefr wrote:

Is the regular search not enough? Why must we throw AI into everything?

Can we please have more substantive critiques other than ai bad (but to answer anecdotally for the past 9 years almost, no, absolutely not)

That's not me saying “AI is bad”, I'm saying “Why must we slap AI onto every single feature, corner of the internet, etc.”

Copying my response from a similar thread; it was implied that the questions would be more relevant here:

Za-Chary wrote:

AI searching is a little different from Google searching, right? Sometimes I've seen AI used to try to find more information about a mathematical concept, for instance, and the AI will give correct explanations, but when providing the source sometimes the AI will give a link to something that doesn't exist. Is this something about AI that could be reasonably fixed? And are such concerns relevant for your suggestion?

Za-Chary wrote:

Copying my response from a similar thread; it was implied that the questions would be more relevant here:

Za-Chary wrote:

AI searching is a little different from Google searching, right? Sometimes I've seen AI used to try to find more information about a mathematical concept, for instance, and the AI will give correct explanations, but when providing the source sometimes the AI will give a link to something that doesn't exist. Is this something about AI that could be reasonably fixed? And are such concerns relevant for your suggestion?

Thank you very much.

cambegonefr wrote:

ajskateboarder wrote:

cambegonefr wrote:

Is the regular search not enough? Why must we throw AI into everything?

Can we please have more substantive critiques other than ai bad (but to answer anecdotally for the past 9 years almost, no, absolutely not)

That's not me saying “AI is bad”, I'm saying “Why must we slap AI onto every single feature, corner of the internet, etc.”

The vibe of the original message kind of expressed that.

I see your criticism on implementing AI into unneccessary corners of the internet, especially in content generation (like people generating AI slop videos and youtube recommending them without anyone even wanting to see it when they realize it is just AI content.

However, I would say that the implementation of AI can be good when it brings a significant benefit.
And I see the potential for this benefit here. This would allow us for way more advanced searching than we currently have, including project contents, which can very much more precise and in depth and allowing you to hit way better results fronm what you want. Also, such a system simply needs AI to work. So, in this case the tool actually can bring large benefits instead of being slapped onto the service with no one really wanting it.

bump

BitcoinFarmer wrote:

cambegonefr wrote:

ajskateboarder wrote:

cambegonefr wrote:

Is the regular search not enough? Why must we throw AI into everything?

Can we please have more substantive critiques other than ai bad (but to answer anecdotally for the past 9 years almost, no, absolutely not)

That's not me saying “AI is bad”, I'm saying “Why must we slap AI onto every single feature, corner of the internet, etc.”

The vibe of the original message kind of expressed that.

I see your criticism on implementing AI into unneccessary corners of the internet, especially in content generation (like people generating AI slop videos and youtube recommending them without anyone even wanting to see it when they realize it is just AI content.

However, I would say that the implementation of AI can be good when it brings a significant benefit.
And I see the potential for this benefit here. This would allow us for way more advanced searching than we currently have, including project contents, which can very much more precise and in depth and allowing you to hit way better results fronm what you want. Also, such a system simply needs AI to work. So, in this case the tool actually can bring large benefits instead of being slapped onto the service with no one really wanting it.

I understand your point and don't disagree with this suggestion, but I feel like it doesn't need to be AI-driven. It can be as simple hitting CTRL+F in your browser to search for a specific word, but just apply that to Scratch projects.

cambegonefr wrote:

I understand your point and don't disagree with this suggestion, but I feel like it doesn't need to be AI-driven. It can be as simple hitting CTRL+F in your browser to search for a specific word, but just apply that to Scratch projects.

My original suggestion did not contain AI, responses gave me the idea to add this and then I created this one that extends the old one with AI.
Find the original one here, it is the same idea but no AI.

Support, the current search tool is absolutely terrible, I have to search the exact name of the project

So I actually have some engineering experience in this, and I can tell you this will not be cheap. Analyzing images and audio files at the scale of the amount of projects Scratch has is going to cost a ton of money. Additionally, being able to actually make sense of the contents of projects and the fact that projects are interactive and not just static content is going to take a massive amount of computing power for what I think is not a huge benefit. Even just using embeddings and similar tools to improve the accuracy of search results based on descriptions and titles is going to cost a lot of money.

jvvg wrote:

So I actually have some engineering experience in this, and I can tell you this will not be cheap. Analyzing images and audio files at the scale of the amount of projects Scratch has is going to cost a ton of money. Additionally, being able to actually make sense of the contents of projects and the fact that projects are interactive and not just static content is going to take a massive amount of computing power for what I think is not a huge benefit. Even just using embeddings and similar tools to improve the accuracy of search results based on descriptions and titles is going to cost a lot of money.

I do agree that this is not a cheap addition. However, Scratch already will add those analyzing features and project crawling and content reading and making sense of it features in a thing called CLA.
The search deserves the same benefits in my opinion.
TAnd those benefits can be massive, in my opinion. This will allow for actually and with a really sharp precision finding projects and parts of projects that you are looking for and you really wanted to find. The current search operations are rather… imprecise and primitive but with this we would have a highly advanced and useful search that allows you to have the results you were actually looking for and make the most use out of them.

This means, out of millions of projects with millions of assets and things you can find exactly what you need.
Anyways, if you would like to keep out the AI and understanding stuff and just focus on already given text, here is my old suggestion.

bump

No support. We don't need to add AI to everything and even then, it would be expensive for the Scratch Team to maintain, especially since they're a nonprofit organization.

IcosahedronsAreCool wrote:

No support. We don't need to add AI to everything and even then, it would be expensive for the Scratch Team to maintain, especially since they're a nonprofit organization.

I see your point, however as the Scratch Website already uses their captabilities on the CLA which will crawl projects as well, this is not that expensive to develop any more, because most of the work (indexing the projects and crawling them) is done anyway. So it is just an extension to what already comes and not that big of an expense any more, whilst bringing a really huge ammount of benefit.


If you seek a “lighter variant” of this suggestion that is simply based off the fact that the CLA does already mean you are able to crawl projects and that does not require AI training and therefore minimizes expenses, then look here.

I don’t Support,

We’re already getting one ai to help with projects which will be really expensive and will take a lot of time to be made.

And now you’re suggesting another one with an entirely different way of function which will increase the sever size needed, increase the money, increase the time.

And besides there are communities to help people find obscure scratch project, like the lost media.

bump