Hello! I'm not sure if this counts as an “advanced topic”, but nowhere else seemed fitting.

I have recently started to get into AI, and my research and experiments have had some surprising results. However, I don't want to keep these experiments all to myself; it'd be a lot more fun to see what the Scratch community could make with neural networks.
As I made a new project to try and create a script that could run an LSTM RNN similar to this program on Github, I suddenly realized that I had no clue on how to make one in something like Scratch.
I wondered, is it even possible to make a working machine learning algorithm in Scratch? Sure, it'd be a nightmare for even the best Scratchers, but is it possible? And if so, how would one accomplish this?

WrongfulRanger0 wrote:

Hello! I'm not sure if this counts as an “advanced topic”, but nowhere else seemed fitting.

I have recently started to get into AI, and my research and experiments have had some surprising results. However, I don't want to keep these experiments all to myself; it'd be a lot more fun to see what the Scratch community could make with neural networks.
As I made a new project to try and create a script that could run an LSTM RNN similar to this program on Github, I suddenly realized that I had no clue on how to make one in something like Scratch.
I wondered, is it even possible to make a working machine learning algorithm in Scratch? Sure, it'd be a nightmare for even the best Scratchers, but is it possible? And if so, how would one accomplish this?

I believe scratch is turing complete so it should be possible if it is possible in any other langs.

It's possible, it's terribly hard, and it's been done. That said, don't expect impressive performance, particularly on large nets :-)

bobbybee wrote:

It's possible, it's terribly hard, and it's been done. That said, don't expect impressive performance, particularly on large nets :-)

Wait, it's been done before already? Could you give me a link to it?

WrongfulRanger0 wrote:

bobbybee wrote:

It's possible, it's terribly hard, and it's been done. That said, don't expect impressive performance, particularly on large nets :-)

Wait, it's been done before already? Could you give me a link to it?

I thought so; don't have the link offhand.

bobbybee wrote:

WrongfulRanger0 wrote:

bobbybee wrote:

It's possible, it's terribly hard, and it's been done. That said, don't expect impressive performance, particularly on large nets :-)

Wait, it's been done before already? Could you give me a link to it?

I thought so; don't have the link offhand.

Just google “Scratch neural network” and you get a bunch

Firedrake969 wrote:

Just google “Scratch neural network” and you get a bunch

Oh, wow, there's some awesome stuff here… A number recognition program, a color-guessing NN, this is pretty cool! If only there was an easier way to implement this technology into a Scratch project…
Thanks!

Here's a nice volleyball game that uses neural network AI.

Wow… That game actually shows the network when you're training it, and by looking for patterns between that's happening on-screen and what the net's doing, I learned quite a bit about how to read neural network maps(or whatever they're called).
Of course, these games and experiments are cool and all, but there still doesn't appear to be any easy way to create a NN in Scratch. I'm not hoping for a “Neural Network” block section for Scratch 3.0, but something like a tutorial could help a lot. The person who made the volleyball game said they're going to make one soon, so that'll be cool!

WrongfulRanger0 wrote:

Wow… That game actually shows the network when you're training it, and by looking for patterns between that's happening on-screen and what the net's doing, I learned quite a bit about how to read neural network maps(or whatever they're called).
Of course, these games and experiments are cool and all, but there still doesn't appear to be any easy way to create a NN in Scratch. I'm not hoping for a “Neural Network” block section for Scratch 3.0, but something like a tutorial could help a lot. The person who made the volleyball game said they're going to make one soon, so that'll be cool!

It's not Scratch, but I like this - http://iamtrask.github.io/2015/07/12/basic-python-network/

WrongfulRanger0 wrote:

The person who made the volleyball game said they're going to make one soon, so that'll be cool!

Ooops. Yes, I said that, but I don't think that that is going to happen. Sorry.
However, If you have questions about NN or my implementations of NN in Scratch (here is another one I made), I will try to answer them. Also, here is a video tutorial, that shows you how to create a c++ NN (about 1h long).
Generally you can use NN for two different kind of tasks. Either you train them to do a task that has a “right” solution (like pattern recognition) or you train them to do a task that doesn't have a correct solution (or you don't know it yourself). AI belongs to the in the second group (I don't know myself, what move is best in a volleyball game). Debugging a NN AI is really really hard. I recommend you to start with a NN from the first group, and maybe later move on to AI.

Be careful you get the AI's consent before terminating it or you could get in trouble with the government.

birdoftheday wrote:

Be careful you get the AI's consent before terminating it or you could get in trouble with the government.

Are you being sarcastic?

MartinBraendli2 wrote:

WrongfulRanger0 wrote:

The person who made the volleyball game said they're going to make one soon, so that'll be cool!

Ooops. Yes, I said that, but I don't think that that is going to happen. Sorry.
However, If you have questions about NN or my implementations of NN in Scratch (here is another one I made), I will try to answer them. Also, here is a video tutorial, that shows you how to create a c++ NN (about 1h long).
Generally you can use NN for two different kind of tasks. Either you train them to do a task that has a “right” solution (like pattern recognition) or you train them to do a task that doesn't have a correct solution (or you don't know it yourself). AI belongs to the in the second group (I don't know myself, what move is best in a volleyball game). Debugging a NN AI is really really hard. I recommend you to start with a NN from the first group, and maybe later move on to AI.

Aww…
Also, I'm already kind of experimenting with the second kind, since I'm using Torch-RNN, so…

You should probably just stay away from creating artificial life forms, who knows what could happen if you give them too much room to think…Whatever, do what you want, just don't blame me when you're taking orders from your own twisted creation…

Well, from my experience, the only way AI could take over the world is if we taught it how.
And I don't have a corpus of various ways to conquer the planet to train my AI on, so I think I'm good.
Thank you for your concern, though.

The AI will learn how to take over the world.

AiyanMind wrote:

The AI will learn how to take over the world.

we're pretty far from that though

Firedrake969 wrote:

AiyanMind wrote:

The AI will learn how to take over the world.

we're pretty far from that though

Very true. For anyone who is genuinely concerned about AI taking over the world, just know that from my experience, the closest AI could do to conquering our planet is make a bunch of plans for a human to do so, and even then, someone would have to intentionally train an AI on tons of other evil plans for this to happen, and even THEN, it'd almost certainly make no sense at all, with either tons of spelling errors or repetition.

Now that we've gotten that out of the way, anything else someone would like to mention? Markov processes, back-propagation, links to cool programs you found on GitHub…? (I'm trying to make my own AI in Scratch based on Torch-RNN, so…)

http://iamtrask.github.io/2015/07/12/basic-python-network/
however, this is just a feedforward network, so look at https://iamtrask.github.io/2015/11/15/anyone-can-code-lstm/ once you read the first link

of course, remember that companies dealing in AI (google and amazon, for instance) are way beyond these relatively simple neural networks - these ones aren't really cutting edge any more

also learn calculus and the rest of the math that goes on behind the scenes so you actually understand what's going on…this is a good resource for that http://neuralnetworksanddeeplearning.com/index.html