Some time back I started designing a way to handle communications between instances of a project, that would support one-to-one interprocess messages and no limit on the number of users. The intent was that you could have multiple instances of player vs player games such as chess, connect4, scrabble etc.

The mechanism I initially tried was to have no master controlling process. When one user tried to send a message to another, there was a complex protocol of claiming exclusive locks and timeout mechanisms to handle clashes where different pairs of users tried to do something at the same time.

It wasn't very reliable and I went back to the drawing room.

I've finally come up with a mechanism which I think is simpler and can be made reliable. The test is online right now at https://scratch.mit.edu/projects/104865199/ - I can't swear that this initial implementation is reliable but I'm confident that in the long term it will be because I think the protocol is sound.

The mechanism is from William Golding's “Lord of the Flies”… where all the children on the island sit in a circle. They have a large conch shell which is passed round the circle. Only the person holding the conch can talk, but they can talk to anyone in the circle, not just the person next to them, and the person they talk to can respond. The person holding the conch shell controls the conversation and then when finished talking passes the conch on to the next person.

This is somewhat like a software Cambridge Ring for the control part but with the optimisation of direct transmission of data packets.

I detect clean shutdowns using the Red Flag trick, and unclean shutdowns are detected when a person holding the conch does not pass it on within a defined timeout. Typically a user will hold the conch for less than .15s however while testing I've set the timeout minimum to 6 seconds so that we don't kick users off because their machine is laggy or their cloud interaction has fallen back to HTML mode.

To avoid the clashes that could occur if multiple users all detected a dead player simultaneously, tests for an inactive player are staged so that different users will check in turn, with a 1 second delay between each player's check. The player who detects a dead player takes the conch from their cold dead hands and becomes the new speaker, then the passing of the conch around the ring resumes again from there.

Please help me test the prototype. https://scratch.mit.edu/projects/104865199/ I want to initially make sure that the code reliably detects logins and logouts. After that is robust, I'll test the interprocess message mechanism (eg how to transmit plays from one user to another). I don't want to test both at once as that was what I did with the previous mechanism and it was very hard to debug.

The way to transmit data will be that you first add the data to an ‘outgoing message’ array, then you add the recipient to a parallel ‘outgoing recipient’ array, and whenever the player holds the conch, every message that is queued in that array will be transmitted to the corresponding recipients.

Incoming messages will similarly be placed in an ‘incoming message’ array and your application can process them and remove them at its convenience - everything is asynchronous and supports the standard Scratch control loop paradigm. Or you can simulate a synchronous transmission by doing a send and waiting for a receive, wrapped in a user-defined block.

thanks!

Graham

I reloaded, and after a few seconds this message starting popping up rapidly.
Forced logout: gtoal

helloandgoodbye9 wrote:

I reloaded, and after a few seconds this message starting popping up rapidly.
Forced logout: gtoal

Ah. should only have been printed once. I'm guessing you were the only user left online when it happened? I'll fix that (tomorrow - it's late…). Meanwhile pressing reload ought to clear it on yours.

thanks

G

It's looking really awesome so far! I'm sure a lot of people will have great uses for this

Yay, I clicked the stop button and I logged out!
Very cool.
But what do you mean ‘Red flag’? And what is Robotron?
EDIT: It's gtoal's second account

This sounds like a really great method for a project that doesn't require too near-real-time interaction (such as the examples you gave), or has only 2 or 3 users at once.

I'm curious to see what happens once it gets to 5 or 6, though… I have a feeling it'll take too long to ‘pass the conch’ around to everyone, so it'll be well over a second between interactions…? (i.e. too long for a near-real-time game like griff's recent bomberman.)

BTW, glad to see you made use of the encoder/decoder

@thisandagain has been logged out over 50 times already….

helloandgoodbye9 wrote:

I reloaded, and after a few seconds this message starting popping up rapidly.
Forced logout: gtoal

Tried it again, now its
Forced logout: thisandagain

BookOwl wrote:

@thisandagain has been logged out over 50 times already….

I think I know the cause of that. It looks like the current user was set to ‘thisandagain’ but thisandagain did not also appear in the logged in user list as expected. So when it tried to remove him from the userlist and failed, it didn't go on to remove him from the ‘active user’ variable as well. Should be fixable.

Overall y'all have triggered two bugs in similar areas and I'll work on fixing those today. Otherwise it looks to be as robust as I hoped. Certainly a lot better than the first attempt I made at a multi-user system. Thanks for the help.

Graham

Or you could create 20 cloud variables, each one used as a communication channel for a single user, and have a central server using Dylan's scratchapi managing all the clients. But that would be cheating.

comp09 wrote:

Or you could create 20 cloud variables, each one used as a communication channel for a single user, and have a central server using Dylan's scratchapi managing all the clients. But that would be cheating.

No need for a cloudvar for each user in that case! –All users can send their update info through a single cloudvar, since your external server can catch every change made by every user, even though they are all on the same cloudvar.

Your server can then collate the data it receives on that cloudvar, and frequently set a second cloudvar to contain data for all users which the project can then use.

If the project takes all user movement, etc., from that second cloudvar, then it allows you to keep all users synchronised, too, which is a nice feature.


Synchronised, unlimited (theoretically) multiplayer through only two cloudvars.

OK, so I do have a few other cloudvars to show some live info, such as how many users have registered, ever, and how many are currently connected, plus one that's not actually in the project itself which is used to detect fallback polling – but they're not needed for the actual multiplayer mechanism.


Been experimenting a bit with that myself for a couple of months now, and it works just fine (only tested up to three players, though, but I think it should be OK up until you saturate your server's ‘net connection and/or blow up the Scratch cloud server ;P).

I guess I could share the project at some point, but it’s pretty boring at the moment – could do with some sort of reasonably simple game (or, at least, a ‘goal’) behind it…

TheLogFather wrote:

some sort of reasonably simple game (or, at least, a 'gtoal') behind it…

ftfy

gtoal wrote:

BookOwl wrote:

@thisandagain has been logged out over 50 times already….

I think I know the cause of that. It looks like the current user was set to ‘thisandagain’ but thisandagain did not also appear in the logged in user list as expected. So when it tried to remove him from the userlist and failed, it didn't go on to remove him from the ‘active user’ variable as well. Should be fixable.

Overall y'all have triggered two bugs in similar areas and I'll work on fixing those today. Otherwise it looks to be as robust as I hoped. Certainly a lot better than the first attempt I made at a multi-user system. Thanks for the help.

Graham

Slowly getting there. One thing I've noticed is that some users take *much* longer to execute their code when they have control compared to others (and this is even before there's actual work to do). Mostly a user takes .13s to hand over the conch. I noticed that when I minimised my window I would sometimes take around 2s instead… then I noticed one user who took over 5s to execute. Whether that was due to his machine being overloaded or a networking issue or what I have no idea.

G

NickyNouse wrote:

TheLogFather wrote:

some sort of reasonably simple game (or, at least, a 'gtoal') behind it…

ftfy

Yeah, that did occur…

The main thing that's really annoying, at the moment, about using the kind of method I talk about above is that your external server loses cloud connection every so often (a dozen times a day, or so), as I reported here, and as demonstrated in this project.

The server can reconnect pretty quickly, of course (a few seconds), but it's still a bit irritating for others who are playing at the time.

Awesome system; so far it has worked for me
And a very clever concept behind it

Sigton

Red flag trick?

gtoal wrote:

One thing I've noticed is that some users take *much* longer to execute their code when they have control compared to others

Probably fallen back to polling.

That'll ruin any kind of multiplayer system that expects reasonably quick turnaround. And especially for a method like this that has to get around to all other players before getting back to you again (just one player on fallback will slow down everyone else).

If you add my fallback detection method into the project, you could throw up a big warning if it detects fallback when the project first starts (or even prevent use of the project, if appropriate). While the project is running, it can also throw up a warning if it detects that the project has randomly lost its fast connection and fallen back to polling (from which it can never recover, unfortunately, until reloading the project).

MegaApuTurkUltra wrote:

Red flag trick?

red stop sign. timer.

TheLogFather wrote:

gtoal wrote:

One thing I've noticed is that some users take *much* longer to execute their code when they have control compared to others

Probably fallen back to polling.

That'll ruin any kind of multiplayer system that expects reasonably quick turnaround. And especially for a method like this that has to get around to all other players before getting back to you again (just one player on fallback will slow down everyone else).

If you add my fallback detection method into the project, you could throw up a big warning if it detects fallback when the project first starts (or even prevent use of the project, if appropriate). While the project is running, it can also throw up a warning if it detects that the project has randomly lost its fast connection and fallen back to polling (from which it can never recover, unfortunately, until reloading the project).

What I may do is have a slow user remove themself from the pool and reinsert after a delay, with the delay increasing every time they reattach. That way if it's a temporary glitch they will get back on eventually and resume where they left off, but if they have an ongoing problem, they'll only reconnect at long intervals and will get kicked off again, so won't affect other players very much.

I'm not overly worried about a few occasional delays of seconds - this is specifically for turn-based games, probably ones where there's a lot of thinking between turns - I don't expect to support fast twitch games. Getting a reliable datagram service is my overriding goal. (Especially if apps can be written in a style that survives a reload…)

Graham

Another thing I've noticed that can cause trouble is that some browsers will slow down (or even almost stop) execution of plugins when the app is put into the background, or on switching to another tab. Similarly, the OS may automatically slow down some apps when not in front. That happens for many recent apps on OSX these days, and it's a setting that you can change (on an app-by-app basis) – it's called “App Nap”. I think Firefox has that behaviour enabled already, which means that anyone running Scratch on Firefox on Mac will cause trouble in a near-real-time multiplayer game if they switch to another app for more than a few seconds.

gtoal wrote:

What I may do is have a slow user remove themself from the pool and reinsert after a delay, with the delay increasing every time they reattach. That way if it's a temporary glitch they will get back on eventually and resume where they left off, but if they have an ongoing problem, they'll only reconnect at long intervals and will get kicked off again, so won't affect other players very much.

Sounds like a plan, yeah…

gtoal wrote:

I'm not overly worried about a few occasional delays of seconds - this is specifically for turn-based games, probably ones where there's a lot of thinking between turns - I don't expect to support fast twitch games. Getting a reliable datagram service is my overriding goal. (Especially if apps can be written in a style that survives a reload…)
Graham

OK, yes, this should be a good method, then.