sharing data across projects

GrandWizzardTheo

Data cloud is great… but how do I share data between two or more projects?

It happily updates data in each open instance of the same project.

Would be nice to be able to monitor data status in MyProjects and how many variables are named there.

thanks - Theodore

feel the database

jokebookservice1

To access Cloud Data of a project using external software, you would do a normal GET request to https://scratch.mit.edu/varserver/<PROJECT_ID>

To then transmit the data to another project, you may want to look into Dyln5797's Python Library which might be able to access variables too, so you would just get and set using their functions, at a interval f about 1 to 10 minutes, and that should be able to sync things up quite well for database-type projects

GrandWizzardTheo

Thanks jokebookservice1

That seems really slow considering the value seems to update almost immediately.

haven't got to learning Python yet. But understand how it could interrogate the value and adapt the other value.

interesting…

jokebookservice1

GrandWizzardTheo wrote:
Thanks jokebookservice1

That seems really slow considering the value seems to update almost immediately.

haven't got to learning Python yet. But understand how it could interrogate the value and adapt the other value.

interesting…

It could update faster at a cost of making your computer work a bit harder, every 5 seconds?

Also, you can use Node.js , somebody lse made one in Node. I believe setting things is something I asked round earlier, and you can make a POST request to /varserver to set variables, but it has to include some headers

Last edited by jokebookservice1 (June 14, 2016 22:49:02)

TheLogFather

jokebookservice1 wrote:
To then transmit the data to another project, you may want to look into Dyln5797's Python Library which might be able to access variables too, so you would just get and set using their functions, at a interval f about 1 to 10 minutes, and that should be able to sync things up quite well for database-type projects

You wouldn't want to send regular requests to the server like that.

Dylan's python scratchapi listens for updates to cloudvars that get sent to it, and you can check to see if there were any by using CloudSession.get_updates (or CloudSession.get_new_values, if you don't mind it only giving the latest change for a var, rather than all changes for the same var since you last checked).

That way you can make it respond almost instantaneously to any change to any cloudvar.

Last edited by TheLogFather (June 15, 2016 00:19:26)

GrandWizzardTheo

This is very interesting… my Dad is helping me with this…

I have invoked scratchapi in terminal which worked after the update pip-8.1.2

in python I logged in

import scratchapi
scratch = scratchapi.ScratchUserSession('Username', ‘password’)

scratch.tools.verify_session() # Should return True

returns True … so it is working!

now I know I have cloud data called

in my project.

what instruction could tell me what the current value of

is?

is it get_var?

thanks - Theodore

GrandWizzardTheo

We made a simple project to count cats

https://scratch.mit.edu/projects/113835404/

We opened it on two computers…

for about 10 mins it worked… when we reduced the number of cats on computer we could see the Cloud Data visible instantly changing on the other computer… GREAT!!!

But then it stopped working… and has never worked since!

Really need some kind of monitor to know the Cloud Data is working

Your thoughts…

Theodore

TheLogFather

GrandWizzardTheo wrote:
for about 10 mins it worked…
…
But then it stopped working… and has never worked since!

Yes, you will find that the port 531 connection gets dropped every so often.

See https://github.com/LLK/scratch-flash/issues/1061 and https://github.com/LLK/scratch-flash/issues/1001

This happens to projects too, but a project will continue to have cloud connection via the slower https polling.
See https://scratch.mit.edu/projects/96991584/ for a project which demonstrates this.

The way I deal with this in my custom cloud clients is to have a server ‘heartbeat’ cloudvar which it increments regularly (how often you do that will depend on circumstances). It then uses “get_vars” (which doesn't need a port 531 connection) to see if the change really made it through. If not then it recreates the CloudSession (as well as checking that the ScratchUserSession is still valid). If the UserSession is not valid, or it still doesn't update correctly, then it tries to sign in again.

Hope that helps!

Last edited by TheLogFather (June 15, 2016 11:44:12)

GrandWizzardTheo

Now I know more…

There is a button to monitor the Cloud Data! Please note that in the wiki help. This switches the browser to the Cloud Data monitor page. But it only updates if you refresh the page… it would be good if this page was set to auto refresh.

Cloud Data only works with Shared Projects. So even though we had made a test project with Cloud Data it was never ever sending anything! When we shared it suddenly started working

It is impossible to Share with yourself. We wanted to open the same Project on two or more machines at school but if we log in on each machine the Cloud Data only updates on one machine. The Cloud Data seems to be completely irreverent of the other open Project. The monitor page sadly confirms this.

What we need is a way to transfer the updating Cloud Data from one Shared Project to another Shared Project… how can this link be made? I am tempted to look at low tech sound triggers! or making a GPIO system back on Raspberry PI 1.4. But there must be a more elegant way.

Thanks - Theodore

lugga

import scratchapi
scratch = None
def run():
    var = scratch.cloud.get_var()
    scratch.cloud.set_var(var)
if __name__ == "__main__":
    scratch = scratchapi.ScratchUserSession('Username', 'password')
    x = scratch.tools.verify_session()
    if x:
        run()

I think this is what you want.

Last edited by lugga (June 16, 2016 08:22:11)

GrandWizzardTheo

Thanks lugga,

is “__main__” the name of the project? or the numerical name of the project? or URL?

I assume I run this in python do I have to invoke anything in terminal? Do I have to run this every time I attempt to perform this feat?

- Theodore

TheLogFather

GrandWizzardTheo wrote:
Cloud Data only works with Shared Projects. So even though we had made a test project with Cloud Data it was never ever sending anything! When we shared it suddenly started working

No, cloud should work just fine on an unshared project.

What might have happened was that the project became disconnected from cloud for some reason, and it required a reload to reconnect. When you share, that reloads the project.

My guess would be that you opened a second instance of the project in another browser tab, perhaps to then go on to look at the cloud log page…? When you open a new instance of a project, the previous one gets disconnected – you can only have one instance of a project connected to cloud from each account.

The above also means that if you want to test interaction between players in a cloud project, then you need to open it with more than one account, unfortunately (which means it has to be shared, unfortunately^2…)

Last edited by TheLogFather (June 16, 2016 13:00:01)

GRUPPOSPORTIVO

Thanks LogFather, now we understand why it has to be shared. Which is no problem except if everyone plays with our project we'll be getting data from all over the globe? which might distort our data!! I will tell Theodore to give our test voting system a name which won't attract millions of followers!

I guess we need to find a way to invite Scratch participants to vote on our system.

The problem I have is my account doesn't allow Cloud Data so I can't test my son's project is communicating. Hope it doesn't take to long to get upgraded.

Thanks - GruppoSportivo

TheLogFather

One trick I've used before to test multi-player cloud is to work on an old previously shared project that was not important, and was perhaps just shared as some kind of simple test, now abandoned.

When you re-share a project, followers don't get notified again, so they won't know that you've just re-shared it.

You can then use another Scratcher account (which you probably don't have at the moment) to load the project while it's shared, and then you can go back and unshare it. You now have two different accounts connected to the project, and so can test interaction between them, but the project doesn't show up anywhere for anyone else.

Last edited by TheLogFather (June 16, 2016 13:06:29)

GrandWizzardTheo

That's a good trick to know when we want to isolate and test the system - thanks - GrandWizzardTheo

lugga

GrandWizzardTheo wrote:
Thanks lugga,

is “__main__” the name of the project? or the numerical name of the project? or URL?

I assume I run this in python do I have to invoke anything in terminal? Do I have to run this every time I attempt to perform this feat?

- Theodore

Hi,
the

if __name__ == "__main__":

part is just so the python scratch knows that it is not being externally called and not to run the python code.
The only parts you should need to modify is the username, password and then fill it setvar and getvar.
Hope this helps

GrandWizzardTheo

Thanks lugga,

Does the project have to be running at the same time? How does it know what project is active?

TheLogFather

The python code shown above is missing important things (such as the project ID to use, and some kind of loop to keep checking and/or setting the cloudvars in the way you want), so it won't work as is.

I'd recommend you read through the README, and any documentation, for whichever of the two APIs you use. That should help give insight into making it work in a way that's more appropriate to your own requirements (as well as getting the parameters right, hopefully…)

Last edited by TheLogFather (June 16, 2016 16:29:14)

Miou4

import scratchapi
import time
#scratch.cloud.set_var('varname',setto,projectid)
while True:
scratch = scratchapi.ScratchUserSession('yourusername', ‘yourpassword’)
bridge1 = scratchapi.ScratchUserSession.cloud.get_var('cloudvar1name',project2id)
bridge2 = scratchapi.ScratchUserSession.cloud.get_var('cloudvar2name',project2id)
print(bridge1)
print(bridge2)
if not bridge1 == bridge2:
scratch.cloud.set_var('cloudvar1name', bridge2, project1id)
print ('Loop')
time.sleep(0.2)

not perfect, but working

Last edited by Miou4 (June 16, 2016 19:18:57)

GrandWizzardTheo

Thanks Miou4

I get Unsupported characters in input when I try to run your script.

Do I make a copy of my project project1id and run that on the other machine project2id

Then you are comparing the 2 cloud variables and copying the second one to match the first?

Are we following this right?

Discuss Scratch