Discuss Scratch

asbqq01
Scratcher
3 posts

Getting project data via HTTP request to 'internalapi' not working?

Hi,

The documentation under 'Scratch File Format' on the wiki says that I ought to be able to download the project.json file via a URL of the form

http://scratch.mit.edu/internalapi/project/id/get/

(see here).

I'm involved in running an after-school club using Scratch (going well — the students are very engaged with their projects) and it would be very handy to have a tool which downloaded the students' projects as backups, permanent records, etc., so this looks ideal.

However, trying this on a shared project I made just now:

http://scratch.mit.edu/internalapi/project/22177160/get/

gives a 404, whether I try in a browser or via ‘wget’ on the command-line.

Am I missing something, or is this functionality currently unavailable?

Thanks very much for any insight.
djdolphin
Scratcher
1000+ posts

Getting project data via HTTP request to 'internalapi' not working?

The url changed to http://projects.scratch.mit.edu/internalapi/project/{project}/get/
asbqq01
Scratcher
3 posts

Getting project data via HTTP request to 'internalapi' not working?

Thanks, djdolphin! That's working nicely. URLs which work for this example project are:

http://scratch.mit.edu/api/v1/project/22177160/?format=json (for the creator, date shared, etc.)
http://projects.scratch.mit.edu/internalapi/project/22177160/get/ (for the JSON of the project)
http://projects.scratch.mit.edu/internalapi/asset/83a9787d4cb6f3b7632b4ddfebf74367.wav/get/ (for one of its assets)

(I don't have a Wiki account so can't update the page, sorry.)
djdolphin
Scratcher
1000+ posts

Getting project data via HTTP request to 'internalapi' not working?

asbqq01 wrote:

Thanks, djdolphin! That's working nicely. URLs which work for this example project are:

http://scratch.mit.edu/api/v1/project/22177160/?format=json (for the creator, date shared, etc.)
http://projects.scratch.mit.edu/internalapi/project/22177160/get/ (for the JSON of the project)
http://projects.scratch.mit.edu/internalapi/asset/83a9787d4cb6f3b7632b4ddfebf74367.wav/get/ (for one of its assets)

(I don't have a Wiki account so can't update the page, sorry.)
I fixed the wiki page.
asbqq01
Scratcher
3 posts

Getting project data via HTTP request to 'internalapi' not working?

Looking again at this, I'm sometimes getting ‘404’ errors for assets. Am I missing something? E.g.,

wget -q -O- http://projects.scratch.mit.edu/internalapi/project/18690738/get/ | grep baseLayerMD5
“baseLayerMD5”: “c4c9443cef113d2f89d646f18c914cb1.png”,
“baseLayerMD5”: “70441496b8c4d5b53b983d1c5d8dd9de.png”,
“baseLayerMD5”: “26fecef75cf3b6e0d98bff5c06475036.svg”,
“baseLayerMD5”: “fbdaf4d1d349edd3ddf3a1c4528aa9ec.svg”,
“baseLayerMD5”: “5976c10412306fc093c1d1930fa05119.svg”,
“baseLayerMD5”: “cdda000264e93b95c5f638fa02f1fb0e.png”,
“baseLayerMD5”: “c0e05836e0efe0ba1fbbbdee1f9c5aae.png”,
“baseLayerMD5”: “d656f30927c1420fe68e1beef6f08cbd.png”,
“baseLayerMD5”: “f495f716613a4bd6968057b29550cc85.png”,
“baseLayerMD5”: “4e8ed954e80e7dc863cc71821aef36b6.png”,
“baseLayerMD5”: “978784738c1d9dd4b1d397fd18bdf406.svg”,
“baseLayerMD5”: “d3c90ee86dabeab614d8b62aaa14a5d8.png”,
“baseLayerMD5”: “b6ac3c9e1789cba2302d2ef82d62d019.svg”,

and some image assets work, e.g.:

wget -O/dev/null http://projects.scratch.mit.edu/internalapi/asset/c4c9443cef113d2f89d646f18c914cb1.png/get/

HTTP request sent, awaiting response… 200 OK
Length: 3775 (3.7K)


but some don't:

wget -O/dev/null http://projects.scratch.mit.edu/internalapi/asset/70441496b8c4d5b53b983d1c5d8dd9de.png/get/

HTTP request sent, awaiting response… 404 Not Found

Any suggestions gratefully received! Not a huge problem if this feature is not fully supported yet, but it would be handy to be able to programmatically download the students' projects. Many thanks.

mobluse
Scratcher
100+ posts

Getting project data via HTTP request to 'internalapi' not working?

Here is a Bash command-line that sometimes works for me in Linux in order to download the project as an sb2-file:
proj=12612306; mkdir sb2tmp; cd sb2tmp; curl -L http://projects.scratch.mit.edu/internalapi/project/$proj/get/ 2> /dev/null | tee project.json | grep -i MD5 | cut -d' ' -f2 | sed 's/[\",]//g' | while read file; do curl -L -o $file http://scratch.mit.edu/internalapi/asset/$file/get 2> /dev/null; done; i=1; while [ $i -le `expr $( ls -1 | wc -l ) - 2` ]; do sed -i "{s/ID\": -1/ID\": $i/;t end;b;:end;n;b end}" project.json; i=`expr $i + 1`; done; i=0; ( grep penLayerMD5 project.json; grep '\(baseLayerMD5\|md5\)' project.json ) | cut -d' ' -f2 | sed 's/[",]//g' | while read file; do ext=`echo $file | cut -d. -f2`; mv $file $i.$ext; i=`expr $i + 1`; done; zip $proj.sb2 *; mv *.sb2 ..; cd ..; rm -r sb2tmp

Last edited by mobluse (Nov. 16, 2014 10:39:12)

procd
Scratcher
100+ posts

Getting project data via HTTP request to 'internalapi' not working?

There's an outstanding, but which is still to be fixed, bug where some of the png assets are given an svg extension instead which you may be hitting.

Powered by DjangoBB