Discuss Scratch

#1May 8, 2014 19:08:14
blob8108
Scratcher
1000+ posts

Forum Archive

I made a temporary mirror of the now-broken forum archives:

http://scratchforums.blob8108.net/forums/
All existing links work: given any link to an old forum page, simply replace “archive.scratch.mit.edu” with “scratchforums.blob8108.net”.

I don't know how long I'll keep it online. But for now, enjoy!


________________________

PS. If you like, you can make your computer use my mirror instead of the archives. You can do this by adding the following line to your `/etc/hosts` file:

188.226.155.121 archive.scratch.mit.edu

Then when you visit http://archive.scratch.mit.edu/, you'll really be visiting my mirror.

You'll have to remember to undo this when I turn off my server or the ST fix the official archive.

Last edited by blob8108 (May 8, 2014 19:09:33)

tosh · slowly becoming a grown-up adult and very confused about it
#2May 8, 2014 22:14:10
djdolphin
Scratcher
1000+ posts

Forum Archive

Gah, you beat me to this. I've been trying to upload 3 gigabytes of files for months. I have a copy of the archive running locally on my computer. Anyway, great job!

Last edited by djdolphin (May 8, 2014 22:16:44)

!
#3May 8, 2014 22:21:30
blob8108
Scratcher
1000+ posts

Forum Archive

djdolphin wrote:

Anyway, great job!
Thanks!
I've been trying to upload 3 gigabytes of files for months.
The hard part is getting things like post redirects to work properly. I'm serving most things as static files, with a bit of Python to do post redirects (so links like viewtopic.php?pid=… still work). The post -> topic mapping is looked up in a database. You really don't want to store each of the 1.4 million posts as a static file. (Since each file takes up at minimum the filesystem's block allocation size – which is usually 1024 bytes.)

The topic pages still take up most of the space (~4 GB). I could probably reduce it further by storing those in a database, too, but that gets messy.
tosh · slowly becoming a grown-up adult and very confused about it
#4May 9, 2014 01:04:15
jvvg
Scratcher
1000+ posts

Forum Archive

Wow, cool! Where are you hosting it?

Professional web developer and lead engineer on the Scratch Wiki
Maybe the Scratch Team isn't so bad — Why the April Fools' Day forum didn't work last year
#5May 9, 2014 01:33:26
blob8108
Scratcher
1000+ posts

Forum Archive

jvvg wrote:

Wow, cool! Where are you hosting it?
On a $5/mo server from http://www.digitalocean.com. They were giving away free $10 codes…
tosh · slowly becoming a grown-up adult and very confused about it
#6May 14, 2014 03:46:51
Swampert11
Scratcher
1000+ posts

Forum Archive

Thank you for posting this!
Do you know a good project that doesn't have a lot of attention? Suggest a project to be put in this studio that is unpopular.
___________________________________________________________________________________

Only a Scratcher <——– Please look at it!
___________________________________________________________________________________

;












#7May 15, 2014 20:05:47
davidkt
Scratcher
1000+ posts

Forum Archive

Or, you could just use the Wayback Machine…
Remember when I looked like this? I still do.


Float, my Scratch 2.0 mod | My (somewhat under-construction) blog
#8May 15, 2014 21:46:16
blob8108
Scratcher
1000+ posts

Forum Archive

davidkt wrote:

Or, you could just use the Wayback Machine…
Sadly, most of the forums aren't in their archive.
tosh · slowly becoming a grown-up adult and very confused about it
#9June 17, 2014 23:38:49
derpmeup
Scratcher
1000+ posts

Forum Archive

davidkt wrote:

Or, you could just use the Wayback Machine…
The Wayback Machine doesn't have every topic archived.
hey bro that's a pretty good meme xD!
#10Sept. 25, 2014 09:02:55
jji7skyline
Scratcher
1000+ posts

Forum Archive

Thanks so much for doing this, it's invaluable! I hope this stays up forever!
Check out the NEW PROBEHEAD ALBUM!
#11Sept. 25, 2014 09:22:35
blob8108
Scratcher
1000+ posts

Forum Archive

jji7skyline wrote:

Thanks so much for doing this, it's invaluable! I hope this stays up forever!
Heh Hosting is a little expensive for a penniless student… I need to sort out giving the ST a copy that works, but making a proper static one is a little tricky
tosh · slowly becoming a grown-up adult and very confused about it
#12Feb. 26, 2015 13:30:49
mighto360
Scratcher
10 posts

Forum Archive

yeah

#13Feb. 27, 2015 20:51:17
comp09
Scratcher
1000+ posts

Forum Archive

blob8108 wrote:

jji7skyline wrote:

Thanks so much for doing this, it's invaluable! I hope this stays up forever!
Heh Hosting is a little expensive for a penniless student… I need to sort out giving the ST a copy that works, but making a proper static one is a little tricky

Are you okay with me pointing my wget request machine-gun at your server to static-ize the website?

Visit the website of Andrew Sun!


#14Feb. 27, 2015 21:15:54
djdolphin
Scratcher
1000+ posts

Forum Archive

comp09 wrote:

blob8108 wrote:

jji7skyline wrote:

Thanks so much for doing this, it's invaluable! I hope this stays up forever!
Heh Hosting is a little expensive for a penniless student… I need to sort out giving the ST a copy that works, but making a proper static one is a little tricky

Are you okay with me pointing my wget request machine-gun at your server to static-ize the website?
Isn't it already static?
!
#15Feb. 27, 2015 21:49:16
comp09
Scratcher
1000+ posts

Forum Archive

djdolphin wrote:

comp09 wrote:

Are you okay with me pointing my wget request machine-gun at your server to static-ize the website?
Isn't it already static?


blob8108 wrote:

djdolphin wrote:

Anyway, great job!
Thanks!
I've been trying to upload 3 gigabytes of files for months.
The hard part is getting things like post redirects to work properly. I'm serving most things as static files, with a bit of Python to do post redirects (so links like viewtopic.php?pid=… still work). The post -> topic mapping is looked up in a database. You really don't want to store each of the 1.4 million posts as a static file. (Since each file takes up at minimum the filesystem's block allocation size – which is usually 1024 bytes.)

The topic pages still take up most of the space (~4 GB). I could probably reduce it further by storing those in a database, too, but that gets messy.

Static-izing the website would make it easier to host on GitHub Pages, though. The post id redirections could be done with a bit of JavaScript hackery.

As a side note, could blob8108 possibly make the database/files available for us? You could 7-zip everything up and upload it to GitHub releases…

Last edited by comp09 (Feb. 27, 2015 21:50:14)


Visit the website of Andrew Sun!


#16Feb. 28, 2015 00:51:39
blob8108
Scratcher
1000+ posts

Forum Archive

comp09 wrote:

The post id redirections could be done with a bit of JavaScript hackery.
Looked into that. Not feasible.
tosh · slowly becoming a grown-up adult and very confused about it
#17Feb. 28, 2015 01:25:30
nXIII
Scratcher
1000+ posts

Forum Archive

blob8108 wrote:

comp09 wrote:

The post id redirections could be done with a bit of JavaScript hackery.
Looked into that. Not feasible.
Couldn't you generate a global post ID → topic ID/page JSON file and then just have viewtopic.php redirect if it gets a post ID?

Last edited by nXIII (Feb. 28, 2015 01:26:12)

♡ nXIII · GitHub
#18Feb. 28, 2015 02:13:33
friendship2468
Scratcher
100+ posts

Forum Archive

Cool
I love HARRRYYYYY POTTTEEERRRR
I am LGBT, panromantic
I code on my iPad mostly
Melanie
#19Feb. 28, 2015 02:25:58
comp09
Scratcher
1000+ posts

Forum Archive

blob8108 wrote:

comp09 wrote:

The post id redirections could be done with a bit of JavaScript hackery.
Looked into that. Not feasible.

You could generate a couple thousand files, each with 1000 topic/page numbers in it. Each file should only be several kilobytes.

For example, to find where the post 1,556,664 is, a script could request the file 1556.txt, which would contain something like the following:
180110,2
162848,1
184942,5
108709,4
111153,7
181590,11
114507,9
117521,1
124810,6
114089,8
...
and so on. The script would go to line 664, which would have “117391,5”, which tells it to go to /forums/viewtopic.php?id=117391&p=5#p1556664.

Sounds feasible to me.

Last edited by comp09 (Feb. 28, 2015 02:26:59)


Visit the website of Andrew Sun!


#20Feb. 28, 2015 12:32:16
blob8108
Scratcher
1000+ posts

Forum Archive

I already thought of that.
tosh · slowly becoming a grown-up adult and very confused about it

Powered by DjangoBB