Hi everyone!

I just recently started working on a robot that will scan scratch and collect comments. This means that I may create a website with the top commenters an api will probably be available at this url. (In test) Do you think this is a good idea? ^^

don't host it on replit or it can break

9gr how?

Chiroyce wrote:

9gr how?

not enough space

Raihan142857 wrote:

Chiroyce wrote:

9gr how?

not enough space

fun fact: i tried doing a similar thing and the json broke itself.

9gr wrote:

Raihan142857 wrote:

Chiroyce wrote:

9gr how?

not enough space

fun fact: i tried doing a similar thing and the json broke itself.

Yeah, nothing is truly free

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

potatophant wrote:

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

imagine how lefty does it with scratchdb

also, each comment seems to be around 300-350 bytes on average, which is over 6kb for 1 scratch-www page, and since you want to get usable data (all comments indexed very quickly) thats gonna be pretty hard to pull off.

metasloth wrote:

potatophant wrote:

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

imagine how lefty does it with scratchdb

I think he posted a screenshot somewhere… if I remember correctly the database is over 20 TB

Raihan142857 wrote:

metasloth wrote:

potatophant wrote:

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

imagine how lefty does it with scratchdb

I think he posted a screenshot somewhere… if I remember correctly the database is over 20 TB

if its 20tb this will take about roughly twice the size as there are more comments than posts

9gr wrote:

Raihan142857 wrote:

metasloth wrote:

potatophant wrote:

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

imagine how lefty does it with scratchdb

I think he posted a screenshot somewhere… if I remember correctly the database is over 20 TB

if its 20tb this will take about roughly twice the size as there are more comments than posts

and crawling through something like this would likely use up a lot of compute power. if your really up for it then sure, but if not then people can just use the scratch api or magnifier api.

Wait i just found this

potatophant wrote:

9gr wrote:

Raihan142857 wrote:

metasloth wrote:

potatophant wrote:

the api itself can be used to get scratch comments but this is interesting. mind you, scratch comments take up an enormous amount of space (have experience from working on magnifier), so if you wanna store anything reasonable id recommend at least a terabyte.

imagine how lefty does it with scratchdb

I think he posted a screenshot somewhere… if I remember correctly the database is over 20 TB

if its 20tb this will take about roughly twice the size as there are more comments than posts

and crawling through something like this would likely use up a lot of compute power. if your really up for it then sure, but if not then people can just use the scratch api or magnifier api.

plus, unlike the forums, there isn't an easy way to find new comments. the forums have a limited number of subsections, and you can really easily find what's new in a topic. with comments, there are millions of places to look for comments, and no centralized place to find them. you would also have to check each comment to see if it had been indexed yet.