Discuss Scratch

mathfreak231
Scratcher
1000+ posts

Scratch Wiki API Tutorial

This article is mostly complete, but it is highly subject to change.

Foreword

Making bots for the Scratch Wiki has become very popular recently, ever since I made the third official bot on the Scratch Wiki. Now, I will share “my” knowledge with you on making a bot for the Scratch Wiki.

What You Will Need

  • A relatively simple way to send HTTP requests in your programming language
  • A way to parse responses (either JSON, XML, serialized PHP, etc.)
  • An account on the Scratch Wiki, other than the one you already use, in the bot group (You can ignore this step if you are making a reference-only bot with relatively small requests.)
  • Basic knowledge of HTTP requests

General Info
Send your requests to the URL http://wiki.scratch.mit.edu/w/api.php. Read-only requests can be sent by any method. Logging in and write API must be a POST request.

Bots (and admins) have higher limits in APIs. They can generally have 10x more pages returned in a single query, and 10x more titles/etc. in a list of inputs.

If you don't have access to the higher limits, you can always grab the “querycontinue” and pass it for successive queries.

Wikis are incredibly case-sensitive, so watch out.

Any instance of <a word like this> or <AWORDLIKETHIS> should be replaced with whatever it calls for, which you can usually figure this out. For example, <USERNAME> should be replaced by your username.

A list of titles, properties, etc. is separated by |. For example, to get info about the pages “Scratch Wiki Home” and “Scratch”, use titles=Scratch%20Wiki%20Home|Scratch.

Basic Parameters
format — In your script, “json”, “xml”, or “php” would be common. Suffix it with “fm” to get it in HTML pretty-print.
action — “login”, “query”, “edit”, “move”, etc.
maxlag — Will cause the request to not go through if the replication lag is more than this amount. Probably not necessary the Scratch Wiki because of its low population.

Logging in

Logging in is likely the most tedious step of any bot program on the Wiki. I'll take you through the process step-by-step:
  1. Pass the following parameters in a POST request: action=login&lgusername=<USERNAME>&lgpassword=<PASSWORD>
  2. The result should be “NeedToken”
  3. Send the same request, only adding a parameter lgtoken found in the response, and a header Cookie found in Set-Cookie of the response header
  4. The result should be “Success”.
  5. You will need to manually construct another cookie:
scratchwikiUserName=<USERNAME>; scratchwikiUserID=<USERID>; scratchwikiToken=<TOKEN>
All of those you should be able to find in the response. Congratulations! You have finally logged on to the Scratch Wiki!

Getting info about pages
Parameters you will always use:
action=query

titles=<The title(s) of the page(s)>
-OR-
pageids=<The ID(s) of the page(s)>
-OR-
revids=<The ID(s) of stored revisions>

prop=<The name(s) of the property module(s). Possible values listed below.>

You will also need more parameters (duh) specific to the property/ies you are getting.
The pages you request are ordered and indexed by their IDs. (It's a hassle to get them when you don't know your IDs.)
Here are all the properties you can grab of a page or pages:

categories (cl) — Lists the categories the page belongs to.
categoryinfo (ci) — Returns category-specific info (such as how many items it has).
duplicatefiles (df) — Returns duplicates found of a given file through hash values; same way as Special:FileDuplicateSearch
extlinks (el) — All external URLs on a page
images (im) — All images used on a given page
imageinfo (ii) — Returns image metadata (such as upload timestamps, file size and MIME type)
info (in) — Returns basic page info (like protection level, url, whether it is watched by the user and how the title is actually displayed) or a token for a write API
links (pl) — Returns all wikilinks from a given page
iwlinks (iw) — Returns all interwiki links from a given page
langlinks (ll) — Returns all language links from a given page (on the Scratch Wiki, there is only de at the dime of writing)
pageprops (pp) — Returns the page's title, ID and namespace
revisions (rv) — Gets information about revisions of pages (such as the content of the page, the user that edited, the ID of the revision and the timestamp of the revision). Is used in three different ways: 1) Gets information about the latest revisions of a set of pages. 2) Gets several revisions of a single page (rvstart, rvend, rvlimit). 3) Gets information about revisions using the revids parameter.
templates (tl) — Lists all pages transcluded on the given page (the {{}})


Listing pages and other things
Parameters you will always use:

action=query
list=<Name of list module. Possible values listed below.>

You will need more parameters (duh) specific to the list you are getting.
Here is a list of all lists (that's a catchy sentence, huh?):
allcategories (ac) — Lists all categories on the wiki, as well as some very basic properties.
allfileusages (af) — Lists all the links to files on the wiki, including nonexistent files.
allimages (ai) — Lists all images on the wiki, as well as the same properties as imageinfo
alllinks (al) — Lists all the internal links on the wiki, including nonexistent files.
allpages (ap) — Lists all pages on the wiki.
alltransclusions (at) — Lists all instances of transclusions on the wiki, including to nonexistent pages.
allusers (au) — Lists all registered users, as well as some properties such as their user rights, edit count and time of registration.
backlinks (bl) — Lists all pages that link to the page given in bltitle/blpageid
blocks (bk) — Lists all blocked users on the wiki, as well as some metadata about the block.
categorymembers (cm) — Lists all members of the category page given in cmtitle/cmpageid, as well as some data about each page as a category member.
deletedrevs (dr) — Lists deleted revisions (admins only ) as well as some data about them.
embeddedin (ei) — Lists pages that transcluse the page given in eititle/eipageid.
exturlusage (eu) — Lists pages that match the URL search term in euquery, searched the same way as Special:LinkSearch, as well as some very basic properties of these pages.
filearchive (fa) — Lists deleted files and some file metadata.
imageusage (iu) — Find all pages that include the image given in iutitle/iupageid.
iwbacklinks (iwbl) — Find all pages that have a given interwiki link. iwblprefix is the interwiki prefix, and iwbltitle is the title of the page on the other wiki.
langbacklinks (lbl) — Find all pages that have a given language interwiki link. lbllang is the language prefix (e.g. “de”) and lbltitle is the title of the page on the other wiki.
logevents (le) — Read the logs. Each log event listed has properties like the type of log, the target page, the preforming user, the time, etc.
pageswithprop (pwp) — Lists pages with the given page property in pwppropname (e.g. hiddencat, notoc, displaytitle, etc.)
pagepropnames (ppn) — Names all the page properties used on the wiki.
protectedtitles (pt) — Lists all nonexistant pages that are protected from creation (Special:ProtectedTitles) as well as some metadata about its protection.
querypage (qp) — Lists pages that are in a special page, given in qppage (e.g. BrokenRedirects, Shortpages, Lonelypages)
random (rn) — Gets a random page or a set of random pages.
recentchanges (rc) — Lists recent changes on the wiki, as well as some data about each revision.
search (sr) — Performs a search like Special:Search using the search term given in srsearch, and returns properties of each matched page.
usercontribs (uc) — Gets all edits by the user given in ucuser.
users (us) — Lists all users on the wiki, as well as some data on them.
watchlist (wl) — Gets recent changes to watchlisted pages.
watchlistraw (wr) — Gets all the pages on the logged-in user's watchlist.

Generators
This combines “prop”'n'“titles” and “list”. Any prop or list that returns a list of pages can also be used as a generator; to use a generator, simply pass the module name to “generator” instead. (Any parameters specific to it will then start with “g”).

An example will better help me illustrate this: Say you want the text of any pages in the category “Category:Roles”. Instead of sending a new request for each page in the list, you can get them all like this:

action=query
generator=categorymembers
gcmtitle=Category:Roles
gcmtype=page
prop=revisions
rvprop=content

See it in action.

Editing and creating pages
Now that we're through with the read API, we can get to the writing API. For page editing and other writing API functions, you or your bot will need to be a “bot” or “sysop”. The first step of editing is to obtain a token, like so:

action=query
titles=<The name of the page you would like to edit. You can also use pageids.>
prop=info
intoken=edit

Then send a POST request. The most common parameters are listed below:

action=edit
title=<Title of the page you are editing. You can also use pageid.>
section=<The number of the section, like how those edit section buttons work. You can also use 0 for the intro paragraph, or ‘new’ for a new section.>
sectiontitle=<The title for a new section.>
text=<New content of the page.>
token=<The token obtained in the info request.>
summary=<edit summary>
bot=<If set and the user is a bot, marks the edit as a bot edit and hides it from Recent Changes by default.>
minor=<If set, marks edit as minor.>
notminor=<If set, overrides “mark all my edits as minor by default” in Preferences.>
createonly=<If set, throws an error if the page already exists.>
nocreate=<If set, throws an error if the page doesn't exist.>
watchlist=<Decide whether to add the page to your watchlist. “watch” or “unwatch”, “preferences” to watch it if “Watch pages I edit by default” in Preferences is set, “nochange” to have the current status remain.>
prependtext=<Add this text to the beginning of the page.>
appendtext=<Add this text to the end of the page.>
redirect=<If set, redirects are automatically followed.>
undo=<Undoes the revision with this ID.>
undoafter=<Undoes all revisions from undo to this one.>

Helpful Links
Auto-generated MediaWiki API documentation page for the Scratch Wiki
Full MediaWiki API documentation

Questions, comments, suggestions? Reply!

Last edited by mathfreak231 (June 12, 2014 16:13:52)


3DS FC: 3239-6065-2543 (Ask me before adding) | AS: Alex~ | X: Alex (DWS)
Ask me on my profile about XYORAS breeding requests!

My browser / operating system: MacOS Macintosh X 10.11.3, Safari 9.0.3, Flash 20.0 (release 0)
djdolphin
Scratcher
1000+ posts

Scratch Wiki API Tutorial

Very comprehensive.

!
Blueinkproductions
Scratcher
1000+ posts

Scratch Wiki API Tutorial

Awwww… Not finished I must wait!

Generation 2: the first time you see this copy and paste it on top of your sig in the scratch forums and increase generation by 1. Social experiment.
____                  _____  _______              
| \ | | | | | |\ | | /
|___/ | | | |__ | | \ | |/
| \ | | | | | | \ | |\
| | | | | | | | \ | | \
|___/ L____ \___/ |_____ ___|___ | \| | \
PRODUCTIONS





















































Here's a hint: support = support.

<shameless self promotion>follow me! follow me! follow me! love my stuff! love my stuff! love my stuff! follow me! love my stuff! remix my stuff! follow me! follow me! </shameless self promotion>
mathfreak231
Scratcher
1000+ posts

Scratch Wiki API Tutorial

bump!

3DS FC: 3239-6065-2543 (Ask me before adding) | AS: Alex~ | X: Alex (DWS)
Ask me on my profile about XYORAS breeding requests!

My browser / operating system: MacOS Macintosh X 10.11.3, Safari 9.0.3, Flash 20.0 (release 0)

Powered by DjangoBB

Standard | Mobile