API: Overhaul token handling

The current token handling is a mess. This simplifies things greatly:
* *All* tokens are obtained from action=query&meta=tokens, rather than
  being spread over action=tokens, action=query&prop=info,
  action=query&prop=revisions, action=query&prop=recentchanges, and
  action=query&prop=users. All these old methods are deprecated.
* Similarly, there is only one hook to register new token types. All old
  hooks are deprecated.
* All tokens are cacheable.
* Most token types are dropped in favor of a 'csrf' token. They already
  were returning the same token anyway.
* All token-using modules will document the required token type in a
  standard manner in action=help and are documented in machine-readable
  fashion in action=paraminfo.

Note this will require updates to all extensions using tokens.

Change-Id: I2793a3f2dd64a4bebb0b4d065e09af1e9f63fb89

RELEASE-NOTES-1.24

@@ -169,6 +169,8 @@ production.
 * (bug 35045) Redirects to sections will now update the URL in browser's address
   bar using the HTML5 History API. When [[Dog]] redirects to [[Animals#Dog]],
   the user will now see "Animals#Dog" in their browser instead of "Dog#Dog".
+* API token handling has been rewritten. Any API module using tokens will need
+  to be updated.
 
 === Bug fixes in 1.24 ===
 * (bug 50572) MediaWiki:Blockip should support gender
@@ -245,6 +247,10 @@ production.
   automatically queries the list of submodule names from the ApiModuleManager.
 * The iwurl parameter to prop=iwlinks is deprecated in favor of iwprop=url, for
   parallelism with prop=langlinks.
+* All tokens should be fetched from action=query&meta=tokens; all other methods
+  of fetching tokens are deprecated. The value needed for meta=tokens's 'type'
+  parameter for each module is documented in the action=help output and is
+  returned from action=paraminfo.
 
 === Languages updated in 1.24 ===
 

docs/hooks.txt

@@ -402,36 +402,39 @@ an action=query submodule. Use this to extend core API modules.
 &$module: Module object
 &$resultPageSet: ApiPageSet object
 
-'APIQueryInfoTokens': Use this hook to add custom tokens to prop=info. Every
-token has an action, which will be used in the intoken parameter and in the
-output (actiontoken="..."), and a callback function which should return the
-token, or false if the user isn't allowed to obtain it. The prototype of the
-callback function is func($pageid, $title), where $pageid is the page ID of the
-page the token is requested for and $title is the associated Title object. In
-the hook, just add your callback to the $tokenFunctions array and return true
-(returning false makes no sense).
+'APIQueryInfoTokens': DEPRECATED! Use ApiQueryTokensRegisterTypes instead.
+Use this hook to add custom tokens to prop=info. Every token has an action,
+which will be used in the intoken parameter and in the output
+(actiontoken="..."), and a callback function which should return the token, or
+false if the user isn't allowed to obtain it. The prototype of the callback
+function is func($pageid, $title), where $pageid is the page ID of the page the
+token is requested for and $title is the associated Title object. In the hook,
+just add your callback to the $tokenFunctions array and return true (returning
+false makes no sense).
 $tokenFunctions: array(action => callback)
 
-'APIQueryRevisionsTokens': Use this hook to add custom tokens to prop=revisions.
-Every token has an action, which will be used in the rvtoken parameter and in
-the output (actiontoken="..."), and a callback function which should return the
-token, or false if the user isn't allowed to obtain it. The prototype of the
-callback function is func($pageid, $title, $rev), where $pageid is the page ID
-of the page associated to the revision the token is requested for, $title the
+'APIQueryRevisionsTokens': DEPRECATED! Use ApiQueryTokensRegisterTypes instead.
+Use this hook to add custom tokens to prop=revisions. Every token has an
+action, which will be used in the rvtoken parameter and in the output
+(actiontoken="..."), and a callback function which should return the token, or
+false if the user isn't allowed to obtain it. The prototype of the callback
+function is func($pageid, $title, $rev), where $pageid is the page ID of the
+page associated to the revision the token is requested for, $title the
 associated Title object and $rev the associated Revision object. In the hook,
 just add your callback to the $tokenFunctions array and return true (returning
 false makes no sense).
 $tokenFunctions: array(action => callback)
 
-'APIQueryRecentChangesTokens': Use this hook to add custom tokens to
-list=recentchanges. Every token has an action, which will be used in the rctoken
-parameter and in the output (actiontoken="..."), and a callback function which
-should return the token, or false if the user isn't allowed to obtain it. The
-prototype of the callback function is func($pageid, $title, $rc), where $pageid
-is the page ID of the page associated to the revision the token is requested
-for, $title the associated Title object and $rc the associated RecentChange
-object. In the hook, just add your callback to the $tokenFunctions array and
-return true (returning false makes no sense).
+'APIQueryRecentChangesTokens': DEPRECATED! Use ApiQueryTokensRegisterTypes instead.
+Use this hook to add custom tokens to list=recentchanges. Every token has an
+action, which will be used in the rctoken parameter and in the output
+(actiontoken="..."), and a callback function which should return the token, or
+false if the user isn't allowed to obtain it. The prototype of the callback
+function is func($pageid, $title, $rc), where $pageid is the page ID of the
+page associated to the revision the token is requested for, $title the
+associated Title object and $rc the associated RecentChange object. In the
+hook, just add your callback to the $tokenFunctions array and return true
+(returning false makes no sense).
 $tokenFunctions: array(action => callback)
 
 'APIQuerySiteInfoGeneralInfo': Use this hook to add extra information to the
@@ -443,13 +446,19 @@ $module: the current ApiQuerySiteInfo module
 sites statistics information.
 &$results: array of results, add things here
 
-'APIQueryUsersTokens': Use this hook to add custom token to list=users. Every
-token has an action, which will be used in the ustoken parameter and in the
-output (actiontoken="..."), and a callback function which should return the
-token, or false if the user isn't allowed to obtain it. The prototype of the
-callback function is func($user) where $user is the User object. In the hook,
-just add your callback to the $tokenFunctions array and return true (returning
-false makes no sense).
+'ApiQueryTokensRegisterTypes': Use this hook to add additional token types to
+action=query&meta=tokens. Note that most modules will probably be able to use
+the 'csrf' token instead of creating their own token types.
+&$salts: array( type => salt to pass to User::getEditToken() )
+
+'APIQueryUsersTokens': DEPRECATED! Use ApiQueryTokensRegisterTypes instead.
+Use this hook to add custom token to list=users. Every token has an action,
+which will be used in the ustoken parameter and in the output
+(actiontoken="..."), and a callback function which should return the token, or
+false if the user isn't allowed to obtain it. The prototype of the callback
+function is func($user) where $user is the User object. In the hook, just add
+your callback to the $tokenFunctions array and return true (returning false
+makes no sense).
 $tokenFunctions: array(action => callback)
 
 'ApiMain::onException': Called by ApiMain::executeActionWithErrorHandling() when
@@ -463,8 +472,8 @@ key for the array that represents the service data. In this data array, the
 key-value-pair identified by the apiLink key is required.
 &$apis: array of services
 
-'ApiTokensGetTokenTypes': Use this hook to extend action=tokens with new token
-types.
+'ApiTokensGetTokenTypes': DEPRECATED! Use ApiQueryTokensRegisterTypes instead.
+Use this hook to extend action=tokens with new token types.
 &$tokenTypes: supported token types in format 'type' => callback function
 used to retrieve this type of tokens.
 

includes/AutoLoader.php

@@ -306,6 +306,7 @@
 	'ApiQuerySiteinfo' => 'includes/api/ApiQuerySiteinfo.php',
 	'ApiQueryStashImageInfo' => 'includes/api/ApiQueryStashImageInfo.php',
 	'ApiQueryTags' => 'includes/api/ApiQueryTags.php',
+	'ApiQueryTokens' => 'includes/api/ApiQueryTokens.php',
 	'ApiQueryUserInfo' => 'includes/api/ApiQueryUserInfo.php',
 	'ApiQueryUsers' => 'includes/api/ApiQueryUsers.php',
 	'ApiQueryWatchlist' => 'includes/api/ApiQueryWatchlist.php',

includes/api/ApiBase.php

@@ -578,6 +578,14 @@ protected function getParamDescription() {
 	 */
 	public function getFinalParams( $flags = 0 ) {
 		$params = $this->getAllowedParams( $flags );
+
+		if ( $this->needsToken() ) {
+			$params['token'] = array(
+				ApiBase::PARAM_TYPE => 'string',
+				ApiBase::PARAM_REQUIRED => true,
+			);
+		}
+
 		wfRunHooks( 'APIGetAllowedParams', array( &$this, &$params, $flags ) );
 
 		return $params;
@@ -591,6 +599,21 @@ public function getFinalParams( $flags = 0 ) {
 	 */
 	public function getFinalParamDescription() {
 		$desc = $this->getParamDescription();
+
+		$tokenType = $this->needsToken();
+		if ( $tokenType ) {
+			if ( !isset( $desc['token'] ) ) {
+				$desc['token'] = array();
+			} elseif ( !is_array( $desc['token'] ) ) {
+				// We ignore a plain-string token, because it's probably an
+				// extension that is supplying the string for BC.
+				$desc['token'] = array();
+			}
+			array_unshift( $desc['token'],
+				"A '$tokenType' token retrieved from action=query&meta=tokens"
+			);
+		}
+
 		wfRunHooks( 'APIGetParamDescription', array( &$this, &$desc ) );
 
 		return $desc;
@@ -1992,31 +2015,84 @@ public function isWriteMode() {
 	 * @return bool
 	 */
 	public function mustBePosted() {
-		return false;
+		return $this->needsToken() !== false;
 	}
 
 	/**
-	 * Returns whether this module requires a token to execute
-	 * It is used to show possible errors in action=paraminfo
-	 * see bug 25248
-	 * @return bool
+	 * Returns the token type this module requires in order to execute.
+	 *
+	 * Modules are strongly encouraged to use the core 'csrf' type unless they
+	 * have specialized security needs. If the token type is not one of the
+	 * core types, you must use the ApiQueryTokensRegisterTypes hook to
+	 * register it.
+	 *
+	 * Returning a non-falsey value here will cause self::getFinalParams() to
+	 * return a required string 'token' parameter and
+	 * self::getFinalParamDescription() to ensure there is standardized
+	 * documentation for it. Also, self::mustBePosted() must return true when
+	 * tokens are used.
+	 *
+	 * In previous versions of MediaWiki, true was a valid return value.
+	 * Returning true will generate errors indicating that the API module needs
+	 * updating.
+	 *
+	 * @return string|false
 	 */
 	public function needsToken() {
 		return false;
 	}
 
 	/**
-	 * Returns the token salt if there is one,
-	 * '' if the module doesn't require a salt,
-	 * else false if the module doesn't need a token
-	 * You have also to override needsToken()
-	 * Value is passed to User::getEditToken
-	 * @return bool|string|array
+	 * Validate the supplied token.
+	 *
+	 * @since 1.24
+	 * @param string $token Supplied token
+	 * @param array $params All supplied parameters for the module
+	 * @return bool
 	 */
-	public function getTokenSalt() {
+	public final function validateToken( $token, array $params ) {
+		$tokenType = $this->needsToken();
+		$salts = ApiQueryTokens::getTokenTypeSalts();
+		if ( !isset( $salts[$tokenType] ) ) {
+			throw new MWException(
+				"Module '{$this->getModuleName()}' tried to use token type '$tokenType' " .
+					'without registering it'
+			);
+		}
+
+		if ( $this->getUser()->matchEditToken(
+			$token,
+			$salts[$tokenType],
+			$this->getRequest()
+		) ) {
+			return true;
+		}
+
+		$webUiSalt = $this->getWebUITokenSalt( $params );
+		if ( $webUiSalt !== null && $this->getUser()->matchEditToken(
+			$token,
+			$webUiSalt,
+			$this->getRequest()
+		) ) {
+			return true;
+		}
+
 		return false;
 	}
 
+	/**
+	 * Fetch the salt used in the Web UI corresponding to this module.
+	 *
+	 * Only override this if the Web UI uses a token with a non-constant salt.
+	 *
+	 * @since 1.24
+	 * @param array $params All supplied parameters for the module
+	 * @return string|array|null
+	 */
+	protected function getWebUITokenSalt( array $params ) {
+		return null;
+	}
+
 	/**
 	 * Gets the user for whom to get the watchlist
 	 *

includes/api/ApiBlock.php

@@ -152,7 +152,6 @@ public function getAllowedParams() {
 				ApiBase::PARAM_TYPE => 'string',
 				ApiBase::PARAM_REQUIRED => true
 			),
-			'token' => null,
 			'expiry' => 'never',
 			'reason' => '',
 			'anononly' => false,
@@ -169,7 +168,6 @@ public function getAllowedParams() {
 	public function getParamDescription() {
 		return array(
 			'user' => 'Username, IP address or IP range you want to block',
-			'token' => 'A block token previously obtained through prop=info',
 			'expiry' => 'Relative expiry time, e.g. \'5 months\' or \'2 weeks\'. ' .
 				'If set to \'infinite\', \'indefinite\' or \'never\', the block will never expire.',
 			'reason' => 'Reason for block',
@@ -192,17 +190,13 @@ public function getDescription() {
 	}
 
 	public function needsToken() {
-		return true;
-	}
-
-	public function getTokenSalt() {
-		return '';
+		return 'csrf';
 	}
 
 	public function getExamples() {
 		return array(
-			'api.php?action=block&user=123.5.5.12&expiry=3%20days&reason=First%20strike',
-			'api.php?action=block&user=Vandal&expiry=never&reason=Vandalism&nocreate=&autoblock=&noemail='
+			'api.php?action=block&user=123.5.5.12&expiry=3%20days&reason=First%20strike&token=123ABC',
+			'api.php?action=block&user=Vandal&expiry=never&reason=Vandalism&nocreate=&autoblock=&noemail=&token=123ABC'
 		);
 	}
 

includes/api/ApiDelete.php

@@ -188,10 +188,6 @@ public function getAllowedParams() {
 			'pageid' => array(
 				ApiBase::PARAM_TYPE => 'integer'
 			),
-			'token' => array(
-				ApiBase::PARAM_TYPE => 'string',
-				ApiBase::PARAM_REQUIRED => true
-			),
 			'reason' => null,
 			'watch' => array(
 				ApiBase::PARAM_DFLT => false,
@@ -220,7 +216,6 @@ public function getParamDescription() {
 		return array(
 			'title' => "Title of the page you want to delete. Cannot be used together with {$p}pageid",
 			'pageid' => "Page ID of the page you want to delete. Cannot be used together with {$p}title",
-			'token' => 'A delete token previously retrieved through prop=info',
 			'reason'
 				=> 'Reason for the deletion. If not set, an automatically generated reason will be used',
 			'watch' => 'Add the page to your watchlist',
@@ -236,11 +231,7 @@ public function getDescription() {
 	}
 
 	public function needsToken() {
-		return true;
-	}
-
-	public function getTokenSalt() {
-		return '';
+		return 'csrf';
 	}
 
 	public function getExamples() {

includes/api/ApiEditPage.php

@@ -513,10 +513,6 @@ public function getAllowedParams() {
 				ApiBase::PARAM_TYPE => 'string',
 			),
 			'text' => null,
-			'token' => array(
-				ApiBase::PARAM_TYPE => 'string',
-				ApiBase::PARAM_REQUIRED => true
-			),
 			'summary' => null,
 			'minor' => false,
 			'notminor' => false,
@@ -575,8 +571,8 @@ public function getParamDescription() {
 			'sectiontitle' => 'The title for a new section',
 			'text' => 'Page content',
 			'token' => array(
-				'Edit token. You can get one of these through prop=info.',
-				"The token should always be sent as the last parameter, or at " .
+				/* Standard description is automatically prepended */
+				'The token should always be sent as the last parameter, or at ' .
 					"least, after the {$p}text parameter"
 			),
 			'summary'
@@ -589,7 +585,8 @@ public function getParamDescription() {
 				'Used to detect edit conflicts; leave unset to ignore conflicts'
 			),
 			'starttimestamp' => array(
-				'Timestamp when you obtained the edit token.',
+				'Timestamp when you began the editing process, e.g. when the current page content ' .
+					'was loaded for editing.',
 				'Used to detect edit conflicts; leave unset to ignore conflicts'
 			),
 			'recreate' => 'Override any errors about the article having been deleted in the meantime',
@@ -616,11 +613,7 @@ public function getParamDescription() {
 	}
 
 	public function needsToken() {
-		return true;
-	}
-
-	public function getTokenSalt() {
-		return '';
+		return 'csrf';
 	}
 
 	public function getExamples() {