Unify JSON serialization format of transactions (Version: 2.0.0-b2)

[intelliot]

There are many API methods that can return transactions. Currently, the return formats are inconsistent. The following table shows where fields are located as of rippled v1.12.0 (unchanged since v1.0.1).

Note: In the following table, the "top" level is with respect to the individual object representing the transaction; for methods that return arrays of transactions, the "top" level is per element of the array. "Transaction Instructions" refer to the canonical fields of the transaction as defined in the transaction format, such as the Account and Flags fields.

Method	Transaction Instructions	Transaction Metadata	hash	ledger_index	inLedger	validated	ledger_hash	Binary blob	date
ledger (with transactions expanded)	Top level	metaData field (JSON) or in meta field¹ (binary)	Top level	Above transaction level	None	Above transaction level	None	tx_blob field¹	None
tx	Top level	meta field	Top level	Top level	Top level	Top level	None	tx field¹	Top level
tx_history (deprecated)	Top level	None	Top level	Top level	Top level	None	None	Not supported	None
account_tx	tx field	meta field	tx field	tx field	tx field	Top level	None	tx_blob field¹	None
transaction_entry (would like to deprecate)	tx_json field	metadata field	tx_json field	Top level	None	Top level	Top level	Not supported	None
sign, sign_for, submit, and submit_multisigned	tx_json field	(N/A)	tx_json field	(N/A)	(N/A)	(N/A)	(N/A)	tx_blob field	None
Streams from subscribe	transaction field	meta field²	Top level	Top level²	None	Top level²	Top level²	Not supported	transaction field
Data API v2 methods	tx field	meta field	Top level	Top level	None	None (Data API only reports validated transaction data)	None	tx¹	Top level (as ISO8601 timestamp, not Ripple time)

¹ Only if the request asked for binary data
² The transactions_proposed stream omits these fields because the transactions' outcomes are not yet final.

Recommendation

Change all of rippled's methods to serialize transactions to JSON in a single consistent format. The format I suggest is essentially the same format the Data API uses, with modifications to accommodate for some formats only the rippled APIs handle. Specifically:

1. Always use tx_blob for binary format. Always use tx_json for JSON format transaction instructions.
2. Only include the canonical transaction instructions ("uppercase fields") and signing fields in the tx_json field with transaction instructions. Move all "lowercase" fields like hash and ledger_index out to the top level of the transaction. (Note: the issuer/currency/value sub-fields of amounts are canonical fields even though they are lowercase.)
3. Always use meta for JSON metadata. Use meta_blob for binary metadata.
4. Remove the inLedger field entirely. (It was a deprecated alias for ledger_index, which is the ledger index of the ledger that includes this transaction.)
5. Add the close_time_iso field, as a sibling to tx_json, to all validated transactions that are from a closed ledger. Most of the time this would be the close time of the parent ledger. Make this a UTC ISO8601 timestamp with whole-seconds resolution, for example 2018-07-22T16:37:55Z. Omit this field from transactions that are not in closed ledgers.
6. Add the ledger_hash field to any transactions when pulling them from a closed ledger. Omit this field from transactions that are not in closed ledgers.
7. Add the validated boolean field to all transactions, even when this information is redundant because of context (for example, listing transactions in a validated ledger).

• Copied from Unify JSON serialization format of transactions clio#722 because this should be implemented in both rippled and Clio.

[pkcs8]

Great to see this initiative to unify output formats. meta, metadata and metaData has been one of my longest pet peeves.

[intelliot]

tx_history is deprecated and I haven't seen any usage of it for many years. I think that if you specify "command": "tx_history" and "api_version": 2, then you should get an error similar to:

{
  "error": "unknownCmd",
  "error_code": 32,
  "error_message": "Unknown method.",
  "status": "error",
  "type": "response",
  "id": 1,
  "request": {
    "id": 1,
    "command": "tx_history",
    "api_version": 2
  }
}

If we want to be friendlier, I think it's also OK to add an informational field that says tx_history is deprecated and was removed in api_version 2.