cloudformation deploy does not honour tabs in JSON templates

[galaxy4public]

Most of our templates are using tabs for the indentation and everything was fine until we tried to feed the template to aws cloudformation deploy which did not like the template despite that every other function in AWS CLI works with the template just fine:

[galaxy@athena templates]$ aws cloudformation validate-template --template-body file://codepipeline.template
{
    "CapabilitiesReason": "The following resource(s) require capabilities: [AWS::IAM::Role]", 
    "Description": "Playing with CodeCommit, CodeBuild, and CodeDeploy", 
    "Parameters": [

and the only function that fails to parse the template is:

[galaxy@athena templates]$ aws cloudformation deploy --stack-name "galaxy-ccc" --template-file codepipeline.template --capabilities CAPABILITY_IAM

while scanning for the next token
found character '\t' that cannot start any token
  in "<string>", line 2, column 1:
    	"AWSTemplateFormatVersion": "20 ... 
    ^

A quick fix is to replace tabs with spaces:

[galaxy@athena templates]$ sed 's,\t, ,g' codepipeline.template > c.template
[galaxy@athena templates]$ aws cloudformation deploy --stack-name "galaxy-ccc" --template-file c.template --capabilities CAPABILITY_IAM
Waiting for changeset to be created..
Waiting for stack create/update to complete

... but it would mean that we would need to fix all our templates which are valid JSON just to workaround a bug in the tool! :(

[jamesls]

The deploy command is a hand written customization. It looks like it's relying on the fact that it can use the yaml parser to parse JSON whereas the rest of the CLI uses the json module.

>>> import yaml
>>> yaml.load(StringIO('{"foo":\t"bar"}'))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/__init__.py", line 71, in load
    return loader.get_single_data()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/constructor.py", line 37, in get_single_data
    node = self.get_single_node()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/composer.py", line 35, in get_single_node
    if not self.check_event(StreamEndEvent):
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/parser.py", line 98, in check_event
    self.current_event = self.state()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/parser.py", line 143, in parse_implicit_document_start
    StreamEndToken):
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/scanner.py", line 116, in check_token
    self.fetch_more_tokens()
  File ".virtualenvs/aws-cli/lib/python2.7/site-packages/yaml/scanner.py", line 257, in fetch_more_tokens
    % ch.encode('utf-8'), self.get_mark())
yaml.scanner.ScannerError: while scanning for the next token
found character '\t' that cannot start any token
  in "<file>", line 1, column 8
>>> import json
>>> json.loads('{"foo":\t"bar"}')
{u'foo': u'bar'}

[jamesls]

We should be able to first try to parse this as JSON and then fall back to yaml if that fails.