Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

encoding Ö (O-Umlaut) error in command 'aws ec2 describe-instances' when redirecting to a file #742

Closed
fietske opened this issue Apr 4, 2014 · 6 comments · Fixed by #756
Assignees
Labels
bug This issue is a bug.

Comments

@fietske
Copy link

fietske commented Apr 4, 2014

we have a tag value with an Ö (O-Umlaut) in the string value.

"aws ec2 describe-instances --output=json" works just fine
"aws ec2 describe-instances --output=text (or output=table" gives the following error:
'ascii' codec can't encode character u'\xf6' in position 60: ordinal not in range(128)

I'd expect it to either go consistently wrong in all output formats or consistently go without error in all output formats.

@jamesls
Copy link
Member

jamesls commented Apr 4, 2014

What version of the CLI are you running? There's a similar bug filed a while back that was fixed in later releases.

@jamesls
Copy link
Member

jamesls commented Apr 4, 2014

Ah found it: #721
Let me know if you're still seeing this issue in the latest version of the CLI.

@fietske
Copy link
Author

fietske commented Apr 4, 2014

1.3.6—
Sent from Mailbox for iPhone

On Fri, Apr 4, 2014 at 11:46 PM, James Saryerwinnie
[email protected] wrote:

What version of the CLI are you running? There's a similar bug filed a while back that was fixed in later releases.

Reply to this email directly or view it on GitHub:
#742 (comment)

@jamesls
Copy link
Member

jamesls commented Apr 5, 2014

Interesting, not seeing that issue:

$ aws --version
aws-cli/1.3.6 Python/2.7.5 Darwin/12.5.0

$ aws ec2 describe-instances --query 'Reservations[].Instances[].Tags' --region us-east-1 --output table
----------------------------
|     DescribeInstances    |
+-------+------------------+
|  Key  |      Value       |
+-------+------------------+
|  Ö    |  Ö               |
+-------+------------------+

 aws ec2 describe-instances --query 'Reservations[].Instances[].Tags' --region us-east-1 --output text
Ö  Ö

It might also be an issue with the terminal encoding. For example, I have my terminal set to use utf-8. We encode the output based on the encoding of sys.stdout. You can check this with:

$ python -c "import sys; print(sys.stdout.encoding)"
UTF-8

@fietske
Copy link
Author

fietske commented Apr 9, 2014

error still there........I just reproduced it and the terminal encoding is UTF-8:

$ aws --version
aws-cli/1.3.6 Python/2.7.5 Darwin/13.1.0
$ python -c "import sys; print(sys.stdout.encoding)"
UTF-8
$ aws ec2 describe-instances --output=table >instances_table
'ascii' codec can't encode character u'\xf6' in position 60: ordinal not in range(128)
(AWSCLI)defssw2s0:.aws mvn$

@jamesls
Copy link
Member

jamesls commented Apr 10, 2014

Ahh, that's the issue, we don't handle the case when we redirect stdout to a file. Reopening...

@jamesls jamesls reopened this Apr 10, 2014
@jamesls jamesls changed the title encoding Ö (O-Umlaut) error in command 'aws ec2 describe-instances' encoding Ö (O-Umlaut) error in command 'aws ec2 describe-instances' when redirecting to a file Apr 10, 2014
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 17, 2014
In python2, if a stream has no encoding, unicode is encoded
using the default encoding.  In python3, sys.stdout is already
text based (it expects str() types).

In python2, we introduce a compat layer that will give you a
stream writer that accepts unicode and encodes the contents into
the local.preferredencoding().

Simple repro: tag an ec2 instance with a unicode char and run:

    aws ec2 describe-instances | grep TAGS

You'll get a ascii encode error.  With this change you won't.

Fixes aws#742.
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 17, 2014
In python2, if a stream has no encoding, unicode is encoded
using the default encoding.  In python3, sys.stdout is already
text based (it expects str() types).

In python2, we introduce a compat layer that will give you a
stream writer that accepts unicode and encodes the contents into
the local.preferredencoding().

Simple repro: tag an ec2 instance with a unicode char and run:

    aws ec2 describe-instances | grep TAGS

You'll get a ascii encode error.  With this change you won't.

Fixes aws#742.
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 17, 2014
In python2, if a stream has no encoding, unicode is encoded
using the default encoding.  In python3, sys.stdout is already
text based (it expects str() types).

In python2, we introduce a compat layer that will give you a
stream writer that accepts unicode and encodes the contents into
the local.preferredencoding().

Simple repro: tag an ec2 instance with a unicode char and run:

    aws ec2 describe-instances | grep TAGS

You'll get a ascii encode error.  With this change you won't.

Fixes aws#742.
jamesls added a commit to jamesls/aws-cli that referenced this issue Apr 17, 2014
In python2, if a stream has no encoding, unicode is encoded
using the default encoding.  In python3, sys.stdout is already
text based (it expects str() types).

In python2, we introduce a compat layer that will give you a
stream writer that accepts unicode and encodes the contents into
the local.preferredencoding().

Simple repro: tag an ec2 instance with a unicode char and run:

    aws ec2 describe-instances | grep TAGS

You'll get a ascii encode error.  With this change you won't.

Fixes aws#742.
jamesls added a commit that referenced this issue Apr 18, 2014
jamesls added a commit that referenced this issue Apr 21, 2014
* release-1.3.7: (28 commits)
  Bumping version to 1.3.7
  Add #742 to changelog
  Add a comment about why get_stdout_text_writer is needed.
  Code review feedback
  Py3 integ test fixes
  Update changelog with #749
  Add compat layer for text based stream writers
  Fix S3 sync issue with keys containing urlencode values
  Add issue to changelog
  Remove print statement in test
  Fix issue with scalar/non-scalar lists
  Fix doc example for s3api put-object
  Refactor load-cli-arg common event code
  Add 750 to the changelog
  Update paramfile custom argument to use events
  Aggregate dupe keys into a list in datapipeline translation
  Add issue to CHANGELOG
  Do not auto parse JSON based on filename
  Update tests to not mock __builtin__.open
  Allow custom param values to be read from files/urls
  ...
thoward-godaddy pushed a commit to thoward-godaddy/aws-cli that referenced this issue Feb 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug This issue is a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants