Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bpo-17239: Disable external entities in SAX parser #9217

Merged
merged 1 commit into from
Sep 23, 2018
Merged

bpo-17239: Disable external entities in SAX parser #9217

merged 1 commit into from
Sep 23, 2018

Conversation

tiran
Copy link
Member

@tiran tiran commented Sep 12, 2018 •
edited by bedevere-bot
Loading

The SAX parser no longer processes general external entities by default
to increase security. Before, the parser created network connections
to fetch remote files or loaded local files from the file system for DTD
and entities.

Signed-off-by: Christian Heimes [email protected]

https://bugs.python.org/issue17239

@tiran tiran requested a review from zooba September 12, 2018 16:26
@csabella
Copy link
Contributor

Since default functionality is changing, should this be included in the What's New in 3.8 page?

vstinner
vstinner approved these changes Sep 13, 2018
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Since there is a way to enable it in Python 3.7 and older, it's fine to change the default.

Should we change the default in Python 3.7 and older? I'm not sure about that.

Doc/library/xml.rst Outdated
@@ -75,6 +75,7 @@ decompression bomb Safe Safe Safe S
2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
the unexpanded entity verbatim.
3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
4. External general entities are no longer processed by default.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe elaborate on "no longer" (just mention Python 3.8?).

Misc/NEWS.d/next/Security/2018-09-11-18-30-55.bpo-17239.kOpwK2.rst Outdated
@@ -0,0 +1,3 @@
The sax parser no longer processes external entities by default. External
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest "The XML save parser". It's not obvious that sax is related to XML if you are not used to XML.

zooba
zooba approved these changes Sep 17, 2018
View reviewed changes
Copy link
Member

@zooba zooba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed with both Cheryl and Victor's suggestions.

@tiran tiran force-pushed the bpo17239-sax-ges branch 2 times, most recently from 15bf855 to dd8ee7e Compare September 17, 2018 21:34
@tiran
Copy link
Member Author

tiran commented Sep 17, 2018

@csabella @vstinner @zooba I have updated the PR.

@zooba
Copy link
Member

zooba commented Sep 17, 2018

LGTM

vstinner
vstinner approved these changes Sep 17, 2018
View reviewed changes
Copy link
Member

@vstinner vstinner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but maybe remove version numbers until they are really fixed?

Doc/library/xml.rst Outdated
@@ -75,6 +75,8 @@ decompression bomb Safe Safe Safe S
2. :mod:`xml.dom.minidom` doesn't expand external entities and simply returns
the unexpanded entity verbatim.
3. :mod:`xmlrpclib` doesn't expand external entities and omits them.
4. External general entities are no longer processed by default since Python
2.7.16, 3.6.7, 3.7.1, and 3.8.0.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might only list 3.8.0, but complete this table once we fixed other branches.

I'm not sure about backporting this backward incompatible change to other branches, but if we do it, we should also fix 3.4 and 3.5, no?

@tiran tiran force-pushed the bpo17239-sax-ges branch 3 times, most recently from d8125c2 to 55db8ce Compare September 22, 2018 05:57
@tiran
bpo-17239: Disable external entities in SAX parser
0c8828c 
The xml.sax and xml.dom.minidom parsers no longer processes external
entities to increase security. Before, the parser created network
connections to fetch remote files or loaded local files from the file
system for DTD and entities.

Signed-off-by: Christian Heimes <[email protected]>
@miss-islington
Copy link
Contributor

@tiran: Status check is done, and it's a success ✅ .

@miss-islington miss-islington merged commit 17b1d5d into python:master Sep 23, 2018
@miss-islington
Copy link
Contributor

Thanks @tiran for the PR 🌮🎉.. I'm working now to backport this PR to: 2.7, 3.6, 3.7.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

@miss-islington
Copy link
Contributor

Sorry, @tiran, I could not cleanly backport this to 3.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 3.7

@miss-islington miss-islington self-assigned this Sep 23, 2018
@miss-islington
Copy link
Contributor

Sorry, @tiran, I could not cleanly backport this to 3.6 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 3.6

@miss-islington
Copy link
Contributor

Sorry, @tiran, I could not cleanly backport this to 2.7 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 17b1d5d4e36aa57a9b25a0e694affbd1ee637e45 2.7

@bedevere-bot
Copy link

GH-9511 is a backport of this pull request to the 3.7 branch.

@bedevere-bot
Copy link

GH-9512 is a backport of this pull request to the 3.6 branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@zooba zooba zooba approved these changes

@vstinner vstinner vstinner approved these changes

Assignees

@miss-islington miss-islington

Labels
type-security A security issue
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@tiran @csabella @zooba @miss-islington @bedevere-bot @vstinner @the-knights-who-say-ni