Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-105704: Disallow square brackets ([ and ]) in domain names for parsed URLs #129418

Merged
merged 4 commits into from
Jan 31, 2025

Conversation

sethmlarson
Copy link
Contributor

@sethmlarson sethmlarson commented Jan 28, 2025

Opening this pull request in favor of #111261, as the original author appears to not be responding to reviews. This moves the error to occur during parsing instead of on the ParseResult properties.

cc @gpshead @orsenthil @pschoen-itsc as reviewers on the previous PR.

self.assertRaises(ValueError, urllib.parse.urlsplit, 'scheme://prefix.[::1]?')
self.assertRaises(ValueError, urllib.parse.urlsplit, 'scheme://[::1].suffix?')
self.assertRaises(ValueError, urllib.parse.urlsplit, 'scheme://user@prefix.[v6a.ip]')
self.assertRaises(ValueError, urllib.parse.urlsplit, 'scheme://user@[v6a.ip].suffix')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It might be helpful to add tests only with [, and tests only with ]. Such URL is also invalid, no?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I'll add tests with that structure too! Thanks :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added in ebf92bb

@sethmlarson sethmlarson requested a review from vstinner January 30, 2025 17:48
Copy link
Member

@orsenthil orsenthil left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Thank you!

@sethmlarson
Copy link
Contributor Author

The failure in Windows free-threading looks unrelated, I can't merge without all the checks passing so if someone else can that would be appreciated 💜

@orsenthil orsenthil enabled auto-merge (squash) January 31, 2025 16:50
@orsenthil orsenthil disabled auto-merge January 31, 2025 16:50
@orsenthil
Copy link
Member

Looks like that's a required to succeed.

@orsenthil orsenthil merged commit d89a5f6 into python:main Jan 31, 2025
43 checks passed
@miss-islington-app
Copy link

Thanks @sethmlarson for the PR, and @orsenthil for merging it 🌮🎉.. I'm working now to backport this PR to: 3.9, 3.10, 3.11, 3.12, 3.13.
🐍🍒⛏🤖

miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 31, 2025
…es for parsed URLs (pythonGH-129418)

* pythongh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <[email protected]>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jan 31, 2025

GH-129526 is a backport of this pull request to the 3.13 branch.

@bedevere-app
Copy link

bedevere-app bot commented Jan 31, 2025

GH-129527 is a backport of this pull request to the 3.12 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.12 bug and security fixes label Jan 31, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 31, 2025
…es for parsed URLs (pythonGH-129418)

* pythongh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <[email protected]>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jan 31, 2025

GH-129528 is a backport of this pull request to the 3.11 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.11 only security fixes label Jan 31, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 31, 2025
…es for parsed URLs (pythonGH-129418)

* pythongh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <[email protected]>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jan 31, 2025

GH-129529 is a backport of this pull request to the 3.10 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.10 only security fixes label Jan 31, 2025
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Jan 31, 2025
…es for parsed URLs (pythonGH-129418)

* pythongh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <[email protected]>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
@bedevere-app
Copy link

bedevere-app bot commented Jan 31, 2025

GH-129530 is a backport of this pull request to the 3.9 branch.

@bedevere-app bedevere-app bot removed the needs backport to 3.9 only security fixes label Jan 31, 2025
@sethmlarson sethmlarson deleted the bracketed-domains branch January 31, 2025 19:31
gpshead pushed a commit that referenced this pull request Feb 2, 2025
…mes for parsed URLs (GH-129418) (GH-129526)

gh-105704: Disallow square brackets (`[` and `]`) in domain names for parsed URLs (GH-129418)

* gh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references



* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
gpshead pushed a commit that referenced this pull request Feb 2, 2025
…mes for parsed URLs (GH-129418) (GH-129527)

gh-105704: Disallow square brackets (`[` and `]`) in domain names for parsed URLs (GH-129418)

* gh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references



* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

(cherry picked from commit d89a5f6)

Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Peter Bierma <[email protected]>
@bedevere-bot
Copy link

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows11 Refleaks 3.13 has failed when building commit 90e526a.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1484/builds/398) and take a look at the build logs.
  4. Check if the failure is related to this commit (90e526a) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1484/builds/398

Failed tests:

  • test_threading

Failed subtests:

  • test_finalize_with_trace - test.test_threading.ThreadTests.test_finalize_with_trace
  • test_daemon_threads_shutdown_stdout_deadlock - test.test_io.CMiscIOTest.test_daemon_threads_shutdown_stdout_deadlock

Summary of the results of the build (if available):

==

Click to see traceback logs
Traceback (most recent call last):
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\test_io.py", line 4625, in test_daemon_threads_shutdown_stdout_deadlock
    self.check_daemon_threads_shutdown_deadlock('stdout')
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\test_io.py", line 4618, in check_daemon_threads_shutdown_deadlock
    self.assertRegex(err, pattern)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^
AssertionError: Regex didn't match: "Fatal Python error: _enter_buffered_busy: could not acquire lock for <(_io\\.)?BufferedWriter name='<stdout>'> at interpreter shutdown, possibly due to daemon threads" not found in 'Windows fatal exception: access violation\n\nWindows fatal exception: access violation\n\n'


Traceback (most recent call last):
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\test_threading.py", line 470, in test_finalize_with_trace
    assert_python_ok("-c", """if 1:
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
        import sys, threading
        ^^^^^^^^^^^^^^^^^^^^^
    ...<17 lines>...
        sys.settrace(func)
        ^^^^^^^^^^^^^^^^^^
        """)
        ^^^^
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\support\script_helper.py", line 180, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\support\script_helper.py", line 165, in _assert_python
    res.fail(cmd_line)
    ~~~~~~~~^^^^^^^^^^
  File "b:\uildarea\3.13.ware-win11.refleak\build\Lib\test\support\script_helper.py", line 75, in fail
    raise AssertionError("Process return code is %d\n"
    ...<13 lines>...
                            err))
AssertionError: Process return code is 3221225477
command line: ['b:\\uildarea\\3.13.ware-win11.refleak\\build\\PCbuild\\amd64\\python_d.exe', '-X', 'faulthandler', '-I', '-c', "if 1:\n            import sys, threading\n\n            # A deadlock-killer, to prevent the\n            # testsuite to hang forever\n            def killer():\n                import os, time\n                time.sleep(2)\n                print('program blocked; aborting')\n                os._exit(2)\n            t = threading.Thread(target=killer)\n            t.daemon = True\n            t.start()\n\n            # This is the trace function\n            def func(frame, event, arg):\n                threading.current_thread()\n                return func\n\n            sys.settrace(func)\n            "]

srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this pull request Feb 7, 2025
…es for parsed URLs (python#129418)

* pythongh-105704: Disallow square brackets ( and ) in domain names for parsed URLs

* Use Sphinx references

Co-authored-by: Peter Bierma <[email protected]>

* Add mismatched bracket test cases, fix news format

* Add more test coverage for ports

---------

Co-authored-by: Peter Bierma <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type-security A security issue
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants