Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retry on tar extraction errors #9753

Merged
merged 1 commit into from
Dec 10, 2024
Merged

Retry on tar extraction errors #9753

merged 1 commit into from
Dec 10, 2024

Conversation

charliermarsh
Copy link
Member

@charliermarsh charliermarsh commented Dec 9, 2024

Summary

So the error here is:

ExtractError("cpython-3.11.11%2B20241206-aarch64-apple-darwin-install_only_stripped.tar.gz", Io(Custom { kind: UnexpectedEof, error: TarError { desc: "failed to unpack `/Users/crmarsh/.local/share/uv/python/.cache/.tmpkqFzqE/python/lib/libpython3.11.dylib`", io: Custom { kind: UnexpectedEof, error: TarError { desc: "failed to unpack `python/lib/libpython3.11.dylib` into `/Users/crmarsh/.local/share/uv/python/.cache/.tmpkqFzqE/python/lib/libpython3.11.dylib`", io: Custom { kind: UnexpectedEof, error: "unexpected end of file" } } } } }))

This isn't a Reqwest error, so we miss it in is_extended_transient_error.

We could add TarError or ExtractError here, but... should we? This PR just extends it to any error that has an IO source. I don't see much of a downside.

Closes #9747.

Test Plan

First, ran: uv run ./scripts/create-python-mirror.py --name cpython --arch aarch64 --os darwin.

Then, dropped this into ./scripts/mirror/server.py:

import os
import random
from http.server import SimpleHTTPRequestHandler, HTTPServer


class GlitchyStaticServer(SimpleHTTPRequestHandler):
    def do_GET(self):
        """Handle GET request."""
        file_path = self.translate_path(self.path)
        
        if not os.path.exists(file_path):
            self.send_error(404, "File not found")
            return
        
        try:
            with open(file_path, 'rb') as f:
                file_content = f.read()

            # Introduce an "unexpected end of file" glitch randomly
            if random.random() < 0.75:  # 75% chance of glitch
                glitch_point = random.randint(1, len(file_content) - 1)
                file_content = file_content[:glitch_point]

            self.send_response(200)
            self.send_header("Content-type", self.guess_type(file_path))
            self.send_header("Content-Length", len(file_content))
            self.end_headers()
            self.wfile.write(file_content)
        
        except Exception as e:
            self.send_error(500, f"Internal Server Error: {e}")
        

def run(server_class=HTTPServer, handler_class=GlitchyStaticServer, port=8080):
    """Run the server."""
    server_address = ('', port)
    httpd = server_class(server_address, handler_class)
    print(f"Serving on port {port} with glitchy behavior")
    httpd.serve_forever()


if __name__ == "__main__":
    run()

Then ran python server.py from that directory.

From there, ran UV_PYTHON_INSTALL_MIRROR="http://localhost:8080" cargo run python install 3.11 --reinstall --verbose to reliably test retries.

@charliermarsh charliermarsh added bug Something isn't working network Network connectivity e.g. proxies, DNS, and SSL labels Dec 9, 2024
Copy link
Member

@konstin konstin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@charliermarsh charliermarsh merged commit eb21e4b into main Dec 10, 2024
64 checks passed
@charliermarsh charliermarsh deleted the charlie/py-retry branch December 10, 2024 12:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working network Network connectivity e.g. proxies, DNS, and SSL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Downloading python from behind corporate proxy fails to download in uv but works with wget and rye
2 participants