Webdriver timed out in container #96

pd-448482 · 2023-05-31T19:36:25Z

Description

Using the latest (4.0) docker container, when I start the container, the webdriver session times out.

To Reproduce

Create config with single flight
Start container with config file mounted
Wait ~10 minutes for timeout

Expected Behavior

Flight scheduled/monitored

Version

Auto-Southwest Check-In v4.0

Additional context

Logs:

2023-05-31 15:22:39 DEBUG MainProcess[log]: Initialized the application
2023-05-31 15:22:39 DEBUG MainProcess[main]: Called with 0 arguments
2023-05-31 15:22:40 DEBUG MainProcess[config]: Reading the configuration file
2023-05-31 15:22:40 DEBUG MainProcess[config]: Adding 0 accounts from configuration file
2023-05-31 15:22:40 DEBUG MainProcess[config]: Setting check fares to True
2023-05-31 15:22:40 DEBUG MainProcess[config]: Adding 1 flights from configuration file
2023-05-31 15:22:40 DEBUG MainProcess[config]: Setting notification level to 1
2023-05-31 15:22:40 DEBUG MainProcess[config]: Using 1 notification services
2023-05-31 15:22:40 DEBUG MainProcess[config]: Setting retrieval interval to 24 hours
2023-05-31 15:22:40 DEBUG MainProcess[main]: Monitoring 0 accounts and 1 flights
2023-05-31 15:22:40 DEBUG Process-1[flight_retriever]: Scheduling reservations for 1 flights
2023-05-31 15:22:40 DEBUG Process-1[checkin_scheduler]: No headers set. Refreshing...
2023-05-31 15:22:40 DEBUG Process-1[checkin_scheduler]: Refreshing headers for current session
2023-05-31 15:22:40 DEBUG Process-1[webdriver]: Starting webdriver for current session
Process Process-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/app/lib/flight_retriever.py", line 36, in monitor_flights
    self._schedule_reservations(flights)
  File "/app/lib/flight_retriever.py", line 64, in _schedule_reservations
    self.checkin_scheduler.schedule(confirmation_numbers)
  File "/app/lib/checkin_scheduler.py", line 36, in schedule
    self.refresh_headers()
  File "/app/lib/checkin_scheduler.py", line 48, in refresh_headers
    webdriver.set_headers()
  File "/app/lib/webdriver.py", line 59, in set_headers
    driver = self._get_driver()
             ^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 142, in _get_driver
    driver = Chrome(
             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seleniumwire/undetected_chromedriver/webdriver.py", line 61, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 429, in __init__
    super(Chrome, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 715, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
  (Session info: chrome=112.0.5615.165)

The text was updated successfully, but these errors were encountered:

jdholtz · 2023-05-31T21:44:38Z

Thanks for filing this issue. Does this issue happen repeatedly or just once?

I added better resistance to random webdriver initialization errors in the next version (will be released soon). If it just happens every once in a while, this added resistance should make this a non-issue

pd-448482 · 2023-05-31T22:26:58Z

It is happening repeatedly. Strangely, I do recall this version of the container working. So I'm not sure what has changed on the local container. Is the bundled chrome version able to auto update?

jdholtz · 2023-05-31T22:36:13Z

It shouldn’t as long as you don’t specify a chrome version in your config file (that is handled in the Docker container itself)

jdholtz · 2023-06-01T04:36:43Z

I just tried on the jdholtz/auto-southwest-check-in:v4.0 image and it has repeatedly started the webdriver in less than a second for me. Therefore, it probably doesn't have anything to do with the image itself. Could something have possibly changed with your network configuration? There aren't too many variables that could make the image behave differently on your machine compared to mine.

Could you also try the :develop image tag and see if that helps at all?

pd-448482 · 2023-06-01T13:39:35Z

I haven't set the chrome version in the config file. So I'm just using the bundled version. I tried the :develop tag and got similar results:

2023-06-01 09:27:13 DEBUG MainProcess[log]: Initialized the application
2023-06-01 09:27:13 DEBUG MainProcess[main]: Called with 0 arguments
2023-06-01 09:27:14 DEBUG MainProcess[config]: Reading the configuration file
2023-06-01 09:27:14 DEBUG MainProcess[config]: Adding 0 accounts from configuration file
2023-06-01 09:27:14 DEBUG MainProcess[config]: Setting check fares to True
2023-06-01 09:27:14 DEBUG MainProcess[config]: Adding 1 flights from configuration file
2023-06-01 09:27:14 DEBUG MainProcess[config]: Setting notification level to 1
2023-06-01 09:27:14 DEBUG MainProcess[config]: Using 1 notification services
2023-06-01 09:27:14 DEBUG MainProcess[config]: Setting retrieval interval to 24 hours
2023-06-01 09:27:14 DEBUG MainProcess[main]: Monitoring 0 accounts and 1 flights
2023-06-01 09:27:14 DEBUG Process-1[flight_retriever]: Scheduling reservations for 1 flights
2023-06-01 09:27:14 DEBUG Process-1[checkin_scheduler]: No headers set. Refreshing...
2023-06-01 09:27:14 DEBUG Process-1[checkin_scheduler]: Refreshing headers for current session
2023-06-01 09:27:14 DEBUG Process-1[webdriver]: Starting webdriver for current session
2023-06-01 09:37:15 DEBUG Process-1[webdriver]: An exception occured when initializing the webdriver: Name: TimeoutException. Error: Message: timeout: Timed out receiving message from renderer: 600.000
  (Session info: chrome=113.0.5672.126)

2023-06-01 09:37:15 DEBUG Process-1[webdriver]: 2 more attempts

jdholtz · 2023-06-05T13:11:11Z

Hey @pldimarco, do you have another machine you could test on to see if you get the same issue?

pd-448482 · 2023-06-06T12:56:25Z

Yeah, I tried it out on my laptop and it worked fine. So on to debugging how a stable image went from working to not working, since the issue occurred before the last release. Seems something with my unraid host then.

Dominick-P · 2023-06-12T03:53:34Z

I'm experiencing the same thing with my container, webdriver will timeout 3 times then hard fail.

             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seleniumwire/undetected_chromedriver/webdriver.py", line 61, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 429, in __init__
    super(Chrome, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 715, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
  (Session info: chrome=114.0.5735.90)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/app/lib/flight_retriever.py", line 38, in monitor_flights
    self._schedule_reservations(flights)
  File "/app/lib/flight_retriever.py", line 66, in _schedule_reservations
    self.checkin_scheduler.schedule(confirmation_numbers)
  File "/app/lib/checkin_scheduler.py", line 36, in schedule
    self.refresh_headers()
  File "/app/lib/checkin_scheduler.py", line 48, in refresh_headers
    webdriver.set_headers()
  File "/app/lib/webdriver.py", line 58, in set_headers
    driver = self._get_driver()
             ^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 139, in _get_driver
    driver = self._init_driver()
             ^^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 181, in _init_driver
    raise RuntimeError(
RuntimeError: Failed to initialize the webdriver after 3 attempts
Process Process-1:
Traceback (most recent call last):
  File "/app/lib/webdriver.py", line 164, in _init_driver
    driver = Chrome(
             ^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/seleniumwire/undetected_chromedriver/webdriver.py", line 61, in __init__
    super().__init__(*args, **kwargs)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 429, in __init__
    super(Chrome, self).__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chrome/webdriver.py", line 84, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/chromium/webdriver.py", line 104, in __init__
    super().__init__(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
    self.start_session(capabilities, browser_profile)
  File "/usr/local/lib/python3.11/site-packages/undetected_chromedriver/__init__.py", line 715, in start_session
    super(selenium.webdriver.chrome.webdriver.WebDriver, self).start_session(
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
    self.error_handler.check_response(response)
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 600.000
  (Session info: chrome=114.0.5735.90)
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/app/lib/flight_retriever.py", line 38, in monitor_flights
    self._schedule_reservations(flights)
  File "/app/lib/flight_retriever.py", line 66, in _schedule_reservations
    self.checkin_scheduler.schedule(confirmation_numbers)
  File "/app/lib/checkin_scheduler.py", line 36, in schedule
    self.refresh_headers()
  File "/app/lib/checkin_scheduler.py", line 48, in refresh_headers
    webdriver.set_headers()
  File "/app/lib/webdriver.py", line 58, in set_headers
    driver = self._get_driver()
             ^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 139, in _get_driver
    driver = self._init_driver()
             ^^^^^^^^^^^^^^^^^^^
  File "/app/lib/webdriver.py", line 181, in _init_driver
    raise RuntimeError(
RuntimeError: Failed to initialize the webdriver after 3 attempts

Dominick-P · 2023-06-12T17:39:49Z

It seems that starting the container in --privileged mode is a workaround to this issue, but is definitely a bandaid solution

stephenpapierski · 2023-06-12T18:05:38Z

I'm getting similar results on Auto-Southwest Check-In v4.1. I'm not using a config file and only passing in USERNAME/PASSWORD. I'm also running on Unraid and setting privileged helps as a temporary workaround.

pd-448482 · 2023-06-12T18:10:47Z

Confirmed that setting privileged on the container has worked around the issue.

jdholtz · 2023-06-13T22:48:07Z

That's good to know that running with a privileged container works around this issue. Since I am not able to reproduce this issue, would you guys be able to manually add some code to see if this issue is solved? If not, I can push a new image with these fixes to which you can test.

On line 207 of lib/webdriver.py, add the following lines of code (make sure the indentation is correct). You can mix and match these lines to see which ones are needed and which ones aren't.

options.add_argument("--disable-gpu")
options.add_argument("--disable-extensions")
options.add_argument("--dns-prefetch-disable")

To add this in the Docker image, you can first run the following command: docker run -d --entrypoint sleep jdholtz/auto-southwest-check-in infinity.

Then, exec into it by doing docker exec -it <container-name> sh.

You can edit the file by using vi lib/webdriver.py and then run the script inside the container by doing python southwest.py USERNAME PASSWORD.

pd-448482 · 2023-06-13T23:11:29Z

@jdholtz I tried out the instructions with a config file and verbose on. And while it does get further, it still seems to fail.

Run logs:

/app # python southwest.py --verbose
2023-06-13 23:08:13 DEBUG MainProcess[log]: Initialized the application
2023-06-13 23:08:13 DEBUG MainProcess[main]: Called with 0 arguments
2023-06-13 23:08:14 DEBUG MainProcess[config]: Reading the configuration file
2023-06-13 23:08:14 DEBUG MainProcess[config]: Adding 0 accounts from configuration file
2023-06-13 23:08:14 DEBUG MainProcess[config]: Setting check fares to True
2023-06-13 23:08:14 DEBUG MainProcess[config]: Adding 1 flights from configuration file
2023-06-13 23:08:14 DEBUG MainProcess[config]: Setting notification level to 1
2023-06-13 23:08:14 DEBUG MainProcess[config]: Using 1 notification services
2023-06-13 23:08:14 DEBUG MainProcess[config]: Setting retrieval interval to 24 hours
2023-06-13 23:08:14 DEBUG MainProcess[main]: Monitoring 0 accounts and 1 flights
2023-06-13 23:08:14 DEBUG Process-1[flight_retriever]: Scheduling reservations for 1 flights
2023-06-13 23:08:14 DEBUG Process-1[checkin_scheduler]: No headers set. Refreshing...
2023-06-13 23:08:14 DEBUG Process-1[checkin_scheduler]: Refreshing headers for current session
2023-06-13 23:08:14 DEBUG Process-1[webdriver]: Starting webdriver for current session
2023-06-13 23:08:14 DEBUG Process-1[webdriver]: Loading Southwest Check-In page
2023-06-13 23:08:44 DEBUG Process-1[webdriver]: Filling out a check-in form to get valid headers
Process Process-1:
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/local/lib/python3.11/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/app/lib/flight_retriever.py", line 38, in monitor_flights
    self._schedule_reservations(flights)
  File "/app/lib/flight_retriever.py", line 66, in _schedule_reservations
    self.checkin_scheduler.schedule(confirmation_numbers)
  File "/app/lib/checkin_scheduler.py", line 36, in schedule
    self.refresh_headers()
  File "/app/lib/checkin_scheduler.py", line 48, in refresh_headers
    webdriver.set_headers()
  File "/app/lib/webdriver.py", line 62, in set_headers
    confirmation_element = WebDriverWait(driver, 30).until(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/selenium/webdriver/support/wait.py", line 95, in until
    raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:

Given the error, is there a way to grab a screenshot on failure? I wonder if the page isn't rendering correctly with the new options.

jdholtz · 2023-06-16T15:54:51Z

Yes, you can grab a screenshot at any point with driver.save_screenshot("file_name.png"). Just place that at any point in the code where you want a screenshot of the page.

It seems like the initial timeout issue is solved, but it just doesn't load the page elements. Have you tried to use each of the new options I suggested individually? You may only need one of them to fix the original issue and this new timeout my be the cause of an option not needed.

stgreenb · 2023-06-17T15:07:13Z

same issue, --privileged fixed it.

jdholtz · 2023-06-18T14:27:02Z

Is everyone who has this issue running the Docker container on an Unraid host? pldimarco and stephenpapierski appear to be so maybe there is a conflict between this script and Unraid hosts.

Dominick-P · 2023-06-18T14:35:52Z

I’m running it on a standard Ubuntu 22.04.1 LTS installation using docker run and portainer for managing my containers

stgreenb · 2023-06-18T15:00:21Z

Ubuntu 22.04.2 LTS

jdholtz · 2023-06-25T13:47:59Z

Hey @pldimarco, any updates to my comment here?

pd-448482 · 2023-06-25T14:34:55Z

@jdholtz

I tried one additional flag at a time and no one flag worked. I didn't try the combinations of 2 flags. But three flags has gotten it to Loading Southwest Check-In page.

I wasn't able to grab a screenshot. Because while I got past the driver = self._init_driver() line, the line where it is stuck is driver.get(CHECKIN_URL). I put a debug line after that statement to confirm it wasn't returning. That's about how far I've gotten with it recently.

jdholtz · 2023-06-25T15:03:15Z

That makes sense. Thanks for trying.

Would you be able to try something else so we can see if this issue is directly with the Chromedriver or with this script?

Enter the Docker container and type python (or python3) to enter the Python terminal. Then paste the following code and see if it still times out:

from seleniumwire.undetected_chromedriver import Chrome, ChromeOptions
options = ChromeOptions()
options.add_argument("--headless=new")
driver = Chrome(driver_executable_path="/usr/bin/chromedriver", options=options)
driver.get("https://mobile.southwest.com/check-in")

If it finishes without timing out, then it is most likely due to the configuration this script uses.

pd-448482 · 2023-06-25T15:44:43Z

I put the code into a script and ran it. And it just hanged. So I put prints between each step, so my script looked like:

from seleniumwire.undetected_chromedriver import Chrome, ChromeOptions
print("1")
options = ChromeOptions()
print("2")
options.add_argument("--headless=new")
print("3")
driver = Chrome(driver_executable_path="/usr/bin/chromedriver", options=options)
print("4")
driver.get("https://mobile.southwest.com/check-in")
print("5")

And the last I saw was 3.

jdholtz · 2023-06-25T16:33:49Z

I really appreciate you taking your time to debug this. Unfortunately, this means the issue doesn’t directly have to do with this script as the issue is reproducible without using this script at all.

Currently, I’m not sure how to debug this further. You (or anyone else having this issue) can try to test the Selenium Docker Image (for Chrome) to see if you still receive a timeout. That repository has also fixed some timeout issues too, so it might be worth it to try to exec into the Southwest script image and do export DBUS_SESSION_BUS_ADDRESS=/dev/null before running the script manually (suggestion from this issue).

wootwoot1234 · 2023-07-02T17:40:08Z

I'm also having this issue running on mac os. It was working before, then I updated it and now it's not. I'm not sure how to fix it.

jdholtz · 2023-07-02T18:29:50Z

Hey @wootwoot1234, it seems like running the Docker container with the --privileged flag works around this issue. You can also try my suggestions about the Selenium Docker image in the above comment to potentially see what the problem is on a deeper level.

wootwoot1234 · 2023-07-02T19:14:58Z

@jdholtz thanks, running it as a docker image worked great except I couldn't get it to load my config file so I had to pass it the username/passwrod as ARGS.

BTW, the docs could be a little clearer. I wasn't clear to me that I was supposed to replace ARGS with my username and password. I eventually figured it out but It might be clearer if instead of ARGS you but USERNAME PASSWORD or CONFIRMATION FIRST LAST

jdholtz · 2023-07-02T19:35:23Z

Good to hear. The paragraph below the docker run command explains how to pass in your config file.

Thanks for your note on the ARGS. I will modify the docs to make it more clear.

jdholtz · 2023-07-07T23:23:17Z

Has anyone that is running into this issue tried my suggestions here?

pd-448482 · 2023-07-08T12:51:51Z

Hey @jdholtz. I went to try out your suggestions here as I was traveling for a bit and I now cannot reproduce the issue. So I disabled privileged on the container and it is working now. I did recently upgrade unraid to 6.12.2 (CC: @stephenpapierski) which may have had an effect. But I know other users were reporting the issue.

jdholtz · 2023-07-08T13:12:29Z

That’s good to hear but strange why this issue has unknowingly disappeared. Thank you for all the debugging you’ve done for this issue!

jdholtz · 2023-07-23T19:50:37Z

Since this issue is resolved for the author and no one else has reported back that the issue still occurs for them after trying my suggestions above, I will close this.

If you are still running into this issue (even after my suggestions), please either comment on this issue or open a separate issue.

txssseal · 2023-08-03T02:57:35Z

Im running into this issue doing a direct install on Ubuntu 22.04.

Process Process-1:
Traceback (most recent call last):
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/webdriver.py", line 164, in _init_driver
    driver = Chrome(
  File "/home/creeks-edge/.local/lib/python3.10/site-packages/seleniumwire/undetected_chromedriver/webdriver.py", line 61, in __init__
    super().__init__(*args, **kwargs)
  File "/home/creeks-edge/.local/lib/python3.10/site-packages/undetected_chromedriver/__init__.py", line 409, in __init__
    browser = subprocess.Popen(
  File "/usr/lib/python3.10/subprocess.py", line 971, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/usr/lib/python3.10/subprocess.py", line 1738, in _execute_child
    and os.path.dirname(executable)
  File "/usr/lib/python3.10/posixpath.py", line 152, in dirname
    p = os.fspath(p)
TypeError: expected str, bytes or os.PathLike object, not NoneType

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap
    self.run()
  File "/usr/lib/python3.10/multiprocessing/process.py", line 108, in run
    self._target(*self._args, **self._kwargs)
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/reservation_monitor.py", line 42, in monitor
    self.checkin_scheduler.refresh_headers()
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/checkin_scheduler.py", line 52, in refresh_headers
    webdriver.set_headers()
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/webdriver.py", line 59, in set_headers
    driver = self._get_driver()
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/webdriver.py", line 139, in _get_driver
    driver = self._init_driver()
  File "/home/creeks-edge/code/auto-southwest-check-in/lib/webdriver.py", line 181, in _init_driver
    raise RuntimeError(
RuntimeError: Failed to initialize the webdriver after 3 attempts

jdholtz · 2023-08-03T03:29:32Z

@txssseal do you have a Chromium browser installed (such as Google Chrome)?

If so, you can try adding the absolute path of your chromedriver location to the config.json option chromedriver_path.

jonoff · 2023-08-17T03:19:54Z

That makes sense. Thanks for trying.

Would you be able to try something else so we can see if this issue is directly with the Chromedriver or with this script?

Enter the Docker container and type python (or python3) to enter the Python terminal. Then paste the following code and see if it still times out:
from seleniumwire.undetected_chromedriver import Chrome, ChromeOptions
options = ChromeOptions()
options.add_argument("--headless=new")
driver = Chrome(driver_executable_path="/usr/bin/chromedriver", options=options)
driver.get("https://mobile.southwest.com/check-in")
If it finishes without timing out, then it is most likely due to the configuration this script uses.

I was able to run the above, but I think you're missing the line that times out,
If I add

confirmation_element = WebDriverWait(driver, 30).until(
           EC.presence_of_element_located((By.NAME, "recordLocator"))
       )

I get the timeout again, and looking at the response, it seems the headers are being rejected, so that element is never found:

'<html><head>\n <title>502 Bad Gateway</title>\n </head>\n <body>\n <h1>502 Bad Gateway</h1>\n <p>ProtocolException(\'Server connection to (\\\'mobile.southwest.com\\\', 443) failed: Error connecting to "mobile.southwest.com": [Errno -3] Try again\')</p>\n \n </body></html>'

jdholtz · 2023-08-17T03:30:30Z

I was able to run the above, but I think you're missing the line that times out,

This timeout happens at different places for everyone affected by the issue (the OP timed out when initializing Chrome, for example).

I'm not able to reproduce this, so it appears to be a network issue that people run into every so often (maybe Southwest has something to do with this, but I'm doubtful). For reference, I am using Chrome v116 (with the workaround in the pinned issue), but I don't know what else could be causing this. It appears to go away for people who run into this issue eventually because no one has reported that they are still running into the timeout.

jonoff · 2023-08-17T03:41:42Z

Good point, for me it just happens to be the 502 error. If you have more debugging ideas, or logs I can capture I can help debug this. I have set my chrome to v114 but will try 116 after reading the workaround.

jdholtz · 2023-08-17T03:46:07Z

It could be a selenium-wire issue. You can try to replace the import statement with from undetected_chromedriver import Chrome, ChromeOptions and see if you still get the timeout.

jonoff · 2023-08-17T04:14:00Z

Thanks, that led me on the right path as the error message was more useful and raised up. Turns out it was higher up the stack and a docker network/DNS resolution issue.

stgreenb · 2023-12-20T21:15:07Z

@jdholtz , Just thought I'd mention that now a few months later (and doing a pull of the latest container), running it still requires --privileged to work.

jdholtz · 2023-12-21T00:21:07Z

running it still requires --privileged to work.

Thanks for the update. I still haven’t run into this issue at all. What OS are you running?

stgreenb · 2023-12-21T00:30:22Z

Ubuntu Server 22.04.3 LTS

…

On Wed, Dec 20, 2023, 7:21 PM Joey Holtzman ***@***.***> wrote: running it still requires --privileged to work. Thanks for the update. I still haven’t run into this issue at all. What OS are you running? — Reply to this email directly, view it on GitHub <#96 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AENAV7DPLFR5C5GXS2SOCQTYKN6H3AVCNFSM6AAAAAAYV5R542VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNRVGMYDONZTGU> . You are receiving this because you commented.Message ID: ***@***.***>

pd-448482 added the bug Something isn't working label May 31, 2023

jdholtz closed this as completed Jul 23, 2023

jdholtz mentioned this issue Sep 11, 2023

Webdriver timeout in container #147

Closed

drippyer mentioned this issue Mar 6, 2024

Forbidden 403 #230

Closed

jdholtz mentioned this issue Aug 14, 2024

Webdriver times out #287

Closed

Webdriver timed out in container #96

Webdriver timed out in container #96

Comments

pd-448482 commented May 31, 2023

Description

To Reproduce

Expected Behavior

Version

Additional context

jdholtz commented May 31, 2023

pd-448482 commented May 31, 2023

jdholtz commented May 31, 2023

jdholtz commented Jun 1, 2023

pd-448482 commented Jun 1, 2023

jdholtz commented Jun 5, 2023

pd-448482 commented Jun 6, 2023

Dominick-P commented Jun 12, 2023 • edited Loading

Dominick-P commented Jun 12, 2023

stephenpapierski commented Jun 12, 2023

pd-448482 commented Jun 12, 2023

jdholtz commented Jun 13, 2023

pd-448482 commented Jun 13, 2023

jdholtz commented Jun 16, 2023

stgreenb commented Jun 17, 2023

jdholtz commented Jun 18, 2023

Dominick-P commented Jun 18, 2023

stgreenb commented Jun 18, 2023

jdholtz commented Jun 25, 2023 • edited Loading

pd-448482 commented Jun 25, 2023

jdholtz commented Jun 25, 2023

pd-448482 commented Jun 25, 2023

jdholtz commented Jun 25, 2023 • edited Loading

wootwoot1234 commented Jul 2, 2023

jdholtz commented Jul 2, 2023

wootwoot1234 commented Jul 2, 2023

jdholtz commented Jul 2, 2023

jdholtz commented Jul 7, 2023

pd-448482 commented Jul 8, 2023

jdholtz commented Jul 8, 2023

jdholtz commented Jul 23, 2023

txssseal commented Aug 3, 2023 • edited Loading

jdholtz commented Aug 3, 2023 • edited Loading

jonoff commented Aug 17, 2023

jdholtz commented Aug 17, 2023

jonoff commented Aug 17, 2023

jdholtz commented Aug 17, 2023

jonoff commented Aug 17, 2023

stgreenb commented Dec 20, 2023

jdholtz commented Dec 21, 2023

stgreenb commented Dec 21, 2023 via email

Dominick-P commented Jun 12, 2023 •

edited

Loading

jdholtz commented Jun 25, 2023 •

edited

Loading

jdholtz commented Jun 25, 2023 •

edited

Loading

txssseal commented Aug 3, 2023 •

edited

Loading

jdholtz commented Aug 3, 2023 •

edited

Loading