Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle venue website timeouts gracefully #14

Open
joshuayoes opened this issue Apr 19, 2022 · 2 comments
Open

Handle venue website timeouts gracefully #14

joshuayoes opened this issue Apr 19, 2022 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@joshuayoes
Copy link
Owner

The most common error in Github Actions is that the venue website times out when we want to scrape it:

There are a few possible reasons:

  • The website provider is timing out our connection because it knows that it is not typical user browser traffic
  • The GitHub Action runner has poor network connectivity (unlikely after many re-tries)
@joshuayoes
Copy link
Owner Author

Possible solution: create a cache of past successful scrapping results, and provide it if venue website is unavailable

@joshuayoes joshuayoes self-assigned this Apr 19, 2022
@joshuayoes joshuayoes added the enhancement New feature or request label Apr 19, 2022
@joshuayoes
Copy link
Owner Author

joshuayoes commented Apr 19, 2022

Database options:

Important:

  • Free
  • Easy way to view and manage data via UI
  • Useful query methods for finding latest venue cache value
  • Able to host locally?

Not Important:

  • Scalability
  • High performance

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant