This repo hosts a tool for scraping URLs added to a Google Sheet. Researchers can conveniently add URLs to a Google Sheet. The tool uses GitHub Actions, Google Sheets API, and Google Drive API to regularly get the URLs added to the Google Sheet, check if they're already scraped, and, if not, scrape them and store the source code and the extracted text in a Google Drive folder. As such, this tool makes it easy and fast for researchers to automatically request a website to be scraped. I built this tool for a project as part of my work at Research Computing and Data Services.
-
Notifications
You must be signed in to change notification settings - Fork 0
emiliolehoucq/web_scraping_tool
Folders and files
Name | Name | Last commit message | Last commit date | |
---|---|---|---|---|
Repository files navigation
About
Tool for scraping websites added to a Google Sheet
Resources
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published