Skip to content

emiliolehoucq/web_scraping_tool

Repository files navigation

This repo hosts a tool for scraping URLs added to a Google Sheet. Researchers can conveniently add URLs to a Google Sheet. The tool uses GitHub Actions, Google Sheets API, and Google Drive API to regularly get the URLs added to the Google Sheet, check if they're already scraped, and, if not, scrape them and store the source code and the extracted text in a Google Drive folder. As such, this tool makes it easy and fast for researchers to automatically request a website to be scraped. I built this tool for a project as part of my work at Research Computing and Data Services.

About

Tool for scraping websites added to a Google Sheet

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages