Document how the new GCP pipeline works

[rviscomi]

We could use the Wiki section of this repo to maintain the docs. It should document how the pipeline works and act as a playbook to handle unexpected issues.

wishlist

• playbook (e.g. how to start the crawl)
• expand on open issues / edge cases (e.g. what happens when a table is deleted mid crawl)
• go back through history and document issues encountered with Beam Python SDK

[giancarloaf]

@tunetheweb heads up, I would love to get some fresh eyes on the process to run the pipeline and improve the documentation to cover everything someone new might need. I'll reach out again once we get closer to "completing" the pipeline so we can work on this together and you can poke holes in everything!

[max-ostapenko]

Closing it as this pipeline was deprecated.
See the new pipeline and docs here: https://github.com/HTTPArchive/dataform?tab=readme-ov-file