Migrate to Cloud Run

[simonw]

Zeit v1 is being turned off in August.

[simonw]

I'm going to switch from Circle CI to GitHub Actions at the same time.

Will base this on https://github.com/simonw/covid-19-datasette/blob/master/.github/workflows/scheduled.yml

[simonw]

I added the GCP_SA_KEY and GCP_SA_EMAIL secrets to this repository.

[simonw]

Here's the Circle CI script I'm replacing: https://github.com/simonw/museums/blob/c0ac71b58b1c958233cfddada8ce6b01b2bfdcb2/.circleci/config.yml

[simonw]

OK, site is deployed at https://niche-museums-j7hipcg4aq-uc.a.run.app/ - but it's showing a 500 error on the index page right now:

The data is there: https://niche-museums-j7hipcg4aq-uc.a.run.app/browse

And it looks like it has the correct list of plugins: https://niche-museums-j7hipcg4aq-uc.a.run.app/-/plugins

[simonw]

I get the same error in my local development environment. Here's the traceback:

  File "/Users/simonw/.local/share/virtualenvs/museums-AcCIO0uJ/lib/python3.7/site-packages/jinja2/asyncsupport.py", line 177, in auto_await
    return await value
  File "/Users/simonw/.local/share/virtualenvs/museums-AcCIO0uJ/lib/python3.7/site-packages/datasette_template_sql/__init__.py", line 8, in execute_sql
    return (await datasette.execute(dbname, sql)).rows
  File "/Users/simonw/.local/share/virtualenvs/museums-AcCIO0uJ/lib/python3.7/site-packages/datasette/app.py", line 350, in execute
    log_sql_errors=log_sql_errors,
  File "/Users/simonw/.local/share/virtualenvs/museums-AcCIO0uJ/lib/python3.7/site-packages/datasette/database.py", line 165, in execute
    with trace("sql", database=self.name, sql=sql.strip(), params=params):
jinja2.exceptions.UndefinedError: 'query' is undefined

[simonw]

The problem occurs here on the index page:

museums/templates/index.html

Lines 76 to 79 in 6422909

	{% set show_links = 1 %}
	{% for museum in sql(query) %}
	{% include "_museum_card.html" %}
	{% endfor %}

The query variable here is empty. It should have been set here:

museums/plugins/index_vars.py

Lines 34 to 43 in 7520afa

	query = f"select {select} from museums {where} order by {order_by}"
	vars.update(
	{
	"query": query,
	"should_next_query": f"select (count(*) - {PAGE_SIZE}) > 0 from museums {where}",
	"next_query": f"select min(id) from ({query})",
	"is_distance_page": is_distance_page,
	}
	)

[simonw]

It looks like the bug originates here: view_name is None when it should be index:

museums/plugins/index_vars.py

Lines 7 to 10 in 7520afa

	def extra_template_vars(request, view_name):
	vars = {"q": request.raw_args.get("q") or ""}
	if view_name == "index":
	# Custom template variables for the homepage

[simonw]

The working version at https://www.niche-museums.com/-/versions reports that the Datasette version is "version": "0+unknown" - and the old Circle CI code was deploying the master branch - I forget why:

museums/.circleci/config.yml

Line 36 in c0ac71b

--branch=master

Looks like I started deploying --master back in November but the commit message doesn't give a reason: a02cb02

[simonw]

My last successful deploy of the old site was c0ac71b on 2nd Feb. At the time Datasette master would have been https://github.com/simonw/datasette/tree/286ed286b68793532c2a38436a08343b45cfbc91

[simonw]

Deploying today's master ( https://github.com/simonw/datasette/tree/e0e7a0facfc935a835cd73c720bc46661462f0b1 ) resulted in the same 500 error. I'm going to see if 286ed286b68793532c2a38436a08343b45cfbc91 works.

I'll test that locally first with this:

pip install https://github.com/simonw/datasette/archive/286ed286b68793532c2a38436a08343b45cfbc91.zip

[simonw]

That fixes it! I'll use this as a temporary fix here, then investigate the underlying Datasette bug.

[simonw]

Next step: map www.niche-museums.com domain to the new Cloud Run instance.

[simonw]

Gonna have to verify that domain with Google first:

[simonw]

For verification I'm following the same steps as here: simonw/datasette#705 (comment)

DNS is hosted by Zeit so I need to run:

now dns add niche-museums.com '@' TXT google-site-verification=98XbgwOi7RdJuO2P6pbEzDwdTJFSVouG0jyRfg2ROnw

[simonw]

Bit of a confusing error message from Google which seems to contradict itself:

Oh! It looks like I bought niche-museums.com through Google Domains which should mean it's verified already.

[simonw]

In that case I'm going to switch it back to using Google nameservers rather than pointing it at Zeit:

[simonw]

Urgh:

[simonw]

I'm going to try setting the CNAME on both Zeit DNS AND Google DNS, to hopefully workaround the potential 48 hour delay.

[simonw]

Ran this:

now dns add niche-museums.com www CNAME ghs.googlehosted.com.

[simonw]

I'm going to make a change that's only visible in the Cloud Run instance so I can tell in my browser if I am seeing the site hosted on Zeit Now or on Cloud Run.

[simonw]

I made the same DNS change on Google Domains: https://domains.google.com/m/registrar/niche-museums.com/dns

[simonw]

Looks like I've broken https://www.niche-museums.com/ - it's returning a TLS error at the moment.

[simonw]

https://niche-museums-j7hipcg4aq-uc.a.run.app/-/metadata now shows the "title" as "www.niche-museums.com"

As soon as https://www.niche-museums.com/-/metadata shows the same thing I can close this ticket.

[simonw]

https://www.sslshopper.com/ssl-checker.html#hostname=www.niche-museums.com says:

[simonw]

[simonw]

So it's working! Just started working on my laptop too.

[simonw]

https://niche-museums.com/ doesn't work. It should redirect to the correct site.

I'm going to try to set that up using Google Domains, in 24 hours time once everything has settled a bit: https://domains.google.com/m/registrar/niche-museums.com/dns

Instructions here: https://www.tachyonstemplates.com/2018/google-domains-forward-root/

[simonw]

Actually I'm going to try pointing https://niche-museums.com/ to the same Cloud Run app, then add some Datasette custom ASGI middleware to handle the no-www to www redirect.

[simonw]

Can't CNAME the root:

[simonw]

In the Cloud Run mapping:

Then on the next page it tells me to setup A and AAAA records:

[simonw]

But... the Google Domains interface doesn't seem to let me setup more than one A record:

[simonw]

Oh, it does, but you have to click that little "+" button:

[simonw]

OK, DNS now looks like this:

Already seeing them on my laptop using dig:

~ $ dig niche-museums.com

; <<>> DiG 9.10.6 <<>> niche-museums.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43118
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1280
;; QUESTION SECTION:
;niche-museums.com.		IN	A

;; ANSWER SECTION:
niche-museums.com.	3600	IN	A	216.239.38.21
niche-museums.com.	3600	IN	A	216.239.36.21
niche-museums.com.	3600	IN	A	216.239.32.21
niche-museums.com.	3600	IN	A	216.239.34.21

;; Query time: 57 msec
;; SERVER: 10.0.0.1#53(10.0.0.1)
;; WHEN: Sat Apr 04 18:15:37 PDT 2020
;; MSG SIZE  rcvd: 110

[simonw]

Still not working, I'll give it a few minutes for certificates to settle down etc.

[simonw]

OK https://niche-museums.com/? is working for me now but not redirecting to www.

[simonw]

Moving that to a separate issue: #21