chore: single top level dev .env file and quicker quickstart

[olizilla]

Use a single top level .env flle to set all dev env vars. With this PR getting started with the api and website is just

• create a magic.link account
• copy .env.tpl to .env in the monorepo root and fill out the magic link keys
• npm i
• npm start

and you'll be off to the races with the api and website 🏇 💨 ✨

This PR includes:

• create a new .env.tpl template that aggregates all the module env needs.
• add top level npm start task to run the api and the website, for a quicker quickstart.
• remove old FAUNA logic from pinpin so we don't have to set the DATABASE var any more.
• Updated README's

TODO:

• warn people that they need to copy the env template and fill out their magic link keys, and maybe copy their SALT over.
• fix it so that the api start command does't fail if you are already running the cluster and db containers.

License: (Apache-2.0 AND MIT)
Signed-off-by: Oli Evans [email protected]

[olizilla]

one command to bind them

❯ npm start                                                                                                   16:02:01

> [email protected] start
> npm-run-all --print-label -p web api

[web] 
[web] > [email protected] web
[web] > npm start -w packages/website
[web] 
[api] 
[api] > [email protected] api
[api] > npm start -w packages/api
[api] 
[web] 
[web] > @web3-storage/[email protected] start
[web] > env-cmd -f ../../.env next dev -p 4000
[web] 
[api] 
[api] > @web3-storage/[email protected] start
[api] > npm-run-all -p cluster:start pg:start -s miniflare
[api] 
[web] ready - started server on 0.0.0.0:4000, url: http://localhost:4000
[web] info  - Loaded env from /Users/oli/Code/web3-storage/web3.storage/packages/website/.env.local
[web] info  - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
[api] 
[api] > @web3-storage/[email protected] cluster:start
[api] > web3-storage-tools cluster --start
[api] 
[api] 
[api] > @web3-storage/[email protected] pg:start
[api] > cd ../db && npm start
[api] 
[api] 
[api] > @web3-storage/[email protected] start
[api] > node scripts/cli.js db --start --project web3-storage
[api] 
[web] Defining routes from exportPathMap
[web] event - compiled successfully
[api] 
[api] > @web3-storage/[email protected] miniflare
[api] > miniflare --env ../../.env --watch
[api] 
[api] 
[api] > @web3-storage/[email protected] build
[api] > node scripts/cli.js build
[api] 
[api] ▲ [WARNING] Non-relative path "packages/blob/src/lib.js" is not allowed when "baseUrl" is not set (did you forget a leading "./"?)
[api] 
[api]     ../../node_modules/ipfs-car/node_modules/@web-std/blob/tsconfig.json:6:24:
[api]       6 │       "@web-std/blob": ["packages/blob/src/lib.js"]
[api]         ╵                         ~~~~~~~~~~~~~~~~~~~~~~~~~~
[api] 
[api] [mf:inf] Build succeeded
[api] [mf:inf] Worker reloaded! (2.15MiB)
[api] [mf:wrn] Worker's uncompressed size exceeds the 1MiB limit! Note that your worker will be compressed during upload so you may still be able to deploy it.
[api] [mf:inf] Listening on :8787
[api] [mf:inf] - http://127.0.0.1:8787
[api] [mf:inf] - http://192.168.1.26:8787

[olizilla]

cron pins still blends

❯ DEBUG="*" npm run start:pins -w packages/cron

> @web3-storage/[email protected] start:pins
> node src/bin/pins.js

  pins:updatePinStatuses 📥 Processing 0 -> 2 +0ms
  pins:updatePinStatuses ⏳ bafkreiabltrd5zm73pvi7plq25pef3hm7jxhbi3kv4hapegrkfpkqtkbme: Checking status... +1ms
  pins:updatePinStatuses ⏳ bafybeidmahqfk7jthbnaspso6skywa7fcqokswrxiw22nzjgnebqkqmrqy: Checking status... +3ms
  pins:updatePinStatuses 📌 bafybeidmahqfk7jthbnaspso6skywa7fcqokswrxiw22nzjgnebqkqmrqy@12D3KooWEuBKsspmFCi9Bpub7LX21NC4YJh8QoZZsqveuxPXubQU: Pinning => Pinned +12ms
  pins:updatePinStatuses 📌 bafkreiabltrd5zm73pvi7plq25pef3hm7jxhbi3kv4hapegrkfpkqtkbme@12D3KooWEuBKsspmFCi9Bpub7LX21NC4YJh8QoZZsqveuxPXubQU: Pinning => Pinned +1ms
  pins:updatePinStatuses ⏳ Updating 2 pins... +0ms
  pins:updatePinStatuses ✅ Updated 2 pins... +10ms
  pins:updatePinStatuses ⏳ Removing 2 pin sync requests... +0ms
  pins:updatePinStatuses ✅ Removed 2 pin sync requests... +10ms
  pins:updatePinStatuses ⏳ Re-queuing 0 pin sync requests... +0ms
  pins:updatePinStatuses ✅ Re-queued 0 pin sync requests... +1ms
  pins:updatePinStatuses 🎉 Done +0ms

[olizilla]

pinpin still blends

❯ DEBUG="*" npm run start -w packages/pinpin

> @web3-storage/[email protected] start
> DEBUG_HIDE_DATE=true DEBUG=pinpin ./src/bin.js

  pinpin 📡 Fetcing Pin Requests from DB +0ms
  pinpin 📥 Processing 3 Pin Requests +35ms
  pinpin 📌 bafkreiabltrd5zm73pvi7plq25pef3hm7jxhbi3kv4hapegrkfpkqtkbme 1/3 pinned on Pinata! Request created: 2022-02-22T15:25:12.062+00:00 +835ms
  pinpin 📌 bafybeidmahqfk7jthbnaspso6skywa7fcqokswrxiw22nzjgnebqkqmrqy 2/3 pinned on Pinata! Request created: 2022-02-22T15:25:41.154+00:00 +17ms
  pinpin 📌 bafybeidt4xhqaizqq2bvferllxcdrk7sfkry72etwbuacfkyqsq3bheody 3/3 pinned on Pinata! Request created: 2022-02-11T12:13:20.299+00:00 +11ms
  pinpin 📡 Deleting 3 processed Pin Requests +0ms
  pinpin 🎉 Done! Pinned 3 of 3 +14ms
  pinpin 📡 Fetcing Pin Requests from DB +1ms
  pinpin 📥 Processing 0 Pin Requests +10ms
  pinpin 📡 Deleting 0 processed Pin Requests +0ms
  pinpin 🎉 Done! Pinned 0 of 0 +11ms
  pinpin Waiting 10s before checking for new Pin Requests +0ms
  pinpin 📡 Fetcing Pin Requests from DB +10s
  pinpin 📥 Processing 0 Pin Requests +11ms
  pinpin 📡 Deleting 0 processed Pin Requests +0ms
  pinpin 🎉 Done! Pinned 0 of 0 +10ms
  pinpin Waiting 10s before checking for new Pin Requests +10s

[vasco-santos]

LGTM ❤️