[p5 web editor - examples] Updater Script

[joeyklee]

→ Step 1: Describe the issue 📝

Did you find a bug? Want to suggest an idea for feature?

• devOps

With the merging of the ml5-examples repo to the ml5-library, we will need to update the script that automatically updates our examples sketches in the p5 web editor.

Right now our batch update script is run by @catarak directly on the web editor server. The script finds all of our sketches using the Github API and updates all of the content but as of now it currently points to the ml5-library repo and not our current ml5-library/examples directory.

The script in question is this one here: https://github.com/ml5js/ml5-examples/blob/release/scripts/batch-update-p5webeditor.js

Current breaking changes

• The current implementation of the updater script traverses the ml5-examples repo an collects the p5 examples to batch upload to the p5 web editor. In updated the script to point to the ml5-examples the changes work as expected, however since we "build" our examples in our Netlify deployment and not stored here in Github, the issue arises that our ml5 library points to localhost and not the ml5@latest cdn URL. @bomanimc - Any thoughts on this would be superb!
  • Some possible solutions:
    • 1. regex replace the ml5 url in the updater script to enforce @latest ml5 lib cdn in the step when we download files. The upside, we don't add more files to our existing ml5-library repo. The downside is that we have to go in and add more logic to the batch upload script but probably better than storing both the dist_examples and the examples
    • 2. include the dist_examples in the ml5-library repo. The upside is that this is a very simple solution. The downside here is that we add a whole bunch of files to the ml5-library repo.
    • 3. Open to suggestions!

Additional notes

• Github will be deprecating the auth_token as a URL parameter, so we will need to update the request style according to the notes given here: https://developer.github.com/changes/2020-02-10-deprecating-auth-through-query-param/

[joeyklee]

Just making a note that I've started a draft here: processing/p5.js-web-editor#1627

[bomanimc]

Another option I wonder about: Could we download the built assets directly from Netlify through an API? Netlify does allow manual downloads of deployed assets. If we could grab it from the script, we'd be able to get all the files we need without having to make any changes.

Opened a question on Netlify here: https://community.netlify.com/t/using-the-api-or-cli-to-download-a-deployed-build/25342

[bomanimc]

Another is that we put together a storage bucket on S3, GCP, or elsewhere and add a script to ml5 to push the example builds to, and then retrieve them from this storage source in the server-side script in the web editor. We'd probably need to pay a bit of money for this, but I imagine we'd only need it until the bulk upload API is available on the p5 web editor side.

[catarak]

@bomanimc are you referring to text assets (js, css, html) or other assets? It honestly might give you some flexibility to host all of the other assets separately and then make theme CORS enabled with the web editor. Then you won't need to re-run the update script in order to update these assets.

[bomanimc]

@catarak referencing the text assets (the html and JS files required for the examples). I agree with that strategy for the other assets though for sure. Do you have any suggestions for the text assets as well?

[catarak]

For those I would upload to the web editor using the API (which I know is not yet public :/, but will be in the near future). For any updates and testing, you can use the staging editor (which I can send you when I have it functioning)

[catarak]

The staging editor is now up at https://stagingeditor.p5js.org (password protected) if you'd like to test any changes to the updater script! Let me know and I'll email you the login.

[bomanimc]

Thanks @catarak! I will take a look at this as soon as possible. May you please email the login to [email protected]?

[bomanimc]

It looks like we've adddressed the scope of this issue! I've opened another issue on the p5.js web editor repo to share another idea about an interim process for updating the examples: processing/p5.js-web-editor#1773