From 7ef48e78ab8544c542ffacf6fab09553a0ae03b7 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 3 Sep 2020 21:57:49 +0300 Subject: [PATCH 1/5] Add workshop agenda --- agenda/README.md | 43 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) create mode 100644 agenda/README.md diff --git a/agenda/README.md b/agenda/README.md new file mode 100644 index 0000000..f3763a7 --- /dev/null +++ b/agenda/README.md @@ -0,0 +1,43 @@ +# W3C Workshop on Web and Machine Learning + +![Agenda header image][header] + +# Agenda for Live Sessions + +## September 16, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200916T14) + +>πŸ”Ž **Scope:** Opportunities and Challenges of Browser-Based Machine Learning +> +>βœ… **Goal:** Determine what are the unique opportunities of browser-based ML, what are the obstacles hindering adoption + +- πŸ‘‹ Introductions +- [πŸ’‘ "Opportunities and Challenges" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Opportunities+and+Challenges%22+sort%3Acomments-desc) + + +## September 22, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200922T14) + +>πŸ”Ž **Scope:** Web Platform Foundations for Machine Learning +> +>βœ… **Goal:** Goal: Understand how machine learning fits into the Web technology stack + +- [πŸ’‘ "Web Platform Foundations" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Web+Platform+Foundations%22+sort%3Acomments-desc) + +## September 23, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200923T14) + +>πŸ”Ž **Scope:** Machine Learning Experiences on the Web: A **Developer’s** Perspective +> +>βœ… **Goal:** Authoring ML experiences on the Web; challenges and opportunities of reusing existing ML models on the Web; on-device training, known technical solutions, gaps + +- [πŸ’‘ "Developer's Perspective" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Developer%27s+Perspective%22+sort%3Acomments-desc) + +## September 29, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200929T14) + +>πŸ”Ž **Scope:** Machine Learning Experiences on the Web: A **User’s** Perspective +> +>βœ… **Goal:** Goal: Web & ML for all: education, learning, accessibility, cross-industry experiences, cross-disciplinary ML: music, art, and media meet ML; Share learnings and best practices across industries + +- [πŸ’‘ "User's Perspective" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22User%27s+Perspective%22+sort%3Acomments-desc) +- πŸ‘ Conclusions & Next Steps + + +[header]: https://www.w3.org/comm/assets/graphics/web-ML/workshop-bg.png "W3C Workshop on Web and Machine Learning header" From 74381d70231a565288e5d622e810dc86fdf2046d Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Fri, 4 Sep 2020 11:37:25 +0300 Subject: [PATCH 2/5] Adjust introduction scope --- agenda/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/agenda/README.md b/agenda/README.md index f3763a7..302231f 100644 --- a/agenda/README.md +++ b/agenda/README.md @@ -10,7 +10,7 @@ > >βœ… **Goal:** Determine what are the unique opportunities of browser-based ML, what are the obstacles hindering adoption -- πŸ‘‹ Introductions +- πŸ‘‹ Introduction to the workshop - [πŸ’‘ "Opportunities and Challenges" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Opportunities+and+Challenges%22+sort%3Acomments-desc) @@ -18,7 +18,7 @@ >πŸ”Ž **Scope:** Web Platform Foundations for Machine Learning > ->βœ… **Goal:** Goal: Understand how machine learning fits into the Web technology stack +>βœ… **Goal:** Understand how machine learning fits into the Web technology stack - [πŸ’‘ "Web Platform Foundations" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Web+Platform+Foundations%22+sort%3Acomments-desc) @@ -34,7 +34,7 @@ >πŸ”Ž **Scope:** Machine Learning Experiences on the Web: A **User’s** Perspective > ->βœ… **Goal:** Goal: Web & ML for all: education, learning, accessibility, cross-industry experiences, cross-disciplinary ML: music, art, and media meet ML; Share learnings and best practices across industries +>βœ… **Goal:** Web & ML for all: education, learning, accessibility, cross-industry experiences, cross-disciplinary ML: music, art, and media meet ML; Share learnings and best practices across industries - [πŸ’‘ "User's Perspective" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22User%27s+Perspective%22+sort%3Acomments-desc) - πŸ‘ Conclusions & Next Steps From dc494a9feab22b055de26908fc018a1cc9b28a55 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Mon, 7 Sep 2020 22:11:30 +0300 Subject: [PATCH 3/5] Add "Opportunities and Challenges" discussion outline --- agenda/README.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/agenda/README.md b/agenda/README.md index 302231f..2e4aeb4 100644 --- a/agenda/README.md +++ b/agenda/README.md @@ -10,9 +10,33 @@ > >βœ… **Goal:** Determine what are the unique opportunities of browser-based ML, what are the obstacles hindering adoption -- πŸ‘‹ Introduction to the workshop -- [πŸ’‘ "Opportunities and Challenges" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Opportunities+and+Challenges%22+sort%3Acomments-desc) +πŸ‘‹ Introduction to the workshop - @anssiko @dontcallmedom +[πŸ’‘ "Opportunities and Challenges" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Opportunities+and+Challenges%22+sort%3Acomments-desc): + +ℹ️ [WebGPU fitness for ML frameworks #66](https://github.com/w3c/machine-learning-workshop/issues/66) - @jasonmayes @Kangz @grorg +- ❓ Does WebGPU expose the right API surface to support ML frameworks interactions with GPUs? +- βœ”οΈ Proposal: New WebGPU extensions for subgroups, cooperative matrix multiply. + +ℹ️ [Applicability to non-browser JS environments #62](https://github.com/w3c/machine-learning-workshop/issues/62) - @jasonmayes @phoddie @huningxin @WenheLI +- ❓ Pay attention to the applicability of the browser-targeted work to non-browser JS environments, in particular Node.js. +- βœ”οΈ Proposal: Extend W3C coordination to TC53 and non-browser projects. + +ℹ️ [Protecting ML models #67](https://github.com/w3c/machine-learning-workshop/issues/67) - @jasonmayes @tidoust @pyu10055 @jbingham +- ❓ Some ML providers need to ensure their ML models cannot be extracted from a browser app. +- βœ”οΈ Proposal: Investigate existing access control mechanisms for video, learnings from 3D assets. + +ℹ️ [Support for Float16 in JS & Wasm environments #64](https://github.com/w3c/machine-learning-workshop/issues/64) - @cynthia @jasonmayes +- ❓ Lack of support for float16 in JS and Wasm environments problematic for quantized models. +- βœ”οΈ Proposal: TBD + +ℹ️ [In-browser training #82](https://github.com/w3c/machine-learning-workshop/issues/82) and [Training across devices #83](https://github.com/w3c/machine-learning-workshop/issues/83) - @irealva @cynthia +- ❓ The current in-browser efforts are focused on inference rather than training. +- βœ”οΈ Proposal: Understand successful real-world usages (e.g. Teachable Machine) and target transfer learning as the initial training use case for related browser API work. + +ℹ️ [Permission model for Machine Learning APIs #72](https://github.com/w3c/machine-learning-workshop/issues/72) - @cynthia @dontcallmedom @anssiko +- ❓ How to design a forward-looking permission model for ML APIs? +- βœ”οΈ Proposal: TBD ## September 22, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200922T14) From 3912ae9406d395c864e9d7864b454cdb841ef044 Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Tue, 8 Sep 2020 17:08:25 +0300 Subject: [PATCH 4/5] Add memory copies to "Opportunities and Challenges" --- agenda/README.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/agenda/README.md b/agenda/README.md index 2e4aeb4..6bdb458 100644 --- a/agenda/README.md +++ b/agenda/README.md @@ -34,6 +34,10 @@ - ❓ The current in-browser efforts are focused on inference rather than training. - βœ”οΈ Proposal: Understand successful real-world usages (e.g. Teachable Machine) and target transfer learning as the initial training use case for related browser API work. +ℹ️ [Memory copies #93](https://github.com/w3c/machine-learning-workshop/issues/93) - @aboba @wchao1115 +- ❓ Machine learning apps within the browser using the media pipeline trigger many more memory copies compared with native applications hindering performance. +- βœ”οΈ Proposal: Introduce a more direct way to feed a video frame, possibly captured from a camera, to a ML model. + ℹ️ [Permission model for Machine Learning APIs #72](https://github.com/w3c/machine-learning-workshop/issues/72) - @cynthia @dontcallmedom @anssiko - ❓ How to design a forward-looking permission model for ML APIs? - βœ”οΈ Proposal: TBD From 10b61332165edab39413df97ec132c1271530b2d Mon Sep 17 00:00:00 2001 From: Anssi Kostiainen Date: Thu, 10 Sep 2020 10:25:21 +0300 Subject: [PATCH 5/5] Add topical organization for Sep 16 --- agenda/README.md | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/agenda/README.md b/agenda/README.md index 6bdb458..8957c18 100644 --- a/agenda/README.md +++ b/agenda/README.md @@ -14,34 +14,48 @@ [πŸ’‘ "Opportunities and Challenges" discussion topics](https://github.com/w3c/machine-learning-workshop/issues?q=is%3Aissue+is%3Aopen+label%3A%22Opportunities+and+Challenges%22+sort%3Acomments-desc): +## Improving existing web platform capabilities + ℹ️ [WebGPU fitness for ML frameworks #66](https://github.com/w3c/machine-learning-workshop/issues/66) - @jasonmayes @Kangz @grorg - ❓ Does WebGPU expose the right API surface to support ML frameworks interactions with GPUs? - βœ”οΈ Proposal: New WebGPU extensions for subgroups, cooperative matrix multiply. +ℹ️ [Support for Float16 in JS & Wasm environments #64](https://github.com/w3c/machine-learning-workshop/issues/64) - @cynthia @jasonmayes +- ❓ Lack of support for float16 in JS and Wasm environments problematic for quantized models. +- βœ”οΈ Proposal: TBD + +ℹ️ [Memory copies #93](https://github.com/w3c/machine-learning-workshop/issues/93) - @aboba @wchao1115 +- ❓ Machine learning apps within the browser using the media pipeline trigger many more memory copies compared with native applications hindering performance. +- βœ”οΈ Proposal: Introduce a more direct way to feed a video frame, possibly captured from a camera, to a ML model. + +ℹ️ [Permission model for Machine Learning APIs #72](https://github.com/w3c/machine-learning-workshop/issues/72) - @cynthia @dontcallmedom @anssiko +- ❓ How to design a forward-looking permission model for ML APIs? +- βœ”οΈ Proposal: TBD + +## Extending beyond the browser + ℹ️ [Applicability to non-browser JS environments #62](https://github.com/w3c/machine-learning-workshop/issues/62) - @jasonmayes @phoddie @huningxin @WenheLI - ❓ Pay attention to the applicability of the browser-targeted work to non-browser JS environments, in particular Node.js. - βœ”οΈ Proposal: Extend W3C coordination to TC53 and non-browser projects. +ℹ️ [Targeting WASI-NN and WebNN together #96](https://github.com/w3c/machine-learning-workshop/issues/96) - @mehmetoguzderin @mingqiusun @abrown +- ❓ Should libraries for browsers and/or Wasm execution environments be able to target WebNN and WASI-NN together? +- βœ”οΈ Proposal: TBD + +## Considerations for creating and deploying models + ℹ️ [Protecting ML models #67](https://github.com/w3c/machine-learning-workshop/issues/67) - @jasonmayes @tidoust @pyu10055 @jbingham - ❓ Some ML providers need to ensure their ML models cannot be extracted from a browser app. - βœ”οΈ Proposal: Investigate existing access control mechanisms for video, learnings from 3D assets. -ℹ️ [Support for Float16 in JS & Wasm environments #64](https://github.com/w3c/machine-learning-workshop/issues/64) - @cynthia @jasonmayes -- ❓ Lack of support for float16 in JS and Wasm environments problematic for quantized models. -- βœ”οΈ Proposal: TBD +ℹ️ [ML model format #74](https://github.com/w3c/machine-learning-workshop/issues/74) - @cynthia @jbingham @wchao1115 +- ❓ There is no standard format for packaging and shipping ML models, model formats evolve rapidly. +- βœ”οΈ Proposal: Initially focus on defining a Web API for accelerating established reusable ML operations instead of standardizing a model format. ℹ️ [In-browser training #82](https://github.com/w3c/machine-learning-workshop/issues/82) and [Training across devices #83](https://github.com/w3c/machine-learning-workshop/issues/83) - @irealva @cynthia - ❓ The current in-browser efforts are focused on inference rather than training. - βœ”οΈ Proposal: Understand successful real-world usages (e.g. Teachable Machine) and target transfer learning as the initial training use case for related browser API work. -ℹ️ [Memory copies #93](https://github.com/w3c/machine-learning-workshop/issues/93) - @aboba @wchao1115 -- ❓ Machine learning apps within the browser using the media pipeline trigger many more memory copies compared with native applications hindering performance. -- βœ”οΈ Proposal: Introduce a more direct way to feed a video frame, possibly captured from a camera, to a ML model. - -ℹ️ [Permission model for Machine Learning APIs #72](https://github.com/w3c/machine-learning-workshop/issues/72) - @cynthia @dontcallmedom @anssiko -- ❓ How to design a forward-looking permission model for ML APIs? -- βœ”οΈ Proposal: TBD - ## September 22, 2020, 2pm UTC [πŸ—“οΈ](https://www.timeanddate.com/worldclock/fixedtime.html?iso=20200922T14) >πŸ”Ž **Scope:** Web Platform Foundations for Machine Learning