Consider real-world usage of WordPress to quantify assumptions made across the Gutenberg project #6228
Comments
johnbillion
changed the title
What is success? Gutenberg will have succeeded when … ? |
|
I believe also that some informal accessibility related user testing was done at CSUN recently. |
|
Thanks for lending your voice to this issue, @johnbillion. As you and many others know, usability testing is one of my favourite topics. We've had small, curated test groups so far because I believe it is important to be aware of who and what we've tested. My hope like most is that as Gutenberg draws closer to merge, we'll see even more tests. Can we ever do too much testing and get too much feedback? Probably not. Adding brand new functionality to 30% of the web isn't without risks. We've iterated a lot on the existing features, so much so it is now time to get a stable product, to then reach out to more and more diverse testing groups. I'd love to hear more about what you, John, and others are building with Gutenberg. What we can do to make that process easier for those creating and those using the experience? What are the stress case stories we need to hear? The more people use, build on, and launch with Gutenberg, the closer we'll get to a product that's ready for 30% of the internet. |
|
@johnbillion I'm actively working on Gutenberg now if there are any specific conversations you'd like to pull me into. |
|
I would first like to applaud hard work, deliberate attempts at inclusion and commendable patience the core Gutenberg team has shown since inception. Often for better, Gutenberg is opinionated. It takes realities of data storage, legacy content and user needs into consideration. But I think my concern is the opinionated nature reproaches some historical control from users (and devs), and that risks alienating some markets. It could also open new doors for WordPress that sustain it ¯_(ツ)_/¯. Metaboxes encouraged users to show-and-hide, rearrange and create many different experiences. This is a paramount strength and weakness. Many meta boxes can be open, however, a single tab/accordion can be open. Gutenberg's reduction is perhaps powerful, perhaps healthy, perhaps enables more complex products going forward, but does reduce the HUD ability of scrolling a long page without interaction and may limit some use cases. I think my strongest concerns are:
|
|
My team and I are just kicking off a project to rebuild a WordPress site for a semi-prominent/well-circulated North American magazine. We're planning to be Gutenberg-first, and are also aiming to use it for managing the homepage layout (in addition to article and static page content). Target launch is late June/early July. I'm hoping to publish blog posts along the way about the experience, and will definitely be encouraging the team to contribute back to Gutenberg, especially as we work with the magazine's editorial team and get a sense of how they use it in their workflow. I agree that seeing Gutenberg in real world situations is critical and hope our experience can be useful in helping it develop. If there are any other specific ways we can help the Gutenberg team, we'd definitely love to do so! |
|
Great note and admirable articulation of a sensitive topic. @johnbillion has identified a number of user personas here who could be negatively impacted by the Gutenberg release. Many of them are users who I'd characterize as I think it's a fair statement that the Many of the developers in this thread represent But regarding these Matt and Automattic have done an okay job at communicating why Gutenberg should be a priority for WordPress, and as project founder he has the BDFL's right to set strategy in a non-democratic manner. But to hedge against a potential unsuccessful launch of Gutenerg I think A8c could help the community by encouraging, assisting, and subsidizing enterprise-level tests prior to 5.0. While public mentions of a release date have been sparse there does still seem to be a rumor that May/June may see 5.0. This timeline seems rushed, and as I pointed out on Twitter, Gutenberg seems to be a misfit in the "deadlines aren't arbitrary" regard already. So delaying further seems fine. |
|
@davisshaver I can only speak for ourselves, but we are absolutely planning to ship regardless of whether Gutenberg has been merged (and are operating under the assumption that it won't have been merged yet). Also worth noting that Automattic is encouraging and assisting enterprise deployments, through WordPress VIP. At the recent #BigWP event in New York, they announced that they'll be publishing some Gutenberg training to the public (potentially tomorrow?) and releasing a more configurable version of the Classic Editor plugin (sometime in June or so). We aren't a VIP partner so it's nice to see them making (some of) these resources available to the public, and I hope they continue to do so! |
|
Agreed! The engineering resources are good and appreciated! My point regarding Automattic was not about documentation or code quality. It's more a general concern about unknown unknowns and how A8c has a special responsibility having mixed its corporate and open source activities. As very well demonstrated by the VIP resources ;). What's not documented–what you will be generating–is the learning from launch in a high volume newsroom. To my knowledge, if you were to launch, you would be among the first organizations to meet the At present it seems the plan is to kick Gutenberg out the door & expect that enterprise newsrooms would switch sometime late this year. But what if we discover something critical during that first period? Before enterprise has migrated, after it's in Core. That period where users are forming opinions about upgrade path and Gutenberg. We only have a single chance to get that right. We all want to prevent any rift in the community or needing to add a LTS version. Personally I don't feel the de-risking to date has been adequate for the scale of the WordPress project. |
|
I'm working on a site at 10up for a government division and we are planning on shipping with Gutenberg as well, regardless of whether it's merged or still a plugin. Timeline to launch is also June/July. Like @chrisvanpatten our initial plan was to really go all-in and use Gutenberg for pages, posts, and even templates like the homepage or other landing pages that would commonly be more or less hard-coded archive-*.php templates with widget areas or customizer controls or whatever. However, some issues we're running into in using Gutenberg that are essentially "blockers" (pun intended) for going all-in:
I have other general concerns, but these are some of the ones that are really hindering current real world use for us. |
|
Awesome write-up, @jasonbahl 💯 |
|
I am currently using Gutenberg for my blog posts, but I do not intend to use it for any pages on websites yet, mainly due to the lack of responsive, non-equal width, and variable width columns, as well as the lack of sections. Without those, I simply can not use Gutenberg for serious page building. I understand that they are coming in the future, but I would really prefer if they came before the WordPress 5.0 release. |
|
I've tested 2.7 recently and one of the main reasons of using Gutenberg, the columns block, continues to be really difficult to work with, so at the moment using Gutenberg on client websites is a no-go for me. |
|
Thank you everyone so far for the awesome feedback. I want to step specifically into answering points if I can to try and make sure we both have issues and more information for everything. @chrisvanpatten really excited to see your launch!
Yes so many ways! We always need code contributions for example PRs and triage to the issues. However, beyond that using and feeding back where the stress points are is an invaluable contribution. Publishing that post you are saying you are going to do is also a huge contribution, thanks. @davisshaver if I may just reflect on your comment here:
Actually the most source of feedback has been from the .org + enterprise. I know that may come as a surprise to many, but it's true. I note feedback as the product is built and influenced by the feedback it gets. Just to ensure everyone has the information, as has been mentioned I would point to https://testgutenberg.com/ and https://vipgutenberg.com/ for some awesome resources. There are also other courses like the awesome ones @zgordon runs: https://gutenberg.courses/. @jasonbahl this is exciting to read!
Let me try and address each point you raise:
Can I check what you mean there and if the following issue helps or not? If that doesn't, can you maybe phrase exactly what is blocking and we can get an issue made for you, or feel free to make one of course.
I don't see an issue for this specifically so wondering if you'd like to make one? I do know of plugins that limit like https://wordpress.org/plugins/block-options/. I would like to dig a bit more there though.
Does this cover that: #5602? I just want to make sure we have an issue relating to this.
I would love to hear those either here or please reach out to me on chat.wordpress.org. |
To clarify this, I mean users inside |
@karmatosed vipgutenberg.com is only available to WordPress VIP customers, correct? That leaves many (most?) developers and agencies out. |
|
@greatislander two of those VIP courses are white labeled versions of what is available at gutenberg.courses and it is for VIP clients only, correct. You can access the same content at gutenberg.courses tho :) They are also launching some free content on vipgutenberg.com today as well. |
|
Thanks for the clarification @zgordon! |
|
@davisshaver I don't want to name names (not my place!) but I can say that there are definitely organisations experimenting with Gutenberg in editorial. I don't know of any larger org using it in production, but it's definitely being exposed to non-developer users in enterprise. |
|
@chrisvanpatten Right, as you noted, this was discussed at the recent Big WP meetup. But my point is that ideally > 0 orgs have tested in production before 5.0 release. I am currently making plans to launch OnwardState.com on Gutenberg next week. |
@davisshaver, there has been some feedback but we can always have more. It also depends on what we're talking about with high volume. It's important to get all stress cases I absolutely agree. That's great you are launching on next week. Anything you can pass on as a lesson there in maybe a blog post? If unable to post public please reach out private. I would love to learn more from everyone as to their experiences. @greatislander: apologies, I should have added that it was coming soon. As @zacgordon says it's all going to have free content as of today - which rocks! Thanks all for the continued tone and quality in this issue, it's incredibly important to get insights like this. |
|
@karmatosed I'll elaborate a bit more on my previous comment.
Currently, when a user selects a Page Template it's trivial to add contextual awareness to metaboxes so that the user can enter content specific for that template. If they switch templates, they will see different metaboxes. Ideally, when a Page Template is selected, a user would be presented with a "Block Template" that allows them to edit the page template. @melchoyce presented a compelling vision for this at LoopConf, but that was just a vision, not current functionality of Gutenberg. There are issues related to this. In fact, searching the Github repo for "page template" shows 136 results currently: https://github.com/WordPress/gutenberg/search?q=page+template&type=Issues Some relevant ones: #3588, #5521 #5537 #4482 (probably more out there that are relevant too)
There are documented ways to whitelist/blacklist, but they're currently not functional. I've elaborated more in this issue: #4848, which is a duplicate of #5895 and in both cases the solution presented is moving Block Registration to the Server, a'la #2751, where I've also written a few books worth of my thoughts 😉 Which leads us to the next point:
I've elaborated quite a bit on why I think the Server Side API needs more attention in #2751. Also engaged in a discussion about the importance of Gutenberg having a Server Side API on make.wordpress.org in regards to the Native iOS and Android Apps plan to support Gutenberg: https://make.wordpress.org/mobile/2018/03/21/gutenberg-on-mobile/#comment-19383 The project we're working on now expressed very early on that they want to have some content be editable by certain roles and locked to other roles. Blocks are a great example of how this could be possible, and there are probably hacky ways to try and do this on the client at some level, but in reality the server needs to have the final say on what users can do with what blocks. The client shouldn't be trusted. The client should provide validation to it's best ability, but the server needs to have the final say. Being able to validate input from the client is pretty critical for anyone to really adopt Gutenberg I would think. There's been prototypes of Collaborative editing on blocks as well. . .without a good Server Side API to know what data can/cannot be a part of a block to act as that intermediate contract negotiator between clients, you're asking for folks to get hacked. If we're talking directly client-to-client, I imagine someone will figure out quite easily how to send something that starts tracking your keystrokes or whatever else. I'm not a hacker, I don't know what a hacker would come up with, but I can tell you it would not be fun to deal with. Having a server do the proper validation on Block input before sending data across the wire to another client is pretty crucial when the day comes to have collaborative editing. FWIW, it does look like there's action on this here: #5802 But, all the resources, documentation, etc are still telling everyone how to do everything on the client. There needs to be a BIG push to get this Server API solidified sooner than later then go through and update all materials out there, including third-party materials like the courses @zgordon and the like have put together on how to properly register blocks via a Server API and letting folks know that client-only registration of blocks is generally not recommended.
|
|
@jasonbahl thanks for those replies. I totally agree that Mel's vision is exciting and it will happen - but it's not happening now. I wanted to say thanks for diving into the repo, so good to see you appearing on a few issues. If you ever do want to point out other concerns or issues please reach out either here or via chat.wordpress.org Slack also. It's important as a team we hear them from everyone. That extends to everyone in this discussion. |
Any clarification on what this means? What part of it isn't happening? Good support for nested blocks? Support for controlling what blocks can be nested within other blocks? The ability to swap block templates on the fly when "Page Template" is changed? Would be good to have clarity on what current features, such as nested blocks, are getting finessed so they work well or removed because they're too hard to get working or whatever. |
Hey, good questions. Let me take a stab at answering, based on where I think we're at right now.
As far as I can tell, these improvements are ongoing: https://github.com/WordPress/gutenberg/issues?q=nested+label%3A%22%5BComponent%5D+Nested+%2F+Inner+Blocks%22 I think we're going to finish up a lot of the bones supporting page layouts for 5.0, but we're not going to see native Gutenberg page layouts until after the initial release. The work I've been doing on being able to select page layouts (in lieu of the old page templates) is still in design, and won't be making it into the initial release of Gutenberg.
This is definitely planned, and I've been working on some mockups around this, but I don't see it happening until after 5.0. I would personally like the second Gutenberg release to focus on pages — page layouts, better blocks for creating complex layouts, more dynamic blocks, etc. — but that's still TBD, based on how the initial release of Gutenberg goes. |
|
@johnbillion, totally agree with everything up to the last section of your ticket. 💯 To use the terminology in an earlier comment, What I think will solve that is (1) in-dashboard promos for the Gutenberg plugin to be installed on .org sites, put in a future 4.9.x release and (2) availability of Gutenberg on I think we've learned a lot of lessons from the REST API merge, and I'm glad you raised the issue. |
|
@jasonbahl, thanks for asking for clarification and @melchoyce thanks for adding your input there, phase two is going to be exciting after laying the groundwork. |
|
@johnbillion Given the timeline proposed at WCEU, are there any specific, actionable steps you'd suggest we take that aren't already underway? |
|
I'm really pleased to hear there's a plan in place to begin user testing of Gutenberg on WordPress.com. It's probably a good idea to open an issue that outlines the plans that are in place for gathering and analysing the feedback that will be submitted by users as part of this testing phase so that it can be tracked, triaged, and actioned by the community. @danielbachhuber Is this something you can coordinate? Unfortunately I'm still short on time. If I had to pick from my original list above, I think it's important to look at the impact that Gutenberg has on the sort of sites that make up the majority of the long tail of WordPress sites but which are (ironically) generally seen less frequently by the core WordPress community: sites that are cobbled together with a large number of plugins including page builders, e-commerce plugins, photography galleries, membership plugins, multilingual plugins, and off-the-shelf themes. There is a real risk of widespread damage to the reputation of WordPress if the Gutenberg editor breaks these sorts of sites en masse or confuses the editing experience. |
|
I’d plus 💯 that last paragraph from @johnbillion if GitHub allowed me to. For example, plugin customer support allows me to see potentially problematic back-ends on an almost daily basis, and that’s probably true for some other support folk, but obviously that doesn’t enable or allow us to test Gutenberg on those sites. What support reps from plugin providers, myself included, could help doing is spread the word. I’m going discuss ways of making WP Media’s support customers aware of Gutenberg with our C level, and I’d like to encourage other support reps from other companies to do the same. |
This! |
Yep, certainly, if it's appropriate / effective. One project to put on everyone's radar is the
We've been discussing this document in our weekly chats for a month or so now. Our original plan was to "test run" the document with DreamHost and InMotion support teams before advertising more broadly. However, it might be worthwhile to publish a block post about it sooner. cc @jadonn @getsource @0aveRyan
Yes, and this is certainly something that the |
|
While the discussion in this issue has surfaced many good points, I think a good rule for our issues is they should be actionable and discrete/specific. This issue is quite broad in its scope and I'm not sure how to measure it "completed". I hope that a lot of the good points around research and testing are internalised by the team working on Gutenberg and on other parts of WordPress. I'm going to close this issue as I don't think it's actionable as-is, but if there are discrete issues to come out of it that can be filed separately, that's cool 👍 |
Issue Overview
The REST API project was a big focus from 2012 up until the completion of its inclusion in WordPress 4.7. Its development spanned four years.
Despite this lengthy period of development, it wasn't until Gutenberg's development started in 2017 that shortcomings of the REST API were really seen on an impactful scale. The
Core REST API Tasklabel is tracking issues in the core REST API that have been identified as a result of what the Gutenberg developers are trying to do with it.All that is to say that the REST API went through four years of development yet has shortcomings that were only identified when developers started building an actual end-user facing product that consumes it. Attempts to replace
admin-ajax.phpfunctionality in the WordPress admin area with REST API calls have also hit problems.What's my point? My point is that the REST API was not exposed to enough real world usage during its development cycle, and this is only now becoming apparent when developers start building innovative applications on it and shortcomings are discovered. Not enough people built things that used the REST API during its development in order to discover whether what was being built actually solved problems.
This ticket will serve to help avoid the same problem happening to the Gutenberg editor. Before a merge proposal can even be considered, a substantial amount of real world usage of Gutenberg needs to be performed, gathered, analysed, iterated upon, and considered as guiding aspects of its continued development.
Real-world usage of Gutenberg covers a much wider spectrum than the REST API. It covers:
I've opened this issue because I believe that the project in its current phase isn't considering real world usage, and I don't believe enough consideration has been put into what problems it's solving. Two open issues illustrate this point:
The result is that the project is making many assumptions about end users, and how they'll use a Gutenberg powered WordPress site. Some user testing has been done (most recently I believe at WCUS in December) but with a narrow and controlled scenario that isn't representative of the way that 30% of the web uses WordPress. I currently enjoy using and experimenting with Gutenberg, but I'm not remotely representative of the target user.
In order to succeed, the project needs to step back somewhat and consider what it's trying to accomplish, for whom, and in what capacity.
I want Gutenberg to succeed. In fact, I think Gutenberg has to succeed, but it won't unless real world usage is moved to the forefront of considerations taken into account during the development of the project.
To that end, a question: How can real-world usage considerations be embedded into the project's ongoing design and development to ensure that it is a success?
Addendum: My own availability to work on this issue, and Gutenberg itself, is unfortunately almost nil (ironically due to working on a Gutenberg-powered project for a client) but I will keep an eye on this issue as and when I can.
Edit: I updated the issue title for clarification.
The text was updated successfully, but these errors were encountered: