Introduce formal fields API with adaptation layer to address metabox concerns

[danielbachhuber]

In reading through #3304, I had an idea: what about formalizing a fields API spec, and then offering an adaptation bridge between ACF, etc. and the fields API spec? I feel this would solve the majority of metabox implementation issues, because most people are using a framework plugin to build custom sites.

[youknowriad]

We should definitely offering a between ACF,CMB... and Gutenberg. Gutenberg will have Templates which in addition to blocks saving to PostMeta are a natural way forward for these plugins.

Whether this bridge needs to rely on a Fields API spec. I don't know. I feel like the blocks would make the Fields API useless and we'd just have to rewrite those plugins twice, instead of one. Which means If we're able to write a bridge to Fields API and we can offer a bridge from Fields API to Gutenberg's API. We can do Plugins => Gutenberg directly.

Also, not sure we can favor a plugin over another. Once Gutenberg's API is "complete", we'd have to reach out to plugin authors to ensure an easy migration path for plugins that want to offer one.

[danielbachhuber]

Which means If we're able to write a bridge to Fields API and we can offer a bridge from Fields API to Gutenberg's API. We can do Plugins => Gutenberg directly.

Right. Two ways of saying the same thing: a formal Fields API for WordPress would also be the formal fields API for Gutenberg.

Also, not sure we can favor a plugin over another. Once Gutenberg's API is "complete", we'd have to reach out to plugin authors to ensure an easy migration path for plugins that want to offer one.

Correct, sorry if I implied otherwise. What I'm suggesting is that having a formal Fields API (that would yield templates in Gutenberg) and then working with plugin authors to implement a bridge between their existing field definitions and the formal Fields API would yield the end result of these major plugins working gracefully with Gutenberg.

[sc0ttkclark]

+1 on Fields API, we're in the process of a revisit of the codebase right now to simplify it to focus on storing/getting configs, with inputs/saving/etc separated out into a different proposal.

[BE-Webdesign]

I feel like the blocks would make the Fields API useless and we'd just have to rewrite those plugins twice, instead of one.

This would be my initial impression as well, but I honestly don't know enough about the Fields API.

[danielbachhuber]

we're in the process of a revisit of the codebase right now to simplify it to focus on storing/getting configs, with inputs/saving/etc separated out into a different proposal.

@sc0ttkclark If this isn't the case already, I'd suggest that the existing Fields API project needs a significant amount of input and guidance from core committers before you invest too much time into refactoring.

This would be my initial impression as well, but I honestly don't know enough about the Fields API.

tl;dr: A declarative abstraction for defining fields and field groups. Most libraries already implement this in some form, and there have been many lessons learned along the way, so most of the effort should be put into standardization. From this standard, Gutenberg templates can be rendered.

Copying @mtias' comment from Slack because I think it's relevant:

matias [1 hour ago]
The spectrum would look something like:

• Classic Editor plugin.
• Detect problematic meta-boxes and load classic mode.
• Metaboxes that can be loaded ok in gutenberg with the non-iframe implementation.
• Bridge (like Fields API) to make this presence smoother.
• Fully native Gutenberg blocks or extensions.

[sc0ttkclark]

@danielbachhuber we've had a lot of eyes on the project the past couple of years, this next effort will be alongside an effort to recruit more Core devs to be part of the next stages.

[StaggerLeee]

I was on my way to say just that.

@sc0ttkclark was often complaining about lack of interested developers for Fields API project.
You behind Shortcake can join, developer(s) of ACF can join, Yoast developers, WooCommerce devs, and some other owners of big plugins and very much code lines. You can together make something very big for Gutenberg. And to save your time in the future.

I would help personally but I am unfortunately not even close good in coding for something like this.

[gschoppe]

To my mind, it would be very useful to provide a simple JSON api to construct custom sets of simple inputs out of basic components (be they blocks or traditional metaboxes). So, if a user just needs a text input, checkbox, and select field, they could simply create and pass Gutenberg a JSON object. The existing block spec would exist for users creating custom interfaces, but simple interfaces could just be a collection of pre-defined components.

This would also reduce number and complexity of enqueued resources.

That way, the same config could drive either metaboxes or Gutenberg, depending on context, and could be a starting point for providing a common interface for various frameworks to use (provided it allows for extensibility to support each of their custom feature sets)

[markkap]

In sports the most important player is the one that didn't play. This discussion reminds that saying.

Complex metas which are the ones that are not likely to work with any simple "iframe like" solution, are also unlikely to be easily defined with the fields API, or json file or whatever. Yoast SEO settings can not be defined with simple schemes and so is probably the publisize feature of jetpack, and just with those two plugins you are talking about more than 10% of wordpress sites without even getting into the mine field of "one off" hacks.

Anything that requires JS that is not supplied by core is unlikely to be automatically transferred from "old way" to "gutenberg way"

[gschoppe]

the JSON config could certainly be supplied via core and inserted with filters, and it probably wouldn't suit yoast. Their integration, despite the popularity of the plugin, is extremely complex and custom, interacting with the editor's content and other aspects of the page DOM, and I can't imagine it working with properly with Gutenberg without converting to JS. However, Yoast also has the resources to do so.

I imagine this JSON config as a 75% solution for plugins and users that don't want to learn/enqueue relatively high-level javascript (yes, it seems simple to those comfortable in modern JS, but to many this will be magic) just to add a few text fields and checkboxes.

If it is extensible, plugins like ACF could add their own JS to it to support more complicated UIs, but it would provide a bridge between metaboxes and Gutenberg, and reduce the number of enqueues needed for simple integrations.

[markkap]

@gschoppe meta boxes are content specific, they are means to extend the basic post type into a product, event or whatever other content type. It is just the easiest path to create a new admin experience to those types, and are not a goal by themselves.

As what is really needed is to be able to change the editing experience based on context, focusing on the mechanics of metaboxes is wrong, as the goal of all sophisticated meta box usages is to transform the whole editing screen in a way which will create a good UX for a specific content. As gutenberg monopolizes the editing screen, the plugins and themes that need to do it will have to integrate with gutenberg to adjust because gutenberg changes the whole structure and HTML of the page.

I am sure yoast did not select his implementation because he likes complexity, it is just that the other option would have been a user hostile UX with metaboxes taking most of the editing screen, and that is just for the SEO stuff.

Sure for ugly looking metaboxes something like the fields API in whatever implementation form, would ease the transfer to ugly looking meta boxes in gutenberg, but users want nice UX, not ugly one, and I am sure that you know as well as me that designing a user centric easy to use edit workflow is one of the most important things in any project. Designs for such a workflow for pre gutenberg era will just break with gutengerg.

In the end, the problem gutenberg faces is technical debt. meta boxes and the editing experience haven't changed in any major way for probably something like a decade. At this point breaking backward compatibility is a must to be able to move forward, the question is where do you move to, is your solution is based on the experience and understanding of what users and developers need gained at that time, or are you going to fall back into empty simple slogans and hacky implementations like "everything is a block" and "Fields API will save us".

[gschoppe]

@markkap, not deriding the rich experiences possible in Gutenberg at all! I thoroughly believe in giving users a rich JavaScript api to build whatever interface they need for Gutenberg. However, the common use case of "I need a check box on this page, that saves to post meta" is not going to go away because of any UI change, and the current JavaScript api makes it a bit clunky for users to iterate quick ui needs like this.

Metaboxes can be enhanced and transformed by Gutenberg, but many users will still just need an easy way to add simple UI elements in fixed positions. That need has been handled by ACF for a long time, but I think Gutenberg can learn from the prevalence of that use case, and move some of that in-house. ACF will still obviously exist for helping users quickly iterate on more complicated UI needs, but something built into WordPress to meet the simple case would be exceedingly useful in lightening the reliance of designer/developers on single point of failure plugins.

Also, in a completely JS admin, without any look backwards, a need still exists for ui primitives to be enqueued in a common way. Otherwise, every plugin, theme, and feature needs to enqueue it's own js, which will each add requests to the admin load time, and will each re-implement the same trivial features over and over.

[StaggerLeee]

Devs of Gutenberg would very much speed up solving of this problem by not insisting on traditional way how core things are done.

I mean contact developers of few most installed plugins and, yes lose additional time, to help them code their Metaboxes and convert them to Gutenberg blocks. Before merging.

ACF, Pods, Yoast, WooCommerce, and few others cannot remember now by heart.
Benefits would be huge from psychological point of view.
"Hej, if those guys converted everything to Gutenberg blocks, and they have thousands and thousands code lines, then I can/have to do it with my small plugin too",

[markkap]

@StaggerLeee support by plugin authors is not the problem. all of them are likely to release gutenberg compatible versions when gutenberg will become a viable product to develop against. The problem is with people that for whatever reason will not be able to upgrade the plugin and/or people that unknowingly will upgrade to 5.0 and get a borked editing experience.

@gschoppe I will try to say the same thing in different way ;) the leading metabox API is ACF. Add to it CMB2 and redux and all what gutenberg team needs to do is to detect the specific patterns and automatically convert them. To get that level of backward compatibility there is no need to introduce an additional step. Those frameworks are not the problem, the problem is with thing which are not based on framework and it is very likely many sites have at least one of them. Even if fields API had landed today, those things are unlikely to have been changed to use it (why would a site owner put more money into a feature he sees no immediate value from it?)

Gutenberg can create UI primitives and probably should do it, but those primitives are unlikely to answer a complexity of even trivial level, like change values in a drop down based on the selection in another drop down. in higher level of complexity you will find the need to communicate between two different meta boxes (drop down changes based on selected category), and for this kind of complexity you will have to integrate with gutenberg to send and receive events something that is unlikely to be done in the kind of API for defining primitives that gutenberg is likely to end using (IIRC this kind of things is something that ACF can not do for example and you need to enhance the ACF code).

[gschoppe]

@markkap I think we are on the same page with the issues with a primitives library and simplified config. At some point, plugin developers will need to write JS to build the perfect interface. I am not suggesting removing that hurdle with a common JSON config of primitives. I'm simply suggesting that it would be a good way to offer options for simple interfaces, and that if thoughtfully designed, it could be extended by plugins to support more use cases.

There are many many plugins out there that modify the post edit page, with nothing more than a checkbox or a single select. Minimizing the impact to these plugins would chip away at the mass of plugins that will have a hard time adapting.

[markkap]

@gschoppe totally agree, some primitives will greatly help the lower level developers (not snubbing at any one, we all been like that at some point) to overcome the complexity of integrating with react. I just hope this short term goal will not distract from the need to rethink the whole extendability model of the edit screen UI and the way data is mapped between UI and the DB.

[danielbachhuber]

Closing in favor of #4151, a broader discussion about Gutenberg integration with existing codebases.