Processor invocation order is unclear

[kduske-n4]

Quoting from the AsciidoctorJ 1.6 Integration Guide

Treeprocessors are called by Asciidoctor at the end of the loading process after Preprocessors, Block processors, Macro processors and Include processors but before Postprocessors that are called after the conversion process.

(emphasis mine)

This is not true at least for inline macros, which are processed after treeprocessors in AsciidoctorJ 1.6.0 beta 3. This is particularly troublesome as nodes that are created in the treeprocessor need to have the correct substitutions set to ensure that inline macros are substituted, but there's no "official" API for that:

• There's no way to query substitutions from a node using the Java API. This would be useful if I want to inherit substitutions or copy them when replacing nodes in a treeprocessor.
• There are no constants and methods in the Options class that allow me to set substitutions. I have to resort to hardcoding the strings.

[mojavelinux]

You're correct that not all macro processors run at the same time. Block macro processors run while the document is being parsed. Once parsing is complete, then treeprocessors are run. The inline macro processors run while the document is being converted. Once the document is converted, then postprocessors are run.

The reason inline macro processors run during conversion is due to asciidoctor/asciidoctor#61. That's just where we are right now (and has always been this way with AsciiDoc). We know we're going to have to make that switch soon enough.

There's an understanding right now that inline macro processors must return converted output (in other words, they should be seen as an extension of the converter, not as an AST or source transformer).

There is a way to convert inline macros to AsciiDoc source in time for conversion. For this, you need to use a treeprocessor that visits the leaf nodes and transforms the raw source. We use this technique in asciidoctor-mathematical to replace inline stem macros with image macros. See https://github.com/asciidoctor/asciidoctor-mathematical/blob/master/lib/asciidoctor-mathematical/extension.rb (Keep in mind, its necessary in some cases to use private APIs as this approach is still maturing).

AsciidoctorJ can only offer what Asciidoctor core offers...and right now, inline macros are just a little quirky until we're able to include inline nodes in the AST.

[kduske-n4]

Thanks for your feedback. I understand that, at least for now, inline macros have to run later during conversions. My suggestion is to make this clearer in the documentation because right now it's a bit confusion for newbies like me.

Regarding the raw processing trick, I have actually done the same thing myself since I had to create a new block to host the output of an inline macro to push it into the sidebar. It's quirky since I have to reproduce some things like argument parsing manually, but it works for now. I'll add my 2 cents about this to #61.

But regarding my original issue, it would be great if the API would support querying and setting the substitutions for blocks.

[mojavelinux]

it would be great if the API would support querying and setting the substitutions for blocks.

The Ruby API allows this. Perhaps this is just missing from the Java API.

[kduske-n4]

Yes, sorry for being unclear: It's missing from the Java API. For now I have hardcoded the substitutions in my processor, but at some point, I'd like to inherit them from other nodes. Once I get back to that, I can try and add the necessary code to AsciidoctorJ and send a pull request.

[mojavelinux]

Great!

I'd like to inherit them from other nodes.

Could you clarify in what way you want to see them inherited? You can express your intent using pseudo code. I'll know what you mean.

In other news, @robertpanzer just added methods to the API to get and set substitutions on a block as those methods were previously missing.

[kduske-n4]

Hi Dan,

inheritance of the substitutions is not what I would want in all situations. What I really want is simply a way to get all substitutions from a block and set them to another, possibly newly created, block. In my particular case, I'm replacing blocks in a tree processor and want the substitutions of the replacement block to be exactly the same as the substitutions of the original block. That is what I meant by "inheritance", which is a confusing term to use here. Sorry.

@robertpanzer thanks for adding these methods!

[mojavelinux]

Excellent. Then that's exactly the direction we are headed!