Added support for plugins

[jcayzac]

This adds support for plugins using the [plugin_name:arg] syntax.

test/index.js shows a basic "youtube" plugin.

[markc]

I'm not sure how to "vote" for this other then to add this +1 comment. I really need something like this but I would rather not maintain a fork.

[rhiokim]

great work :-) +1

[ghost]

+1

[potench]

I pulled this feature into my instance but it didn't work.
While trying to fix it, I arrived at a vastly different solution.
Please provide feedback as I've made a few assumptions outlined below that might need adjustment.
If you like it, I'll PR it and we can close this issue.

Helper Support

Here's my commit, it includes tests and is ready for use now should you need it immediately.

Configuration

// helpers can be added to the options object 
marked.setOptions({
    gfm: true,
    tables: true,
    breaks: false,
    pedantic: false,
    sanitize: true,
    smartLists: true,
    langPrefix: 'language-',
    helpers : {
        youtube : function (url) {
            return '<iframe src="https://www.youtube.com/embed/' + url + '"></iframe>'
        }
    }
});

// or you can add helpers through the setHelpers() method
marked.setHelpers({
    helpers : {
        annotation : function (txt, msg) {
            return '<a class="annotation" data-msg="' + msg + '">' + txt + '</a>'
        }
    }
});

Usage

{NameOfHelper::var1:var2:var3...} 
or
<!--- {NameOfHelper::var1:var2:var3...} -->

• You'll probably want to wrap helpers in comment blocks so that other parsers hide your helper syntax
• You can pass multiple variables to the helper separating each by a colon :
• The parser simply ignores helpers that don't exist so {missing::on:purpose} will just be ignored (until you add a helper method called missing)

Example

## My Example markdown file

There is inline-lexer support for helpers: {annotation::test:1}

 * You can trigger them in a list or other markdown semantic {annotation::test:2}

Or you can trigger them at the block level like the following example:

{youtube::g2FOLrC2e6E} 

-----------

### Bonus

Wrap your helpers in comment tags <!-- {annotation::test:3} --> so other parsers don't display them.

Note that some markdown engines (ghm) use <!--- (three dashes) so this example also works:
<!--- 
{annotation::test:4} 
-->

[potench]

Also, @chjj I have a quick question.
Unrelated to the above commit some of the tests (10,11,14,19) in master are failing in node 0.8.8.
Do you want these updated or am I missing something?

[lepture]

I am doing it in another way, like sundown #129

[potench]

@lepture I don't think these are the same - you are customizing the output for each instance of a particular match vs specifying a unique helper or plugin independent of existing markdown matches. If I wanted to customize a heading, I'd use CSS.

[rhiokim]

it didn't work fine in this case

[only:link](http://example.com)

[acamarata]

Has any method for helpers, plugins been accept? I have a handful of custom things I wanted to throw in, thanks.

[aplib]

Could you explain a little more?
Clarify the question: do you want to parse another markup (mediawiki?) Or simply add a simple substitution?

[acamarata]

Both are welcome as I want simple substitutions for quoting specific
references that are common used {Stats:Blah Blah Blah:p34} will be used a
lot, and in Mediawiki we did this with Templates, I would love to be able
to code some substitutions the best way, best practices, etc. But there is
a real need for some higher level and some core Mediawiki functions, which
would be great as an addon/plugin - but wondering how I could get
References, footnotes, citations, and maybe even some form of templates
though substitutions could work for some/all of those also.

So both, two questions.

On Thu, Sep 5, 2013 at 9:03 PM, vadim baklanov [email protected]:

Could you explain a little more?
Clarify the question: do you want to parse another markup (mediawiki?) Or
simply add a simple substitution?

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/35#issuecomment-23869661
.

[aplib]

I implemented some mechanism to include foreign content in markdown tex, look here http://aplib.github.io/markdown-site-template/index.html. If something is unclear ready to explain

[acamarata]

I'll take a look at stuff like:
http://aplib.github.io/markdown-site-template/components/controls.alert.html

I would want to implement something like [[Page]] which would be red if no
page exists and blue if it does, which would require some logic etc. I
also don't see anything really robust enough to replace Mediawiki functions
all out, but I could work on it. It does seem a start on setting up some
substitutions but I like what I saw of plugins and addhelps in the two
links above... but seems they are not pulled into the project.

On Thu, Sep 5, 2013 at 9:12 PM, vadim baklanov [email protected]:

Or rather there is not a single mechanism, there is the templating and
components

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/35#issuecomment-23870290
.

[aplib]

for example, I added a component based on the code from the repository https://github.com/cscott/instaview
You can see it here http://aplib.github.io/markdown-site-template/components/wiki.instaview.html
This plugin adds wiki markup.

[SteveBenz]

I agree with the direction that the original change and the extensions that @potench proposed. This is a great way to solve a bunch of problems, as all the +1's suggest. The idea of supporting for compat makes me happy too, given that these plug-ins, by their very nature, are going to be site-dependent.

Is there some prior art that the ":" syntax for parameter-separation is mimicking? I'd suggest two alternatives to that mechanism. The first of which is to not do any parameter-breaking at all and make the add-on break down the parameter list if it needs to.

If you really feel that a more sophisticated means would be helpful, then I'd say the argument should be treated as a query string. I prefer that because:

1. It's a well-known syntax
2. It's flexible
3. It has a known escaping language (e.g. what if you needed to have an argument with a colon in it?)
4. It'd allow the plug-in to take a more flexible parameter list. (I've been thinking about using this sort of thing to allow markdown to reference code samples; in that space so customers may want to reference samples with a line & file, sometimes by an ID, maybe by a file name and a region name, etc.) With this sort of thing, you can enrich your plug-in without breaking compat with old versions pretty easily.

Another nit: I'm not sure I like the switch to "{" from "[". I mean, I believe "[" alone is wrong, because that overloads the hyperlink semantics in a way that might be confusing or conflicting. But introducing a whole new character seems like too heavy a hammer. Instead, I'd propose looking at the image inclusion syntax ("![...]") as a model, because it does something generally similar. My proposal for a character would be '$' because it's got some variable-substitution connotations that seem appropriate here. So, for example "$[youtube:g2FOLrC2e6E]", would be what I'd go for.

Details aside, this is an important change, any hope of some progress here, @chjj ? Anything I could do to help it along?

[commenthol]

I like the idea of having plugins in markdown.
The proposal here adds this on a block level but not on an inline level and I am wondering if the syntax proposed fits all possible use-cases.

Currently I am working on a markdown preprocessor markedpp which adds support for TOC, numberedheadings, inlining files, aso ...

I encountered the problem that there is no clear defined markdown syntax for extensions, and I really wish there would be one.

This syntax should be suitable on block and on inline level.

In markedpp I needed to specify a command (comparable to plugin/extension) as well as some parameters and opted for the following "bang" syntax which is derived from the image tag.

!<command> (<options>)

where:

• <command> : A word of [a-z]+ chars defining the extension
• <options> : Optional. A space separated list of keys or key-value pairs enclosed in normal brackets.
  • key-value pairs are separated by a = char e.g. key=value which allows assignment of numbers, strings
  • keys lack the = char which allows assignment of booleans (or value combined with the used <command>).
  • Arrays can be assigned to a key with key="value1;value2;value3" (The separator ";" char would need to be discussed... or could be left to the used <command>)
  • If spaces are used in keys then these need to be escaped with \, alternatively " can be used e.g. key="value with spaces".

This allows to specify e.g.

!toc

!toc (level=2 omit="Table of Contents")

!include (filename.ext)

In order to reuse this information for another pre-process run a "closing" tag is required as well. Therefore the alternative syntax is allowed as well:

<!-- !toc -->
* [Heading 1](#heading-1)
<!-- toc! -->

This works well on block level, where each command is surrounded by newlines, but not on an inline level. I am not firm if there are languages which allow a ! in front of a character (e.g. something like spanish with their ¡). Or imagine a typo e.g. "Last word !Next sentence". !Next would be interpreted as a command... :(

While reading @SteveBenz comment I really like the idea of using the $[]() syntax which really can be used everywhere.

So what about this?

$[<command>](<options>)

which would allow e.g.:

$[toc](level=2)

or

<!-- $[toc](level=2) -->
* [Heading 1](#heading-1)
<!-- [toc]$ -->

or if you want to define a plugin which itself neads some input:
e.g. a uml-parser which generates a SVG displaying a UML sequence chart:

``` $[uml](chart=sequence)
a --> b : Hello!
a <-- b : How are you?
a --> b : Thanks, I am fine.
```

What do you think?

[SteveBenz]

@commenthol has a lot of good ideas. I like your adaption of what I suggested. Using the () for the argument list is a fantastic idea, because I think it looks clearer and it doesn't rely on introducing some new magic character into the markdown syntax. $plugin is quite consistent with the prior art and I think it strikes the right balance between clarity and backwards compatibility.

I am really solidifying my belief that argument parsing should be punted to the add-in itself. Maybe nice markdown libraries can offer some libraries and good documentation can suggest some best practices, but it shouldn't be a part of the language.

I think you should separate your thinking about ``` from plug-ins. I think you could do it with custom rendering better. If you had a custom renderer for ```, you could go with:

```umlsequence
a --> b : Hello!
a <-- b : How are you?
a --> b : Thanks, I am fine.

Your custom renderer could pick that apart and generate some nice svgl from that text. To me I don't see it as all that different from syntax highlighting. The reason I say that is because if your markdown got copied to some other host that didn't support any of your markdown extensions, the foreign renderer would simply render the text as-is, and that wouldn't be the end of the world. The reader could still get the gist of the diagram.

Although this belongs in another thread, I'd claim that we need to allow for richer metadata than just a language after the ```. I'd like to see the ability to include other metadata there - here, for example, the language is UML, but you shouldn't, as you are now, forced to somehow pretend that "umlsequence" is a language, when in real life, "uml" is the language and "sequence" is the kind of diagram.

For another example, suppose you have a custom renderer that not only knows how to highlight, but also to generate hyperlinks to class names mentioned in the code. For that guy to really work right, it'd need to know the "using" statements associated with the block of code. Right now there's no way to convey that 'using' data... But all that'd be needed to fix that would be to loosen up the regex that matches fenced ticks.

[SteveBenz]

In re-reading the comments, I think we should highlight why we need a new syntax for plug-in calls and why Custom Renderers, while they're wonderful things, are insufficient. Suppose, for example, that we create some functionality and use a custom renderer to make it happen. I could have markdown like:

Take a look at the following sample:

[sample:source=foo.cpp&name=hello world]

That'd be easy to implement with custom renderers and it'd generate great results on the site with the renderer. But if I copy the markdown to some other system where the renderer isn't there, I'm going to see my sample block replaced with a dysfunctional hyperlink. It'd be much better for the reader if the code had been left as-is. At least then I could glean that the markdown was broken and should have included a block of code.

Moreover, if I (as the person who copied the markdown to the new host) want to fix up the markdown to look good on the target system (even if it doesn't have the fancy plug-ins installed), I can't reliably find the plug-in calls because there's no clear syntax for them.

[markc]

As far as using non-existent plug-ins on other markdown sites without the required extra code maybe it would be a good idea to embed the whole plug-in system within the current link semantics using just # " as the trigger for plug-ins... ie;

[text to replace with inline plug-in output](# "text red,bold")
[text to replace with referenced plug-in output][1]
[1]: # "text red, bold"

is rendered like...

text to replace with inline plug-in output
text to replace with referenced plug-in output

In the above examples # " (hash + space + double quote) is a special sequence that triggers a plug-in handler which in this case is called text with arguments of red,bold. As you can see, without the special text plug-in handler being available (which could turn the previous text red and bold) the area just appears as a normal, but useless, link.

[howardroark]

Hey @commenthol and @SteveBenz I think what you are describing could be better referred to as "Macros". I have seen it asked for out of Markdown countless times. The idea being that there is one reserved syntax for the purpose of building your own custom markup output that your sub group of Markdown users can benefit from. Like UML diagrams! You just build your own SVG/canvas madness that picks up on the HTML output after the fact.

If you are curious, there is a long discussion about the idea on a fork of this repo over here:
jonschlinkert#9

The project itself is being rebuilt under a new name with learnings from this one:
https://github.com/jonschlinkert/remarkable

They plan to get block macros in there as a default and throw in a couple tickets from this very project. You may want to watch it, it has solid admins behind it :)

[jcayzac]

Hi there,

There were lots of good comments here, highlighting use cases that weren't taken into account by my code, and suggesting alternative mechanisms for extending Markdown.

I myself believe this might be out of scope for a Markdown processor after all, and more in line with what's expected of a templating engine —I'm now using Pug as my document format, which is extensible and allows inline Markdown too.

I'm closing this pull request, as it's old and has failed to get consensus.