For an extra five minutes learning you get a boatload (think container ship) more features[1] - it compiles to DocBook: a mature, actually standardised, highly structured format, and from that you get HTML, EPUB, PDF, slideshows, and man pages for free.[0] For math you get MathML, ASCIIMath, and LaTeX (along with a number of ways to render them.) It has a super nice syntax, is equally good at little docs and huge books, and you could theoretically write a proper academic paper in it with the LaTeX backend. And you always know what's going to happen when you try to mix bold and italic...
Also endorsed by Linus.[2]
[0]: http://www.methods.co.nz/asciidoc/#_overview_and_examples
[1]: Well, five minutes to be able to do everything Markdown can do; everything else will take a bit longer
[2]: https://plus.google.com/+LinusTorvalds/posts/X2XVf9Q7MfV (comments)Edit: I was wrong about Github not supporting AsciiDoc
I don't have much experience with AsciiDoc, but I've encountered other limitations in writing pandoc's AsciiDoc renderer. For example, I didn't see any way to include multiple paragraphs (or other block content) in an AsciiDoc footnote. That's a deal-breaker for many academics.
I also want to emphasize something that is often not mentioned in comparisons of Markdown and AsciiDoc. As John Gruber emphasizes in his Markdown documentation, Markdown emphasizes ease of reading in source format. AsciiDoc has different priorities, and it sacrifices the readability of the source document to get them. Here's an example involving nested lists, from the AsciiDoc manual:
1. List item one.
+
List item one continued with a second paragraph followed by an
Indented block.
+
.................
$ ls *.sh
$ mv *.sh ~/tmp
.................
+
List item continued with a third paragraph.
2. List item two continued with an open block.
+
--
This paragraph is part of the preceding list item.
a. This list is nested and does not require explicit item continuation.
+
This paragraph is part of the preceding list item.
b. List item b.
This paragraph belongs to item two of the outer list.
--
Markdown equivalent: 1. List item one.
List item one continued with a second paragraph followed by an
Indented block.
$ ls *.sh
$ mv *.sh ~/tmp
List item continued with a third paragraph.
2. List item two continued with an open block.
This paragraph is part of the preceding list item.
1. This list is nested and does not require explicit item continuation.
This paragraph is part of the preceding list item.
2. List item b.
This paragraph belongs to item two of the outer list.
Note that the AsciiDoc is at least as easy to write, perhaps easier, because you don't need to worry about indentation. But the Markdown source is more readable; the indentation makes clear the structure of the list in a way that mirrors how it would be displayed in a browser or on the page.Markdown's syntax constrains how much it can ever do, whereas AsciiDoc is designed to be extensible, providing macros at the "language" level and underneath that there's the whole DocBook toolchain: it seems to me that the clever move is leverage all this, rather than create yet-another-Markdown (and it's not like the syntax tradeoff is at all dramatic.) I think the footnote issue you mention can be done in block macros, but I'm not sure.
I might be getting a bit utopian here, but what would be really great is if all these markup languages sought to have their canonical implementation in pandoc, which would allow for the standardisation of a set of pass-throughs/filters/annotations for things like equations and citations.
Thanks for pandoc, btw. Just out of interest, is there a reason you haven't attempted an AsciiDoc reader? (Or have you?) Assuming it'd be quite a bit more work than the others, what with macros and so forth.
You stated about the footnote support being a deal-breaker for many academics, the poor equation support is a deal-breaker for me. Most of what I write is full of equations, especially aligned equations. It's necessary to reference a specific equation that might be on a different page (or far above it on an HTML page).
It looks like Asciidoc has different ways of handling LaTeX equations for non-LaTeX output. It can render a PNG, it can use a Javascript solution, or it can output MathML.
In addition to default asciidoctor style, there are other themes available - http://themes.asciidoctor.org/preview/ (See bottom right for theme switcher).
You can also take a look at Pro Git book styles - http://git-scm.com/book/en/v2 which was written in asciidoc - https://github.com/progit/progit2
Both Common Mark and Pandoc serve different purposes: the first is an initiative to counter fragmentation/balkanization of the Markdown ecosystem and in being so has to reckon with backward compatibility, consensus and adoption. The latter is a document conversion library, which, by design, needs to reckon with interchangeability between formats and may hence be hampered by the lowest common denominator as regards feature support. Internally, Pandoc keeps an Abstract Syntax Tree (accessible in json format), and defaults on its own flavour of Pandoc Markdown (featuring a Markdown superset of content element types).
Both the Common Mark community and John MacFarlane have made it clear their first and foremost focus is on standardization, not so much on extending the feature set. Yet, scholars and technical writers are in dire need for something more heavy-weight than the rather small set of features offered by Common Mark implementations or Pandoc Markdown. Hence the Scholarly Markdown initiative and the scholdoc reference implementation (Pandoc fork).
More on how Scholarly Markdown came about, can be read on the blog of one of it’s pacemakers, Martin Fenner; e.g. http://blog.martinfenner.org/2013/11/17/the-grammar-of-schol..., http://blogs.plos.org/mfenner/2012/12/18/additional-markdown...
CommonMark is currently focused on the task of giving a decent spec for the core syntax and robust, efficient implementations; extensions will wait til that project is done, but certainly aren't ruled out.
Pandoc has always been in the game of extending the feature set. Here are just some of the Markdown extensions pandoc supports: LaTeX math (which can be rendered in a variety of formats, including native Word and MathML), LaTeX macros, inline LaTeX, automatically numbered examples and cross-references to these, automatically generated citations (using CSL styles), super and subscripts, strikeout, figures, YAML metadata, definition lists, several styles of tables, fenced code blocks with syntax highlighting, header identifiers, and footnotes. scholdoc just adds a few things on top of all this (and many of them could be implemented in pandoc filters). As noted in one of the other comments on this thread, most of the features scholdoc adds are under active discussion in pandoc as well. So it's not that pandoc and scholdoc have different aims; pandoc just moves more slowly, because it has to worry about how features are implemented in many more output formats, and it operates under some other constraints that scholdoc rejects (e.g. trying to avoid the use of English words like "Figure" for syntax cues).
Basically this is forked because I've been wanting to see how much change to the AST is needed to include most of the academic-specific features. Since Pandoc's AST definition is a separate package from Pandoc itself and could possibly be a dependency of other projects, I thought it would be best to figure most of it out first and end up with just one proposal. Scholdoc has a much more limited number of input/output syntax, so it has much more flexibility when it comes to adding new document element types.
Consider this a self-motivated skunkworks project for Pandoc.
Internal referencing and attributes on figures are two things that are currently being discussed for pandoc. The discussion has been going on for quite a while though - hence people making forks.
Discussion on internal referencing: https://github.com/jgm/pandoc/issues/813
Discussion on image attributes: https://github.com/jgm/pandoc/issues/261
This raises a couple more questions for me.
First, when searching I was able to find Martin Fenner's very interesting blog posts about ideas for a "Scholarly Markdown" and, as those issues and the first link on the scholarlymarkdown.com site reference, he appears to be associated with a separate "scholmd" project, also called "Scholarly Markdown," which is apparently a related project that itself is a fork of the Python markdown science project:
scholmd:
Markdown Science:
https://github.com/karthik/markdown_science
However, it's unclear what all of the relationships are between all of these projects and forks.
Secondly, since some (or all?) of the changes are being discussed in the Pandoc issue tracker, are these changes intended to be submitted to Pandoc in pull requests? I don't currently see any.
(except for really simple cases)
https://raw.githubusercontent.com/softcover/softcover_book/m...
The beautiful idea in markdown is that it allows you to mix (non-container) HTML tags in with the .md and it just works. Softcover markdown is in the same vein, allowing the more readable markdown for main copy, and intermix LaTeX tags as required. Beautiful if you ask me. Or at least beautifuler than ```math ... ```math.
The "backward compatibility" of ScholarlyMarkdown with basic markdown is a cool feature as many tools/plarforms exist that "support" .md now, but to preview you'll still need something that renders the equations, so strictly speaking ScholarlyMarkdown is a new markup langauge.
For example, they do not plan to add syntax highlighting blocks (the ```some code``` on GitHub) to their implementation, because they believe that it is outside the scope of markdown. Then, because a lot of people actually need this feature, they still have to patch or extend or plug-in the functionality into any implementation of CommonMark, leading to fragmentation again. And frankly, the reason that I start writing more markdown is precisely because of the syntax highlighting ability. Oh, and to write scientific articles, math formulas are a deal-breaker.
They want to have a standard, unambiguous syntax specification, a suite of comprehensive tests and a cleanly implemented parser. They want to unify the community of markdown users and developers. All of those are commendable goals. But at the end of the day, it doesn't satisfy my needs, so I'd rather use a messy, poorly specified markdown flavor or even just render the markdown with GitHub's service.
To correct the record, fenced code blocks have been there from the beginning: http://spec.commonmark.org/0.18/#fenced-code-blocks.
Lastly, why is everyone hellbent on following the original syntax like it's the word of God? Anytime I hear someone complain about an issue in Markdown, people point to the 2004 syntax and how it's true to spec. Users don't care, they want something usable. I'd be happy if someone creates a completely new spec, and I never hear the word Markdown again. I'm tired of the same issues being passed down.
That's what LaTeX does too. Its markup, generally, is semantic.
As a side note, a lot of Markdown + LaTeX + Code can be done in IPython Notebook. (Though, there are some things absent, like referencing citations or other equations).
https://github.com/timtylin/scholdoc-examples
More will be added as I tweak the syntax. My hidden goal is to add enough to ScholarlyMarkdown to be able to reproduce around 70% of ArXiv.
http://www.dllu.net/programming/dllup/
which handles math using svgtex instead of clientside MathJax for faster rendering. It also compiles to both html5 and LaTeX. However, dllup is overall less polished, missing some features (labelling equations, sections).
Babel and Org have clunky syntax but the mode takes care of that. The combination isn't perfect but it's very powerful, in particular the ability to chain several languages together in flexible ways.
Coming up with more ways to combine pretty formatting with syntax highlighting is polishing a pretty smooth surface. I'm dreaming of a really slick syntax and operating environment, like babel if it wasn't tacked onto org. In the meantime, using what we have.
Of course, having spent the past few months writing an app that basically reinvents this same wheel (http://www.eqeditor.com/writer/) I suppose I'm in good company.
I'm imagining some kind of online component like ShareLaTeX that will become a clearinghouse for a number of tested and proven conversion paths, and can handle compiling a ScholarlyMarkdown document to different formats. This project won't go anywhere if this can't at least be done for major houses like Elsevier, PNAS, Phys Rev, etc.
There should be a nice ScholarlyMarkdown -> LaTeX cross compiler for starting simple documents in TeX and then sharing with collaborators (pick some obvious defaults or allow a config file to get fancy). But more importantly, if I am joining a project that already has a bunch of LaTeX wizardry going on, I should be able to seamlessly and implicitly edit the text parts in ScholarlyMarkdown without my collaborators knowing.
Can ScholarlyMarkdown do this? If not then I'm not really interested.
I'm currently using a workflow in my thesis where I use ScholarlyMarkdown to write individual chapters for final inclusion into an existing LaTeX book document. I find that ScholarlyMarkdown works quite well this way, and it potentially allows collaborators, since individual parts are isolated.
I'd love a tool that works with this philosophy, and I feel certain anything like ScholarlyMarkdown won't catch on in my field (theoretical computer science) without such tools.
As you probably know, TeX is a full programming language (albeit an odd one), which means that to compile (La)TeX to Markdown, Markdown needs to support all the features of TeX, which means that it has to be Turing-complete. Not to mention the fact that parsing TeX is kind of nuts too.
If anyone edits the TeX to fall outside that sub-set, it's reasonable to just embed those parts verbatim in the resulting Markdown.
I built something similar to that, but instead of forking Pandoc (and investing the time to learn Haskell), I just added filters through Python. It's not at fast, but you can intercept mostly anything.
Thus, the flow ends up as:
paper.md -> Pandoc -> paper.json + metadata -> paper.tex -> paper.pdf
paper.tex -> paper.md + metadata ->(merge) paper.tex -> paper.pdf
Edit: yes, I was missing something. Scholdoc supports MathJax (which has browser, Office, and LaTeX support).
Small issue: The title isn't showing up in Firefox (or Chrome) because there are two title elements and the first one is blank.
It's more difficult but I feel like there's a lot more potential.
Additionally there is a very stable, performant and flexible reference implementation implemented as a web app (mediawiki) with excellent import/export in XML format (and the list goes on ...), versioning with syntax-highlighted diff, etc etc etc.
What have I missed? :)
Nobody likes MediaWiki syntax, not even its own users. It's awful. The only reason it's still used at Wikipedia is because there's too much content to reasonably convert.
=> built