Myst Markdown – Markdown for technical/scientific document | Hacker News Reader

Myst Markdown – Markdown for technical/scientific document

mystmd.org

195 pointsgmgn44 comments

As a notorious Markdown critic, I have to say, their syntax really looks like a decent attempt of support for technical and scientific documents. Having used reStructuredText for a thesis with a lot of bells and whistles, and having used org-mode mostly ever since, I am not just admitting it willy-nilly. Then of course some of its syntax and syntax concepts are straight out of reStructuredText's book, so it makes sense, that it would inherit the capabilities as well.

What I haven't seen at a quick glance is the ability to link to any place in the document (sink) from withing the document itself (source). Maybe I did not look far enough in the examples. reStructuredTex can do this, and org-mode can do that too. And I don't mean just linking to existing headings. The other required things like footnotes and citations and so on seem to be there. Now if this reaches mainstream and we start writing documentation in it, there is great potential to have finally good documentation, not limited by the likes of github markdown. Of course same is true for reStructuredText and org-mode. From that perspective, one could ask, why yet another format was necessary, rather than improving tooling for lets say reStructuredText.

Org-mode has a new maintainer, and he has invited contributions of source code:

https://emacsconf.org/2024/talks/org-update/

Yeah, I have seen the talk! He already reviewed some change I once sent and some bug report. Was very responsive and also gave me hints for what part I would need to change to make a specific thing happen. Very helpful.

> there is great potential to have finally good documentation

Yes, true, no good documentation has ever been written in Markdown. Good point.

Most Markdown dialects will support some sort of label or 'span' naming, which provides link targets (if only by dropping down to literal HTML <span>/<div> elements), and it looks like this does too: https://mystmd.org/guide/cross-references#label-anything

> What I haven't seen at a quick glance is the ability to link to any place in the document (sink) from withing the document itself (source).

You can create an anchor:

    (my-anchor)=

...and then link to it:

    [see here](#my-anchor)

(https://mystmd.org/guide/cross-references)

This is a lovely comment to read, thank you.

The only implementation of restructured text is the docutils-Sphinx ecosystem. MyST Markdown was designed as a nicer language for Markdown-familiar authoring: https://executablebooks.org/en/latest/blog/2020-08-07-announ...

In the myst-md project, we're applying many of the lessons that we learned in the Jupyter Book project. One of the big takeaways is that the tooling is equally as important as the underlying markup, and docutils/Sphinx was holding us back in a few areas.

If you like links within documents, take a look at embedding (https://mystmd.org/guide/embed) and cross-referencing (https://mystmd.org/guide/cross-references)

One of the most powerful features that I pushed the MyST team to incorporate is xref (linking External MyST projects via borrowed syntax from ReStructuredText) https://mystmd.org/guide/external-references#myst-xref Which allows supplying a list of external references, and complete metadata: https://mystmd.org/guide/website-metadata#myst-xref-json

No relation to the best-selling 90's PC game "Myst", in case anyone was wondering like I was.

I clicked the link thinking I would see some Myst imagery. Nope. Very disappointed.

Indeed. Which makes it very hard to google.

I miss playing that game. My father bought the special edition which came with a journal that you could write your notes in it. And we played together taking notes and figuring out the puzzles.

I really enjoyed the recent VR remaster re-release of Riven.

I don’t have a vr device. Is it in normal mode?

Yes, you can play the rerelease of Riven without VR. I did that a few months ago.

I was about to warn that this app may be haunted by Sirrus and Achenar. You don't take the name of Myst in vain.

I've really had a pet peeve about footnotes on mobile and this does the crime. If you click a footnote and it jumps very far away and doesn't have a return navigation, I have left your page usually after the second time I see a footnote and didn't note my exact scroll. A small thing but pretty please don't make me have to hunt for where I was...

Not me mining HN for feature requests... ;)

Oh, it turns out we already do this! A small bugfix to make these always-visible on mobile displays :)

https://github.com/jupyter-book/myst-theme/issues/513

Nice! Glad to see. Huge quality of life improvement

After reading the introductory page, I don't see any benefit over Quarto, and in fact some limitations in comparison.

I was going to ask about comparison with quarto. I just managed to get a good quarto setup with typst to typeset documents and I'm curious about this alternative.

I'd use myst for docs, it's simpler, lighter, and interlinks well.

Quarto is more for publications.

MyST-MD is also aimed at publications! See the SciPy proceedings for some examples: https://proceedings.scipy.org/2024

Quarto rendering is quite slow compared to Myst. It is more focused on R Markdown and Observable JS.

The main _technical_ difference between the two tools is the underlying approach to how we handle ASTs. MyST-MD leverages the unified-js tooling around AST handling, whereas Quarto uses Pandoc.

For UX, we think that MyST has a lot of nice bells and whistles, but we really need to author a blog post or documentation page to clarify the user-facing differences!

This is my favorite syntax for technical writing, but unfortunately the only complete implementation is in Python. There are half-finished JS projects (last I checked) but nothing else. I really wish they were complete so we could use this in more places.

I opted for Djot (https://djot.net) as a second-place alternative with support for many more languages. My use case was attempting a Sphinx alternative in TypeScript. (Link for the curious: https://steveasleep.com/djockey/)

MyST-MD is intended to be

> a Sphinx alternative in TypeScript.

We have some unfortunate naming that follows from history. The Python tools you're referring to are a distinct project. The Jupyter Book project saw the development of many of those tools, and is now pivotting to a new engine MyST-MD that this article links to!

Why do you say that these tools aren't complete? Whilst we are certainly not "done", these tools are being used in many places, such as https://proceedings.scipy.org/2024

Looking forward to hearing from you!

I can't wait for HTML export for Typst.

I am getting a bit tired of all the markdown flavours and extensions

I note that it provides a different dialect of md compared to Obsidian, especially the syntax for admonitions. If one would stray that far, why not AsciiDoc?

Myst provides an AST-spec, but I doubt it would be easy to integrate that into Obsidian. Maybe other can shed light on it?

Instead of AsciiDoc, it originates from the Sphinx/ReStrucuredText ecosystem. See https://mystmd.org/guide/background

> If one would stray that far, why not AsciiDoc?

And why AsciiDoc in particular? It lists the inspirations for the beyond-Markdown features.

This reminds me of a project I worked on during my PhD, where you create a network of scientific documents and notes/threads via Markdown, with a similarly structured "rabbit-hole" linking system: https://github.com/rkp8000/hypothesize . I'm not really a software engineer so never made it ready for general use, but I'm very happy to see a similar idea turned into something real! Kudos to the authors.

Looks interesting. The name suggests it uses RestructuredText rather than Markdown. Was that the original plan?

Is it appropriate for distill-like articles?

Their documents say it is a CommonMark superset inspired by Sphinx and reStructuredText.

It's Markdown, but you can use it within the same Sphinx ecosystem as reStructuredText.

I use it for a few of my projects and it's fantastic - https://llm.datasette.io for example uses MyST Markdown: https://github.com/simonw/llm/tree/main/docs

Is there a way to write a myst markdown file for one page, a tex file for another linked page, and have them both compile to html seamlessly? I have been doing this with markdown + pandoc for my web page, and I wonder if there is something richer than plain markdown to use for my non-latex content.

mystmd supports parsing .tex files as well as MyST Markdown. There are some technical limitations here given the difficulty in parsing Markdown, but it's certainly usable.

Is there a visual editor and auto-formatter that would ease the transition for less technical folk? Ideally, it'd be a containerized web service with git integration, where proposed changes go to a branch.

This is such a complicated domain that you have to have some kind of function-call looking syntax for extensibility (compared to basic lightweight markup which doesn’t). So that’s fine.

    {rolename}`and here is my role's content!`

I would personally prefer something like Tex here. You just need backslashes, square brackets and braces.

I guess this is inspired by Restructured Text. Not a fan of that—it looks like the most punctuation-heavy lightweight markup language.

Cool to see that Jupyter is giving MyST a lot of support and backing. Sounds like not only does it now have feature parity with reST but may even be surpassing reST now.

It's in Hugo already.

Do you have a link? I searched and couldn’t find it just two days ago. The only thing I find is that you can achieve similar thing with shortcuts und passthrough.

I’m using Sphinx/Myst/pydata theme for my blog which is a very cool combination. But Hugo seems more established.

https://gohugo.io/content-management/mathematics/

For more complex data visuals there's D3: https://observablehq.com/@d3/gallery

Reply on news.ycombinator.com