-[* Dashes, Brackets and Stars -[*

md"
### Pain Points and Potential of Markdown in Pluto

An opinionated 'lightning talk' by Jeremiah Lewis

2021-04-08
"

Pluto.jl is a great tool

But unfortunately it's easy to end up with something that looks like:

0️⃣ Truncated History of v0.x Julia

'Nothing else can solve my problem except Julia' era.

Language

• High performance

• Replace two languages (e.g. Python and C++) with one new one

• Explosion of 'Julia for X' packages, from power grids to differential equations

Community

• Community-driven language design effort 'after the emergence of data science'

• Specialist community with lots of scientific users

• Hype 'spikes' in general data science community which would quickly diminish

1️⃣ Truncated History of Post v1.x Julia

'Why would I need to use Python' era.

Language

• Easy to write code

• Package manager best in class (vs Python, R)

• Ecosystem mature enough to compete on 'just another data science project'

Community

• Subject matter developers come together within ecosystems like Queryverse, JuliaData, SciML

• Increased, durable visibility in general data science community

• Lots of content generation: blog posts, high volume Discourse, active Slack, Twitter community

Two Waves of Adoption, Two Target Audiences

1. Performance Pros

• Significant general computing experience

• Often become Julia package maintainers

• Used to language's 'quirks'

• Sold on Julia's performance advantages

2. Amateur Adopters

• Limited time writing Julia code

• In general, skew towards 'computing' newcomers

• Intrigued by Julia's ease of use

In Pluto, combining text and code with markdown is as easy as $\mathrm{\$}(1+1)$ = 2

...but not so fast...

For example, as reported here, the following code:

md"$(var) value is has newline. But next inline use of $(var) works as expected"

Yields this output:

1

value is has newline. But next inline use of 1 works as expected

$$(1+1)\$(1+1)\$(1+1)$$

(1 + 1)

2 2

$(1+1)$(1 + 1)

Why has progress on Markdown stalled?

• Markdown is part of stdlib and used throughout core language for docs

• Existing features of 'dollar math' and 'dollar interpolation' relied upon by advanced users, but mean that 'intuitive' parsing is at times impossible

• Test suite is not comprehensive, changes to code can have unknown consequences

But Julia manages to elegantly solve far more difficult issues...

Why does Julia struggle with Markdown?

• Julia solves 'two-language problem' by creating a 'two-community' problem

• 'Pros' and 'Adopters' share a single language

• 'Pros' are familiar with (avoiding) quirks, use Julia for comparative performance advantages

• 'Adopters' demand 'ease of use', but often not (yet) in a position to contribute code towards this goal

CommonMark.jl Resolves Markdown Issues

• Consistent, normed Markdown implementation

• Independent spec test suite which is used across many languages beyond Julia

• Dollar math, which is related to many issues and ambiguities is deactivated by default, instead just use ```$math$ !

There are some things CommonMark.jl can solve, for everything else, there's community building...

• Julia solves two-language problem, a technical hurdle

• This creates the 'two-communities sharing one language problem', a human one

• In order to maximize the potential of Julia's technical advances, it will be necessary to align the interests of the different Julia adopters