Extension Tokens

These tokens do not form part of the core CommonMark specification, but are commonly implemented extended syntax elements.

Token	Description	Example
Strikethrough	a line through the text	~~some text~~
Math	Dollar $ enclosed math. Two $ characters wrap multi-line math.	$a=1$ $b=2$ $$ a=1 $$
FrontMatter	A YAML block at the start of the document enclosed by ---	--- key: value ---
Table TableRow TableCell	Markdown table style, with pipe delimitation, and (optional) colon specified alignment.	| left | centre | right | | :--- | :----: | ----: | | a | b | c |
Footnote FootReference	A definition for a referencing footnote, that will usually be moved to the bottom of the document by a renderer.	Something that needs further explanation.[^ref] [^ref]: Some footnote text

Strikethrough

class mistletoe.span_tokens_ext.Strikethrough(*, content: Optional[str] = None, children: Optional[list] = None, position: Tuple[int, int] = None)

Bases: mistletoe.base_elements.SpanToken

Strikethrough tokens: ~~some text~~.

Must be ordered after CoreTokens in the parsing list.

pattern = re.compile('(?<!\\\\)(?:\\\\\\\\)*~~(.+?)~~', re.DOTALL)

parse_group = 1

parse_inner = True

Math

class mistletoe.span_tokens_ext.Math(*, content: Optional[str] = None, children: Optional[list] = None, position: Tuple[int, int] = None)

Bases: mistletoe.base_elements.SpanToken

Dollar Math tokens (single or double): $a=1$.

Must be ordered after CoreTokens in the parsing list.

pattern = re.compile('(?<!\\\\)(?:\\\\\\\\)*(\\${1,2})([^\\$]+?)\\1')

parse_inner = False

parse_group = 0

FrontMatter

class mistletoe.block_tokens.FrontMatter(*, content: Union[str, dict], position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.BlockToken

Front matter YAML block, on the first line of the document.

---
a: b
c: d
---

NOTE: The content of the block should be valid YAML, but its parsing (and hence syntax testing) is deferred to the renderers. This is so that, given ‘bad’ YAML, the rest of the of document will still be parsed, and then the renderers can apply there own error reporting.

Not included in the parsing process, but called by Document.read, if front_matter=True, and stored on Document.front_matter in the syntax tree.

Parameters

• content (str, dict) – Source text (should be valid YAML)

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

get_data() → dict

Return the de-serialized front matter data (requires pyyaml).

classmethod start(line: str) → bool

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines: mistletoe.base_elements.SourceLines)

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

Table

class mistletoe.block_tokens_ext.Table(*, children: List[mistletoe.block_tokens_ext.TableRow], header: Optional[mistletoe.block_tokens_ext.TableRow] = None, column_align: list, position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.BlockToken

Table token.

Note: header delimiters must be of at least length 3 (—)

Example:

| Left Align  |   Centered  | Right Align   |
| :---        |    :----:   |          ---: |
| Header      | Title       | Here's this   |
| Paragraph   | Text        | And more      |

Parameters

• children (List[mistletoe.block_tokens_ext.TableRow]) – Child tokens list

• header (mistletoe.block_tokens_ext.TableRow, NoneType) – The header row (default: None)

• column_align (list) – align options for columns (left=None (default), center=0, right=1)

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

static split_delimiter(delimiter: str)

Helper function; returns a list of align options.

Parameters

delimiter – e.g.: | :— | :—: | —: |

Returns

a list of align options (None, 0 or 1).

static parse_align(column)

Helper function; returns align option from cell content.

Returns

None if align = left; 0 if align = center; 1 if align = right.

static start(line)

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines: mistletoe.base_elements.SourceLines)

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

TableRow

class mistletoe.block_tokens_ext.TableRow(*, children: List[mistletoe.block_tokens_ext.TableCell], row_align: list, position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.BlockToken

Table row token.

Parameters

• children (List[mistletoe.block_tokens_ext.TableCell]) – Child tokens list

• row_align (list) – align options for columns (left=None (default), center=0, right=1)

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

classmethod read(line, row_align=None, lineno=0, lines: mistletoe.base_elements.SourceLines = None)

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

TableCell

class mistletoe.block_tokens_ext.TableCell(*, children: List[mistletoe.base_elements.Token], align: Optional[int], position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.BlockToken

Table cell token.

Boundary between span-level and block-level tokens.

Parameters

• children (List[mistletoe.base_elements.Token]) – Child tokens list

• align (int, NoneType) – align options for the cell (left=None (default), center=0, right=1)

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

classmethod read(content, align=None, expand_spans=False, lineno=0, lines: mistletoe.base_elements.SourceLines = None)

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

Footnote

class mistletoe.block_tokens_ext.Footnote(*, target: str, children: List[mistletoe.base_elements.Token], position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.BlockToken

Footnote token. (“[^a]: the footnote body”)

As outlined in markdownguide and php-markdown; Footnote definitions can be found anywhere in the document, but footnotes will always be listed in the order they are referenced to in the text (and will not be shown if they are not referenced).

NOTE: currently this only supports single line footnotes, but it is intended that this will eventually support multi-line.

Footnotes are stored in the Document.footnotes in the final syntax tree.

This should be ordered for parsing, before the LinkDefinition token

Parameters

• target (str) – footnote reference target

• children (List[mistletoe.base_elements.Token]) – Child tokens list

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

label_pattern = re.compile('^[ \\n]{0,3}\\[\\^([a-zA-Z0-9#@]+)\\]\\:\\s*(.*)$')

classmethod start(line)

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines: mistletoe.base_elements.SourceLines)

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

FootReference

class mistletoe.span_tokens_ext.FootReference(*, target: str, position: mistletoe.base_elements.Position = None)

Bases: mistletoe.base_elements.SpanToken

Footnote reference tokens. (“[^a]”)

As outlined in markdownguide and php-markdown; When you create a footnote, a superscript number with a link appears where you added the footnote reference. Readers can click the link to jump to the content of the footnote at the bottom of the page.

Unlike, the implementations above, it is allowed to have multiple footnote references per footnote definition.

Must be ordered after CoreTokens in the token parsing list.

Parameters

• target (str) – footnote reference target

• position (mistletoe.base_elements.Position) – Line position in source text (default: None)

pattern = re.compile('^\\[\\^([a-zA-Z0-9#@]+)\\]')

parse_inner = False

parse_group = 0

Core Span Tokens Core Renderers