Core Block Tokens

These block tokens are defined in the CommonMark specification.

Document

class mistletoe.block_tokens.Document(*, children: List[mistletoe.base_elements.Token], link_definitions: dict, front_matter: Optional[mistletoe.block_tokens.FrontMatter] = None)[source]

Bases: mistletoe.base_elements.BlockToken

Document container.

Parameters
classmethod read(lines, start_line: int = 0, reset_definitions=True, store_definitions=False, front_matter=False)[source]

Read a document

Parameters
  • lines – Lines or string to parse

  • start_line – The initial line (used for nested parsing)

  • reset_definitions – remove any previously stored link_definitions

  • store_definitions – store LinkDefinitions or ignore them

  • front_matter – search for an initial YAML block front matter block (note this is not strictly CommonMark compliant)

Atx Heading

class mistletoe.block_tokens.Heading(*, level: int, children: List[mistletoe.base_elements.Token], position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Heading token. ([“### some heading ###n”])

Boundary between span-level and block-level tokens.

Parameters
classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines, expand_spans=False)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

Setext Heading

class mistletoe.block_tokens.SetextHeading(*, level: int, children: List[mistletoe.base_elements.Token], position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Setext headings.

Not included in the parsing process, but returned by Paragraph.read.

Parameters
classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

Paragraph

class mistletoe.block_tokens.Paragraph(*, children: List[mistletoe.base_elements.Token], position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Paragraph token. ([“somen”, “continuousn”, “linesn”])

Boundary between span-level and block-level tokens.

Parameters
static start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines, expand_spans=False)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

Quote

class mistletoe.block_tokens.Quote(*, children: List[mistletoe.base_elements.Token], position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Quote token. ([“> # headingn”, “> paragraphn”]).

Parameters
static start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

BlockCode

class mistletoe.block_tokens.BlockCode(*, children: List[mistletoe.base_elements.Token], language: str = '', position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Indented code.

Parameters
  • children (List[mistletoe.base_elements.Token]) – Child tokens list

  • language (str) – The code language (for sytax highlighting) (default: “”)

  • position (Tuple[int, int]) – Line position in source text (start, end)

static start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

CodeFence

class mistletoe.block_tokens.CodeFence(*, children: List[mistletoe.base_elements.Token], language: str = '', arguments: str = '', position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Code fence. ([“`sh\n", "rm -rf /", ..., "`”])

Boundary between span-level and block-level tokens.

Parameters
  • children (List[mistletoe.base_elements.Token]) – Child tokens list

  • language (str) – The code language (for sytax highlighting) (default: “”)

  • arguments (str) – Any string occuring after the language (default: “”)

  • position (Tuple[int, int]) – Line position in source text (start, end)

classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

List

class mistletoe.block_tokens.List(*, children: List[mistletoe.base_elements.Token], loose: bool, start_at: Optional[int], position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

List token (unordered or ordered)

Parameters
  • children (List[mistletoe.base_elements.Token]) – Child tokens list

  • loose (bool) – Whether list items are separated by blank lines

  • start_at (int, NoneType) – None if unordered, starting number if ordered.

  • position (Tuple[int, int]) – Line position in source text (start, end)

classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

ListItem

class mistletoe.block_tokens.ListItem(*, children: List[mistletoe.base_elements.Token], loose: bool, leader: str, prepend, next_marker, position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

List items.

Not included in the parsing process, but called by List.

Parameters
  • children (List[mistletoe.base_elements.Token]) – Child tokens list

  • loose (bool) – Whether list items are separated by blank lines

  • leader (str) – The prefix number or bullet point.

  • prepend (Any) –

  • next_marker (Any) –

  • position (Tuple[int, int]) – Line position in source text (start, end)

classmethod parse_marker(line)[source]

Returns a pair (prepend, leader) if the line has a valid leader.

classmethod read(lines, prev_marker=None)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

LinkDefinition

class mistletoe.block_tokens.LinkDefinition(*, definitions: list, position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

LinkDefinition token: [ref]: url “title”

Parameters
  • definitions (list) – list of (label, dest, title)

  • position (Tuple[int, int]) – Line position in source text (start, end)

classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

ThematicBreak

class mistletoe.block_tokens.ThematicBreak(*, position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Thematic break token (a.k.a. horizontal rule.)

Parameters

position (Tuple[int, int]) – Line position in source text (start, end)

classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.

HTMLBlock

class mistletoe.block_tokens.HTMLBlock(*, content: str, position: Tuple[int, int])[source]

Bases: mistletoe.base_elements.BlockToken

Block-level HTML token.

Parameters
  • content (str) – literal strings rendered as-is

  • position (Tuple[int, int]) – Line position in source text (start, end)

classmethod start(line)[source]

Takes a line from the document as argument, and returns a boolean representing whether that line marks the start of the current token. Every subclass of BlockToken must define a start function (see block_tokenizer.tokenize_main).

classmethod read(lines)[source]

takes the rest of the lines in the document as an iterator (including the start line), and consumes all the lines that should be read into this token.

The default is to stop at an empty line.