Welcome to pyparsing-highlighting’s documentation!

• View on GitHub

Syntax highlighting with pyparsing, supporting both HTML output and prompt_toolkit–style terminal output. The PPHighlighter class can also be used as a lexer for syntax highlighting as you type in prompt_toolkit. It is compatible with existing Pygments styles.

Requirements

• Python 3.5+
• prompt_toolkit 2.0+
• Pygments
• pyparsing

Installation

pip3 install -U pyparsing-highlighting

Or, after cloning the repository on GitHub:

python3 setup.py install

Examples

The following code demonstrates the use of PPHighlighter:

from pp_highlighting import PPHighlighter
from prompt_toolkit.styles import Style
import pyparsing as pp
from pyparsing import pyparsing_common as ppc

def parser_factory(styler):
    a = styler('class:int', ppc.integer)
    return pp.delimitedList(a)

pph = PPHighlighter(parser_factory)
style = Style([('int', '#528f50')])
pph.print('1, 2, 3', style=style)

This prints out the following to the terminal:

The following code generates HTML:

pph.highlight_html('1, 2, 3')

The output is:

<pre class="highlight"><span class="int">1</span>, <span class="int">2</span>, <span class="int">3</span></pre>

There is also a lower-level API—pph.highlight('1, 2, 3') returns the following:

FormattedText([('class:int', '1'), ('', ', '), ('class:int', '2'), ('', ', '), ('class:int', '3')])

A FormattedText instance can be passed to prompt_toolkit.print_formatted_text(), along with a Style mapping the class names to colors, for display on the terminal.

PPHighlighter can also be passed to a prompt_toolkit.PromptSession as the lexer argument, which will perform syntax highlighting as you type. For examples of this, see examples/calc.py, examples/json_pph.py, and examples/sexp.py. The examples can be run by (from the project root directory):

python3 -m examples.calc
python3 -m examples.json_pph
python3 -m examples.sexp

Testing

(From the project root directory):

To run the unit tests:

python3 -m unittest

To run the regression benchmark:

python3 -m tests.benchmark

Module pp_highlighting

Syntax highlighting for prompt_toolkit and HTML with pyparsing.

pp_highlighting.dummy_styler = <pp_highlighting.pp_highlighter.DummyStyler object>

DummyStyler – An importable instance of DummyStyler to pass to parser factories.

class pp_highlighting.DummyStyler

Bases: pp_highlighting.pp_highlighter.Styler

A drop-in replacement for Styler which, when called, merely returns a copy of the given parse expression without capturing text or applying styles. To aid in testing whether a parser factory has been passed a DummyStyler object, bool(DummyStyler()) is False.

__call__(style, expr)

Returns a copy of the given parse expression.

Parameters:

• style (Union[pygments.token.Token, str]) – Ignored.
• expr (Union[pyparsing.ParserElement, str]) – Copied, unless it is a string literal, in which case it will be wrapped by pyparsing.ParserElement._literalStringClass (default pyparsing.Literal).

Returns:

pyparsing.ParserElement – A copy of the input parse expression.

class pp_highlighting.PPHighlighter(parser_factory, *, uses_pygments_tokens=False)

Bases: prompt_toolkit.lexers.base.Lexer

Syntax highlighting for prompt_toolkit and HTML with pyparsing.

This class can be used to highlight text via its highlight() method (for prompt_toolkit.print_formatted_text()—see the prompt_toolkit documentation for details), its highlight_html() method, its print() method, and by passing it as the lexer argument to a prompt_toolkit.PromptSession.

__init__(parser_factory, *, uses_pygments_tokens=False)

Constructs a new PPHighlighter.

You should supply a parser factory, a function that takes one argument and returns a parse expression. PPHighlighter will pass a Styler object as the argument (see Styler for more details).

Examples

>>> def parser_factory(styler):
>>>     a = styler('class:int', ppc.integer)
>>>     return pp.delimitedList(a)
>>> pph = PPHighlighter(parser_factory)
>>> pph.highlight('1, 2, 3')
FormattedText([('class:int', '1'), ('', ', '), ('class:int', '2'),
('', ', '), ('class:int', '3')])

FormattedText instances can be passed to prompt_toolkit.print_formatted_text().

Parameters:

• parser_factory (Callable[[Styler], pyparsing.ParserElement]) – The parser factory.
• uses_pygments_tokens (bool) – Whether or not the parser is styled using Pygments tokens.

highlight(s)

Highlights a string, returning a list of fragments suitable for prompt_toolkit.print_formatted_text().

Parameters:s (str) – The input string. Returns:prompt_toolkit.formatted_text.FormattedText – The resulting list of prompt_toolkit text fragments.

lex_document(document)

Takes a Document and returns a callable that takes a line number and returns a list of (style_str, text) tuples for that line.

XXX: Note that in the past, this was supposed to return a list

of (Token, text) tuples, just like a Pygments lexer.

highlight_html(s, *, css_class='highlight')

Highlights a string, returning HTML.

Only CSS class names are currently supported. Parts of the style string that do not begin with class: will be ignored. If there are dots in the class name, they will be turned into hyphens.

Parameters:

• s (str) – The input string.
• css_class (str) – The CSS class for the wrapping tag.

Returns:

str – The generated HTML.

print(*values, **kwargs)

Highlights and prints the values to a stream, or to sys.stdout by default. It calls prompt_toolkit.print_formatted_text() internally and takes the same keyword arguments as it (compatible with the builtin print()).

Default values of keyword-only arguments:

print(*values, sep=' ', end='\n', file=None, flush=False,
      style=None, output=None, color_depth=None,
      style_transformation=None,
      include_default_pygments_style=None)

class pp_highlighting.PPValidator(expr, *, multiline=True, move_cursor_to_end=False)

Bases: prompt_toolkit.validation.Validator

A prompt_toolkit Validator for pyparsing.

__init__(expr, *, multiline=True, move_cursor_to_end=False)

Constructs a new PPValidator.

Parameters:

• expr (pyparsing.ParserElement) – The parser to use for validation.
• multiline (bool) – Whether to include the line number in the error message.
• move_cursor_to_end (bool) – Whether to move the cursor to the end of the input if a non-pyparsing exception was raised during parsing.

validate(document)

Validate the input. If invalid, this should raise a ValidationError.

Parameters:document – Document instance.

class pp_highlighting.Styler

Bases: object

Wraps pyparsing parse expressions to capture styled text fragments.

__init__()

Initialize self. See help(type(self)) for accurate signature.

__call__(style, expr)

Wraps the given parse expression to capture the original text it matched, and returns the modified parse expression. The style argument can be either a prompt_toolkit style string or a Pygments token.

Parameters:

• style (Union[pygments.token.Token, str]) – The style to set for this text fragment, as a string or a Pygments token.
• expr (Union[pyparsing.ParserElement, str]) – The pyparsing parser to wrap. If a literal string is specified, it will be wrapped by pyparsing.ParserElement._literalStringClass (default pyparsing.Literal).

Returns:

pyparsing.ParserElement – The wrapped parser.

clear()

Removes all captured styled text fragments.

delete(loc)

Removes the styled text fragment starting at a given location if it exists.

Parameters:loc (int) – The styled text fragment to delete’s start location.

get(loc)

Returns the styled text fragment starting at a given location if it exists, else None.

Parameters:loc (int) – The styled text fragment’s start location. Returns:Optional[Tuple[Union[pygments.token.Token, str], str]] – The styled text fragment, if it exists.

locs()

Returns a sorted list of styled text start locations.

Returns:List[int] – A sorted list of styled text start locations.

Indices and tables

• Index
• Module Index