Lexer Listings

6.5

Lexer Listings

Tero Hasu

This library implements something similar to scribble/manual module’s codeblock, codeblock0, code, and typeset-code. However, the typesetting forms and functions here assume foreign code, and no #lang to specify the language used. Instead, one must explicitly specify a color lexer function to use, one with the same signature as that of default-lexer and racket-lexer.

This package presently includes only one lexer, namely the cxx-lexer in the lexer-listings/cxx module.

1 Language-Agnostic API

(require lexer-listings)

package: lexer-listings

The lexcodeblock, lexcodeblock0, lexcode, and typeset-lexcode forms are used for typesetting foreign code, with a lexer explicitly specified as the first argument.

syntax
(lexcode lexer-expr str-expr ...+)

str-expr : string?

Similar to code, but parses code from strings into the inline text of the document using the specified lexer. The str-expressions must yield strings of code to be tokenized.

For example,

This is @lexcode[cxx-lexer]{1 + 2}.

produces the typeset result:

This is 1 + 2.

syntax
(lexcodeblock lexer-expr str-expr ...+)

str-expr : string?

Similar to codeblock, but parses code from strings into a block in the document using the lexer specified by lexer-expr. The str-expressions must yield strings of code to be tokenized.

For example,

@lexcodeblock[cxx-lexer]|{
  int f(int const& x) {
    if (f <= 1) return 1;
    else        return x*f(x-1);
  }
}|

produces the typeset result:

int f(int const& x) {
if (f <= 1) return 1;
else return x*f(x-1);
}

syntax
(lexcodeblock0 lexer-expr option ... str-expr ...+)

Similar to codeblock0, but uses the specified lexer.

procedure
(typeset-lexcode strs ...) → block?
strs : string?

Similar to typeset-code, but uses the specified lexer.

value
foreign-keyword-color : style?
value
foreign-name-color : style?
value
foreign-symbol-color : style?

Additional styles supported by typeset-lexcode (in addition to Racket’s meta-color, value-color, etc.). These are intended to be used for foreign keywords, other predefined names, and non-predefined names, respectively. These styles correspond to the additional token types 'frg-keyword, 'frg-name, 'frg-symbol, as may be assigned by a language-specific color lexer.

When rendering a document, the styles translate as FrgKw, FrgName, and FrgSym. If tokens of the associated types are emitted, definitions for those style classes must then be provided in a renderer-specific way. See Manual CSS Style Classes for a list of Racket manual style classes, which can serve as examples for providing definitions for FrgKw, etc. See this package’s "style.css" for an example definition for CSS.

2 Acknowledgements

This document and parts of the software are based on Leif Andersen’s work on the r-lexer package. The rest of the software is based on code from Racket’s scribble-lib package.