= HIGHLIGHT PLUGIN MANUAL
André Simon
v3.59, October 2020
:lang: en
:toc: left
:toc-title: Contents
:toclevels: 4
:sectnums:
:sectnumlevels: 2
:sectanchors:
// Cross References:
:xrefstyle: short
:section-refsig: Sect.
// Misc Settings:
:experimental: true
:icons: font
:linkattrs: true
// GitHub Settings to enable Admonitions Icons in preview:
ifdef::env-github[]
:caution-caption: :fire:
:important-caption: :heavy_exclamation_mark:
:note-caption: :information_source:
:tip-caption: :bulb:
:warning-caption: :warning:
endif::[]
// =====================================
// Custom Attributes for Reference Links
// =====================================
// Folders:
:plugins: pass:q[link:plugins/[`plugins/`^]]
// Plugins files:
:bash_functions_lua: pass:q[link:plugins/bash_functions.lua[`bash_functions.lua`^]]
:cpp_ref_cplusplus_com_lua: pass:q[link:plugins/cpp_ref_cplusplus_com.lua[`cpp_ref_cplusplus_com.lua`^]]
:ctags_html_tooltips_lua: pass:q[link:plugins/ctags_html_tooltips.lua[`ctags_html_tooltips.lua`^]]
:outhtml_codefold_lua: pass:q[link:plugins/outhtml_codefold.lua[`outhtml_codefold.lua`^]]
:outhtml_curly_brackets_matcher_lua: pass:q[link:plugins/outhtml_curly_brackets_matcher.lua[`outhtml_curly_brackets_matcher.lua`^]]
:outhtml_keyword_matcher_lua: pass:q[link:plugins/outhtml_keyword_matcher.lua[`outhtml_keyword_matcher.lua`^]]
:theme_invert_lua: pass:q[link:plugins/theme_invert.lua[`theme_invert.lua`^]]
== ABOUT
The plug-in interface allows modifications of syntax parsing, colouring and
the document's header and footer.
A common task would be to define a new set of syntax keywords, or to add items
to existing keyword groups. More advanced plug-ins add tooltips based on ctags
input or source code folding to the output (see <<EXAMPLES>>).
== SCRIPT STRUCTURE
The following script contains the basic plug-in structure which has no effect
on the output:
[source,lua]
--------------------------------------------------------------------------------
Description="Plugin Boilerplate"
-- optional parameter desc: syntax description
function syntaxUpdate(desc)
end
function themeUpdate(desc)
end
function formatUpdate(desc)
end
Plugins={
{ Type="theme", Chunk=themeUpdate },
{ Type="lang", Chunk=syntaxUpdate },
{ Type="format", Chunk=formatUpdate },
}
--------------------------------------------------------------------------------
The first line contains a description which gives a short summary of the
plug-in effects.
The next lines contain function definitions:
* The `syntaxUpdate` function is applied on syntax definition scripts (`+*.lang+`),
whereas the `themeUpdate` function is applied on colour themes (`+*.theme+`).
* The `formatUpdate` is not executed in a syntax or theme Lua state.
It just returns strings to override or enhance the document header and footer.
* The `desc` parameter contains the description string of the loaded syntax
definition or colour theme. This information can be used to restrict
modifications to certain kinds of input (i.e. only C code should be
enhanced with syslog keywords).
Finally the `Plugins` array connects the functions to the Lua states which
are created when highlight loads a lang or theme script.
In this example, `themeUpdate` is connected to the theme state, and `syntaxUpdate`
to the lang state.
It is not required to always define both connections if your plugin only affects
one of the config script types.
== SYNTAX CHUNK ELEMENTS
The following list includes all variables and constants you may use to adjust
the parser's syntax highlighting.
Syntax definition items:
[horizontal]
`Comments` :: table
`Description` :: string
`Digits` :: string
`EnableIndentation` :: boolean
`Identifiers` :: string
`IgnoreCase` :: boolean
`Keywords` :: table
`NestedSections` :: table
`Operators` :: string
`PreProcessor` :: table
`Strings` :: table
Document modification items:
[horizontal]
`HeaderInjection` :: string
`FooterInjection` :: string
Read only (internal highlighting states):
[horizontal]
`HL_STANDARD` :: number
`HL_BLOCK_COMMENT` :: number
`HL_BLOCK_COMMENT_END` :: number
`HL_EMBEDDED_CODE_BEGIN` :: number
`HL_EMBEDDED_CODE_END` :: number
`HL_ESC_SEQ` :: number
`HL_ESC_SEQ_END` :: number
`HL_IDENTIFIER_BEGIN` :: number
`HL_IDENTIFIER_END` :: number
`HL_KEYWORD` :: number
`HL_KEYWORD_END` :: number
`HL_LINENUMBER` :: number
`HL_LINE_COMMENT` :: number
`HL_LINE_COMMENT_END` :: number
`HL_NUMBER` :: number
`HL_OPERATOR` :: number
`HL_OPERATOR_END` :: number
`HL_PREPROC` :: number
`HL_PREPROC_END` :: number
`HL_PREPROC_STRING` :: number
`HL_STRING` :: number
`HL_STRING_END` :: number
`HL_UNKNOWN` :: number
`HL_REJECT` :: number
Read only (output document format):
[horizontal]
`HL_OUTPUT` :: number (selected format)
`HL_FORMAT_HTML` :: number
`HL_FORMAT_XHTML` :: number
`HL_FORMAT_TEX` :: number
`HL_FORMAT_LATEX` :: number
`HL_FORMAT_RTF` :: number
`HL_FORMAT_ANSI` :: number
`HL_FORMAT_XTERM256` :: number
`HL_FORMAT_TRUECOLOR` :: number
`HL_FORMAT_SVG` :: number
`HL_FORMAT_BBCODE` :: number
`HL_FORMAT_PANGO` :: number
`HL_FORMAT_ODT` :: number
Read only (other):
[horizontal]
`HL_PLUGIN_PARAM` :: string (set with `--plug-in-param`)
`HL_LANG_DIR` :: string (path of language definition directory)
Functions:
[horizontal]
`AddKeyword` :: function
`RemoveKeyword` :: function
`AddPersistentState` :: function
`OnStateChange` :: function
`OverrideParam` :: function
`Decorate` :: function
`DecorateLineBegin` :: function
`DecorateLineEnd` :: function
`StoreValue` :: function
[IMPORTANT]
================================================================================
Functions will only be executed if they are defined as local functions within the
`lang` chunk referenced in the `Plugins` array.
They will be ignored when defined elsewhere in the script.
================================================================================
The functions `AddKeyword`, `RemoveKeyword` and `OnStateChange` are also useful
in language definitions without a plug-in use case.
=== FUNCTIONS ADDKEYWORD/REMOVEKEYWORD
[[AddKeyword]]
The `AddKeyword` function will add a keyword to one of the the internal keyword
lists. It has no effect if the keyword was added before.
Keywords added with `AddKeyword` will remain active for all files of the same
syntax if highlight is in batch mode.
................................................................................
AddKeyword(keyword, kwGroupID)
Parameters: keyword: string which should be added to a keyword list
kwGroupID: keyword group ID of the keyword
Returns: true if successful
................................................................................
[[RemoveKeyword]]
The `RemoveKeyword` function erases the given keyword from the internal list.
................................................................................
RemoveKeyword(keyword)
Parameters: keyword: string which should be removed from the keyword list
Returns: true if successful
................................................................................
=== FUNCTION ADDPERSISTENTSTATE
[[AddPersistentState]]
This function enables storage of keywords and keyword ranges in a plug-in file.
If the syntax contains elements which depend on a context, you can highlight
them although this context is lost in other input files or code sections.
The invocation of AddPersistentState will cause highlight to save a plugin as
temporary file and parse input files using this plug-in again if necessary.
................................................................................
AddPersistentState(keyword, kwGroupID)
Parameters: keyword: string which should be added to a keyword list
kwGroupID: keyword group ID of the keyword
Returns: true if successful
................................................................................
................................................................................
AddPersistentState(lineno, kwGroupID, column, length)
Parameters: lineno: line number
kwGroupID: the keyword group ID
column: column
length: length of the keyword
Returns: true if successful
................................................................................
=== FUNCTION ONSTATECHANGE
[[OnStateChange]]
This function is a hook which is called if an internal state changes (e.g. from
`HL_STANDARD` to `HL_KEYWORD` if a keyword is found). It can be used to alter
the new state or to manipulate syntax elements like keyword lists.
................................................................................
OnStateChange(oldState, newState, token, kwGroupID, lineno, column)
Hook Event: Highlighting parser state change
Parameters: oldState: old state
newState: intended new state
token: the current token which triggered the new state
kwGroupID: if newState is HL_KEYWORD, the parameter
contains the keyword group ID
lineno: line number (since 3.50)
column: line column (since 3.50)
Returns: Correct state to continue OR HL_REJECT
................................................................................
Returns `HL_REJECT` if the recognized token and state should be discarded;
the first character of token will be outputted and highlighted as `oldState`.
Example for its usage in a plugin context:
[source,lua]
--------------------------------------------------------------------------------
function OnStateChange(oldState, newState, token, kwgroup)
if newState==HL_KEYWORD and kwgroup==5 then
AddKeyword(token, 5)
end
return newState
end
--------------------------------------------------------------------------------
This function adds the current keyword to the internal keyword list if the
keyword belongs to keyword group 5. If keyword group 5 is defined by a regex,
this token will be recognized later as a keyword even if the regular expression
does no longer match.
[IMPORTANT]
================================================================================
If both the syntax file and the plug-in script contain an `OnStateChange`
function, the plug-in definition will replace the function of the syntax file.
In order to prevent this, the existing function can be assigned to an new
variable and called using this name in the plug-in `OnStateChange` code chunk.
See this example in the plug-in `bash_functions.lua`:
================================================================================
[source,lua]
--------------------------------------------------------------------------------
function syntaxUpdate(desc)
if desc=="Bash" then
table.insert( Keywords,
{ Id=5, Regex=[[(\w+)\s*\(]]
} )
if OnStateChange ~= nil then
OrigOnStateChange = OnStateChange;
end
-- add keywords to list 5 if pattern matches
function OnStateChange(oldState, newState, token, kwgroup)
if newState==HL_KEYWORD and kwgroup==5 then
AddKeyword(token, 5)
return newState
end
if OrigOnStateChange then
return OrigOnStateChange(oldState, newState, token, kwgroup)
end
end
end
end
--------------------------------------------------------------------------------
=== FUNCTION OVERRIDEPARAM
[[OverrideParam]]
This function enables modification of internal default values concerning the
parser or output formats. This function is experimental and subject to change.
Currently it supports these parameters:
[horizontal]
`state.string.raw` :: `true` or `false`
`format.maskws` :: `true` or `false`
`format.spacer` :: whitespace format string
See `langDefs/toml.lang` and `plugins/outhtml_ie7_webctrl.lua` for examples.
................................................................................
OverrideParam(paramName, paramVal)
Parameters: paramName: name of configuration item
paramVal: value
Returns: true if successful
................................................................................
=== DECORATE FUNCTIONS
[[Decorate]]
The `Decorate` function is a hook which is called if a syntax token has been
identified. It can be used to alter the token or to add additional text in the
target output format (e.g. hyperlinks).
................................................................................
Decorate(token, state, kwGroupID, lineHasStmt, lineno, column)
Hook Event: Token identification
Parameters: token: current token
state: current state
kwGroupID: if state is HL_KEYWORD, the parameter
contains the keyword group ID
lineHasStmt: true if current line contains a statement
lineno: line number (since 3.60)
column: line column (since 3.60)
Returns: Altered token string or nothing if original token should be
outputted
................................................................................
Examples:
[source,lua]
--------------------------------------------------------------------------------
function Decorate(token, state)
if (state == HL_KEYWORD) then
return string.upper(token)
end
end
--------------------------------------------------------------------------------
This function converts all keywords to upper case.
The functions `DecorateLineBegin` and `DecorateLineEnd` are called if a new line
starts or ends. They can be used to add special formatting to lines of code.
[[DecorateLineBegin]]
................................................................................
DecorateLineBegin(lineNumber)
Hook Event: output of a new line
Parameters: lineNumber: the current line number
Returns: A string to be prepended to a new line (or nothing)
................................................................................
[[DecorateLineEnd]]
................................................................................
DecorateLineEnd(lineNumber)
Hook Event: output of a line ending
Parameters: lineNumber: the current line number
Returns: A string to be appended to a line (or nothing)
................................................................................
[IMPORTANT]
================================================================================
The return value of `Decorate` functions will be embedded in the formatting tags
of the output format.
The return values are not modified or validated by highlight.
================================================================================
=== FUNCTION STOREVALUE
[[StoreValue]]
................................................................................
StoreValue(name, value)
Set or get a value to exchange information across Lua states
Parameters: name: the parameter name
value: the parameter value
Returns: if value is not set, the previously assigned value of name
................................................................................
== THEME CHUNK ELEMENTS
The following list includes all those items which you can overwrite or extend to
adjust the formatting (colour and font attributes) of the output:
Output formatting items:
[horizontal]
`Default` :: table
`Canvas` :: table
`Number` :: table
`Escape` :: table
`String` :: table
`StringPreProc` :: table
`BlockComment` :: table
`PreProcessor` :: table
`LineNum` :: table
`Operator` :: table
`LineComment` :: table
`Keywords` :: table
Read only (output document format):
[horizontal]
`HL_OUTPUT` :: number
`HL_FORMAT_HTML` :: number
`HL_FORMAT_XHTML` :: number
`HL_FORMAT_TEX` :: number
`HL_FORMAT_LATEX` :: number
`HL_FORMAT_RTF` :: number
`HL_FORMAT_ANSI` :: number
`HL_FORMAT_XTERM256` :: number
`HL_FORMAT_TRUECOLOR` :: number
`HL_FORMAT_SVG` :: number
`HL_FORMAT_BBCODE` :: number
`HL_FORMAT_PANGO` :: number
`HL_FORMAT_ODT` :: number
Add additional styling information:
[horizontal]
`Injections` :: table
[horizontal]
`StoreValue` :: function
=== FUNCTION STOREVALUE
................................................................................
Same as the definition given above.
................................................................................
== FORMAT CHUNK ELEMENTS
Read only (output document format):
[horizontal]
`HL_OUTPUT` :: number
`HL_FORMAT_HTML` :: number
`HL_FORMAT_XHTML` :: number
`HL_FORMAT_TEX` :: number
`HL_FORMAT_LATEX` :: number
`HL_FORMAT_RTF` :: number
`HL_FORMAT_ANSI` :: number
`HL_FORMAT_XTERM256` :: number
`HL_FORMAT_TRUECOLOR` :: number
`HL_FORMAT_SVG` :: number
`HL_FORMAT_BBCODE` :: number
`HL_FORMAT_PANGO` :: number
`HL_FORMAT_ODT` :: number
Functions:
[horizontal]
`DocumentHeader` :: function
`DocumentFooter` :: function
=== DOCUMENTHEADER FUNCTION
[[DocumentHeader]]
................................................................................
DocumentHeader(numFiles, currFile, options)
Hook Event: output of a new file's header
Parameters: numFiles: number of files to be generated
currFile: current file counter
options: Map of the following options
options.title: document title
options.encoding: document encoding
options.fragment: true if header/footer should not be outputted
options.font: font name
options.fontsize: font size
Returns: [string, boolean?] (or nothing)
The string contains the new document header
The boolean value indicates if the string should replace the default
header (false=default) or if it should be appended to it (true).
................................................................................
=== DOCUMENTFOOTER FUNCTION
[[DocumentFooter]]
................................................................................
DocumentFooter(numFiles, currFile, options)
Hook Event: output of a new file's footer
Parameters: see DocumentHeader
Returns: [string, boolean?] (or nothing)
The string contains the new document footer
The boolean value indicates if the string should replace the default
footer (false=default) or if it should precede it (true).
................................................................................
== STEP BY STEP
This example will add reference hyperlinks to Qt keywords:
[source,lua]
--------------------------------------------------------------------------------
-- first add a description of what the plug-in does
Description="Add qtproject.org reference links to HTML, LaTeX or RTF output"
-- define the plugin categories (e.g., supported output formats, languages)
Categories = {"c++", "qt" }
-- the syntaxUpdate function contains code related to syntax recognition
function syntaxUpdate(desc)
-- if the current file is no C++ file we exit
if desc~="C and C++" then
return
end
-- this function returns a qt-project reference link of the given token
function getURL(token)
-- generate the URL
url='http://qt-project.org/doc/qt-4.8/'..string.lower(token).. '.html'
-- embed the URL in a hyperlink according to the output format
-- first HTML, then LaTeX and RTF
if (HL_OUTPUT== HL_FORMAT_HTML or HL_OUTPUT == HL_FORMAT_XHTML) then
return '<a class="hl" target="new" href="'
.. url .. '">'.. token .. '</a>'
elseif (HL_OUTPUT == HL_FORMAT_LATEX) then
return '\\href{'..url..'}{'..token..'}'
elseif (HL_OUTPUT == HL_FORMAT_RTF) then
return '{{\\field{\\*\\fldinst HYPERLINK "'
..url..'" }{\\fldrslt\\ul\\ulc0 '..token..'}}}'
end
end
-- the Decorate function will be invoked for every recognized token
function Decorate(token, state)
-- we are only interested in keywords, preprocessor or default items
if (state ~= HL_STANDARD and state ~= HL_KEYWORD and
state ~=HL_PREPROC) then
return
end
-- Qt keywords start with Q, followed by an upper and a lower case letter
-- if this pattern applies to the token, we return the URL
-- if we return nothing, the token is outputted as is
if string.find(token, "Q%u%l")==1 then
return getURL(token)
end
end
end
-- the themeUpdate function contains code related to the theme
function themeUpdate(desc)
-- the Injections table can be used to add style information to the theme
-- HTML: we add additional CSS style information to beautify hyperlinks,
-- they should have the same color as their surrounding tags
if (HL_OUTPUT == HL_FORMAT_HTML or HL_OUTPUT == HL_FORMAT_XHTML) then
Injections[#Injections+1]=
"a.hl, a.hl:visited {color:inherit;font-weight:inherit;}"
-- LaTeX: hyperlinks require the hyperref package, so we add this here
-- the colorlinks and pdfborderstyle options remove ugly boxes in the output
elseif (HL_OUTPUT==HL_FORMAT_LATEX) then
Injections[#Injections+1]=
"\\usepackage[colorlinks=false, pdfborderstyle={/S/U/W 1}]{hyperref}"
end
end
-- let highlight load the chunks
Plugins={
{ Type="lang", Chunk=syntaxUpdate },
{ Type="theme", Chunk=themeUpdate },
}
--------------------------------------------------------------------------------
== EXAMPLES
The {plugins} directory contains example scripts, including:
{bash_functions_lua}::
+
[horizontal]
Description ::: Add function names to keyword list.
Features ::: Adds new keyword group based on a regex, defines `OnStateChange`,
uses `AddKeyword`.
{theme_invert_lua}::
+
[horizontal]
Description ::: Invert colours of the original theme.
Features ::: Modifies all color attributes of the theme script, uses Lua
pattern matching.
{cpp_ref_cplusplus_com_lua}::
+
[horizontal]
Description ::: Add cplusplus.com reference links to HTML, LaTeX or RTF output of
C++ code.
Features ::: Uses `Decorate` to add hyperlinks for a defined set of C++ keywords. +
Adds CSS styles with `Injections`.
{ctags_html_tooltips_lua}::
+
[horizontal]
Description ::: Add tooltips based on a ctags file (default input file: tags).
Features ::: Uses file input (defined by cli option `--plug-in-param`) and
parses tags data before `Decorate` is called.
{outhtml_curly_brackets_matcher_lua}::
+
[horizontal]
Description ::: Shows matching curly brackets in HTML output.
Features ::: Uses `Decorate` to add span tags with unique ids to opening and
closing brackets. +
Adds JavaScript with `HeaderInjection` variable. +
Inserts additional CSS styles with `Injections` variable.
{outhtml_keyword_matcher_lua}::
+
[horizontal]
Description ::: Shows matching keywords in HTML output.
Features ::: Uses `Decorate` to add span tags with unique ids to keywords. +
Uses `OnStateChange` to assign an internal ID to each keyword. +
Adds JavaScript with `HeaderInjection` variable. +
Inserts additional CSS styles with `Injections` variable. +
{outhtml_codefold_lua}::
+
[horizontal]
Description ::: Adds code folding for C style languages, Pascal, Lua and Ruby to
HTML output.
Features ::: Uses `DecorateLineBegin` and `DecorateLineEnd` to add ID-spans to each
line. +
Applies `Decorate` to each code block delimiter to add `onClick` event
handlers. +
Adds JavaScript with `HeaderInjection` and `FooterInjection` variables. +
Inserts additional CSS styles with `Injections` variable.
== PLUG-IN USAGE
[discrete]
=== Command line interface
Run highlight `--list-scripts=plugins` to show all installed plug-ins.
Use `--plug-in` to load a plug-in script file.
This option can be applied more than once to apply several plug-ins.
Omit the `.lua` suffix.
You can store your plug-in scripts for testing in `~/.highlight/plugins`.
.Example
................................................................................
highlight my.cpp -Ilz --plug-in=html_curly_brackets_matcher > ~/test_out/my.html
................................................................................
[discrete]
=== GUI
Add the plug-in scripts in the plug-in selection tab and enable them using the
checkboxes.
// EOF //