191110.1 · d5fb4df33d4adc936a20e2dc30683e3c28b29ac9
released 1.5.8 - updated patterns, speed improvements, and fixed options -Y and -T
reading declarations
@@ -45,7 +45,7 @@ Table of contents - [Search this but not that with -v, -f, -L, and (?^pattern)](#not) - [Matching empty patterns with -Y](#empty) - [Searching ASCII and Unicode files](#unicode)- - [Matching multiple lines of text with -o](#only)+ - [Matching multiple lines of text](#multiline) - [Searching source code using -f, -o, -O, and -t](#source) - [Find files by file signature and shebang "magic bytes" with -M and -t](#magic) - [Searching binary files with -U, -W, and -X](#binary)@@ -53,8 +53,10 @@ Table of contents - [Using gitignore-style globs to select directories and files to search](#gitignore) - [Counting matching lines with -c and -g](#count) - [Displaying file, line, column, and byte offset info with -H, -n, -k, -b, and -T](#fields)- - [Customizing the output in JSON, XML, CSV, C++, and replacing matches using group captures](#format) - [Displaying colors with --color](#color)+ - [Output matches in JSON, XML, CSV, C++](#json)+ - [Customizing the output with --format](#format)+ - [Replacing matches with --format group captures](#replace) - [Limiting the number of matches with -m, --max-depth, and --max-files](#max) - [Searching compressed files with -z](#gz) - [More examples](#more)@@ -102,13 +104,10 @@ Introduction: why use ugrep? source code. This feature supports matching that could otherwise only be done with utilities like `sed`, for example: - ugrep -o '.*begin(.|\n)*?end.*' myfile.txt+ ugrep '.*begin(.|\n)*?end.*' myfile.txt matches all lines between a line containing `begin` and the first line after- that containing `end` by using lazy repetition `*?`. Multi-line matching is- performed with any one of the options `-o` (only matching), `-c` (count),- `-N` (only line number), `-l` (files with match), `-L` (files without match),- or `-q` (quiet), + that containing `end` by using lazy repetition `*?`. - **ugrep includes a growing [database of source code search patterns](https://github.com/Genivia/ugrep/tree/master/patterns)**. So you@@ -131,17 +130,16 @@ Introduction: why use ugrep? example to find exact matches of `main` in C/C++ source code while skipping strings and comments that may have a match with `main` in them: - ugrep -Ro -tc++ -nw 'main' -f c/zap_strings -f c/zap_comments+ ugrep -R -tc++ -nw 'main' -f c/zap_strings -f c/zap_comments - where `-R` is recursive search while following symlinks, `-o` for multi-line- matches (since strings and comments may span multiple lines), `-tc++`- searches C/C++ source code files, `-n` shows line numbers in the output, `-w`- matches exact words (for example, `mainly` won't be matched), and the `-f`- options specify two predefined installed patterns to match and skip strings- and comments in the input. As another example, it is now easy to search a- PHP file while zapping past any HTML between PHP code segments:+ where `-R` is recursive search while following symlinks `-tc++` searches+ C/C++ source code files, `-n` shows line numbers in the output, `-w` matches+ exact words (for example, `mainly` won't be matched), and the `-f` options+ specify two predefined installed patterns to match and skip strings and+ comments in the input. As another example, it is now easy to search a PHP+ file while zapping past any HTML between PHP code segments: - ugrep -o '.*IsInjected.*' -f php/zap_html myfile.php+ ugrep 'IsInjected' -f php/zap_html myfile.php - **ugrep produces hexdumps for binary matches** to search for binary patterns of bytes, for example:@@ -256,13 +254,6 @@ Note: most of the ugrep tests produce better performance results without `mmap` (option `--no-mmap`), which may be counter-intuitive. See our [TODO](#todo) below. -With respect to T8: **ugrep** and ripgrep use threads to search files-simultaneously. Ripgrep has a CPU utilization of 627.2% for this concurrent-search. By contrast, **ugrep** appears more efficient with a CPU utilization-of 276% for the same concurrent search. This means that spawning just two-or three threads (as specified by `-J2` or `-J3`) yields about the same fast-execution times for **ugrep** on this machine.- In some cases we decided in favor of features and safety over performance. For example, **ugrep** considers files binary when containing invalid UTF encodings or a NUL (`\0`). GNU and BSD grep only check for NUL, which is faster but may@@ -422,8 +413,8 @@ select single- and multi-line comments in the current file: :grep -f c++/comments % Only the first line of a multi-line comment is shown in quickfix, to save-space. To show all lines of a multi-line match produced with option `-o`,-remove `%-G` from `grepformat` in `.vimrc`.+space. To show all lines of a multi-line match, remove `%-G` from `grepformat`+in `.vimrc`. <a name="comparison"/> @@ -482,12 +473,8 @@ few: BSD grep POSIX ERE and support Unicode pattern matching and most of the PCRE syntax with some limitations, see further below. Extended regular expression (ERE) syntax is the default (i.e. option `-E`, as egrep).-- **ugrep** uses incremental matching. When one or more of the options `-q`- (quiet), `-o` (only matching), `-c` (count), `-N` (only line number), `-l`- (file with match), or `-L` (files without match) is used, **ugrep** performs- an even faster search of the input file instead of reading the input- line-by-line as other grep tools do. This allows matching patterns that- span multiple lines.+- **ugrep** uses incremental matching. This allows matching patterns that span+ multiple lines. - **ugrep** supports *negative patterns* of the form `(?^X)` to skip input that matches `X`. Negative patterns can be used to skip strings and comments when searching for identifiers in source code and find matches that aren't in@@ -552,30 +539,29 @@ numbers next to the lines matched: This search query also finds `main` in strings and comment blocks. With **ugrep** we can use *negative patterns* of the form `(?^...)` to skip-unwanted matches in C/C++ quoted strings and comment blocks. Because strings-and comment blocks may span multiple lines, we should use `-o`:+unwanted matches in C/C++ quoted strings and comment blocks: - ugrep -r -o -nkw 'main' '(?^"(\\.|\\\r?\n|[^\\\n"])*"|//.*|/\*([^*]|(\*+[^*/]))*\*+\/)' myproject+ ugrep -r -nkw 'main' '(?^"(\\.|\\\r?\n|[^\\\n"])*"|//.*|/\*([^*]|(\*+[^*/]))*\*+\/)' myproject This is a lot of work to type in correctly! If you are like me, I'm lazy and don't want to spend time fiddling with regex patterns when I am working on something more important. There is an easier way by using **ugrep**'s predefined patterns (`-f`) that are installed with the tool: - ugrep -r -o -nkw 'main' -f c/zap_strings -f c/zap_comments myproject+ ugrep -r -nkw 'main' -f c/zap_strings -f c/zap_comments myproject This query also searches through other files than C/C++ source code, like READMEs, Makefiles, and so on. We're also skipping symlinks with `-r`. So let's refine this query by selecting C/C++ files only using option `-tc,c++` and include symlinks to files and directories with `-R`: - ugrep -R -o -tc,c++ -nkw 'main' -f c/zap_strings -f c/zap_comments myproject+ ugrep -R -tc,c++ -nkw 'main' -f c/zap_strings -f c/zap_comments myproject What if we are only looking for the identifier `main` but not as a function `main(`? We can use a negative pattern for this to skip unwanted `main\h*(` pattern matches: - ugrep -R -o -tc,c++ -nkw -e 'main' -e '(?^main\h*\()' -f c/zap_strings -f c/zap_comments myproject+ ugrep -R -tc,c++ -nkw -e 'main' -e '(?^main\h*\()' -f c/zap_strings -f c/zap_comments myproject This uses the `-e` option to explicitly specify several patterns, which is essentially forming the pattern `main|(?^main\h*\()', where `\h` matches space@@ -587,7 +573,7 @@ blocks. To do so we can first select the comment blocks with **ugrep**'s predefined `c/comments` pattern AND THEN select lines with `FIXME` using a pipe: - ugrep -R -o -tc,c++ -nk -f c/comments myproject | ugrep -w 'FIXME'+ ugrep -R -tc,c++ -nk -f c/comments myproject | ugrep -w 'FIXME' Filtering results with pipes is generally easier than using AND-OR logic that some search tools use. This approach follows the Unix spirit to keep utilities@@ -596,7 +582,7 @@ simple and use them in combination for more complex tasks. Say we want to produce a sorted list of all identifiers found in Java source code while skipping strings and comments: - ugrep -R -o -tjava -f java/names -f java/zap_strings -f java/zap_comments myproject | sort -u+ ugrep -R -tjava -f java/names -f java/zap_strings -f java/zap_comments myproject | sort -u This matches Java Unicode identifiers using the regex `\p{JavaIdentifierStart}\p{JavaIdentifierPart}*` defined in@@ -613,7 +599,7 @@ files with extension `.c` or `.cpp` by using option `-Oc,cpp` and by using a predefined pattern `functions` that is installed with the tool to produce all function definitions. Then we select the one we want: - ugrep -R -o -Oc,cpp -nk -f c/functions | ugrep 'qsort'+ ugrep -R -Oc,cpp -nk -f c/functions | ugrep 'qsort' Note that we could have used `-tc,c++` to select C/C++ files, but this also includes header files when we want to only search `.c` and `.cpp` files. To@@ -748,7 +734,7 @@ To search for lines with the word `display` in file `myfile.py` skipping this word in strings and comments, where `-f` specifies patterns in files which are predefined patterns in this case: - ugrep -on -w 'display' -f python/zap_strings -f python/zap_comments myfile.py+ ugrep -n -w 'display' -f python/zap_strings -f python/zap_comments myfile.py To recursively list all Python files that do not contain the word `display`, allowing the word to occur in strings and comments:@@ -815,16 +801,21 @@ To check that a file contains Unicode: ugrep -q '[^[:ascii:]]' myfile && echo "contains Unicode" -<a name="only"/>+<a name="multiline"/>++### Matching multiple lines of text -### Matching multiple lines of text with -o+Multiple lines may be matched by patterns that match newline `\n` characters,+unless one or more context options `-A`, `-B`, `-C`, `-y` is used, or `-v` that+apply to lines. Use option `-o` to output the match only, not the full+lines(s) that match. -To match C/C++ `/*...*/` multi-line comments:+To match C/C++ `/*...*/` multi-line comments, color highlighted: - ugrep -o '/\*([^*]|(\*+[^*/]))*\*+\/' myfile.cpp+ ugrep --color '/\*([^*]|(\*+[^*/]))*\*+\/' myfile.cpp To match C/C++ comments using the predefined `c/comments` patterns with-`-f c/comments`:+`-f c/comments`, restricted to the matching part only with option `-o`: ugrep -of c/comments myfile.cpp @@ -835,33 +826,34 @@ Same as `sed -n '/begin/,/end/p'`: to match all lines between a line containing <a name="source"/> -### Searching source code using -f, -o, -O, and -t+### Searching source code using -f, -O, and -t To recursively display function definitions in C/C++ files (`.h`, `.hpp`, `.c`, `.cpp` etc.) with line numbers with `-tc++`, `-o`, `-n`, and `-f c++/functions`: - ugrep -Ron -tc++ -f c++/functions+ ugrep -R -on -tc++ -f c++/functions To recursively display function definitions in `.c` and `.cpp` files with line numbers with `-Oc,cpp`, `-o`, `-n`, and `-f c++/functions`: - ugrep -Ron -Oc,cpp -f c++/functions+ ugrep -R -on -Oc,cpp -f c++/functions -To search for patterns starting with `FIXME` in C/C++ comments, excluding-`FIXME` in multi-line strings:+To search for lines with `FIXME` in C/C++ comments, excluding `FIXME` in+multi-line strings: - ugrep -on 'FIXME.*' -f c++/zap_strings myfile.cpp+ ugrep -n 'FIXME' -f c++/zap_strings myfile.cpp -To read patterns starting with `TODO` and `FIXME` from standard input to match-with, while excluding matches in C++ strings:+To read patterns `TODO` and `FIXME` from standard input to match lines in the+input, while excluding matches in C++ strings: ugrep -on -f - -f c++/zap_strings myfile.cpp <<END- TODO.*- FIXME.*+ TODO+ FIXME END -To display XML element and attribute tags in an XML file, excluding tags that-are placed in (multi-line) comments:+To display XML element and attribute tags in an XML file, restricted to the+matching part with `-o`, excluding tags that are placed in (multi-line)+comments: ugrep -o -f xml/tags -f xml/zap_comments myfile.xml @@ -1071,71 +1063,115 @@ To display the line and column numbers of matches in XML with `--xml`: ugrep -r -nk --xml -tc++ 'main' -<a name="format"/>+<a name="color"/>++### Displaying colors with --color++To produce color-highlighted results:++ ugrep --color -R -n -k -tc++ 'FIXME.*'++To page through the color-highlighted results with pager `less -R`:++ ugrep --color --pager -R -n -k -tc++ 'FIXME.*'++To use predefined patterns to list all `#include` and `#define` in C++ files:++ ugrep --color -R -n -tc++ -f c++/includes -f c++/defines++To list all `#define FOO...` macros in C++ files, color-highlighted:++ ugrep --color=always -R -n -tc++ -f c++/defines | ugrep 'FOO.*'++Same, but restricted to `.cpp` files only:++ ugrep --color=always -R -n -Ocpp -f c++/defines | ugrep 'FOO.*'++To monitor the system log for bug reports:++ tail -f /var/log/system.log | ugrep --color -i -w 'bug'++To search tarballs for archived PDF files (assuming bash is our shell):++ for tb in *.tar *.tar.gz; do echo "$tb"; tar tfz "$tb" | ugrep '.*\.pdf$'; done++<a name="json"/> -### Customizing the output in JSON, XML, CSV, C++, and replacing matches with group captures using --format+### Output matches in JSON, XML, CSV, C++ To recursively search for lines with `TODO` and display C++ file matches in-JSON with line number properties (note that we use pattern `.*TODO.*` to+JSON with line number properties: include the whole line containing `TODO` in the results): - ugrep -rtc++ -n --json '.*TODO.*'+ ugrep -rtc++ -n --json 'TODO' To recursively search for lines with `TODO` and display C++ file matches in XML with line and column number attributes: - ugrep -rtc++ -nk --xml '.*TODO.*'+ ugrep -rtc++ -nk --xml 'TODO' To recursively search for lines with `TODO` and display C++ file matches in CSV format with file pathname, line number, and column number fields: - ugrep -rtc++ --csv -Hnk '.*TODO.*'+ ugrep -rtc++ --csv -Hnk 'TODO' -To extract a table from an HTML file and put it in C/C++ source code:+To extract a table from an HTML file and put it in C/C++ source code using+`-o`: - ugrep --cpp '<tr>.*</tr>' index.html > table.cpp+ ugrep -o --cpp '<tr>.*</tr>' index.html > table.cpp -To extract table cells from an HTML file using Perl matching (`-P`) to support-group captures with lazy quantifier `(.*?)`, and translate the matches to a-comma-separated list with format `%,%1` (conditional comma and group capture):+<a name="format"/> - ugrep -P '<td>(.*?)</td>' --format='%,%1' index.html+### Customizing the output with --format -The same as above, but displaying the replaced matches line-by-line:+To output matching lines faster by omitting the header output and binary match+checks, using `--format` with field `%O` (output matching line as is) and field+`%~` (output newline): - ugrep -P '<td>(.*?)</td>' --format='%1\n' index.html+ ugrep --format='%O%~' 'href=' index.html -<a name="color"/>+The same as above, but also displaying the line and column numbers: -### Displaying colors with --color+ ugrep --format='%n%k: %O%~' 'href=' index.html -To produce color-highlighted results:+To string together the line numbers of matches, separated by commas with field+`%,`: - ugrep --color -R -n -k -tc++ 'FIXME.*'+ ugrep --format='%,%n' 'href=' index.html -To page through the color-highlighted results with pager `less -R`:+To output the matching part of a line only with field `%o` (or option `-o` with+field `%O`): - ugrep --color --pager -R -n -k -tc++ 'FIXME.*'+ ugrep --format='%o%~' "href=[\"'][^\"'\n][\"']" index.html -To use predefined patterns to list all `#include` and `#define` in C++ files:+To string together the pattern matches as CSV-formatted strings with field `%v`+separated by commas with field `%,`: - ugrep --color -R -n -tc++ -f c++/includes -f c++/defines+ ugrep --format='%,%v' "href=[\"'][^\"'\n][\"']" index.html -To list all `#define FOO...` macros in C++ files, color-highlighted:+To output matches in JSON, using formatting options that produce the same+output as `--json`: - ugrep --color=always -R -n -tc++ -f c++/defines | ugrep 'FOO.*'+ ugrep --format-begin='[' \+ --format-open='%,%~ {%~ %[,%~ ]$%["file": ]H"matches": [' \+ --format='%,%~ { %[, ]$%["line": ]N%["column": ]K%["offset": ]B"match": %J }' \+ --format-close='%~ ]%~ }' \+ --format-end='%~]%~' \+ 'href=' index.html -Same, but restricted to `.cpp` files only:+<a name="replace"/> - ugrep --color=always -R -n -Ocpp -f c++/defines | ugrep 'FOO.*'+### Replacing matches with --format group captures -To monitor the system log for bug reports:+To extract table cells from an HTML file using Perl matching (`-P`) to support+group captures with lazy quantifier `(.*?)`, and translate the matches to a+comma-separated list with format `%,%1` (conditional comma and group capture): - tail -f /var/log/system.log | ugrep --color -i -w 'bug'+ ugrep -P '<td>(.*?)</td>' --format='%,%1' index.html -To search tarballs for archived PDF files (assuming bash is our shell):+The same as above, but displaying the replaced matches line-by-line: - for tb in *.tar *.tar.gz; do echo "$tb"; tar tfz "$tb" | ugrep '.*\.pdf$'; done+ ugrep -P '<td>(.*?)</td>' --format='%1\n' index.html <a name="max"/> @@ -1170,6 +1206,15 @@ Zlib-compressed files with extension `.gz` are decompressed and searched. ### More examples +To search for pattern `-o` in `script.sh` using `-e` to explicitly specify a+pattern to prevent pattern `-o` from being interpreted as an option:++ ugrep -n -e '-o' script.sh++Alternatively, using `--` to end the list of command arguments:++ ugrep -n -- '-o' script.sh+ To list all text files (.txt and .md) that do not properly end with a `\n` (`-o` is required to match `\n` or `\z`): @@ -1212,20 +1257,19 @@ Man page line if the regular expression (RE) in the pattern matches the input line without its trailing newline. An empty expression matches every line. Each input line that matches at least one of the patterns is- written to the standard output. To search for patterns that span mul-- tiple lines, use option -o.+ written to the standard output. - The ugrep utility normalizes and decodes encoded input to search for- the specified ASCII/Unicode patterns. If the input contains a UTF BOM- indicating UTF-8, UTF-16, or UTF-32 input, then ugrep normalizes the+ The ugrep utility normalizes and decodes encoded input to search for+ the specified ASCII/Unicode patterns. If the input contains a UTF BOM+ indicating UTF-8, UTF-16, or UTF-32 input, then ugrep normalizes the input to UTF-8. If no UTF BOM is present, then ugrep assumes the input- is ASCII, UTF-8, or raw binary. To explicitly specify an input encod-+ is ASCII, UTF-8, or raw binary. To explicitly specify an input encod- ing to decode, use option -Q, --encoding. The following options are available: -A NUM, --after-context=NUM- Print NUM lines of trailing context after matching lines.+ Print NUM lines of trailing context after matching lines. Places a --group-separator between contiguous groups of matches. See also the -B, -C, and -y options. @@ -1236,125 +1280,125 @@ Man page if the terminal driver interprets some of it as commands. -B NUM, --before-context=NUM- Print NUM lines of leading context before matching lines.+ Print NUM lines of leading context before matching lines. Places a --group-separator between contiguous groups of matches. See also the -A, -C, and -y options. -b, --byte-offset- The offset in bytes of a matched line is displayed in front of+ The offset in bytes of a matched line is displayed in front of the respective matched line. When used with option -g, displays- the offset in bytes of each pattern matched. Byte offsets are- exact for binary, ASCII, and UTF-8 input. Otherwise, the byte+ the offset in bytes of each pattern matched. Byte offsets are+ exact for binary, ASCII, and UTF-8 input. Otherwise, the byte offset in the UTF-8-converted input is displayed. --binary-files=TYPE- Controls searching and reporting pattern matches in binary- files. Options are `binary', `without-match`, `text`, `hex`,- and `with-hex'. The default is `binary' to search binary files- and to report a match without displaying the match. `with-- out-match' ignores binary matches. `text' treats all binary- files as text, which might output binary garbage to the termi-- nal, which can have problematic consequences if the terminal- driver interprets some of it as commands. `hex' reports all- matches in hexadecimal. `with-hex` only reports binary matches- in hexadecimal, leaving text matches alone. A match is consid-- ered binary if a match contains a zero byte or invalid UTF+ Controls searching and reporting pattern matches in binary+ files. Options are `binary', `without-match`, `text`, `hex`,+ and `with-hex'. The default is `binary' to search binary files+ and to report a match without displaying the match. `with-+ out-match' ignores binary matches. `text' treats all binary+ files as text, which might output binary garbage to the termi-+ nal, which can have problematic consequences if the terminal+ driver interprets some of it as commands. `hex' reports all+ matches in hexadecimal. `with-hex` only reports binary matches+ in hexadecimal, leaving text matches alone. A match is consid-+ ered binary if a match contains a zero byte or invalid UTF encoding. See also the -a, -I, -U, -W, and -X options. --break- Groups matches per file and adds a line break between results+ Groups matches per file and adds a line break between results from different files. -C[NUM], --context[=NUM] Print NUM lines of leading and trailing context surrounding each match. The default is 2 and is equivalent to -A 2 -B 2. Places- a --group-separator between contiguous groups of matches. No+ a --group-separator between contiguous groups of matches. No whitespace may be given between -C and its argument NUM. -c, --count- Only a count of selected lines is written to standard output.+ Only a count of selected lines is written to standard output. If -g or -o is specified, counts the number of patterns matched. If -v is specified, counts the number of non-matching lines. --color[=WHEN], --colour[=WHEN]- Mark up the matching text with the expression stored in the- GREP_COLOR or GREP_COLORS environment variable. The possible+ Mark up the matching text with the expression stored in the+ GREP_COLOR or GREP_COLORS environment variable. The possible values of WHEN can be `never', `always', or `auto', where `auto' marks up matches only when output on a terminal. --cpp Output file matches in C++. See also option --format. - --csv Output file matches in CSV. Use options -H, -n, -k, and -b to+ --csv Output file matches in CSV. Use options -H, -n, -k, and -b to specify additional fields. See also option --format. -D ACTION, --devices=ACTION- If an input file is a device, FIFO or socket, use ACTION to- process it. By default, ACTION is `skip', which means that+ If an input file is a device, FIFO or socket, use ACTION to+ process it. By default, ACTION is `skip', which means that devices are silently skipped. If ACTION is `read', devices read just as if they were ordinary files. -d ACTION, --directories=ACTION- If an input file is a directory, use ACTION to process it. By- default, ACTION is `read', i.e., read directories just as if- they were ordinary files. If ACTION is `skip', silently skip- directories. If ACTION is `recurse', read all files under each- directory, recursively, following symbolic links only if they- are on the command line. This is equivalent to the -r option.- If ACTION is `dereference-recurse', read all files under each+ If an input file is a directory, use ACTION to process it. By+ default, ACTION is `read', i.e., read directories just as if+ they were ordinary files. If ACTION is `skip', silently skip+ directories. If ACTION is `recurse', read all files under each+ directory, recursively, following symbolic links only if they+ are on the command line. This is equivalent to the -r option.+ If ACTION is `dereference-recurse', read all files under each directory, recursively, following symbolic links. This is equivalent to the -R option. -E, --extended-regexp- Interpret patterns as extended regular expressions (EREs). This+ Interpret patterns as extended regular expressions (EREs). This is the default. -e PATTERN, --regexp=PATTERN- Specify a PATTERN used during the search of the input: an input- line is selected if it matches any of the specified patterns.- This option is most useful when multiple -e options are used to- specify multiple patterns, when a pattern begins with a dash- (`-'), to specify a pattern after option -f or after the FILE+ Specify a PATTERN used during the search of the input: an input+ line is selected if it matches any of the specified patterns.+ This option is most useful when multiple -e options are used to+ specify multiple patterns, when a pattern begins with a dash+ (`-'), to specify a pattern after option -f or after the FILE arguments. --exclude=GLOB Skip files whose name matches GLOB (using wildcard matching). A- glob can use *, ?, and [...] as wildcards, and \ to quote a- wildcard or backslash character literally. If GLOB contains /,- full pathnames are matched. Otherwise basenames are matched.- Note that --exclude patterns take priority over --include pat-+ glob can use *, ?, and [...] as wildcards, and \ to quote a+ wildcard or backslash character literally. If GLOB contains /,+ full pathnames are matched. Otherwise basenames are matched.+ Note that --exclude patterns take priority over --include pat- terns. This option may be repeated. --exclude-dir=GLOB- Exclude directories whose name matches GLOB from recursive+ Exclude directories whose name matches GLOB from recursive searches. If GLOB contains /, full pathnames are matched. Oth-- erwise basenames are matched. Note that --exclude-dir patterns- take priority over --include-dir patterns. This option may be+ erwise basenames are matched. Note that --exclude-dir patterns+ take priority over --include-dir patterns. This option may be repeated. --exclude-from=FILE- Read the globs from FILE and skip files and directories whose+ Read the globs from FILE and skip files and directories whose name matches one or more globs (as if specified by --exclude and- --exclude-dir). Lines starting with a `#' and empty lines in+ --exclude-dir). Lines starting with a `#' and empty lines in FILE ignored. When FILE is a `-', standard input is read. This option may be repeated. -F, --fixed-strings- Interpret pattern as a set of fixed strings, separated by new-- lines, any of which is to be matched. This makes ugrep behave+ Interpret pattern as a set of fixed strings, separated by new-+ lines, any of which is to be matched. This makes ugrep behave as fgrep. If PATTERN or -e PATTERN is also specified, then this option does not apply to -f FILE patterns. -f FILE, --file=FILE- Read one or more newline-separated patterns from FILE. Empty- pattern lines in FILE are not processed. If FILE does not+ Read one or more newline-separated patterns from FILE. Empty+ pattern lines in FILE are not processed. If FILE does not exist, the GREP_PATH environment variable is used as the path to- FILE. If that fails, looks for FILE in- /usr/local/share/ugrep/patterns. When FILE is a `-', standard+ FILE. If that fails, looks for FILE in+ /usr/local/share/ugrep/patterns. When FILE is a `-', standard input is read. This option may be repeated. --format=FORMAT Output FORMAT-formatted matches. See `man ugrep' section FORMAT- for the `%' fields. Options -A, -B, -C, -y, and -v are dis-+ for the `%' fields. Options -A, -B, -C, -y, and -v are dis- abled. --free-space@@ -1365,131 +1409,131 @@ Man page behave as traditional grep. -g, --no-group- Do not group multiple pattern matches on the same matched line.+ Do not group multiple pattern matches on the same matched line. Output the matched line again for each additional pattern match,- using `+' as the field separator for each additional match.+ using `+' as the field separator. --group-separator=SEP Use SEP as a group separator for context options -A, -B, and -C. By default SEP is a double hyphen (`--'). -H, --with-filename- Always print the filename with output lines. This is the+ Always print the filename with output lines. This is the default when there is more than one file to search. -h, --no-filename- Never print filenames with output lines. This is the default- when there is only one file (or only standard input) to search.+ Never print filenames with output lines. This is the default+ when there is only one file (or only standard input) to search. --help Print a help message. - -I Ignore matches in binary files. This option is equivalent to+ -I Ignore matches in binary files. This option is equivalent to the --binary-files=without-match option. -i, --ignore-case- Perform case insensitive matching. By default, ugrep is case+ Perform case insensitive matching. By default, ugrep is case sensitive. This option applies to ASCII letters only. --include=GLOB Search only files whose name matches GLOB (using wildcard match-- ing). A glob can use *, ?, and [...] as wildcards, and \ to+ ing). A glob can use *, ?, and [...] as wildcards, and \ to quote a wildcard or backslash character literally. If GLOB con-- tains /, file pathnames are matched. Otherwise file basenames- are matched. Note that --exclude patterns take priority over+ tains /, file pathnames are matched. Otherwise file basenames+ are matched. Note that --exclude patterns take priority over --include patterns. This option may be repeated. --include-dir=GLOB- Only directories whose name matches GLOB are included in recur-- sive searches. If GLOB contains /, full pathnames are matched.- Otherwise basenames are matched. Note that --exclude-dir pat-- terns take priority over --include-dir patterns. This option+ Only directories whose name matches GLOB are included in recur-+ sive searches. If GLOB contains /, full pathnames are matched.+ Otherwise basenames are matched. Note that --exclude-dir pat-+ terns take priority over --include-dir patterns. This option may be repeated. --include-from=FILE- Read the globs from FILE and search only files and directories- whose name matches one or more globs (as if specified by- --include and --include-dir). Lines starting with a `#' and- empty lines in FILE are ignored. When FILE is a `-', standard+ Read the globs from FILE and search only files and directories+ whose name matches one or more globs (as if specified by+ --include and --include-dir). Lines starting with a `#' and+ empty lines in FILE are ignored. When FILE is a `-', standard input is read. This option may be repeated. -J NUM, --jobs=NUM- Specifies the number of threads spawned to search files. By+ Specifies the number of threads spawned to search files. By default, an optimum number of threads is spawned to search files- simultaneously. -J1 disables threading: files are matched in- the same order as the files specified.+ simultaneously. -J1 disables threading: files are searched in+ the same order as specified. -j, --smart-case Perform case insensitive matching unless PATTERN contains a cap- ital letter. Case insensitive matching applies to ASCII letters only. - --json Output file matches in JSON. Use options -H, -n, -k, and -b+ --json Output file matches in JSON. Use options -H, -n, -k, and -b to specify additional properties. See also option --format. -k, --column-number- The column number of a matched pattern is displayed in front of- the respective matched line, starting at column 1. Tabs are+ The column number of a matched pattern is displayed in front of+ the respective matched line, starting at column 1. Tabs are expanded when columns are counted, see option --tabs. -L, --files-without-match- Only the names of files not containing selected lines are writ-- ten to standard output. Pathnames are listed once per file+ Only the names of files not containing selected lines are writ-+ ten to standard output. Pathnames are listed once per file searched. If the standard input is searched, the string ``(standard input)'' is written. -l, --files-with-matches Only the names of files containing selected lines are written to- standard output. ugrep will only search a file until a match- has been found, making searches potentially less expensive.- Pathnames are listed once per file searched. If the standard+ standard output. ugrep will only search a file until a match+ has been found, making searches potentially less expensive.+ Pathnames are listed once per file searched. If the standard input is searched, the string ``(standard input)'' is written. --label[=LABEL]- Displays the LABEL value when input is read from standard input+ Displays the LABEL value when input is read from standard input where a file name would normally be printed in the output. This option applies to options -H, -L, and -l. --line-buffered- Force output to be line buffered. By default, output is line- buffered when standard output is a terminal and block buffered+ Force output to be line buffered. By default, output is line+ buffered when standard output is a terminal and block buffered otherwise. -M MAGIC, --file-magic=MAGIC- Only files matching the signature pattern `MAGIC' are searched.- The signature "magic bytes" at the start of a file are compared- to the `MAGIC' regex pattern. When matching, the file will be- searched. This option may be repeated and may be combined with- options -O and -t to expand the search. This option is rela-- tively slow as every file on the search path is read to compare+ Only files matching the signature pattern `MAGIC' are searched.+ The signature "magic bytes" at the start of a file are compared+ to the `MAGIC' regex pattern. When matching, the file will be+ searched. This option may be repeated and may be combined with+ options -O and -t to expand the search. This option is rela-+ tively slow as every file on the search path is read to compare `MAGIC'. -m NUM, --max-count=NUM- Stop reading the input after NUM matches for each file pro-+ Stop reading the input after NUM matches for each file pro- cessed. --max-depth=NUM- Restrict recursive search to NUM (NUM > 0) directories deep,+ Restrict recursive search to NUM (NUM > 0) directories deep, where --max-depth=1 searches the specified path without visiting- sub-directories. By comparison, -dskip skips all directories+ sub-directories. By comparison, -dskip skips all directories even when they are on the command line. --max-files=NUM- If -R or -r is specified, restrict the number of files matched- to NUM. If -J1 is specified, files are matched in the same- order as the files specified.+ If -R or -r is specified, restrict the number of files matched+ to NUM. Specify -J1 to produce replicable results by ensuring+ that files are searched in the same order as specified. -N, --only-line-number The line number of the matching line in the file is output with-- out displaying the match. The line number counter is reset for+ out displaying the match. The line number counter is reset for each file processed. -n, --line-number- Each output line is preceded by its relative line number in the- file, starting at line 1. The line number counter is reset for+ Each output line is preceded by its relative line number in the+ file, starting at line 1. The line number counter is reset for each file processed. --no-group-separator- Removes the group separator line from the output for context+ Removes the group separator line from the output for context options -A, -B, and -C. --no-hidden@@ -1501,15 +1545,14 @@ Man page -O EXTENSIONS, --file-extensions=EXTENSIONS Search only files whose file name extensions match the specified- comma-separated list of file name EXTENSIONS. This option is+ comma-separated list of file name EXTENSIONS. This option is the same as specifying --include='*.ext' for each extension name- `ext' in the EXTENSIONS list. This option may be repeated and+ `ext' in the EXTENSIONS list. This option may be repeated and may be combined with options -M and -t to expand the search. -o, --only-matching- Prints only the matching part of lines and allows pattern- matches across newlines to span multiple lines. Line numbers- for multi-line matches are displayed with option -n, using `|'+ Prints only the matching part of lines. When multiple lines+ match, the line numbers with option -n are displayed using `|' as the field separator for each additional line matched by the pattern. This option cannot be combined with options -A, -B, -C, -v, and -y.@@ -1538,21 +1581,20 @@ Man page -q, --quiet, --silent Quiet mode: suppress normal output. ugrep will only search until a match has been found, making searches potentially less- expensive. Allows a pattern match to span multiple lines.+ expensive. -R, --dereference-recursive Recursively read all files under each directory. Follow all symbolic links, unlike -r. If -J1 is specified, files are- matched in the same order as the files specified.+ searched in the same order as specified. -r, --recursive Recursively read all files under each directory, following sym- bolic links only if they are on the command line. If -J1 is- specified, files are matched in the same order as the files- specified.+ specified, files are searched in the same order as specified. -S, --dereference- If -r is specified, all symbolic links are followed, like -R.+ If -r is specified, all symbolic links are followed, like -R. The default is not to follow symbolic links. -s, --no-messages@@ -1560,39 +1602,39 @@ Man page their error messages are suppressed). --separator=SEP- Use SEP as field separator between file name, line number, col-+ Use SEP as field separator between file name, line number, col- umn number, byte offset, and the matched line. The default is a colon (`:'). --stats- Display statistics on the number of files and directories+ Display statistics on the number of files and directories searched. -T, --initial-tab- Add a tab space to separate the file name, line number, column+ Add a tab space to separate the file name, line number, column number, and byte offset with the matched line. -t TYPES, --file-type=TYPES- Search only files associated with TYPES, a comma-separated list+ Search only files associated with TYPES, a comma-separated list of file types. Each file type corresponds to a set of file name extensions passed to option -O. For capitalized file types, the- search is expanded to include files found on the search path- with matching file signature magic bytes passed to option -M.- This option may be repeated. The possible values of TYPES can- be (use option -tlist to display a detailed list): `action-- script', `ada', `asm', `asp', `aspx', `autoconf', `automake',- `awk', `Awk', `basic', `batch', `bison', `c', `c++', `clojure',+ search is expanded to include files found on the search path+ with matching file signature magic bytes passed to option -M.+ This option may be repeated. The possible values of TYPES can+ be (use option -tlist to display a detailed list): `action-+ script', `ada', `asm', `asp', `aspx', `autoconf', `automake',+ `awk', `Awk', `basic', `batch', `bison', `c', `c++', `clojure', `csharp', `css', `csv', `dart', `Dart', `delphi', `elisp',- `elixir', `erlang', `fortran', `gif', `Gif', `go', `groovy',- `gsp', `haskell', `html', `jade', `java', `jpeg', `Jpeg', `js',- `json', `jsp', `julia', `kotlin', `less', `lex', `lisp', `lua',- `m4', `make', `markdown', `matlab', `node', `Node', `objc',- `objc++', `ocaml', `parrot', `pascal', `pdf', `Pdf', `perl',- `Perl', `php', `Php', `png', `Png', `prolog', `python',+ `elixir', `erlang', `fortran', `gif', `Gif', `go', `groovy',+ `gsp', `haskell', `html', `jade', `java', `jpeg', `Jpeg', `js',+ `json', `jsp', `julia', `kotlin', `less', `lex', `lisp', `lua',+ `m4', `make', `markdown', `matlab', `node', `Node', `objc',+ `objc++', `ocaml', `parrot', `pascal', `pdf', `Pdf', `perl',+ `Perl', `php', `Php', `png', `Png', `prolog', `python', `Python', `r', `rpm', `Rpm', `rst', `rtf', `Rtf', `ruby',- `Ruby', `rust', `scala', `scheme', `shell', `Shell',- `smalltalk', `sql', `svg', `swift', `tcl', `tex', `text',- `tiff', `Tiff', `tt', `typescript', `verilog', `vhdl', `vim',+ `Ruby', `rust', `scala', `scheme', `shell', `Shell',+ `smalltalk', `sql', `svg', `swift', `tcl', `tex', `text',+ `tiff', `Tiff', `tt', `typescript', `verilog', `vhdl', `vim', `xml', `Xml', `yacc', `yaml' --tabs=NUM@@ -1601,25 +1643,25 @@ Man page -U, --binary Disables Unicode matching for binary file matching, forcing PAT-- TERN to match bytes, not Unicode characters. For example, -U- '\xa3' matches byte A3 (hex) instead of the Unicode code point+ TERN to match bytes, not Unicode characters. For example, -U+ '\xa3' matches byte A3 (hex) instead of the Unicode code point U+00A3 represented by the two-byte UTF-8 sequence C2 A3. -V, --version Display version information and exit. -v, --invert-match- Selected lines are those not matching any of the specified pat-+ Selected lines are those not matching any of the specified pat- terns. -W, --with-hex- Only output binary matches in hexadecimal, leaving text matches+ Only output binary matches in hexadecimal, leaving text matches alone. This option is equivalent to the --binary-files=with-hex option. -w, --word-regexp The PATTERN or -e PATTERN are searched for as a word (as if sur-- rounded by \< and \>). If PATTERN or -e PATTERN is also speci-+ rounded by \< and \>). If PATTERN or -e PATTERN is also speci- fied, then this option does not apply to -f FILE patterns. -X, --hex@@ -1627,23 +1669,23 @@ Man page --binary-files=hex option. -x, --line-regexp- Only input lines selected against the entire PATTERN or -e PAT-- TERN are considered to be matching lines (as if surrounded by ^- and $). If PATTERN or -e PATTERN is also specified, then this+ Only input lines selected against the entire PATTERN or -e PAT-+ TERN are considered to be matching lines (as if surrounded by ^+ and $). If PATTERN or -e PATTERN is also specified, then this option does not apply to -f FILE patterns. - --xml Output file matches in XML. Use options -H, -n, -k, and -b to+ --xml Output file matches in XML. Use options -H, -n, -k, and -b to specify additional attributes. See also option --format. -Y, --empty- Permits empty matches, such as `^\h*$' to match blank lines.+ Permits empty matches, such as `^\h*$' to match blank lines. Empty matches are disabled by default. Note that empty-matching- patterns such as `x?' and `x*' match all input, not only lines+ patterns such as `x?' and `x*' match all input, not only lines with `x'. -y, --any-line Any matching or non-matching line is output. Non-matching lines- are output with the `-' separator as context of the matching+ are output with the `-' separator as context of the matching lines. See also the -A, -B, and -C options. -Z, --null@@ -1657,15 +1699,14 @@ Man page ine the working directory. Use `--' before the FILE arguments to allow file and directory names to start with a `-'. - The regular expression pattern syntax is an extended form of the POSIX+ The regular expression pattern syntax is an extended form of the POSIX ERE syntax. For an overview of the syntax see README.md or visit: https://github.com/Genivia/ugrep - Note that `.' matches any non-newline character. Matching a newline- character `\n' is not possible unless one or more of the options -c,- -L, -l, -N, -o, or -q are used (in any combination, but not combined- with option -v) to allow a pattern match to span multiple lines.+ Note that `.' matches any non-newline character. Pattern `\n' matches+ a newline and multiple lines may be matched, unless one or more of the+ context options -A, -B, -C or -y is used, or option -v. EXIT STATUS The ugrep utility exits with one of the following values:@@ -1894,10 +1935,6 @@ Man page reverses the separator to the default separator or the separator speci- fied by --separator. - Matches are formatted without context. To output the entire line with- the match, use pattern '.*PATTERN.*' to match the line before and after- the match.- Additional formatting options: --format-begin=FORMAT@@ -1951,11 +1988,11 @@ Man page using recursive search, with one line of context before and after each matched line: - $ ugrep --color -C1 -R -n -k -tc++ 'FIXME.*'+ $ ugrep --color -C1 -R -n -k -tc++ 'FIXME' To list all C/C++ comments in a file displaying their line and column- numbers using options -n and -k, and option -o that allows for matching- patterns across multiple lines:+ numbers using options -n and -k, and option -o to restrict the match to+ the pattern only: $ ugrep -nko -e '//.*' -e '/\*([^*]|(\*+[^*/]))*\*+\/' myfile @@ -2005,7 +2042,7 @@ Man page - ugrep 1.5.7 November 07, 2019 UGREP(1)+ ugrep 1.5.8 November 10, 2019 UGREP(1) <a name="patterns"/> @@ -2058,8 +2095,8 @@ sub-expression patterns `φ` and `ψ`: `(?^φ)` | matches `φ` and ignores it (top-level `φ` with nothing following after the negative pattern) `^φ` | matches `φ` at the begin of input or begin of a line (top-level `φ`, not nested in a sub-pattern) `φ$` | matches `φ` at the end of input or end of a line (top-level `φ`, not nested in a sub-pattern)- `\Aφ` | matches `φ` at the begin of input (requires one or more of `-c`, `-L`, `-l`, `-N`, `-o`, `-q`) (top-level `φ`, not nested in a sub-pattern)- `φ\z` | matches `φ` at the end of input (requires one or more of `-c`, `-L`, `-l`, `-N`, `-o`, `-q`) (top-level `φ`, not nested in a sub-pattern)+ `\Aφ` | matches `φ` at the begin of input (top-level `φ`, not nested in a sub-pattern)+ `φ\z` | matches `φ` at the end of input (top-level `φ`, not nested in a sub-pattern) `\bφ` | matches `φ` starting at a word boundary (top-level `φ`, not nested in a sub-pattern) `φ\b` | matches `φ` ending at a word boundary (top-level `φ`, not nested in a sub-pattern) `\Bφ` | matches `φ` starting at a non-word boundary (top-level `φ`, not nested in a sub-pattern)@@ -2068,8 +2105,8 @@ sub-expression patterns `φ` and `ψ`: `\>φ` | matches `φ` that starts a non-word (top-level `φ`, not for sub-patterns `φ`) `φ\<` | matches `φ` that ends a non-word (top-level `φ`, not nested in a sub-pattern) `φ\>` | matches `φ` that ends a word (top-level `φ`, not nested in a sub-pattern)- `\i` | matches an indent (requires one or more of `-c`, `-L`, `-l`, `-N`, `-o`, `-q`)- `\j` | matches a dedent (requires one or more of `-c`, `-L`, `-l`, `-N`, `-o`, `-q`)+ `\i` | matches an indent+ `\j` | matches a dedent `(?i:φ)` | matches `φ` ignoring case `(?s:φ)` | `.` (dot) in `φ` matches newline `(?x:φ)` | ignore all whitespace and comments in `φ`
bin/linux/ugrep
bin/macosx/ugrep
bin/windows/ugrep.exe
@@ -756,7 +756,7 @@ class AbstractMatcher { } return utf8(tmp); }- /// Put back one character (8-bit) on the input character sequence for matching, invalidates the current match info and text, unput is not honored when matching in-place using buffer(base, size) and nothing has been read yet.+ /// Put back one character (8-bit) on the input character sequence for matching, DANGER: invalidates the previous text() pointer and match info, unput is not honored when matching in-place using buffer(base, size) and nothing has been read yet. void unput(char c) ///< 8-bit character to put back { DBGLOG("AbstractMatcher::unput()");@@ -806,12 +806,36 @@ class AbstractMatcher { #endif } #if defined(WITH_SPAN)+ /// Returns pointer to the begin of line in the buffer before the matched text.+ inline const char *bol()+ /// @returns pointer to the begin of line+ {+ (void)lineno();+ return bol_;+ }+ /// Returns pointer to the end of line in the buffer after the matched text, DANGER: invalidates previous bol() and text() pointers, only use eol() before bol() and text().+ inline const char *eol()+ /// @returns pointer to the end of line+ {+ if (chr_ == '\n' || txt_[len_] == '\n')+ return txt_ + len_;+ while (pos_ < end_ || wrap())+ {+ char *s = static_cast<char*>(std::memchr(buf_ + pos_, '\n', end_ - pos_));+ if (s != NULL)+ return s;+ (void)grow();+ pos_ = end_;+ end_ += get(buf_ + end_, blk_ ? blk_ : max_ - end_ - 1);+ }+ eof_ = true;+ return buf_ + end_;+ } /// Returns the byte offset of the match from the start of the line. size_t border() /// @returns border offset {- (void)lineno();- return txt_ - bol_;+ return txt_ - bol(); } /// Enlarge the match to span the entire line of input (excluding \n), return text(). const char *span()@@ -824,26 +848,9 @@ class AbstractMatcher { if (chr_ == '\n') return txt_; reset_text();- while (true)- {- char *s = static_cast<char*>(std::memchr(buf_ + pos_, '\n', end_ - pos_));- if (s != NULL)- {- set_current(s - buf_);- len_ = s - txt_;- return text();- }- (void)grow();- pos_ = end_;- end_ += get(buf_ + end_, blk_ ? blk_ : max_ - end_ - 1);- if (pos_ >= end_ && !wrap())- {- eof_ = true;- break;- }- }- set_current(end_);- len_ = end_ - (txt_ - buf_);+ const char *e = eol();+ set_current(e - buf_);+ len_ = e - bol_; return text(); } /// Returns the line of input (excluding \n) as a string containing the matched text as a substring.@@ -851,38 +858,18 @@ class AbstractMatcher { /// @returns matching line as a string { DBGLOG("AbstractMatcher::line()");- (void)lineno();- if (chr_ == '\n' || txt_[len_] == '\n' || at_end())- return std::string(bol_, txt_ - bol_ + len_);- std::string ln(bol_, txt_ - bol_ + len_);- ln.push_back(chr_ == '\0' ? txt_[len_] : chr_);- size_t loc = pos_ + 1;- while (true)- {- if (loc < end_)- {- char *s = static_cast<char*>(std::memchr(buf_ + loc, '\n', end_ - loc));- if (s != NULL)- return ln.append(buf_ + pos_ + 1, s - buf_ - pos_ - 1);- }- (void)grow();- loc = end_;- end_ += get(buf_ + end_, blk_ ? blk_ : max_ - end_ - 1);- if (loc >= end_ && !wrap())- {- eof_ = true;- break;- }- }- if (pos_ < end_)- ln.append(buf_ + pos_ + 1, end_ - pos_ - 1);- return ln;+ const char *e = eol(); // warning: must call eol() before bol()+ const char *b = bol();+ return std::string(b, e - b); } /// Returns the line of input (excluding \n) as a wide string containing the matched text as a substring. std::wstring wline() /// @returns matching line as a wide string {- return wcs(line());+ DBGLOG("AbstractMatcher::wline()");+ const char *e = eol(); // warning: must call eol() before bol()+ const char *b = bol();+ return wcs(b, e - b); } #endif /// Skip input until the specified character is consumed or EOF is reached.
@@ -421,15 +421,15 @@ unrolled: pos_ = cur_; if (at_end()) {- set_current(cur_); DBGLOG("Reject empty match at EOF"); if (!opt_.N || !at_bob()) // allow FIND and SCAN with "N" to match empty input, with ^$ etc. cap_ = 0;+ set_current(cur_); } else if (method == Const::FIND) {- set_current(++cur_); // skip one char to keep searching DBGLOG("Reject empty match and continue?");+ set_current(++cur_); // skip one char to keep searching if (cap_ == 0 || !opt_.N || (!bol && c1 == '\n')) // allow FIND with "N" to match an empty line, with ^$ etc. goto scan; DBGLOG("Accept empty match");
@@ -27,7 +27,6 @@ match one or more patterns. By default, a pattern matches an input line if the regular expression (RE) in the pattern matches the input line without its trailing newline. An empty expression matches every line. Each input line that matches at least one of the patterns is written to the standard output.-To search for patterns that span multiple lines, use option \fB-o\fR. .PP The \fBugrep\fR utility normalizes and decodes encoded input to search for the specified ASCII/Unicode patterns. If the input contains a UTF BOM indicating@@ -70,10 +69,9 @@ syntax. For an overview of the syntax see README.md or visit: .IP https://github.com/Genivia/ugrep .PP-Note that `.' matches any non-newline character. Matching a newline character-`\\n' is not possible unless one or more of the options \fB-c\fR, \fB-L\fR,-\fB-l\fR, \fB-N\fR, \fB-o\fR, or \fB-q\fR are used (in any combination, but not-combined with option \fB-v\fR) to allow a pattern match to span multiple lines.+Note that `.' matches any non-newline character. Pattern `\\n' matches a+newline and multiple lines may be matched, unless one or more of the context+options \fB-A\fR, \fB-B\fR, \fB-C\fR or \fB-y\fR is used, or option \fB-v\fR. .SH "EXIT STATUS" The \fBugrep\fR utility exits with one of the following values: .IP 0@@ -284,9 +282,6 @@ The separator used by \fB%P\fR, \fB%H\fR, \fB%N\fR, \fB%K\fR, \fB%B\fR, and \fB%$\fR, reverses the separator to the default separator or the separator specified by \fB--separator\fR. .PP-Matches are formatted without context. To output the entire line with the match,-use pattern '.*\fIPATTERN\fR.*' to match the line before and after the match.-.PP Additional formatting options: .IP \fB--format-begin\fR=\fIFORMAT\fR the \fIFORMAT\fR when beginning the search.@@ -334,11 +329,11 @@ $ ugrep -q '[^[:ascii:]]' myfile && echo "contains Unicode" To display the line and column number of all `FIXME' in all C++ files using recursive search, with one line of context before and after each matched line: .IP-$ ugrep --color -C1 -R -n -k -tc++ 'FIXME.*'+$ ugrep --color -C1 -R -n -k -tc++ 'FIXME' .PP To list all C/C++ comments in a file displaying their line and column numbers-using options \fB-n\fR and \fB-k\fR, and option \fB-o\fR that allows for-matching patterns across multiple lines:+using options \fB-n\fR and \fB-k\fR, and option \fB-o\fR to restrict the match+to the pattern only: .IP $ ugrep -nko -e '//.*' -e '/\\*([^*]|(\\*+[^*/]))*\\*+\\/' myfile .PP
@@ -1,4 +1,4 @@-.TH UGREP "1" "November 07, 2019" "ugrep 1.5.7" "User Commands"+.TH UGREP "1" "November 10, 2019" "ugrep 1.5.8" "User Commands" .SH NAME \fBugrep\fR -- universal file pattern searcher .SH SYNOPSIS@@ -12,7 +12,6 @@ match one or more patterns. By default, a pattern matches an input line if the regular expression (RE) in the pattern matches the input line without its trailing newline. An empty expression matches every line. Each input line that matches at least one of the patterns is written to the standard output.-To search for patterns that span multiple lines, use option \fB-o\fR. .PP The \fBugrep\fR utility normalizes and decodes encoded input to search for the specified ASCII/Unicode patterns. If the input contains a UTF BOM indicating@@ -166,7 +165,7 @@ behave as traditional grep. \fB\-g\fR, \fB\-\-no\-group\fR Do not group multiple pattern matches on the same matched line. Output the matched line again for each additional pattern match,-using `+' as the field separator for each additional match.+using `+' as the field separator. .TP \fB\-\-group\-separator\fR=\fISEP\fR Use SEP as a group separator for context options \fB\-A\fR, \fB\-B\fR, and \fB\-C\fR. By@@ -216,8 +215,8 @@ This option may be repeated. \fB\-J\fR \fINUM\fR, \fB\-\-jobs\fR=\fINUM\fR Specifies the number of threads spawned to search files. By default, an optimum number of threads is spawned to search files-simultaneously. \fB\-J\fR1 disables threading: files are matched in the-same order as the files specified.+simultaneously. \fB\-J\fR1 disables threading: files are searched in the+same order as specified. .TP \fB\-j\fR, \fB\-\-smart\-case\fR Perform case insensitive matching unless PATTERN contains a capital@@ -274,8 +273,8 @@ when they are on the command line. .TP \fB\-\-max\-files\fR=\fINUM\fR If \fB\-R\fR or \fB\-r\fR is specified, restrict the number of files matched to-NUM. If \fB\-J\fR1 is specified, files are matched in the same order as-the files specified.+NUM. Specify \fB\-J\fR1 to produce replicable results by ensuring that+files are searched in the same order as specified. .TP \fB\-N\fR, \fB\-\-only\-line\-number\fR The line number of the matching line in the file is output without@@ -306,9 +305,8 @@ in the EXTENSIONS list. This option may be repeated and may be combined with options \fB\-M\fR and \fB\-t\fR to expand the search. .TP \fB\-o\fR, \fB\-\-only\-matching\fR-Prints only the matching part of lines and allows pattern matches-across newlines to span multiple lines. Line numbers for-multi\-line matches are displayed with option \fB\-n\fR, using `|' as the+Prints only the matching part of lines. When multiple lines match,+the line numbers with option \fB\-n\fR are displayed using `|' as the field separator for each additional line matched by the pattern. This option cannot be combined with options \fB\-A\fR, \fB\-B\fR, \fB\-C\fR, \fB\-v\fR, and \fB\-y\fR. .TP@@ -334,17 +332,16 @@ The input file encoding. The possible values of ENCODING can be: \fB\-q\fR, \fB\-\-quiet\fR, \fB\-\-silent\fR Quiet mode: suppress normal output. ugrep will only search until a match has been found, making searches potentially less expensive.-Allows a pattern match to span multiple lines. .TP \fB\-R\fR, \fB\-\-dereference\-recursive\fR Recursively read all files under each directory. Follow all-symbolic links, unlike \fB\-r\fR. If \fB\-J\fR1 is specified, files are matched-in the same order as the files specified.+symbolic links, unlike \fB\-r\fR. If \fB\-J\fR1 is specified, files are searched+in the same order as specified. .TP \fB\-r\fR, \fB\-\-recursive\fR Recursively read all files under each directory, following symbolic links only if they are on the command line. If \fB\-J\fR1 is specified,-files are matched in the same order as the files specified.+files are searched in the same order as specified. .TP \fB\-S\fR, \fB\-\-dereference\fR If \fB\-r\fR is specified, all symbolic links are followed, like \fB\-R\fR. The@@ -456,10 +453,9 @@ syntax. For an overview of the syntax see README.md or visit: .IP https://github.com/Genivia/ugrep .PP-Note that `.' matches any non-newline character. Matching a newline character-`\\n' is not possible unless one or more of the options \fB-c\fR, \fB-L\fR,-\fB-l\fR, \fB-N\fR, \fB-o\fR, or \fB-q\fR are used (in any combination, but not-combined with option \fB-v\fR) to allow a pattern match to span multiple lines.+Note that `.' matches any non-newline character. Pattern `\\n' matches a+newline and multiple lines may be matched, unless one or more of the context+options \fB-A\fR, \fB-B\fR, \fB-C\fR or \fB-y\fR is used, or option \fB-v\fR. .SH "EXIT STATUS" The \fBugrep\fR utility exits with one of the following values: .IP 0@@ -670,9 +666,6 @@ The separator used by \fB%P\fR, \fB%H\fR, \fB%N\fR, \fB%K\fR, \fB%B\fR, and \fB%$\fR, reverses the separator to the default separator or the separator specified by \fB--separator\fR. .PP-Matches are formatted without context. To output the entire line with the match,-use pattern '.*\fIPATTERN\fR.*' to match the line before and after the match.-.PP Additional formatting options: .IP \fB--format-begin\fR=\fIFORMAT\fR the \fIFORMAT\fR when beginning the search.@@ -720,11 +713,11 @@ $ ugrep -q '[^[:ascii:]]' myfile && echo "contains Unicode" To display the line and column number of all `FIXME' in all C++ files using recursive search, with one line of context before and after each matched line: .IP-$ ugrep --color -C1 -R -n -k -tc++ 'FIXME.*'+$ ugrep --color -C1 -R -n -k -tc++ 'FIXME' .PP To list all C/C++ comments in a file displaying their line and column numbers-using options \fB-n\fR and \fB-k\fR, and option \fB-o\fR that allows for-matching patterns across multiple lines:+using options \fB-n\fR and \fB-k\fR, and option \fB-o\fR to restrict the match+to the pattern only: .IP $ ugrep -nko -e '//.*' -e '/\\*([^*]|(\\*+[^*/]))*\\*+\\/' myfile .PP
@@ -9,22 +9,20 @@ The list of patterns defined in this directory will expand over time. For example, to display class definitions in C++ files in the working directory - ugrep -r -o -tc++ -f c++/classes+ ugrep -r -tc++ -f c++/classes To display Java identifiers (Unicode) with the line numbers of the matches, but skipping all matches of identifiers in comments and strings: - ugrep -r -n -o -f java/names -f java/zap_comments -f java/zap_strings--Some patterns automatically enable ugrep option `-o` to match multiple lines.-These pattern files start with `###-o` to enable option `-o` that is required-to match the pattern across multiple lines. For example, strings and comments-may span multiple lines, such as Python docstrings, requiring option `-o`.+ ugrep -r -n -f java/names -f java/zap_comments -f java/zap_strings Empty lines and lines starting with a `#` are ignored. -Patterns requiring Unicode matching are placed in Unicode mode with `(?u:X)`,-just in case to prevent ugrep option -U from disabling them.+Lines starting with `###` followed by `-o` auto-enables option `-o` for all+patterns matched.++Patterns `X` requiring Unicode matching should be placed in Unicode mode with+`(?u:X)`, just in case to prevent ugrep option -U from disabling them. We love your contributions to this effort! ❤️
@@ -3,19 +3,19 @@ C++ patterns C++ with Unicode identifiers. -- `classes` matches class definitions, auto-enables ugrep option -o-- `comments` matches comments, auto-enables ugrep option -o-- `defines` matches `#define` lines, use ugrep option -o to match multi-line `#define`-- `directives` matches #-directives, use ugrep option -o to match multi-line directives-- `doc_comments` matches Doxygen comments, auto-enables ugrep option -o-- `enums` matches enumerations, auto-enables ugrep option -o-- `functions` matches function definitions, recommend ugrep option -o+- `classes` matches class definitions+- `comments` matches comments+- `defines` matches `#define` lines+- `directives` matches #-directives+- `doc_comments` matches Doxygen comments+- `enums` matches enumerations+- `functions` matches function definitions - `includes` matches `#include` lines - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o to match multi-line strings-- `structs` matches struct definitions, auto-enables ugrep option -o+- `strings` matches strings+- `structs` matches struct definitions - `typedefs` matches `typedef` lines - `zap_commands` removes command keywords from matches-- `zap_comments` removes comments from matches, recommend ugrep option -o+- `zap_comments` removes comments from matches - `zap_directives` removes #-directive lines from matches-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `zap_strings` removes strings from matches
@@ -1,2 +1 @@-###-o \<class\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,3 +1,2 @@-###-o //.* /\*([^*]|(\*+[^*/]))*\*+\/
@@ -1,3 +1,2 @@-###-o //[/!].* /\*[*!]([^*]|(\*+[^*/]))*\*+\/
@@ -1,2 +1 @@-###-o \<enum\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,3 +1,2 @@-###-o "(\\.|\\\r?\n|[^\\\n"])*" '(\\.|[^\\\n'])*'
@@ -1,2 +1 @@-###-o \<struct\h+[^{\n]*(?=\{|\n\h*\{)
@@ -3,18 +3,18 @@ C patterns Traditional C with non-Unicode identifiers. -- `comments` matches comments, auto-enables ugrep option -o-- `defines` matches `#define` lines, use ugrep option -o to match multi-line `#define`-- `directives` matches #-directives, use ugrep option -o to match multi-line directives-- `doc_comments` matches Doxygen comments, auto-enables ugrep option -o-- `enums` matches enumerations, auto-enables ugrep option -o-- `functions` matches function definitions, recommend ugrep option -o+- `comments` matches comment+-u `defines` matches `#define` lines+- `directives` matches #-directives+- `doc_comments` matches Doxygen comments+- `enums` matches enumerations+- `functions` matches function definitions - `includes` matches `#include` lines - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o to match multi-line strings-- `structs` matches struct definitions, auto-enables ugrep option -o+- `strings` matches strings+- `structs` matches struct definitions - `typedefs` matches `typedef` lines - `zap_commands` removes command keywords from matches-- `zap_comments` removes comments from matches, recommend ugrep option -o+- `zap_comments` removes comments from matches - `zap_directives` removes #-directive lines from matches-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `zap_strings` removes strings from matches
@@ -1,3 +1,2 @@-###-o //.* /\*([^*]|(\*+[^*/]))*\*+\/
@@ -1,3 +1,2 @@-###-o //[/!].* /\*[*!]([^*]|(\*+[^*/]))*\*+\/
@@ -1,2 +1 @@-###-o \<enum\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,3 +1,2 @@-###-o "(\\.|\\\r?\n|[^\\\n"])*" '(\\.|[^\\\n'])*'
@@ -1,2 +1 @@-###-o \<struct\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,12 +1,12 @@ Java patterns ============= -- `classes` matches class definitions, auto-enables ugrep option -o-- `comments` matches comments, auto-enables ugrep option -o-- `doc_comments` matches Doxygen comments, auto-enables ugrep option -o-- `enums` matches enumerations, auto-enables ugrep option -o-- `functions` matches function definitions, recommend ugrep option -o+- `classes` matches class definitions+- `comments` matches comments+- `doc_comments` matches Doxygen comments+- `enums` matches enumerations+- `functions` matches function definitions - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o to match multi-line strings-- `zap_comments` removes comments from matches, recommend ugrep option -o-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `strings` matches strings+- `zap_comments` removes comments from matches+- `zap_strings` removes strings from matches
@@ -1,2 +1 @@-###-o \<class\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,3 +1,2 @@-###-o //.* /\*([^*]|(\*+[^*/]))*\*+\/
@@ -1,3 +1,2 @@-###-o //[/!].* /\*[*!]([^*]|(\*+[^*/]))*\*+\/
@@ -1,2 +1 @@-###-o \<enum\h+[^{\n]*(?=\{|\n\h*\{)
@@ -1,3 +1,2 @@-###-o "(\\.|\\\r?\n|[^\\\n"])*" '(\\.|[^\\\n'])*'
@@ -2,10 +2,10 @@ Javascript patterns =================== - `classes` matches class definitions-- `comments` matches comments, auto-enables ugrep option -o+- `comments` matches comments - `dom` matches DOM document method invocations - `functions` matches function definitions - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o to match multi-line strings-- `zap_comments` removes comments from matches, recommend ugrep option -o-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `strings` matches strings+- `zap_comments` removes comments from matches+- `zap_strings` removes strings from matches
@@ -1,3 +1,2 @@-###-o //.* /\*([^*]|(\*+[^*/]))*\*+\/
@@ -1,2 +1 @@-###-o "(\\.|\\\r?\n|[^\\\n"])*"
@@ -1,8 +1,8 @@ Markdown patterns ================= -- `code` matches inline code, auto-enables ugrep option -o-- `codeblocks` matches code blocks, auto-enables ugrep option -o+- `code` matches inline code+- `codeblocks` matches code blocks - `images` matches markdown and HTML images - `links` matches markdown and HTML hyperlinks-- `sections` matches markdown sections, auto-enables option -o+- `sections` matches markdown sections
@@ -1,2 +1 @@-###-o `[^`\n]+`
@@ -1,4 +1,3 @@-###-o ^```+([^`]|`[^`]|``[^`])+\n``` ^~~~+([^~]|~[^~]|~~[^~])+\n~~~ ^(\t|[ ]{4}).*$
@@ -1,3 +1,2 @@-###-o ^.+(?=\r?\n(===|---)) ^#{1,6}\h+.*
@@ -1,10 +1,10 @@ PHP patterns ============ -- `comments` matches comments, auto-enables ugrep option -o+- `comments` matches comments - `functions` matches function definitions - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o to match multi-line strings-- `zap_comments` removes comments from matches, recommend ugrep option -o-- `zap_html` removes HTML content outside of `<?php...?>` blocks, auto-enables ugrep option -o-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `strings` matches strings+- `zap_comments` removes comments from matches+- `zap_html` removes HTML content outside of `<?php...?>` blocks+- `zap_strings` removes strings from matches
@@ -1,4 +1,3 @@-###-o #.* //.* /\*([^*]|(\*+[^*/]))*\*+\/
@@ -1,2 +1 @@-###-o "(\\.|\\\r?\n|[^\\\n"])*"
@@ -1,2 +1 @@-###-o (?^^(\?>|\h*<!DOCTYPE|\h*<html>)(.|\n)*?\n(<\?php\n|\z))
@@ -3,9 +3,9 @@ Python patterns - `comments` matches Python #-comments - `defs` matches Python `def` lines-- `doc_strings` matches Python docstrings, auto-enables ugrep option -o+- `doc_strings` matches Python docstrings - `imports` matches Python `import` lines - `names` matches Python identifiers (and keywords)-- `string` matches Python strings, auto-enables ugrep option -o+- `string` matches Python strings - `zap_comments` removes Python comments from matches-- `zap_strings` removes Python strings from matches, recommend ugrep option -o+- `zap_strings` removes Python strings from matches
@@ -1,2 +1 @@-###-o ^\h+"""("{0,2}[^\\"]|\\.)*"""
@@ -1,4 +1,3 @@-###-o '([^\\\n']|\\.)*' "([^\\\n"]|\\.)*" '''('{0,2}[^\\']|\\.)*'''
@@ -7,10 +7,10 @@ using back references (not yet supported) and lazy quantifiers, using regex `%q(.).*?\1`. - `classes` matches class definition lines-- `comments` matches comments, auto-enables ugrep option -o+- `comments` matches comments - `defs` matches function definition lines - `modules` matches module lines - `names` matches identifiers (and keywords)-- `strings` matches strings, auto-enables ugrep option -o-- `zap_comments` removes comments from matches, recommend ugrep option -o-- `zap_strings` removes strings from matches, recommend ugrep option -o+- `strings` matches strings+- `zap_comments` removes comments from matches+- `zap_strings` removes strings from matches
@@ -1,3 +1,2 @@-###-o #.* ^=begin(.|\n[^=]|\n=e[^n]|\n=en[^d])*\n=end
@@ -1,4 +1,3 @@-###-o '([^'\\]*|\\(.|\n))*' "([^"\\]*|\\(.|\n))*" %q/[^/]*/
@@ -3,10 +3,10 @@ XML patterns - `attributes` matches attributes without their values - `attributes_and_values` matches attributes and values-- `comments` matches comments, auto-enables ugrep option -o+- `comments` matches comments - `elements` matches element tags without their values-- `pis` matches processing instructions, recommend ugrep option -o+- `pis` matches processing instructions - `tags` matches tag names (element and attribute tags)-- `zap_comments` removes comments from matches, recommend ugrep option -o-- `zap_pis` removes processing instructions from matches, recommend ugrep option -o+- `zap_comments` removes comments from matches+- `zap_pis` removes processing instructions from matches - `zap_strings` removes attribute string values from matches
@@ -1,2 +1 @@-###-o <!--([^-]|-[^-]|--+[^->])*-*-->
@@ -1,2 +1 @@-###-o <\?([^?]|\?[^>])*\?>
@@ -146,7 +146,7 @@ Prebuilt executables are located in ugrep/bin. #endif // ugrep version info-#define UGREP_VERSION "1.5.7"+#define UGREP_VERSION "1.5.8" // ugrep platform -- see configure.ac #if !defined(PLATFORM)@@ -235,7 +235,7 @@ std::set<ino_t> visited; // ugrep command-line options bool flag_with_filename = false; bool flag_no_filename = false;-bool flag_no_labels = false;+bool flag_no_header = false; bool flag_no_group = false; bool flag_no_messages = false; bool flag_no_hidden = false;@@ -726,6 +726,96 @@ struct Output { chr(*s++); } + // output a match+ void mat(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ str(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ str(b, e - b);+ }+ }++ // output a quoted match with escapes for \ and "+ void quote(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ quote(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ quote(b, e - b);+ }+ }++ // output a match as a string in C/C+++ void cpp(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ cpp(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ cpp(b, e - b);+ }+ }++ // output a match as a quoted string in CSV+ void csv(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ csv(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ csv(b, e - b);+ }+ }++ // output a match as a quoted string in JSON+ void json(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ json(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ json(b, e - b);+ }+ }++ // output a match in XML+ void xml(reflex::AbstractMatcher *matcher)+ {+ if (flag_only_matching)+ {+ xml(matcher->begin(), matcher->size());+ }+ else+ {+ const char *e = matcher->eol(); // warning: must call eol() before bol()+ const char *b = matcher->bol();+ xml(b, e - b);+ }+ }+ // output a number with field width w (padded with spaces) void num(size_t i, size_t w = 1) {@@ -775,12 +865,19 @@ struct Output { flush(); } - // flush the buffers- void flush()+ // acquire synchronization lock on the master's mutex, if not owned already+ void acquire() { // if multi-threaded and lock is not owned already, then lock on master's mutex if (lock != NULL && !lock->owns_lock()) lock->lock();+ }++ // flush the buffers and acquire lock+ void flush()+ {+ // if multi-threaded and lock is not owned already, then lock on master's mutex+ acquire(); // flush the buffers container to the designated output file, pipe, or stream for (Buffers::iterator i = buffers.begin(); i != buf; ++i)@@ -821,7 +918,7 @@ struct Output { lock = new std::unique_lock<std::mutex>(mutex, std::defer_lock); } - // flush and release synchronization on the master's mutex, if one was assigned before with sync()+ // flush and release synchronization lock on the master's mutex, if one was assigned before with sync() void release() { flush();@@ -843,48 +940,18 @@ struct Output { // output a quoted string with escapes for \ and " void quote(const char *data, size_t size); - // output a quoted string with escapes for \ and "- void quote(const std::string& s)- {- quote(s.c_str(), s.size());- }-- // output string in C/C+++ // output quoted string in C/C++ void cpp(const char *data, size_t size); - // output string in C/C++- void cpp(const std::string& s)- {- cpp(s.c_str(), s.size());- }-- // output string in JSON- void json(const char *data, size_t size);-- // output string in JSON- void json(const std::string& s)- {- json(s.c_str(), s.size());- }- // output quoted string in CSV void csv(const char *data, size_t size); - // output quoted string in CSV- void csv(const std::string& s)- {- csv(s.c_str(), s.size());- }+ // output quoted string in JSON+ void json(const char *data, size_t size); - // output string in XML+ // output in XML void xml(const char *data, size_t size); - // output string in XML- void xml(const std::string& s)- {- xml(s.c_str(), s.size());- }- std::unique_lock<std::mutex> *lock; // synchronization lock FILE *file; // output stream Dump dump; // hex dump state@@ -1269,10 +1336,7 @@ void Output::format(const char *format, const char *pathname, size_t matches, re break; case 'O':- if (flag_only_matching)- str(matcher->begin(), matcher->size());- else- str(matcher->line());+ mat(matcher); break; case 'o':@@ -1280,10 +1344,7 @@ void Output::format(const char *format, const char *pathname, size_t matches, re break; case 'Q':- if (flag_only_matching)- quote(matcher->begin(), matcher->size());- else- quote(matcher->line());+ quote(matcher); break; case 'q':@@ -1291,47 +1352,59 @@ void Output::format(const char *format, const char *pathname, size_t matches, re break; case 'C':- if (flag_only_matching)- cpp(matcher->begin(), matcher->size());+ if (flag_count)+ chr('"'), num(matches), chr('"'); else- cpp(matcher->line());+ cpp(matcher); break; case 'c':- cpp(matcher->begin(), matcher->size());+ if (flag_count)+ chr('"'), num(matches), chr('"');+ else+ cpp(matcher->begin(), matcher->size()); break; case 'V':- if (flag_only_matching)- csv(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches); else- csv(matcher->line());+ csv(matcher); break; case 'v':- csv(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches);+ else+ csv(matcher->begin(), matcher->size()); break; case 'J':- if (flag_only_matching)- json(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches); else- json(matcher->line());+ json(matcher); break; case 'j':- json(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches);+ else+ json(matcher->begin(), matcher->size()); break; case 'X':- if (flag_only_matching)- xml(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches); else- xml(matcher->line());+ xml(matcher); break; case 'x':- xml(matcher->begin(), matcher->size());+ if (flag_count)+ num(matches);+ else+ xml(matcher->begin(), matcher->size()); break; case '$':@@ -1470,8 +1543,8 @@ void Output::cpp(const char *data, size_t size) chr('"'); } -// output string in JSON-void Output::json(const char *data, size_t size)+// output quoted string in CSV+void Output::csv(const char *data, size_t size) { const char *s = data; const char *e = data + size;@@ -1485,7 +1558,13 @@ void Output::json(const char *data, size_t size) if ((c & 0x80) == 0) {- if (c < 0x20 || c == '"' || c == '\\')+ if (c == '"')+ {+ str(t, s - t);+ t = s + 1;+ str("\"\"");+ }+ else if ((c < 0x20 && c != '\t') || c == '\\') { str(t, s - t); t = s + 1;@@ -1520,8 +1599,8 @@ void Output::json(const char *data, size_t size) } else {- str("\\u");- hex(c, 4);+ str("\\x");+ hex(c, 2); } } }@@ -1532,8 +1611,8 @@ void Output::json(const char *data, size_t size) chr('"'); } -// output quoted string in CSV-void Output::csv(const char *data, size_t size)+// output quoted string in JSON+void Output::json(const char *data, size_t size) { const char *s = data; const char *e = data + size;@@ -1547,13 +1626,7 @@ void Output::csv(const char *data, size_t size) if ((c & 0x80) == 0) {- if (c == '"')- {- str(t, s - t);- t = s + 1;- str("\"\"");- }- else if ((c < 0x20 && c != '\t') || c == '\\')+ if (c < 0x20 || c == '"' || c == '\\') { str(t, s - t); t = s + 1;@@ -1588,8 +1661,8 @@ void Output::csv(const char *data, size_t size) } else {- str("\\x");- hex(c, 2);+ str("\\u");+ hex(c, 4); } } }@@ -1600,7 +1673,7 @@ void Output::csv(const char *data, size_t size) chr('"'); } -// output string in XML+// output in XML void Output::xml(const char *data, size_t size) { const char *s = data;@@ -2764,7 +2837,7 @@ int main(int argc, char **argv) // read standard input flag_stdin = true; }- else if (options && pattern == NULL && flag_file.empty())+ else if (pattern == NULL && flag_file.empty()) { // no regex pattern specified yet, so assume it is PATTERN pattern = arg;@@ -2823,7 +2896,12 @@ int main(int argc, char **argv) // empty PATTERN matches everything if (pattern.empty()) {- regex.append(".*\\n?|");+ if (flag_line_regexp)+ regex.append("^$|");+ else+ regex.append(".*\\n?|");++ flag_empty = true; // we're matching empty lines, so enable -Y } else {@@ -2853,11 +2931,14 @@ int main(int argc, char **argv) // remove the ending '|' from the |-concatenated regexes in the regex string regex.pop_back(); - // -x or -w- if (flag_line_regexp)- regex.insert(0, "^(").append(")$"); // make the regex line-anchored- else if (flag_word_regexp)- regex.insert(0, "\\<(").append(")\\>"); // make the regex word-anchored+ if (regex != "^$")+ {+ // -x or -w+ if (flag_line_regexp)+ regex.insert(0, "^(").append(")$"); // make the regex line-anchored+ else if (flag_word_regexp)+ regex.insert(0, "\\<(").append(")\\>"); // make the regex word-anchored+ } // -x and -w do not apply to patterns in -f FILE when PATTERN or -e PATTERN is specified flag_line_regexp = false;@@ -2990,10 +3071,6 @@ int main(int argc, char **argv) if (flag_invert_match) flag_no_group = flag_only_matching = false; - // -c with -o is the same as -c with -g- if (flag_count && flag_only_matching)- flag_no_group = true;- // normalize -R (--dereference-recurse) option if (strcmp(flag_directories, "dereference-recurse") == 0) {@@ -3029,7 +3106,7 @@ int main(int argc, char **argv) // if no display options -H, -n, -N, -k, -b are set, enable --no-labels to suppress labels for speed if (!flag_with_filename && !flag_line_number && !flag_only_line_number && !flag_column_number && !flag_byte_offset)- flag_no_labels = true;+ flag_no_header = true; // normalize --cpp, --csv, --json, --xml if (flag_cpp)@@ -3037,7 +3114,7 @@ int main(int argc, char **argv) flag_format_begin = "const struct grep {\n const char *file;\n size_t line;\n size_t column;\n size_t offset;\n const char *match;\n} matches[] = {\n"; flag_format_open = " // %f\n"; flag_format = " { %h, %n, %k, %b, %C },\n";- flag_format_close = "\n";+ flag_format_close = " \n"; flag_format_end = " { NULL, 0, 0, 0, NULL }\n};\n"; } else if (flag_csv)@@ -3464,6 +3541,10 @@ int main(int argc, char **argv) exit(EXIT_ERROR); } + // --format-begin+ if (!flag_quiet && flag_format_begin != NULL)+ format(flag_format_begin, 0);+ try { // -U: set flags to convert regex to Unicode@@ -3513,7 +3594,8 @@ int main(int argc, char **argv) #ifdef HAVE_BOOST_REGEX // construct the Boost.Regex NFA-based Perl pattern matcher std::string pattern(reflex::BoostPerlMatcher::convert(regex, convert_flags));- reflex::BoostPerlMatcher matcher(pattern, matcher_options.c_str());+ reflex::BoostPerlMatcher matcher(pattern, reflex::Input(), matcher_options.c_str());+ if (threads > 1) { GrepMaster grep(output, &matcher);@@ -3532,7 +3614,8 @@ int main(int argc, char **argv) { // construct the RE/flex DFA pattern matcher and start matching files reflex::Pattern pattern(reflex::Matcher::convert(regex, convert_flags), "r");- reflex::Matcher matcher(pattern, matcher_options.c_str());+ reflex::Matcher matcher(pattern, reflex::Input(), matcher_options.c_str());+ if (threads > 1) { GrepMaster grep(output, &matcher);@@ -3635,6 +3718,10 @@ int main(int argc, char **argv) abort("Exception: ", error.what()); } + // --format-end+ if (!flag_quiet && flag_format_end != NULL)+ format(flag_format_end, stats.found_files());+ // --stats: display stats when we're done if (flag_stats) stats.report();@@ -3650,10 +3737,6 @@ int main(int argc, char **argv) // search the specified files or standard input for pattern matches void ugrep(reflex::Matcher& magic, Grep& grep, std::vector<const char*>& files) {- // --format-begin- if (flag_format_begin != NULL)- format(flag_format_begin, 0);- if (!flag_stdin && files.empty()) { recurse(1, magic, grep, ".");@@ -3686,10 +3769,6 @@ void ugrep(reflex::Matcher& magic, Grep& grep, std::vector<const char*>& files) find(1, magic, grep, file, basename, DIRENT_TYPE_UNKNOWN, 0, !flag_no_dereference); } }-- // --format-end- if (flag_format_end != NULL)- format(flag_format_end, stats.found_files()); } // search file or directory for pattern matches@@ -4115,26 +4194,32 @@ void Grep::search(const char *pathname) matches = !matches; if (matches > 0)+ {+ // --format-open or format-close: acquire lock early before stats.found()+ if (flag_files_with_match && (flag_format_open != NULL || flag_format_close != NULL))+ out.acquire();+ if (!stats.found()) goto exit_search; - // -l or -L- if (matches > 0 && flag_files_with_match)- {- if (flag_format)- {- if (flag_format_open != NULL)- out.format(flag_format_open, pathname, stats.found_files(), matcher);- out.format(flag_format, pathname, 1, matcher);- if (flag_format_close != NULL)- out.format(flag_format_close, pathname, stats.found_files(), matcher);- }- else+ // -l or -L+ if (flag_files_with_match) {- out.str(color_fn);- out.str(pathname);- out.str(color_off);- out.chr(flag_null ? '\0' : '\n');+ if (flag_format)+ {+ if (flag_format_open != NULL)+ out.format(flag_format_open, pathname, stats.found_files(), matcher);+ out.format(flag_format, pathname, 1, matcher);+ if (flag_format_close != NULL)+ out.format(flag_format_close, pathname, stats.found_files(), matcher);+ }+ else+ {+ out.str(color_fn);+ out.str(pathname);+ out.str(color_off);+ out.chr(flag_null ? '\0' : '\n');+ } } } }@@ -4142,7 +4227,7 @@ void Grep::search(const char *pathname) { // -c: count the number of lines/patterns matched - if (flag_no_group)+ if (flag_no_group || flag_only_matching) { // -c with -g or -o: count the number of patterns matched in the file @@ -4185,6 +4270,10 @@ void Grep::search(const char *pathname) matches = matcher->lineno() - matches - 1; } + // --format: acquire lock early before stats.found()+ if ((flag_format_open != NULL || flag_format_close != NULL))+ out.acquire();+ if (!stats.found()) goto exit_search; @@ -4227,9 +4316,13 @@ void Grep::search(const char *pathname) while (matcher->find()) {- // --format-open+ // output --format-open if (matches == 0) {+ // --format-open or --format-close: acquire lock early before stats.found()+ if (flag_format_open != NULL || flag_format_close != NULL)+ out.acquire();+ if (!stats.found()) goto exit_search; @@ -4239,23 +4332,25 @@ void Grep::search(const char *pathname) ++matches; + // output --format out.format(flag_format, pathname, matches, matcher); // -m: max number of matches reached? if (flag_max_count > 0 && matches >= flag_max_count) break;++ if (flag_line_buffered)+ out.flush(); } - // --format-close+ // output --format-close if (matches > 0 && flag_format_close != NULL) out.format(flag_format_close, pathname, stats.found_files(), matcher); }- else if (flag_only_matching || flag_only_line_number)+ else if (flag_only_line_number) {- // -o or -N+ // -N - bool hex = false;- bool binary = flag_hex; size_t lineno = 0; const char *separator = flag_separator; @@ -4269,234 +4364,376 @@ void Grep::search(const char *pathname) if (lineno != current_lineno || flag_no_group) {+ if (matches == 0)+ if (!stats.found())+ goto exit_search;++ ++matches;++ out.header(pathname, lineno, matcher, matcher->first(), separator, false);+ // -m: max number of matches reached? if (flag_max_count > 0 && matches >= flag_max_count) break; + if (flag_line_buffered)+ out.flush();+ lineno = current_lineno;+ }+ }+ }+ else if (flag_only_matching)+ {+ // -o++ bool hex = false;+ size_t lineno = 0; + read_file();++ while (matcher->find())+ {+ const char *begin = matcher->begin();+ size_t size = matcher->size();++ bool binary = flag_hex || (!flag_text && is_binary(begin, size));++ size_t current_lineno = matcher->lineno();++ if (lineno != current_lineno || flag_no_group)+ { if (matches == 0) if (!stats.found()) goto exit_search; ++matches; - if (!flag_no_labels)+ if (!flag_no_header) { if (hex && out.dump.offset < matcher->first()) out.dump.done(flag_separator);+ hex = false; - binary = flag_hex || (!flag_text && is_binary(matcher->begin(), matcher->size()));+ const char *separator = lineno != current_lineno ? flag_separator : "+"; - out.header(pathname, lineno, matcher, matcher->first(), separator, binary);+ out.header(pathname, current_lineno, matcher, matcher->first(), separator, binary); }++ lineno = current_lineno; } - if (!flag_only_line_number)+ if (binary) {- if (flag_hex)+ if (flag_hex || flag_with_hex) { if (hex) out.dump.next(matcher->first(), flag_separator);- out.dump.hex(Output::Dump::HEX_MATCH, matcher->first(), matcher->begin(), matcher->size(), flag_separator);++ out.dump.hex(Output::Dump::HEX_MATCH, matcher->first(), begin, size, flag_separator); hex = true; }- else if (binary)+ else if (!flag_binary_without_matches) {- if (flag_with_hex)- {- if (hex)- out.dump.next(matcher->first(), flag_separator);- out.dump.hex(Output::Dump::HEX_MATCH, matcher->first(), matcher->begin(), matcher->size(), flag_separator);- hex = true;- }- else if (!flag_binary_without_matches)- {- out.binary_file_matches(pathname);- }+ out.binary_file_matches(pathname);+ matches = 1;+ goto done_search; }- else- {- const char *begin = matcher->begin();- size_t size = matcher->size();-- if (flag_line_number)- {- // -o with -n: echo multi-line matches line-by-line-- const char *from = begin;- const char *to;-- while ((to = static_cast<const char*>(memchr(from, '\n', size - (from - begin)))) != NULL)- {- out.str(color_ms);- out.str(from, to - from + 1);- out.str(color_off);+ }+ else+ {+ if (hex && out.dump.offset < matcher->first())+ out.dump.done(flag_separator);+ hex = false; - if (to + 1 < begin + size)- out.header(pathname, ++lineno, NULL, matcher->first() + (to - begin) + 1, "|", binary);+ if (!flag_no_header)+ {+ // -o with -n: echo multi-line matches line-by-line - from = to + 1;- }+ const char *from = begin;+ const char *to; - out.str(color_ms);- out.str(from, size - (from - begin));- out.str(color_off);- if (size == 0 || begin[size - 1] != '\n')- out.chr('\n');- }- else+ while ((to = static_cast<const char*>(memchr(from, '\n', size - (from - begin)))) != NULL) { out.str(color_ms);- out.str(begin, size);+ out.str(from, to - from + 1); out.str(color_off);- if (size == 0 || begin[size - 1] != '\n')- out.chr('\n');++ if (to + 1 <= begin + size)+ out.header(pathname, ++lineno, NULL, matcher->first() + (to - begin) + 1, "|", false);++ from = to + 1; } - if (flag_line_buffered)- out.flush();+ size -= from - begin;+ begin = from; }++ out.str(color_ms);+ out.str(begin, size);+ out.str(color_off);++ if (size == 0 || begin[size - 1] != '\n')+ out.chr('\n'); }++ // -m: max number of matches reached?+ if (flag_max_count > 0 && matches >= flag_max_count)+ break;++ if (flag_line_buffered)+ out.flush(); } if (hex)- out.dump.done(separator);+ out.dump.done(flag_separator); } else {- // read input line-by-line and display lines that match the pattern-- // mmap base and size, set with read_file() and mmap.file()- const char *base = NULL;- size_t size = 0;-- bool is_mmap = mmap.file(input, base, size);-- if (is_mmap && flag_before_context == 0 && flag_after_context == 0 && !flag_any_line && !flag_invert_match && !flag_no_group)+ if (flag_before_context == 0 && flag_after_context == 0 && !flag_any_line && !flag_invert_match) {- // this branch is the same as the next branch but optimized for mmap() when options -A, -B, -C, -g, -v, -y are not used+ // options -A, -B, -C, -v, -y are not used - const char *here = base;- size_t left = size;+ bool binary = false;+ std::string rest_line;+ const char *rest_line_data = NULL;+ size_t rest_line_size = 0;+ size_t rest_line_last = 0;+ size_t lineno = 0; - size_t byte_offset = 0;- size_t lineno = 1;+ read_file(); - while (true)+ while (matcher->find()) {- const char *line = here;-- // read the next line from mmap- if (getline(here, left))- break;-- bool binary = flag_hex;-- size_t last = UNDEFINED;+ if (matches == 0)+ if (!stats.found())+ goto exit_search; - // the current input line to match- read_line(matcher, line, here - line);+ size_t current_lineno = matcher->lineno(); - // search the line for pattern matches- while (matcher->find())+ if (lineno != current_lineno || flag_no_group) {- if (last == UNDEFINED)+ if (rest_line_data != NULL) {- if (matches == 0)- if (!stats.found())- goto exit_search;-- if (!flag_text && !flag_hex)+ if (binary) {- if (is_binary(line, here - line))- {- if (flag_binary_without_matches)- {- matches = 0;- break;- }- binary = true;- }-- if (binary && !flag_with_hex)- {- out.binary_file_matches(pathname);- matches = 1;- goto done_search;- }+ out.dump.hex(Output::Dump::HEX_LINE, rest_line_last, rest_line_data, rest_line_size, flag_separator);+ out.dump.done(flag_separator);+ }+ else+ {+ out.str(rest_line_data, rest_line_size); } - if (!flag_no_labels)- out.header(pathname, lineno, matcher, byte_offset, flag_separator, binary);+ if (flag_line_buffered)+ out.flush();+ } - ++matches;+ // -m: max number of matches reached?+ if (flag_max_count > 0 && matches >= flag_max_count)+ break; - last = 0;+ const char *eol = matcher->eol(); // warning: call eol() before bol()+ const char *bol = matcher->bol(); - // if no color highlighting, then display the matched line at once- if (flag_color == NULL)- break;+ ++matches;++ binary = flag_hex || (!flag_text && is_binary(bol, eol - bol));++ if (!flag_no_header)+ {+ const char *separator = lineno != current_lineno ? flag_separator : "+";++ out.header(pathname, current_lineno, matcher, matcher->first(), separator, binary); } + lineno = current_lineno;++ size_t border = matcher->border();+ size_t first = matcher->first();+ const char *begin = matcher->begin();+ const char *end = matcher->end();+ size_t size = matcher->size();+ if (binary) {- out.dump.hex(Output::Dump::HEX_LINE, byte_offset + last, line + last, matcher->first() - last, flag_separator);- out.dump.hex(Output::Dump::HEX_MATCH, byte_offset + matcher->first(), matcher->begin(), matcher->size(), flag_separator);+ if (flag_hex || flag_with_hex)+ {+ out.dump.hex(Output::Dump::HEX_LINE, first - border, bol, border, flag_separator);+ out.dump.hex(Output::Dump::HEX_MATCH, first, begin, size, flag_separator);++ rest_line.assign(end, eol - end).push_back('\n'); // but there may not be a \n at EOF+ rest_line_data = rest_line.c_str();+ rest_line_size = rest_line.size();+ rest_line_last = matcher->last();++ lineno += matcher->lines() - 1;+ }+ else if (!flag_binary_without_matches)+ {+ out.binary_file_matches(pathname);+ matches = 1;+ goto done_search;+ } } else { out.str(color_sl);- out.str(line + last, matcher->first() - last);+ out.str(bol, border); out.str(color_off);++ if (!flag_no_header)+ {+ // -n: echo multi-line matches line-by-line++ const char *from = begin;+ const char *to;+ size_t num = 1;++ while ((to = static_cast<const char*>(memchr(from, '\n', size - (from - begin)))) != NULL)+ {+ out.str(color_ms);+ out.str(from, to - from + 1);+ out.str(color_off);++ if (to + 1 <= begin + size)+ out.header(pathname, lineno + num, NULL, first + (to - begin) + 1, "|", false);++ from = to + 1;+ ++num;+ }++ size -= from - begin;+ begin = from;+ }+ out.str(color_ms);- out.str(matcher->begin(), matcher->size());+ out.str(begin, size); out.str(color_off);- } - last = matcher->last();+ rest_line.assign(end, eol - end).push_back('\n'); // but there may not be a \n at EOF+ rest_line_data = rest_line.c_str();+ rest_line_size = rest_line.size();+ rest_line_last = matcher->last(); - // skip any further empty pattern matches- if (last == 0)- break;+ lineno += matcher->lines() - 1;+ } }-- if (last != UNDEFINED)+ else {- if (binary)- {- out.dump.hex(Output::Dump::HEX_LINE, byte_offset + last, line + last, here - line - last, flag_separator);- out.dump.done(flag_separator);- }- else+ size_t size = matcher->size();++ if (size > 0) {- out.str(color_sl);- out.str(line + last, here - line - last);- out.str(color_off);- }+ size_t lines = matcher->lines(); - if (flag_line_buffered)- out.flush();- }+ if (lines > 1 || flag_color)+ {+ size_t first = matcher->first();+ size_t last = matcher->last();+ const char *begin = matcher->begin(); - // -m: max number of matches reached?- if (flag_max_count > 0 && matches >= flag_max_count)- break;+ if (binary)+ {+ out.dump.hex(Output::Dump::HEX_LINE, rest_line_last, rest_line_data, first - rest_line_last, flag_separator);+ out.dump.hex(Output::Dump::HEX_MATCH, first, begin, size, flag_separator);+ }+ else+ {+ out.str(color_sl);+ out.str(rest_line_data, first - rest_line_last);+ out.str(color_off); - // update byte offset and line number- byte_offset += here - line;- ++lineno;+ if (lines > 1 && !flag_no_header)+ {+ // -n: echo multi-line matches line-by-line++ const char *from = begin;+ const char *to;+ size_t num = 1;++ while ((to = static_cast<const char*>(memchr(from, '\n', size - (from - begin)))) != NULL)+ {+ out.str(color_ms);+ out.str(from, to - from + 1);+ out.str(color_off);++ if (to + 1 <= begin + size)+ out.header(pathname, lineno + num, NULL, first + (to - begin) + 1, "|", false);++ from = to + 1;+ ++num;+ }++ size -= from - begin;+ begin = from;+ }++ out.str(color_ms);+ out.str(begin, size);+ out.str(color_off);+ }++ if (lines == 1)+ {+ rest_line_data += last - rest_line_last;+ rest_line_size -= last - rest_line_last;+ rest_line_last = last;+ }+ else+ {+ const char *eol = matcher->eol(); // warning: call eol() before end()+ const char *end = matcher->end();++ bool rest_binary = flag_hex || (!flag_text && is_binary(end, eol - end));++ if (binary && !rest_binary)+ {+ out.dump.done(flag_separator);+ out.header(pathname, lineno + lines - 1, matcher, last, flag_separator, false);+ }+ else if (!binary && rest_binary)+ {+ out.nl();+ out.header(pathname, lineno + lines - 1, matcher, last, flag_separator, true);+ }++ binary = rest_binary;++ rest_line.assign(end, eol - end).push_back('\n'); // but there may not be a \n at EOF+ rest_line_data = rest_line.c_str();+ rest_line_size = rest_line.size();+ rest_line_last = last;++ lineno += lines - 1;+ }+ }+ }+ } }++ if (rest_line_data != NULL)+ {+ if (binary)+ out.dump.hex(Output::Dump::HEX_LINE, rest_line_last, rest_line_data, rest_line_size, flag_separator);+ else+ out.str(rest_line_data, rest_line_size);+ }++ if (binary)+ out.dump.done(flag_separator); } else { // read input line-by-line and display lines that match the pattern with context lines - // TODO: replace line-by-line reading with block reading to improve speed+ // mmap base and size, set with mmap.file()+ const char *base = NULL;+ size_t size = 0; reflex::BufferedInput buffered_input; - if (!is_mmap)+ if (!mmap.file(input, base, size)) buffered_input = input; const char *here = base;@@ -4569,7 +4806,7 @@ void Grep::search(const char *pathname) if (!stats.found()) goto exit_search; - if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, lineno, matcher, byte_offset, "-", binary[current]); last = 0;@@ -4635,7 +4872,7 @@ void Grep::search(const char *pathname) { out.binary_file_matches(pathname); matches = 1;- break;+ goto done_search; } if (after_context)@@ -4686,7 +4923,7 @@ void Grep::search(const char *pathname) { if (last == UNDEFINED) {- if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, begin, matcher, byte_offsets[begin_context], "-", binary[begin_context]); last = 0;@@ -4743,7 +4980,7 @@ void Grep::search(const char *pathname) before = lineno; } - if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, lineno, NULL, byte_offsets[current], flag_separator, binary[current]); if (binary[current])@@ -4826,7 +5063,7 @@ void Grep::search(const char *pathname) { size_t begin_context = begin % (flag_before_context + 1); - if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, begin, NULL, byte_offsets[begin_context], "-", binary[begin_context]); if (binary[begin_context])@@ -4853,9 +5090,9 @@ void Grep::search(const char *pathname) { // -g: do not group matches on a single line but on multiple lines, counting each match separately - const char *separator = (last == UNDEFINED ? flag_separator : "+");+ const char *separator = last == UNDEFINED ? flag_separator : "+"; - if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, lineno, matcher, byte_offset + matcher->first(), separator, binary[current]); out.str(color_sl);@@ -4878,7 +5115,7 @@ void Grep::search(const char *pathname) { if (last == UNDEFINED) {- if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, lineno, matcher, byte_offset, flag_separator, binary[current]); ++matches;@@ -4931,7 +5168,7 @@ void Grep::search(const char *pathname) // -A NUM: show context after matched lines, simulates BSD grep -A // display line as part of the after context of the matched line- if (!flag_no_labels)+ if (!flag_no_header) out.header(pathname, lineno, NULL, byte_offsets[current], "-", binary[current]); if (binary[current])@@ -4968,11 +5205,11 @@ done_search: out.chr('\n'); } +exit_search:+ // flush and release output to allow other workers to output results out.release(); -exit_search:- close_file(); } @@ -5266,7 +5503,7 @@ void help(const char *message, const char *arg) -g, --no-group\n\ Do not group multiple pattern matches on the same matched line.\n\ Output the matched line again for each additional pattern match,\n\- using `+' as the field separator for each additional match.\n\+ using `+' as the field separator.\n\ --group-separator=SEP\n\ Use SEP as a group separator for context options -A, -B, and -C. By\n\ default SEP is a double hyphen (`--').\n\@@ -5306,8 +5543,8 @@ void help(const char *message, const char *arg) -J NUM, --jobs=NUM\n\ Specifies the number of threads spawned to search files. By\n\ default, an optimum number of threads is spawned to search files\n\- simultaneously. -J1 disables threading: files are matched in the\n\- same order as the files specified.\n\+ simultaneously. -J1 disables threading: files are searched in the\n\+ same order as specified.\n\ -j, --smart-case\n\ Perform case insensitive matching unless PATTERN contains a capital\n\ letter. Case insensitive matching applies to ASCII letters only.\n\@@ -5353,8 +5590,8 @@ void help(const char *message, const char *arg) when they are on the command line.\n\ --max-files=NUM\n\ If -R or -r is specified, restrict the number of files matched to\n\- NUM. If -J1 is specified, files are matched in the same order as\n\- the files specified.\n\+ NUM. Specify -J1 to produce replicable results by ensuring that\n\+ files are searched in the same order as specified.\n\ -N, --only-line-number\n\ The line number of the matching line in the file is output without\n\ displaying the match. The line number counter is reset for each\n\@@ -5378,9 +5615,8 @@ void help(const char *message, const char *arg) in the EXTENSIONS list. This option may be repeated and may be\n\ combined with options -M and -t to expand the search.\n\ -o, --only-matching\n\- Prints only the matching part of lines and allows pattern matches\n\- across newlines to span multiple lines. Line numbers for\n\- multi-line matches are displayed with option -n, using `|' as the\n\+ Prints only the matching part of lines. When multiple lines match,\n\+ the line numbers with option -n are displayed using `|' as the\n\ field separator for each additional line matched by the pattern.\n\ This option cannot be combined with options -A, -B, -C, -v, and -y.\n\ -P, --perl-regexp\n\@@ -5405,15 +5641,14 @@ void help(const char *message, const char *arg) -q, --quiet, --silent\n\ Quiet mode: suppress normal output. ugrep will only search until a\n\ match has been found, making searches potentially less expensive.\n\- Allows a pattern match to span multiple lines.\n\ -R, --dereference-recursive\n\ Recursively read all files under each directory. Follow all\n\- symbolic links, unlike -r. If -J1 is specified, files are matched\n\- in the same order as the files specified.\n\+ symbolic links, unlike -r. If -J1 is specified, files are searched\n\+ in the same order as specified.\n\ -r, --recursive\n\ Recursively read all files under each directory, following symbolic\n\ links only if they are on the command line. If -J1 is specified,\n\- files are matched in the same order as the files specified.\n\+ files are searched in the same order as specified.\n\ -S, --dereference\n\ If -r is specified, all symbolic links are followed, like -R. The\n\ default is not to follow symbolic links.\n\