std.regex

Regular expressions

are a commonly used method of pattern matching on strings, with regex being a catchy word for a pattern in this domain specific language. Typical problems usually solved by regular expressions include validation of user input and the ubiquitous find & replace in text processing utilities.

Synopsis

Create a regex at runtime: met on 24/01/1970. 7/8/99 wasn't as hot as 7/8/2022. ) --- import std.regex; import std.stdio; // Print out all possible dd/mm/yy(yy) dates found in user input. auto r = regex(r"\b[0-9][0-9]?/[0-9][0-9]?/[0-9]0-9?\b"); foreach (line; stdin.byLine) { // matchAll() returns a range that can be iterated // to get all subsequent matches. foreach (c; matchAll(line, r)) writeln(c.hit); } ---

Create a static regex at compile-time, which contains fast native code:

import std.regex;
auto ctr = ctRegex!(`^.*/([^/]+)/?$`);

// It works just like a normal regex:
auto c2 = matchFirst("foo/bar", ctr);   // First match found here, if any
assert(!c2.empty);   // Be sure to check if there is a match before examining contents!
assert(c2[1] == "bar");   // Captures is a range of submatches: 0 = full match.

Multi-pattern regex:

import std.regex;
auto multi = regex([`\d+,\d+`, `([a-z]+):(\d+)`]);
auto m = "abc:43 12,34".matchAll(multi);
assert(m.front.whichPattern == 2);
assert(m.front[1] == "abc");
assert(m.front[2] == "43");
m.popFront();
assert(m.front.whichPattern == 1);
assert(m.front[0] == "12,34");

Captures and opCast!bool:

import std.regex;
// The result of `matchAll/matchFirst` is directly testable with `if/assert/while`,
// e.g. test if a string consists of letters only:
assert(matchFirst("LettersOnly", `^\p{L}+$`));

// And we can take advantage of the ability to define a variable in the IfCondition:
if (const captures = matchFirst("At l34st one digit, but maybe more...", `((\d)(\d*))`))
{
    assert(captures[2] == "3");
    assert(captures[3] == "4");
    assert(captures[1] == "34");
}

Syntax and general information

The general usage guideline is to keep regex complexity on the side of simplicity, as its capabilities reside in purely character-level manipulation. As such it's ill-suited for tasks involving higher level invariants like matching an integer number bounded in an [a,b] interval. Checks of this sort of are better addressed by additional post-processing.

The basic syntax shouldn't surprise experienced users of regular expressions. For an introduction to std.regex see a

short tour of the module API

and its abilities.

There are other web resources on regular expressions to help newcomers, and a good reference with tutorial can easily be found.

This library uses a remarkably common ECMAScript syntax flavor with the following extensions:

Named subexpressions, with Python syntax.
Unicode properties such as Scripts, Blocks and common binary properties e.g Alphabetic, White_Space, Hex_Digit etc.
Arbitrary length and complexity lookbehind, including lookahead in lookbehind and vise-versa.

Pattern syntax

std.regex operates on codepoint level,

'character' in this table denotes a single Unicode codepoint.

Pattern element	Semantics
Atoms	Match single characters
any character except [{\|+?()^$*	Matches the character itself.
.	In single line mode matches any character. Otherwise it matches any character except '\n' and '\r'.
[class]	Matches a single character that belongs to this character class.
[^class]	Matches a single character that does not belong to this character class.
\cC	Matches the control character corresponding to letter C
\xXX	Matches a character with hexadecimal value of XX.
\uXXXX	Matches a character with hexadecimal value of XXXX.
\U00YYYYYY	Matches a character with hexadecimal value of YYYYYY.
\f	Matches a formfeed character.
\n	Matches a linefeed character.
\r	Matches a carriage return character.
\t	Matches a tab character.
\v	Matches a vertical tab character.
\d	Matches any Unicode digit.
\D	Matches any character except Unicode digits.
\w	Matches any word character (note: this includes numbers).
\W	Matches any non-word character.
\s	Matches whitespace, same as \p{White_Space}.
\S	Matches any character except those recognized as \s .
\\\\	Matches \ character.
\c where c is one of [\|+?()*	Matches the character c itself.
\p{PropertyName}	Matches a character that belongs to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {, }.
\P{PropertyName}	Matches a character that does not belong to the Unicode PropertyName set. Single letter abbreviations can be used without surrounding {, }.
\p{InBasicLatin}	Matches any character that is part of the BasicLatin Unicode block.
\P{InBasicLatin}	Matches any character except ones in the BasicLatin Unicode block.
\p{Cyrillic}	Matches any character that is part of Cyrillic script.
\P{Cyrillic}	Matches any character except ones in Cyrillic script.
Quantifiers	Specify repetition of other elements
*	Matches previous character/subexpression 0 or more times. Greedy version - tries as many times as possible.
*?	Matches previous character/subexpression 0 or more times. Lazy version - stops as early as possible.
+	Matches previous character/subexpression 1 or more times. Greedy version - tries as many times as possible.
+?	Matches previous character/subexpression 1 or more times. Lazy version - stops as early as possible.
?	Matches previous character/subexpression 0 or 1 time. Greedy version - tries as many times as possible.
??	Matches previous character/subexpression 0 or 1 time. Lazy version - stops as early as possible.
{n}	Matches previous character/subexpression exactly n times.
{n,}	Matches previous character/subexpression n times or more. Greedy version - tries as many times as possible.
{n,}?	Matches previous character/subexpression n times or more. Lazy version - stops as early as possible.
{n,m}	Matches previous character/subexpression n to m times. Greedy version - tries as many times as possible, but no more than m times.
{n,m}?	Matches previous character/subexpression n to m times. Lazy version - stops as early as possible, but no less then n times.
Other	Subexpressions & alternations
(regex)	Matches subexpression regex, saving matched portion of text for later retrieval.
(?#comment)	An inline comment that is ignored while matching.
(?:regex)	Matches subexpression regex, not saving matched portion of text. Useful to speed up matching.
A\|B	Matches subexpression A, or failing that, matches B.
(?P<name≥regex)	Matches named subexpression regex labeling it with name 'name'. When referring to a matched portion of text, names work like aliases in addition to direct numbers.
Assertions	Match position rather than character
^	Matches at the beginning of input or line (in multiline mode).
$	Matches at the end of input or line (in multiline mode).
\b	Matches at word boundary.
\B	Matches when not at word boundary.
(?=regex)	Zero-width lookahead assertion. Matches at a point where the subexpression regex could be matched starting from the current position.
(?!regex)	Zero-width negative lookahead assertion. Matches at a point where the subexpression regex could not be matched starting from the current position.
(?<=regex)	Zero-width lookbehind assertion. Matches at a point where the subexpression regex could be matched ending at the current position (matching goes backwards).
(?<!regex)	Zero-width negative lookbehind assertion. Matches at a point where the subexpression regex could not be matched ending at the current position (matching goes backwards).

Character classes

Pattern element	Semantics
Any atom	Has the same meaning as outside of a character class, except for ] which must be written as \\]
a-z	Includes characters a, b, c, ..., z.
[a\|\|b], [a--b], [a~~b], [a&&b]	Where a, b are arbitrary classes, means union, set difference, symmetric set difference, and intersection respectively. Any sequence of character class elements implicitly forms a union.

Regex flags

Flag	Semantics
g	Global regex, repeat over the whole input.
i	Case insensitive matching.
m	Multi-line mode, match ^, $ on start and end line separators as well as start and end of input.
s	Single-line mode, makes . match '\n' and '\r' as well.
x	Free-form syntax, ignores whitespace in pattern, useful for formatting complex regular expressions.

Unicode support

This library provides full Level 1 support* according to

UTS 18. Specifically:

1.1 Hex notation via any of \uxxxx, \U00YYYYYY, \xZZ.
1.2 Unicode properties.
1.3 Character classes with set operations.
1.4 Word boundaries use the full set of "word" characters.
1.5 Using simple casefolding to match case
insensitively across the full range of codepoints.

1.6 Respecting line breaks as any of
\u000A | \u000B | \u000C | \u000D | \u0085 | \u2028 | \u2029 | \u000D\u000A.

1.7 Operating on codepoint level.

*With exception of point 1.1.1, as of yet, normalization of input is expected to be enforced by user.

Replace format string

A set of functions in this module that do the substitution rely on a simple format to guide the process. In particular the table below applies to the format argument of

replaceFirst and replaceAll.

The format string can reference parts of match using the following notation.

Format specifier	Replaced by
$&	the whole match.
$`	part of input preceding the match.
$'	part of input following the match.
$$	'$' character.
\c , where c is any character	the character c itself.
\\\\	'\\' character.
$1 .. $99	submatch number 1 to 99 respectively.

Slicing and zero memory allocations orientation

All matches returned by pattern matching functionality in this library are slices of the original input. The notable exception is the replace family of functions that generate a new string from the input.

In cases where producing the replacement is the ultimate goal

replaceFirstInto and replaceAllInto could come in handy

as functions that avoid allocations even for replacement.

Copyright

License

Boost License 1.0.

Authors

Dmitry Olshansky,

API and utility constructs are modeled after the original std.regex by Walter Bright and Andrei Alexandrescu.

Source: std/regex/package.d

tmpl ctRegexImpl

alias Regex RegexException StaticRegex

struct Captures CTRegexWrapper RegexMatch Splitter

fn bmatch escaper match matchAll matchFirst matchMany matchOnce matchOnceImpl regex regexImpl replace replaceAll replaceAllInto replaceAllWith replaceCapturesInto replaceFirst replaceFirstInto replaceFirstWith replaceFmt replaceMatchesInto split splitter

Types 7

aliasRegex(Char) = std.regex.internal.ir.Regex!(Char)

Regex object holds regular expression pattern in compiled form.

Instances of this object are constructed via calls to regex. This is an intended form for caching and storage of frequently used regular expressions.

Example:

Test if this object doesn't contain any compiled pattern.

Regex!char r;
assert(r.empty);
r = regex(""); // Note: "" is a valid regex pattern.
assert(!r.empty);

Getting a range of all the named captures in the regex. ---- import std.range; import std.algorithm;

auto re = regex((?P<name>\w+) = (?P<var>\d+)); auto nc = re.namedCaptures; static assert(isRandomAccessRange!(typeof(nc))); assert(!nc.empty); assert(nc.length == 2); assert(nc.equal(["name", "var"])); assert(nc[0] == "name"); assert(nc[1..$].equal(["var"])); ----

aliasStaticRegex = Regex

A StaticRegex is Regex object that contains D code specially generated at compile-time to speed up matching.

No longer used, kept as alias to Regex for backwards compatibility.

private structCTRegexWrapper(Char)

Fields

private immutable(Regex!Char) * re

Methods

@property @trusted ref getRe() const

structCaptures(R) if (isSomeString!R)

Captures object contains submatches captured during a call to match or iteration over RegexMatch range.

First element of range is the whole match.

Fields

Store matches

const(NamedGroup)[] _names

R _input

int _nMatch

uint _f

Methods

inout(R) getMatch(size_t index) inout

R pre() @propertySlice of input prior to the match.

R post() @propertySlice of input immediately after the match.

R hit() @propertySlice of matched portion of input.

R front() @propertyRange interface.

R back() @propertyditto

void popFront()ditto

void popBack()ditto

bool empty() @property constditto

inout(R) opIndex()(size_t i) inoutditto

bool opCast(T: bool)() @safe const nothrowExplicit cast to bool. Useful as a shorthand for !(x.empty) in if and assert statements.

int whichPattern() @safe @property const nothrowNumber of pattern matched counting, where 1 - the first pattern. Returns 0 on no match.

R opIndex(String)(String i) if (isSomeString!String)Lookup named submatch.

size_t length() @property constNumber of matches in this object.

@property ref captures(){A hook for compatibility with original std.regex.

Constructors

this(R input, uint n, const(NamedGroup)[] named)

this(ref RegexMatch!R rmatch)

structRegexMatch(R) if (isSomeString!R)

A regex engine state, as returned by match family of functions.

Effectively it's a forward range of Captures!R, produced by lazily searching for matches in a given input.

Fields

Matcher!Char _engine

Rebindable!(const MatcherFactory!Char) _factory

R _input

Captures!R _captures

Methods

R pre() @propertyShorthands for front.pre, front.post, front.hit.

R post() @propertyditto

R hit() @propertyditto

inout(Captures!R) front() @property inoutFunctionality for processing subsequent matches of global regexes via range interface: --- import std.regex; auto m = matchAll("Hello, world!", regex(`\w+`)); assert(m.front.hit == "Hello"); m.popF...

void popFront()ditto

auto save(){ditto

bool empty() @property constTest if this match object is empty.

T opCast(T: bool)(){Same as !(x.empty), provided for its convenience in conditional statements.

inout(Captures!R) captures() @property inoutSame as .front, provided for compatibility with original std.regex.

Constructors

this(R input, RegEx prog)

Destructors

~this

structSplitter(Flag!"keepSeparators" keepSeparators = No.keepSeparators, Range, alias RegEx = Regex) if (isSomeString!Range && isRegexFor!(RegEx, Range))

Splits a string r using a regular expression pat as a separator.

Parameters

`keepSeparators`	flag to specify if the matches should be in the resulting range
`r`	the string to split
`pat`	the pattern to split on

Returns

A lazy range of strings

Fields

Range _input

size_t _offset

Rx _match

Methods

auto ref opSlice()

Range front() @propertyForward range primitives.

bool empty() @propertyditto

void popFront()ditto

@property auto save()ditto

Constructors

this(Range input, RegEx separator)

aliasRegexException = std.regex.internal.ir.RegexException

Exception object thrown in case of errors during regex compilation.

Functions 34

fnauto regex(S : C[], C)(const S[] patterns, const(char)[] flags = "") if (isSomeString!(S)) @trustedCompile regular expression pattern for the later execution. Returns: `Regex` object that works on inputs having the same character width as `pattern`.

fnauto regex(S)(S pattern, const(char)[] flags = "") if (isSomeString!(S)) @trustedditto

fnauto regexImpl(S)(const S pattern, const(char)[] flags = "") if (isSomeString!(typeof(pattern)))

private fnauto matchOnceImpl(RegEx, R)(R input, const auto ref RegEx prog) @trusted

private fn@safe auto matchOnce(RegEx, R)(R input, const auto ref RegEx prog) pure

private fnauto matchMany(RegEx, R)(R input, auto ref RegEx re) @safe

private fn

void replaceCapturesInto(alias  output,  Sink,  R,  T)(ref  Sink sink,  R input,  T captures) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R)

@trusted

private fn

void replaceMatchesInto(alias  output,  Sink,  R,  T)(ref  Sink sink,  R input,  T matches) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R)

private fnR replaceFirstWith(alias output, R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))

private fn

R replaceAllWith(alias  output,
         alias  method = matchAll,  R,  RegEx)(R input,  RegEx re) if (isSomeString!R &&  isRegexFor!(RegEx,  R))

fnauto match(R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))Start matching `input` to regex pattern `re`, using Thompson NFA matching scheme.

fnauto match(R, String)(R input, String re) if (isSomeString!R && isSomeString!String)ditto

fnauto matchFirst(R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))Find the first (leftmost) slice of the `input` that matches the pattern `re`. This function picks the most suitable regular expression engine depending on the pattern properties.

fnauto matchFirst(R, String)(R input, String re) if (isSomeString!R && isSomeString!String)ditto

fnauto matchFirst(R, String)(R input, String[] re...) if (isSomeString!R && isSomeString!String)ditto

fnauto matchAll(R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))Initiate a search for all non-overlapping matches to the pattern `re` in the given `input`. The result is a lazy range of matches generated as they are encountered in the input going left to right.

fnauto matchAll(R, String)(R input, String re) if (isSomeString!R && isSomeString!String)ditto

fnauto matchAll(R, String)(R input, String[] re...) if (isSomeString!R && isSomeString!String)ditto

fnauto bmatch(R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))Start matching of `input` to regex pattern `re`, using traditional https://en.wikipedia.org/wiki/Backtracking matching scheme.

fnauto bmatch(R, String)(R input, String re) if (isSomeString!R && isSomeString!String)ditto

void replaceFmt(R,  Capt,  OutR)(R format,  Capt captures,  OutR sink,  bool  ignoreBadSubs =  false) if (isOutputRange!(OutR,  ElementEncodingType!R[])  &&
     isOutputRange!(OutR,  ElementEncodingType!(Capt.String)[]))

R replaceFirst(R,  C,  RegEx)(R input,  RegEx re,  const(C)[]  format) if (isSomeString!R &&  is(C :  dchar)  &&  isRegexFor!(RegEx,  R))

Construct a new string from `input` by replacing the first match with a string generated from it according to the `format` specifier.

fnR replaceFirst(alias fun, R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))This is a general replacement tool that construct a new string by replacing matches of pattern `re` in the `input`. Unlike the other overload there is no format string instead captures are passed t...

void replaceFirstInto(Sink,  R,  C,  RegEx)(ref  Sink sink,  R input,  RegEx re,  const(C)[]  format) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R
    &&  is(C :  dchar)  &&  isRegexFor!(RegEx,  R))

@trustedA variation on replaceFirst that instead of allocating a new string on each call outputs the result piece-wise to the `sink`. In particular this enables efficient construction of a final output inc...

void replaceFirstInto(alias  fun,  Sink,  R,  RegEx)(Sink sink,  R input,  RegEx re) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R &&  isRegexFor!(RegEx,  R))

@trustedditto

R replaceAll(R,  C,  RegEx)(R input,  RegEx re,  const(C)[]  format) if (isSomeString!R &&  is(C :  dchar)  &&  isRegexFor!(RegEx,  R))

@trustedConstruct a new string from `input` by replacing all of the fragments that match a pattern `re` with a string generated from the match according to the `format` specifier.

fnR replaceAll(alias fun, R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R)) @trustedThis is a general replacement tool that construct a new string by replacing matches of pattern `re` in the `input`. Unlike the other overload there is no format string instead captures are passed t...

void replaceAllInto(Sink,  R,  C,  RegEx)(Sink sink,  R input,  RegEx re,  const(C)[]  format) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R
    &&  is(C :  dchar)  &&  isRegexFor!(RegEx,  R))

@trustedA variation on replaceAll that instead of allocating a new string on each call outputs the result piece-wise to the `sink`. In particular this enables efficient construction of a final output incre...

void replaceAllInto(alias  fun,  Sink,  R,  RegEx)(Sink sink,  R input,  RegEx re) if (isOutputRange!(Sink,  dchar)  &&  isSomeString!R &&  isRegexFor!(RegEx,  R))

@trustedditto

R replace(alias  scheme =  match,  R,  C,  RegEx)(R input,  RegEx re,  const(C)[]  format) if (isSomeString!R &&  isRegexFor!(RegEx,  R))

Old API for replacement, operation depends on flags of pattern `re`. With "g" flag it performs the equivalent of replaceAll otherwise it works the same as replaceFirst.

fnR replace(alias fun, R, RegEx)(R input, RegEx re) if (isSomeString!R && isRegexFor!(RegEx, R))ditto

Splitter!(keepSeparators,  Range,  RegEx) splitter(
     Flag!"keepSeparators" keepSeparators =  No.keepSeparators,  Range,  RegEx)(Range r,  RegEx pat) if (
     is(BasicElementOf!Range :  dchar)  &&  isRegexFor!(RegEx,  Range))

ditto

fnString[] split(String, RegEx)(String input, RegEx rx) if (isSomeString!String && isRegexFor!(RegEx, String)) @trustedAn eager version of `splitter` that creates an array with splitted slices of `input`.

fnauto escaper(Range)(Range r)A range that lazily produces a string output escaped to be used inside of a regular expression.

Templates 1

tmplctRegexImpl(alias pattern, string flags = "")

Functions

bool func(BacktrackingMatcher!Char matcher)

std.regex

Synopsis

See Also

Syntax and general information

Pattern syntax

Character classes

Regex flags

Unicode support

Replace format string

Slicing and zero memory allocations orientation

Copyright

License

Authors

Types 7

Parameters

Returns

Functions 34

Templates 1