Regex Without Tears
The 20% of regular expressions that solves 95% of problems — tokens, anchors, groups, lookarounds, flags.
A regular expression describes a set of strings; the engine walks the input left to right and reports the first place the pattern fits. Most of the language is three ideas: tokens (what one character may be), quantifiers (how many times), and groups (what to capture or alternate). Anchors and lookarounds add conditions without consuming characters.
The syntax shown is the Perl-style core shared by JavaScript, Python, Java, PCRE and ripgrep. Flavors diverge at the edges — named groups, lookbehind, \d in POSIX tools — and the flavor sections call out where.
.+ swallows the rest of the input, finds no > at the end, and gives characters back one at a time. Lazy .+? takes one character and grows only until > can match.Core tokens
- . \d \w \s
- any char · digit · word char [A-Za-z0-9_] · whitespace (capitals negate)
- * + ? {n,m}
- 0+ · 1+ · 0/1 · between n and m — all greedy by default
- Lazy: *? +?
- shortest match —
<.+?>stops at the first > - [abc] [^abc] [a-z]
- set · negated set · range; inside sets most magic is literal
- a|b
- alternation — scope it with groups:
gr(a|e)y - {3} {2,}
- exactly 3 · 2 or more —
\d{3}is three digits - \. \* \( \\
- escape a metacharacter to match it literally —
\.is a real dot - \t \n \r
- tab · newline · carriage return (Windows lines end in
\r\n) - [\s\S]
- truly any character, newlines included — for engines without the
sflag - \p{L} \p{Lu}
- any letter / uppercase letter in any script — JS with
u, PCRE, Java; not Pythonre
Anchors & groups
- ^ $ \b
- line start · line end · word boundary (
\bcat\b≠ category) - (x) (?:x)
- capturing vs non-capturing group
- (?<name>x)
- named capture →
groups.name - \1 backreference
- match the same text again —
(\w+) \1finds doubled words - Replace with $1
- captured groups in replacements:
(\d+)-(\d+)→$2-$1 - \B
- NOT a word boundary —
\Bcatmatches inside bobcat, not the word cat - \k<name>
- backreference by name in JS, .NET and PCRE; Python writes
(?P=name) - $& and $<name>
- JS replacement strings: the whole match · a named group
- (?i)
- inline flag at the pattern start — Python, PCRE, Java, Go; Node 22 JavaScript rejects it
Lookarounds & flags
- (?=x) (?!x)
- lookahead: followed by / NOT followed by (zero-width)
- (?<=x) (?<!x)
- lookbehind: preceded by / not preceded by
- Price example
(?<=\$)\d+matches 42 in $42 without eating the $- Flags g i m s
- all matches · ignore case · ^$ per line · dot matches newline
- ^(?=.*\d)(?=.*[A-Z]).{8,}$
- stacked lookaheads AND conditions: a digit, an uppercase letter, 8+ characters
- Flag u (JS)
- Unicode mode:
\p{…}, whole code points such as emoji, stricter syntax - re.X / (?x)
- verbose mode: whitespace ignored and
#comments allowed — for long patterns - Lookbehind width
- JavaScript allows any length; Python needs a fixed width (
(?<=a+)is an error)
Flavors: JavaScript & Python
- str.match(/x/g)
- all matches as plain strings, groups dropped —
matchAllkeeps groups per match - re.test(s) with /g
- stateful:
lastIndexadvances, so calling it twice on one string can return false - s.replace(/x/g, '$1')
- needs the
gflag (orreplaceAll) to replace every match, not just the first - new RegExp(userInput)
- escape it first —
.*(in user text are live metacharacters - r'\d+'
- Python raw string: backslashes reach the regex engine untouched — always use one
- re.match vs re.search
matchanchors at the start,searchscans,fullmatchmust cover everything- (?P<name>…)
- Python’s named group; reuse it as
(?P=name), replace with\g<name> - re.sub(r"(\d+)", r"<\1>", s)
- Python replacements use
\1or\g<1>, never$1 - \d in Python 3
- matches any Unicode digit, Arabic-Indic too —
re.ASCIIor[0-9]for 0–9 only
Flavors: grep, sed, rg
- grep -E 'a|b+'
- extended regex:
+ ? | ( ) {}work unescaped — plain grep needs backslashes - grep -oE '[0-9]+'
- print only the matching part, one match per line
- \d in grep -E / sed
- not POSIX — GNU grep reads it as a plain
d; use[0-9]or[[:digit:]] - grep -P '(?<=id )\d+'
- Perl-compatible: lookarounds and
\d— GNU grep only; macOS grep has no-P - sed -E 's/(a+)b/\1/g'
- ERE in sed; backreferences are
\1, andgreplaces every match on the line - rg -P '(\w+) \1'
- ripgrep’s default engine has no backreferences or lookaround;
-Pswitches to PCRE2 - Go, Rust, RE2
- linear-time engines: no backreferences or lookaround, but immune to catastrophic backtracking
Performance & safety
- (a+)+$
- nested quantifiers backtrack exponentially on a near miss like
aaaa…b - ReDoS
- never run user-supplied patterns, or fragile ones on user input, without a timeout or linear engine
- a++ (?>…)
- possessive / atomic: never backtrack into it — PCRE, Java, Python 3.11+
- "[^"]*" not ".*"
- a negated class stops at the delimiter; greedy
.*runs to the end, then backtracks
Battle-tested snippets
- Trim
^\s+|\s+$→ replace with empty (needs thegflag)- Number with decimals
-?\d+(\.\d+)?- ISO date
\d{4}-\d{2}-\d{2}— shape only: 2024-13-99 passes, so validate the value in code- Duplicate words
\b(\w+)\s+\1\b- The prime rule
- if the regex needs comments, it should be code. Emails/URLs: use a library
- Thousands separators
\B(?=(\d{3})+(?!\d))→ replace with,turns 1234567 into 1,234,567- Hex color
^#?([0-9a-fA-F]{3}){1,2}$— 3 or 6 hex digits- UUID
^[0-9a-f]{8}-([0-9a-f]{4}-){3}[0-9a-f]{12}$with theiflag- URL slug
^[a-z0-9]+(?:-[a-z0-9]+)*$— no leading, trailing or doubled hyphens- Collapse whitespace
\s+→ one space, then trim the ends- key=value pairs
(\w+)=("[^"]*"|\S+)— quoted values may contain spaces
Worked example: Parse an access log, then count the errors
One line of a web server’s access log in the combined format:
203.0.113.7 - - [28/Oct/2026:10:15:32 +0000] "GET /api/users?id=42 HTTP/1.1" 404 512Build the pattern one field at a time, giving each field the tightest token that fits:
^(?<ip>\S+) \S+ \S+: the client address and two dash fields, each a run of non-spaces.\[(?<time>[^\]]+)\]: everything up to the closing bracket, using a negated class rather than.*?."(?<method>[A-Z]+) (?<path>[^ "?]+)[^"]*": the request. The path stops at a space, a quote or?, so the query string is dropped.(?<status>\d{3}) (?<bytes>\d+|-)$: the status, then a size that may be-.
Run it over five lines (two 404s on /api/users, one 500 on /api/login, two 200s) in Node:
const errs = {};
for (const line of log.split('\n')) {
const m = line.match(re);
if (m && +m.groups.status >= 400) {
const k = m.groups.status + ' ' + m.groups.path;
errs[k] = (errs[k] ?? 0) + 1;
}
}
// { '404 /api/users': 2, '500 /api/login': 1 }For a quick count in the shell a looser pattern is enough, because the status is the only 3-digit number right after a closing quote:
$ grep -cE '" [45][0-9]{2} ' access.log
3A line that does not fit returns null rather than a wrong field. Count those too: a sudden rise means the log format changed.
Recipes
Reformat dates (YYYY-MM-DD to DD/MM/YYYY) in three tools
# sed (ERE): \1 \2 \3 are the captured groups
sed -E 's/([0-9]{4})-([0-9]{2})-([0-9]{2})/\3\/\2\/\1/g' report.txt
// JavaScript, with named groups
text.replace(/(?<y>\d{4})-(?<m>\d{2})-(?<d>\d{2})/g, '$<d>/$<m>/$<y>')
# Python
re.sub(r'(\d{4})-(\d{2})-(\d{2})', r'\3/\2/\1', text)One pattern, three replacement syntaxes: \3 in sed and Python, $<d> or $3 in JavaScript. Use [0-9] in sed, where \d is not POSIX.
Validate a password policy with lookaheads
^(?=.*[a-z])(?=.*[A-Z])(?=.*\d)(?=.*[^A-Za-z0-9]).{12,}$
(?=.*[a-z]) a lowercase letter somewhere
(?=.*[A-Z]) an uppercase letter somewhere
(?=.*\d) a digit somewhere
(?=.*[^A-Za-z0-9]) a symbol somewhere
.{12,} at least 12 characters in totalEach lookahead checks from the start without consuming anything, so the conditions combine as AND. If you need a specific error message per rule, test the rules separately in code instead.
Extract every match with its groups
// JavaScript
for (const m of log.matchAll(/(?<level>ERROR|WARN) (?<code>\d+)/g)) {
console.log(m.groups.level, m.groups.code);
}
# Python
for m in re.finditer(r'(?P<level>ERROR|WARN) (?P<code>\d+)', log):
print(m['level'], m['code'])
# shell: count them
grep -oE '(ERROR|WARN) [0-9]+' app.log | sort | uniq -cmatchAll throws a TypeError without the g flag. String.match with g would return only the full matches and drop the groups.
Build a pattern from user input safely
// JavaScript: escape metacharacters, then build
const escaped = input.replace(/[.*+?^$()|[\]{}\\]/g, '\\$&');
const re = new RegExp('(?<!\\w)' + escaped + '(?!\\w)', 'i');
# Python
pattern = re.compile(r'(?<!\w)' + re.escape(user_input) + r'(?!\w)', re.I)Unescaped, an input like c++ or ( changes the pattern or throws. The lookarounds act as word boundaries that also work when the input starts or ends with a symbol — \bc\+\+\b never matches “C++ ”.
Spot catastrophic backtracking and fix it
// a 26-letter line ending in "!" (a near miss)
/^(\w+\s?)+$/.test('a'.repeat(26) + '!') // seconds: exponential backtracking
/^(?:\w+\s)*\w+$/.test('a'.repeat(26) + '!') // microseconds: one way to split the inputBecause \s? is optional, the first pattern can split a run of letters into words in 2^n ways and tries every one before failing. Requiring a separator in each repetition removes the ambiguity.
When to use it, and when not
Regex is the right tool for text with a flat, regular shape: pulling fields out of log lines, search-and-replace across a codebase, checking that an input looks like a postcode or a SKU, splitting on messy delimiters. One line of pattern replaces a page of index arithmetic.
- Nested structure: use a parser. HTML, JSON and balanced brackets need counting, which a true regular expression cannot do. Use an HTML parser,
JSON.parseor a real grammar. - CSV: use a CSV reader. Quoted fields can hold commas, quotes and newlines, and
split(',')or a hand-made pattern gets one of them wrong. - Emails, URLs, dates: use the platform.
new URL(), a date parser, and for email a confirmation message, because the only real test of an address is delivering to it. - Fixed text: use string methods.
startswith,in,includesandsplitare clearer, and there are no metacharacters to escape.
Trade-offs and limits
The big split is the engine. A backtracking engine tries alternatives one at a time, which buys rich features at the price of a worst case that can be exponential. An automaton engine such as RE2 tracks every alternative at once in time linear in the input, and drops the features that need to remember what was matched.
| Engine | Used by | Backreferences, lookaround | Worst case |
|---|---|---|---|
| Backtracking | JavaScript, Python re, Java, PCRE, Perl | Yes | Exponential on ambiguous patterns |
| Automaton | RE2, Go regexp, Rust regex, ripgrep’s default | No | Linear in the input |
The other cost is readability. A dense pattern is write-only unless you split it with verbose mode, name its groups and keep test strings beside it. Two simple patterns and a line of code usually beat one clever pattern.
Common mistakes
- Validating without anchors.
/\d{3}/.test('ab1234cd')is true, because the engine looks for the pattern anywhere. Use^…$orre.fullmatch. In Python$also matches before a final newline, sore.match(r'^\d{3}$', '123\n')succeeds;fullmatchand\Zdo not. [A-z]as “any letter”. The range runs through the code points between Z and a, so it also matches[ \ ] ^ _and the backtick. Write[A-Za-z].- Alternation swallowing the anchors.
^cat|dog$means “cat at the start, or dog at the end”, so it matches “xdog”. Group it:^(?:cat|dog)$. - Expecting a repeated group to keep every repetition.
(\d)+on “123” captures only the last digit, 3. Capture the run with(\d+), or collect each match withfindallormatchAll. - Capturing groups in split. Python’s
re.split(r'(,)', 'a,b')returns['a', ',', 'b']: captured separators come back in the list. Use(?:…)when you only need grouping.
Check yourself
What does re.findall(r'(\w)(\d)', 'a1 b2') return?
[('a', '1'), ('b', '2')]. With groups in the pattern, findall returns the groups as tuples, not the whole matches a1 and b2.
Does ^\d{3}$ match "123\n" in JavaScript’s test? In Python’s re.match?
JavaScript: no, false. Python: yes, because its $ also matches just before a trailing newline.
How many matches does /\bcat\b/g find in “cat concat cat’s”?
2: the first word, and the cat in cat’s, since the apostrophe is not a word character. Inside “concat” there is no boundary before the c.
On <a>x<b>, what does <[^>]+> match first, and how does it differ from the lazy pattern?
<a>, the same text, but with no backtracking: [^>] can never step past a >, so there is nothing to give back.
Questions about Regex Without Tears Cheat Sheet
What is the difference between greedy and lazy quantifiers?
A greedy quantifier such as * or + matches as much text as it can, then gives characters back one at a time only if the rest of the pattern fails. A lazy quantifier, written with a trailing question mark as in *? or +?, matches as little as possible and extends only when it has to. On the text "[a][b]", the pattern \[.+\] matches the whole string, while \[.+?\] stops at the first closing bracket and matches "[a]". Lazy is the quick fix for over-matching, but a negated class such as \[[^\]]*\] is usually faster and states the intent more precisely.
How do I match any character including newlines?
By default the dot matches any character except a line break. Turn on single-line or dotall mode to change that: the s flag in JavaScript (as in /a.b/s), re.DOTALL or an inline (?s) in Python, and (?s) in PCRE, Java and Go. In engines without such a flag, use a class that covers everything, such as [\s\S], which means any whitespace or any non-whitespace character. Do not confuse this with the m flag: multiline mode changes what ^ and $ match, letting them match at each line start and end, and does nothing to the dot.
What is a lookahead in regex?
A lookahead is a zero-width assertion: it checks what comes next without consuming it, so the match position does not move. (?=x) succeeds when x follows, and (?!x) succeeds when it does not. Lookbehinds, written with a less-than sign after the question mark, check what comes before. Because they consume nothing, several lookaheads at the start of a pattern combine as AND conditions, which is how password rules are usually written. They also let you match a number only when it follows a dollar sign, without including the sign in the match.
Why does my regex work in JavaScript but not in grep or Python?
Each tool implements its own flavor. POSIX tools like grep -E and sed -E do not support \d, lazy quantifiers or lookarounds, so use [0-9] there, or GNU grep -P for Perl syntax. Python spells named groups with an extra P after the question mark, uses \1 rather than $1 in replacements, requires fixed-width lookbehind, and lacks \p{L}. ripgrep, Go and RE2 run in linear time and therefore drop backreferences and lookaround entirely. In code, also watch string escaping: a backslash in a normal string literal must be doubled, which is why Python uses raw strings.