Built-in Regex terms are not JSON-quoted inside containers (list[types.email] generates [[email protected]])

Describe the issue as clearly as possible:

Built-in Regex terms from outlines.types are not JSON-quoted when they appear inside a container, so list[types.email] generates [[email protected]] instead of ["[email protected]"]. The result does not parse as JSON.

This came out of review on #1961, which quotes date/time/datetime in containers. That fix is keyed to the terms python_types_to_terms returns for a Python type. But the same function has if isinstance(ptype, Term): return ptype at the top, so a built-in term handed in directly lands in the same branch of _ensure_json_quoted and comes back bare. Credit to @Sanjays2402 for spotting it.

Dict keys are already fine, since _ensure_json_quoted is called there with quote_regex=True. The gap is list items and dict values.

There are 29 built-in Regex terms. Three of them, date/time/datetime, are #1961. The other 26 fall into four groups.

Already correct, 1 term: string is "[^"]*", so it carries its own quotes and list[types.string] already matches ["abc"].

Bare is the right shape, 6 terms: integer, number, boolean, digit, latitude, longitude. Quoting these would break them.

Valid JSON strings once quoted. These are the real bug, 14 terms: email, uuid4, ipv4, ipv6, mac_address, semver, slug, hex_color, hex_str, credit_card, char, bic, e164, iban.

Quoting is necessary but not sufficient, 5 terms: newline ((\r\n|\r|\n)), whitespace (\s), sentence ([A-Z].*\s*[.!?]), paragraph (same, ending in \n+). Three different treatments hide in this bucket, so a fix that handles one and assumes it covers the rest will be wrong. Counting the characters JSON must escape (", the backslash, and U+0000 through U+001F) that each pattern admits anywhere inside a match:

newline      2    '\n' '\r'
whitespace   9    '\t' '\n' '\x0b' '\x0c' '\r' '\x1c' '\x1d' '\x1e' '\x1f'
sentence     34   all of them
paragraph    34   all of them

For newline and whitespace, escaping control characters is enough. For sentence and paragraph it is not, because .* also admits the backslash and a bare quote, and those two fail differently from a control character:

sentence matches 'A\.'   quote-only gives ["A\."]   json.loads -> Invalid \escape: line 1 column 4 (char 3)
sentence matches 'A".'   quote-only gives ["A"."]   json.loads -> Expecting ',' delimiter: line 1 column 5 (char 4)

So those two want whatever json.dumps does to a string, not an escape list assembled by hand. string stays out of the bucket entirely: "[^"]*" needs both quotes, so it never fullmatches a lone quote. Credit to @ErenAta16 for working the three cases apart.

isbn belongs in this bucket too, for a different reason (credit to @ErenAta16 for catching it). Its pattern carries four $ anchors inside lookaheads, so once it sits inside \[(...)\] the alternation is unsatisfiable and nothing matches list[isbn] at all, quoted or bare. Dropping the anchors pins them as the cause:

Regex(types.isbn.pattern.replace('$', ''))   in a list, bare    '[9783161484100]'    -> True

That line is False with the anchors in place. The quoted form stays False either way, since nothing on main quotes isbn, so isbn needs both the anchor fix and the quoting fix.

Note the predictor is anchor semantics, not the $ character. email has two $ characters inside a character class and list[email] matches [[email protected]] fine. Dropping the anchors changes what bare isbn accepts, so it needs the same maintainer call as the other four.

These failures are not all loud. json.loads reads a lone control character between brackets as whitespace:

json.loads('[\n]')     []
json.loads('[\t]')     []
json.loads('["a"]')    ['a']

So list[types.newline] generates text a parser accepts as an empty array. The caller asked for one element, the generation produced one, and the parse yields zero, with no exception raised anywhere. That is the worst case in this issue, worse than the JSONDecodeError rows, because nothing surfaces.

So the naive fix of growing the temporal tuple one type at a time is wrong twice over: it misses 14 terms today, and it would quietly mis-signal the other 5 as fixed.

One design question for a maintainer before anyone writes this. The cleanest shape I can see is to invert the rule: identity-match a short denylist of JSON-scalar built-ins and quote every other built-in term, leaving user-supplied Regex alone as it is today. That stops the list growing for new string-shaped types, and only a new numeric type would ever need adding. Whether the 4 escaping terms should be excluded from that, or fixed in the same pass, is the part worth a decision, keeping in mind that two of them need control-character escaping and two need full string escaping.

Steps/code to reproduce the bug:

import re
import outlines.types as types
from outlines.types.dsl import to_regex, python_types_to_terms

def matches(annotation, sample):
    return bool(re.fullmatch(to_regex(python_types_to_terms(annotation)), sample))

# list items
print(matches(list[types.email], '[[email protected]]'))      # True, not valid JSON
print(matches(list[types.email], '["[email protected]"]'))    # False, valid JSON rejected

# dict values
print(matches(dict[str, types.email], '{"k":[email protected]}'))    # True

# dict keys already handled by quote_regex=True
print(matches(dict[types.email, str], '{"[email protected]":"v"}'))  # True

Swap types.email for uuid4, ipv4, ipv6, mac_address, semver, slug, hex_color, hex_str, credit_card, char, bic, e164 or iban for the same result. isbn behaves differently: nothing matches, see the anchor note above.

Expected result:

Containers holding a string-shaped built-in term generate valid JSON:

    list[types.email]        matches ["[email protected]"]  and rejects [[email protected]]
    dict[str, types.email]   matches {"k":"[email protected]"}

Numeric and boolean built-ins keep their current bare output:

    list[int]                matches [5]
    list[bool]               matches [True]

On that last line: types.boolean is (True|False), so list[bool] matches [True] and rejects [true]. Python literals, not JSON. latitude and longitude have the sibling problem, a leading + that JSON rejects, so list[types.latitude] matches [+45.5] and json.loads refuses it. Both are separate bugs from the quoting one and not in scope here. The scalar denylist keeps all three bare either way, so neither changes the shape proposed above.

Error message:

No exception. The generated regex silently accepts non-JSON output.

Outlines/Python version information:

Version information

main @ be2cd15, re-verified on main @ 7d06847 (also reproduces on the #1961 branch)

Context for the issue:

Happy to open the PR once there is a call on the two questions above: whether to invert to a scalar denylist, and what to do about the 4 control-character terms.