Hidden rules with regex tokens silently match empty strings instead of reporting errors

Problem

When using hidden rules (rules starting with _) with regex tokens that require at least one character (like /[A-Za-z0-9_]+/), the parser silently matches empty strings at zero-width positions instead of reporting syntax errors. This behavior is inconsistent with non-hidden rules and string literal tokens.

Steps to reproduce

Grammar 1: Hidden rule with regex (problematic).

module.exports = grammar({
  name: 'test',
  rules: {
    source_file: $ => seq('.', $.id),
    id: $ => $._id,
    _id: _ => token(/[A-Za-z0-9_]+/),
  },
});

Input file 1.txt:

.

Command:

tree-sitter generate
tree-sitter parse 1.txt

Actual Output:

(source_file [0, 0] - [1, 0]
  (id [0, 1] - [0, 1]))

Expected Behavior: Should report a MISSING error for id since there's no content after the dot.

Grammar 2: Non-hidden rule with regex (correct).

module.exports = grammar({
  name: 'test',
  rules: {
    source_file: $ => seq('.', $.id),
    id: _ => token(/[A-Za-z0-9_]+/),
  },
});

Output:

(source_file [0, 0] - [1, 0]
  (id [0, 1] - [0, 1]))
1.txt   Parse:    0.87 ms            2 bytes/ms (MISSING id [0, 1] - [0, 1])

This correctly reports MISSING!

Grammar 3: Direct hidden rule reference.

module.exports = grammar({
  name: 'test',
  rules: {
    source_file: $ => seq('.', $._id),
    _id: _ => token(/[A-Za-z0-9_]+/),
  },
});

Output:

(source_file [0, 0] - [1, 0])

Again, silently fails without error!

Grammar 4: Hidden rule with string literal.

module.exports = grammar({
  name: 'test',
  rules: {
    source_file: $ => seq('.', $._id),
    _id: _ => token('a'),
  },
});

Output:

(source_file [0, 0] - [1, 0])
1.txt   Parse:    0.02 ms           89 bytes/ms (MISSING "a" [0, 1] - [0, 1])

String literals correctly report errors!

Expected behavior

Provide consistent error reporting behavior

Tree-sitter version (tree-sitter --version)

tree-sitter 0.25.10 (da6fe9beb4f7f67beb75914ca8e0d48ae48d6406)

Operating system/version

macOS Sequoia 15.5 arm64