#11878 Refactor data-schema core around inspectable schema objects

Summary

Refactor @remix-run/data-schema so built-in schemas are plain objects we can inspect. Put the data that describes each rule on the schema object itself. Validators should read that data from the object instead of hiding it in a closure.

The schema object should be the one source of truth for validation and for tools that need to inspect a schema. Do not add a second description tree like schema['~def'] next to the validator.

This is close to the approach Valibot uses. A schema is still a small object with a ~run method, but it also has fields for its kind, object entries, array item, union members, wrapper source, and other options.

schema object
├── validation method
└── rule data
    ├── kind
    ├── child schemas
    └── options

This would let a future remix/data-schema/json-schema export walk a data-schema object without making the main data-schema module import the JSON Schema converter. It should also avoid the bundle and memory cost of keeping the same data in both a closure and a separate description object.

Current and target shape

Today, schema constructors close over their arguments. This works for validation, but other tools cannot see how the schema was built. For example, there is no way to inspect an object schema and find its entries or unknown-key behavior.

function object(entries, options) {
  return createBuiltinSchema(function validate(value, context) {
    // `entries` and `options` are only visible inside this closure.
  })
}

The goal is to store this data on the schema object. The exact property names and types do not need to match this example.

function object(entries, options) {
  return createBuiltinSchema({
    kind: 'object',
    entries,
    unknownKeys: options?.unknownKeys ?? 'strip',

    '~run'(value, context) {
      // Validation reads `this.entries` and `this.unknownKeys`.
    },
  })
}

Schemas will still contain functions because they still need to validate values. The important part is that the data that describes a rule is no longer available only inside a closure.

We also need to keep the exact order of composed operations. For example, these operations cannot be swapped:

number()
  .transform((value) => String(value))
  .refine((value) => value.length === 2)

The schema object should describe each stage in order. A simple model might look like this:

{
  kind: 'refine',
  source: {
    kind: 'transform',
    source: { kind: 'number' },
    transform: value => String(value),
  },
  predicate: value => value.length === 2,
}

The final shape is up to the implementation, but it must keep the source and order of every pipe, refine, and transform step.

Requirements

Keep the current public constructors, chainable methods, Schema type, inference helpers, parsing functions, and Standard Schema validation behavior.
Keep the current behavior for primitives, objects, arrays, records, tuples, maps, sets, unions, variants, optional and nullable values, defaults, lazy schemas, coercion, and form data.
Store each built-in schema's kind, child schemas, and options on the schema object instead of keeping them only in its validator closure.
Make pipe, refine, and transform ordered and inspectable without changing their input and output types.
Keep checks inspectable through their existing code and values where possible. Custom predicates and transforms may stay as functions, but the schema should make it clear that they cannot be described as plain data.
Keep createSchema(validator) working for custom schemas. Custom validators may remain opaque, but wrappers around them must keep their validation context and issue paths correct.
Keep the mutable path optimization for built-in schemas and the stable path objects passed to custom schemas.
Keep abortEarly, error maps, locale handling, issue order, issue paths, unknown-key modes, default factories, sparse arrays, and custom iterators working as they do today.
Make sure recursive schemas made with lazy() can be built, inspected, and validated without resolving an endless cycle.
Do not add a second public definition API or copy the same constructor data into both a closure and a ~def object.
Keep the inspectable fields private for now. We can define a public inspection API later if we need one.
Do not add JSON Schema conversion in this issue. A later change can add toJsonSchema() and toStandardJsonSchema() from a separate remix/data-schema/json-schema export.
A future converter may depend on data-schema internals. The main data-schema module must never import the converter.

remix/data-schema/json-schema → data-schema schema objects
remix/data-schema             ✕ JSON Schema converter

What must not change

The schema objects will have a new internal shape, but their public behavior should stay the same:

schema['~standard'].validate(value, options) must work exactly as it does today.
parse() and parseSafe() must keep accepting any Standard Schema-compatible schema, not just data-schema objects.
InferInput and InferOutput must return the same types for every constructor and order of operations.
Custom schemas must keep receiving stable context.path values. They must never see the mutable path used inside built-in schemas.
Chained operations must run in the same order and stop or collect issues under the same conditions.
Named imports must stay tree-shakable. Importing a primitive schema must not pull in other validators or future JSON Schema code.

Performance

This refactor must keep data-schema's current size and speed. Compare the old and new code on the same machine and runtime. Source line count is not a useful measure here.

Measure:

Minified and compressed bundle size for a few named imports, including one primitive schema and one nested object schema.
Cold import heap and RSS.
Schema construction speed.
Retained heap and RSS per schema.
Valid and invalid object and array validation speed.
Peak and retained memory during validation.

Store constructor data once. If a schema object already has entries, source, checks, or members, its validator should read those fields instead of closing over another copy of the same references.

Implementation plan

Define an internal object model for built-in schemas, their child schemas, and their options without adding a second definition tree.
Update schema creation and the Standard Schema bridge so built-in validators read rule data from their schema objects.
Move primitive, collection, object, wrapper, union, and variant schemas to the new object model without changing their public types or behavior.
Move coercion, lazy, and form-data schemas to the same object model while keeping their current package boundaries.
Store pipe, refine, and transform as ordered steps that keep their source schema and operation.
Keep the current createSchema() API and make it clear that custom validators cannot always be inspected.
Keep the mutable path for built-in validation and stable paths for custom schemas across nested schemas and wrappers.
Add tests for the inspectable shape of each kind of schema and the order of composed operations.
Keep all current behavior and type tests passing, including tests for inference, issue paths, abort behavior, defaults, unknown keys, lazy recursion, sparse arrays, and custom iterators.
Add or update benchmarks for schema construction, retained schema memory, named-import bundle size, and cold import cost.
Record before-and-after benchmark results and fix any clear drop in bundle size, memory use, schema construction speed, or validation speed.
Update extension docs if the guidance for createSchema() changes, while keeping current public examples valid.
Add the needed @remix-run/data-schema change file, and only update the main Remix release notes if the public API changes.

Required checks

pnpm --filter @remix-run/data-schema run test --quiet passes.
pnpm --filter @remix-run/data-schema run typecheck passes.
pnpm --filter @remix-run/data-schema run build passes.
The data-schema benchmarks pass and include before-and-after results from several runs.
A few named-import bundles show that tree shaking still works and that unrelated schema types and JSON Schema code are not included.
Cold import cost, retained schema memory, schema construction, validation speed, and validation memory stay within normal run-to-run changes.
pnpm run validate-package-meta, pnpm run lint, and pnpm run format:check pass.
pnpm run test:changed and pnpm run typecheck:changed pass.

Related work

#11747 explores JSON Schema conversion and shows why data-schema needs rule data that other code can inspect.
Valibot uses schema objects that can be inspected and keeps JSON Schema support in a separate @valibot/to-json-schema package: https://valibot.dev/guides/json-schema/