polyglot

Polyglot

Rust/Wasm-powered SQL transpiler for more than 30 SQL dialects, inspired by sqlglot.

Polyglot parses, generates, transpiles, and formats SQL across more than 30 SQL dialects. It ships as:

• a Rust crate (polyglot-sql)
• a TypeScript/WASM SDK (@polyglot-sql/sdk)
• a Python package (polyglot-sql)
• a Go SDK (github.com/tobilg/polyglot/packages/go) backed by the native FFI library

There's also a playground where you can try it out in the browser, as well as Rust API Docs, TypeScript API Docs, and Python API Docs.

Release notes are tracked in CHANGELOG.md.

Features

• Transpile SQL between any pair of more than 30 SQL dialects
• Parse SQL into a fully-typed AST
• Generate SQL back from AST nodes
• Format / pretty-print SQL
• Fluent builder API for constructing queries programmatically
• Validation with syntax, semantic, and schema-aware checks
• Column lineage and OpenLineage-compatible payload generation
• Compact query analysis facts for projections, relations, CTEs, and set operations
• AST visitor utilities for walking, transforming, and analyzing queries
• Stack-safety hardening on native targets via default-on stacker
• C FFI shared/static library for multi-language bindings (polyglot-sql-ffi)
• Python bindings powered by PyO3 (polyglot-sql on PyPI)

Supported Dialects

Quick Start

Rust

use polyglot_sql::{transpile, DialectType};

// Transpile MySQL to PostgreSQL
let result = transpile(
    "SELECT IFNULL(a, b) FROM t",
    DialectType::MySQL,
    DialectType::Postgres,
).unwrap();
assert_eq!(result[0], "SELECT COALESCE(a, b) FROM t");

use polyglot_sql::builder::*;

// Fluent query builder
let query = select(["id", "name"])
    .from("users")
    .where_(col("age").gt(lit(18)))
    .order_by(["name"])
    .limit(10)
    .build();

See the full Rust crate README for more examples.

TypeScript

npm install @polyglot-sql/sdk

import { transpile, Dialect } from '@polyglot-sql/sdk';

// Transpile MySQL to PostgreSQL
const result = transpile(
  'SELECT IFNULL(a, b) FROM t',
  Dialect.MySQL,
  Dialect.PostgreSQL,
);
console.log(result.sql[0]); // SELECT COALESCE(a, b) FROM t

import { select, col, lit } from '@polyglot-sql/sdk';

// Fluent query builder
const sql = select('id', 'name')
  .from('users')
  .where(col('age').gt(lit(18)))
  .orderBy(col('name').asc())
  .limit(10)
  .toSql('postgresql');

See the full TypeScript SDK README for more examples.

Python

pip install polyglot-sql

import polyglot_sql

result = polyglot_sql.transpile(
    "SELECT IFNULL(a, b) FROM t",
    read="mysql",
    write="postgres",
)
print(result[0])  # SELECT COALESCE(a, b) FROM t

See the full Python bindings README.

Go

go get github.com/tobilg/polyglot/packages/go

The Go module contains the PureGo wrapper only. Runtime API calls require a separate matching polyglot-sql-ffi shared library (.so, .dylib, or .dll) from the same Polyglot release or a local FFI build:

cargo build -p polyglot-sql-ffi --profile ffi_release
export POLYGLOT_SQL_FFI_PATH="$PWD/target/ffi_release/libpolyglot_sql_ffi.so"

import (
    "fmt"

    polyglot "github.com/tobilg/polyglot/packages/go"
)

client, err := polyglot.OpenDefault()
if err != nil {
    panic(err)
}
defer client.Close()

result, err := client.Transpile(
    "SELECT IFNULL(a, b) FROM t",
    "mysql",
    "postgres",
)
if err != nil {
    panic(err)
}
fmt.Println(result[0]) // SELECT COALESCE(a, b) FROM t

The Go SDK uses PureGo over polyglot-sql-ffi; it does not download native libraries or bundle release artifacts in the Go module. Build/download the FFI shared library and set POLYGLOT_SQL_FFI_PATH or pass its path to polyglot.Open. See the full Go SDK README.

Lineage and OpenLineage Output

Polyglot can trace column lineage through SQL queries and can generate OpenLineage-compatible JSON payloads from that analysis. The OpenLineage support currently produces columnLineage dataset facets, optional schema facets, and JobEvent / RunEvent payloads for supported query shapes such as SELECT, set-operation queries, INSERT ... SELECT, and CREATE TABLE AS SELECT.

Lineage can be selected by output name or by zero-based output ordinal. Ordered output metadata preserves unnamed projections and unresolved wildcards, so callers can decide whether a positional lookup is complete before tracing it. Immediate set-operation branch roots include the operator, original zero-based branch ordinal, and ALL flag. For OpenLineage, both UNION branches are direct value dependencies; the right branch of EXCEPT and INTERSECT is an indirect FILTER dependency.

OpenLineage transport and client emission are intentionally out of scope: Polyglot builds payloads for callers to inspect, persist, or send through their own infrastructure.

• TypeScript SDK examples: packages/sdk/README.md
• Python examples: crates/polyglot-sql-python/README.md
• Go examples: packages/go/README.md

Compact Query Analysis

For applications that need summary facts instead of a full AST or full lineage graph, analyze_query / analyzeQuery returns a compact payload with output projections, direct visible relations, transitive physical baseTables, CTE names and top-level cteFacts, original starProjections, set-operation branches, transform kinds, conservative projection nullability, optional type hints, and upstream column references. The API is additive and uses the same optional ValidationSchema shape as schema-aware validation and lineage. Each set-operation branch is classified as a value or filter contribution. columnUses separately groups references used in joins, filters, grouping, HAVING/QUALIFY, window partition/order/frame expressions, ordering, aggregate filters/ordering, and set-operation filter branches. Each fact includes a context, scopePath, expressionPath, dialect-rendered expressionSql, and references using the existing source identity and confidence vocabulary. For example, SELECT o.id FROM orders o WHERE o.amount > 0 reports an amount use with context filter; the id projection's upstream facts are unchanged.

Paths distinguish CTEs, derived tables, subqueries and nested set-operation branches (for example root.branches[1]). They are deterministic within an analysis, not persistent identifiers across edits. References preserve repeated occurrences; multiple upstream dependencies may share a use's location. Optional span objects contain half-open start/end Unicode-character offsets into the original SQL, not UTF-8 bytes or JavaScript UTF-16 offsets. Reference spans locate the original use, including uses of CTE columns or output aliases. Complete expression spans are returned only when available; generated SQL is not used to invent source locations. Implicit NATURAL JOIN keys need complete source columns; otherwise an unknown wildcard reference records that gap. These are direct query-analysis facts, not cross-query impact analysis.

Validation uses broad type families, while query analysis preserves detailed schema type strings such as DECIMAL(10,2) for typeHint values when they can be parsed. For physical relations, name remains the qualified display name while catalog, schema, and table expose parsed identifier parts for consumers that need to distinguish qualifiers from table names.

Validation schema JSON uses:

{
  "strict": true,
  "tables": [
    {
      "name": "orders",
      "schema": "analytics",
      "aliases": ["o"],
      "primaryKey": ["id"],
      "uniqueKeys": [["external_id"]],
      "foreignKeys": [
        {
          "columns": ["customer_id"],
          "references": { "table": "customers", "columns": ["id"] }
        }
      ],
      "columns": [
        { "name": "id", "type": "INT", "nullable": false, "primaryKey": true },
        { "name": "amount", "type": "DECIMAL(10,2)", "nullable": true }
      ]
    }
  ]
}

Use the type key for column types in JSON. dataType / data_type are not accepted aliases for this schema payload.

Format Guard Rails

SQL formatting runs through guard limits in Rust core to prevent pathological inputs from exhausting memory:

• maxInputBytes: 16 MiB (default)
• maxTokens: 1_000_000 (default)
• maxAstNodes: 1_000_000 (default)
• maxSetOpChain: 256 (default)

Guard failures return error codes in the message (E_GUARD_INPUT_TOO_LARGE, E_GUARD_TOKEN_BUDGET_EXCEEDED, E_GUARD_AST_BUDGET_EXCEEDED, E_GUARD_SET_OP_CHAIN_EXCEEDED).

Configuration surface by runtime:

• Rust: configurable via format_with_options.
• WASM: configurable via format_sql_with_options / format_sql_with_options_value.
• TypeScript SDK: configurable via formatWithOptions.
• C FFI: configurable via polyglot_format_with_options.
• Python: configurable via keyword-only format_sql(..., max_*) overrides.

WASM low-level example (from polyglot-sql-wasm exports):

import init, { format_sql_with_options } from "./polyglot_sql_wasm.js";

await init();
const raw = format_sql_with_options(
  "SELECT a,b FROM t",
  "generic",
  JSON.stringify({
    maxInputBytes: 2 * 1024 * 1024,
    maxTokens: 250000,
    maxAstNodes: 250000,
    maxSetOpChain: 128
  }),
);
const result = JSON.parse(raw);

Stack Safety

On native Rust builds, polyglot-sql enables the optional stacker feature by default. This adds stack-growth protection around the deepest parser / generator / transpile entry points so pathological or heavily nested SQL is less likely to abort the process with a stack overflow.

Important scope notes:

• Rust, C FFI, and Python native builds inherit this by default.
• WASM does not use stacker; polyglot-sql-wasm depends on the core crate with default-features = false.
• A few paths still use explicitly larger thread stacks as defense-in-depth for very deep workloads, especially some test harnesses, the Python worker thread, and the bench_json example.

If you want to disable stacker for a native Rust build, turn off default features and opt back into the ones you need:

[dependencies]
polyglot-sql = { version = "0.11.0", default-features = false, features = ["all-dialects", "transpile"] }

That can reduce overhead slightly on trusted inputs, but you lose the default stack-growth protection for deeply nested SQL.

Project Structure

…

Examples

Standalone example projects are available in the examples/ directory. Each one pulls the latest published package and can be run independently.

Rust

cargo run --manifest-path examples/rust/Cargo.toml

TypeScript

cd examples/typescript
pnpm install --ignore-workspace && pnpm start

Building from Source

# Build Rust core
cargo build -p