#132319 Embeddable resources: generic embeddings and hybrid search

Make manifest-declared resources embeddable and hybrid-searchable through Grafana's existing unified-storage embedding pipeline. Dashboard extraction and hybrid ranking already exist; this work adds generic resource support and migrates the dashboard caller to the shared API.

The SDK declaration work is complete in https://github.com/grafana/grafana-app-sdk/pull/1541. Grafana's dependency upgrade will wait for a release containing that PR.

Agreed design

Declare embedding inputs independently of search fields, under each API version's embed.fields. These fields generate embedding text; they do not enable filtering by those fields.
Resolve input paths using each object's stored apiVersion, including historical unserved versions. Different versions may use different paths.
Keep one manual embed[resource].reembedVersion per resource across all API versions. Input edits alone do not require a bump; increasing the number intentionally requests re-embedding through the existing version machinery.
Custom builders, including dashboards, own their version in Go and omit both declarative embedding inputs and the root embedding configuration entry.
Reuse vector_allowed_internal_collections as the single resource allowlist for both generation and query access. Existing global indexing/provider settings continue to apply; preserve today's dashboard defaults.
Mount the generic hybrid endpoint from search.hybrid and the API server's enable_hybrid_api switch. Route mounting does not depend on storage-server configuration. Generic filtering initially supports UID/folder; existing dashboard-specific filters remain dashboard-specific.
Reuse existing reconciliation, deletion, content-version backfills and provider batching support. Keep storage-server support and API-layer changes in separate PRs/deployments, with compatibility during version skew.

Remaining work

The enrollment sub-issue contains two implementation stages suitable for separate PRs: configuration/builder selection, followed by catalog provisioning and storage execution.

Suggested order

The generic builder, backfill page batching, and generic hybrid API contract can start while the SDK release is pending. The builder can initially use an unwired provider interface.

After the release, land the SDK/provider integration and storage enrollment support. Roll out storage support before enabling the API routes. Use a resource fully in unified storage for the first end-to-end rollout. Migrate the command palette in a separate frontend PR; retire the old dashboard endpoint only after deployed clients have moved.

Completion criteria

An enrolled generic resource can backfill existing objects, process new writes/updates/deletes, and serve authorized hybrid queries.
Tests cover different paths across stored API versions, historical unserved versions, custom-builder precedence, manual version increases, and safe skips when declarations are missing. Missing declarations preserve existing embeddings.
Generation and query access use the single agreed allowlist. Unsupported/disallowed requests fail before consuming provider quota.
Existing dashboard behavior and older API/storage combinations remain compatible through rollout.
The first generic-resource rollout is verified, the command palette uses the generic API, and the legacy dashboard endpoint is retired after migration.

Deferred

Keep the current event-triggered initial backfill: the first processed write, including an eligible replayed write, initializes a resource's backfill job. A quiet newly enrolled resource is not guaranteed an immediate backfill. Guaranteed bootstrap on enrollment and durable recovery of missed backfill events remain follow-ups.

Other deferred work includes automatic embedding-definition hashes, mandatory identical declarations across API versions, arbitrary filtering on embedding inputs, generic document chunking, and automatic cleanup of retained vector rows on unenrollment.