Granular OpenLineage event emission on `feast apply` for precise FeatureView lineage

Is your feature request related to a problem? Please describe. When openlineage.emit_on_apply: true is enabled in feature_store.yaml, running feast apply currently aggregates all defined data sources, entities, and feature views into a single bulk definition job event (feast.<project>.feature_views).

Because OpenLineage lineage engines (including Feast's lineage consumer) infer dataset-to-dataset relationships from job inputs and outputs, placing all data sources as inputs and all feature views as outputs of the same job causes the lineage graph to form a full $(M \times N)$ Cartesian product. Every data source appears connected to every feature view (as dashed derived edges), regardless of whether they are actually related in the feature store definitions.

In feature stores with multiple data sources and feature views, this makes the OpenLineage lineage graph cluttered and inaccurate for downstream consumers and the UI.

Describe the solution you'd like

FeastOpenLineageEmitter.emit_apply() could emit more granular, per-FeatureView definition events instead of a single monolithic job event. Specifically:

1. For each FeatureView / StreamFeatureView / OnDemandFeatureView, emit a distinct definition event where:
   • inputs: Only the data sources and entities declared for that specific feature view.
   • outputs: That specific feature view dataset.
2. Maintain the existing pattern used for FeatureService definitions (which are already emitted individually per service). This ensures that OpenLineage accurately captures the true 1:1 or N:1 dependency graph between sources and feature views both in Feast UI and in external lineage backends (e.g., Marquez, DataHub, Atlan).

Describe alternatives you've considered

1. Relying on the UI's Feast Registry Lineage toggle:
   • While the built-in "Feast registry lineage" view correctly parses definitions directly from the registry proto, it does not help external OpenLineage consumers/catalogs that rely on standard OpenLineage event streams emitted during CI/CD or feast apply.
2. Filtering or matching datasets in the backend processor:
   • Having OpenLineageProcessor inspect facets to filter out unrelated edges is fragile. Emitting accurate event boundaries at the source (FeastOpenLineageEmitter) adheres more closely to OpenLineage design principles.

Additional context

Technical Details

• Bulk emission in emitter: In sdk/python/feast/openlineage/emitter.py within emit_apply(), all sources and entities are collected into fv_inputs, and all feature views are collected into fv_outputs:

  python复制

  # sdk/python/feast/openlineage/emitter.py
  result1 = self._client.emit_run_event(
      job_name=feature_views_job_name(project),
      run_id=str(uuid.uuid4()),
      event_type=RunState.COMPLETE,
      inputs=fv_inputs,
      outputs=fv_outputs,
      job_facets=job_facets,
      namespace=namespace,
  )

• Transitive edge derivation: In sdk/python/feast/openlineage/processor.py, _build_dataset_to_dataset_edges() maps all job inputs to all job outputs:

  python复制

  # sdk/python/feast/openlineage/processor.py
  for inp in inputs:
      for out in outputs:
          self._store.upsert_lineage_edge(
              source_type="dataset",
              source_name=inp["name"],
              target_type="dataset",
              target_name=out["name"],
              edge_type="derived",
          )

• Existing helper: feature_view_to_job() and emit_feature_view_lineage() in sdk/python/feast/openlineage/mappers.py already implement per-view mapping logic, which can be reused or aligned with emit_apply().