[BUG] Scheduled pipeline run recorded LOCAL_PYTHON instead of configured K8S executor and never dispatched

Mage version

0.9.77

Describe the bug

We observed a production scheduled pipeline run that was created successfully, marked as started, and had block runs created, but never dispatched execution and never created a Kubernetes Job.

• Pipeline run created successfully
• started_at populated
• Pipeline status transitioned to RUNNING
• Eight block runs created
• All block runs remained with:
  • started_at = NULL
  • completed_at = NULL
• No Kubernetes Job was created
• No execution pod was created
• Scheduler continued emitting heartbeat messages for the run
• Pipeline run eventually required manual cancellation

The most notable anomaly is that the failed run recorded: executor_type = LOCAL_PYTHON

while:

• The pipeline's metadata.yaml specifies: executor_type: k8s
• All blocks in the pipeline are configured for K8S execution.
• The workspace has DEFAULT_EXECUTOR_TYPE=k8s.
• All successful executions of this pipeline over at least the previous month recorded: executor_type = k8s

Pipeline completed successfully on a manual rerun. That database record showed the executor_type = k8s.

Trigger Configuration

The trigger configuration in logs was identical between the failed run and successful rerun.

Pipeline Configuration

The pipeline metadata.yaml contains: executor_type: k8s

Workspace Configuration

DEFAULT_EXECUTOR_TYPE is also configured as k8s

To reproduce

No response

Expected behavior

Based on our reading of:

• pipeline_scheduler_original.py
• executor_factory.py
• pipeline.py
• k8s_pipeline_executor.py

the run should resolve to the K8S executor and create a Kubernetes Job.

Screenshots

Operating system

No response

Additional context

No response