Platform Issue: streams that finish mid-sync keep reporting "Syncing" until the whole job ends, and their elapsed timer counts from the wrong transition

Platform Version: 2.1.0 (self-hosted via abctl). Verified still present on airbyte-platform@main as of 2026-09-16.

What step the error happened: During the Sync

Filing here because airbyte-platform auto-closes incoming PRs and redirects to this repo. A branch with the proposed fix is linked at the bottom.

Summary

On a connection's Streams tab, while a sync is running, two separate defects make the per-stream view unusable for tracking progress:

1. Every stream reports Syncing for the entire job, including streams that emitted COMPLETE hours earlier. Streams only flip to Synced when the whole job ends. A stream that fails mid-job likewise keeps showing Syncing instead of Incomplete.
2. The elapsed timer counts from each stream's most recent status transition, not from when it started. For a stream that already completed, the UI renders a "running for 1h 12m" timer that is really measuring time since it finished, and it keeps ticking upward.

Net effect: on a long sync with many streams, the Streams tab cannot tell you which streams are done, which failed, or how long any of them took. Every row reads "syncing" with a plausible-looking but meaningless duration.

This is easiest to see on a connection with many streams of very different sizes — small streams finish in the first minutes and then sit there claiming to be syncing for hours.

Root cause

1. Status: a running job marks every stream as running

computeStreamStatus.ts:

const isRunning =
  isSyncing ||
  statuses[0].runState === StreamStatusRunState.RUNNING ||
  statuses[0].runState === StreamStatusRunState.RATE_LIMITED;

isSyncing is !!syncProgressItem from useStreamsStatuses.ts, and useStreamsSyncProgress is gated on connectionStatus.status === ConnectionSyncStatus.running. The sync-progress payload contains an entry for every stream the running job has touched — finished ones included — so isSyncing stays true for a completed stream until the job ends.

isRunning then short-circuits the classification:

if (isRunning) {
  ...
  if (recordsExtracted && recordsExtracted > 0) {
    if (runningJobConfigType === "sync") {
      return { status: StreamStatusType.Syncing, ... };   // <-- returns here
    }
  }
}

// ...never reached while the job runs:
if (lastSuccessfulSync) {
  return { status: StreamStatusType.Synced, ... };
}

The stream's own COMPLETE record is sitting right there in statuses — lastSuccessfulSync even computes it a few lines earlier — but the Synced branch is below the isRunning branch, so it is unreachable for the rest of the job. The Incomplete branch is unreachable for the same reason.

2. Timer: elapsed is measured from the newest transition

useUiStreamsStates.ts:

uiState.activeJobStartedAt =
  currentJobId === streamStatus?.relevantHistory[0]?.jobId
    ? streamStatus?.relevantHistory[0]?.transitionedAt
    : undefined;

relevantHistory is sorted descending by transitionedAt (sortStreamStatuses in useStreamsStatuses.ts), so [0] is the newest transition — which for a finished stream is its COMPLETE. That value is assigned to a field named activeJobStartedAt and rendered as the running timer in LatestSyncCell.tsx:

const start = dayjs(syncStartedAt);
const end = dayjs(Date.now());
const hours = Math.abs(end.diff(start, "hour"));
const minutes = Math.abs(end.diff(start, "minute")) - hours * 60;

So a completed stream shows now − (time it completed), counting up. A genuinely running stream shows time since its RUNNING transition, which is roughly correct — hence a mix of plausible and nonsensical durations in the same table.

The Math.abs() calls suggest negative durations were hit at some point and clamped rather than traced back to the source.

#12996 ("check that latest status matches current job for calculating time elapsed") added the currentJobId === ...jobId guard, which correctly stops a previous job's timestamp leaking in. It does not address a terminal transition within the current job being used as a start time.

Steps to reproduce

1. Create a connection with several streams of very different sizes (e.g. one large stream and a handful of small ones).
2. Trigger a sync and open the connection's Streams tab.
3. Wait for the small streams to finish — confirm via the job logs or the stream_statuses table that they have emitted COMPLETE.
4. Observe: those streams still show Syncing, with an elapsed timer that counts up from the moment they completed.
5. They only settle to Synced once the large stream finishes and the job ends.

Expected behaviour

• A stream that reached a terminal run state within the currently-running job shows Synced (or Incomplete on failure), not Syncing.
• The elapsed timer for a running stream measures from that stream's first transition in the current job.
• A stream the current job has not started yet still shows Queued — a terminal status left over from a previous job must not be mistaken for having finished this one.

Proposed fix

Branch: IvanKaramazov/airbyte-platform@fix/stream-status-completed-streams-show-syncing

Two small changes plus regression tests:

computeStreamStatus.ts — thread the running job's id in, and exclude streams that already reached a terminal state within it. The jobId comparison is what keeps the not-yet-started case behaving as it does today:

const hasFinishedInRunningJob =
  runningJobId !== undefined &&
  statuses[0].jobId === runningJobId &&
  (statuses[0].runState === StreamStatusRunState.COMPLETE ||
    statuses[0].runState === StreamStatusRunState.INCOMPLETE);

const isRunning =
  !hasFinishedInRunningJob &&
  (isSyncing ||
    statuses[0].runState === StreamStatusRunState.RUNNING ||
    statuses[0].runState === StreamStatusRunState.RATE_LIMITED);

useUiStreamsStates.ts — take the oldest transition belonging to the current job as the start time:

const currentJobHistory =
  streamStatus?.relevantHistory.filter((status) => status.jobId === currentJobId) ?? [];
uiState.activeJobStartedAt = currentJobHistory.at(-1)?.transitionedAt;

With the first change, a completed stream's status is no longer in activeStatuses, so LatestSyncCell stops rendering the elapsed timer for it at all; the second change makes the timer correct for streams that genuinely are running.

Verification

Four cases were added to computeStreamStatus.test.ts: completed-in-running-job → Synced, failed-in-running-job → Incomplete, actually-running → Syncing (unchanged), and not-yet-started-with-a-previous-job-COMPLETE → Queued (unchanged).

jest src/area/connection/utils/computeStreamStatus.test.ts → 33 passed, 33 total (29 pre-existing + 4 new). No pre-existing case changed behaviour.

useStreamsStatuses.test.ts and useUiStreamsStates.test.ts could not be executed here: both fail at import with RangeError: Invalid array length out of area/connectorBuilder/components/constants, because this environment skipped scripts/load-declarative-schema.sh. I confirmed the identical failure occurs on unmodified main, so it is environmental rather than a regression from this change — but those two suites are worth a run in proper CI.