Real-time track generation leaves valid driving points unassigned between tracks

Environment

• Observed on Dawarich 1.14.1, self-hosted with Docker Compose on Debian 13, x86-64.
• Rails environment: development
• Client details are not relevant because the failure occurs in server-side track generation after the points have been stored.
• All three relevant settings match the Dawarich 1.14.1 defaults: meters_between_routes = 500, minutes_between_routes = 30, gps_filtering_enabled = true.
• Source review of 1.14.5 shows the same relevant selection, splitting, and catch-up rules. I have not replayed this case on 1.14.5.

Observed behavior

A drive is present in the raw Routes layer, but a continuous section is missing from the server-generated Tracks layer. Read-only database checks found that 33 consecutive points were stored successfully but still had track_id IS NULL the following day. Their anomaly values were NULL, so the generation query's anomaly IS NOT TRUE condition includes them.

The selected points were recorded approximately every 54–55 seconds. Consecutive distances inside the missing section were 581–1958 metres. Upload delay was not the cause: the maximum positive arrival delay in the attached excerpt was four seconds. Small negative differences were also observed, so these are approximate comparisons of device and server clocks.

The preceding track ends at relative time +00:54. The next track starts at +34:49, leaving a 33-minute-55-second gap between generated track boundaries. Its first two points are only six seconds and 192 metres apart. Those two points form a track, whereas the preceding 33 points remain unassigned.

The attachment contains 37 points: the last two points of track A, the 33 unassigned points, and the first two points of track B. The rest of the journey is omitted. The original track IDs have been replaced by A/B aliases. All absolute timestamps are shifted together to a randomly selected period; relative intervals and the route geometry are preserved.

Expected behavior

Valid points from a continuous drive should not remain permanently excluded from generated tracks simply because normal movement between recorded samples exceeds the distance threshold. If such points are intentionally split during real-time processing, a later catch-up pass should reconsider unassigned points between existing tracks.

Suspected mechanism, supported by database observations and source review

1. Tracks::IncrementalGenerator selects only untracked points from the last six hours. The already assigned endpoint of track A is therefore unavailable as a seed for the next point, even though that next point is only 104 metres away.
2. Track.segment_points_in_sql starts a new segment when the distance between consecutive selected points exceeds the configured threshold. HAVING count(*) >= 2 then discards each singleton segment. All 33 intermediate points meet this pattern.
3. A new track starts exactly at the later close pair (six seconds / 192 metres). Its start is more than 30 minutes after track A ends, so the incremental merge condition does not join A and B.
4. Orphan reabsorption only considers points whose timestamps are already inside a track's start/end window. The missing points lie between those windows.
5. Tracks::DailyGenerationJob starts after the maximum existing track end. Once a later track exists, this older interior gap falls before its starting point. The real-time lookback also eventually expires.

This explains the observed point memberships and exact restart boundary without requiring missing uploads or GPS anomalies. No job execution trace or controlled replay has yet been collected; the sequence above is a source-based diagnosis, not a claim that every historical job invocation was observed.

Suggested reproduction (not yet executed as a controlled replay)

1. Use one tracker and the settings above. Use the attached relative timestamps, translated near the current test clock; importing the shifted historical timestamps would exercise a different path.
2. Start with the first two points already assigned to an existing track A.
3. Feed the following points in timestamp order through the normal real-time ingestion path, preserving the sampling intervals and allowing the debounced generator to run.
4. The first unassigned point is close to the previous assigned endpoint, but the subsequent unassigned-to-unassigned distances all exceed 500 metres. Inspect track_id while generation runs.
5. Feed the final close pair after more than 30 minutes from A's end. Inspect whether B forms while the 33 intermediate points remain unassigned.
6. Run the ordinary periodic catch-up after B exists and check whether the interior gap is reconsidered.

A bulk GPX import or full track recalculation is not equivalent to this reproduction: the chunk processor uses a different, time-based segmentation path.

Relevant code in 1.14.5

• Untracked-only selection, distance splitting, and minimum segment size
• Incremental lookback and preceding-track merge
• Orphan reabsorption restricted to an existing track window
• Periodic processing starts after the latest track

Related reports

• #2188 describes missing connections at driving speeds, but was closed after reports of improvement in 1.0.2–1.0.4. This report concerns persisted unassigned points in 1.14.1.
• #2463 / #2756 concern overlapping tracks and late-arriving points inside an existing track window. Here arrivals were prompt and the points lie between two track windows.

Attachments

• points.json: 37 actual coordinate samples, anonymized timestamps/device/track identifiers, observed membership, anomaly flags, approximate arrival delays, and rounded distance measurements. All points belong to one anonymized device.
• route-excerpt.gpx: the same clipped coordinates and shifted times for inspection. Importing this GPX is not a real-time reproduction.
• route-comparison.png: the clipped raw route versus observed generated-track membership.
• sampling-and-membership.png: sampling distances, the 500 m threshold, and observed membership.

Both images are derived visualizations, not application screenshots. Original screenshots and full raw logs are excluded. The real road geometry is intentionally retained with the data owner's permission; the full trip, real recording date, original identifiers, and device metadata are not included.

points.json track-gap-data.zip