#2696·labelme

feat: manage AI models and background downloads in Settings

Author: wkentaroCreated Sep 16, 2026Updated Sep 17, 2026
Labelsready-for-agenttype:feature

Problem

AI Assist and AI Text Prompt list models as selectable before their files are downloaded. First use then starts a blocking download dialog. Users need to see what is ready, prepare models in Settings, and keep annotating during downloads.

Confirmed behavior

  • Settings → AI Models lists the currently supported models with capabilities, download size, status, license/details links, and relevant use restrictions. Require acknowledgment only where the actual terms require it.

  • Both annotation-panel model pickers contain downloaded models plus Manage models…. An empty picker offers Download a model…. Completing a download makes the model selectable, without selecting it or running inference.

  • Both AI Assist and AI Text Prompt remember model selections across restarts. Implement AI Text Prompt persistence as the first small PR. When remembered files are missing, show the unavailable state and offer restoration or another choice, without silently downloading or switching models.

  • Download one model at a time with a visible queue. Downloads continue while Settings is closed and while users annotate manually or use another downloaded model. Users can cancel the current download or remove queued items.

  • After failure, retain Failed and Retry for that model and continue the queue. Retry appends the model to the end.

  • On quit, stop safely and discard the queue. On relaunch, downloads start only on explicit user request. Reuse completed files, restarting an unfinished file. Byte-level resume is deferred.

  • Delete requires confirmation, releases any loaded session, and clears affected selections. A shared model such as SAM3 updates both panels. Annotation files are untouched.

  • A completed download includes shared dependencies required by the model’s supported workflows, avoiding a hidden additional first-use download. Recognize and check existing local/offline files. Downloaded describes verified local files, not a promise of hardware compatibility.

  • Model management remains accessible with command-line overrides while existing restrictions on editing Settings remain enforced.

Scope and implementation order

  1. Persist AI Text Prompt model choice in a separate small PR, using the existing configuration/apply path.

  2. Add model management, background queue, accurate picker states, and model details. Reuse the existing Settings surface and download dependencies; introduce no second cache or alternate model catalog.

Cache-directory customization, new model families, downloads after quitting, automatic queue restart, and byte-level resume are outside this work. Related but distinct: #1428 (custom cache location), #2353 (large-file progress counts), and #2537 (custom ONNX models).

Acceptance checks

Verify selection persistence and Settings synchronization; empty/missing/downloaded model states; no automatic inference or model switching after download; Settings-close and annotation behavior during transfer; FIFO, failure, retry, and cancellation; clean shutdown; deletion of selected/shared models; preservation of annotations; offline readiness including shared NMS; existing-cache validation; CLI override access; and progress for files larger than 2 GiB. Use small local HTTP test artifacts for transfer lifecycle checks, with no multi-GB network requirement in CI.

Keep application Settings persistence separate from derived download/cache state. Preserve Prompt Compatibility checks before inference. Ship user-facing towncrier fragments and prepare PRs for human review; merging is handled by the maintainer.

Local QA — 2026-09-17

Tested branch feat/ai-model-manager at 561276d9 plus local, uncommitted QA fixes using Computer-Use on macOS, with isolated settings, output files, and model cache.

Verified in the real application: model download and completion; queueing and queued cancellation; active cancellation; closing Settings during transfer; real EfficientSam inference and saved annotations; model selection and the Manage models entry; selected-model deletion with annotation preservation; cleared selection after restart; failure/Retry and continued queue processing against an intentionally unavailable local endpoint; full diagnostic details; Settings search; keyboard navigation; and light/dark presentation. A manual rectangle was also saved after starting a real background download, and the transfer completed successfully.

Fixed locally:

  • Cleared AI selections displayed the first model in Settings instead of an empty selection. Both model Settings now support (none).
  • Searching for downloads or model names did not find AI Models. Search now exposes the model-management section.
  • Keyboard navigation could focus an offscreen setting. Section navigation now enters that section and focused controls scroll into view.
  • Long download errors made the page scroll horizontally. Failed rows retain Retry and expose the complete diagnostic in a Details dialog.
  • Download buttons had an ambiguous, static accessible name. Accessible names now reflect Download, Cancel, or Retry.
  • Model links were not keyboard-accessible, the license itself was not linked, and dark-theme links had poor contrast. Links now support keyboard access, include the license URL, and follow the theme text color while retaining underlines.
  • Transfer startup showed a blank filename and 0 B / ?; it now shows Downloading… until progress identifies the file.

Validation: 200 targeted tests passed, including new regressions for cleared selections, model discovery/focus, failure layout/details/Retry, and rendered link colors after theme changes. Ruff, formatting, Gruff, type checks, all 20 translation catalogs, and whitespace checks passed. Changes remain uncommitted for review.

Remaining requirement: pre-download byte sizes are not shown. The installed osam 0.8.0 model metadata supplies license/source information but no download-size field. This needs authoritative size metadata or a separately designed asynchronous size lookup; this QA pass does not substitute guesses or duplicate the model catalog.

Limits: live desktop coverage was macOS and real inference used EfficientSam. Shared SAM3 selection/deletion and large-byte-count behavior were covered by automated local-artifact tests, not multi-GB downloads. Computer-Use did not dismiss the native deletion confirmation with Escape; mouse Cancel and the safe default action worked. Physical-keyboard behavior of that native dialog remains unverified.