#5773·photoprism

AI: Make single-word label normalization optional

Author: lastzeroCreated Aug 8, 2026Updated Sep 16, 2026
Labelstestedailabels

As an operator who generates labels with a vision model, I want to choose whether label names are collapsed to a single word, so that compound subjects like "ferris wheel" stay searchable instead of being stored as "Ferris".

Problem

Labels returned by the Ollama and OpenAI engines pass through normalizeLabelResult in internal/ai/vision/label_normalizer.go. When a name contains more than one word, the normalizer splits it into tokens, returns the first token that resolves against the canonical label vocabulary, and otherwise falls back to the first token. For a compound noun the surviving word is usually not the subject, and which word survives is not predictable:

Model returns PhotoPrism stores
ferris wheel Ferris
amusement park Park
roller coaster Roller
lifeguard tower Tower
traffic light Traffic
solar panel Solar
trash can Trash
ski-lift Ski

Ferris, Roller, and Solar are not terms anyone will search for. Models that instead satisfy the single-word instruction by concatenating (ferriswheel, amusementpark) produce a token that matches nothing either. Neither outcome can be repaired afterwards, because the original phrase is discarded before the label is stored.

The behavior is currently unconditional, which conflicts with the configuration surface we already expose. System can be set to a prompt that does not require single-word nouns, but any descriptive phrase the model then returns is collapsed anyway — so the option does not have the effect its wording implies.

Nothing downstream requires a single word. entity.NewLabel already accepts multi-word names and slugs them as expected (Ferris Wheelferris-wheel), several fixtures use them, and the canonical vocabulary itself contains hundreds of multi-word entries such as sea lion and theme park. Only the out-of-vocabulary fallback is lossy.

Proposal

Make single-word collapsing a choice rather than a fixed rule, defaulting per model so that each engine gets the behavior its output actually calls for.

The setting belongs on the model, as a sibling of System and Format, rather than under OptionsOptions carries sampling parameters that are passed through to the service, and a PhotoPrism-side post-processing rule must not travel to Ollama or OpenAI:

yaml
Models:
  - Type: labels
    Name: <model>
    Engine: ollama
    System: |
      You are a PhotoPrism vision model. Output concise JSON that matches the schema.
    Normalize: phrase   # single-word | phrase | false
Value Behavior
(unset) Resolves to the default for the model — see below
single-word Current behavior; collapse to one token
phrase Keep multi-word names, normalizing casing and separators, and match the whole phrase against the canonical vocabulary before falling back to the phrase itself
false Store the name the model returned, with whitespace and case cleanup only, and no vocabulary name mapping

off, none, no, and disabled are accepted as aliases of false, matching how Run and Service.Think already accept several spellings.

The default depends on the model, because the failure modes do. A cloud model defaults to phrase; everything else keeps single-word. A model counts as cloud-hosted when its version tag is cloud, when the engine is openai and the identifier belongs to one of OpenAI's own families (gpt, chatgpt, o1, o3, o4, matched as a whole family so a local model merely starting with the same letters is not misread), or when its resolved endpoint is the Ollama Cloud host — all read from the model rather than from a process-wide default, so a configuration that reaches both a local instance and a cloud service classifies each entry on its own.

The distinction is measured rather than assumed. Across the benchmark image set, every multi-word label the cloud models returned was a real compound — ferris wheel, amusement park, roller coaster, ski-lift — at a rate of 0-2%, and the default mangles all four. Models small enough for an 8 GB GPU return 3-19% multi-word names and mix real compounds with filler such as city_name, text_on_sign, and photo list, which collapsing keeps in check.

The gap is widest outside English, where a compound subject is normally two words. A name written in a non-Latin script is therefore never collapsed, whatever the mode says: the canonical vocabulary is English, so tokenizing حمار وحشي (zebra) has nothing to resolve against and could only reduce it to حمار (donkey), and גלגל ענק (ferris wheel) to גלגל (wheel).

The condition is the script rather than the language, because the two behave differently. A Latin-script name can still resolve token by token and the surviving word is often the right one — Spanish noria gigante keeps the head noun Noria — while a non-Latin name can only ever be truncated. A name mixing scripts keeps the configured behavior, so شاطئ beach still resolves to Beach through the vocabulary.

false disables name rewriting, not quality filtering: a label whose name matches a vocabulary rule still inherits that rule's confidence threshold, priority, and categories, so labels that are dropped today (for example background, threshold 1.0) continue to be dropped. That keeps the mode purely about naming.

Technical Considerations

  • normalizeLabelResult is shared by the Ollama and OpenAI engines, so the setting must reach both. The TensorFlow path is unaffected — it never calls the normalizer.
  • The default must be resolvable per engine and per model, not as a single package-wide constant. ONNX-based label models (#5769) classify against a vocabulary that is multi-word by construction — 474 of the 1000 entries in the current class list contain a space — so an engine that emits great white shark needs phrase as its own default rather than an explicit override on every model entry. The per-model check is what lets a cloud model keep the cloud default when it is reached through a local instance that proxies the request, and conversely what keeps a local model on single-word when OLLAMA_BASE_URL points at the cloud.
  • Confidence and topicality thresholds, category merging, and priority assignment must keep working unchanged in every mode. The code path is identical; only which vocabulary rule is found differs, and therefore which per-label threshold applies.
  • Multi-word labels flowing into search and album matching overlap with #5714. Exact-phrase lookup already resolves today (label:"sea lion" matches a Sea Lion label by name and by slug). What still misses is the inflected or partial phrase — the slug helper splits on the separator and never emits a whole-phrase slug — and the title, caption, subject, and keyword updaters, which match single tokens only. Those belong to #5714.
  • Whether a phrase-mode label should also register its head noun as a separate label is worth considering, but it is a larger change and does not block this one.

Implementation

Shipped on develop in Vision: Add Normalize option for label names #5773 and Vision: Keep label names written in another script whole #5773. Model.IsCloud() decides the default, pairing with ollama.IsCloudUrl() and openai.IsCloudModel(), and the mode is carried to the response parsers on the request object without being serialized. photoprism vision ls reports the effective mode for the models it applies to.

Verified against a live service on the same picture and prompt: minimax-m3:cloud stores Ferris Wheel, Amusement Park, and Roller Coaster, where the same model pinned to single-word stores Ferris, Park, Roller, and Trash.

The published user documentation describes the option. The release notes are the one item still outstanding, since the change is only visible to installations using a cloud model and no release has been tagged since it landed.

Acceptance Criteria

  • Label normalization MUST offer a way to keep multi-word names instead of collapsing them to a single token.
  • The default for self-hosted Ollama MUST remain the current single-word behavior.
  • The default for cloud models MUST be phrase, since their multi-word output is measurably deliberate rather than accidental.
  • Because installations using a cloud model will see compound label names after upgrading, the release notes MUST call the change out.
  • Whether a model is cloud-hosted MUST be decided per model, not per engine, so that a configuration mixing a local instance with a cloud service gets the right default for each entry.
  • The setting MUST live on the model rather than under Options, and MUST NOT be sent to the vision service.
  • An unset value MUST resolve from the model, then its engine, so a future engine whose vocabulary is multi-word can ship the right behavior without a per-model override.
  • Model and configuration inspection (photoprism vision ls) SHOULD report the effective mode for models it applies to.
  • The setting MUST apply to both the Ollama and the OpenAI engines.
  • An unrecognized value MUST be reported once when the configuration is loaded, and MUST fall back to the default instead of failing the load.
  • Confidence, topicality, category, and priority handling MUST behave identically in every mode, including the mode that keeps the model's own name.
  • Phrase mode MUST match a multi-word name against the canonical label vocabulary as a whole, including its singular form, before falling back to storing the phrase verbatim.
  • A label name written in a script the canonical vocabulary cannot represent MUST be kept whole in every mode, since collapsing it can only change the subject.
  • Tests MUST cover each mode with at least one compound name (for example ferris wheel), one already-single-word name, one name that resolves through the vocabulary, and one name whose rule threshold drops it.
  • The new setting MUST be documented in the vision configuration reference and in the Ollama engine documentation.
  • The new setting SHOULD be described in the published user documentation, so operators find it without reading the package docs.
  • Search behavior for stored multi-word labels SHOULD be consistent with #5714.
  • A phrase-mode label MAY additionally register its head noun as a separate label; this is out of scope here.