#31514 [Bug] Facial recognition data not correctly shared (some detected faces missing for shared user)

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

The bug

I was looking forward to the facial recognition data sharing feature and after it landed I immediately tested it out on my main server. On that server I have the following users:

Family: partner shares all photos to all family members, most photos are stored in an external library (contains old photos from before immich became relevant. It currently contains ~43k photos)
Personal accounts of family members. This is where all family members upload their own photos via the immich app. All family members have access to the main family photos via partner sharing. Family members don't partner share their own photos with the family account.

After the update landed, I placed all family members + the family account in a new sharing group and reset the facial recognition data. My expectation was that I would be able to see all detected faces across all family photos and my personal photos. However, after the reprocessing of the images was complete, I saw that for many photos that have faces in them (that belong to the family account) I could not see those detected faces in my personal account. Also curious is that when I navigate to "explore -> people" I can see 366 detected faces on my personal account and 2663 faces for the family account (but maybe this is related to #31434).

The OS that Immich Server is running on

NixOS

Version of Immich Server

v3.2.0

Version of Immich Mobile App

v3.2.0

Platform with the issue

Server
Web
Mobile

Device make and model

No response

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/data
      - /etc/localtime:/etc/localtime:ro
      - /mnt/truenas_multimedia/imagedata/Digital/OwnPics:/mnt/nas/OwnPics:ro
      - /mnt/user1_ownpics:/mnt/nas/OwnPics_user1:ro
    env_file:
      - .env
    ports:
      - '127.0.0.1:22803:2283'
    depends_on:
      - redis
      - database
    restart: unless-stopped
    healthcheck:
      disable: false
    user: "1000:1000"

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: unless-stopped
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:eaba718fecd1196d88533de7ba49bf903ad33664a92debb24660a922ecd9cac8
    healthcheck:
      test: redis-cli ping || exit 1
    restart: unless-stopped

  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.3.0-pgvectors0.3.0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
      DB_STORAGE_TYPE: 'SSD'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    restart: unless-stopped

volumes:
  model-cache:
    #prometheus-data:
    #grafana-data:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/truenas_immich
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=<secret>

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

# Enable grafana metrics
IMMICH_TELEMETRY_INCLUDE=all

# Helmet Configuration
IMMICH_HELMET_FILE=true

Reproduction steps

Create and setup new immich instance with default docker-compose.yaml and .env contents
Modify the docker-compose file to contain another bind mount to an external directory
Create two new users (test1 (plays role of family user) and test2 (plays role of family member)). Make test1 partner share with test2. Do not yet create the sharing group between test1 and test2.
Create an external library and assign it to test1. Needs to point to the folder configured in the docker-compose file
Add photos to the external library and rescan it. I added 5 pictures: On all 5 pictures was the same person (a) and on 3 of those pictures where also the same 3 other people (b, c, d).
Upload photos using the web interface (logged in as test1): 4 total, all 4 contain (a), 1 contains (b, c)
(Still logged in as test1) view the people page, you will see detected faces
Create a facial recognition sharing group between test1 and test2 (login as test2 and approve it, then login as test1 again). Then reset the facial recognition data.
Wait a short moment until immich has processed the images, then take a look at explore -> people (still logged in as test1), detected faces will be visible
Logout as test1 and login as test2.
Enable "Show in timeline" for the photos from test1

Now there are two places where the faces should be visible but they are not:

People page in explore -> people: Does not contain any recognized faces (this might be linked to #31434).
People information when inspecting a shared image in the timeline that has people on it: no faces shown (also not hidden)

Relevant log output

Additional information

No response