immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

New models search only works when lowercased #4100

New Issue
Closed
opened 2026-02-05 09:43:40 +03:00 by OVERLORD · 5 comments
OVERLORD commented 2026-02-05 09:43:40 +03:00
Owner
Copy Link

Originally created by @benwoo1110 on GitHub (Aug 17, 2024).

The bug

I switched from the default ViT-B-32__openai model to the new ViT-B-16-SigLIP__webli and re-ran all the smart search. I noticed the results when search string has uppercase letters will return completely wrong results, as can be seen in the screenshot below:

6197393944636145716_121

6197393944636145715_121

The OS that Immich Server is running on

Raspberry Pi OS Lite (Bookworm)

Version of Immich Server

v1.112.1

Version of Immich Mobile App

v1.112.1

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    environment:
      DB_HOSTNAME: ${DB_HOSTNAME}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_USERNAME: ${DB_USERNAME}
      DB_DATABASE_NAME: ${DB_DATABASE_NAME}
      REDIS_HOSTNAME: ${REDIS_HOSTNAME}
    volumes:
      - /srv/dev-disk-by-uuid-460edec2-7136-4c17-b8c5-811412cb11ae/immich:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /srv/dev-disk-by-uuid-460edec2-7136-4c17-b8c5-811412cb11ae:/usr/src/app/external
    ports:
      - 2283:3001
    dns:
      - 192.168.1.23
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false
    deploy:
      resources:
        limits:
          cpus: '3.00'

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:release
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    environment:
      DB_HOSTNAME: ${DB_HOSTNAME}
      DB_PASSWORD: ${DB_PASSWORD}
      DB_USERNAME: ${DB_USERNAME}
      DB_DATABASE_NAME: ${DB_DATABASE_NAME}
      REDIS_HOSTNAME: ${REDIS_HOSTNAME}
    volumes:
      - /opt/immich/model-cache:/cache
    dns:
      - 192.168.1.23
    restart: always
    healthcheck:
      disable: false
    deploy:
      resources:
        limits:
          cpus: '1.50'

  redis:
    container_name: immich_redis
    image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5
    dns:
      - 192.168.1.23
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - /opt/immich/pgdata:/var/lib/postgresql/data
    dns:
      - 192.168.1.23
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 10m
      start_interval: 1m
      start_period: 10m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=1GB", "-c", "shared_buffers=256MB", "-c", "wal_compression=off"]
    restart: always

Your .env content

DB_PASSWORD=<REDACTED>
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Change the smart search model used from `ViT-B-32__openai` to `ViT-B-16-SigLIP__webli` in the web UI settings page.
2. Re-run all for smart search.
3. Tested search with various search strings with/without capitalisation.

Relevant log output

No response

Additional information

No response

Originally created by @benwoo1110 on GitHub (Aug 17, 2024). ### The bug I switched from the default `ViT-B-32__openai` model to the new `ViT-B-16-SigLIP__webli` and re-ran all the smart search. I noticed the results when search string has uppercase letters will return completely wrong results, as can be seen in the screenshot below: ![6197393944636145716_121](https://github.com/user-attachments/assets/dec40607-1c50-451d-9339-5cbda960b9d2) ![6197393944636145715_121](https://github.com/user-attachments/assets/1e681390-7ab6-432a-9c75-1b2ac350c4ba) ### The OS that Immich Server is running on Raspberry Pi OS Lite (Bookworm) ### Version of Immich Server v1.112.1 ### Version of Immich Mobile App v1.112.1 ### Platform with the issue - [ ] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:release environment: DB_HOSTNAME: ${DB_HOSTNAME} DB_PASSWORD: ${DB_PASSWORD} DB_USERNAME: ${DB_USERNAME} DB_DATABASE_NAME: ${DB_DATABASE_NAME} REDIS_HOSTNAME: ${REDIS_HOSTNAME} volumes: - /srv/dev-disk-by-uuid-460edec2-7136-4c17-b8c5-811412cb11ae/immich:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /srv/dev-disk-by-uuid-460edec2-7136-4c17-b8c5-811412cb11ae:/usr/src/app/external ports: - 2283:3001 dns: - 192.168.1.23 depends_on: - redis - database restart: always healthcheck: disable: false deploy: resources: limits: cpus: '3.00' immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:release # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable environment: DB_HOSTNAME: ${DB_HOSTNAME} DB_PASSWORD: ${DB_PASSWORD} DB_USERNAME: ${DB_USERNAME} DB_DATABASE_NAME: ${DB_DATABASE_NAME} REDIS_HOSTNAME: ${REDIS_HOSTNAME} volumes: - /opt/immich/model-cache:/cache dns: - 192.168.1.23 restart: always healthcheck: disable: false deploy: resources: limits: cpus: '1.50' redis: container_name: immich_redis image: registry.hub.docker.com/library/redis:6.2-alpine@sha256:51d6c56749a4243096327e3fb964a48ed92254357108449cb6e23999c37773c5 dns: - 192.168.1.23 healthcheck: test: redis-cli ping || exit 1 restart: always database: container_name: immich_postgres image: registry.hub.docker.com/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - /opt/immich/pgdata:/var/lib/postgresql/data dns: - 192.168.1.23 healthcheck: test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 10m start_interval: 1m start_period: 10m command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=1GB", "-c", "shared_buffers=256MB", "-c", "wal_compression=off"] restart: always ``` ### Your .env content ```Shell DB_PASSWORD=<REDACTED> DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps ```bash 1. Change the smart search model used from `ViT-B-32__openai` to `ViT-B-16-SigLIP__webli` in the web UI settings page. 2. Re-run all for smart search. 3. Tested search with various search strings with/without capitalisation. ``` ### Relevant log output _No response_ ### Additional information _No response_
OVERLORD added the 🧠machine-learning label 2026-02-05 09:43:40 +03:00
OVERLORD commented 2026-02-05 09:43:43 +03:00
Author
Owner
Copy Link

@JordyEGNL commented on GitHub (Aug 17, 2024):

Can also confirm that this is an issue on the web version.

afbeelding

@JordyEGNL commented on GitHub (Aug 17, 2024): Can also confirm that this is an issue on the web version. ![afbeelding](https://github.com/user-attachments/assets/be04f9ae-d8c4-4044-960c-e17dee644731)
OVERLORD commented 2026-02-05 09:43:44 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Aug 17, 2024):

interesting! @mertalev I assume the fix for this is to make all search queries as lower cases?

@alextran1502 commented on GitHub (Aug 17, 2024): interesting! @mertalev I assume the fix for this is to make all search queries as lower cases?
OVERLORD commented 2026-02-05 09:43:45 +03:00
Author
Owner
Copy Link

@mertalev commented on GitHub (Aug 17, 2024):

No, this should be handled in the tokenizer used in machine learning. Different models want different inputs.

@mertalev commented on GitHub (Aug 17, 2024): No, this should be handled in the tokenizer used in machine learning. Different models want different inputs.
OVERLORD commented 2026-02-05 09:43:46 +03:00
Author
Owner
Copy Link

@benwoo1110 commented on GitHub (Aug 18, 2024):

I guess Immich can do some form of input sanitisation/validation based on the ml model used. E.g. If we know this mode only support lowercased characters we can just lowercase it before passing to the tokeniser. If it's a completely unsupported character by the model e.g. emoji or chinese, Immich should throw an error instead of letting the search return useless results.

This should be possible if the models used have a documentation of the supported characters as their input.

@benwoo1110 commented on GitHub (Aug 18, 2024): I guess Immich can do some form of input sanitisation/validation based on the ml model used. E.g. If we know this mode only support lowercased characters we can just lowercase it before passing to the tokeniser. If it's a completely unsupported character by the model e.g. emoji or chinese, Immich should throw an error instead of letting the search return useless results. This should be possible if the models used have a documentation of the supported characters as their input.
OVERLORD commented 2026-02-05 09:43:47 +03:00
Author
Owner
Copy Link

@mertalev commented on GitHub (Aug 18, 2024):

#11881 should fix the casing issue. Checking unsupported characters could also be nice. I'm actually not sure what the tokenizer does when it encounters an unsupported character.

@mertalev commented on GitHub (Aug 18, 2024): #11881 should fix the casing issue. Checking unsupported characters could also be nice. I'm actually not sure what the tokenizer does when it encounters an unsupported character.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label 🧠machine-learning
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4100
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?