immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-03-01 11:20:12 +03:00
Code Issues 783 Packages Projects Releases 12 Wiki Activity

Duplicates from External Library are ending up in Timeline and "Duplicates" View, but it is actually the same exact file #5331

New Issue
Closed
opened 2026-02-05 11:21:12 +03:00 by OVERLORD · 6 comments
OVERLORD commented 2026-02-05 11:21:12 +03:00
Owner
Copy Link

Originally created by @jimmywan on GitHub (Feb 6, 2025).

The bug

I have two external libraries, the second of which is 99% videos. My timeline and "Review Duplicates" tool are showing two entries for the same file. If I navigate to the path by clicking on the filename, it brings me to the Explorer View and all files are show twice.

The external library is pointing to a filesystem housed on a Synology NAS that is mounted as read-only in my container. I have not noticed any files from my first external library showing up as duplicates.

The OS that Immich Server is running on

DSM 7.2.2-72806 Update 2, running portainer 2.21.2

Version of Immich Server

1.125.7

Version of Immich Mobile App

1.125.5 build.181

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    user: 1026:101
    security_opt:
      - no-new-privileges:true
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    devices:
      - /dev/dri:/dev/dri
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /volume1/photo/OriginalPics:/usr/src/app/external/OriginalPics:ro
      - /volume1/photo/RawPics:/usr/src/app/external/RawPics:ro
      - /volume1/photo/SharedPics:/usr/src/app/external/SharedPics:ro
      - /volume1/video/Dance/2024-2025 CCDC Prep Purple:/usr/src/app/external/2024-2025 CCDC Prep Purple:ro

    env_file:
      - stack.env
    ports:
      - 8212:2283
    dns: 192.168.10.2
    # networks:
    #   immich_network:
    #     ipv4_address: 172.16.0.1
    depends_on:
      redis:
          condition: service_healthy
      database:
          condition: service_started
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    # image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
    # image: ghcr.io/immich-app/immich-machine-learning:main-openvino
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    user: 1026:101
    security_opt:
      - no-new-privileges:true

    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    # device_cgroup_rules:
    #   - 'c 189:* rmw'
    # devices:
    #   - /dev/dri:/dev/dri
    volumes:
      # - /dev/bus/usb:/dev/bus/usb
      - ${MODEL_CACHE_LOCATION}:/cache:rw
      - ${MODEL_DOTCACHE_LOCATION}:/.cache:rw
      - ${MODEL_DOTCONFIG_LOCATION}:/.config:rw
    env_file:
      - stack.env
    dns: 192.168.10.2
    # networks:
    #   immich_network:
    #     ipv4_address: 172.16.0.2
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:905c4ee67b8e0aa955331960d2aa745781e6bd89afc44a8584bfd13bc890f0ae
    user: 1026:101
    security_opt:
      - no-new-privileges:true
    volumes:
      - ${REDIS_LOCATION}:/data:rw
    # networks:
    #   immich_network:
    #     ipv4_address: 172.16.0.3
    healthcheck:
      test: redis-cli ping || exit 1
    restart: on-failure:5

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0

    security_opt:
      - no-new-privileges:true
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    # networks:
    #   immich_network:
    #     ipv4_address: 172.16.0.4
    healthcheck:
      test: >-
        pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1;
        Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align
        --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')";
        echo "checksum failure count is $$Chksum";
        [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      # start_interval: 30s
      start_period: 5m
    command: >-
      postgres
      -c shared_preload_libraries=vectors.so
      -c 'search_path="$$user", public, vectors'
      -c logging_collector=on
      -c max_wal_size=2GB
      -c shared_buffers=512MB
      -c wal_compression=on
    restart: always

networks:
  pihole_bridge:
    external: true
    name: dns_bridge

Your .env content

UPLOAD_LOCATION=/volume1/docker/immich/upload
DB_DATA_LOCATION=/volume1/docker/immich/db
MODEL_CACHE_LOCATION=/volume1/docker/immich/cache
TZ=America/Chicago
IMMICH_VERSION=v1.125.7
DB_USERNAME=@@@@@@
DB_DATABASE_NAME=@@@@@@
DB_PASSWORD=@@@@@@
REDIS_LOCATION=/volume1/docker/immich/redis
MODEL_DOTCACHE_LOCATION=/volume1/docker/immich/.cache
MODEL_DOTCONFIG_LOCATION=/volume1/docker/immich/.config

Reproduction steps

Steps A:

  1. Browse to Immich Timeline, scroll to date in question.
  2. See duplicate thumbnails pointing at the same file.
  3. Click on File Location (i) for both thumbnails and note path is identical.

Steps B:

  1. Utilities -> Review Duplicates
  2. See identified "duplicate" that points to the same exact path twice. It's not two copies in different folders, I double checked.

Relevant log output

at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 8  - 02/05/2025, 12:00:00 AM   ERROR [Microservices:JobService] Object:

{

  "id": "224c3970-e49d-43d3-8f5c-7738fb7036df"

}

[Nest] 8  - 02/05/2025, 12:00:00 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string

[Nest] 8  - 02/05/2025, 12:00:00 AM   ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:54:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:164:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:114:25)

    at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)

    at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:130:13)

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 8  - 02/05/2025, 12:00:00 AM   ERROR [Microservices:JobService] Object:

{

  "id": "2aaab635-7333-4a25-8845-d41fba204860"

}

[Nest] 8  - 02/05/2025, 12:00:02 AM   ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string

[Nest] 8  - 02/05/2025, 12:00:02 AM   ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string

    at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17)

    at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:54:68)

    at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:164:63)

    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)

    at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:114:25)

    at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28)

    at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:130:13)

    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)

    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

[Nest] 8  - 02/05/2025, 12:00:02 AM   ERROR [Microservices:JobService] Object:

{

  "id": "7ecd75ba-ec05-4837-bcba-780faa28d75b"

}

[Nest] 8  - 02/05/2025, 2:00:00 AM     LOG [Microservices:BackupService] Database Backup Starting. Database Version: 14

[Nest] 8  - 02/05/2025, 2:01:34 AM     LOG [Microservices:BackupService] Database Backup Success

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM     LOG [Microservices:LibraryService] Importing new library asset: /usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg

[Nest] 8  - 02/05/2025, 12:07:21 PM     LOG [Microservices:LibraryService] Importing new library asset: /usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_M.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_S.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_M.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_S.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:21 PM   ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg'

[Nest] 8  - 02/05/2025, 12:07:23 PM    WARN [Microservices:MetadataService] Asset 5fb70fcd-3009-4659-b577-1236e6f5cf55 has no time zone information

[Nest] 8  - 02/05/2025, 12:07:23 PM    WARN [Microservices:MetadataService] Asset 0f09c4fb-c22f-478f-978b-3f9b4ee9fc22 has no time zone information

Additional information

NOTE: I only noticed this problem AFTER upgrading from 1.125.1 to 1.125.7 and enabled File Watching.
Note that my external library is mounted read-only and does include the exclusion string **/@eaDir/**, so I'm not sure why those paths are referenced in the logs I pasted above.

NOTE: The timeline on my android app does NOT show duplicates. This problem only occurs on the web view from my desktop browser.

Version History:

  • Installed 1.125.7 on Feb 3, 2025
  • Installed 1.125.1 on Jan 23, 2025
  • Installed 1.123.0 on Dec 19, 2024
  • Installed 1.122.3 on Dec 14, 2024
  • Installed 1.122.2 on Dec 8, 2024
Originally created by @jimmywan on GitHub (Feb 6, 2025). ### The bug I have two external libraries, the second of which is 99% videos. My timeline and "Review Duplicates" tool are showing two entries for the same file. If I navigate to the path by clicking on the filename, it brings me to the Explorer View and all files are show twice. The external library is pointing to a filesystem housed on a Synology NAS that is mounted as read-only in my container. I have not noticed any files from my first external library showing up as duplicates. ### The OS that Immich Server is running on DSM 7.2.2-72806 Update 2, running portainer 2.21.2 ### Version of Immich Server 1.125.7 ### Version of Immich Mobile App 1.125.5 build.181 ### Platform with the issue - [x] Server - [x] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} user: 1026:101 security_opt: - no-new-privileges:true # extends: # file: hwaccel.transcoding.yml # service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding devices: - /dev/dri:/dev/dri volumes: # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /volume1/photo/OriginalPics:/usr/src/app/external/OriginalPics:ro - /volume1/photo/RawPics:/usr/src/app/external/RawPics:ro - /volume1/photo/SharedPics:/usr/src/app/external/SharedPics:ro - /volume1/video/Dance/2024-2025 CCDC Prep Purple:/usr/src/app/external/2024-2025 CCDC Prep Purple:ro env_file: - stack.env ports: - 8212:2283 dns: 192.168.10.2 # networks: # immich_network: # ipv4_address: 172.16.0.1 depends_on: redis: condition: service_healthy database: condition: service_started restart: always healthcheck: disable: false immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda # image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino # image: ghcr.io/immich-app/immich-machine-learning:main-openvino image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} user: 1026:101 security_opt: - no-new-privileges:true # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable # device_cgroup_rules: # - 'c 189:* rmw' # devices: # - /dev/dri:/dev/dri volumes: # - /dev/bus/usb:/dev/bus/usb - ${MODEL_CACHE_LOCATION}:/cache:rw - ${MODEL_DOTCACHE_LOCATION}:/.cache:rw - ${MODEL_DOTCONFIG_LOCATION}:/.config:rw env_file: - stack.env dns: 192.168.10.2 # networks: # immich_network: # ipv4_address: 172.16.0.2 restart: always healthcheck: disable: false redis: container_name: immich_redis image: docker.io/redis:6.2-alpine@sha256:905c4ee67b8e0aa955331960d2aa745781e6bd89afc44a8584bfd13bc890f0ae user: 1026:101 security_opt: - no-new-privileges:true volumes: - ${REDIS_LOCATION}:/data:rw # networks: # immich_network: # ipv4_address: 172.16.0.3 healthcheck: test: redis-cli ping || exit 1 restart: on-failure:5 database: container_name: immich_postgres image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 security_opt: - no-new-privileges:true environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' volumes: # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file - ${DB_DATA_LOCATION}:/var/lib/postgresql/data # networks: # immich_network: # ipv4_address: 172.16.0.4 healthcheck: test: >- pg_isready --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" || exit 1; Chksum="$$(psql --dbname="$${POSTGRES_DB}" --username="$${POSTGRES_USER}" --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 5m # start_interval: 30s start_period: 5m command: >- postgres -c shared_preload_libraries=vectors.so -c 'search_path="$$user", public, vectors' -c logging_collector=on -c max_wal_size=2GB -c shared_buffers=512MB -c wal_compression=on restart: always networks: pihole_bridge: external: true name: dns_bridge ``` ### Your .env content ```Shell UPLOAD_LOCATION=/volume1/docker/immich/upload DB_DATA_LOCATION=/volume1/docker/immich/db MODEL_CACHE_LOCATION=/volume1/docker/immich/cache TZ=America/Chicago IMMICH_VERSION=v1.125.7 DB_USERNAME=@@@@@@ DB_DATABASE_NAME=@@@@@@ DB_PASSWORD=@@@@@@ REDIS_LOCATION=/volume1/docker/immich/redis MODEL_DOTCACHE_LOCATION=/volume1/docker/immich/.cache MODEL_DOTCONFIG_LOCATION=/volume1/docker/immich/.config ``` ### Reproduction steps Steps A: 1. Browse to Immich Timeline, scroll to date in question. 2. See duplicate thumbnails pointing at the same file. 3. Click on File Location (i) for both thumbnails and note path is identical. Steps B: 1. Utilities -> Review Duplicates 2. See identified "duplicate" that points to the same exact path twice. It's not two copies in different folders, I double checked. ### Relevant log output ```shell at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 8 - 02/05/2025, 12:00:00 AM ERROR [Microservices:JobService] Object: { "id": "224c3970-e49d-43d3-8f5c-7738fb7036df" } [Nest] 8 - 02/05/2025, 12:00:00 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 8 - 02/05/2025, 12:00:00 AM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:54:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:164:63) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:114:25) at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28) at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:130:13) at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 8 - 02/05/2025, 12:00:00 AM ERROR [Microservices:JobService] Object: { "id": "2aaab635-7333-4a25-8845-d41fba204860" } [Nest] 8 - 02/05/2025, 12:00:02 AM ERROR [Microservices:JobService] Unable to run job handler (thumbnailGeneration/generate-thumbnails): Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string [Nest] 8 - 02/05/2025, 12:00:02 AM ERROR [Microservices:JobService] Error: VipsJpeg: Message codes are not supported, error message is in msg_parm.s string at Sharp.toBuffer (/usr/src/app/node_modules/sharp/lib/output.js:163:17) at MediaRepository.decodeImage (/usr/src/app/dist/repositories/media.repository.js:54:68) at MediaService.generateImageThumbnails (/usr/src/app/dist/services/media.service.js:164:63) at process.processTicksAndRejections (node:internal/process/task_queues:105:5) at async MediaService.handleGenerateThumbnails (/usr/src/app/dist/services/media.service.js:114:25) at async JobService.onJobStart (/usr/src/app/dist/services/job.service.js:148:28) at async EventRepository.onEvent (/usr/src/app/dist/repositories/event.repository.js:130:13) at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) [Nest] 8 - 02/05/2025, 12:00:02 AM ERROR [Microservices:JobService] Object: { "id": "7ecd75ba-ec05-4837-bcba-780faa28d75b" } [Nest] 8 - 02/05/2025, 2:00:00 AM LOG [Microservices:BackupService] Database Backup Starting. Database Version: 14 [Nest] 8 - 02/05/2025, 2:01:34 AM LOG [Microservices:BackupService] Database Backup Success [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM LOG [Microservices:LibraryService] Importing new library asset: /usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg [Nest] 8 - 02/05/2025, 12:07:21 PM LOG [Microservices:LibraryService] Importing new library asset: /usr/src/app/external/REDACTEDFOLDERNAME/IMG4235789500694256002.jpg [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_M.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_S.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_M.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg/SYNOFILE_THUMB_S.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library fa949762-60de-411e-a51c-e01b0ebd91c3 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:21 PM ERROR [Microservices:LibraryService] Library watcher for library a747c829-7e16-4618-95bb-73d7a123fb19 encountered error: Error: ENOSPC: System limit for number of file watchers reached, watch '/usr/src/app/external/REDACTEDFOLDERNAME/@eaDir/IMG4235789500694256002.jpg' [Nest] 8 - 02/05/2025, 12:07:23 PM WARN [Microservices:MetadataService] Asset 5fb70fcd-3009-4659-b577-1236e6f5cf55 has no time zone information [Nest] 8 - 02/05/2025, 12:07:23 PM WARN [Microservices:MetadataService] Asset 0f09c4fb-c22f-478f-978b-3f9b4ee9fc22 has no time zone information ``` ### Additional information NOTE: I only noticed this problem AFTER upgrading from 1.125.1 to 1.125.7 and enabled File Watching. Note that my external library is mounted read-only and does include the exclusion string `**/@eaDir/**`, so I'm not sure why those paths are referenced in the logs I pasted above. NOTE: The timeline on my android app does NOT show duplicates. This problem only occurs on the web view from my desktop browser. Version History: - Installed 1.125.7 on Feb 3, 2025 - Installed 1.125.1 on Jan 23, 2025 - Installed 1.123.0 on Dec 19, 2024 - Installed 1.122.3 on Dec 14, 2024 - Installed 1.122.2 on Dec 8, 2024
OVERLORD commented 2026-02-05 11:21:15 +03:00
Author
Owner
Copy Link

@jimmywan commented on GitHub (Feb 6, 2025):

If it helps, it does look like the server has two metadata entries pointing to the same physical path:
http://192.168.1.6:8212/folders/photos/4c7ccb43-f643-49bc-9591-2283d494569d?path=MYREDACTEDPATH
http://192.168.1.6:8212/folders/photos/ab75f16e-e836-480b-95dc-fe9c088ff5c4?path=MYREDACTEDPATH

This is the browser URL when viewing the two assets.

@jimmywan commented on GitHub (Feb 6, 2025): If it helps, it does look like the server has two metadata entries pointing to the same physical path: `http://192.168.1.6:8212/folders/photos/4c7ccb43-f643-49bc-9591-2283d494569d?path=MYREDACTEDPATH` `http://192.168.1.6:8212/folders/photos/ab75f16e-e836-480b-95dc-fe9c088ff5c4?path=MYREDACTEDPATH` This is the browser URL when viewing the two assets.
OVERLORD commented 2026-02-05 11:21:15 +03:00
Author
Owner
Copy Link

@clueo8 commented on GitHub (Feb 24, 2025):

I am noticing this as well, with the duplicates actually being the same file/filename but resides within a different sub-directory of the external storage location.

@clueo8 commented on GitHub (Feb 24, 2025): I am noticing this as well, with the duplicates actually being the same file/filename but resides within a different sub-directory of the external storage location.
OVERLORD commented 2026-02-05 11:21:17 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Feb 24, 2025):

but resides within a different sub-directory of the external storage location.

In that case they're treated as two independent files and everything is working as expected.

cc @etnoy any idea on the original issue, why might there be multiple entries for the same path?

@bo0tzz commented on GitHub (Feb 24, 2025): > but resides within a different sub-directory of the external storage location. In that case they're treated as two independent files and everything is working as expected. cc @etnoy any idea on the original issue, why might there be multiple entries for the same path?
OVERLORD commented 2026-02-05 11:21:17 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Feb 24, 2025):

If it helps, it does look like the server has two metadata entries pointing to the same physical path: http://192.168.1.6:8212/folders/photos/4c7ccb43-f643-49bc-9591-2283d494569d?path=MYREDACTEDPATH http://192.168.1.6:8212/folders/photos/ab75f16e-e836-480b-95dc-fe9c088ff5c4?path=MYREDACTEDPATH

This is the browser URL when viewing the two assets.

Can you post the import paths for all external libraries? I think you've imported the same file multiple times

@etnoy commented on GitHub (Feb 24, 2025): > If it helps, it does look like the server has two metadata entries pointing to the same physical path: `http://192.168.1.6:8212/folders/photos/4c7ccb43-f643-49bc-9591-2283d494569d?path=MYREDACTEDPATH` `http://192.168.1.6:8212/folders/photos/ab75f16e-e836-480b-95dc-fe9c088ff5c4?path=MYREDACTEDPATH` > > This is the browser URL when viewing the two assets. Can you post the import paths for all external libraries? I think you've imported the same file multiple times
OVERLORD commented 2026-02-05 11:21:18 +03:00
Author
Owner
Copy Link

@jimmywan commented on GitHub (Feb 25, 2025):

Oh you're right, how silly of me:
Library 1: /usr/src/app/external
Library 2: /usr/src/app/external/Videos/XXX-YYY-ZZZ

I thought I had put them in parallel dirs, but didn't.
Any suggestions on the best way to clean this up with the least amount of reprocessing? My directory structure actually looks like this:

All under /usr/app/external
/RAWPics
/OriginalPics
/SharedPics
/Videos

Should I just split each into its own external library? I've been meaning to figure out how to reduce the amount of time spent rescanning as many of these folders are fairly static.

@jimmywan commented on GitHub (Feb 25, 2025): Oh you're right, how silly of me: Library 1: /usr/src/app/external Library 2: /usr/src/app/external/Videos/XXX-YYY-ZZZ I thought I had put them in parallel dirs, but didn't. Any suggestions on the best way to clean this up with the least amount of reprocessing? My directory structure actually looks like this: All under /usr/app/external /RAWPics /OriginalPics /SharedPics /Videos Should I just split each into its own external library? I've been meaning to figure out how to reduce the amount of time spent rescanning as many of these folders are fairly static.
OVERLORD commented 2026-02-05 11:21:19 +03:00
Author
Owner
Copy Link

@etnoy commented on GitHub (Feb 25, 2025):

Oh you're right, how silly of me: Library 1: /usr/src/app/external Library 2: /usr/src/app/external/Videos/XXX-YYY-ZZZ

I thought I had put them in parallel dirs, but didn't. Any suggestions on the best way to clean this up with the least amount of reprocessing? My directory structure actually looks like this:

All under /usr/app/external /RAWPics /OriginalPics /SharedPics /Videos

Should I just split each into its own external library? I've been meaning to figure out how to reduce the amount of time spent rescanning as many of these folders are fairly static.

Yes, I would put each in a separate library. My long term plan is to allow to do things per external library but we're not there yet. If you do it that way, you'll be futureproofed. Other than that, you are free to do it whatever way you want.

@etnoy commented on GitHub (Feb 25, 2025): > Oh you're right, how silly of me: Library 1: /usr/src/app/external Library 2: /usr/src/app/external/Videos/XXX-YYY-ZZZ > > I thought I had put them in parallel dirs, but didn't. Any suggestions on the best way to clean this up with the least amount of reprocessing? My directory structure actually looks like this: > > All under /usr/app/external /RAWPics /OriginalPics /SharedPics /Videos > > Should I just split each into its own external library? I've been meaning to figure out how to reduce the amount of time spent rescanning as many of these folders are fairly static. Yes, I would put each in a separate library. My long term plan is to allow to do things per external library but we're not there yet. If you do it that way, you'll be futureproofed. Other than that, you are free to do it whatever way you want.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#5331
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?