immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 11 Packages Projects Releases 10 Wiki Activity

[BUG] Path not found errors during thumbnail creation in 1.52.0 #776

New Issue
Closed
opened 2026-02-04 22:26:49 +03:00 by OVERLORD · 6 comments
OVERLORD commented 2026-02-04 22:26:49 +03:00
Owner
Copy Link

Originally created by @raisinbear on GitHub (Mar 29, 2023).

The bug

Hi,

thanks again for this update with great new features!
I noticed during upload of a bunch of images that a lot of errors along the lines of

immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM     LOG [MediaService] Start Generating Video Thumbnail
immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM   ERROR [MediaService] Failed to generate thumbnail for asset: 39e6bfb5-feec-421b-9cd2-d5e927be9300/VIDEO
immich_microservices  | Error: ffmpeg exited with code 1: upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/c58a8764-7c7d-4fd7-b488-7f5c6e6ee67a.mov: No such file or directory
immich_microservices  | 
immich_microservices  |     at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/processor.js:182:22)
immich_microservices  |     at ChildProcess.emit (node:events:527:28)
immich_microservices  |     at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12)
immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=50780146-1436-44cd-89fa-85e4103f8629)
immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM   ERROR [MediaService] Failed to generate thumbnail for asset: 50780146-1436-44cd-89fa-85e4103f8629/IMAGE
immich_microservices  | Error: File not found - /usr/src/app/upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/43141149-ea82-47b1-9c32-e67dda20a947.HEIC
immich_microservices  |     at BinaryExtractionTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:41:29)
immich_microservices  |     at BinaryExtractionTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dis
t/Task.js:146:40)
immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM    WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=2b4644d9-df11-4672-b66d-5db7306b832d)
immich_microservices  | [Nest] 1  - 03/29/2023, 9:21:15 AM   ERROR [MediaService] Failed to generate thumbnail for asset: 2b4644d9-df11-4672-b66d-5db7306b832d/IMAGE
immich_microservices  | Error: File not found - /usr/src/app/upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/24abe953-dc7e-4d51-922f-c058e75d1970.HEIC
immich_microservices  |     at BinaryExtractionTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:41:29)
immich_microservices  |     at BinaryExtractionTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dist/Task.js:146:40)

were showing up in the log and the new images weren't showing up in the mobile app due to the missing thumbnails.
It looks like the thumbnail processor is still looking in the upload path although the images were already moved to their final location under library/. On first glance this makes sense to me, because since 1.52.0, a job called STORAGE_TEMPLATE_MIGRATION_SINGLE, which didn't exist in the last release, is triggered by the metadata-extraction.processor. If my guessing is correct, this would move the uploaded file to it's designated place in the file system and leave the probably more time-consuming thumbnail generation job without the image in the still expected upload location.

The OS that Immich Server is running on

Debian 11

Version of Immich Server

v1.52.0

Version of Immich Mobile App

v1.52.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: ["/bin/sh", "./start-server.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:release
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - redis
      - database
      - typesense
    restart: always

#  immich-machine-learning:
#    container_name: immich_machine_learning
#    image: ghcr.io/immich-app/immich-machine-learning:release
#    volumes:
#      - ${UPLOAD_LOCATION}:/usr/src/app/upload
#      - model-cache:/cache
#    env_file:
#      - .env
#    environment:
#      - NODE_ENV=production
#    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:release
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.0
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
    logging:
      driver: none
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:release
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

###################################################################################
# Database
###################################################################################

DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_PASSWORD=postgres
DB_DATABASE_NAME=immich

# Optional Database settings:
# DB_PORT=5432

###################################################################################
# Redis
###################################################################################

REDIS_HOSTNAME=immich_redis

# Optional Redis settings:
# REDIS_PORT=6379
# REDIS_DBINDEX=0
# REDIS_PASSWORD=
# REDIS_SOCKET=

###################################################################################
# Upload File Location
#
# This is the location where uploaded files are stored.
###################################################################################

UPLOAD_LOCATION=/home/immich/data

###################################################################################
# Typesense
###################################################################################
TYPESENSE_API_KEY=[...removed...]
# TYPESENSE_ENABLED=false

###################################################################################
# Reverse Geocoding
#
# Reverse geocoding is done locally which has a small impact on memory usage
# This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable
# This ranges from 0-3 with 3 being the most precise
# 3 - Cities > 500 population: ~200MB RAM
# 2 - Cities > 1000 population: ~150MB RAM
# 1 - Cities > 5000 population: ~80MB RAM
# 0 - Cities > 15000 population: ~40MB RAM
####################################################################################

# DISABLE_REVERSE_GEOCODING=false
# REVERSE_GEOCODING_PRECISION=3

####################################################################################
# WEB - Optional
#
# Custom message on the login page, should be written in HTML form.
# For example:
# PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>"
####################################################################################

PUBLIC_LOGIN_PAGE_MESSAGE=

####################################################################################
# Alternative Service Addresses - Optional
#
# This is an advanced feature for users who may be running their immich services on different hosts.
# It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers.
# Note: immich-microservices is bound to 3002, but no references are made
####################################################################################

IMMICH_WEB_URL=http://immich-web:3000
IMMICH_SERVER_URL=http://immich-server:3001
IMMICH_MACHINE_LEARNING_URL=False

####################################################################################
# Alternative API's External Address - Optional
#
# This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery.
# You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash.
# NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api
# Examples: http://localhost:3001, http://immich-api.example.com, etc
####################################################################################

#IMMICH_API_URL_EXTERNAL=http://localhost:3001

Reproduction steps

1. Upload images, might take a bunch. With one or two I couldn't reproduce it for whatever reason. With ~16 I always can.
2. Observe immich console output / logs.

Additional information

No response

Originally created by @raisinbear on GitHub (Mar 29, 2023). ### The bug Hi, thanks again for this update with great new features! I noticed during upload of a bunch of images that a lot of errors along the lines of ``` immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM LOG [MediaService] Start Generating Video Thumbnail immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM ERROR [MediaService] Failed to generate thumbnail for asset: 39e6bfb5-feec-421b-9cd2-d5e927be9300/VIDEO immich_microservices | Error: ffmpeg exited with code 1: upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/c58a8764-7c7d-4fd7-b488-7f5c6e6ee67a.mov: No such file or directory immich_microservices | immich_microservices | at ChildProcess.<anonymous> (/usr/src/app/node_modules/fluent-ffmpeg/lib/processor.js:182:22) immich_microservices | at ChildProcess.emit (node:events:527:28) immich_microservices | at Process.ChildProcess._handle.onexit (node:internal/child_process:291:12) immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=50780146-1436-44cd-89fa-85e4103f8629) immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM ERROR [MediaService] Failed to generate thumbnail for asset: 50780146-1436-44cd-89fa-85e4103f8629/IMAGE immich_microservices | Error: File not found - /usr/src/app/upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/43141149-ea82-47b1-9c32-e67dda20a947.HEIC immich_microservices | at BinaryExtractionTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:41:29) immich_microservices | at BinaryExtractionTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dis t/Task.js:146:40) immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM WARN [MediaService] Failed to generate jpeg thumbnail using sharp, trying with exiftool-vendored (asset=2b4644d9-df11-4672-b66d-5db7306b832d) immich_microservices | [Nest] 1 - 03/29/2023, 9:21:15 AM ERROR [MediaService] Failed to generate thumbnail for asset: 2b4644d9-df11-4672-b66d-5db7306b832d/IMAGE immich_microservices | Error: File not found - /usr/src/app/upload/upload/ea8f7f01-33a1-4b8a-85d4-15ab6689bc27/24abe953-dc7e-4d51-922f-c058e75d1970.HEIC immich_microservices | at BinaryExtractionTask.parser (/usr/src/app/node_modules/exiftool-vendored/dist/ExifToolTask.js:41:29) immich_microservices | at BinaryExtractionTask._Task_resolve (/usr/src/app/node_modules/batch-cluster/dist/Task.js:146:40) ``` were showing up in the log and the new images weren't showing up in the mobile app due to the missing thumbnails. It looks like the thumbnail processor is still looking in the upload path although the images were already moved to their final location under `library/`. On first glance this makes sense to me, because since `1.52.0`, a job called `STORAGE_TEMPLATE_MIGRATION_SINGLE`, which didn't exist in the last release, is triggered by the `metadata-extraction.processor`. If my guessing is correct, this would move the uploaded file to it's designated place in the file system and leave the probably more time-consuming thumbnail generation job without the image in the still expected upload location. ### The OS that Immich Server is running on Debian 11 ### Version of Immich Server v1.52.0 ### Version of Immich Mobile App v1.52.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:release entrypoint: ["/bin/sh", "./start-server.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:release entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload env_file: - .env environment: - NODE_ENV=production depends_on: - redis - database - typesense restart: always # immich-machine-learning: # container_name: immich_machine_learning # image: ghcr.io/immich-app/immich-machine-learning:release # volumes: # - ${UPLOAD_LOCATION}:/usr/src/app/upload # - model-cache:/cache # env_file: # - .env # environment: # - NODE_ENV=production # restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:release entrypoint: ["/bin/sh", "./entrypoint.sh"] env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.0 environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data logging: driver: none volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2 restart: always database: container_name: immich_postgres image: postgres:14 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} PG_DATA: /var/lib/postgresql/data volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:release environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 2283:8080 logging: driver: none depends_on: - immich-server restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell ################################################################################### # Database ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_PASSWORD=postgres DB_DATABASE_NAME=immich # Optional Database settings: # DB_PORT=5432 ################################################################################### # Redis ################################################################################### REDIS_HOSTNAME=immich_redis # Optional Redis settings: # REDIS_PORT=6379 # REDIS_DBINDEX=0 # REDIS_PASSWORD= # REDIS_SOCKET= ################################################################################### # Upload File Location # # This is the location where uploaded files are stored. ################################################################################### UPLOAD_LOCATION=/home/immich/data ################################################################################### # Typesense ################################################################################### TYPESENSE_API_KEY=[...removed...] # TYPESENSE_ENABLED=false ################################################################################### # Reverse Geocoding # # Reverse geocoding is done locally which has a small impact on memory usage # This memory usage can be altered by changing the REVERSE_GEOCODING_PRECISION variable # This ranges from 0-3 with 3 being the most precise # 3 - Cities > 500 population: ~200MB RAM # 2 - Cities > 1000 population: ~150MB RAM # 1 - Cities > 5000 population: ~80MB RAM # 0 - Cities > 15000 population: ~40MB RAM #################################################################################### # DISABLE_REVERSE_GEOCODING=false # REVERSE_GEOCODING_PRECISION=3 #################################################################################### # WEB - Optional # # Custom message on the login page, should be written in HTML form. # For example: # PUBLIC_LOGIN_PAGE_MESSAGE="This is a demo instance of Immich.<br><br>Email: <i>demo@demo.de</i><br>Password: <i>demo</i>" #################################################################################### PUBLIC_LOGIN_PAGE_MESSAGE= #################################################################################### # Alternative Service Addresses - Optional # # This is an advanced feature for users who may be running their immich services on different hosts. # It will not change which address or port that services bind to within their containers, but it will change where other services look for their peers. # Note: immich-microservices is bound to 3002, but no references are made #################################################################################### IMMICH_WEB_URL=http://immich-web:3000 IMMICH_SERVER_URL=http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL=False #################################################################################### # Alternative API's External Address - Optional # # This is an advanced feature used to control the public server endpoint returned to clients during Well-known discovery. # You should only use this if you want mobile apps to access the immich API over a custom URL. Do not include trailing slash. # NOTE: At this time, the web app will not be affected by this setting and will continue to use the relative path: /api # Examples: http://localhost:3001, http://immich-api.example.com, etc #################################################################################### #IMMICH_API_URL_EXTERNAL=http://localhost:3001 ``` ### Reproduction steps ```bash 1. Upload images, might take a bunch. With one or two I couldn't reproduce it for whatever reason. With ~16 I always can. 2. Observe immich console output / logs. ``` ### Additional information _No response_
OVERLORD commented 2026-02-04 22:26:57 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Mar 29, 2023):

Did you run the storage migration job after updating to let the server move files to the new paths? I misread, these are newly uploaded images.

@bo0tzz commented on GitHub (Mar 29, 2023): ~~Did you run the storage migration job after updating to let the server move files to the new paths?~~ I misread, these are newly uploaded images.
OVERLORD commented 2026-02-04 22:27:00 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Mar 29, 2023):

@jrasm91 I think this scenario can happen if the EXIF date is different than ctime in the metadata extraction job, right?

@alextran1502 commented on GitHub (Mar 29, 2023): @jrasm91 I think this scenario can happen if the EXIF date is different than ctime in the metadata extraction job, right?
OVERLORD commented 2026-02-04 22:27:07 +03:00
Author
Owner
Copy Link

@jrasm91 commented on GitHub (Mar 29, 2023):

Basically. More generally, it's an issue of the AssetEntity that we add queue can become old/stale. It looks like the storage migration may update the path around the same time the asset is being processed by the thumbnail jobs, but the data in the queue still has the old path. It's not a bad idea for each job to load the asset from the database (by id), and use the latest values, rather than relying on the queue being the "source of truth".

@jrasm91 commented on GitHub (Mar 29, 2023): Basically. More generally, it's an issue of the `AssetEntity` that we add queue can become old/stale. It looks like the storage migration may update the path around the same time the asset is being processed by the thumbnail jobs, but the data in the queue still has the old path. It's not a bad idea for each job to load the asset from the database (by id), and use the latest values, rather than relying on the queue being the "source of truth".
OVERLORD commented 2026-02-04 22:27:11 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Mar 29, 2023):

It's not a bad idea for each job to load the asset from the database (by id)

I thought we'd changed to doing that a while ago, if not we definitely should.

@bo0tzz commented on GitHub (Mar 29, 2023): > It's not a bad idea for each job to load the asset from the database (by id) I thought we'd changed to doing that a while ago, if not we definitely should.
OVERLORD commented 2026-02-04 22:27:14 +03:00
Author
Owner
Copy Link

@jrasm91 commented on GitHub (Mar 29, 2023):

We changed it for the search jobs, those use asset ids, but everything else is still by AssetEntity

@jrasm91 commented on GitHub (Mar 29, 2023): We changed it for the search jobs, those use asset ids, but everything else is still by `AssetEntity`
OVERLORD commented 2026-02-04 22:27:17 +03:00
Author
Owner
Copy Link

@rhullah commented on GitHub (Apr 11, 2023):

Did you noticed that the path it could find had a double upload folder at the beginning: upload/upload/...

Mine is only a single upload folder. Any idea where the additional one would have come from?

@rhullah commented on GitHub (Apr 11, 2023): Did you noticed that the path it could find had a double `upload` folder at the beginning: `upload/upload/...` Mine is only a single upload folder. Any idea where the additional one would have come from?
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#776
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?