immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

Attempting to finish incomplete move Failed, some files were missing #3593

New Issue
Closed
opened 2026-02-05 09:05:35 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 09:05:35 +03:00
Owner
Copy Link

Originally created by @piscesvivian on GitHub (Jun 20, 2024).

The bug

  1. I don't know how, may be server restart / upload failure / task failure ? some files were missing.
  2. when I run storage template convert task, there server got a lots of ERROR

The OS that Immich Server is running on

Fedora 39

Version of Immich Server

v1.106.4

Version of Immich Mobile App

v1.106.4

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    volumes:
      - ${ML_MODEL_CACHE}:/cache
    env_file:
      - .env
    restart: always
    extends:
      file: hwaccel.ml.yml
      service: cuda

  redis:
    container_name: immich_redis
    image: m.daocloud.io/docker.io/library/redis:6.2-alpine
    restart: always

  database:
    container_name: immich_postgres
    image: m.daocloud.io/docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    ports:
      - 5432:5432
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data:z
      # - pgdata:/var/lib/postgresql/data
    restart: always

# volumes:
  # pgdata:
  # model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/f/immich-app/library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich
DB_DATA_LOCATION=./postgresql/data

REDIS_HOSTNAME=immich_redis
HF_ENDPOINT=https://hf-mirror.com

ML_MODEL_CACHE=./models

Reproduction steps

1. You need some file not exists on disk, but I don't know how it happends.
2. Login as Admin
3. Go to PAGE http://127.0.0.1:2283/admin/jobs-status
4. RUN 存储模板转换 [Apply the current 存储模板 to previously uploaded assets]
5. Monitor the server log, and get the error

Relevant log output

[Nest] 14  - 06/20/2024, 3:39:07 PM     LOG [Microservices:MediaService] Attempting to finish incomplete move: upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC => upload/library/c6e140e1-badd-4b5b-b93f-5fceab63da13/2023/01/20/IMG_2577.HEIC
�[Nest] 14  - 06/20/2024, 3:39:07 PM     LOG [Microservices:MediaService] Found file at new location
�[Nest] 14  - 06/20/2024, 3:39:07 PM   ERROR [Microservices:StorageTemplateService] Problem applying storage template
[Nest] 14  - 06/20/2024, 3:39:07 PM   ERROR [Microservices:StorageTemplateService] Error: ENOENT: no such file or directory, stat 'upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC'
    at async Object.stat (node:internal/fs/promises:1029:18)
    at async StorageCore.verifyNewPathContentsMatchesExpected (/usr/src/app/dist/cores/storage.core.js:176:25)
    at async StorageCore.moveFile (/usr/src/app/dist/cores/storage.core.js:132:19)
    at async /usr/src/app/dist/services/storage-template.service.js:154:17
    at async /usr/src/app/dist/repositories/database.repository.js:177:23
�[Nest] 14  - 06/20/2024, 3:39:07 PM   ERROR [Microservices:StorageTemplateService] Object:
{
  "id": "35111153-6a9f-460f-9e76-06c2da588c1d",
  "oldPath": "upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC",
  "newPath": "upload/library/c6e140e1-badd-4b5b-b93f-5fceab63da13/2023/01/20/IMG_2577+1.HEIC"
}

Additional information

Is there a function to check file on disk and if it not exists, then clear the related data.
Also, there is a lots of invalid file(MOV / HEIC / JPEG), in upload/upload folders, it couldn't open by windows photo app or FFMpeg.
Can we supply a way to sync disk with db status, so that they can be consistent

Originally created by @piscesvivian on GitHub (Jun 20, 2024). ### The bug 1. I don't know how, may be server restart / upload failure / task failure ? some files were missing. 2. when I run storage template convert task, there server got a lots of ERROR ### The OS that Immich Server is running on Fedora 39 ### Version of Immich Server v1.106.4 ### Version of Immich Mobile App v1.106.4 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always extends: file: hwaccel.transcoding.yml service: nvenc immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda volumes: - ${ML_MODEL_CACHE}:/cache env_file: - .env restart: always extends: file: hwaccel.ml.yml service: cuda redis: container_name: immich_redis image: m.daocloud.io/docker.io/library/redis:6.2-alpine restart: always database: container_name: immich_postgres image: m.daocloud.io/docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} ports: - 5432:5432 volumes: - ${DB_DATA_LOCATION}:/var/lib/postgresql/data:z # - pgdata:/var/lib/postgresql/data restart: always # volumes: # pgdata: # model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/mnt/f/immich-app/library # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY=some-random-text DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich DB_DATA_LOCATION=./postgresql/data REDIS_HOSTNAME=immich_redis HF_ENDPOINT=https://hf-mirror.com ML_MODEL_CACHE=./models ``` ### Reproduction steps ```bash 1. You need some file not exists on disk, but I don't know how it happends. 2. Login as Admin 3. Go to PAGE http://127.0.0.1:2283/admin/jobs-status 4. RUN 存储模板转换 [Apply the current 存储模板 to previously uploaded assets] 5. Monitor the server log, and get the error ``` ### Relevant log output ```shell [Nest] 14 - 06/20/2024, 3:39:07 PM LOG [Microservices:MediaService] Attempting to finish incomplete move: upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC => upload/library/c6e140e1-badd-4b5b-b93f-5fceab63da13/2023/01/20/IMG_2577.HEIC �[Nest] 14 - 06/20/2024, 3:39:07 PM LOG [Microservices:MediaService] Found file at new location �[Nest] 14 - 06/20/2024, 3:39:07 PM ERROR [Microservices:StorageTemplateService] Problem applying storage template [Nest] 14 - 06/20/2024, 3:39:07 PM ERROR [Microservices:StorageTemplateService] Error: ENOENT: no such file or directory, stat 'upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC' at async Object.stat (node:internal/fs/promises:1029:18) at async StorageCore.verifyNewPathContentsMatchesExpected (/usr/src/app/dist/cores/storage.core.js:176:25) at async StorageCore.moveFile (/usr/src/app/dist/cores/storage.core.js:132:19) at async /usr/src/app/dist/services/storage-template.service.js:154:17 at async /usr/src/app/dist/repositories/database.repository.js:177:23 �[Nest] 14 - 06/20/2024, 3:39:07 PM ERROR [Microservices:StorageTemplateService] Object: { "id": "35111153-6a9f-460f-9e76-06c2da588c1d", "oldPath": "upload/upload/c6e140e1-badd-4b5b-b93f-5fceab63da13/bf/22/bf22018f-ef6e-4ead-a005-6962a398e501.HEIC", "newPath": "upload/library/c6e140e1-badd-4b5b-b93f-5fceab63da13/2023/01/20/IMG_2577+1.HEIC" } ``` ### Additional information Is there a function to check file on disk and if it not exists, then clear the related data. Also, there is a lots of invalid file(MOV / HEIC / JPEG), in upload/upload folders, it couldn't open by windows photo app or FFMpeg. Can we supply a way to sync disk with db status, so that they can be consistent
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#3593
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?