immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

Storage migration complains about non-existent sidecars #2076

New Issue
Closed
opened 2026-02-05 04:59:50 +03:00 by OVERLORD · 4 comments
OVERLORD commented 2026-02-05 04:59:50 +03:00
Owner
Copy Link

Originally created by @mmomjian on GitHub (Jan 28, 2024).

The bug

Non-existent sidecar files are referenced in the database.

The OS that Immich Server is running on

Docker on Debian 12

Version of Immich Server

v1.93.3

Version of Immich Mobile App

n/a

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

n/a

Your .env content

n/a

Reproduction steps

1. Import photos from Google Photos using a storage template
2. Change the template, and run a migration job in the server jobs
3. view logs of immich-microservices

Additional information

The immich-microservices logfiles will contain lines like:

[Nest] 8  - 01/28/2024, 6:37:33 PM     LOG [StorageCore] Attempting to finish incomplete move: upload/library/<uuid>/<oldpath>/HEIC.xmp => upload/library/<uuid>/<newpath>/IMG_0157.HEIC.xmp
[Nest] 8  - 01/28/2024, 6:37:33 PM    WARN [StorageCore] Unable to complete move. File does not exist at either location.

I can confirm based on backups that this file did not have an XMP associated with it. When reviewing storage-template.service.ts , it looks like it might try to move a sidecar regardless of if it exists or not - not really sure.

Postgres reports:

immich=# select "originalPath", "sidecarPath" from assets WHERE "originalPath" LIKE '%Newport%IMG_0157.HEIC%';
                                         originalPath                                          |                                            sidecarPath                                      
-----------------------------------------------------------------------------------------------+---------------------------------------------------------------------------------------------------
 upload/library/uuid/path/IMG_0157.HEIC | upload/library/uuid/path/IMG_0157.HEIC.xmp
(1 row)

These images were uploaded using immich-go, so it's possible that it somehow create a false XMP path? I'm unsure how that all works.

Originally created by @mmomjian on GitHub (Jan 28, 2024). ### The bug Non-existent sidecar files are referenced in the database. ### The OS that Immich Server is running on Docker on Debian 12 ### Version of Immich Server v1.93.3 ### Version of Immich Mobile App n/a ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML n/a ``` ### Your .env content ```Shell n/a ``` ### Reproduction steps ```bash 1. Import photos from Google Photos using a storage template 2. Change the template, and run a migration job in the server jobs 3. view logs of immich-microservices ``` ### Additional information The immich-microservices logfiles will contain lines like: ``` [Nest] 8 - 01/28/2024, 6:37:33 PM LOG [StorageCore] Attempting to finish incomplete move: upload/library/<uuid>/<oldpath>/HEIC.xmp => upload/library/<uuid>/<newpath>/IMG_0157.HEIC.xmp [Nest] 8 - 01/28/2024, 6:37:33 PM WARN [StorageCore] Unable to complete move. File does not exist at either location. ``` I can confirm based on backups that this file did not have an XMP associated with it. When reviewing storage-template.service.ts , it looks like it might try to move a sidecar regardless of if it exists or not - not really sure. Postgres reports: ``` immich=# select "originalPath", "sidecarPath" from assets WHERE "originalPath" LIKE '%Newport%IMG_0157.HEIC%'; originalPath | sidecarPath -----------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------- upload/library/uuid/path/IMG_0157.HEIC | upload/library/uuid/path/IMG_0157.HEIC.xmp (1 row) ``` These images were uploaded using immich-go, so it's possible that it somehow create a false XMP path? I'm unsure how that all works.
OVERLORD commented 2026-02-05 04:59:54 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Jan 28, 2024):

cc @simulot, I think this may have been an immich-go bug in the past, right?

@bo0tzz commented on GitHub (Jan 28, 2024): cc @simulot, I think this may have been an immich-go bug in the past, right?
OVERLORD commented 2026-02-05 05:00:03 +03:00
Author
Owner
Copy Link

@mmomjian commented on GitHub (Jan 28, 2024):

On further research, I have 5 images (out of ~50k) that seem to have an unmigrated sidecar file from upload/upload. My upload/ folder does not contain any files, just empty folders. It's not clear to me that an issue in immich-go would lead to this kind of database error - whatever the source of the missing XMP files, shouldn't Immich check for the existence of said file before loading it into the DB?

immich=# select "originalPath", "sidecarPath" from assets WHERE "sidecarPath" LIKE '%upload/upload%';
                                            originalPath                                             |                                              sidecarPath           
-----------------------------------------------------------------------------------------------------+--------------------------------------------------------------------------------------------------------
 upload/library/e46649fa-6d49-4b4d-b923-3450743816ee/IMG/redacted/IMG_2993+1.HEIC        | upload/upload/e46649fa-6d49-4b4d-b923-3450743816ee/2d/0e/2d0e693e-fb6c-47ad-ab3d-8571594bb1e2.HEIC.xmp

It seems to appear that both of these issues would be fixed (sort of) by draft PR #6293 - this would not fix the missing data, but it would fix the reference to non-existent files.

@mmomjian commented on GitHub (Jan 28, 2024): On further research, I have 5 images (out of ~50k) that seem to have an unmigrated sidecar file from upload/upload. My upload/ folder does not contain any files, just empty folders. It's not clear to me that an issue in immich-go would lead to this kind of database error - whatever the source of the missing XMP files, shouldn't Immich check for the existence of said file before loading it into the DB? ``` immich=# select "originalPath", "sidecarPath" from assets WHERE "sidecarPath" LIKE '%upload/upload%'; originalPath | sidecarPath -----------------------------------------------------------------------------------------------------+-------------------------------------------------------------------------------------------------------- upload/library/e46649fa-6d49-4b4d-b923-3450743816ee/IMG/redacted/IMG_2993+1.HEIC | upload/upload/e46649fa-6d49-4b4d-b923-3450743816ee/2d/0e/2d0e693e-fb6c-47ad-ab3d-8571594bb1e2.HEIC.xmp ``` It seems to appear that both of these issues would be fixed (sort of) by draft PR #6293 - this would not fix the missing data, but it would fix the reference to non-existent files.
OVERLORD commented 2026-02-05 05:00:13 +03:00
Author
Owner
Copy Link

@simulot commented on GitHub (Jan 29, 2024):

immich-go used to create XMP files from google photo import. As far as I know this is fixed since ... long. (i need to dig into commits)

But in this case, I think that the immich server just check the existence of an XMP files

@simulot commented on GitHub (Jan 29, 2024): immich-go used to create XMP files from google photo import. As far as I know this is fixed since ... long. (i need to dig into commits) But in this case, I think that the immich server just check the existence of an XMP files
OVERLORD commented 2026-02-05 05:00:18 +03:00
Author
Owner
Copy Link

@jrasm91 commented on GitHub (Mar 22, 2024):

I believe we handle deleted xmp files now.

@jrasm91 commented on GitHub (Mar 22, 2024): I believe we handle deleted xmp files now.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#2076
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?