immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 08:41:28 +03:00
Code Issues 26 Packages Projects Releases 10 Wiki Activity

[BUG] Small number of missed photos #1496

New Issue
Closed
opened 2026-02-05 02:04:30 +03:00 by OVERLORD · 9 comments
OVERLORD commented 2026-02-05 02:04:30 +03:00
Owner
Copy Link

Originally created by @aquasync on GitHub (Oct 23, 2023).

The bug

Hi,

I just set up a fresh install of the latest version of immich, and tried to sync the photo library from an iphone. This mostly worked fine, however I noticed a few photos with missing thumbnails in the timeline view when accessing via the web interface.

Some of them have exif data listed, others don't (says "NO EXIF INFO AVAILABLE"). In both cases the filenames correspond to photos that are on the iphone, but didn't end up in the library. There are however 4 images left in the upload folder with guid filenames. When using the immich app on the iphone, the broken thumbnail images do appear, and clicking on them shows the (temporary) guid filenames used in the upload process, not their original filenames (eg IMG_1094.HEIC). It says that all 1889 files were uploaded successfully.

When I go to repair in the admin page, it just says "UNTRACKS FILES (2)", and lists only 2 of them. That is curious, and looking in the assets table in the database, it seems that the other 2 are being referenced directly from the upload dir:

                                         originalPath                                         |                 originalFileName
----------------------------------------------------------------------------------------------+---------------------------------------------------
...
 upload/library/xxxxxxx/2023/IMG_1067.HEIC                                                    | IMG_1067
 upload/library/xxxxxxx/2023/IMG_1068.HEIC                                                    | IMG_1068
 upload/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/b8a11332-4b49-4670-8507-f636106894a6.HEIC | IMG_1069
 upload/library/xxxxxxx/2023/IMG_1070.HEIC                                                    | IMG_1070
 upload/library/xxxxxxx/2023/IMG_1071.HEIC                                                    | IMG_1071
...
 upload/library/xxxxxxx/2023/IMG_1092.HEIC                                                    | IMG_1092
 upload/library/xxxxxxx/2023/IMG_1093.HEIC                                                    | IMG_1093
 upload/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/3acc3342-d4f1-4047-b4e9-5b1d95741aee.HEIC | IMG_1094
 upload/library/xxxxxxx/2023/IMG_1095.HEIC                                                    | IMG_1095
 upload/library/xxxxxxx/2023/IMG_1096.HEIC                                                    | IMG_1096

Checking the 2 untracked guids, they both seem to be corrupted (partial uploads I guess). Finally, the front end says 1835 photos & 54 videos, which aligns with the 1889 assets shown in the iphone app. However just checking the file counts in the library folder (find -type f) I get 1893 files. With the 4 additional files in the upload folder, presumably that means I am missing an additional 2 photos? Curiously select count(*) from assets gives me only 1895, so that doesn't seem to agree with the frontend anyway?

I suppose the main concerns here are:

  • How to ensure it doesn't leave some photos in the upload folder, despite adding them to the library. This doesn't seem to be intended behaviour, and they don't seem to have been added correctly as they are not visible in the web interface (just missing thumbnail icon).
  • How to fix the incomplete/untracked files. Am I just supposed to delete them and then they get re-uploaded?
  • Is there some way to ensure the mobile app will retry these failed uploads? It seems to think everything has gone fine, however some media hasn't been backed up successfully.

The OS that Immich Server is running on

Ubuntu 20.04

Version of Immich Server

v1.82.1

Version of Immich Mobile App

v1.80.0 build.120

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start.sh", "immich"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: ["start.sh", "microservices"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 0.0.0.0:2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/pool0/immich/library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Clean install of latest immich server & client
2. Sync library from an iphone

Additional information

No response

Originally created by @aquasync on GitHub (Oct 23, 2023). ### The bug Hi, I just set up a fresh install of the latest version of immich, and tried to sync the photo library from an iphone. This mostly worked fine, however I noticed a few photos with missing thumbnails in the timeline view when accessing via the web interface. Some of them have exif data listed, others don't (says "NO EXIF INFO AVAILABLE"). In both cases the filenames correspond to photos that are on the iphone, but didn't end up in the library. There are however 4 images left in the upload folder with guid filenames. When using the immich app on the iphone, the broken thumbnail images do appear, and clicking on them shows the (temporary) guid filenames used in the upload process, not their original filenames (eg IMG_1094.HEIC). It says that all 1889 files were uploaded successfully. When I go to repair in the admin page, it just says "UNTRACKS FILES (2)", and lists only 2 of them. That is curious, and looking in the assets table in the database, it seems that the other 2 are being referenced directly from the upload dir: ``` originalPath | originalFileName ----------------------------------------------------------------------------------------------+--------------------------------------------------- ... upload/library/xxxxxxx/2023/IMG_1067.HEIC | IMG_1067 upload/library/xxxxxxx/2023/IMG_1068.HEIC | IMG_1068 upload/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/b8a11332-4b49-4670-8507-f636106894a6.HEIC | IMG_1069 upload/library/xxxxxxx/2023/IMG_1070.HEIC | IMG_1070 upload/library/xxxxxxx/2023/IMG_1071.HEIC | IMG_1071 ... upload/library/xxxxxxx/2023/IMG_1092.HEIC | IMG_1092 upload/library/xxxxxxx/2023/IMG_1093.HEIC | IMG_1093 upload/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/3acc3342-d4f1-4047-b4e9-5b1d95741aee.HEIC | IMG_1094 upload/library/xxxxxxx/2023/IMG_1095.HEIC | IMG_1095 upload/library/xxxxxxx/2023/IMG_1096.HEIC | IMG_1096 ``` Checking the 2 untracked guids, they both seem to be corrupted (partial uploads I guess). Finally, the front end says 1835 photos & 54 videos, which aligns with the 1889 assets shown in the iphone app. However just checking the file counts in the library folder (find -type f) I get 1893 files. With the 4 additional files in the upload folder, presumably that means I am missing an additional 2 photos? Curiously select count(*) from assets gives me only 1895, so that doesn't seem to agree with the frontend anyway? I suppose the main concerns here are: * How to ensure it doesn't leave some photos in the upload folder, despite adding them to the library. This doesn't seem to be intended behaviour, and they don't seem to have been added correctly as they are not visible in the web interface (just missing thumbnail icon). * How to fix the incomplete/untracked files. Am I just supposed to delete them and then they get re-uploaded? * Is there some way to ensure the mobile app will retry these failed uploads? It seems to think everything has gone fine, however some media hasn't been backed up successfully. ### The OS that Immich Server is running on Ubuntu 20.04 ### Version of Immich Server v1.82.1 ### Version of Immich Mobile App v1.80.0 build.120 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: ["start.sh", "immich"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: ["start.sh", "microservices"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release} env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data # remove this to get debug messages - GLOG_minloglevel=1 volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release} environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 0.0.0.0:2283:8080 depends_on: - immich-server - immich-web restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/pool0/immich/library # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY=some-random-text DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps ```bash 1. Clean install of latest immich server & client 2. Sync library from an iphone ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 02:04:34 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Oct 23, 2023):

We added a table to keep track of the move. Can you help query all the entries in the move_history table?

This is the issue with a race condition while handling the process of moving the files from upload folder to its respective library path to conform with the storage template.

If there are entries in move_history table, you can fix it with running the Storage Migration job to re-trigger the move again.

Let me know how it goes.

@alextran1502 commented on GitHub (Oct 23, 2023): We added a table to keep track of the move. Can you help query all the entries in the `move_history` table? This is the issue with a race condition while handling the process of moving the files from `upload` folder to its respective `library` path to conform with the storage template. If there are entries in `move_history` table, you can fix it with running the `Storage Migration` job to re-trigger the move again. Let me know how it goes.
OVERLORD commented 2026-02-05 02:04:36 +03:00
Author
Owner
Copy Link

@aquasync commented on GitHub (Oct 24, 2023):

Hi @alextran1502, unfortunately the move_history table is empty.

@aquasync commented on GitHub (Oct 24, 2023): Hi @alextran1502, unfortunately the move_history table is empty.
OVERLORD commented 2026-02-05 02:04:40 +03:00
Author
Owner
Copy Link

@aquasync commented on GitHub (Oct 24, 2023):

I've looked into the 2 untracked files a bit more closely - they're both just partial uploads of files that did make it in to the library:

$ cmp /pool0/immich/library/library/xxxxxxx/2023/IMG_2009.HEIC /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/9b13b2d4-2a69-4e76-a7b5-0d5f004fd36c.HEIC
cmp: EOF on /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/9b13b2d4-2a69-4e76-a7b5-0d5f004fd36c.HEIC after byte 1864479, line 7200
$ cmp /pool0/immich/library/library/xxxxxxx/2023/IMG_0117.HEIC /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/d5b08250-ced3-4064-9746-e3ffd54d0141.HEIC
cmp: EOF on /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/d5b08250-ced3-4064-9746-e3ffd54d0141.HEIC after byte 1588437, in line 6326

So that makes perfect sense. Also looking into the item counts, the reason I was confused was due to 6 live photos - they end up with .mov files with isVisible = false, so that all seems to line up too.

I manually triggered the storage template migration and that picked up the 2 files stuck in upload - still nothing in the move_history table though?

So the only bug then is this race condition. Also relatedly: not sure why that should be a problems for the web interface? Maybe it refuses to read files that are outside of the library folder, and it works fine on the iphone app because it just uses the local images? After running storage migration they do work fine in the web now though.

@aquasync commented on GitHub (Oct 24, 2023): I've looked into the 2 untracked files a bit more closely - they're both just partial uploads of files that did make it in to the library: ``` $ cmp /pool0/immich/library/library/xxxxxxx/2023/IMG_2009.HEIC /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/9b13b2d4-2a69-4e76-a7b5-0d5f004fd36c.HEIC cmp: EOF on /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/9b13b2d4-2a69-4e76-a7b5-0d5f004fd36c.HEIC after byte 1864479, line 7200 $ cmp /pool0/immich/library/library/xxxxxxx/2023/IMG_0117.HEIC /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/d5b08250-ced3-4064-9746-e3ffd54d0141.HEIC cmp: EOF on /pool0/immich/library/upload/badf4376-3bc7-4ee4-8f8c-12be683a8c4e/d5b08250-ced3-4064-9746-e3ffd54d0141.HEIC after byte 1588437, in line 6326 ``` So that makes perfect sense. Also looking into the item counts, the reason I was confused was due to 6 live photos - they end up with .mov files with isVisible = false, so that all seems to line up too. I manually triggered the storage template migration and that picked up the 2 files stuck in upload - still nothing in the move_history table though? So the only bug then is this race condition. Also relatedly: not sure why that should be a problems for the web interface? Maybe it refuses to read files that are outside of the library folder, and it works fine on the iphone app because it just uses the local images? After running storage migration they do work fine in the web now though.
OVERLORD commented 2026-02-05 02:04:47 +03:00
Author
Owner
Copy Link

@yx043749 commented on GitHub (Oct 24, 2023):

same issue/problem，while Sync library from an iphone

@yx043749 commented on GitHub (Oct 24, 2023): same issue/problem，while Sync library from an iphone
OVERLORD commented 2026-02-05 02:04:54 +03:00
Author
Owner
Copy Link

@rlegan commented on GitHub (Nov 12, 2023):

same issue here when syncing with an iphone on the latest version (1.85.0) both on mobile and web apps

@rlegan commented on GitHub (Nov 12, 2023): same issue here when syncing with an iphone on the latest version (1.85.0) both on mobile and web apps
OVERLORD commented 2026-02-05 02:04:58 +03:00
Author
Owner
Copy Link

@agross commented on GitHub (Nov 18, 2023):

I also have a file that is reported as being untracked while also being visible in the library. The move_history table is empty.

Asset details:

2023-11-18_11-53-08

"Untracked files" table:

2023-11-18_11-53-54 
$ stat /fs/data/immich/app/library/agross/😺/2023/09/16/IMG_6556.MOV
  File: /fs/data/immich/app/library/agross/😺/2023/09/16/IMG_6556.MOV
  Size: 29408944        Blocks: 57440      IO Block: 4096   regular file
Device: 253,1   Inode: 692287260   Links: 1
Access: (0644/-rw-r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:object_r:default_t:s0
Access: 2023-11-18 11:48:03.453264637 +0100
Modify: 2023-10-22 09:46:52.398559580 +0200
Change: 2023-10-31 01:12:12.600693740 +0100
 Birth: 2023-10-22 09:46:40.048498576 +0200
@agross commented on GitHub (Nov 18, 2023): I also have a file that is reported as being untracked while also being visible in the library. The `move_history` table is empty. Asset details: <img width="319" alt="2023-11-18_11-53-08" src="https://github.com/immich-app/immich/assets/16245/8cf71a96-2497-4933-9c32-f2749582a0ac"> "Untracked files" table: <img width="573" alt="2023-11-18_11-53-54" src="https://github.com/immich-app/immich/assets/16245/14ae4b22-2267-4ef0-9a79-f9dc27bb47b6"> ```sh $ stat /fs/data/immich/app/library/agross/😺/2023/09/16/IMG_6556.MOV File: /fs/data/immich/app/library/agross/😺/2023/09/16/IMG_6556.MOV Size: 29408944 Blocks: 57440 IO Block: 4096 regular file Device: 253,1 Inode: 692287260 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: system_u:object_r:default_t:s0 Access: 2023-11-18 11:48:03.453264637 +0100 Modify: 2023-10-22 09:46:52.398559580 +0200 Change: 2023-10-31 01:12:12.600693740 +0100 Birth: 2023-10-22 09:46:40.048498576 +0200 ```
OVERLORD commented 2026-02-05 02:05:01 +03:00
Author
Owner
Copy Link

@streetcap commented on GitHub (Nov 22, 2023):

I am have been having the same problem for the past several releases. Currently on v1.88.2.

@streetcap commented on GitHub (Nov 22, 2023): I am have been having the same problem for the past several releases. Currently on v1.88.2.
OVERLORD commented 2026-02-05 02:05:03 +03:00
Author
Owner
Copy Link

@jrasm91 commented on GitHub (Jan 13, 2024):

Untracked files, especially in the upload folder can show up for partial/failed uploads, but the mobile app will automatically try them again. Besides that, is there an issue still occurring?

@jrasm91 commented on GitHub (Jan 13, 2024): Untracked files, especially in the upload folder can show up for partial/failed uploads, but the mobile app will automatically try them again. Besides that, is there an issue still occurring?
OVERLORD commented 2026-02-05 02:05:04 +03:00
Author
Owner
Copy Link

@agross commented on GitHub (Jan 14, 2024):

Besides that, is there an issue still occurring?

I still have untracked files in the library folder.

@agross commented on GitHub (Jan 14, 2024): > Besides that, is there an issue still occurring? I still have untracked files in the library folder.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1496
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?