immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 12 Packages Projects Releases 10 Wiki Activity

[BUG] Redundant re-uploading of all files instead of uploading only new files when retrying upload of the same folder #1556

New Issue
Closed
opened 2026-02-05 02:21:19 +03:00 by OVERLORD · 1 comment
OVERLORD commented 2026-02-05 02:21:19 +03:00
Owner
Copy Link

Originally created by @LonnyX on GitHub (Nov 2, 2023).

The bug

Upon retrying to upload a folder, the system redundantly re-uploads all files within the folder, even if it has already detected that some of these files have been uploaded previously. This behavior results in unnecessary redundancy and prolongs the uploading process.

As you can see below, instead of uploading 983 (remaining files) it will re-upload the whole folder.
Capture d’écran 2023-11-02 à 21 46 57

The OS that Immich Server is running on

Raspberry Pi OS Lite 12 (bookworm)

Version of Immich Server

v1.84.0

Version of Immich Mobile App

doesn't matter

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start.sh", "immich"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: ["start.sh", "microservices"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/media/devmon/Elements/immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=********
DB_PASSWORD=*********

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

IMMICH_MACHINE_LEARNING_URL=false
TYPESENSE_ENABLED=false

Reproduction steps

1. Upload a folder with multiple files.
2. Wait a bit and kill the upload process
3. Note how many files were successfully uploaded.
4. Retry the upload of the same folder.
5. Observe the system's behavior during the re-uploading process. (see my screenshot)

Additional information

  1. Install immich on Raspberry Pi 4
  2. Attempt to run bulk upload 1000 photos & videos
  3. Observe the behavior of Docker when the CPU load is high.
### Tasks
Originally created by @LonnyX on GitHub (Nov 2, 2023). ### The bug Upon retrying to upload a folder, the system redundantly re-uploads all files within the folder, even if it has already detected that some of these files have been uploaded previously. This behavior results in unnecessary redundancy and prolongs the uploading process. As you can see below, instead of uploading 983 (remaining files) it will re-upload the whole folder. <img width="1013" alt="Capture d’écran 2023-11-02 à 21 46 57" src="https://github.com/immich-app/immich/assets/10662320/e99af010-8b55-4604-9563-b3ec71e54c86"> ### The OS that Immich Server is running on Raspberry Pi OS Lite 12 (bookworm) ### Version of Immich Server v1.84.0 ### Version of Immich Mobile App doesn't matter ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: ["start.sh", "immich"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: ["start.sh", "microservices"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release} env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data # remove this to get debug messages - GLOG_minloglevel=1 volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release} ports: - 2283:8080 depends_on: - immich-server - immich-web restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/media/devmon/Elements/immich # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY=******** DB_PASSWORD=********* # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis IMMICH_MACHINE_LEARNING_URL=false TYPESENSE_ENABLED=false ``` ### Reproduction steps ```bash 1. Upload a folder with multiple files. 2. Wait a bit and kill the upload process 3. Note how many files were successfully uploaded. 4. Retry the upload of the same folder. 5. Observe the system's behavior during the re-uploading process. (see my screenshot) ``` ### Additional information 1. Install immich on Raspberry Pi 4 2. Attempt to run bulk upload 1000 photos & videos 3. Observe the behavior of Docker when the CPU load is high. ```[tasklist] ### Tasks ```
OVERLORD commented 2026-02-05 02:21:27 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Nov 3, 2023):

Hello, if the file is duplicated and rejected from the server, the CLI will try to re-upload the next time because the CLI client is stateless. It can only keep track of the content that has been uploaded to the server

@alextran1502 commented on GitHub (Nov 3, 2023): Hello, if the file is duplicated and rejected from the server, the CLI will try to re-upload the next time because the CLI client is stateless. It can only keep track of the content that has been uploaded to the server
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1556
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?