immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 12 Packages Projects Releases 10 Wiki Activity

[BUG] Bulk Upload CLI fails to upload all files #1518

New Issue
Closed
opened 2026-02-05 02:12:25 +03:00 by OVERLORD · 4 comments
OVERLORD commented 2026-02-05 02:12:25 +03:00
Owner
Copy Link

Originally created by @ioslife on GitHub (Oct 26, 2023).

The bug

I am attempting to bulk upload 28,778 files via the CLI. On initial run, it says it will upload all 28778. That failed and now it says it will upload 12,094. I run that and it says it uploaded 12094/28778, but if I run the immich upload again, it still says 12,094 to upload.

Screenshot 2023-10-26 at 8 39 06 AM

The OS that Immich Server is running on

unraid

Version of Immich Server

v1.82.0

Version of Immich Mobile App

v1.82.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start.sh", "immich"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: ["start.sh", "microservices"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
      - typesense
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  immich-web:
    container_name: immich_web
    image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release}
    env_file:
      - .env
    restart: always

  typesense:
    container_name: immich_typesense
    image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd
    environment:
      - TYPESENSE_API_KEY=${TYPESENSE_API_KEY}
      - TYPESENSE_DATA_DIR=/data
      # remove this to get debug messages
      - GLOG_minloglevel=1
    volumes:
      - tsdata:/data
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

  immich-proxy:
    container_name: immich_proxy
    image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release}
    environment:
      # Make sure these values get passed through from the env file
      - IMMICH_SERVER_URL
      - IMMICH_WEB_URL
    ports:
      - 2283:8080
    depends_on:
      - immich-server
      - immich-web
    restart: always

volumes:
  pgdata:
  model-cache:
  tsdata:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/user/data/immich_photos

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
TYPESENSE_API_KEY=some-random-text
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. upload files via cli
2. it fails
3. upload again
4. it says complete
5. upload again
6. it says same amount of photos to upload

Additional information

No response

Originally created by @ioslife on GitHub (Oct 26, 2023). ### The bug I am attempting to bulk upload 28,778 files via the CLI. On initial run, it says it will upload all 28778. That failed and now it says it will upload 12,094. I run that and it says it uploaded 12094/28778, but if I run the immich upload again, it still says 12,094 to upload. <img width="600" alt="Screenshot 2023-10-26 at 8 39 06 AM" src="https://github.com/immich-app/immich/assets/5478660/6177acfe-25ab-4373-a97f-97df1fdfcb0f"> ### The OS that Immich Server is running on unraid ### Version of Immich Server v1.82.0 ### Version of Immich Mobile App v1.82.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: ["start.sh", "immich"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: ["start.sh", "microservices"] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database - typesense restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always immich-web: container_name: immich_web image: ghcr.io/immich-app/immich-web:${IMMICH_VERSION:-release} env_file: - .env restart: always typesense: container_name: immich_typesense image: typesense/typesense:0.24.1@sha256:9bcff2b829f12074426ca044b56160ca9d777a0c488303469143dd9f8259d4dd environment: - TYPESENSE_API_KEY=${TYPESENSE_API_KEY} - TYPESENSE_DATA_DIR=/data # remove this to get debug messages - GLOG_minloglevel=1 volumes: - tsdata:/data restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3 restart: always database: container_name: immich_postgres image: postgres:14-alpine@sha256:28407a9961e76f2d285dc6991e8e48893503cc3836a4755bbc2d40bcc272a441 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always immich-proxy: container_name: immich_proxy image: ghcr.io/immich-app/immich-proxy:${IMMICH_VERSION:-release} environment: # Make sure these values get passed through from the env file - IMMICH_SERVER_URL - IMMICH_WEB_URL ports: - 2283:8080 depends_on: - immich-server - immich-web restart: always volumes: pgdata: model-cache: tsdata: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/mnt/user/data/immich_photos # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords TYPESENSE_API_KEY=some-random-text DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps ```bash 1. upload files via cli 2. it fails 3. upload again 4. it says complete 5. upload again 6. it says same amount of photos to upload ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 02:12:35 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Oct 26, 2023):

12,094 is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server.

@alextran1502 commented on GitHub (Oct 26, 2023): `12,094` is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server.
OVERLORD commented 2026-02-05 02:12:40 +03:00
Author
Owner
Copy Link

@ioslife commented on GitHub (Oct 26, 2023):

12,094 is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server.

So does that mean that the other 16,684 successfully uploaded at some point?

@ioslife commented on GitHub (Oct 26, 2023): > `12,094` is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server. So does that mean that the other 16,684 successfully uploaded at some point?
OVERLORD commented 2026-02-05 02:12:43 +03:00
Author
Owner
Copy Link

@alextran1502 commented on GitHub (Oct 26, 2023):

12,094 is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server.

So does that mean that the other 16,684 successfully uploaded at some point?

Yes

@alextran1502 commented on GitHub (Oct 26, 2023): > > `12,094` is the number of duplicated assets that are already on the server. Since the CLI is stateless and that specific version of the CLI doesn't have client-side hashing, it will reupload the duplicated file and then get rejected by the server. > > > > So does that mean that the other 16,684 successfully uploaded at some point? Yes
OVERLORD commented 2026-02-05 02:12:50 +03:00
Author
Owner
Copy Link

@ioslife commented on GitHub (Oct 26, 2023):

Thanks Alex, you rock.

@ioslife commented on GitHub (Oct 26, 2023): Thanks Alex, you rock.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1518
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?