immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

Storage Template Migration job stuck at Waiting #7799

New Issue
Closed
opened 2026-02-05 13:18:44 +03:00 by OVERLORD · 2 comments
OVERLORD commented 2026-02-05 13:18:44 +03:00
Owner
Copy Link

Originally created by @firmanraharjo on GitHub (Nov 10, 2025).

I have searched the existing issues, both open and closed, to make sure this is not a duplicate report.

  • Yes

The bug

The Storage Template Migration job is stuck at waiting. I've tried clicking start multiple times, and it simply increments the number of jobs in the Waiting stage.

The OS that Immich Server is running on

Ubuntu 24.04.2 LTS

Version of Immich Server

v2.2.3

Version of Immich Mobile App

v2.2.3

Platform with the issue

  • Server
  • Web
  • Mobile

Device make and model

No response

Your docker-compose.yml content

immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - ${EXTERNAL_PATH}:/usr/src/app/external
    env_file:
      - .env
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.immich-server.entrypoints=web,websecure"
      - "traefik.http.routers.immich-server.rule=Host(`hostname.com`)"
      - "traefik.http.routers.immich-server.tls=true"
      - "traefik.http.routers.immich-server.tls.certresolver=production"
      - "traefik.http.routers.immich-server.middlewares=chain-basic@file"
    ports:
      - "4002:3001"
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends:
      file: hwaccel.ml.yml
      service: cuda
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:328fe6a5822256d065debb36617a8169dbfbd77b797c525288e465f56c1d392b
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=***
# The location where your database files are stored
DB_DATA_LOCATION=***

EXTERNAL_PATH=***

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=***

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=***

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=***

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=***
DB_DATABASE_NAME=***

Reproduction steps

  1. Launch immich container using docker-compose up
  2. Enable Storage Template Engine in the Admin settings
  3. Click Start on the Storage Template Migration job in the Jobs tab
    ...

Relevant log output

[Nest] 23  - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~3wbo4d2g] 

[Nest] 23  - 11/10/2025, 5:57:25 AM   DEBUG [Api:JobService~bfel52pg] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:25 AM   DEBUG [Api:LoggingInterceptor~bfel52pg] PUT /api/jobs/storageTemplateMigration 200 16.01ms 10.0.0.1

command=start force=false

[Nest] 23  - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~bfel52pg] 

[Nest] 23  - 11/10/2025, 5:57:25 AM   DEBUG [Api:JobService~03k2qjql] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:25 AM   DEBUG [Api:LoggingInterceptor~03k2qjql] PUT /api/jobs/storageTemplateMigration 200 11.34ms 10.0.0.1

command=start force=false

[Nest] 23  - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~03k2qjql] 

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:JobService~ya7hhei8] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:LoggingInterceptor~ya7hhei8] PUT /api/jobs/storageTemplateMigration 200 18.96ms 10.0.0.1

command=start force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~ya7hhei8] 

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:JobService~hcwewzr9] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:LoggingInterceptor~hcwewzr9] PUT /api/jobs/storageTemplateMigration 200 23.92ms 10.0.0.1

command=start force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~hcwewzr9] 

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:JobService~vpv8x5mm] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM   DEBUG [Api:LoggingInterceptor~vpv8x5mm] PUT /api/jobs/storageTemplateMigration 200 14.77ms 10.0.0.1

command=start force=false

[Nest] 23  - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~vpv8x5mm] 

[Nest] 23  - 11/10/2025, 5:57:27 AM   DEBUG [Api:JobService~1w54fuie] Handling command: queue=storageTemplateMigration,command=start,force=false

[Nest] 23  - 11/10/2025, 5:57:27 AM   DEBUG [Api:LoggingInterceptor~1w54fuie] PUT /api/jobs/storageTemplateMigration 200 6.25ms 10.0.0.1

command=start force=false

Additional information

Image

No response

Originally created by @firmanraharjo on GitHub (Nov 10, 2025). ### I have searched the existing issues, both open and closed, to make sure this is not a duplicate report. - [x] Yes ### The bug The Storage Template Migration job is stuck at waiting. I've tried clicking start multiple times, and it simply increments the number of jobs in the Waiting stage. ### The OS that Immich Server is running on Ubuntu 24.04.2 LTS ### Version of Immich Server v2.2.3 ### Version of Immich Mobile App v2.2.3 ### Platform with the issue - [x] Server - [ ] Web - [ ] Mobile ### Device make and model _No response_ ### Your docker-compose.yml content ```YAML immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} extends: file: hwaccel.transcoding.yml service: nvenc volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - ${EXTERNAL_PATH}:/usr/src/app/external env_file: - .env labels: - "traefik.enable=true" - "traefik.http.routers.immich-server.entrypoints=web,websecure" - "traefik.http.routers.immich-server.rule=Host(`hostname.com`)" - "traefik.http.routers.immich-server.tls=true" - "traefik.http.routers.immich-server.tls.certresolver=production" - "traefik.http.routers.immich-server.middlewares=chain-basic@file" ports: - "4002:3001" depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda extends: file: hwaccel.ml.yml service: cuda volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: docker.io/redis:6.2-alpine@sha256:328fe6a5822256d065debb36617a8169dbfbd77b797c525288e465f56c1d392b healthcheck: test: redis-cli ping || exit 1 restart: always database: container_name: immich_postgres image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' volumes: - ${DB_DATA_LOCATION}:/var/lib/postgresql/data healthcheck: test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 5m start_interval: 30s start_period: 5m command: ["postgres", "-c" ,"shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"] restart: always ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=*** # The location where your database files are stored DB_DATA_LOCATION=*** EXTERNAL_PATH=*** # To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List # TZ=*** # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=*** # Connection secret for postgres. You should change it to a random password DB_PASSWORD=*** # The values below this line do not need to be changed ################################################################################### DB_USERNAME=*** DB_DATABASE_NAME=*** ``` ### Reproduction steps 1. Launch immich container using docker-compose up 2. Enable Storage Template Engine in the Admin settings 3. Click Start on the Storage Template Migration job in the Jobs tab ... ### Relevant log output ```shell [Nest] 23 - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~3wbo4d2g] [Nest] 23 - 11/10/2025, 5:57:25 AM DEBUG [Api:JobService~bfel52pg] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:25 AM DEBUG [Api:LoggingInterceptor~bfel52pg] PUT /api/jobs/storageTemplateMigration 200 16.01ms 10.0.0.1 command=start force=false [Nest] 23 - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~bfel52pg] [Nest] 23 - 11/10/2025, 5:57:25 AM DEBUG [Api:JobService~03k2qjql] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:25 AM DEBUG [Api:LoggingInterceptor~03k2qjql] PUT /api/jobs/storageTemplateMigration 200 11.34ms 10.0.0.1 command=start force=false [Nest] 23 - 11/10/2025, 5:57:25 AM VERBOSE [Api:LoggingInterceptor~03k2qjql] [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:JobService~ya7hhei8] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:LoggingInterceptor~ya7hhei8] PUT /api/jobs/storageTemplateMigration 200 18.96ms 10.0.0.1 command=start force=false [Nest] 23 - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~ya7hhei8] [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:JobService~hcwewzr9] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:LoggingInterceptor~hcwewzr9] PUT /api/jobs/storageTemplateMigration 200 23.92ms 10.0.0.1 command=start force=false [Nest] 23 - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~hcwewzr9] [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:JobService~vpv8x5mm] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:26 AM DEBUG [Api:LoggingInterceptor~vpv8x5mm] PUT /api/jobs/storageTemplateMigration 200 14.77ms 10.0.0.1 command=start force=false [Nest] 23 - 11/10/2025, 5:57:26 AM VERBOSE [Api:LoggingInterceptor~vpv8x5mm] [Nest] 23 - 11/10/2025, 5:57:27 AM DEBUG [Api:JobService~1w54fuie] Handling command: queue=storageTemplateMigration,command=start,force=false [Nest] 23 - 11/10/2025, 5:57:27 AM DEBUG [Api:LoggingInterceptor~1w54fuie] PUT /api/jobs/storageTemplateMigration 200 6.25ms 10.0.0.1 command=start force=false ``` ### Additional information <img width="787" height="211" alt="Image" src="https://github.com/user-attachments/assets/a621268f-7afd-44b3-9e5a-ad3e39b7b476" /> _No response_
OVERLORD commented 2026-02-05 13:18:47 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Nov 10, 2025):

Do you use any remote workers? Can you try restarting the stack?

@bo0tzz commented on GitHub (Nov 10, 2025): Do you use any remote workers? Can you try restarting the stack?
OVERLORD commented 2026-02-05 13:18:48 +03:00
Author
Owner
Copy Link

@firmanraharjo commented on GitHub (Nov 11, 2025):

Nope, all the containers are on the same machine. But I did try restarting the stack a couple of times and after the 2nd restart it ended up working.

@firmanraharjo commented on GitHub (Nov 11, 2025): Nope, all the containers are on the same machine. But I did try restarting the stack a couple of times and after the 2nd restart it ended up working.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#7799
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?