Upgrade from 1.106.4 -> 1.107.0, immich can't start #3774

Closed
opened 2026-02-05 09:13:33 +03:00 by OVERLORD · 0 comments
Owner

Originally created by @bbklopfer on GitHub (Jul 2, 2024).

The bug

Hi, immich was working with 1.106.4. I upgraded to 1.107.0 (docker compose pull && docker compose down && docker compose up). It now fails with some entered unreachable code error, and gets stuck in a restart loop. docker compose up output attached.

The OS that Immich Server is running on

Debian Bookworm on Orange Pi 5+

Version of Immich Server

v1.107.0

Version of Immich Mobile App

N/A

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

# version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always


  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:b6124ab2e45cc332e16398022a411d7e37181f21ff7874835e0180f56a09e82a
    restart: always

  database:
    container_name: immich_postgres
#    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# Having trouble with my Samsung showing incorrect dates.
TZ=America/Los_Angeles
# The location where your uploaded files are stored
UPLOAD_LOCATION=./library

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=*****

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Update from 1.106.4 to 1.107.0
2. Start immich the usual way

Relevant log output

Container immich_postgres  Created
 Container immich_machine_learning  Created
 Container immich_redis  Created
 Container immich_server  Created
Attaching to immich_machine_learning, immich_postgres, immich_redis, immich_server
immich_redis             | 1:C 02 Jul 2024 17:19:41.537 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
immich_redis             | 1:C 02 Jul 2024 17:19:41.537 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started
immich_redis             | 1:C 02 Jul 2024 17:19:41.537 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
immich_redis             | 1:M 02 Jul 2024 17:19:41.540 * monotonic clock: POSIX clock_gettime
immich_redis             | 1:M 02 Jul 2024 17:19:41.542 * Running mode=standalone, port=6379.
immich_redis             | 1:M 02 Jul 2024 17:19:41.542 # Server initialized
immich_redis             | 1:M 02 Jul 2024 17:19:41.549 * Loading RDB produced by version 6.2.14
immich_redis             | 1:M 02 Jul 2024 17:19:41.550 * RDB age 54 seconds
immich_redis             | 1:M 02 Jul 2024 17:19:41.550 * RDB memory usage when created 0.81 Mb
immich_redis             | 1:M 02 Jul 2024 17:19:41.550 # Done loading RDB, keys loaded: 14, keys expired: 0.
immich_redis             | 1:M 02 Jul 2024 17:19:41.550 * DB loaded from disk: 0.000 seconds
immich_redis             | 1:M 02 Jul 2024 17:19:41.550 * Ready to accept connections
immich_postgres          | 
immich_postgres          | PostgreSQL Database directory appears to contain a database; Skipping initialization
immich_postgres          | 
immich_postgres          | 2024-07-02 17:19:41.718 UTC [1] LOG:  starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
immich_postgres          | 2024-07-02 17:19:41.718 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
immich_postgres          | 2024-07-02 17:19:41.718 UTC [1] LOG:  listening on IPv6 address "::", port 5432
immich_postgres          | 2024-07-02 17:19:41.721 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
immich_postgres          | 2024-07-02 17:19:41.727 UTC [27] LOG:  database system was shut down at 2024-07-02 17:18:47 UTC
immich_postgres          | 2024-07-02 17:19:41.746 UTC [1] LOG:  database system is ready to accept connections
immich_server            | Detected CPU Cores: 8
immich_machine_learning  | [07/02/24 10:19:43] INFO     Starting gunicorn 22.0.0                           
immich_machine_learning  | [07/02/24 10:19:43] INFO     Listening at: http://[::]:3003 (9)                 
immich_machine_learning  | [07/02/24 10:19:43] INFO     Using worker: app.config.CustomUvicornWorker       
immich_machine_learning  | [07/02/24 10:19:43] INFO     Booting worker with pid: 10                        
immich_server            | Starting api worker
immich_server            | Starting microservices worker
immich_machine_learning  | [07/02/24 10:19:50] INFO     Started server process [10]                        
immich_machine_learning  | [07/02/24 10:19:50] INFO     Waiting for application startup.                   
immich_machine_learning  | [07/02/24 10:19:50] INFO     Created in-memory cache with unloading after 300s  
immich_machine_learning  |                              of inactivity.                                     
immich_machine_learning  | [07/02/24 10:19:50] INFO     Initialized request thread pool with 8 threads.    
immich_machine_learning  | [07/02/24 10:19:50] INFO     Application startup complete.                      
immich_server            | [Nest] 7  - 07/02/2024, 10:19:51 AM     LOG [Microservices:EventRepository] Initialized websocket server
immich_server            | [Nest] 17  - 07/02/2024, 10:19:52 AM     LOG [Api:EventRepository] Initialized websocket server
immich_postgres          | 2024-07-02 17:19:52.687 UTC [37] ERROR:  internal error: entered unreachable code
immich_postgres          | 2024-07-02 17:19:52.687 UTC [37] STATEMENT:  
immich_postgres          | 	            CREATE INDEX face_index ON face_search
immich_postgres          | 	            USING hnsw (embedding vector_cosine_ops)
immich_postgres          | 	            WITH (ef_construction = 300, m = 16)
immich_server            | Migration "AddFaceSearchRelation1718486162779" failed, error: internal error: entered unreachable code
immich_server            | QueryFailedError: internal error: entered unreachable code
immich_server            |     at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19)
immich_server            |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_server            |     at async AddFaceSearchRelation1718486162779.up (/usr/src/app/dist/migrations/1718486162779-AddFaceSearchRelation.js:23:9)
immich_server            |     at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
immich_server            |     at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:265:35)
immich_server            |     at async DatabaseRepository.runMigrations (/usr/src/app/dist/repositories/database.repository.js:169:9)
immich_server            |     at async /usr/src/app/dist/services/database.service.js:134:17
immich_server            |     at async /usr/src/app/dist/repositories/database.repository.js:177:23 {
immich_server            |   query: '\n' +
immich_server            |     '            CREATE INDEX face_index ON face_search\n' +
immich_server            |     '            USING hnsw (embedding vector_cosine_ops)\n' +
immich_server            |     '            WITH (ef_construction = 300, m = 16)',
immich_server            |   parameters: undefined,
immich_server            |   driverError: error: internal error: entered unreachable code
immich_server            |       at /usr/src/app/node_modules/pg/lib/client.js:526:17
immich_server            |       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_server            |       at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25)
immich_server            |       at async AddFaceSearchRelation1718486162779.up (/usr/src/app/dist/migrations/1718486162779-AddFaceSearchRelation.js:23:9)
immich_server            |       at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17)
immich_server            |       at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:265:35)
immich_server            |       at async DatabaseRepository.runMigrations (/usr/src/app/dist/repositories/database.repository.js:169:9)
immich_server            |       at async /usr/src/app/dist/services/database.service.js:134:17
immich_server            |       at async /usr/src/app/dist/repositories/database.repository.js:177:23 {
immich_server            |     length: 82,
immich_server            |     severity: 'ERROR',
immich_server            |     code: 'XX000',
immich_server            |     detail: undefined,
immich_server            |     hint: undefined,
immich_server            |     position: undefined,
immich_server            |     internalPosition: undefined,
immich_server            |     internalQuery: undefined,
immich_server            |     where: undefined,
immich_server            |     schema: undefined,
immich_server            |     table: undefined,
immich_server            |     column: undefined,
immich_server            |     dataType: undefined,
immich_server            |     constraint: undefined,
immich_server            |     file: 'utils.rs',
immich_server            |     line: '21',
immich_server            |     routine: undefined
immich_server            |   },
immich_server            |   length: 82,
immich_server            |   severity: 'ERROR',
immich_server            |   code: 'XX000',
immich_server            |   detail: undefined,
immich_server            |   hint: undefined,
immich_server            |   position: undefined,
immich_server            |   internalPosition: undefined,
immich_server            |   internalQuery: undefined,
immich_server            |   where: undefined,
immich_server            |   schema: undefined,
immich_server            |   table: undefined,
immich_server            |   column: undefined,
immich_server            |   dataType: undefined,
immich_server            |   constraint: undefined,
immich_server            |   file: 'utils.rs',
immich_server            |   line: '21',
immich_server            |   routine: undefined
immich_server            | }
immich_server            | microservices worker exited with code 1
immich_postgres          | 2024-07-02 17:19:53.556 UTC [38] LOG:  unexpected EOF on client connection with an open transaction

immich_server exited with code 0
immich_server            | Detected CPU Cores: 8

Additional information

No response

Originally created by @bbklopfer on GitHub (Jul 2, 2024). ### The bug Hi, immich was working with 1.106.4. I upgraded to 1.107.0 (`docker compose pull && docker compose down && docker compose up`). It now fails with some `entered unreachable code` error, and gets stuck in a restart loop. `docker compose up` output attached. ### The OS that Immich Server is running on Debian Bookworm on Orange Pi 5+ ### Version of Immich Server v1.107.0 ### Version of Immich Mobile App N/A ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML # version: "3.8" # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:b6124ab2e45cc332e16398022a411d7e37181f21ff7874835e0180f56a09e82a restart: always database: container_name: immich_postgres # image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always volumes: pgdata: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # Having trouble with my Samsung showing incorrect dates. TZ=America/Los_Angeles # The location where your uploaded files are stored UPLOAD_LOCATION=./library # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password DB_PASSWORD=***** # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis ``` ### Reproduction steps ```bash 1. Update from 1.106.4 to 1.107.0 2. Start immich the usual way ``` ### Relevant log output ```shell Container immich_postgres Created Container immich_machine_learning Created Container immich_redis Created Container immich_server Created Attaching to immich_machine_learning, immich_postgres, immich_redis, immich_server immich_redis | 1:C 02 Jul 2024 17:19:41.537 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo immich_redis | 1:C 02 Jul 2024 17:19:41.537 # Redis version=6.2.14, bits=64, commit=00000000, modified=0, pid=1, just started immich_redis | 1:C 02 Jul 2024 17:19:41.537 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf immich_redis | 1:M 02 Jul 2024 17:19:41.540 * monotonic clock: POSIX clock_gettime immich_redis | 1:M 02 Jul 2024 17:19:41.542 * Running mode=standalone, port=6379. immich_redis | 1:M 02 Jul 2024 17:19:41.542 # Server initialized immich_redis | 1:M 02 Jul 2024 17:19:41.549 * Loading RDB produced by version 6.2.14 immich_redis | 1:M 02 Jul 2024 17:19:41.550 * RDB age 54 seconds immich_redis | 1:M 02 Jul 2024 17:19:41.550 * RDB memory usage when created 0.81 Mb immich_redis | 1:M 02 Jul 2024 17:19:41.550 # Done loading RDB, keys loaded: 14, keys expired: 0. immich_redis | 1:M 02 Jul 2024 17:19:41.550 * DB loaded from disk: 0.000 seconds immich_redis | 1:M 02 Jul 2024 17:19:41.550 * Ready to accept connections immich_postgres | immich_postgres | PostgreSQL Database directory appears to contain a database; Skipping initialization immich_postgres | immich_postgres | 2024-07-02 17:19:41.718 UTC [1] LOG: starting PostgreSQL 14.10 (Debian 14.10-1.pgdg120+1) on aarch64-unknown-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit immich_postgres | 2024-07-02 17:19:41.718 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432 immich_postgres | 2024-07-02 17:19:41.718 UTC [1] LOG: listening on IPv6 address "::", port 5432 immich_postgres | 2024-07-02 17:19:41.721 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432" immich_postgres | 2024-07-02 17:19:41.727 UTC [27] LOG: database system was shut down at 2024-07-02 17:18:47 UTC immich_postgres | 2024-07-02 17:19:41.746 UTC [1] LOG: database system is ready to accept connections immich_server | Detected CPU Cores: 8 immich_machine_learning | [07/02/24 10:19:43] INFO  Starting gunicorn 22.0.0 immich_machine_learning | [07/02/24 10:19:43] INFO  Listening at: http://[::]:3003 (9) immich_machine_learning | [07/02/24 10:19:43] INFO  Using worker: app.config.CustomUvicornWorker immich_machine_learning | [07/02/24 10:19:43] INFO  Booting worker with pid: 10 immich_server | Starting api worker immich_server | Starting microservices worker immich_machine_learning | [07/02/24 10:19:50] INFO  Started server process [10] immich_machine_learning | [07/02/24 10:19:50] INFO  Waiting for application startup. immich_machine_learning | [07/02/24 10:19:50] INFO  Created in-memory cache with unloading after 300s immich_machine_learning |   of inactivity. immich_machine_learning | [07/02/24 10:19:50] INFO  Initialized request thread pool with 8 threads. immich_machine_learning | [07/02/24 10:19:50] INFO  Application startup complete. immich_server | [Nest] 7 - 07/02/2024, 10:19:51 AM  LOG [Microservices:EventRepository] Initialized websocket server immich_server | [Nest] 17 - 07/02/2024, 10:19:52 AM  LOG [Api:EventRepository] Initialized websocket server immich_postgres | 2024-07-02 17:19:52.687 UTC [37] ERROR: internal error: entered unreachable code immich_postgres | 2024-07-02 17:19:52.687 UTC [37] STATEMENT: immich_postgres | CREATE INDEX face_index ON face_search immich_postgres | USING hnsw (embedding vector_cosine_ops) immich_postgres | WITH (ef_construction = 300, m = 16) immich_server | Migration "AddFaceSearchRelation1718486162779" failed, error: internal error: entered unreachable code immich_server | QueryFailedError: internal error: entered unreachable code immich_server | at PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:219:19) immich_server | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_server | at async AddFaceSearchRelation1718486162779.up (/usr/src/app/dist/migrations/1718486162779-AddFaceSearchRelation.js:23:9) immich_server | at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17) immich_server | at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:265:35) immich_server | at async DatabaseRepository.runMigrations (/usr/src/app/dist/repositories/database.repository.js:169:9) immich_server | at async /usr/src/app/dist/services/database.service.js:134:17 immich_server | at async /usr/src/app/dist/repositories/database.repository.js:177:23 { immich_server | query: '\n' + immich_server | ' CREATE INDEX face_index ON face_search\n' + immich_server | ' USING hnsw (embedding vector_cosine_ops)\n' + immich_server | ' WITH (ef_construction = 300, m = 16)', immich_server | parameters: undefined, immich_server | driverError: error: internal error: entered unreachable code immich_server | at /usr/src/app/node_modules/pg/lib/client.js:526:17 immich_server | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_server | at async PostgresQueryRunner.query (/usr/src/app/node_modules/typeorm/driver/postgres/PostgresQueryRunner.js:184:25) immich_server | at async AddFaceSearchRelation1718486162779.up (/usr/src/app/dist/migrations/1718486162779-AddFaceSearchRelation.js:23:9) immich_server | at async MigrationExecutor.executePendingMigrations (/usr/src/app/node_modules/typeorm/migration/MigrationExecutor.js:225:17) immich_server | at async DataSource.runMigrations (/usr/src/app/node_modules/typeorm/data-source/DataSource.js:265:35) immich_server | at async DatabaseRepository.runMigrations (/usr/src/app/dist/repositories/database.repository.js:169:9) immich_server | at async /usr/src/app/dist/services/database.service.js:134:17 immich_server | at async /usr/src/app/dist/repositories/database.repository.js:177:23 { immich_server | length: 82, immich_server | severity: 'ERROR', immich_server | code: 'XX000', immich_server | detail: undefined, immich_server | hint: undefined, immich_server | position: undefined, immich_server | internalPosition: undefined, immich_server | internalQuery: undefined, immich_server | where: undefined, immich_server | schema: undefined, immich_server | table: undefined, immich_server | column: undefined, immich_server | dataType: undefined, immich_server | constraint: undefined, immich_server | file: 'utils.rs', immich_server | line: '21', immich_server | routine: undefined immich_server | }, immich_server | length: 82, immich_server | severity: 'ERROR', immich_server | code: 'XX000', immich_server | detail: undefined, immich_server | hint: undefined, immich_server | position: undefined, immich_server | internalPosition: undefined, immich_server | internalQuery: undefined, immich_server | where: undefined, immich_server | schema: undefined, immich_server | table: undefined, immich_server | column: undefined, immich_server | dataType: undefined, immich_server | constraint: undefined, immich_server | file: 'utils.rs', immich_server | line: '21', immich_server | routine: undefined immich_server | } immich_server | microservices worker exited with code 1 immich_postgres | 2024-07-02 17:19:53.556 UTC [38] LOG: unexpected EOF on client connection with an open transaction immich_server exited with code 0 immich_server | Detected CPU Cores: 8 ``` ### Additional information _No response_
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#3774