immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

immich-machine-learning keeps restarting after upgrade to 1.117 #4483

New Issue
Closed
opened 2026-02-05 10:35:14 +03:00 by OVERLORD · 7 comments
OVERLORD commented 2026-02-05 10:35:14 +03:00
Owner
Copy Link

Originally created by @smalecwks on GitHub (Oct 6, 2024).

The bug

Hi,

After server upgrade to 1.117 machine_learning keeps restarting, one additional external library in docer-compose, upload location moved to addtinal drive.

After rollback to 1.116 everything works fine. System proxmox, updated.

6711a859daed ghcr.io/immich-app/immich-machine-learning:release "tini -- ./start.sh" About a minute ago Restarting (1) 4 seconds ago immich_machine_learning

Logs for container:

IndentationError: unindent does not match any outer indentation level
Traceback (most recent call last):
File "/opt/venv/bin/gunicorn", line 5, in
from gunicorn.app.wsgiapp import run
File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in
from gunicorn.app.base import Application
File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in
from gunicorn import util
File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in
import email.utils
File "/usr/local/lib/python3.11/email/utils.py", line 28, in
import random
File "/usr/local/lib/python3.11/random.py", line 381
$ # pick an element in x[:i+1] with which to exchange x[i]

The OS that Immich Server is running on

Proxmox (debain)

Version of Immich Server

1.117.0

Version of Immich Mobile App

1.117.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /NAS/Fotki:/NAS/Fotki
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792
    healthcheck:
      test: redis-cli ping || exit 1
    restart: always

  database:
    container_name: immich_postgres
    image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    healthcheck:
      test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1
      interval: 5m
      start_interval: 30s
      start_period: 5m
    command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"]
    restart: always

volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/NAS/immich
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
#IMMICH_VERSION="v1.116.0"

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=postgres

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

Keeps restarting after start

Relevant log output

IndentationError: unindent does not match any outer indentation level
Traceback (most recent call last):
  File "/opt/venv/bin/gunicorn", line 5, in <module>
    from gunicorn.app.wsgiapp import run
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in <module>
    from gunicorn.app.base import Application
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in <module>
    from gunicorn import util
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in <module>
    import email.utils
  File "/usr/local/lib/python3.11/email/utils.py", line 28, in <module>
    import random
  File "/usr/local/lib/python3.11/random.py", line 381
    $        # pick an element in x[:i+1] with which to exchange x[i]
                                                                     ^
IndentationError: unindent does not match any outer indentation level
Traceback (most recent call last):
  File "/opt/venv/bin/gunicorn", line 5, in <module>
    from gunicorn.app.wsgiapp import run
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in <module>
    from gunicorn.app.base import Application
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in <module>
    from gunicorn import util
  File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in <module>
    import email.utils
  File "/usr/local/lib/python3.11/email/utils.py", line 28, in <module>
    import random
  File "/usr/local/lib/python3.11/random.py", line 381
    $        # pick an element in x[:i+1] with which to exchange x[i]
                                                                     ^
IndentationError: unindent does not match any outer indentation level

Additional information

No response

Originally created by @smalecwks on GitHub (Oct 6, 2024). ### The bug Hi, After server upgrade to 1.117 machine_learning keeps restarting, one additional external library in docer-compose, upload location moved to addtinal drive. After rollback to 1.116 everything works fine. System proxmox, updated. 6711a859daed ghcr.io/immich-app/immich-machine-learning:release "tini -- ./start.sh" About a minute ago Restarting (1) 4 seconds ago immich_machine_learning Logs for container: IndentationError: unindent does not match any outer indentation level Traceback (most recent call last): File "/opt/venv/bin/gunicorn", line 5, in <module> from gunicorn.app.wsgiapp import run File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in <module> from gunicorn.app.base import Application File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in <module> from gunicorn import util File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in <module> import email.utils File "/usr/local/lib/python3.11/email/utils.py", line 28, in <module> import random File "/usr/local/lib/python3.11/random.py", line 381 $ # pick an element in x[:i+1] with which to exchange x[i] ### The OS that Immich Server is running on Proxmox (debain) ### Version of Immich Server 1.117.0 ### Version of Immich Mobile App 1.117.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.transcoding.yml # service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding volumes: # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /NAS/Fotki:/NAS/Fotki env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always healthcheck: disable: false immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} # extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cpu # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable volumes: - model-cache:/cache env_file: - .env restart: always healthcheck: disable: false redis: container_name: immich_redis image: docker.io/redis:6.2-alpine@sha256:2d1463258f2764328496376f5d965f20c6a67f66ea2b06dc42af351f75248792 healthcheck: test: redis-cli ping || exit 1 restart: always database: container_name: immich_postgres image: docker.io/tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} POSTGRES_INITDB_ARGS: '--data-checksums' volumes: # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file - ${DB_DATA_LOCATION}:/var/lib/postgresql/data healthcheck: test: pg_isready --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' || exit 1; Chksum="$$(psql --dbname='${DB_DATABASE_NAME}' --username='${DB_USERNAME}' --tuples-only --no-align --command='SELECT COALESCE(SUM(checksum_failures), 0) FROM pg_stat_database')"; echo "checksum failure count is $$Chksum"; [ "$$Chksum" = '0' ] || exit 1 interval: 5m start_interval: 30s start_period: 5m command: ["postgres", "-c", "shared_preload_libraries=vectors.so", "-c", 'search_path="$$user", public, vectors', "-c", "logging_collector=on", "-c", "max_wal_size=2GB", "-c", "shared_buffers=512MB", "-c", "wal_compression=on"] restart: always volumes: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/NAS/immich # The location where your database files are stored DB_DATA_LOCATION=./postgres # To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List # TZ=Etc/UTC # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release #IMMICH_VERSION="v1.116.0" # Connection secret for postgres. You should change it to a random password # Please use only the characters `A-Za-z0-9`, without special characters or spaces DB_PASSWORD=postgres # The values below this line do not need to be changed ################################################################################### DB_USERNAME=postgres DB_DATABASE_NAME=immich ``` ### Reproduction steps Keeps restarting after start ### Relevant log output ```shell IndentationError: unindent does not match any outer indentation level Traceback (most recent call last): File "/opt/venv/bin/gunicorn", line 5, in <module> from gunicorn.app.wsgiapp import run File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in <module> from gunicorn.app.base import Application File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in <module> from gunicorn import util File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in <module> import email.utils File "/usr/local/lib/python3.11/email/utils.py", line 28, in <module> import random File "/usr/local/lib/python3.11/random.py", line 381 $ # pick an element in x[:i+1] with which to exchange x[i] ^ IndentationError: unindent does not match any outer indentation level Traceback (most recent call last): File "/opt/venv/bin/gunicorn", line 5, in <module> from gunicorn.app.wsgiapp import run File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/wsgiapp.py", line 8, in <module> from gunicorn.app.base import Application File "/opt/venv/lib/python3.11/site-packages/gunicorn/app/base.py", line 10, in <module> from gunicorn import util File "/opt/venv/lib/python3.11/site-packages/gunicorn/util.py", line 5, in <module> import email.utils File "/usr/local/lib/python3.11/email/utils.py", line 28, in <module> import random File "/usr/local/lib/python3.11/random.py", line 381 $ # pick an element in x[:i+1] with which to exchange x[i] ^ IndentationError: unindent does not match any outer indentation level ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 10:35:20 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Oct 6, 2024):

Can you try deleting the ML image and pulling it again?

@bo0tzz commented on GitHub (Oct 6, 2024): Can you try deleting the ML image and pulling it again?
OVERLORD commented 2026-02-05 10:35:22 +03:00
Author
Owner
Copy Link

@zeusiot commented on GitHub (Oct 6, 2024):

I'm having is restarting after upgrade do 1.117 in the same way the logs showing:

     Address family not supported by protocol

[10/06/24 10:58:53] ERROR connection to ('::', 3003) failed: [Errno 97]
Address family not supported by protocol
[10/06/24 10:58:54] ERROR connection to ('::', 3003) failed: [Errno 97]
Address family not supported by protocol
[10/06/24 10:58:55] ERROR Can't connect to ('::', 3003)

@zeusiot commented on GitHub (Oct 6, 2024): I'm having is restarting after upgrade do 1.117 in the same way the logs showing: Address family not supported by protocol [10/06/24 10:58:53] ERROR connection to ('::', 3003) failed: [Errno 97] Address family not supported by protocol [10/06/24 10:58:54] ERROR connection to ('::', 3003) failed: [Errno 97] Address family not supported by protocol [10/06/24 10:58:55] ERROR Can't connect to ('::', 3003)
OVERLORD commented 2026-02-05 10:35:26 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Oct 6, 2024):

@zeusiot that's a different error. Please open a QA thread at https://github.com/immich-app/immich/discussions/categories/q-a (or on discord).

@bo0tzz commented on GitHub (Oct 6, 2024): @zeusiot that's a different error. Please open a QA thread at https://github.com/immich-app/immich/discussions/categories/q-a (or on discord).
OVERLORD commented 2026-02-05 10:35:30 +03:00
Author
Owner
Copy Link

@smalecwks commented on GitHub (Oct 6, 2024):

How to delete just ml image?

Regards,
Lukasz

⁣
​

W dniu 6 paź 2024, 12:00, o 12:00, użytkownik bo0tzz @.***> napisał:

Can you try deleting the ML image and pulling it again?

--
Reply to this email directly or view it on GitHub:
https://github.com/immich-app/immich/issues/13220#issuecomment-2395374983
You are receiving this because you authored the thread.

Message ID: @.***>

@smalecwks commented on GitHub (Oct 6, 2024): How to delete just ml image? Regards, Lukasz ⁣ ​ W dniu 6 paź 2024, 12:00, o 12:00, użytkownik bo0tzz ***@***.***> napisał: >Can you try deleting the ML image and pulling it again? > >-- >Reply to this email directly or view it on GitHub: >https://github.com/immich-app/immich/issues/13220#issuecomment-2395374983 >You are receiving this because you authored the thread. > >Message ID: ***@***.***>
OVERLORD commented 2026-02-05 10:35:30 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Oct 6, 2024):

Stop the containers, then docker image rm ghcr.io/immich-app/immich-machine-learning:release.

@bo0tzz commented on GitHub (Oct 6, 2024): Stop the containers, then `docker image rm ghcr.io/immich-app/immich-machine-learning:release`.
OVERLORD commented 2026-02-05 10:35:31 +03:00
Author
Owner
Copy Link

@smalecwks commented on GitHub (Oct 6, 2024):

Ok, it seems this solved the problem. Thank you for help.

Regards,
Lukasz

⁣
​

W dniu 6 paź 2024, 15:18, o 15:18, użytkownik bo0tzz @.***> napisał:

Stop the containers, then docker image rm ghcr.io/immich-app/immich-machine-learning:release.

--
Reply to this email directly or view it on GitHub:
https://github.com/immich-app/immich/issues/13220#issuecomment-2395437769
You are receiving this because you authored the thread.

Message ID: @.***>

@smalecwks commented on GitHub (Oct 6, 2024): Ok, it seems this solved the problem. Thank you for help. Regards, Lukasz ⁣ ​ W dniu 6 paź 2024, 15:18, o 15:18, użytkownik bo0tzz ***@***.***> napisał: >Stop the containers, then `docker image rm >ghcr.io/immich-app/immich-machine-learning:release`. > >-- >Reply to this email directly or view it on GitHub: >https://github.com/immich-app/immich/issues/13220#issuecomment-2395437769 >You are receiving this because you authored the thread. > >Message ID: ***@***.***>
OVERLORD commented 2026-02-05 10:35:35 +03:00
Author
Owner
Copy Link

@smalecwks commented on GitHub (Oct 6, 2024):

Ok, it seems this solved the problem. Thank you for help.

Regards,
Lukasz

⁣
​

W dniu 6 paź 2024, 15:18, o 15:18, użytkownik bo0tzz @.***> napisał:

Stop the containers, then docker image rm ghcr.io/immich-app/immich-machine-learning:release.

--
Reply to this email directly or view it on GitHub:
https://github.com/immich-app/immich/issues/13220#issuecomment-2395437769
You are receiving this because you authored the thread.

Message ID: @.***>

@smalecwks commented on GitHub (Oct 6, 2024): Ok, it seems this solved the problem. Thank you for help. Regards, Lukasz ⁣ ​ W dniu 6 paź 2024, 15:18, o 15:18, użytkownik bo0tzz ***@***.***> napisał: >Stop the containers, then `docker image rm >ghcr.io/immich-app/immich-machine-learning:release`. > >-- >Reply to this email directly or view it on GitHub: >https://github.com/immich-app/immich/issues/13220#issuecomment-2395437769 >You are receiving this because you authored the thread. > >Message ID: ***@***.***>
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4483
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?