[BUG] Container immich-machine-learning keep crashing #970

Closed
opened 2026-02-04 23:44:48 +03:00 by OVERLORD · 3 comments
Owner

Originally created by @jpoggi on GitHub (Jun 19, 2023).

Hi,

Since a few updates one container on the stack keep crashing in a loop with this error message:

immich-machine-learning  | INFO:     Started server process [1]
immich-machine-learning  | INFO:     Waiting for application startup.
immich-machine-learning  | Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration.
immich-machine-learning exited with code 0

Can you point me the right direction on how to debug this ?

Thanks.

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.62.0

Version of Immich Mobile App

v1.62.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

services:
  redis.server.immich:
    container_name: redis.server.immich
    image: redis:7
    restart: unless-stopped
    mem_limit: 512m
    volumes:
      - /srv/immich/redis_01:/data
    healthcheck:
      test: ["CMD", "redis-cli", "--raw", "incr", "ping"]
      interval: 60s
      retries: 5
      start_period: 10s
      timeout: 30s

  postgres.server.immich:
    container_name: postgres.server.immich
    image: postgres:15
    restart: unless-stopped
    mem_limit: 512m
    environment:
      POSTGRES_DB: immich
      POSTGRES_USER: immich
      POSTGRES_PASSWORD: $DB_PASSWORD
      PG_DATA: /var/lib/postgresql/data
    volumes:
      - /srv/immich/db_01:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD", "pg_isready", "-q", "-d", "immich", "-U", "immich"]
      interval: 60s
      retries: 5
      start_period: 10s
      timeout: 30s

  immich-server:
    container_name: immich-server
    image: ghcr.io/immich-app/immich-server:v1.62.0
    entrypoint: ["/bin/sh", "./start-server.sh"]
    restart: unless-stopped
    mem_limit: 512m
    volumes:
      - /srv/immich/vol_01:/usr/src/app/upload
    environment:
      TYPESENSE_API_KEY: $TYPESENSE_API_KEY
      DB_HOSTNAME: postgres.server.immich
      DB_DATABASE_NAME: immich
      DB_USERNAME: immich
      DB_PASSWORD: $DB_PASSWORD
      REDIS_HOSTNAME: redis.server.immich
    depends_on:
      - redis.server.immich
      - postgres.server.immich

  immich-microservices:
    container_name: immich-microservices
    image: ghcr.io/immich-app/immich-server:v1.62.0
    restart: unless-stopped
    mem_limit: 1024m
    entrypoint: ["/bin/sh", "./start-microservices.sh"]
    volumes:
      - /srv/immich/vol_01:/usr/src/app/upload
    environment:
      TYPESENSE_API_KEY: $TYPESENSE_API_KEY
      DB_HOSTNAME: postgres.server.immich
      DB_DATABASE_NAME: immich
      DB_USERNAME: immich
      DB_PASSWORD: $DB_PASSWORD
      REDIS_HOSTNAME: redis.server.immich
    depends_on:
      - redis.server.immich
      - postgres.server.immich

  immich-machine-learning:
    container_name: immich-machine-learning
    image: ghcr.io/immich-app/immich-machine-learning:v1.62.0
    restart: unless-stopped
    mem_limit: 512m
    volumes:
      - /srv/immich/vol_01:/usr/src/app/upload
      - /srv/immich/vol_02:/cache

  typesense:
    container_name: typesense
    image: typesense/typesense:0.24.0
    restart: always
    environment:
      TYPESENSE_API_KEY: $TYPESENSE_API_KEY
      TYPESENSE_DATA_DIR: /data
    logging:
      driver: none
    volumes:
      - /srv/immich/vol_03:/data

  immich-web:
    container_name: immich-web
    image: ghcr.io/immich-app/immich-web:v1.62.0
    restart: unless-stopped
    mem_limit: 512m
    environment:
      PUBLIC_LOGIN_PAGE_MESSAGE: Welcome on Immich.
    entrypoint: ["/bin/sh", "./entrypoint.sh"]

  immich-proxy:
    container_name: immich-proxy
    image: ghcr.io/immich-app/immich-proxy:v1.62.0
    restart: unless-stopped
    mem_limit: 512m
    environment:
      IMMICH_WEB_URL: http://immich-web:3000
      IMMICH_SERVER_URL: http://immich-server:3001
      IMMICH_MACHINE_LEARNING_URL: http://immich-machine-learning:3003
    ports:
      - 2283:8080
    logging:
      driver: none
    depends_on:
      - immich-server

Your .env content

NA

Reproduction steps

1. Just reboot the stack.

Additional information

Docker version

docker --version
Docker version 24.0.2, build cb74dfc

Docker compose version:

Docker Compose version v2.18.1
Originally created by @jpoggi on GitHub (Jun 19, 2023). Hi, Since a few updates one container on the stack keep crashing in a loop with this error message: ``` immich-machine-learning | INFO: Started server process [1] immich-machine-learning | INFO: Waiting for application startup. immich-machine-learning | Could not find image processor class in the image processor config or the model config. Loading based on pattern matching with the model's feature extractor configuration. immich-machine-learning exited with code 0 ``` Can you point me the right direction on how to debug this ? Thanks. ### The OS that Immich Server is running on Ubuntu 22.04 ### Version of Immich Server v1.62.0 ### Version of Immich Mobile App v1.62.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML services: redis.server.immich: container_name: redis.server.immich image: redis:7 restart: unless-stopped mem_limit: 512m volumes: - /srv/immich/redis_01:/data healthcheck: test: ["CMD", "redis-cli", "--raw", "incr", "ping"] interval: 60s retries: 5 start_period: 10s timeout: 30s postgres.server.immich: container_name: postgres.server.immich image: postgres:15 restart: unless-stopped mem_limit: 512m environment: POSTGRES_DB: immich POSTGRES_USER: immich POSTGRES_PASSWORD: $DB_PASSWORD PG_DATA: /var/lib/postgresql/data volumes: - /srv/immich/db_01:/var/lib/postgresql/data healthcheck: test: ["CMD", "pg_isready", "-q", "-d", "immich", "-U", "immich"] interval: 60s retries: 5 start_period: 10s timeout: 30s immich-server: container_name: immich-server image: ghcr.io/immich-app/immich-server:v1.62.0 entrypoint: ["/bin/sh", "./start-server.sh"] restart: unless-stopped mem_limit: 512m volumes: - /srv/immich/vol_01:/usr/src/app/upload environment: TYPESENSE_API_KEY: $TYPESENSE_API_KEY DB_HOSTNAME: postgres.server.immich DB_DATABASE_NAME: immich DB_USERNAME: immich DB_PASSWORD: $DB_PASSWORD REDIS_HOSTNAME: redis.server.immich depends_on: - redis.server.immich - postgres.server.immich immich-microservices: container_name: immich-microservices image: ghcr.io/immich-app/immich-server:v1.62.0 restart: unless-stopped mem_limit: 1024m entrypoint: ["/bin/sh", "./start-microservices.sh"] volumes: - /srv/immich/vol_01:/usr/src/app/upload environment: TYPESENSE_API_KEY: $TYPESENSE_API_KEY DB_HOSTNAME: postgres.server.immich DB_DATABASE_NAME: immich DB_USERNAME: immich DB_PASSWORD: $DB_PASSWORD REDIS_HOSTNAME: redis.server.immich depends_on: - redis.server.immich - postgres.server.immich immich-machine-learning: container_name: immich-machine-learning image: ghcr.io/immich-app/immich-machine-learning:v1.62.0 restart: unless-stopped mem_limit: 512m volumes: - /srv/immich/vol_01:/usr/src/app/upload - /srv/immich/vol_02:/cache typesense: container_name: typesense image: typesense/typesense:0.24.0 restart: always environment: TYPESENSE_API_KEY: $TYPESENSE_API_KEY TYPESENSE_DATA_DIR: /data logging: driver: none volumes: - /srv/immich/vol_03:/data immich-web: container_name: immich-web image: ghcr.io/immich-app/immich-web:v1.62.0 restart: unless-stopped mem_limit: 512m environment: PUBLIC_LOGIN_PAGE_MESSAGE: Welcome on Immich. entrypoint: ["/bin/sh", "./entrypoint.sh"] immich-proxy: container_name: immich-proxy image: ghcr.io/immich-app/immich-proxy:v1.62.0 restart: unless-stopped mem_limit: 512m environment: IMMICH_WEB_URL: http://immich-web:3000 IMMICH_SERVER_URL: http://immich-server:3001 IMMICH_MACHINE_LEARNING_URL: http://immich-machine-learning:3003 ports: - 2283:8080 logging: driver: none depends_on: - immich-server ``` ### Your .env content ```Shell NA ``` ### Reproduction steps ```bash 1. Just reboot the stack. ``` ### Additional information Docker version ``` docker --version Docker version 24.0.2, build cb74dfc ``` Docker compose version: ``` Docker Compose version v2.18.1 ```
Author
Owner

@mertalev commented on GitHub (Jun 19, 2023):

Does this still happen when mem_limit is removed? For the ML logs, the first three lines are normal and the last one means it exited without error, so I imagine it's docker that's shutting it down because it's using more memory than allowed.

@mertalev commented on GitHub (Jun 19, 2023): Does this still happen when `mem_limit` is removed? For the ML logs, the first three lines are normal and the last one means it exited without error, so I imagine it's docker that's shutting it down because it's using more memory than allowed.
Author
Owner

@alextran1502 commented on GitHub (Jun 19, 2023):

machine-learning will need at least 2~3GB of RAM to load all the models I believe.

@alextran1502 commented on GitHub (Jun 19, 2023): machine-learning will need at least 2~3GB of RAM to load all the models I believe.
Author
Owner

@jpoggi commented on GitHub (Jun 19, 2023):

It seems ok now.

Container is consuming around 2gb of memory without setting any limitations.

2ba47c878de0   immich-machine-learning        0.17%     1.869GiB / 15.31GiB   12.21%    290MB / 719kB     2.6GB / 2.52GB    29

I can't believe it was just a memory issue.

Thanks for the quick and just answer.

Regards.

@jpoggi commented on GitHub (Jun 19, 2023): It seems ok now. Container is consuming around 2gb of memory without setting any limitations. ``` 2ba47c878de0 immich-machine-learning 0.17% 1.869GiB / 15.31GiB 12.21% 290MB / 719kB 2.6GB / 2.52GB 29 ``` I can't believe it was just a memory issue. Thanks for the quick and just answer. Regards.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#970