[BUG] Message in dmesg "traps: gunicorn[119727] trap invalid opcode ip:7fa3580045f2 sp:7ffdfa43c970 error:0 in libtorch_cpu.so[7fa34792f000+16075000]" #1910

Closed
opened 2026-02-05 04:25:00 +03:00 by OVERLORD · 2 comments
Owner

Originally created by @ajm83 on GitHub (Dec 31, 2023).

The bug

I am using the Immich docker image, I noticed I get this message in my dmesg log approx 1-2 times per minute.

traps: gunicorn[119727] trap invalid opcode ip:7fa3580045f2 sp:7ffdfa43c970 error:0 in libtorch_cpu.so[7fa34792f000+16075000]

I suppose that the message means the AI library is built with features not compatible with my CPU? This is fine for me, I don't particularly need that function and my CPU is fairly old (HP Microserver N40L with AMD Turion II Neo).

However the reason I am reporting this as a bug is that I switched this off in the Immich admin GUI, but I still get the above message logged.

I also tried setting IMMICH_MACHINE_LEARNING_ENABLED=false in the .env file, this has also not stopped the message.

[Sun Dec 31 10:39:16 2023] traps: gunicorn[121530] trap invalid opcode ip:7f62fee045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62ee72f000+16075000] [Sun Dec 31 10:39:29 2023] traps: gunicorn[121602] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:39:42 2023] traps: gunicorn[121608] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:39:55 2023] traps: gunicorn[121616] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:07 2023] traps: gunicorn[121624] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:22 2023] traps: gunicorn[121629] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:45 2023] traps: gunicorn[121671] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:41:12 2023] traps: gunicorn[121732] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000]

I think that the IMMICH_MACHINE_LEARNING_ENABLED=false setting or the admin GUI option should prevent the immich-machine-learning from starting (or prevent it from attempting to run the job every minute in any case).

Thanks

The OS that Immich Server is running on

Ubuntu 22.04

Version of Immich Server

v1.91.4

Version of Immich Mobile App

v1.91.4

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

version: "3.8"

#
# WARNING: Make sure to use the docker-compose.yml of the current release:
#
# https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
#
# The compose file on main may not be compatible with the latest release.
#

name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - 2283:3001
    depends_on:
      - redis
      - database
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.yml
    #   service: hwaccel
    command: [ "start.sh", "microservices" ]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

  immich-machine-learning:
    container_name: immich_machine_learning
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:b6124ab2e45cc332e16398022a411d7e37181f21ff7874835e0180f56a09e82a
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=/mnt/pool/Photos/Immich/

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
DB_PASSWORD=password123lol

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis


IMMICH_MACHINE_LEARNING_ENABLED=false

Reproduction steps

1. Have N40L hardware
2. Start docker
3. Check dmesg output

Additional information

No response

Originally created by @ajm83 on GitHub (Dec 31, 2023). ### The bug I am using the Immich docker image, I noticed I get this message in my dmesg log approx 1-2 times per minute. `traps: gunicorn[119727] trap invalid opcode ip:7fa3580045f2 sp:7ffdfa43c970 error:0 in libtorch_cpu.so[7fa34792f000+16075000]` I suppose that the message means the AI library is built with features not compatible with my CPU? This is fine for me, I don't particularly need that function and my CPU is fairly old (HP Microserver N40L with AMD Turion II Neo). However the reason I am reporting this as a bug is that I switched this off in the Immich admin GUI, but I still get the above message logged. I also tried setting IMMICH_MACHINE_LEARNING_ENABLED=false in the .env file, this has also not stopped the message. `[Sun Dec 31 10:39:16 2023] traps: gunicorn[121530] trap invalid opcode ip:7f62fee045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62ee72f000+16075000] [Sun Dec 31 10:39:29 2023] traps: gunicorn[121602] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:39:42 2023] traps: gunicorn[121608] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:39:55 2023] traps: gunicorn[121616] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:07 2023] traps: gunicorn[121624] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:22 2023] traps: gunicorn[121629] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:40:45 2023] traps: gunicorn[121671] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000] [Sun Dec 31 10:41:12 2023] traps: gunicorn[121732] trap invalid opcode ip:7f62ff6045f2 sp:7ffeaa571fe0 error:0 in libtorch_cpu.so[7f62eef2f000+16075000]` I think that the IMMICH_MACHINE_LEARNING_ENABLED=false setting or the admin GUI option should prevent the immich-machine-learning from starting (or prevent it from attempting to run the job every minute in any case). Thanks ### The OS that Immich Server is running on Ubuntu 22.04 ### Version of Immich Server v1.91.4 ### Version of Immich Mobile App v1.91.4 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML version: "3.8" # # WARNING: Make sure to use the docker-compose.yml of the current release: # # https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml # # The compose file on main may not be compatible with the latest release. # name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: [ "start.sh", "immich" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} # extends: # file: hwaccel.yml # service: hwaccel command: [ "start.sh", "microservices" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:b6124ab2e45cc332e16398022a411d7e37181f21ff7874835e0180f56a09e82a restart: always database: container_name: immich_postgres image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always volumes: pgdata: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored UPLOAD_LOCATION=/mnt/pool/Photos/Immich/ # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password DB_PASSWORD=password123lol # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=postgres DB_DATABASE_NAME=immich REDIS_HOSTNAME=immich_redis IMMICH_MACHINE_LEARNING_ENABLED=false ``` ### Reproduction steps ```bash 1. Have N40L hardware 2. Start docker 3. Check dmesg output ``` ### Additional information _No response_
Author
Owner

@bo0tzz commented on GitHub (Dec 31, 2023):

Turning off ML in the settings just makes it so the features are disabled/no requests to the ML container are made. You still need to remove it from the docker-compose file yourself.

@bo0tzz commented on GitHub (Dec 31, 2023): Turning off ML in the settings just makes it so the features are disabled/no requests to the ML container are made. You still need to remove it from the docker-compose file yourself.
Author
Owner

@LudwikO commented on GitHub (Jan 8, 2024):

@ajm83 - I know that the issue is closed now, but I think you may find it interesting/useful: I had very similar (or maybe even the same) bug on also quite old AMD E-350 processor. I believe that it may be something connected with lack of AVX. Anyway, I managed to get it to work, by building machine learning container from source by myself on this old machine.

@LudwikO commented on GitHub (Jan 8, 2024): @ajm83 - I know that the issue is closed now, but I think you may find it interesting/useful: I had very similar (or maybe even the same) bug on also quite old AMD E-350 processor. I believe that it may be something connected with lack of AVX. Anyway, I managed to get it to work, by building machine learning container from source by myself on this old machine.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1910