immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-03-01 11:20:12 +03:00
Code Issues 783 Packages Projects Releases 12 Wiki Activity

CUDA acceleration complains "Could not load ANN shared libraries" #4366

New Issue
Closed
opened 2026-02-05 10:15:22 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 10:15:22 +03:00
Owner
Copy Link

Originally created by @kutsyy on GitHub (Sep 19, 2024).

The bug

I am running remote machine learning on windows 11 with and trying to enable CUDA based hardware acceleration. hardware acceleration doesn't start and log shows:

DEBUG Could not load ANN shared libraries, using ONNX:
libmali.so: cannot open shared object file: No such
file or directory

which is not CUDA

The OS that Immich Server is running on

Windows 10

Version of Immich Server

V1.115.0

Version of Immich Mobile App

NA

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

name: immich_remote_ml

services: 
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    #extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities:
                - gpu
    volumes:
      - model-cache:/cache
    restart: always
    env_file:
      - .env
    ports:
      - 3003:3003
    
volumes:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

IMMICH_LOG_LEVEL=verbose
# The location where your uploaded files are stored
UPLOAD_LOCATION=./library
# The location where your database files are stored
DB_DATA_LOCATION=./postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
# TZ=Etc/UTC

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=XXX

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Reproduction steps

  1. docker compose up -d
  2. docker logs immich_machine_learning

...

Relevant log output

> docker logs --follow  immich_machine_learning
[09/19/24 13:44:01] INFO     Starting gunicorn 23.0.0
[09/19/24 13:44:01] INFO     Listening at: http://[::]:3003 (9)
[09/19/24 13:44:01] INFO     Using worker: app.config.CustomUvicornWorker
[09/19/24 13:44:01] INFO     Booting worker with pid: 10
[09/19/24 13:44:03] DEBUG    Could not load ANN shared libraries, using ONNX:
                             libmali.so: cannot open shared object file: No such
                             file or directory
[09/19/24 13:44:07] INFO     Started server process [10]
[09/19/24 13:44:07] INFO     Waiting for application startup.
[09/19/24 13:44:07] INFO     Created in-memory cache with unloading after 300s
                             of inactivity.
[09/19/24 13:44:07] INFO     Initialized request thread pool with 12 threads.
[09/19/24 13:44:07] DEBUG    Checking for inactivity...
[09/19/24 13:44:07] INFO     Application startup complete.
[09/19/24 13:44:17] DEBUG    Checking for inactivity...
[09/19/24 13:44:27] DEBUG    Checking for inactivity...

Additional information

nvidia-smi
Thu Sep 19 07:16:59 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 560.94 Driver Version: 560.94 CUDA Version: 12.6 |
|-----------------------------------------+------------------------+----------------------+
| GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce GTX 970 WDDM | 00000000:01:00.0 Off | N/A |
| 0% 36C P8 15W / 163W | 958MiB / 4096MiB | 1% Default |
| | | N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|=========================================================================================|
| 0 N/A N/A 3896 C+G ...1.0_x64__8wekyb3d8bbwe\Video.UI.exe N/A |
| 0 N/A N/A 4088 C+G ...Mozilla Thunderbird\thunderbird.exe N/A |
| 0 N/A N/A 5772 C+G ...24.4.2\host-windows-x64\nsys-ui.exe N/A |
| 0 N/A N/A 8268 C+G ...\Docker\frontend\Docker Desktop.exe N/A |
| 0 N/A N/A 9640 C+G ...\Chrome Beta\Application\chrome.exe N/A |
| 0 N/A N/A 11888 C+G C:\Windows\explorer.exe N/A |
| 0 N/A N/A 12328 C+G ...2txyewy\StartMenuExperienceHost.exe N/A |
| 0 N/A N/A 12980 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 13400 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A |
| 0 N/A N/A 13440 C+G ...ejd91yc\AdobeNotificationClient.exe N/A |
| 0 N/A N/A 13920 C+G ...\Chrome Beta\Application\chrome.exe N/A |
| 0 N/A N/A 13948 C+G ...on\128.0.2739.79\msedgewebview2.exe N/A |
| 0 N/A N/A 14508 C+G ...5n1h2txyewy\ShellExperienceHost.exe N/A |
| 0 N/A N/A 16024 C+G ...CBS_cw5n1h2txyewy\TextInputHost.exe N/A |
| 0 N/A N/A 22712 C+G ...on\128.0.2739.79\msedgewebview2.exe N/A |
| 0 N/A N/A 23000 C+G ...8.0_x64__v10z8vjag6ke6\HP.Smart.exe N/A |
+-----------------------------------------------------------------------------------------+

Originally created by @kutsyy on GitHub (Sep 19, 2024). ### The bug I am running remote machine learning on windows 11 with and trying to enable CUDA based hardware acceleration. hardware acceleration doesn't start and log shows: DEBUG Could not load ANN shared libraries, using ONNX: libmali.so: cannot open shared object file: No such file or directory which is not CUDA ### The OS that Immich Server is running on Windows 10 ### Version of Immich Server V1.115.0 ### Version of Immich Mobile App NA ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML name: immich_remote_ml services: immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release} #extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration # file: hwaccel.ml.yml # service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable deploy: resources: reservations: devices: - driver: nvidia count: 1 capabilities: - gpu volumes: - model-cache:/cache restart: always env_file: - .env ports: - 3003:3003 volumes: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables IMMICH_LOG_LEVEL=verbose # The location where your uploaded files are stored UPLOAD_LOCATION=./library # The location where your database files are stored DB_DATA_LOCATION=./postgres # To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List # TZ=Etc/UTC # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secret for postgres. You should change it to a random password # Please use only the characters `A-Za-z0-9`, without special characters or spaces DB_PASSWORD=XXX # The values below this line do not need to be changed ################################################################################### DB_USERNAME=postgres DB_DATABASE_NAME=immich ``` ### Reproduction steps 1. docker compose up -d 2. docker logs immich_machine_learning 3. ... ### Relevant log output ```shell > docker logs --follow immich_machine_learning [09/19/24 13:44:01] INFO Starting gunicorn 23.0.0 [09/19/24 13:44:01] INFO Listening at: http://[::]:3003 (9) [09/19/24 13:44:01] INFO Using worker: app.config.CustomUvicornWorker [09/19/24 13:44:01] INFO Booting worker with pid: 10 [09/19/24 13:44:03] DEBUG Could not load ANN shared libraries, using ONNX: libmali.so: cannot open shared object file: No such file or directory [09/19/24 13:44:07] INFO Started server process [10] [09/19/24 13:44:07] INFO Waiting for application startup. [09/19/24 13:44:07] INFO Created in-memory cache with unloading after 300s of inactivity. [09/19/24 13:44:07] INFO Initialized request thread pool with 12 threads. [09/19/24 13:44:07] DEBUG Checking for inactivity... [09/19/24 13:44:07] INFO Application startup complete. [09/19/24 13:44:17] DEBUG Checking for inactivity... [09/19/24 13:44:27] DEBUG Checking for inactivity... ``` ### Additional information > nvidia-smi Thu Sep 19 07:16:59 2024 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 560.94 Driver Version: 560.94 CUDA Version: 12.6 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Driver-Model | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce GTX 970 WDDM | 00000000:01:00.0 Off | N/A | | 0% 36C P8 15W / 163W | 958MiB / 4096MiB | 1% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 3896 C+G ...1.0_x64__8wekyb3d8bbwe\Video.UI.exe N/A | | 0 N/A N/A 4088 C+G ...Mozilla Thunderbird\thunderbird.exe N/A | | 0 N/A N/A 5772 C+G ...24.4.2\host-windows-x64\nsys-ui.exe N/A | | 0 N/A N/A 8268 C+G ...\Docker\frontend\Docker Desktop.exe N/A | | 0 N/A N/A 9640 C+G ...\Chrome Beta\Application\chrome.exe N/A | | 0 N/A N/A 11888 C+G C:\Windows\explorer.exe N/A | | 0 N/A N/A 12328 C+G ...2txyewy\StartMenuExperienceHost.exe N/A | | 0 N/A N/A 12980 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A | | 0 N/A N/A 13400 C+G ....Search_cw5n1h2txyewy\SearchApp.exe N/A | | 0 N/A N/A 13440 C+G ...ejd91yc\AdobeNotificationClient.exe N/A | | 0 N/A N/A 13920 C+G ...\Chrome Beta\Application\chrome.exe N/A | | 0 N/A N/A 13948 C+G ...on\128.0.2739.79\msedgewebview2.exe N/A | | 0 N/A N/A 14508 C+G ...5n1h2txyewy\ShellExperienceHost.exe N/A | | 0 N/A N/A 16024 C+G ...CBS_cw5n1h2txyewy\TextInputHost.exe N/A | | 0 N/A N/A 22712 C+G ...on\128.0.2739.79\msedgewebview2.exe N/A | | 0 N/A N/A 23000 C+G ...8.0_x64__v10z8vjag6ke6\HP.Smart.exe N/A | +-----------------------------------------------------------------------------------------+
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#4366
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?