immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 26 Packages Projects Releases 10 Wiki Activity

Issues with OpenVINO #2276

New Issue
Closed
opened 2026-02-05 05:55:29 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 05:55:29 +03:00
Owner
Copy Link

Originally created by @DX37 on GitHub (Feb 28, 2024).

The bug

OpenVINO HW acceleration still not working properly.
Logs:

immich_machine_learning  | [02/28/24 09:02:45] ERROR    Exception in ASGI application                      
immich_machine_learning  |                                                                                 
immich_machine_learning  |                              ╭─────── Traceback (most recent call last) ───────╮
immich_machine_learning  |                              │ /usr/src/app/main.py:108 in predict             │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   105 │                                         │
immich_machine_learning  |                              │   106 │   model = await load(await model_cache. │
immich_machine_learning  |                              │   107 │   model.configure(**kwargs)             │
immich_machine_learning  |                              │ ❱ 108 │   outputs = await run(model.predict, in │
immich_machine_learning  |                              │   109 │   return ORJSONResponse(outputs)        │
immich_machine_learning  |                              │   110                                           │
immich_machine_learning  |                              │   111                                           │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/main.py:115 in run                 │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   112 async def run(func: Callable[..., Any], i │
immich_machine_learning  |                              │   113 │   if thread_pool is None:               │
immich_machine_learning  |                              │   114 │   │   return func(inputs)               │
immich_machine_learning  |                              │ ❱ 115 │   return await asyncio.get_running_loop │
immich_machine_learning  |                              │   116                                           │
immich_machine_learning  |                              │   117                                           │
immich_machine_learning  |                              │   118 async def load(model: InferenceModel) ->  │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/lib/python3.10/concurrent/futures/thread.p │
immich_machine_learning  |                              │ y:58 in run                                     │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/base.py:60 in predict       │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    57 │   │   self.load()                       │
immich_machine_learning  |                              │    58 │   │   if model_kwargs:                  │
immich_machine_learning  |                              │    59 │   │   │   self.configure(**model_kwargs │ 
immich_machine_learning  |                              │ ❱  60 │   │   return self._predict(inputs)      │ 
immich_machine_learning  |                              │    61 │                                         │ 
immich_machine_learning  |                              │    62 │   @abstractmethod                       │
immich_machine_learning  |                              │    63 │   def _predict(self, inputs: Any) -> An │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /usr/src/app/models/clip.py:52 in _predict      │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │    49 │   │   │   case Image.Image():           │
immich_machine_learning  |                              │    50 │   │   │   │   if self.mode == "text":   │
immich_machine_learning  |                              │    51 │   │   │   │   │   raise TypeError("Cann │
immich_machine_learning  |                              │ ❱  52 │   │   │   │   outputs: NDArray[np.float │
immich_machine_learning  |                              │       self.transform(image_or_text))[0][0]      │
immich_machine_learning  |                              │    53 │   │   │   case str():                   │
immich_machine_learning  |                              │    54 │   │   │   │   if self.mode == "vision": │
immich_machine_learning  |                              │    55 │   │   │   │   │   raise TypeError("Cann │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │ /opt/venv/lib/python3.10/site-packages/onnxrunt │
immich_machine_learning  |                              │ ime/capi/onnxruntime_inference_collection.py:21 │
immich_machine_learning  |                              │ 9 in run                                        │
immich_machine_learning  |                              │                                                 │
immich_machine_learning  |                              │   216 │   │   if not output_names:              │
immich_machine_learning  |                              │   217 │   │   │   output_names = [output.name f │
immich_machine_learning  |                              │   218 │   │   try:                              │
immich_machine_learning  |                              │ ❱ 219 │   │   │   return self._sess.run(output_ │
immich_machine_learning  |                              │   220 │   │   except C.EPFail as err:           │
immich_machine_learning  |                              │   221 │   │   │   if self._enable_fallback:     │
immich_machine_learning  |                              │   222 │   │   │   │   print(f"EP Error: {str(er │
immich_machine_learning  |                              ╰─────────────────────────────────────────────────╯
immich_machine_learning  |                              Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero     
immich_machine_learning  |                              status code returned while running                 
immich_machine_learning  |                              OpenVINO-EP-subgraph_3 node.                       
immich_machine_learning  |                              Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap
immich_machine_learning  |                              h_3_0' Status Message:                             
immich_machine_learning  |                              /home/onnxruntimedev/onnxruntime/onnxruntime/core/p
immich_machine_learning  |                              roviders/openvino/ov_interface.cc:53               
immich_machine_learning  |                              onnxruntime::openvino_ep::OVExeNetwork             
immich_machine_learning  |                              onnxruntime::openvino_ep::OVCore::LoadNetwork(const
immich_machine_learning  |                              string&, std::string&, ov::AnyMap&, std::string)   
immich_machine_learning  |                              [OpenVINO-EP]  Exception while Loading Network for 
immich_machine_learning  |                              graph:                                             
immich_machine_learning  |                              OpenVINOExecutionProvider_OpenVINO-EP-subgraph_3_0C
immich_machine_learning  |                              heck 'false' failed at                             
immich_machine_learning  |                              src/inference/src/core.cpp:149:                    
immich_machine_learning  |                              invalid external data:                             
immich_machine_learning  |                              ExternalDataInfo(data_full_path:                   
immich_machine_learning  |                              Constant_1863_attr__value, offset: 0, data_length: 
immich_machine_learning  |                              0)

immich_microservices  | [Nest] 7  - 02/28/2024, 9:02:45 AM   ERROR [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error
immich_microservices  | [Nest] 7  - 02/28/2024, 9:02:45 AM   ERROR [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error
immich_microservices  |     at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:22:19)
immich_microservices  |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices  |     at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/domain/smart-info/smart-info.service.js:73:31)
immich_microservices  |     at async /usr/src/app/dist/domain/job/job.service.js:137:37
immich_microservices  |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices  |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices  | [Nest] 7  - 02/28/2024, 9:02:45 AM   ERROR [JobService] Object:
immich_microservices  | {
immich_microservices  |   "id": "e260343b-cb91-4bba-82bf-7180db2a2899"
immich_microservices  | }

The OS that Immich Server is running on

Arch Linux

Version of Immich Server

v1.96.0

Version of Immich Mobile App

v1.96.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

name: immich
                                                           
services:   
  immich-server:   
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: [ "start.sh", "immich" ]
    volumes:                                                                                                          
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    env_file:                      
      - .env                                                                                                                                                                                                                                
    ports:                                                                                                            
      - 2283:3001
    depends_on:
      - redis   
      - database                       
    restart: always                
                                                           
  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding
      file: hwaccel.transcoding.yml
      service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    command: [ "start.sh", "microservices" ]
    volumes:                          
      - ${UPLOAD_LOCATION}:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
      - /srv/4tb/samba/public/Медиа/Фото:/mnt/media/4tb_photos:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - model-cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:afb290a0a0d0b2bd7537b62ebff1eb84d045c757c1c31ca2ca48c79536c0de82
    restart: always

  database:
    container_name: immich_postgres
    #image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee
    image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - pgdata:/var/lib/postgresql/data
    restart: always

volumes:
  pgdata:
  model-cache:

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
#UPLOAD_LOCATION=/srv/4tb/samba/backups/immich
UPLOAD_LOCATION=/srv/immich

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release

# Connection secrets for postgres and typesense. You should change these to random passwords
DB_PASSWORD=<redacted>

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=<redacted>
DB_DATABASE_NAME=<redacted>

REDIS_HOSTNAME=immich_redis

PUBLIC_LOGIN_PAGE_MESSAGE=DX37 Immich
TZ=Asia/Novosibirsk

Reproduction steps

1. Change machine learning container to openvino variant
2. `docker compose pull` and `docker compose up -d`
3. Try to start Smart Search job for all assets

Additional information

CPU is Intel Core i5-10500.
Using nllb-clip-large-siglip__v1 multilingual model for Smart Search.

Originally created by @DX37 on GitHub (Feb 28, 2024). ### The bug OpenVINO HW acceleration still not working properly. Logs: ``` immich_machine_learning | [02/28/24 09:02:45] ERROR Exception in ASGI application immich_machine_learning | immich_machine_learning | ╭─────── Traceback (most recent call last) ───────╮ immich_machine_learning | │ /usr/src/app/main.py:108 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 105 │ │ immich_machine_learning | │ 106 │ model = await load(await model_cache. │ immich_machine_learning | │ 107 │ model.configure(**kwargs) │ immich_machine_learning | │ ❱ 108 │ outputs = await run(model.predict, in │ immich_machine_learning | │ 109 │ return ORJSONResponse(outputs) │ immich_machine_learning | │ 110 │ immich_machine_learning | │ 111 │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/main.py:115 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 112 async def run(func: Callable[..., Any], i │ immich_machine_learning | │ 113 │ if thread_pool is None: │ immich_machine_learning | │ 114 │ │ return func(inputs) │ immich_machine_learning | │ ❱ 115 │ return await asyncio.get_running_loop │ immich_machine_learning | │ 116 │ immich_machine_learning | │ 117 │ immich_machine_learning | │ 118 async def load(model: InferenceModel) -> │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/lib/python3.10/concurrent/futures/thread.p │ immich_machine_learning | │ y:58 in run │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/base.py:60 in predict │ immich_machine_learning | │ │ immich_machine_learning | │ 57 │ │ self.load() │ immich_machine_learning | │ 58 │ │ if model_kwargs: │ immich_machine_learning | │ 59 │ │ │ self.configure(**model_kwargs │ immich_machine_learning | │ ❱ 60 │ │ return self._predict(inputs) │ immich_machine_learning | │ 61 │ │ immich_machine_learning | │ 62 │ @abstractmethod │ immich_machine_learning | │ 63 │ def _predict(self, inputs: Any) -> An │ immich_machine_learning | │ │ immich_machine_learning | │ /usr/src/app/models/clip.py:52 in _predict │ immich_machine_learning | │ │ immich_machine_learning | │ 49 │ │ │ case Image.Image(): │ immich_machine_learning | │ 50 │ │ │ │ if self.mode == "text": │ immich_machine_learning | │ 51 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ ❱ 52 │ │ │ │ outputs: NDArray[np.float │ immich_machine_learning | │ self.transform(image_or_text))[0][0] │ immich_machine_learning | │ 53 │ │ │ case str(): │ immich_machine_learning | │ 54 │ │ │ │ if self.mode == "vision": │ immich_machine_learning | │ 55 │ │ │ │ │ raise TypeError("Cann │ immich_machine_learning | │ │ immich_machine_learning | │ /opt/venv/lib/python3.10/site-packages/onnxrunt │ immich_machine_learning | │ ime/capi/onnxruntime_inference_collection.py:21 │ immich_machine_learning | │ 9 in run │ immich_machine_learning | │ │ immich_machine_learning | │ 216 │ │ if not output_names: │ immich_machine_learning | │ 217 │ │ │ output_names = [output.name f │ immich_machine_learning | │ 218 │ │ try: │ immich_machine_learning | │ ❱ 219 │ │ │ return self._sess.run(output_ │ immich_machine_learning | │ 220 │ │ except C.EPFail as err: │ immich_machine_learning | │ 221 │ │ │ if self._enable_fallback: │ immich_machine_learning | │ 222 │ │ │ │ print(f"EP Error: {str(er │ immich_machine_learning | ╰─────────────────────────────────────────────────╯ immich_machine_learning | Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero immich_machine_learning | status code returned while running immich_machine_learning | OpenVINO-EP-subgraph_3 node. immich_machine_learning | Name:'OpenVINOExecutionProvider_OpenVINO-EP-subgrap immich_machine_learning | h_3_0' Status Message: immich_machine_learning | /home/onnxruntimedev/onnxruntime/onnxruntime/core/p immich_machine_learning | roviders/openvino/ov_interface.cc:53 immich_machine_learning | onnxruntime::openvino_ep::OVExeNetwork immich_machine_learning | onnxruntime::openvino_ep::OVCore::LoadNetwork(const immich_machine_learning | string&, std::string&, ov::AnyMap&, std::string) immich_machine_learning | [OpenVINO-EP] Exception while Loading Network for immich_machine_learning | graph: immich_machine_learning | OpenVINOExecutionProvider_OpenVINO-EP-subgraph_3_0C immich_machine_learning | heck 'false' failed at immich_machine_learning | src/inference/src/core.cpp:149: immich_machine_learning | invalid external data: immich_machine_learning | ExternalDataInfo(data_full_path: immich_machine_learning | Constant_1863_attr__value, offset: 0, data_length: immich_machine_learning | 0) ``` ``` immich_microservices | [Nest] 7 - 02/28/2024, 9:02:45 AM ERROR [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | [Nest] 7 - 02/28/2024, 9:02:45 AM ERROR [JobService] Error: Machine learning request for clip failed with status 500: Internal Server Error immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/infra/repositories/machine-learning.repository.js:22:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/domain/smart-info/smart-info.service.js:73:31) immich_microservices | at async /usr/src/app/dist/domain/job/job.service.js:137:37 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) immich_microservices | [Nest] 7 - 02/28/2024, 9:02:45 AM ERROR [JobService] Object: immich_microservices | { immich_microservices | "id": "e260343b-cb91-4bba-82bf-7180db2a2899" immich_microservices | } ``` ### The OS that Immich Server is running on Arch Linux ### Version of Immich Server v1.96.0 ### Version of Immich Mobile App v1.96.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML name: immich services: immich-server: container_name: immich_server image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} command: [ "start.sh", "immich" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro env_file: - .env ports: - 2283:3001 depends_on: - redis - database restart: always immich-microservices: container_name: immich_microservices image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release} extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/hardware-transcoding file: hwaccel.transcoding.yml service: quicksync # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding command: [ "start.sh", "microservices" ] volumes: - ${UPLOAD_LOCATION}:/usr/src/app/upload - /etc/localtime:/etc/localtime:ro - /srv/4tb/samba/public/Медиа/Фото:/mnt/media/4tb_photos:ro env_file: - .env depends_on: - redis - database restart: always immich-machine-learning: container_name: immich_machine_learning # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag. # Example tag: ${IMMICH_VERSION:-release}-cuda image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-openvino extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration file: hwaccel.ml.yml service: openvino # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable volumes: - model-cache:/cache env_file: - .env restart: always redis: container_name: immich_redis image: redis:6.2-alpine@sha256:afb290a0a0d0b2bd7537b62ebff1eb84d045c757c1c31ca2ca48c79536c0de82 restart: always database: container_name: immich_postgres #image: tensorchord/pgvecto-rs:pg14-v0.1.11@sha256:0335a1a22f8c5dd1b697f14f079934f5152eaaa216c09b61e293be285491f8ee image: tensorchord/pgvecto-rs:pg14-v0.2.0@sha256:90724186f0a3517cf6914295b5ab410db9ce23190a2d9d0b9dd6463e3fa298f0 env_file: - .env environment: POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_USER: ${DB_USERNAME} POSTGRES_DB: ${DB_DATABASE_NAME} volumes: - pgdata:/var/lib/postgresql/data restart: always volumes: pgdata: model-cache: ``` ### Your .env content ```Shell # You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables # The location where your uploaded files are stored #UPLOAD_LOCATION=/srv/4tb/samba/backups/immich UPLOAD_LOCATION=/srv/immich # The Immich version to use. You can pin this to a specific version like "v1.71.0" IMMICH_VERSION=release # Connection secrets for postgres and typesense. You should change these to random passwords DB_PASSWORD=<redacted> # The values below this line do not need to be changed ################################################################################### DB_HOSTNAME=immich_postgres DB_USERNAME=<redacted> DB_DATABASE_NAME=<redacted> REDIS_HOSTNAME=immich_redis PUBLIC_LOGIN_PAGE_MESSAGE=DX37 Immich TZ=Asia/Novosibirsk ``` ### Reproduction steps ```bash 1. Change machine learning container to openvino variant 2. `docker compose pull` and `docker compose up -d` 3. Try to start Smart Search job for all assets ``` ### Additional information CPU is Intel Core i5-10500. Using `nllb-clip-large-siglip__v1` multilingual model for Smart Search.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#2276
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?