[BUG] failed to download ViT-B-32 #1516

Closed
opened 2026-02-05 02:12:22 +03:00 by OVERLORD · 1 comment
Owner

Originally created by @t3dium on GitHub (Oct 27, 2023).

The bug

The VIT-B-32 model fails to download in immich, leading to encoding clips failing. Logs are shown below.

Immich's machine learning container:

Actions

      

    raw_response = await run_endpoint_function(

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function

    return await dependant.call(**values)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 75, in predict

    model = await load(await app.state.model_cache.get(model_name, model_type, **kwargs))

            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 101, in load

    await loop.run_in_executor(app.state.thread_pool, _load)

  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run

    result = self.fn(*self.args, **self.kwargs)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 94, in _load

    model.load()

  File "/usr/src/app/models/base.py", line 63, in load

    self.download()

  File "/usr/src/app/models/base.py", line 58, in download

    self._download()

  File "/usr/src/app/models/clip.py", line 43, in _download

    self._download_model(*models[0])

  File "/usr/src/app/models/clip.py", line 101, in _download_model

    download_model(

  File "/opt/venv/lib/python3.11/site-packages/clip_server/model/pretrained_models.py", line 239, in download_model

    raise RuntimeError(

RuntimeError: Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx within retry limit 3

[10/27/23 00:43:56] INFO     Downloading clip model 'ViT-B-32::openai'.This may 

                             take a while.                                      

Failed to download 

https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572

76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 0th

attempt

Failed to download 

https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572

76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 1th

attempt

Failed to download 

https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572

76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 2th

attempt

  textual.onnx 0.0% • 0/100 bytes • ? • -:--:--

                                               

Exception in ASGI application

Traceback (most recent call last):

  File "/opt/venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi

    result = await app(  # type: ignore[func-returns-value]

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__

    return await self.app(scope, receive, send)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/fastapi/applications.py", line 276, in __call__

    await super().__call__(scope, receive, send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__

    await self.middleware_stack(scope, receive, send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__

    raise exc

  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__

    await self.app(scope, receive, _send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__

    raise exc

  File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__

    await self.app(scope, receive, sender)

  File "/opt/venv/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__

    raise e

  File "/opt/venv/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__

    await self.app(scope, receive, send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__

    await route.handle(scope, receive, send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle

    await self.app(scope, receive, send)

  File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 66, in app

    response = await func(request)

               ^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app

    raw_response = await run_endpoint_function(

                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function

    return await dependant.call(**values)

           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 75, in predict

    model = await load(await app.state.model_cache.get(model_name, model_type, **kwargs))

            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 101, in load

    await loop.run_in_executor(app.state.thread_pool, _load)

  File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run

    result = self.fn(*self.args, **self.kwargs)

             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

  File "/usr/src/app/main.py", line 94, in _load

    model.load()

  File "/usr/src/app/models/base.py", line 63, in load

    self.download()

  File "/usr/src/app/models/base.py", line 58, in download

    self._download()

  File "/usr/src/app/models/clip.py", line 43, in _download

    self._download_model(*models[0])

  File "/usr/src/app/models/clip.py", line 101, in _download_model

    download_model(

  File "/opt/venv/lib/python3.11/site-packages/clip_server/model/pretrained_models.py", line 239, in download_model

    raise RuntimeError(

RuntimeError: Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx within retry limit 3

Also I think it would be best if there's a limit on how many times the download link is queried per a given time period, rather than infinitely trying every few seconds. Otherwise it's going to look malicious on amazon's end if this is occurring on every new immich user's device.

The OS that Immich Server is running on

Dietpi (debian based)

Version of Immich Server

v1.82.1

Version of Immich Mobile App

n/a

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

I followed the default config https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml

Your .env content

https://github.com/immich-app/immich/releases/latest/download/example.env

Reproduction steps

1. Started the immich stack
2. Uploaded photos
3. Tried searching for images

Additional information

No response

Originally created by @t3dium on GitHub (Oct 27, 2023). ### The bug The VIT-B-32 model fails to download in immich, leading to encoding clips failing. Logs are shown below. Immich's machine learning container: ``` Actions raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 75, in predict model = await load(await app.state.model_cache.get(model_name, model_type, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 101, in load await loop.run_in_executor(app.state.thread_pool, _load) File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 94, in _load model.load() File "/usr/src/app/models/base.py", line 63, in load self.download() File "/usr/src/app/models/base.py", line 58, in download self._download() File "/usr/src/app/models/clip.py", line 43, in _download self._download_model(*models[0]) File "/usr/src/app/models/clip.py", line 101, in _download_model download_model( File "/opt/venv/lib/python3.11/site-packages/clip_server/model/pretrained_models.py", line 239, in download_model raise RuntimeError( RuntimeError: Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx within retry limit 3 [10/27/23 00:43:56] INFO Downloading clip model 'ViT-B-32::openai'.This may take a while. Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572 76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 0th attempt Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572 76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 1th attempt Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d536572 76696365/onnx/ViT-B-32/textual.onnx with <HTTPError 403: 'Forbidden'> at the 2th attempt textual.onnx 0.0% • 0/100 bytes • ? • -:--:-- Exception in ASGI application Traceback (most recent call last): File "/opt/venv/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 435, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 78, in __call__ return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-packages/fastapi/applications.py", line 276, in __call__ await super().__call__(scope, receive, send) File "/opt/venv/lib/python3.11/site-packages/starlette/applications.py", line 122, in __call__ await self.middleware_stack(scope, receive, send) File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 184, in __call__ raise exc File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/errors.py", line 162, in __call__ await self.app(scope, receive, _send) File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 79, in __call__ raise exc File "/opt/venv/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 68, in __call__ await self.app(scope, receive, sender) File "/opt/venv/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 21, in __call__ raise e File "/opt/venv/lib/python3.11/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__ await self.app(scope, receive, send) File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 718, in __call__ await route.handle(scope, receive, send) File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 276, in handle await self.app(scope, receive, send) File "/opt/venv/lib/python3.11/site-packages/starlette/routing.py", line 66, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 237, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/venv/lib/python3.11/site-packages/fastapi/routing.py", line 163, in run_endpoint_function return await dependant.call(**values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 75, in predict model = await load(await app.state.model_cache.get(model_name, model_type, **kwargs)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 101, in load await loop.run_in_executor(app.state.thread_pool, _load) File "/usr/local/lib/python3.11/concurrent/futures/thread.py", line 58, in run result = self.fn(*self.args, **self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/src/app/main.py", line 94, in _load model.load() File "/usr/src/app/models/base.py", line 63, in load self.download() File "/usr/src/app/models/base.py", line 58, in download self._download() File "/usr/src/app/models/clip.py", line 43, in _download self._download_model(*models[0]) File "/usr/src/app/models/clip.py", line 101, in _download_model download_model( File "/opt/venv/lib/python3.11/site-packages/clip_server/model/pretrained_models.py", line 239, in download_model raise RuntimeError( RuntimeError: Failed to download https://clip-as-service.s3.us-east-2.amazonaws.com/models-436c69702d61732d53657276696365/onnx/ViT-B-32/textual.onnx within retry limit 3 ``` Also I think it would be best if there's a limit on how many times the download link is queried per a given time period, rather than infinitely trying every few seconds. Otherwise it's going to look malicious on amazon's end if this is occurring on every new immich user's device. ### The OS that Immich Server is running on Dietpi (debian based) ### Version of Immich Server v1.82.1 ### Version of Immich Mobile App n/a ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML I followed the default config https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml ``` ### Your .env content ```Shell https://github.com/immich-app/immich/releases/latest/download/example.env ``` ### Reproduction steps ```bash 1. Started the immich stack 2. Uploaded photos 3. Tried searching for images ``` ### Additional information _No response_
Author
Owner

@tdopierre commented on GitHub (Oct 27, 2023):

This is known issue, please see https://github.com/immich-app/immich/issues/4117#issuecomment-1772790612

@tdopierre commented on GitHub (Oct 27, 2023): This is known issue, please see https://github.com/immich-app/immich/issues/4117#issuecomment-1772790612
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#1516