Microservices error contacting Machnie Learning when URL has trailing slash #3219

Closed
opened 2026-02-05 08:04:36 +03:00 by OVERLORD · 0 comments
Owner

Originally created by @AlmightyFrog on GitHub (May 22, 2024).

The bug

I was doing tests doing load balancing ML jobs to several systems in parallel. When i changed it back from nginx load balancer to direct container configured it like this http://immich-machine-learning:3003/ including a trailing slash
image

This seems to break Microservices container issuing Machinelearning

immich_microservices           | [Nest] 7  - 05/22/2024, 7:35:18 PM   ERROR [ImmichMicroservices] [JobService] Object:
immich_microservices           | {
immich_microservices           |   "id": "d63c6a02-d2d7-4df9-be46-17c3432d4281"
immich_microservices           | }
immich_microservices           |
immich_microservices           | [Nest] 7  - 05/22/2024, 7:35:18 PM   ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 404: Not Found
immich_microservices           | [Nest] 7  - 05/22/2024, 7:35:18 PM   ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for clip failed with status 404: Not Found
immich_microservices           |     at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19)
immich_microservices           |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices           |     at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:86:31)
immich_microservices           |     at async /usr/src/app/dist/services/job.service.js:145:36
immich_microservices           |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices           |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)

When I change it to http://immich-machine-learning:3003 everything works fine.

Can someone else please confirm that this is a real issue, and if so, either sanitize the input or give a warning or at least document it somewhere that traling slash seems to make an issue?
Thanks!


I tried to get the exact url which is making issues by switching back to nginx load balancer and logging all access+error, but there then now only following popped up which i guess is something completely else.

==> access.log <==
172.16.19.7 - - [22/May/2024:20:01:37 +0000] "POST /predict HTTP/1.1" 504 167 "-" "node"

==> error.log <==
2024/05/22 20:01:37 [error] 29#29: *49 client intended to send too large body: 1266390 bytes, client: 172.16.19.7, server: , request: "POST /predict HTTP/1.1", host: "immich-machine-learning-nginx:3003"

The OS that Immich Server is running on

Arch

Version of Immich Server

v1.105.1

Version of Immich Mobile App

does not matter

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

does not matter

Your .env content

pretty much default.

Reproduction steps

1. Add slash
2. Issue e.g. face detection
3. notice error on console log

Relevant log output

No response

Additional information

No response

Originally created by @AlmightyFrog on GitHub (May 22, 2024). ### The bug I was doing tests doing load balancing ML jobs to several systems in parallel. When i changed it back from nginx load balancer to direct container configured it like this `http://immich-machine-learning:3003/` including a trailing slash ![image](https://github.com/immich-app/immich/assets/2421469/df36fdac-82bc-4b8b-a984-68a5cbcc01c5) This seems to break Microservices container issuing Machinelearning ``` immich_microservices | [Nest] 7 - 05/22/2024, 7:35:18 PM ERROR [ImmichMicroservices] [JobService] Object: immich_microservices | { immich_microservices | "id": "d63c6a02-d2d7-4df9-be46-17c3432d4281" immich_microservices | } immich_microservices | immich_microservices | [Nest] 7 - 05/22/2024, 7:35:18 PM ERROR [ImmichMicroservices] [JobService] Unable to run job handler (smartSearch/smart-search): Error: Machine learning request for clip failed with status 404: Not Found immich_microservices | [Nest] 7 - 05/22/2024, 7:35:18 PM ERROR [ImmichMicroservices] [JobService] Error: Machine learning request for clip failed with status 404: Not Found immich_microservices | at MachineLearningRepository.predict (/usr/src/app/dist/repositories/machine-learning.repository.js:23:19) immich_microservices | at process.processTicksAndRejections (node:internal/process/task_queues:95:5) immich_microservices | at async SmartInfoService.handleEncodeClip (/usr/src/app/dist/services/smart-info.service.js:86:31) immich_microservices | at async /usr/src/app/dist/services/job.service.js:145:36 immich_microservices | at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28) immich_microservices | at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24) ``` When I change it to `http://immich-machine-learning:3003` everything works fine. Can someone else please confirm that this is a real issue, and if so, either sanitize the input or give a warning or at least document it somewhere that traling slash seems to make an issue? Thanks! --- I tried to get the exact url which is making issues by switching back to nginx load balancer and logging all access+error, but there then now only following popped up which i guess is something completely else. ``` ==> access.log <== 172.16.19.7 - - [22/May/2024:20:01:37 +0000] "POST /predict HTTP/1.1" 504 167 "-" "node" ==> error.log <== 2024/05/22 20:01:37 [error] 29#29: *49 client intended to send too large body: 1266390 bytes, client: 172.16.19.7, server: , request: "POST /predict HTTP/1.1", host: "immich-machine-learning-nginx:3003" ``` ### The OS that Immich Server is running on Arch ### Version of Immich Server v1.105.1 ### Version of Immich Mobile App does not matter ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML does not matter ``` ### Your .env content ```Shell pretty much default. ``` ### Reproduction steps ```bash 1. Add slash 2. Issue e.g. face detection 3. notice error on console log ``` ### Relevant log output _No response_ ### Additional information _No response_
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#3219