immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-25 03:12:21 +03:00
Code Issues 783 Packages Projects Releases 12 Wiki Activity

/api/server-info/ping doesn't check connection to postgres #3380

New Issue
Closed
opened 2026-02-05 08:13:52 +03:00 by OVERLORD · 3 comments
OVERLORD commented 2026-02-05 08:13:52 +03:00
Owner
Copy Link

Originally created by @dani on GitHub (May 30, 2024).

The bug

I'm using Immich on a Nomad orchestrator, and using /api/server-info/ping endpoint to check server's health. A regularly have an issue where connection to postgres times out (the initial problem is likely due to my system which is slowed down during backup, but Immich server is not reconnecting correctly to postgres, and always report a timeout even if the postgres server is working correctly once backups ended). When the postgres connection is down, /api/server-info/ping is still reporting an OK status, so my orchestrator cannot detect something is wrong, and restart Immich automatically

The OS that Immich Server is running on

Docker (on Nomad orchestrator)

Version of Immich Server

v1.105.1

Version of Immich Mobile App

v1.104.0

Platform with the issue

  • Server
  • Web
  • Mobile

Your docker-compose.yml content

Not using docker-compose, but a Nomad job (https://git.lapiole.org/nomad/immich/src/branch/master/example/immich.nomad.hcl)

Your .env content

REDIS_HOSTNAME=127.0.0.1
IMMICH_MEDIA_LOCATION=/data
LANG=fr_FR.utf8
NODE_OPTIONS=--max-old-space-size=512
TZ=Europe/Paris
HTTP_PROXY=http://immich:XXXX@127.0.0.1:3128
HTTPS_PROXY=http://immich:XXXX@127.0.0.1:3128
DB_URL=postgres://immich:XXXX@127.0.0.1:5432/immich

Reproduction steps

1. Have postgres not reachable
2. Query /api/server-info/ping

Relevant log output

When Immich can't reach postgres, I see these kind of error

2024-05-30 09:05:34.000	    at process.processTimers (node:internal/timers:514:7)
2024-05-30 09:05:34.000	    at listOnTimeout (node:internal/timers:573:17)
2024-05-30 09:05:34.000	    at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27)
2024-05-30 09:05:34.000	Error: timeout exceeded when trying to connect
2024-05-30 09:05:34.000	[Nest] 7  - 30/05/2024 09:05:34   ERROR [ImmichServer] [ExceptionsHandler] timeout exceeded when trying to connect
2024-05-30 09:05:39.000	"GET /api/user/me HTTP/1.1" 500 - 0 52 10005 10004 "10.99.20.2" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" "8e9d4f07-e874-4c5c-a16b-e15bf397205a" "photos.consul" "127.0.0.1:3001"
2024-05-30 09:05:44.000	    at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27)
2024-05-30 09:05:44.000	    at process.processTimers (node:internal/timers:514:7)] Error: timeout exceeded when trying to connect
2024-05-30 09:05:44.000	    at process.processTimers (node:internal/timers:514:7)] Failed to get server config
2024-05-30 09:05:44.000	[Nest] 7  - 30/05/2024 09:05:44   ERROR [ImmichServer] [Error: timeout exceeded when trying to connect
2024-05-30 09:05:44.000	    at listOnTimeout (node:internal/timers:573:17)

I don't know why Immich "think" there's a timeout because postgres is reachable and correctly working. But anyway, the health endpoint should report something is wrong. As you can see a call to /api/user/me responds with a 500 error (when a user tries to access the web interface)



### Additional information

_No response_
Originally created by @dani on GitHub (May 30, 2024). ### The bug I'm using Immich on a Nomad orchestrator, and using /api/server-info/ping endpoint to check server's health. A regularly have an issue where connection to postgres times out (the initial problem is likely due to my system which is slowed down during backup, but Immich server is not reconnecting correctly to postgres, and always report a timeout even if the postgres server is working correctly once backups ended). When the postgres connection is down, /api/server-info/ping is still reporting an OK status, so my orchestrator cannot detect something is wrong, and restart Immich automatically ### The OS that Immich Server is running on Docker (on Nomad orchestrator) ### Version of Immich Server v1.105.1 ### Version of Immich Mobile App v1.104.0 ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML Not using docker-compose, but a Nomad job (https://git.lapiole.org/nomad/immich/src/branch/master/example/immich.nomad.hcl) ``` ### Your .env content ```Shell REDIS_HOSTNAME=127.0.0.1 IMMICH_MEDIA_LOCATION=/data LANG=fr_FR.utf8 NODE_OPTIONS=--max-old-space-size=512 TZ=Europe/Paris HTTP_PROXY=http://immich:XXXX@127.0.0.1:3128 HTTPS_PROXY=http://immich:XXXX@127.0.0.1:3128 DB_URL=postgres://immich:XXXX@127.0.0.1:5432/immich ``` ### Reproduction steps ```bash 1. Have postgres not reachable 2. Query /api/server-info/ping ``` ### Relevant log output ```shell When Immich can't reach postgres, I see these kind of error 2024-05-30 09:05:34.000 at process.processTimers (node:internal/timers:514:7) 2024-05-30 09:05:34.000 at listOnTimeout (node:internal/timers:573:17) 2024-05-30 09:05:34.000 at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27) 2024-05-30 09:05:34.000 Error: timeout exceeded when trying to connect 2024-05-30 09:05:34.000 [Nest] 7 - 30/05/2024 09:05:34 ERROR [ImmichServer] [ExceptionsHandler] timeout exceeded when trying to connect 2024-05-30 09:05:39.000 "GET /api/user/me HTTP/1.1" 500 - 0 52 10005 10004 "10.99.20.2" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/125.0.0.0 Safari/537.36" "8e9d4f07-e874-4c5c-a16b-e15bf397205a" "photos.consul" "127.0.0.1:3001" 2024-05-30 09:05:44.000 at Timeout._onTimeout (/usr/src/app/node_modules/pg-pool/index.js:205:27) 2024-05-30 09:05:44.000 at process.processTimers (node:internal/timers:514:7)] Error: timeout exceeded when trying to connect 2024-05-30 09:05:44.000 at process.processTimers (node:internal/timers:514:7)] Failed to get server config 2024-05-30 09:05:44.000 [Nest] 7 - 30/05/2024 09:05:44 ERROR [ImmichServer] [Error: timeout exceeded when trying to connect 2024-05-30 09:05:44.000 at listOnTimeout (node:internal/timers:573:17) ``` I don't know why Immich "think" there's a timeout because postgres is reachable and correctly working. But anyway, the health endpoint should report something is wrong. As you can see a call to /api/user/me responds with a 500 error (when a user tries to access the web interface) ``` ### Additional information _No response_
OVERLORD commented 2026-02-05 08:13:58 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (May 30, 2024):

/ping isn't really a healthcheck endpoint, it's just hardcoded to return "pong"

@bo0tzz commented on GitHub (May 30, 2024): /ping isn't really a healthcheck endpoint, it's just hardcoded to return "pong"
OVERLORD commented 2026-02-05 08:14:02 +03:00
Author
Owner
Copy Link

@brandonkal commented on GitHub (Aug 17, 2024):

Is there a healthcheck endpoint?

@brandonkal commented on GitHub (Aug 17, 2024): Is there a healthcheck endpoint?
OVERLORD commented 2026-02-05 08:14:06 +03:00
Author
Owner
Copy Link

@bo0tzz commented on GitHub (Aug 17, 2024):

No, but iirc there is an open request for adding one and we would like to at some point.

@bo0tzz commented on GitHub (Aug 17, 2024): No, but iirc there is an open request for adding one and we would like to at some point.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#3380
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?