immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-02-05 00:30:57 +03:00
Code Issues 11 Packages Projects Releases 10 Wiki Activity

[Bug]: Handle SIGTERM #669

New Issue
Closed
opened 2026-02-04 21:47:14 +03:00 by OVERLORD · 8 comments
OVERLORD commented 2026-02-04 21:47:14 +03:00
Owner
Copy Link

Originally created by @bt90 on GitHub (Feb 6, 2023).

The node processes started by Immich currently don't handle the SIGTERM signal. This is required to let Docker gracefully shutdown the containers.

Ideally this should close any open connections and also be logged.

Originally created by @bt90 on GitHub (Feb 6, 2023). The node processes started by Immich currently don't handle the `SIGTERM` signal. This is required to let Docker gracefully shutdown the containers. Ideally this should close any open connections and also be logged.
OVERLORD added the 🗄️server label 2026-02-04 21:47:14 +03:00
OVERLORD commented 2026-02-04 21:47:34 +03:00
Author
Owner
Copy Link

@bt90 commented on GitHub (Feb 6, 2023):

Affected:

  • machine-learning
  • microservices
  • server
  • web
@bt90 commented on GitHub (Feb 6, 2023): Affected: * [ ] machine-learning * [ ] microservices * [ ] server * [ ] web
OVERLORD commented 2026-02-04 21:47:43 +03:00
Author
Owner
Copy Link

@bertmelis commented on GitHub (Mar 22, 2023):

Just a comment to give this issue a bit more attention. Especially for my Podman installation where I run containers using systemd, I have to hack a bit to manage the units.

Maybe this is a solution: https://stackoverflow.com/questions/43584760/gunicorn-graceful-stopping-with-docker-compose (for ML)

For the services, catching the signals in the script might work?

I'm just thinking out loud here. I'm a mere beginner when it comes to containers.

@bertmelis commented on GitHub (Mar 22, 2023): Just a comment to give this issue a bit more attention. Especially for my Podman installation where I run containers using systemd, I have to hack a bit to manage the units. Maybe this is a solution: https://stackoverflow.com/questions/43584760/gunicorn-graceful-stopping-with-docker-compose (for ML) For the services, catching the signals in the script might work? I'm just thinking out loud here. I'm a mere beginner when it comes to containers.
OVERLORD commented 2026-02-04 21:47:50 +03:00
Author
Owner
Copy Link

@bt90 commented on GitHub (Mar 22, 2023):

The container itself should already be setup correctly. It's just the node logic which is missing here.

@bt90 commented on GitHub (Mar 22, 2023): The container itself should already be setup correctly. It's just the node logic which is missing here.
OVERLORD commented 2026-02-04 21:47:56 +03:00
Author
Owner
Copy Link

@bertmelis commented on GitHub (Mar 22, 2023):

You mean the containers itself stop with SIGTERM but the business logic isn't gracefully aborted?

Then I have a bug in my setup because some of my containers still fallback to SIGKILL after the standard 10s timeout.

@bertmelis commented on GitHub (Mar 22, 2023): You mean the containers itself stop with SIGTERM but the business logic isn't gracefully aborted? Then I have a bug in my setup because some of my containers still fallback to SIGKILL after the standard 10s timeout.
OVERLORD commented 2026-02-04 21:48:01 +03:00
Author
Owner
Copy Link

@bt90 commented on GitHub (Mar 22, 2023):

I should phrase it differently. The signal from the docker daemon is correctly handed to the node process which ignores it at the moment. The daemon ultimately waits 10s until it kills the processes in a non-graceful manner using SIGKILL.

tl;dr immich processes need to handle SIGTERM

@bt90 commented on GitHub (Mar 22, 2023): I should phrase it differently. The signal from the docker daemon is correctly handed to the node process which ignores it at the moment. The daemon ultimately waits 10s until it kills the processes in a non-graceful manner using SIGKILL. tl;dr immich processes need to handle SIGTERM
OVERLORD commented 2026-02-04 21:48:04 +03:00
Author
Owner
Copy Link

@bt90 commented on GitHub (Jul 20, 2023):

https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals

Looks like we need to add an init process to our docker containers. Maybe we can get away with a simple change to our compose file?

https://docs.docker.com/compose/compose-file/05-services/#init

@bt90 commented on GitHub (Jul 20, 2023): https://github.com/nodejs/docker-node/blob/main/docs/BestPractices.md#handling-kernel-signals Looks like we need to add an init process to our docker containers. Maybe we can get away with a simple change to our compose file? https://docs.docker.com/compose/compose-file/05-services/#init
OVERLORD commented 2026-02-04 21:48:08 +03:00
Author
Owner
Copy Link

@ldobbelsteen commented on GitHub (Apr 9, 2025):

Both the main server and machine learning containers still exit with code 143 when receiving a SIGTERM. Is this the intentional behavior? I think it is normal practice to exit using code 0 if a SIGTERM is handled gracefully instead of 143.

I personally run Immich using Podman Quadlets, and systemd interprets an exit code 143 as a failure by default. Also on a different system using Synology Container Manager, I get a system warning notification if I stop the Immich container gracefully. Both these issues can be solved by surpressing exit code 143 warnings, but I think using exit code 0 is cleaner.

@ldobbelsteen commented on GitHub (Apr 9, 2025): Both the main server and machine learning containers still exit with code 143 when receiving a SIGTERM. Is this the intentional behavior? I think it is normal practice to exit using code 0 if a SIGTERM is handled gracefully instead of 143. I personally run Immich using Podman Quadlets, and systemd interprets an exit code 143 as a failure by default. Also on a different system using Synology Container Manager, I get a system warning notification if I stop the Immich container gracefully. Both these issues can be solved by surpressing exit code 143 warnings, but I think using exit code 0 is cleaner.
OVERLORD commented 2026-02-04 21:48:20 +03:00
Author
Owner
Copy Link

@bt90 commented on GitHub (Apr 9, 2025):

That seems to be a nodejs thing: https://nodejs.org/api/process.html#process_signal_events

It may be worth opening a new issue for signal handling in the application code.

@bt90 commented on GitHub (Apr 9, 2025): That seems to be a nodejs thing: https://nodejs.org/api/process.html#process_signal_events It may be worth opening a new issue for signal handling in the application code.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:translation
cli
date-time
documentation
external-library
format
good first issue
needs-answer
nice to have
sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label 🗄️server
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#669
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?