immich-app/immich
1
0
Fork 0
mirror of https://github.com/immich-app/immich.git synced 2026-07-25 14:00:45 +03:00
Code Issues 783 Packages Projects Releases 22 Wiki Activity

[PR #3934] [MERGED] chore(ml): memory optimisations #10167

New Issue
Closed
opened 2026-02-05 14:17:03 +03:00 by OVERLORD · 0 comments
OVERLORD commented 2026-02-05 14:17:03 +03:00
Owner
Copy Link

📋 Pull Request Information

Original PR: https://github.com/immich-app/immich/pull/3934
Author: @mertalev
Created: 8/31/2023
Status: ✅ Merged
Merged: 9/1/2023
Merged by: @alextran1502

Base: main ← Head: chore/ml-mem-optimisations

📝 Commits (2)

  • b6dc936 mimalloc, gunicorn, no mem arena, optional threads
  • 81a3c46 formatting

📊 Changes

8 files changed (+122 additions, -107 deletions)

View changed files

📝 machine-learning/Dockerfile (+5 -4)
📝 machine-learning/app/config.py (+11 -8)
📝 machine-learning/app/main.py (+5 -18)
📝 machine-learning/app/models/base.py (+1 -0)
➕ machine-learning/log_conf.json (+17 -0)
📝 machine-learning/poetry.lock (+68 -76)
📝 machine-learning/pyproject.toml (+2 -1)
➕ machine-learning/start.sh (+13 -0)

📄 Description

Description

This is a goody bag of memory optimisations that bring down RAM usage considerably.

  • Disabled memory arena for ONNX Runtime
    • This preallocated memory for each model session, increasing RAM usage
  • Use mimalloc
    • WIth the memory arena disabled, mimalloc (also used in microservices) is responsible for memory allocation
  • Optional thread pool
    • For particularly RAM-constrained environments, bringing down job concurrency to 1 and disabling the thread pool led to the greatest savings in RAM
  • Gunicorn
    • Gunicorn consistently had lower RAM usage than using Uvicorn directly, possibly because it's better at freeing request payloads from memory

Model unloading remains disabled by default as it is still leaky even with these optimisations in place.

How Has This Been Tested?

After starting the containers, I simultaneously started each ML job and tracked RAM usage. All told, the default configuration now moves between 1.25-1.4gb, while setting job concurrency to 1 and disabling the thread pool brought it under 1.1gb (with a peak usage of 1.25gb during the initial model load).

🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/immich-app/immich/pull/3934 **Author:** [@mertalev](https://github.com/mertalev) **Created:** 8/31/2023 **Status:** ✅ Merged **Merged:** 9/1/2023 **Merged by:** [@alextran1502](https://github.com/alextran1502) **Base:** `main` ← **Head:** `chore/ml-mem-optimisations` --- ### 📝 Commits (2) - [`b6dc936`](https://github.com/immich-app/immich/commit/b6dc9361efd6ba19253f36ddd5b7a11919fde97c) mimalloc, gunicorn, no mem arena, optional threads - [`81a3c46`](https://github.com/immich-app/immich/commit/81a3c46f076f04f855137243e10911d3b451aa48) formatting ### 📊 Changes **8 files changed** (+122 additions, -107 deletions) <details> <summary>View changed files</summary> 📝 `machine-learning/Dockerfile` (+5 -4) 📝 `machine-learning/app/config.py` (+11 -8) 📝 `machine-learning/app/main.py` (+5 -18) 📝 `machine-learning/app/models/base.py` (+1 -0) ➕ `machine-learning/log_conf.json` (+17 -0) 📝 `machine-learning/poetry.lock` (+68 -76) 📝 `machine-learning/pyproject.toml` (+2 -1) ➕ `machine-learning/start.sh` (+13 -0) </details> ### 📄 Description ## Description This is a goody bag of memory optimisations that bring down RAM usage considerably. - Disabled memory arena for ONNX Runtime * This preallocated memory for each model session, increasing RAM usage - Use mimalloc * WIth the memory arena disabled, mimalloc (also used in microservices) is responsible for memory allocation - Optional thread pool * For particularly RAM-constrained environments, bringing down job concurrency to 1 and disabling the thread pool led to the greatest savings in RAM - Gunicorn * Gunicorn consistently had lower RAM usage than using Uvicorn directly, possibly because it's better at freeing request payloads from memory Model unloading remains disabled by default as it is still leaky even with these optimisations in place. ## How Has This Been Tested? After starting the containers, I simultaneously started each ML job and tracked RAM usage. All told, the default configuration now moves between 1.25-1.4gb, while setting job concurrency to 1 and disabling the thread pool brought it under 1.1gb (with a peak usage of 1.25gb during the initial model load). --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2026-02-05 14:17:03 +03:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
accessibility
changelog:enhancement
changelog:security
changelog:skip
changelog:translation
cli
date-time
dependencies
documentation
external-library
format
good first issue
mobile-beta
mobile-beta
mobile-beta
needs-answer
nice to have
pull-request
Mirrored from GitHub Pull Request
 sharing
tech-debt
📱mobile
🖥️web
🗄️server
🧠machine-learning
No Label pull-request
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
OVERLORD
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: immich-app/immich#10167
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?