[PR #13791] [CLOSED] Parallelize library.db migration steps, minor fixes #13710

Closed
opened 2026-02-07 07:17:49 +03:00 by OVERLORD · 0 comments
Owner

📋 Pull Request Information

Original PR: https://github.com/jellyfin/jellyfin/pull/13791
Author: @ferferga
Created: 3/28/2025
Status: Closed

Base: masterHead: perf-library-migration-2


📝 Commits (8)

  • 0ab8ce7 Fix incorrect logging of UserData entries
  • eed90f3 Perform all leftover cleanup first
  • 5572c27 Run every parallelizable library migration step in tasks
  • cdb1e50 Defer committing to the end of library.db migration
  • a5dc983 Fix elapsed time not being calculated correctly for base items, processed entities not corresponding to the real ones
  • c55f6c9 Fix context disposed on commit, disable change tracking
  • db37f36 Save BaseItem changes early on
  • 8b02315 Further clarify initial cleanup message

📊 Changes

1 file changed (+274 additions, -184 deletions)

View changed files

📝 Jellyfin.Server/Migrations/Routines/MigrateLibraryDb.cs (+274 -184)

📄 Description

This PR further reduces the migration time from 04:26 (achieved through #13749) to 03:45 in my Raspberry Pi by parallelizing everything that can be executed in parallel.

The downside is that this increase a little the RAM usage because all the tasks are performed as early as possible and flushing changes to the DB is deferred to the end. The biggest one though (BaseItems) is flushed to the DB as soon as possible. I don't think this will ever be a problem though, since my bottleneck now is I/O bound, and we can assume a correlation between the library size and better I/O speeds for flushing to the database (and, as soon as changes are flushed, space is freed). My 4 GB Pi 4 could handle it with a lot of headroom.

Also took the chance to fix the logger messages (for people entries, MediaStreamInfos count was shown instead, for example) and improve the wording (in my opinion) of the logged messages.


🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.

## 📋 Pull Request Information **Original PR:** https://github.com/jellyfin/jellyfin/pull/13791 **Author:** [@ferferga](https://github.com/ferferga) **Created:** 3/28/2025 **Status:** ❌ Closed **Base:** `master` ← **Head:** `perf-library-migration-2` --- ### 📝 Commits (8) - [`0ab8ce7`](https://github.com/jellyfin/jellyfin/commit/0ab8ce7cb095eef46608e93d0319a7dd1959a59b) Fix incorrect logging of UserData entries - [`eed90f3`](https://github.com/jellyfin/jellyfin/commit/eed90f39ce101a3aee5cbb753893c53f824ea17d) Perform all leftover cleanup first - [`5572c27`](https://github.com/jellyfin/jellyfin/commit/5572c27eb6c355ea111635c6c37dec5db7d2d098) Run every parallelizable library migration step in tasks - [`cdb1e50`](https://github.com/jellyfin/jellyfin/commit/cdb1e508a8a5ddc0fb8419b81c637bb97b0f65a6) Defer committing to the end of library.db migration - [`a5dc983`](https://github.com/jellyfin/jellyfin/commit/a5dc983cec4a3afcd9235b384d58e0d10f69a992) Fix elapsed time not being calculated correctly for base items, processed entities not corresponding to the real ones - [`c55f6c9`](https://github.com/jellyfin/jellyfin/commit/c55f6c925456a05257fb0936e6673bc3d0a8a9fd) Fix context disposed on commit, disable change tracking - [`db37f36`](https://github.com/jellyfin/jellyfin/commit/db37f36ce0a8b38ad7396527c318f32ebda587dd) Save BaseItem changes early on - [`8b02315`](https://github.com/jellyfin/jellyfin/commit/8b0231520d02228af48216976b0d3a182ed0e711) Further clarify initial cleanup message ### 📊 Changes **1 file changed** (+274 additions, -184 deletions) <details> <summary>View changed files</summary> 📝 `Jellyfin.Server/Migrations/Routines/MigrateLibraryDb.cs` (+274 -184) </details> ### 📄 Description This PR further reduces the migration time from 04:26 (achieved through #13749) to **03:45** in my Raspberry Pi by parallelizing everything that can be executed in parallel. The downside is that this increase a little the RAM usage because all the tasks are performed as early as possible and flushing changes to the DB is deferred to the end. The biggest one though (BaseItems) is flushed to the DB as soon as possible. I don't think this will ever be a problem though, since my bottleneck now is I/O bound, and we can assume a correlation between the library size and better I/O speeds for flushing to the database (and, as soon as changes are flushed, space is freed). My 4 GB Pi 4 could handle it with a lot of headroom. Also took the chance to fix the logger messages [(for people entries, ``MediaStreamInfos`` count was shown instead, for example)](https://github.com/jellyfin/jellyfin/blob/master/Jellyfin.Server/Migrations/Routines/MigrateLibraryDb.cs#L275) and improve the wording (in my opinion) of the logged messages. --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>
OVERLORD added the pull-request label 2026-02-07 07:17:49 +03:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/jellyfin#13710