So slow, even it knows that. #2278

Closed
opened 2026-02-06 21:55:43 +03:00 by OVERLORD · 2 comments
Owner

Originally created by @IngwiePhoenix on GitHub (Nov 22, 2020).

Describe the bug
Updating a library, letting it scrub for metadata or even just sending HTTP responses, is awfuly slow...

Some excerpts:

[2020-11-22 09:51:09.171 +00:00] [ERR] Failed to retrieve episode images for series "76695"
TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:3]
   at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap)
   at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken)
[2020-11-22 09:51:16.924 +00:00] [ERR] Failed to retrieve episode images for series "76695"
TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:2]
   at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap)
   at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken)
[2020-11-22 09:51:23.214 +00:00] [ERR] Failed to retrieve episode images for series "76695"
TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:1]
   at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap)
   at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken)
   at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken)

(The exceptions themselves are okay - but look at the time inbetween them,,,)

It really feels like the task scheduler is limiting itself a lot. Or is this depending on the disk I/O? In that case, it could actually be that I am using a WebDAV mount as my library source. But sending HTTP requests - or even processing them internally - really shouldn't take so long.

System (please complete the following information):

  • OS: Debian 10
  • Virtualization: None, bare-metal, installation via APT
  • Clients: Browser
  • Browser: Opera GX
  • Jellyfin Version: The latest from the APT repo.
$ dpkg -l | grep jellyfin
ii  jellyfin                          10.6.4-1                                         all          Provides the Jellyfin Free Software Media System
ii  jellyfin-ffmpeg                   4.3.1-1-buster                                   amd64        Tools for transcoding, streaming and playing of multimedia files
ii  jellyfin-server                   10.6.4-1                                         amd64        Jellyfin is the Free Software Media System.
ii  jellyfin-web                      10.6.4-1                                         all          Jellyfin is the Free Software Media System.
  • Playback: Not part of this issue.
  • Installed Plugins: Anime
  • Reverse Proxy: NGINX
  • Base URL: None
  • Networking: Host (dedicated server)
  • Storage: WebDAV mount via davfs2 on $jellyfin_home/mounts/seafile/ with -o rw,uid=jellyfin,gid=jellyfin

To Reproduce
A little convoluted to reproduce, actually. But I'll give you the gist of it.

  1. Set up Seafile (i.e. Docker)
  2. Install davfs2
  3. Use davfs2 to mount your Seafile data
  4. Add a library using the Seafile mount
  5. Try to update the library - it'll be very, very slow when fetching metadata

Expected behavior
A speedy lookup, really. That, and faster HTTP responses. It's incredibly slow sometimes. Right now, it's been five minutes since I sent a trigger to update metadata manually and the circle is still spinning...

Logs
I posted snippets above.

Screenshots
All I could give you is a spinning circle... I don't think you need that :)

Additional context
Maybe my Server hardware?

$ cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 2293.222
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x
topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep
t vpid xsaveopt dtherm arat pln pts
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips        : 6784.40
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 2375.907
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 1
cpu cores       : 2
apicid          : 2
initial apicid  : 2
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x
topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep
t vpid xsaveopt dtherm arat pln pts
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips        : 6784.40
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 2
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 2317.685
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 0
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x
topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep
t vpid xsaveopt dtherm arat pln pts
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips        : 6784.40
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 42
model name      : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz
stepping        : 7
microcode       : 0x28
cpu MHz         : 2402.731
cache size      : 3072 KB
physical id     : 0
siblings        : 4
core id         : 1
cpu cores       : 2
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x
topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep
t vpid xsaveopt dtherm arat pln pts
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit
bogomips        : 6784.40
clflush size    : 64
cache_alignment : 64
address sizes   : 36 bits physical, 48 bits virtual
power management:

$ free -m
              total        used        free      shared  buff/cache   available
Mem:           7885        5334         127         208        2423        2052
Swap:           510         510           0

If you need something else, please let me know!

Originally created by @IngwiePhoenix on GitHub (Nov 22, 2020). **Describe the bug** Updating a library, letting it scrub for metadata or even just sending HTTP responses, is awfuly slow... Some excerpts: ``` [2020-11-22 09:51:09.171 +00:00] [ERR] Failed to retrieve episode images for series "76695" TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:3] at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap) at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken) [2020-11-22 09:51:16.924 +00:00] [ERR] Failed to retrieve episode images for series "76695" TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:2] at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap) at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken) [2020-11-22 09:51:23.214 +00:00] [ERR] Failed to retrieve episode images for series "76695" TvDbSharper.TvDbServerException: The given series ID does not exist; No results for your query: map[AiredSeason:2 EpisodeNumber:1] at TvDbSharper.Infrastructure.Parser.Parse[T](ApiResponse response, IReadOnlyDictionary`2 errorMap) at TvDbSharper.Clients.SeriesClient.GetEpisodesAsync(Int32 seriesId, Int32 page, EpisodeQuery query, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.TryGetValue[T](String key, String language, Func`1 resultFactory) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbClientManager.GetEpisodeTvdbId(Int32 seriesTvdbId, EpisodeQuery episodeQuery, String language, CancellationToken cancellationToken) at MediaBrowser.Providers.Plugins.TheTvdb.TvdbEpisodeImageProvider.GetImages(BaseItem item, CancellationToken cancellationToken) ``` (The exceptions themselves are okay - but look at the time inbetween them,,,) It really feels like the task scheduler is limiting itself a lot. Or is this depending on the disk I/O? In that case, it could actually be that I am using a WebDAV mount as my library source. But sending HTTP requests - or even processing them internally - really shouldn't take so long. **System (please complete the following information):** - OS: Debian 10 - Virtualization: None, bare-metal, installation via APT - Clients: Browser - Browser: Opera GX - Jellyfin Version: The latest from the APT repo. ``` $ dpkg -l | grep jellyfin ii jellyfin 10.6.4-1 all Provides the Jellyfin Free Software Media System ii jellyfin-ffmpeg 4.3.1-1-buster amd64 Tools for transcoding, streaming and playing of multimedia files ii jellyfin-server 10.6.4-1 amd64 Jellyfin is the Free Software Media System. ii jellyfin-web 10.6.4-1 all Jellyfin is the Free Software Media System. ``` - Playback: Not part of this issue. - Installed Plugins: Anime - Reverse Proxy: NGINX - Base URL: None - Networking: Host (dedicated server) - Storage: WebDAV mount via `davfs2` on `$jellyfin_home/mounts/seafile/` with `-o rw,uid=jellyfin,gid=jellyfin` **To Reproduce** A little convoluted to reproduce, actually. But I'll give you the gist of it. 1. Set up Seafile (i.e. Docker) 2. Install `davfs2` 3. Use `davfs2` to mount your Seafile data 4. Add a library using the Seafile mount 5. Try to update the library - it'll be very, very slow when fetching metadata **Expected behavior** A speedy lookup, really. That, and faster HTTP responses. It's incredibly slow sometimes. Right now, it's been five minutes since I sent a trigger to update metadata manually and the circle is still spinning... **Logs** I posted snippets above. **Screenshots** All I could give you is a spinning circle... I don't think you need that :) **Additional context** Maybe my Server hardware? ``` $ cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz stepping : 7 microcode : 0x28 cpu MHz : 2293.222 cache size : 3072 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep t vpid xsaveopt dtherm arat pln pts bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit bogomips : 6784.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz stepping : 7 microcode : 0x28 cpu MHz : 2375.907 cache size : 3072 KB physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 2 initial apicid : 2 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep t vpid xsaveopt dtherm arat pln pts bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit bogomips : 6784.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 2 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz stepping : 7 microcode : 0x28 cpu MHz : 2317.685 cache size : 3072 KB physical id : 0 siblings : 4 core id : 0 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep t vpid xsaveopt dtherm arat pln pts bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit bogomips : 6784.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 3 vendor_id : GenuineIntel cpu family : 6 model : 42 model name : Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz stepping : 7 microcode : 0x28 cpu MHz : 2402.731 cache size : 3072 KB physical id : 0 siblings : 4 core id : 1 cpu cores : 2 apicid : 3 initial apicid : 3 fpu : yes fpu_exception : yes cpuid level : 13 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl x topology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 popcnt tsc_deadline_timer xsave avx lahf_lm epb pti tpr_shadow vnmi flexpriority ep t vpid xsaveopt dtherm arat pln pts bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit bogomips : 6784.40 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: $ free -m total used free shared buff/cache available Mem: 7885 5334 127 208 2423 2052 Swap: 510 510 0 ``` If you need something else, please let me know!
OVERLORD added the bugstale labels 2026-02-06 21:55:43 +03:00
Author
Owner

@ferferga commented on GitHub (Nov 22, 2020):

Have you tried other provider? Server does scanning sequentially instead of parallel, so yes, it takes a lot. We're working to fix that. However, TVDB also plays a key role here, their API is not very good and they have huge response times sometimes. Swicth to TMDB instead and let's see what happens.

@ferferga commented on GitHub (Nov 22, 2020): Have you tried other provider? Server does scanning sequentially instead of parallel, so yes, it takes a lot. We're working to fix that. However, TVDB also plays a key role here, their API is not very good and they have huge response times sometimes. Swicth to TMDB instead and let's see what happens.
Author
Owner

@ferferga commented on GitHub (Dec 1, 2020):

@IngwiePhoenix Any news about this?

@ferferga commented on GitHub (Dec 1, 2020): @IngwiePhoenix Any news about this?
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/jellyfin#2278