mirror of
https://github.com/jellyfin/jellyfin.git
synced 2026-05-04 18:09:12 +03:00
Server unavailable, invalid response, too many open files in system #1032
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @n-peugnet on GitHub (Nov 5, 2019).
Describe the bug
Impossible to connect to Jellyfin while the application is still running.
To Reproduce
I don't really know how or when the problem appeared
Expected behavior
The app should be accessible
Logs
Full log: https://public.club1.fr/other/jellyfin_011.log (the too many open files error starts appearing at line 12391)
Screenshots
System (please complete the following information):
Additional context
@JustAMan commented on GitHub (Nov 22, 2019):
When this thing happens next time please try to see which files are opened by Jellyfin server (e.g. by
ls -l /proc/JELLYFIN_PID/fdwhereJELLYFIN_PIDis the PID of Jellyfin server).@zeldatp151 commented on GitHub (Feb 23, 2020):
I am also having this issue, restarting the server temporarily fixes it for a few hours.
Here is a paste of the output requested https://pastebin.com/U4yiJw7c
note that I restarted the server at midnight
@JustAMan commented on GitHub (Feb 25, 2020):
So it'd loads of sockets left open... now I wonder what sockets are those. @cvium @Bond-009 do you have any ideas?
@PrplHaz4 commented on GitHub (Feb 25, 2020):
Bonkers - if you disable the DLNA server and Automatic Port Mapping (Networking) does it still happen? Is there anything interesting in the logs prior to the Failed to bind to port 1900 error?
Looks like it might be trying to find an announce port but can't get one for whatever reason...
@Bond-009 commented on GitHub (Feb 25, 2020):
@JustAMan Well we keep every HttpClient we ever use...
@JustAMan commented on GitHub (Feb 26, 2020):
So what
shows when it has so many handles open?
@zeldatp151 commented on GitHub (Feb 26, 2020):
I am currently away from home on travel, but I will try to get the requested information as soon as possible.
@zeldatp151 commented on GitHub (Mar 3, 2020):
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
jellyfin 873 jellyfin cwd unknown /proc/873/cwd (readlink: Permission denied)
jellyfin 873 jellyfin rtd unknown /proc/873/root (readlink: Permission denied)
jellyfin 873 jellyfin txt unknown /proc/873/exe (readlink: Permission denied)
jellyfin 873 jellyfin NOFD /proc/873/fd (opendir: Permission denied)
Also, I want to mention that the issue is getting worse, I increased my open file limit originally to 500k, and it worked ok for about 3 days, then I restarted the server, and it was broken within a day, that is when I made my first post here about it. after that I increased my limit to 999999, and it worked fine for a week, and now it works for a few hours after a reboot.
@JustAMan commented on GitHub (Mar 3, 2020):
This
lsofdoesn't showing what I expected...Try this:
@zeldatp151 commented on GitHub (Mar 3, 2020):
Here is the pastebin for that output, I have sanitized my username and system names.
https://pastebin.com/HGEWk2Dj
@JustAMan commented on GitHub (Mar 4, 2020):
There seem to be slightly less than 500 files open, and nothing that stands out, which is strange.
Can you let it grind to "too many open files", then gather same
lsofthingy?..@zeldatp151 commented on GitHub (Mar 7, 2020):
OK, So pastebin wouldn't let me paste this becaues it was too large, and it broke justpasteme, os here it is on privatebin, just know that this will expire in 23 hours, and it takes a minute to load
https://privatebin.net/?ce604df1ed866350#9F4ZMhCeJtwFQjnsSpRUKpRDG3gVUedYH4mL3XLRcMXk
I'm also now sure that the issue is caused by the anidb agent. I really don't want to disable that plugin, because it works much better than tvdb for a lot of stuff, but I probably need to for now.
Edit: here is one on pastebin (https://pastebin.com/SyAkabNh). I trimmed ~1000 lines off the end to get it under the limit for pastebin, all of the trimmed lines are some veriation of :
jellyfin 2368 jellyfin 2653u IPv4 38064946 0t0 TCP :35757->anidb-prod.anidb.net:80 (CLOSE_WAIT)
https://pastebin.com/SyAkabNh
@JustAMan commented on GitHub (Mar 10, 2020):
Is it possible for you to uninstall Anime plugin? I guess some of those issues might be related to that.
@zeldatp151 commented on GitHub (Mar 10, 2020):
I could, but I would really prefer not to, a large portion of my library gets it's metadata via the anime plugin, as the material is either not on tvdb, or it is wrong on tvdb.
@JustAMan commented on GitHub (Mar 11, 2020):
Let's try uninstalling it temporarily and see if it's related to that. If it is, we're closing at pinpointing the issue.
@AnotherDevlop commented on GitHub (Apr 13, 2020):
Same problem here. Although it doesn't seems directly related to aniDB, since I have open a lot of socket files to a unknown ip too. However, I will disable aniDB and see if happen, and report here in a couple of days.
Logs
Problem seems to be on line 2008, "System.IO.IOException: Function not implemented" can be ignored safely since the fs, where movies are, is readonly.
jellyfin_log_sanitized.txt
jellyfin_ls_fd_log.txt
jellyfin_lsof_log.txt
System:
@zeldatp151 commented on GitHub (Apr 13, 2020):
I should add that mine stopped having the issue with the anime plugin update a few weeks ago.
Or it has at least stopped crashing, I should check the open files to see if anything weird is going on still.
On Sun, Apr 12, 2020 at 10:06 PM, AnotherDevlopnotifications@github.com wrote:
Same problem here. Although it doesn't seems directly related to aniDB, since I have open a lot of socket files to a unknown ip too. However, I will disable aniDB and see if happen, and report here in a couple of days.
Logs
Problem seems to be on line 2008, "System.IO.IOException: Function not implemented" can be ignored safely since the fs, where movies are, is readonly.
jellyfin_log_sanitized.txt
jellyfin_ls_fd_log.txt
jellyfin_lsof_log.txt
System:
OS: Ubuntu server 18.04
Browser: Firefox
Jellyfin Version: 10.4.3.0
Reverse proxy: apache
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or unsubscribe.
@JustAMan commented on GitHub (Apr 13, 2020):
This indeed seems the Anime plugin problem
@jellyfin/backend can we have a look at it and see if it's not properly disposing connections? There's hundreds of those in
CLOSE_WAITstate there.@AnotherDevlop commented on GitHub (Apr 14, 2020):
Like I said, there is the report without anime plugin. Still happens, now the culprit is opensubtitles plugin.
Logs
jellyfin_log_2_sanitized.txt
jellyfin_ls_fd_log_2.txt
jellyfin_lsof_log_2.txt
@stale[bot] commented on GitHub (Aug 12, 2020):
This issue has gone 120 days without comment. To avoid abandoned issues, it will be closed in 21 days if there are no new comments.
If you're the original submitter of this issue, please comment confirming if this issue still affects you in the latest release or nightlies, or close the issue if it has been fixed. If you're another user also affected by this bug, please comment confirming so. Either action will remove the stale label.
This bot exists to prevent issues from becoming stale and forgotten. Jellyfin is always moving forward, and bugs are often fixed as side effects of other changes. We therefore ask that bug report authors remain vigilant about their issues to ensure they are closed if fixed, or re-confirmed - perhaps with fresh logs or reproduction examples - regularly. If you have any questions you can reach us on Matrix or Social Media.