mirror of
https://github.com/jellyfin/jellyfin.git
synced 2026-07-16 05:43:54 +03:00
GetOptimalVideoStream decides on Transcode, but WebOS client able to play Remux and Direct Stream #6778
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @quoushimitch on GitHub (Feb 28, 2025).
Description of the bug
I've discovered certain video files in my media library that cause odd behavior when I try to play them. To preface: I understand that the webOS client has many compromises compared to an Android STB or Apple TV. But working within the boundaries of the video/audio/subtitle codecs that LG officially supports through their web APIs, I have been curating my library to only include files that I believe to be compatible with Direct Play. The video files in question which transcode all have the same MediaInfo general encoding parameters (part of series), shown below:
MediaInfo for Transcoded Videos
` GeneralUnique ID : 1234 (0x1234)
Complete name : /data/Media/Video#1.mkv
Format : Matroska
Format version : Version 4
File size : 6.90 GiB
Duration : 24 min 10 s
Overall bit rate mode : Variable
Overall bit rate : 41.1 Mb/s
Frame rate : 23.976 FPS
Movie name : S01E01 - Title
Encoded date : 2025-01-27 10:46:37 UTC
Writing application : mkvmerge v88.0 ('All I Know') 64-bit
Writing library : libebml v1.4.5 + libmatroska v1.7.1
Cover : Yes
Attachments : Bahnschrift.ttf / Acme-Regular.ttf / ArchitectsDaughter-Regular.ttf / Audiowide-Regular.ttf / augie.tff / AVENIRNEXTROUNDEDPRO-DEMI.OTF / BadScript-Regular.ttf / BalsamiqSans-Bold.ttf / BalsamiqSans-Regular.ttf / BESTAGON-ITALIC.OTF / BESTAGON-REGULAR.OTF / BESTAGON-THICK ITALIC.OTF / BESTAGON-THICK.OTF / BESTAGON-THIN ITALIC.OTF / BESTAGON-THIN.OTF / BilboSwashCaps-Regular.ttf / Caveat-VariableFont_wght (1).ttf / Caveat-VariableFont_wght.ttf / CHOPPERRR.otf / CraftyGirls-Regular.ttf / CURLZ__.TTF / FFFFORWA.TTF / GHelveticaNeueLTStd-LtEx.otf / GochiHand-Regular.ttf / HERCULES.TTF / Kalam-Regular.ttf / KleeOne-Regular.ttf / LATO-EXTRABOLDITALIC.TTF / LATO-EXTRABOLD_0.TTF / LATO-EXTRABOLD_1.TTF / LovedbytheKing-Regular.ttf / Martel-Regular.ttf / Merriweather-Bold.ttf / NOODLESOUP.TTF / PatrickHand-Regular.ttf / PermanentMarker-Regular.ttf / PlayfairDisplay-VariableFont_wght.ttf / Proxima Nova Extrabold.otf / RabioHead.ttf / Ramabhadra-Regular.ttf / ReenieBeanie-Regular.ttf / ROBOTO-REGULAR_0.TTF / SF-UI-Display-Regular.otf / Sofia_Pro_Soft_Regular.ttf / SqueakyChalkSound-ZG8.ttf / Sriracha-Regular.ttf / SUBWT___.ttf / TavernloreBB-Regular.otf / Bahnschrift.ttf / Acme-Regular.ttf / ArchitectsDaughter-Regular.ttf / Audiowide-Regular.ttf / augie.tff / AVENIRNEXTROUNDEDPRO-DEMI.OTF / BadScript-Regular.ttf / BalsamiqSans-Bold.ttf / BalsamiqSans-Regular.ttf / BESTAGON-ITALIC.OTF / BESTAGON-REGULAR.OTF / BESTAGON-THICK ITALIC.OTF / BESTAGON-THICK.OTF / BESTAGON-THIN ITALIC.OTF / BESTAGON-THIN.OTF / BilboSwashCaps-Regular.ttf / Caveat-VariableFont_wght (1).ttf / Caveat-VariableFont_wght.ttf / CHOPPERRR.otf / CraftyGirls-Regular.ttf / CURLZ__.TTF / FFFFORWA.TTF / GHelveticaNeueLTStd-LtEx.otf / GochiHand-Regular.ttf / HERCULES.TTF / Kalam-Regular.ttf / KleeOne-Regular.ttf / LATO-EXTRABOLDITALIC.TTF / LATO-EXTRABOLD_0.TTF / LATO-EXTRABOLD_1.TTF / LovedbytheKing-Regular.ttf / Martel-Regular.ttf / Merriweather-Bold.ttf / NOODLESOUP.TTF / PatrickHand-Regular.ttf / PermanentMarker-Regular.ttf / PlayfairDisplay-VariableFont_wght.ttf / Proxima Nova Extrabold.otf / RabioHead.ttf / Ramabhadra-Regular.ttf / ReenieBeanie-Regular.ttf / ROBOTO-REGULAR_0.TTF / SF-UI-Display-Regular.otf / Sofia_Pro_Soft_Regular.ttf / SqueakyChalkSound-ZG8.ttf / Sriracha-Regular.ttf / SUBWT___.ttf / TavernloreBB-Regular.otf / Sofia_Pro_Soft_Regular.ttf / SqueakyChalkSound-ZG8.ttf / Sriracha-Regular.ttf / SUBWT___.ttf / TavernloreBB-Regular.otf / courbd.ttf / Cucho Bold_1.otf / Mijas-Ultra_0.otf / 000GoombellaTB-Regular.ttf / AstounderSquaredBB.otf / giaothong2.ttf / 000SkippySharpiCiel-Regular.ttf / 000BillyTheFlyingRobotLCBB-Regular.otf / SVN-Avo bold.ttf / iCielPequena.otf / 000 Augie [TeddyBear].ttf / 000 10 Cent Comics [TeddyBear].ttf / 000 Akbar [TeddyBear].ttf / 000 Chinacat [TeddyBear].ttf / 000 Clairvoyant BB [TeddyBear].ttf / 000 Blambot Pro Lite TB.ttf / 000 CCGabrielBautistaLito TB.ttf / SVN-Bree Regular.ttf / SVN-Amsi Narw Bold.ttf / SVN-Bali Script.ttf / iciel Cadena.ttf / UVNVietSach.TTF / SVN-Bear.ttf / UVNAnhHai_R.TTF / 9Slide03-Jura-Medium_1.ttf / A-otfMaruFolkProM.otf / A-otfハッピーnStdR.otf / A-otfハッピーnStdR.otf / Aqua_pfont-str.ttf / Bahnschrift.ttf / Bahnschrift-Bold.ttf / BrokeStone.otf / DftegakimegumiStdW3.otf / EverythingRises.ttf / FiraSansMedium.ttf / FiraSansMediumItalic.ttf / Fot-カラットStdUb-Black.otf / HelveticaLtBold.ttf / NotoSans.ttf / NotoSansMedium.ttf / Notosans-regular-mary.ttf / SegoeUi.ttf / SegoeUiSemibold-SemiBold.ttf / Acme-Regular.ttf / ArchitectsDaughter-Regular.ttf / Audiowide-Regular.ttf / augie.tff / AVENIRNEXTROUNDEDPRO-DEMI.OTF / BadScript-Regular.ttf / BalsamiqSans-Bold.ttf / BalsamiqSans-Regular.ttf / BESTAGON-ITALIC.OTF / BESTAGON-REGULAR.OTF / BESTAGON-THICK ITALIC.OTF / BESTAGON-THICK.OTF / BESTAGON-THIN ITALIC.OTF / BESTAGON-THIN.OTF / BilboSwashCaps-Regular.ttf / Caveat-VariableFont_wght (1).ttf / Caveat-VariableFont_wght.ttf / CHOPPERRR.otf / CraftyGirls-Regular.ttf / CURLZ__.TTF / FFFFORWA.TTF / GHelveticaNeueLTStd-LtEx.otf / GochiHand-Regular.ttf / HERCULES.TTF / Kalam-Regular.ttf / KleeOne-Regular.ttf / LATO-EXTRABOLD_0.TTF / LATO-EXTRABOLD_1.TTF / LATO-EXTRABOLDITALIC.TTF / LovedbytheKing-Regular.ttf / Martel-Regular.ttf / Merriweather-Bold.ttf / NOODLESOUP.TTF / PatrickHand-Regular.ttf / PermanentMarker-Regular.ttf / PlayfairDisplay-VariableFont_wght.ttf / Proxima Nova Extrabold.otf / RabioHead.ttf / Ramabhadra-Regular.ttf / ReenieBeanie-Regular.ttf / ROBOTO-REGULAR_0.TTF / SF-UI-Display-Regular.otf / GandhiSans-Italic.otf / GandhiSans-Regular.otf / cover.jpg / arial.ttf / arialbd.ttf / comic.ttf / comicbd.ttf / CONSOLA.ttf / CONSOLAB.ttf / Roboto-Medium.ttf / Roboto-MediumItalic.ttf / times.ttf / timesbd.ttf / trebuc.ttf / trebucbd.ttf / verdana.ttf / verdanab.ttf / tahoma.ttf / times.ttf / trebuc.ttf / verdana.ttf / AdobeArabic-Bold.otf / Arial_Black.ttf / arialbd.ttf / comic.ttf / cour.ttf / georgia.ttf / impact.ttf / RixLoveFool.ttf / GandhiSans-Bold.otf / GandhiSans-BoldItalic.otf / NotoSansJP-Bold.ttf / Ampersand.ttf / Arial.ttf / DjbChalkItUp.TTF / DkDownwardFall.TTF / Dotf1.ttf / FaracoHand.TTF / Fixedsysttf-SemiBold.ttf / Grain.ttf / HelveticaNeue.otf / Panton-Bold.OTF
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Format settings, Slice count : 4 slices per frame
Codec ID : V_MPEG4/ISO/AVC
Duration : 24 min 1 s
Bit rate mode : Variable
Bit rate : 36.7 Mb/s
Maximum bit rate : 39.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.738
Time code of first frame : 01:00:00:00
Stream size : 6.16 GiB (89%)
Default : Yes
Forced : No
Audio #1
ID : 2
Format : PCM
Format settings : Little / Signed
Codec ID : A_PCM/INT/LIT
Duration : 24 min 1 s
Bit rate mode : Constant
Bit rate : 2 304 kb/s
Channel(s) : 2 channels
Sampling rate : 48.0 kHz
Frame rate : 200.000 FPS (240 SPF)
Bit depth : 24 bits
Stream size : 396 MiB (6%)
Title : Commentary
Language : English
Default : No
Forced : No
Audio #2
ID : 3
Format : FLAC
Format/Info : Free Lossless Audio Codec
Codec ID : A_FLAC
Duration : 24 min 1 s
Bit rate mode : Variable
Bit rate : 1 346 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 11.719 FPS (4096 SPF)
Bit depth : 24 bits
Compression mode : Lossless
Stream size : 231 MiB (3%)
Title : Japanese / FLAC / 2.0 / 48 kHz / 1346 kbps
Writing library : libFLAC 1.4.3 (2023-06-23)
Language : Japanese
Default : Yes
Forced : No
MD5 of the unencoded content : D1234
Audio #3
ID : 4
Format : E-AC-3
Format/Info : Enhanced AC-3
Commercial name : Dolby Digital Plus
Codec ID : A_EAC3
Duration : 24 min 0 s
Bit rate mode : Constant
Bit rate : 224 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 31.250 FPS (1536 SPF)
Compression mode : Lossy
Delay relative to video : 7 ms
Stream size : 38.5 MiB (1%)
Title : English / E-AC-3 / 2.0 / 48 kHz / 224 kbps
Language : English
Service kind : Complete Main
Default : No
Forced : No
Text #1
ID : 5
Format : ASS
Codec ID : S_TEXT/ASS
Codec ID/Info : Advanced Sub Station Alpha
Duration : 17 min 55 s
Bit rate : 16.7 kb/s
Frame rate : 0.617 FPS
Count of elements : 663
Compression mode : Lossless
Stream size : 2.14 MiB (0%)
Title : English / Forced / ASS
Language : English
Default : No
Forced : Yes
Text #2
ID : 6
Format : ASS
Codec ID : S_TEXT/ASS
Codec ID/Info : Advanced Sub Station Alpha
Duration : 23 min 49 s
Bit rate : 12.7 kb/s
Frame rate : 0.707 FPS
Count of elements : 1011
Compression mode : Lossless
Stream size : 2.16 MiB (0%)
Title : English / Full / ASS
Language : English
Default : Yes
Forced : No
Menu
00:00:00.000 : en:Prologue
...
`
For these files, the web player lists "Reason for Transcoding" as "There was an error starting direct playback". The debug logs show an attempt at Direct Play, but a result of "DirectPlayError". From my research, this only happens when there's a bug in Jellyfin OR the browser advertises a codec in its profile that it doesn't actually support. Originally, I thought this might have been a file with some odd unsupported codec combination. However, I found that the file also plays as a Direct Stream by disabling the user setting "Allow video playback that requires transcoding". But it turns out, this isn't even required as a simple edit to the jellyfin-web build causes the backend to return a Remux. It seems something I don't understand is happening in the SetDeviceSpecificData function for this to happen. The only difference? "allowVideoStreamCopy=false" is omitted from the client request parameters, due to, forcing EnableDirectStream and AllowVideoStreamCopy to TRUE in the jellyfin-web bundle.
Log attached is for the default case, where the video transcodes without any modification.
Please see my initial forum post for more information, including images showing the different playback states.
I'm curious to learn the logic behind why forcing "no transcode" in user settings results in a different output type than my web bundle hack, and why neither is able to Direct Play (maybe the TV really just can't do it?). Regardless, these are more desirable than transcoding and should be fixed!
Reproduction steps
What is the current bug behavior?
The video will take longer to start than usual, and when it starts, it will have a "Play Method" of "transcoding", Video codec of "HEVC" (I enabled transcode to HEVC), and Audio codec of "FLAC (direct)".
What is the expected correct behavior?
The source video codec of H264 High (AVC), subtitles (ASS), and audio codec(s) (FLAC, E-AC-3, and PCM) are all supported by LG for WebOS apps. The video should Direct Play - but workarounds at least get it to Remux instead of Transcode.
Jellyfin Server version
10.10.0+
Specify commit id
No response
Specify unstable release number
No response
Specify version number
No response
Specify the build version
10.11.0 (bd73283bf-dirty)
Environment
Jellyfin logs
FFmpeg logs
Client / Browser logs
Unable to capture client logs without remote debugging - see post for images of Playback Info popup
Relevant screenshots or videos
No response
Additional information
I've attached the MediaInfo for a file that contains a similar encoding, but which DIrect Play works without any modifications:
MediaInfo
GeneralUnique ID : 5678 (0x5678)
Complete name : /data/Media/Video#2.mkv
Format : Matroska
Format version : Version 4
File size : 6.66 GiB
Duration : 24 min 20 s
Overall bit rate mode : Variable
Overall bit rate : 38.6 Mb/s
Frame rate : 23.976 FPS
Encoded date : 2025-01-15 10:48:19 UTC
Writing application : mkvmerge v89.0 ('And the Melody Still Lingers On (Night in Tunisia)') 64-bit
Writing library : libebml v1.4.5 + libmatroska v1.7.1
Attachments : arial.ttf / arialbd.ttf / comic.ttf / comicbd.ttf / CONSOLA.ttf / CONSOLAB.ttf / Dragon-Serial-Heavy-Regular.ttf / ebrimabd.ttf / FOT-GMaruGoPro-B.otf / FOT-UDMarugo_LargePr6-B.otf / Kan2EngGSZS3.otf / MyriadPro (Bold).otf / nobilty.ttf / nyala.ttf / pastel crayon.ttf / PostAntiqua-SemiBold.ttf / puffmod1.ttf / Rakoon Personal Use.otf / Rambla.otf / Roboto-Medium.ttf / Roboto-MediumItalic.ttf / Rubik-Regular_0.ttf / Rubik-SemiBoldItalic-NEW.ttf / Rubik-SemiBold-NEW.ttf / RyoGothicPlusN-Regular.otf / Salsa-Regular.otf / segoeui.ttf / squeaky chalk sound.ttf / Stoobs.otf / times.ttf / timesbd.ttf / trebuc.ttf / trebucbd.ttf / VAGRoundedBT-Regular.otf / verdana.ttf / verdanab.ttf / Vogel Normal.ttf
Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings : CABAC / 4 Ref Frames
Format settings, CABAC : Yes
Format settings, Reference frames : 4 frames
Format settings, Slice count : 4 slices per frame
Codec ID : V_MPEG4/ISO/AVC
Duration : 24 min 40 s
Bit rate mode : Constant
Bit rate : 39.0 Mb/s
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.784
Time code of first frame : 01:00:00:00
Stream size : 6.41 GiB (96%)
Title : Title
Default : Yes
Forced : No
Audio
ID : 2
Format : FLAC
Format/Info : Free Lossless Audio Codec
Codec ID : A_FLAC
Duration : 24 min 40 s
Bit rate mode : Variable
Bit rate : 1 320 kb/s
Channel(s) : 2 channels
Channel layout : L R
Sampling rate : 48.0 kHz
Frame rate : 10.417 FPS (4608 SPF)
Bit depth : 24 bits
Compression mode : Lossless
Stream size : 233 MiB (3%)
Title : Jpn FLAC 2.0
Writing library : Lavf61.5.101
Language : Japanese
Default : Yes
Forced : No
MD5 of the unencoded content : B5678
Text #1
ID : 3
Format : ASS
Codec ID : S_TEXT/ASS
Codec ID/Info : Advanced Sub Station Alpha
Duration : 24 min 19 s
Bit rate : 8 621 b/s
Frame rate : 3.074 FPS
Count of elements : 4488
Compression mode : Lossless
Stream size : 1.50 MiB (0%)
Title : English Title
Language : English
Default : Yes
Forced : No
Text #2
ID : 4
Format : ASS
Codec ID : S_TEXT/ASS
Codec ID/Info : Advanced Sub Station Alpha
Duration : 23 min 9 s
Bit rate : 169 b/s
Frame rate : 0.293 FPS
Count of elements : 407
Compression mode : Lossless
Stream size : 28.8 KiB (0%)
Title : English Two
Language : English
Default : No
Forced : No
Menu
00:00:00.000 : ja:Intro
@quoushimitch commented on GitHub (Feb 28, 2025):
Hey @felix920506, I agree that there will likely also need to be a report in the jellyfin-web repo, but the backend returning Remux and/or Direct Stream inconsistently is why I reported it here. The client only changes one web parameter related to copying - nothing about direct stream is passed from the client to the backend.
@gnattu commented on GitHub (Feb 28, 2025):
My theory: It is mkv. It always has been mkv on webOS causing problems. Because the client think this file can be directly played but it actually cannot, playback conservatively fallback to full transcode because the client has no idea what is going wrong with the player, just like you have no idea either. Can you remux the file to mp4 manually to see if it direct plays? I think it should.
@quoushimitch commented on GitHub (Feb 28, 2025):
You were on the spot with this one, it did direct play when I remuxed into mp4. So I suppose this issue really consists of multiple: (1) Jellyfin accepts that LG supports this from an API call, but LG is seemingly wrong. This issue is probably a jellyfin-web issue. (2) When a request comes in to try to transcode, the best profile for a situation where only a remux is needed should be a remux. But for some reason, the backend chooses to transcode.
Thanks for helping narrow this down.
@gnattu commented on GitHub (Feb 28, 2025):
Wrong and wrong. The situation here is simple. The device profile of webOS supports mkv (and it really does for a lot of videos, but some of them won't play like yours). And when it does not play as it normally would, the player simply crashed and the client will simply disable direct play and remux to force a full transcode as the safest fallback because the problem is not detectable when the player crashed. It is not a jellyfin-web issue nor backend issue. There is no "best profile" because both remux and direct play is fully disabled in this case. There is no "backend chooses to transcode", it is client requires it to be transcoded to recover from a playback failure.
@quoushimitch commented on GitHub (Feb 28, 2025):
I would respect that being the case, but even if all you have mentioned is true, then I still think there is an oversight with how the transcode is chosen. This is likely an issue for jellyfin-web, so I am trying to clarify before I open an issue there. When the client "crashes", it should not immediately resort to a full transcode. This is what I understood isAlreadyFallbacking to protect against. At this stage, it should still allow video copy and direct stream, and ideally the remux is chosen and played. Only once the transcode attempt fails should it attempt a full transcode. Is this sound and expected behavior?
This still doesn't explain why Jellyfin chooses a Remux when allowVideoStreamCopy is changed to true (check made here) compared to why a Direct Stream is chosen instead when the server user config disallows transcoding. Once that is resolved, I think there is no more work that could be done on the backend for this issue.
@gnattu commented on GitHub (Feb 28, 2025):
It should. This is the most unideal case and in this case the client should try to make that video to still play, and not ideally because this is already not an ideal state. We only fallback to remux for livetv streams because those streams are known to be unreliable and a lot of browsers will fail to direct play those anyway, but we are not doing the same for local videos because when a local video crashed a player for most of the time a remux cannot fix it, and just fail twice is just sacrificing other user's use case for your use case because fail twice will spawn ffmpeg multiple times and the loading time will also be increased.
The server has a lot of "undefined" behavior when transcoding is disabled. That option was buggy for years and it does not even prevent video transcoding for all cases if the client request is weird enough to skip all checks in the stream builder. I would assume the audio copy is disabled somewhere during the pipeline but I have to investigate deep enough for that. But this is still irrelevant for your problem. There are a lot of fallbacks implemented during emby time which is still affecting us because its random transcoding fallbacks caused a LOT of undefined behavior for our playback API. Probably I should just fail all playback request went into the HLS controller if transcoding is disabled for safety, but I don't want to break current clients and facing the complaint for numerous users relying on current behavior.
@jellyfin-bot commented on GitHub (Jun 29, 2025):
This issue has gone 120 days without an update and will be closed within 21 days if there is no new activity. To prevent this issue from being closed, please confirm the issue has not already been fixed by providing updated examples or logs.
If you have any questions you can use one of several ways to contact us.
@jellyfin-bot commented on GitHub (Jul 20, 2025):
This issue was closed due to inactivity.