Adding a track to a playlist changes all absolute paths to relative paths in m3u8 playlist #975

Closed
opened 2026-02-06 20:19:42 +03:00 by OVERLORD · 20 comments
Owner

Originally created by @SegiH on GitHub (Oct 9, 2019).

Describe the bug
I am using Jellyfin 10.4 on Windows 10 64 bit. I use m3u8 playlists instead of the default XML format because they are easier to work with and back up. I have a music library that points to E:\Music and a folder called Playlists in E:\Music which contains 28 playlist m3u8 files. This has worked perfectly pre 10.4. Before updating to 10.4, when I would add a track to the playlist, if I open the m3u8, you can see that a new entry has been added for the song with the absolute path to the song.

After upgrading to 10.4, when you add a song to an m3u8 playlist, it replaces all absolute paths to all songs in the playlists with a relative path ..\

For example E:\Music\Beethoven\Immortal Beloved\02 Fur Elise.mp3 would be replaced with ..\Beethoven\Immortal Beloved\02 Fur Elise.mp3

It does not do this for only the song that you have just added but every single song in the playlist.

I tested this in the web UI and on Android and the problem exists on both platforms so it is a server related issue.

To Reproduce

  1. Open an m3u8 based playlist and verify that it has absolute paths to the files
  2. Add a track to the playlist in the web UI or in the Android app
  3. Open the same m3u8 based playlist
  4. All absolute paths are replaced with ..\

Expected behavior

I would expect that the server not replace all of the absolute paths with relative ones.

Logs
After replacing the paths with relative ones, an error appears in the server logs for each track in the playlist:

[2019-10-09 10:03:08.339 -07:00] [WRN] Unable to find linked item at path "..\Beethoven\Immortal Beloved\02 Fur Elise.mp3"

Screenshots
N/A

System (please complete the following information):

  • OS: Windows 10 64 bit
  • Browser: Firefox
  • Jellyfin Version: 10.4.0
  • Reverse proxy: None

Additional context

Originally created by @SegiH on GitHub (Oct 9, 2019). **Describe the bug** I am using Jellyfin 10.4 on Windows 10 64 bit. I use m3u8 playlists instead of the default XML format because they are easier to work with and back up. I have a music library that points to E:\Music and a folder called Playlists in E:\Music which contains 28 playlist m3u8 files. This has worked perfectly pre 10.4. Before updating to 10.4, when I would add a track to the playlist, if I open the m3u8, you can see that a new entry has been added for the song with the absolute path to the song. <!-- A clear and concise description of what the bug is. --> After upgrading to 10.4, when you add a song to an m3u8 playlist, it replaces all absolute paths to all songs in the playlists with a relative path ..\ For example E:\Music\Beethoven\Immortal Beloved\02 Fur Elise.mp3 would be replaced with ..\Beethoven\Immortal Beloved\02 Fur Elise.mp3 It does not do this for only the song that you have just added but every single song in the playlist. I tested this in the web UI and on Android and the problem exists on both platforms so it is a server related issue. **To Reproduce** <!-- Steps to reproduce the behavior: --> 1. Open an m3u8 based playlist and verify that it has absolute paths to the files 2. Add a track to the playlist in the web UI or in the Android app 3. Open the same m3u8 based playlist 4. All absolute paths are replaced with ..\ **Expected behavior** <!-- A clear and concise description of what you expected to happen. --> I would expect that the server not replace all of the absolute paths with relative ones. **Logs** After replacing the paths with relative ones, an error appears in the server logs for each track in the playlist: [2019-10-09 10:03:08.339 -07:00] [WRN] Unable to find linked item at path "..\Beethoven\Immortal Beloved\02 Fur Elise.mp3" **Screenshots** N/A **System (please complete the following information):** - OS: Windows 10 64 bit - Browser: Firefox - Jellyfin Version: 10.4.0 - Reverse proxy: None **Additional context** <!-- Add any other context about the problem here. -->
OVERLORD added the bugconfirmedregression labels 2026-02-06 20:19:42 +03:00
Author
Owner

@anthonylavado commented on GitHub (Oct 15, 2019):

Is it this by any chance?
0ccbc2f374/Emby.Server.Implementations/Playlists/PlaylistManager.cs (L410-L437)

Specifically L418, and L436-437


Edit: Maybe it's not? According to the blame, that was already there since last year. Unless this is just showing up now

@anthonylavado commented on GitHub (Oct 15, 2019): Is it this by any chance? https://github.com/jellyfin/jellyfin/blob/0ccbc2f37444d7deb7981a1e6ac2fafc1a518750/Emby.Server.Implementations/Playlists/PlaylistManager.cs#L410-L437 Specifically L418, and L436-437 --- **Edit:** Maybe it's not? According to the blame, that was already there since last year. Unless this is just showing up now
Author
Owner

@JustAMan commented on GitHub (Oct 16, 2019):

This looks very suspicious indeed, and we might've changed something in the code flow (like uncomment some call to a function) which made this particular (unchanged) part to start executing now.

@JustAMan commented on GitHub (Oct 16, 2019): This looks very suspicious indeed, and we might've changed something in the code flow (like uncomment some call to a function) which made this particular (unchanged) part to start executing now.
Author
Owner

@SegiH commented on GitHub (Oct 16, 2019):

This seems to be the more likely explanation. It was working fine in 10.3.6.

Based on the release date of 10.3.6 and looking at the commits to this cs class there doesn't seem to be any significant changes.

Also want to add that in 10.4.0 if you add a song to an m3u8 playlist it removes all songs that were previously in the playlist and replaces it with the song that you have just added so it only has 1 track

@SegiH commented on GitHub (Oct 16, 2019): This seems to be the more likely explanation. It was working fine in 10.3.6. Based on the release date of 10.3.6 and looking at the commits to this cs class there doesn't seem to be any significant changes. Also want to add that in 10.4.0 if you add a song to an m3u8 playlist it removes all songs that were previously in the playlist and replaces it with the song that you have just added so it only has 1 track
Author
Owner

@SegiH commented on GitHub (Oct 21, 2019):

As far as previous playlist items being removed, I wonder if in the method AddToPlaylistInternal in PlaylistManager.cs which looks like this:

     private void AddToPlaylistInternal(string playlistId, IEnumerable<Guid> itemIds, User user, DtoOptions options)
    {
        var playlist = _libraryManager.GetItemById(playlistId) as Playlist;

        if (playlist == null)
        {
            throw new ArgumentException("No Playlist exists with the supplied Id");
        }

        var list = new List<LinkedChild>();

        var items = GetPlaylistItems(itemIds, playlist.MediaType, user, options)
            .Where(i => i.SupportsAddingToPlaylist)
            .ToList();

        foreach (var item in items)
        {
            if (PlaylistHasItem(playlistId, item.Id.ToString()) == false)
            {
                list.Add(LinkedChild.Create(item));
            }
            else
            {
                throw new ArgumentException("The Playlist " + playlist.Name + " already has the item " + item.Name + " with the item ID " + item.Id);
            }
        }

        var newList = playlist.LinkedChildren.ToList();
        newList.AddRange(list);
        playlist.LinkedChildren = newList.ToArray();

        playlist.UpdateToRepository(ItemUpdateType.MetadataEdit, CancellationToken.None);

        if (playlist.IsFile)
        {
            SavePlaylistFile(playlist);
        }

        _providerManager.QueueRefresh(playlist.Id, new MetadataRefreshOptions(new DirectoryService(_logger, _fileSystem))
        {
            ForceSave = true

        }, RefreshPriority.High);
    }

`

Could the line
var items = GetPlaylistItems(itemIds, playlist.MediaType, user, options) .Where(i => i.SupportsAddingToPlaylist) .ToList();
, specifically i.SupportsAddingToPlaylist cause the previous playlist items to be removed from the playlist ?

@SegiH commented on GitHub (Oct 21, 2019): As far as previous playlist items being removed, I wonder if in the method AddToPlaylistInternal in PlaylistManager.cs which looks like this: private void AddToPlaylistInternal(string playlistId, IEnumerable<Guid> itemIds, User user, DtoOptions options) { var playlist = _libraryManager.GetItemById(playlistId) as Playlist; if (playlist == null) { throw new ArgumentException("No Playlist exists with the supplied Id"); } var list = new List<LinkedChild>(); var items = GetPlaylistItems(itemIds, playlist.MediaType, user, options) .Where(i => i.SupportsAddingToPlaylist) .ToList(); foreach (var item in items) { if (PlaylistHasItem(playlistId, item.Id.ToString()) == false) { list.Add(LinkedChild.Create(item)); } else { throw new ArgumentException("The Playlist " + playlist.Name + " already has the item " + item.Name + " with the item ID " + item.Id); } } var newList = playlist.LinkedChildren.ToList(); newList.AddRange(list); playlist.LinkedChildren = newList.ToArray(); playlist.UpdateToRepository(ItemUpdateType.MetadataEdit, CancellationToken.None); if (playlist.IsFile) { SavePlaylistFile(playlist); } _providerManager.QueueRefresh(playlist.Id, new MetadataRefreshOptions(new DirectoryService(_logger, _fileSystem)) { ForceSave = true }, RefreshPriority.High); } ` Could the line ` var items = GetPlaylistItems(itemIds, playlist.MediaType, user, options) .Where(i => i.SupportsAddingToPlaylist) .ToList(); ` , specifically i.SupportsAddingToPlaylist cause the previous playlist items to be removed from the playlist ?
Author
Owner

@SegiH commented on GitHub (Nov 4, 2019):

Downgraded to 10.3.7 a few minutes ago and can confirm that the problem isn't there on 10.3.7

@SegiH commented on GitHub (Nov 4, 2019): Downgraded to 10.3.7 a few minutes ago and can confirm that the problem isn't there on 10.3.7
Author
Owner

@anthonylavado commented on GitHub (Nov 6, 2019):

@SegiH I'm having this issue happen to me in 10.3.7, with a .M3U playlist (on my Mac).

It starts like this:

#EXTM3U
#EXTALB:dark before dawn
#EXTART:ten and tracer
#EXTINF:211,vic chance
/Volumes/Library/Sample Library v1/Music/Ten and Tracer/Dark Before Dawn [8bp025]/8bp025-04-ten-and-tracer-vic-chance.mp3

and then when I add a song using the interface, it changes to:

#EXTM3U
#EXTALB:dark before dawn
#EXTART:ten and tracer
#EXTINF:211,vic chance
Ten and Tracer/Dark Before Dawn [8bp025]/8bp025-04-ten-and-tracer-vic-chance.mp3
#EXTALB:Promo
#EXTART:Binaerpilot
#EXTINF:166,Otosclerosis
Binaerpilot/Promo/10_binaerpilot_-_otosclerosis.mp3
@anthonylavado commented on GitHub (Nov 6, 2019): @SegiH I'm having this issue happen to me in 10.3.7, with a .M3U playlist (on my Mac). It starts like this: ``` #EXTM3U #EXTALB:dark before dawn #EXTART:ten and tracer #EXTINF:211,vic chance /Volumes/Library/Sample Library v1/Music/Ten and Tracer/Dark Before Dawn [8bp025]/8bp025-04-ten-and-tracer-vic-chance.mp3 ``` and then when I add a song using the interface, it changes to: ``` #EXTM3U #EXTALB:dark before dawn #EXTART:ten and tracer #EXTINF:211,vic chance Ten and Tracer/Dark Before Dawn [8bp025]/8bp025-04-ten-and-tracer-vic-chance.mp3 #EXTALB:Promo #EXTART:Binaerpilot #EXTINF:166,Otosclerosis Binaerpilot/Promo/10_binaerpilot_-_otosclerosis.mp3 ```
Author
Owner

@anthonylavado commented on GitHub (Nov 6, 2019):

And I can confirm the same behaviour in the latest version of Emby (4.2.1.0 at the time of writing). This is apparently expected behaviour.

Maybe there is an issue with permissions when you're running Jellyfin 10.3.7, such that it can't modify the playlist file, but it will modify the library database entries? Could you attach a playlist of yours that has this issue?

@anthonylavado commented on GitHub (Nov 6, 2019): And I can confirm the same behaviour in the latest version of Emby (4.2.1.0 at the time of writing). This is apparently expected behaviour. Maybe there is an issue with permissions when you're running Jellyfin 10.3.7, such that it can't modify the playlist file, but it will modify the library database entries? Could you attach a playlist of yours that has this issue?
Author
Owner

@JustAMan commented on GitHub (Nov 6, 2019):

This is apparently expected behaviour.

That is a very strange expected behaviour to see... looks more like unnoticed bug or incorrect design decision to me.

@JustAMan commented on GitHub (Nov 6, 2019): > This is apparently expected behaviour. That is a very _strange_ expected behaviour to see... looks more like unnoticed bug or incorrect design decision to me.
Author
Owner

@anthonylavado commented on GitHub (Nov 6, 2019):

@JustAMan Everywhere I look, it seems that it’s because it can only access things in your library, and thus the paths need to match.

https://emby.media/community/index.php?/topic/73348-m3u-music-playlist/page-3

@anthonylavado commented on GitHub (Nov 6, 2019): @JustAMan Everywhere I look, it seems that it’s because it can only access things in your library, and thus the paths need to match. https://emby.media/community/index.php?/topic/73348-m3u-music-playlist/page-3
Author
Owner

@SegiH commented on GitHub (Nov 6, 2019):

I downgraded from 10.4.0 to 10.3.7 and strangely, I am now seeing this behavior where the tracks are added using a relative path to the m3u8 file.

For example, this is the contents of Classical.m3u8

#EXTM3U
#EXTALB:Immortal Beloved#EXTART:Beethoven, Ludwig van#EXTINF:180,Fur Elise
E:\Music\Beethoven\Immortal Beloved\02 Fur Elise.mp3
#EXTART:Beethoven, Ludwig van#EXTINF:900,Moonlight Sonata
E:\Music\Beethoven\Misc\Moonlight Sonata.mp3
#EXTART:Schubert#EXTINF:113,Moment Musicaux No. 3 in F Minor
E:\Music\Schubert\Moment Musicaux No. 3 in F Minor.mp3
#EXTART:Tchaikovsky#EXTINF:450,Waltz of the Flowers
E:\Music\Tchaikovsky\Waltz of the Flowers.mp3
#EXTART:Vivaldi#EXTINF:2519,Four Seasons
E:\Music\Vivaldi\Four Seasons.mp3
#EXTART:Carly Commando#EXTINF:345,Everyday
E:\Music\Carly Commando\00 Everyday.mp3
#EXTART:Carly Commando#EXTINF:345,Everyday
E:\Music\Carly Commando\00 Everyday.mp3

After adding a track to this m3u8 playlist, all paths that begin with E:\Music\ are replaced with ..\ for all existing tracks as well as the track that was just added.

On 10.4.0, the problem is even worse. When you add a song to a playlist, Jellyfin automatically removes all of the previous tracks and only retains the one that you just added, also with a relative path.

What is strange is that before upgrading to to 10.4.0, tracks to an m3u8 playlist were added with an absolute path not a relative one. It almost seems like after upgrading to 10.4.0 then downgrading back to 10.3.7 causes this bug to happen again. I know for sure that songs were NOT added to an m3u8 with a relative playlist when I was originally on 10.3.7 and before upgrading to 10.4.0. I had edited the m3u8 previously when I was on 10.3.7 (and before upgrading to 10.4) and the paths were always absolute paths.

@SegiH commented on GitHub (Nov 6, 2019): I downgraded from 10.4.0 to 10.3.7 and strangely, I am now seeing this behavior where the tracks are added using a relative path to the m3u8 file. For example, this is the contents of Classical.m3u8 ``` #EXTM3U #EXTALB:Immortal Beloved#EXTART:Beethoven, Ludwig van#EXTINF:180,Fur Elise E:\Music\Beethoven\Immortal Beloved\02 Fur Elise.mp3 #EXTART:Beethoven, Ludwig van#EXTINF:900,Moonlight Sonata E:\Music\Beethoven\Misc\Moonlight Sonata.mp3 #EXTART:Schubert#EXTINF:113,Moment Musicaux No. 3 in F Minor E:\Music\Schubert\Moment Musicaux No. 3 in F Minor.mp3 #EXTART:Tchaikovsky#EXTINF:450,Waltz of the Flowers E:\Music\Tchaikovsky\Waltz of the Flowers.mp3 #EXTART:Vivaldi#EXTINF:2519,Four Seasons E:\Music\Vivaldi\Four Seasons.mp3 #EXTART:Carly Commando#EXTINF:345,Everyday E:\Music\Carly Commando\00 Everyday.mp3 #EXTART:Carly Commando#EXTINF:345,Everyday E:\Music\Carly Commando\00 Everyday.mp3 ``` After adding a track to this m3u8 playlist, all paths that begin with E:\Music\ are replaced with ..\ for all existing tracks as well as the track that was just added. On 10.4.0, the problem is even worse. When you add a song to a playlist, Jellyfin automatically removes all of the previous tracks and only retains the one that you just added, also with a relative path. What is strange is that before upgrading to to 10.4.0, tracks to an m3u8 playlist were added with an absolute path not a relative one. It almost seems like after upgrading to 10.4.0 then downgrading back to 10.3.7 causes this bug to happen again. I know for sure that songs were NOT added to an m3u8 with a relative playlist when I was originally on 10.3.7 and before upgrading to 10.4.0. I had edited the m3u8 previously when I was on 10.3.7 (and before upgrading to 10.4) and the paths were always absolute paths.
Author
Owner

@anthonylavado commented on GitHub (Nov 6, 2019):

I bisected from 10.4.0 all the way back to 10.2.2, setting up entirely new libraries (erasing the whole configuration each time), and this still happened with the playlist samples I posted.

@anthonylavado commented on GitHub (Nov 6, 2019): I bisected from 10.4.0 all the way back to 10.2.2, setting up entirely new libraries (erasing the whole configuration each time), and this still happened with the playlist samples I posted.
Author
Owner

@anthonylavado commented on GitHub (Nov 6, 2019):

@SegiH It would have been interesting to know if when you added music before, if it actually got saved in to the playlist file or not.

@anthonylavado commented on GitHub (Nov 6, 2019): @SegiH It would have been interesting to know if when you added music before, if it actually got saved in to the playlist file or not.
Author
Owner

@SegiH commented on GitHub (Nov 6, 2019):

What do you mean exactly ?

@SegiH commented on GitHub (Nov 6, 2019): What do you mean exactly ?
Author
Owner

@anthonylavado commented on GitHub (Nov 10, 2019):

Investigation Notes

All testing carried out using 10.4.1. I did check 10.3.7 behaviour, see the Conclusion section for details.

Initial Setup

Starting with a clean system, I have imported a music library containing a playlist. The playlist is structured as follows:

#EXTM3U
#EXTALB:Immortal Beloved#EXTART:Beethoven, Ludwig van#EXTINF:180,Fur Elise
/Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3
#EXTART:Beethoven, Ludwig van#EXTINF:900,Moonlight Sonata
/Volumes/Library/Sample/Music/Beethoven/Misc/Moonlight Sonata.mp3
#EXTART:Schubert#EXTINF:113,Moment Musicaux No. 3 in F Minor
/Volumes/Library/Sample/Music/Schubert/Moment Musicaux No. 3 in F Minor.mp3
#EXTART:Tchaikovsky#EXTINF:450,Waltz of the Flowers
/Volumes/Library/Sample/Music/Tchaikovsky/Waltz of the Flowers.mp3
#EXTART:Vivaldi#EXTINF:2519,Four Seasons
/Volumes/Library/Sample/Music/Vivaldi/Four Seasons.mp3
#EXTART:Carly Commando#EXTINF:345,Everyday
/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3
#EXTART:Carly Commando#EXTINF:345,Everyday
/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3

The files are in those respective folders. The playlist is named Classical.m3u8, and is located in /Volumes/Library/Sample/Music/Playlists.

I have verified that the music files in this playlist work as intended.

Library Database

Upon import, the playlist gets copied in to the database. In Library.db, there is a table called "TypedBaseItem". The following query returns all playlists:
SELECT * FROM TypedBaseItems WHERE TYPE = "MediaBrowser.Controller.Playlists.Playlist"

In Library.db, there is a column named Data, where JSON may be stored. In the case of the playlist, it is this:

{
    "Shares": [],
    "IsRoot": false,
    "LinkedChildren": [{
        "Path": "/Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3",
        "Type": "Manual",
        "ItemId": "2edc10e15d1cc7e39b58159644b0dde8"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Beethoven/Misc/Moonlight Sonata.mp3",
        "Type": "Manual",
        "ItemId": "7ee6563913fa5d14fcbbbf497e1412af"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Schubert/Moment Musicaux No. 3 in F Minor.mp3",
        "Type": "Manual",
        "ItemId": "b4a2b8393b72ba7986f7e01c83ce48e5"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Tchaikovsky/Waltz of the Flowers.mp3",
        "Type": "Manual",
        "ItemId": "6ce9beb02a253c9b844df7e0c3a3a801"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Vivaldi/Four Seasons.mp3",
        "Type": "Manual",
        "ItemId": "a7fffc286eb40ba80080d777c44bde67"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3",
        "Type": "Manual",
        "ItemId": "9c958b319088e8c5ee78a0ec7e850436"
    }, {
        "Path": "/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3",
        "Type": "Manual",
        "ItemId": "9c958b319088e8c5ee78a0ec7e850436"
    }],
    "RemoteTrailers": [],
    "IsHD": false,
    "IsShortcut": false,
    "Width": 0,
    "Height": 0,
    "ExtraIds": [],
    "SupportsExternalTransfer": false
}

Adding A New Song, different root folder

I added a new song from a different root folder. At this point, the following happened:

  • The playlist file is modified to have relative paths (e.g. Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3 becomes ../Beethoven/Immortal Beloved/02 Fur Elise.mp3)
  • The paths are also updated in the JSON in the database
  • The ItemId entries for all playlist files get nulled (e.g. 2edc10e15d1cc7e39b58159644b0dde8 to 00000000000000000000000000000000)

Viewing the Playlist

When you go to view the playlist in the browser, it is empty. This is because of the null ItemId. When the ItemId is null, the server doesn't know which folder to start from for the relative path search. As a result, the entry is considered "Invalid". In the logs/console, this is confirmed with a log entry for each file, indicating "Unable to find linked item at path". Attempting to play the playlist in this broken state just results in a "PlaybackError" with message "PlaybackErrorNoCompatibleStream".

Recovery Method

If the ItemId for the songs are inserted back in to the database JSON, the item will return in the playlist. This is presumably because it has a full absolute path (in the column named "Path").

The Library Becomes The Truth

If you attempt to fix the issue by putting absolute paths back in the playlist file, this will not affect the entry in the database. This is because the system considers the database version to be the authoritative copy, and the disk one to be simply a backup.

Adding a song, again, deletes data

If you continue to add another song to the playlist, all the now invalid entries (the original list, the first song added), are removed. This happens to the playlist entry in the library, and also to the file on disk!

This is presumably because the invalid entries are considered not to exist, and playlists are written to when updated. At this point, your original playlist is lost, and it is not possible to restore it.


An Alternate Scenario

When a new song is added from the Music library, to the playlist, there are two different scenarios to consider:

  1. The new song is added from a path that only requires going forward (it is in the same folder, or a subfolder)
  2. The new song is added from a path that requires going up a folder at any point

Note that in each case, the playlist is written to when it is updated.

Scenario 1

In this case, assume the following structure:

.
└── MyMedia
    └── Music
        ├── Coldplay
        │   └── Parachutes
        │       └── Yellow.mp3
        ├── Home
        │   └── Odyssey
        │       └── Resonance.mp3
        └── Playlist.m3u

The playlist may have an absolute path in it:

/MyMedia/Music/Coldplay/Parachutes/Yellow.mp3

When the song "Resonance" is added, the paths are written, relative to the playlist:

Coldplay/Parachutes/Yellow.mp3
Home/Odyssey/Resonance.mp3

This will work, and will not result in the blanking of ItemId in the database. Adding further songs from subfolders will work.

Scenario 2

In this case, assume the following structure:

.
└── MyMedia
    └── Music
        ├── Coldplay
        │   └── Parachutes
        │       └── Yellow.mp3
        ├── Home
        │   └── Odyssey
        │       └── Resonance.mp3
        └── Playlists
            └── Playlist.m3u

The playlist may have an absolute path in it:

/MyMedia/Music/Coldplay/Parachutes/Yellow.mp3

When the song "Resonance" is added, the paths are written relative to the playlist:

../Coldplay/Parachutes/Yellow.mp3
../Home/Odyssey/Resonance.mp3

This will fail.
This results in the blanking of ItemId in the database, and further songs will also fail, resulting in the wiping of "invalid" entries from the playlist file on disk.


Conclusion

There are several issues at hand:

  • Playlists are written to when updated, but in a destructive fashion (it modifies the existing valid content)
  • Adding paths to a playlist that requires going up at least one folder results in bad item links in the database (blank ItemId)
  • If not recovered by the user, adding more items results in data loss (the playlist file is cleared of "invalid entries")

Rewriting the paths to be relative appears to be inherited code from Emby (see forum post).

However, in 10.3.7, adding an item does not blank the ItemId in the database. This behaviour appears to be new to versions 10.4.0+. As a result, playlists are breaking.

What changed that caused this? Was it the change in database journalling? Was it an unrelated change in a query?

@anthonylavado commented on GitHub (Nov 10, 2019): # Investigation Notes All testing carried out using 10.4.1. I did check 10.3.7 behaviour, see the Conclusion section for details. ## Initial Setup Starting with a clean system, I have imported a music library containing a playlist. The playlist is structured as follows: ``` #EXTM3U #EXTALB:Immortal Beloved#EXTART:Beethoven, Ludwig van#EXTINF:180,Fur Elise /Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3 #EXTART:Beethoven, Ludwig van#EXTINF:900,Moonlight Sonata /Volumes/Library/Sample/Music/Beethoven/Misc/Moonlight Sonata.mp3 #EXTART:Schubert#EXTINF:113,Moment Musicaux No. 3 in F Minor /Volumes/Library/Sample/Music/Schubert/Moment Musicaux No. 3 in F Minor.mp3 #EXTART:Tchaikovsky#EXTINF:450,Waltz of the Flowers /Volumes/Library/Sample/Music/Tchaikovsky/Waltz of the Flowers.mp3 #EXTART:Vivaldi#EXTINF:2519,Four Seasons /Volumes/Library/Sample/Music/Vivaldi/Four Seasons.mp3 #EXTART:Carly Commando#EXTINF:345,Everyday /Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3 #EXTART:Carly Commando#EXTINF:345,Everyday /Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3 ``` The files are in those respective folders. The playlist is named `Classical.m3u8`, and is located in `/Volumes/Library/Sample/Music/Playlists`. I have verified that the music files in this playlist work as intended. ## Library Database Upon import, the playlist gets copied in to the database. In `Library.db`, there is a table called "TypedBaseItem". The following query returns all playlists: `SELECT * FROM TypedBaseItems WHERE TYPE = "MediaBrowser.Controller.Playlists.Playlist"` In `Library.db`, there is a column named `Data`, where JSON may be stored. In the case of the playlist, it is this: ``` { "Shares": [], "IsRoot": false, "LinkedChildren": [{ "Path": "/Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3", "Type": "Manual", "ItemId": "2edc10e15d1cc7e39b58159644b0dde8" }, { "Path": "/Volumes/Library/Sample/Music/Beethoven/Misc/Moonlight Sonata.mp3", "Type": "Manual", "ItemId": "7ee6563913fa5d14fcbbbf497e1412af" }, { "Path": "/Volumes/Library/Sample/Music/Schubert/Moment Musicaux No. 3 in F Minor.mp3", "Type": "Manual", "ItemId": "b4a2b8393b72ba7986f7e01c83ce48e5" }, { "Path": "/Volumes/Library/Sample/Music/Tchaikovsky/Waltz of the Flowers.mp3", "Type": "Manual", "ItemId": "6ce9beb02a253c9b844df7e0c3a3a801" }, { "Path": "/Volumes/Library/Sample/Music/Vivaldi/Four Seasons.mp3", "Type": "Manual", "ItemId": "a7fffc286eb40ba80080d777c44bde67" }, { "Path": "/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3", "Type": "Manual", "ItemId": "9c958b319088e8c5ee78a0ec7e850436" }, { "Path": "/Volumes/Library/Sample/Music/Carly Commando/00 Everyday.mp3", "Type": "Manual", "ItemId": "9c958b319088e8c5ee78a0ec7e850436" }], "RemoteTrailers": [], "IsHD": false, "IsShortcut": false, "Width": 0, "Height": 0, "ExtraIds": [], "SupportsExternalTransfer": false } ``` ## Adding A New Song, different root folder I added a new song from a different root folder. At this point, the following happened: * The playlist file is modified to have relative paths (e.g. `Volumes/Library/Sample/Music/Beethoven/Immortal Beloved/02 Fur Elise.mp3` becomes `../Beethoven/Immortal Beloved/02 Fur Elise.mp3`) * The paths are also updated in the JSON in the database * The ItemId entries for all playlist files get nulled (e.g. `2edc10e15d1cc7e39b58159644b0dde8` to `00000000000000000000000000000000`) ### Viewing the Playlist When you go to view the playlist in the browser, it is empty. This is because of the null ItemId. When the ItemId is null, the server doesn't know which folder to start from for the relative path search. As a result, the entry is considered "Invalid". In the logs/console, this is confirmed with a log entry for each file, indicating "Unable to find linked item at path". Attempting to play the playlist in this broken state just results in a "PlaybackError" with message "PlaybackErrorNoCompatibleStream". ### Recovery Method If the ItemId for the songs are inserted back in to the database JSON, the item will return in the playlist. This is presumably because it has a full absolute path (in the column named "Path"). ### The Library Becomes The Truth If you attempt to fix the issue by putting absolute paths back in the playlist file, this will not affect the entry in the database. This is because the system considers the database version to be the authoritative copy, and the disk one to be simply a backup. ### Adding a song, again, deletes data If you continue to add another song to the playlist, all the now invalid entries (the original list, the first song added), are removed. This happens to the playlist entry in the library, and **also to the file on disk!** This is presumably because the invalid entries are considered not to exist, and playlists are written to when updated. At this point, your original playlist is lost, and it is not possible to restore it. ----- # An Alternate Scenario When a new song is added from the Music library, to the playlist, there are two different scenarios to consider: 1. The new song is added from a path that only requires going forward (it is in the same folder, or a subfolder) 2. The new song is added from a path that requires going up a folder at any point Note that in each case, **the playlist is written to when it is updated**. ## Scenario 1 In this case, assume the following structure: ``` . └── MyMedia └── Music ├── Coldplay │   └── Parachutes │   └── Yellow.mp3 ├── Home │   └── Odyssey │   └── Resonance.mp3 └── Playlist.m3u ``` The playlist may have an absolute path in it: ``` /MyMedia/Music/Coldplay/Parachutes/Yellow.mp3 ``` When the song "Resonance" is added, the paths are written, relative to the playlist: ``` Coldplay/Parachutes/Yellow.mp3 Home/Odyssey/Resonance.mp3 ``` This will work, and will not result in the blanking of ItemId in the database. Adding further songs from subfolders will work. ## Scenario 2 In this case, assume the following structure: ``` . └── MyMedia └── Music ├── Coldplay │   └── Parachutes │   └── Yellow.mp3 ├── Home │   └── Odyssey │   └── Resonance.mp3 └── Playlists └── Playlist.m3u ``` The playlist may have an absolute path in it: ``` /MyMedia/Music/Coldplay/Parachutes/Yellow.mp3 ``` When the song "Resonance" is added, the paths are written relative to the playlist: ``` ../Coldplay/Parachutes/Yellow.mp3 ../Home/Odyssey/Resonance.mp3 ``` **This will fail.** This results in the blanking of ItemId in the database, and further songs will also fail, resulting in the wiping of "invalid" entries from the playlist file on disk. ---- # Conclusion There are several issues at hand: * Playlists are written to when updated, but in a destructive fashion (it modifies the existing valid content) * Adding paths to a playlist that requires going up at least one folder results in bad item links in the database (blank ItemId) * If not recovered by the user, adding more items results in data loss (the playlist file is cleared of "invalid entries") Rewriting the paths to be relative appears to be inherited code from Emby (see [forum post](https://emby.media/community/index.php?/topic/73348-m3u-music-playlist/page-2)). However, in 10.3.7, adding an item does not blank the ItemId in the database. This behaviour appears to be new to versions 10.4.0+. As a result, playlists are breaking. What changed that caused this? Was it the change in database journalling? Was it an unrelated change in a query?
Author
Owner

@anthonylavado commented on GitHub (Jan 7, 2020):

Using this query to search for commits that might have affected this: is:pr is:closed merged:2019-07-24..2019-11-04 base:master

@anthonylavado commented on GitHub (Jan 7, 2020): Using this query to search for commits that might have affected this: `is:pr is:closed merged:2019-07-24..2019-11-04 base:master`
Author
Owner

@anthonylavado commented on GitHub (Jan 8, 2020):

So in this section of code:
76b4ba3c5e/Emby.Server.Implementations/Playlists/PlaylistManager.cs (L462-L465)

If I take line 464 and change it to:

return itemPath;

Then the playlist modifications succeed.

@anthonylavado commented on GitHub (Jan 8, 2020): So in this section of code: https://github.com/jellyfin/jellyfin/blob/76b4ba3c5e65cd6034c86df6f937fbfaa9e371be/Emby.Server.Implementations/Playlists/PlaylistManager.cs#L462-L465 If I take line 464 and change it to: ```c# return itemPath; ``` Then the playlist modifications succeed.
Author
Owner

@dkanada commented on GitHub (Jan 8, 2020):

Now we need to think of the ramifications from changing that line for other playlists.

@dkanada commented on GitHub (Jan 8, 2020): Now we need to think of the ramifications from changing that line for other playlists.
Author
Owner

@anthonylavado commented on GitHub (Jan 8, 2020):

@dkanada That's true. This was just part of the investigation. Testing this in 10.3.7 just now confirms that the file paths are made relative, exactly as what happens in 10.4.x. It seems that starting in the newer version, it can't find the files/make the relative path, so then it "loses" it, and thus the item is scheduled for removal.

@anthonylavado commented on GitHub (Jan 8, 2020): @dkanada That's true. This was just part of the investigation. Testing this in 10.3.7 just now confirms that the file paths are made relative, exactly as what happens in 10.4.x. It seems that starting in the newer version, it can't find the files/make the relative path, so then it "loses" it, and thus the item is scheduled for removal.
Author
Owner

@SegiH commented on GitHub (Jan 8, 2020):

Great to hear that you found a possible solution

@SegiH commented on GitHub (Jan 8, 2020): Great to hear that you found a possible solution
Author
Owner

@anthonylavado commented on GitHub (Jan 8, 2020):

https://github.com/jellyfin/jellyfin/pull/1168

In versions prior to 10.4.z, under MakeAbsolutePath, this is the try for the update:
595a68b822/Emby.Server.Implementations/IO/ManagedFileSystem.cs (L126-L131)

In 10.4.z, this is the same code:
d8c3b26fa6/Emby.Server.Implementations/IO/ManagedFileSystem.cs (L108-L111)
This is the source of the issue with file paths.

@anthonylavado commented on GitHub (Jan 8, 2020): https://github.com/jellyfin/jellyfin/pull/1168 In versions prior to 10.4.z, under `MakeAbsolutePath`, this is the `try` for the update: https://github.com/jellyfin/jellyfin/blob/595a68b822654b03c32a35060bf97234dbc6ef7c/Emby.Server.Implementations/IO/ManagedFileSystem.cs#L126-L131 In 10.4.z, this is the same code: https://github.com/jellyfin/jellyfin/blob/d8c3b26fa686e440912a45d82bb9866b4b66822c/Emby.Server.Implementations/IO/ManagedFileSystem.cs#L108-L111 This is the source of the issue with file paths.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: starred/jellyfin#975