More about Special Characters

[MarmiteSandwich]

I am creating playlists from iTunes, using a utility which claims to know about Unicode and has an option to cover this (java -jar itunesexport.jar -useM3U8Ext). I have many artists or albums which contain these characters, which then find their way into folder and file names, obviously. I have been using Minimserver under Windows for years and the .m3u8 extension for playlist files works splendidly.

Now I am migrating the Minimserver to Linux (Turnkey - Debian 8), along with the data and playlists from the Windows environment. Minimserver will find and serve up all of these files, no problem. The playlists with special characters work for most files, but some characters seem to cause problems and the Minimserver log on scanning says things like "Error: playlist Music/_Playlists/StrangeCharacters.m3u8: unknown folder for ../Rokia Traoré/Bowmboï/01 M'Bifo.m4a". Other files in the same playlist with special characters in the artist/album name play ok, e.g. "../Toumani Diabate's Symmetric Orchestra/Boulevard De L'Indépendance/02 Boulevard de l'Indépendance.m4a".

Some characters seem to be more special than others. All suggestions welcome.
Marmite.

[simoncn]

This might be an issue with NFC (composed) vs NFD (decomposed) encodings of Unicode characters in filenames and playlists. See this thread for some information about this. As you will see from the referenced thread, I have made changes to MinimServer that fix some cases of this but there might be other cases that are still causing issues.

[MarmiteSandwich]

(25-04-2017 20:12)simoncn Wrote:  This might be an issue with NFC (composed) vs NFD (decomposed) encodings of Unicode characters in filenames and playlists. See this thread for some information about this. As you will see from the referenced thread, I have made changes to MinimServer that fix some cases of this but there might be other cases that are still causing issues.

Dunno if this helps, but it doesn't seem to be consistent. This entry in the playlist doesn't work:

Code:

../Meshell Ndegéocello/The Spirit Music Jamia_ Dance Of The Inf/06 The Chosen.mp3
or
../Meshell Ndegéocello/The Spirit Music Jamia_ Dance Of The Inf/06 The Chosen.mp3

The second version is how Wordpad displays it, vs notepad for the first. This entry in a playlist works fine:

Code:

../Toumani Diabate's Symmetric Orchestra/Boulevard De L'Indépendance/02 Boulevard de l'Indépendance.m4a
or
../Toumani Diabate's Symmetric Orchestra/Boulevard De L'Indépendance/02 Boulevard de l'Indépendance.m4a

Both entries contain the same accented character, which is not very special, which suggests there is some other issue. I will continue looking for clues.
Marmite

[MarmiteSandwich]

(25-04-2017 20:54)MarmiteSandwich Wrote:  I will continue looking for clues.
Marmite

OK, I eliminated that particular inconsistency by deleting the offending folder in the Linux instance and recreating it from the Windows original. But the inconsistency remains with these 2:

works:

Code:

../Meshell Ndegéocello/The Spirit Music Jamia_ Dance Of The Inf/06 The Chosen.mp3

doesn't work:

Code:

../Rokia Traoré/Tchamantché/08 Tchamantché.m4a

Both folders on the Linux instance have been deleted and recreated from the original and contain the same accented character. Other entries which don't work are:

Code:

../Rubén González/Introducing...Rubén González/04 Melodía Del Rio.m4a
../João Gilberto/João Voz E Violão/05 Desafinado.mp3
../Antônio Carlos Jobim/Antônio Carlos Jobim And Friends/04 Once I Loved.m4a
../Cesária Évora/São Vicente/12 Negue.m4a
../Rokia Traoré/Bowmboï/01 M'Bifo.m4a

The following entries work:

Code:

../Camarón de la Isla/Grands Cantaores du Flamenco/01 Bulerías de la perla.m4a
../Carlos Montoya/Spanish Guitar Magic/2-07 Malagueñas Boleras.mp3
../Gotan Project/Lunático/05 Lunático.mp3
../Orchestra Baobab/Specialist In All Styles/03 Dée Moo Wóor.m4a
../Vieux Farka Touré/Fondo/02 Aï Haïra.m4a

Stumped.

[simoncn]

From your reference to Notepad and Wordpad, it sounds like you are looking on Windows for the cause of the problem. I suggest you look on Linux instead as the files might not have arrived there in the same condition that they left Windows.

Running the ls -l command from a Terminal window (to show actual filenames) and the cat command (to show paths in .m3u8 files) can be helpful. Also, check the start of the MinimServer log to see what platform default charset is being used.

When posting paths that work or don't work, it would be helpful to say whether they are taken from running cat on a .m3u8 file or ls -l on a directory.

[MarmiteSandwich]

I will do this. First I plan to reformat the Linux disk into a Linux native format, not FAT32 as at present. This might take some time...

[simoncn]

(26-04-2017 09:31)MarmiteSandwich Wrote:  I will do this. First I plan to reformat the Linux disk into a Linux native format, not FAT32 as at present. This might take some time...

The disk format will not make any difference to your issue.

It is likely that the playlist entries are correct and the issue is with the filenames on the Linux machine. I suggest you take one of the files or folders that doesn't match the playlist entry and run the command

ls -l >~/output.txt

from the containing folder to get the actual characters in the file or folder name. If you attach the output.txt file to a post here, I will take a look.

[MarmiteSandwich]

(26-04-2017 09:56)simoncn Wrote:  It is likely that the playlist entries are correct and the issue is with the filenames on the Linux machine. I suggest you take one of the files or folders that doesn't match the playlist entry and run the command

ls -l >~/output.txt

from the containing folder to get the actual characters in the file or folder name. If you attach the output.txt file to a post here, I will take a look.

OK, here are all the errors, along with the line from the playlist which caused the error.

Code:

Error: playlist _Playlists/Blues & Ballads.m3u8: unknown folder for ../Rubén González/Introducing...Rubén González/04 Melodía Del Rio.m4a
../Rubén González/Introducing...Rubén González/04 Melodía Del Rio.m4a
Error: playlist _Playlists/Blues & Ballads.m3u8: unknown folder for ../João Gilberto/João Voz E Violão/05 Desafinado.mp3
../João Gilberto/João Voz E Violão/05 Desafinado.mp3
Error: playlist _Playlists/CoversForBabyBoomers.m3u8: unknown folder for ../Rokia Traoré/Tchamantché/09 The Man I Love.m4a
../Rokia Traoré/Tchamantché/09 The Man I Love.m4a
Error: playlist _Playlists/Herts.m3u8: unknown folder for ../Antônio Carlos Jobim/Antônio Carlos Jobim And Friends/04 Once I Loved.m4a
../Antônio Carlos Jobim/Antônio Carlos Jobim And Friends/04 Once I Loved.m4a
Error: playlist _Playlists/Herts.m3u8: unknown folder for ../Rokia Traoré/Bowmboï/10 Bowmboï.m4a
../Rokia Traoré/Bowmboï/10 Bowmboï.m4a
Error: playlist _Playlists/New2015_16.m3u8: unknown folder for ../Cesária Évora/São Vicente/12 Negue.m4a
../Cesária Évora/São Vicente/12 Negue.m4a

And I am attaching the best output.txt file I could manage. It shows the highest level folders (artist), within which the album folder and track file reside. I couldn't actually open any of them, because I couldn't find a way of typing the special characters into PuTTY which would work.

Investigation much appreciated.
Marmite

[simoncn]

Please attach the output.txt file. You might have forgotten to click the Add Attachment button.

[MarmiteSandwich]

(27-04-2017 18:05)simoncn Wrote:  Please attach the output.txt file. You might have forgotten to click the Add Attachment button.

Quite so.