Tags used as Sort Order Only ( Feature Request? )

[sidepipe]

Firstly, thanks Simon for giving us the almost perfect media server - but more than that for being so responsive to users... certain other authors could learn a thing or two

Wasn't sure where to post this, so it ended up in "General", feel free to move it!

There have been various threads about sort order etc, and I see that there has now been a kludge implemented to ignore "the" for example. I like artists ordered by surname where appropriate.. my current "solution" is to use the album artist tag to create my sort field, so I'd tag "Amos, Tori" for example. This isn't ideal though, because I don't like them showing up in the list that way.

It seems to me that the simple answer to all this would be to introduce a list of tags used to sort albums/artists which is just that - a sort list which does not get displayed itself but is just used for an ORDERBY or some such. We can already index custom tags and so I'm supposing that it wouldn't be that tricky to implement.

Unfortunately the best example here is iTunes, much as I dislike it It already has "Sort Artist" and "Sort Album Artist" tags ( which I think are actually soar and soaa ) which do this very thing... it even auto creates Sort Artist without the "The" if appropriate. So, in this example, you would add soaa and soar to the list of custom tags to index, and then specify the album sort order as something like "soaa,soar". If both are empty or not present, then Minim should fall back to using its previous fields.

It seems to me that this would give a generic and fairly easy to implement way of sorting based on user preferences, without resorting to separate files etc. You would not need the kludge for "The" any more either.

Apologies if this has been suggested before - even more so if its implemented and I didn't know . I'm very new to Minimserver, coming here because I gave up trying to get Twonky to sort stuff as I wanted, and this is actually the final piece in the jigsaw.

[simoncn]

(05-01-2013 08:47)sidepipe Wrote:  Firstly, thanks Simon for giving us the almost perfect media server - but more than that for being so responsive to users... certain other authors could learn a thing or two

Wasn't sure where to post this, so it ended up in "General", feel free to move it!

There have been various threads about sort order etc, and I see that there has now been a kludge implemented to ignore "the" for example. I like artists ordered by surname where appropriate.. my current "solution" is to use the album artist tag to create my sort field, so I'd tag "Amos, Tori" for example. This isn't ideal though, because I don't like them showing up in the list that way.

It seems to me that the simple answer to all this would be to introduce a list of tags used to sort albums/artists which is just that - a sort list which does not get displayed itself but is just used for an ORDERBY or some such. We can already index custom tags and so I'm supposing that it wouldn't be that tricky to implement.

Unfortunately the best example here is iTunes, much as I dislike it It already has "Sort Artist" and "Sort Album Artist" tags ( which I think are actually soar and soaa ) which do this very thing... it even auto creates Sort Artist without the "The" if appropriate. So, in this example, you would add soaa and soar to the list of custom tags to index, and then specify the album sort order as something like "soaa,soar". If both are empty or not present, then Minim should fall back to using its previous fields.

It seems to me that this would give a generic and fairly easy to implement way of sorting based on user preferences, without resorting to separate files etc. You would not need the kludge for "The" any more either.

Apologies if this has been suggested before - even more so if its implemented and I didn't know . I'm very new to Minimserver, coming here because I gave up trying to get Twonky to sort stuff as I wanted, and this is actually the final piece in the jigsaw.

Thanks for this suggestion. There has been quite a bit of discussion about this on the Linn forums, and this is a good opportunity for me to give a summary on this forum of how things stand.

I'm aware of the "separate sort tag" approach uses by iTunes and some other servers. This provides a lot of flexibility, but it requires all the tags to be entered twice. Doing this initially and then maintaining it could be quite a bit of extra work.

I've chosen a different approach for MinimServer. This isn't yet fully implemented, though I've already done quite a lot of work in the codebase to provide the necessary underpinnings to make this work efficiently.

The chosen approach is for any tag to have:

• an index value, which is shown in the MinimServer index for browsing
• a sort value, which is used by the MinimServer index for sorting
• a display value, which is sent by MinimServer to the control point for displaying in the playlist, Now Playing screen, etc.
  

By default, all these values are the same. MinimServer will provide various customizations to allow any one of these to be different from the other two. This means you'll be able to have:

index value == sort value, different display value
index value == display value, different sort value
display value == sort value, different index value

One of the customizations supported will be "lastNameFirst". This means you'll be able to tag an artist as "Amos, Tori" and have this used as the sort value. For the index value and/or the display value, you can apply the "lastNameFirst" transformation, and MinimServer will automatically transform this to "Tori Amos".

Another customization will be "ignoreThe". The means you'll be able to tag an artist as "The Beatles" and have that used as the display value and index value. For the sort value, you can apply the "ignoreThe" transformation, and MinimServer will automatically transform this to "Beatles, The".

Edit: updated the previous paragraph.

Note: As you'll see from this, the current treatment of "ignoreThe" isn't a kludge, but a first step towards a much more general solution to the sort order problem.

It will also be possible to use an explicit tag to set the sort value. For example, a FLAC file could contain the tags
ARTIST=Tori Amos
ARTISTSORT=Amos, Tori
which would set the sort value to "Amos, Tori" and the display value and index value to "Tori Amos". For mp4 and ALAC files, the corresponding iTunes tags would be used.

One problem with the explicit tagging approach is that it's possible to have different files with the same ARTIST tags but with different ARTISTSORT tags. I haven't decided yet how MinimServer should handle this. My feeling is that this would be an oversight by the user, rather than a deliberate intention. MinimServer could either report the conflict and ignore both ARTISTSORT tags, or could choose the first one it sees and ignore the other(s).

Is it possible for this conflict situation to arise in iTunes? If so, how does iTunes handle it?

[sidepipe]

Hi Simon,

It sounds as though I will be able to do what I wanted once you have completed this work. As for the work of entering the tags twice my thoughts would be:

1) As long as the sort order falls back to the normal Album Artist/Artist if the additional custom tags aren't present, then this is only true for the exceptions, not for all tags.

2) Some software such as iTunes already does some of the work ( creating the "The" exception. )

3) There are plenty of scriptable tag editors which could create these tags external to the server.

Whilst I am far from an expert on tagging etc, I have quite a bit of software design experience... and my opinion is that you should keep things as simple as possible, and keep them to the function that they are intended for. IMHO it would be far better to keep the server side as simple as possible. As soon as you start building rules in to the server, you also need a million exceptions to those rules... you will have a never ending stream of "can it do this" or "it should do that!" Having said that, it's your baby, not mine

As for how iTunes handles conflicts... I believe that it doesn't try. Each album ( indeed each track ) is treated individually, so if you have two albums whose sort fields are different, they will show up in different places in the listing, even though the Album Artist ( and hence the list entry ) is the same. So, sticking with Tori Amos, your Artists listing would look like ( for example: )

The Alan Parsons Project
Tori Amos
The Art of Noise
...
Texas
Tori Amos
Travis

The two Tori Amos entries would have different albums under them based on which ones generated the particular artist list entry. As you say, this would be the fault/choice of the user rather than the job of the server to fix.

[simoncn]

(05-01-2013 10:03)sidepipe Wrote:  Hi Simon,

It sounds as though I will be able to do what I wanted once you have completed this work. As for the work of entering the tags twice my thoughts would be:

1) As long as the sort order falls back to the normal Album Artist/Artist if the additional custom tags aren't present, then this is only true for the exceptions, not for all tags.

2) Some software such as iTunes already does some of the work ( creating the "The" exception. )

3) There are plenty of scriptable tag editors which could create these tags external to the server.

Whilst I am far from an expert on tagging etc, I have quite a bit of software design experience... and my opinion is that you should keep things as simple as possible, and keep them to the function that they are intended for. IMHO it would be far better to keep the server side as simple as possible. As soon as you start building rules in to the server, you also need a million exceptions to those rules... you will have a never ending stream of "can it do this" or "it should do that!" Having said that, it's your baby, not mine

As for how iTunes handles conflicts... I believe that it doesn't try. Each album ( indeed each track ) is treated individually, so if you have two albums whose sort fields are different, they will show up in different places in the listing, even though the Album Artist ( and hence the list entry ) is the same. So, sticking with Tori Amos, your Artists listing would look like ( for example: )

The Alan Parsons Project
Tori Amos
The Art of Noise
...
Texas
Tori Amos
Travis

The two Tori Amos entries would have different albums under them based on which ones generated the particular artist list entry. As you say, this would be the fault/choice of the user rather than the job of the server to fix.

Thanks for these comments. I think I understand your perspective, but I take a somewhat different view. I get more questions and problem reports concerning tagging issues than anything else, and I think for the great majority of users it's extremely important to keep the tagging in their library as simple and consistent as possible. MinimServer is designed around this philosophy and does not take the view that tagging mistakes by the user should result in confusing or inconsistent output when browsing the library.

MinimServer is also designed for high performance, and allowing the same artist to appear in multiple positions in a sorted list would require some design changes that would harm performance. I'm fine with making design changes to provide useful functionality, but I'm not keen to do this to work around a situation created by faulty tagging.

[bbrip]

Simon, I am absolutely with you here, and thing all that double tagging approach taken by iTunes just to get the sorting right hasn't been thought out correctly, certainly not the greatest flash of geniality on display by late Steve Your approach is significantly more efficient, avoiding duplicate tagging efforts for all fields you may want to sort differently then you display them.

Keep on along those lines. Looking forward to the next release already

[Manul]

I'd like to add a vote for building some kind of support for iTunes' way of handling sort order into MinimServer. It may not be the best solution, but I guess I'm probably not the only one who has already put some effort into getting the sort order right via iTunes and I would be very happy if I didn't have to duplicate that effort.

As for conflicting entries for the same artist, I guess there's three ways of handling them:

• Just take the first entry for the sort order and ignore the others,
• ignore all sort order entries in case of conflicting ones,
• or just treat conflicting entries as separate artists which happen to have the same display name.
  

My personal favourite would be the last solution, which I guess could be easy to implement, but I could live with all of them, since I'd try to sort out conflicting entries anyway.

[bbrip]

To be honest, I believe iTunes / Apple have got this sorting thing seriously wrong. So why copy or cater for a poor implementation? They may right this wrong eventually.

Pressure is on Apple these days which is usually good for customers. As long as they were the heros they had no reason to listen.

bbrip

[simoncn]

(03-02-2013 21:23)bbrip Wrote:  To be honest, I believe iTunes / Apple have got this sorting thing seriously wrong. So why copy or cater for a poor implementation? They may right this wrong eventually.

Pressure is on Apple these days which is usually good for customers. As long as they were the heros they had no reason to listen.

bbrip

I agree that Apple have got this wrong, and I'm not planning to copy it.

The MinimServer approach that I'm working on will enable each artist to have:

• a value in the ARTIST tag
• a value for display on the control point (the display value)
• a value for display in the index (the index value)
• a value for sorting the index entry (the sort value)
  

The ARTIST tag value is the "primary key" for the artist, and it won't be possible for two artists to have the same "primary key" and have differences in the other values. This constraint is needed to enable an efficient implementation in MinimServer and provide a good experience for the user.

Subject to this constraint, there will be various options for how the other values are provided or derived. One of these options will be iTunes-style tagging.

[Manul]

(03-02-2013 22:03)simoncn Wrote:  The MinimServer approach that I'm working on will enable each artist to have:

• a value in the ARTIST tag
• a value for display on the control point (the display value)
• a value for display in the index (the index value)
• a value for sorting the index entry (the sort value)
  

The ARTIST tag value is the "primary key" for the artist, and it won't be possible for two artists to have the same "primary key" and have differences in the other values. This constraint is needed to enable an efficient implementation in MinimServer and provide a good experience for the user.

Subject to this constraint, there will be various options for how the other values are provided or derived. One of these options will be iTunes-style tagging.

Thanks for clearing this up, this is close to how I understood your approach - I was just thinking the "primary key" would be some kind of internal index, independent from the four visible attributes.

Your approach sounds very good though, and as long as there is a way to derive these values from existing iTunes tags, I'll probably be quite happy with whatever you come up with.

[rjplummer]

You need to consider the case of multiple artists. Consider "Chester & Lester" by Chet Atkins and Les Paul. Most people would like to see this album under both "A" & "P". This means you have two ARTIST tags and two ARTISTSORT tags. But the tagging specs don't allow you to assume anything about the order of multiple tags so you can't know for sure that Atkins, Chet is the sort for "Chet Atkins" and not the sort for "Les Paul"

Also, the idea of of a lastNameFirst flag is problematic. Consider "Jerry Jeff Walker" and "Eddie Van Halen". And again with multiiple artists, how do you decide which artist(s) are affected by the lastNameFirst flag.

If you are going to use flags, I'd just suggest a word order value that would override the sort order. Hence, "Eddie Van Halen" would use 231, "Jerry Jeff Walker" would use 312, "Gladys Knight and the Pips" would use 215.

Ideally, you would store this with the Artist. For instance you could use the caret to separate the metadata from the displayable part of the tag, e.g., for the Chester and Lester example, you'd have two ARTIST tags, "Chet Atkins^21" and "Les Paul^21" and Bob Dylan and the Band would also have two ARTIST tags, "Bob Dylan^21" and "The Band^2"