Limit of 65000 folders

[simoncn]

These artwork folders should not be counted if MinimServer is working as intended.

First, MinimServer scans the folder "artist A". This contains audio files so it is added to the "all folders" list and is included in the 65000 limit.

Second, MinimServer scans the folder "artist A/Artwork". This is temporarily added to the "all folders" list and is included in the 65000 limit. When MinimServer finds that this folder does not contain audio files, it is removed from the "all folders" list and is not included in the 65000 limit.

Please do the following:

1) In the log window, set the logging level to Verbose
2) Do a rescan
3) When the rescan hits the 65000 limit, copy the mnimserver.log file from the MinimServer/appData folder, zip this file (it will be large) and attach the zipped file to a post here.

This should enable me to see whether any folders that don't contain audio files are being included in the 65000 limit.

[9dIe2Hd0VGy3Pw0zYPYs]

Hi Simon,
it looks like I have to many folder with double-CDs.
Some of them I stored as Album\CD1 and Album\CD2.
I did not find any folder in the verly large log file, which doesn't contain music.
So I hope that you can find a way to remove the 65k folder limit in newer version.
It would took a long time to find all these double-CD folders and change them into multi CD folder.

[simbun]

If your files are tagged accurately it should be trivial to automate the reorganisation - assuming you want to - with tools such as foobar (using File Operations), having said that, the fact you have Genre in your tree hierarchy will make it difficult to replicate unless you only have one genre in your music tags too.

Today, folder structure is largely irrelevant (at least for me) as all browsing is via tags, so I decided that as long as it was consistent (so it could easily be checked), choosing a simple structure that was driven wholly from the tags (so I could easily change it should I need to) met all my needs.

I use the following naming standard.

Code:

/music/%ALBUMARTISTSORT%/albumartist.jpg                                    no use for this yet
/music/%ALBUMARTISTSORT%/%DATE% - %ALBUM%/
/music/%ALBUMARTISTSORT%/%DATE% - %ALBUM%/%DISCNUMBER%.%TRACKNUMBER%.flac
/music/%ALBUMARTISTSORT%/%DATE% - %ALBUM%/folder.jpg                        probably not needed as most players will pull it from tags
/music/%ALBUMARTISTSORT%/%DATE% - %ALBUM%cd$DISCNUMBER%.cue                 used for AccurateRip verification

An advantage of having the music files simply called %DISCNUMBER%.%TRACKNUMBER%.flac is that you don't need to worry about all the characters that aren't valid in filenames on your OS, plus it probably helps with indexing (Sonos had a 65,000 track limit but your tags and filenames could reduce that).

I understand there's no one right way to do this, so I hope I haven't caused any offense by offering up my implementation.

[9dIe2Hd0VGy3Pw0zYPYs]

Thanks for your advice, maybe it is helpfull for other people.
The problem is, I didn't tagged the discnumer when I startet ripping.
I used CD1 and CD2 for multidisc albums, because I had no server that can handel it. I hadn't Minim at this time.
So this are bad things out of the history.

I like my folderstructure. It is very helpfull sometimes.
Some people like roon, I don't.
So, everyone is different.

[DavidHB]

(19-04-2020 10:03)9dIe2Hd0VGy3Pw0zYPYs Wrote:  The problem is, I didn't tagged the discnumber when I started ripping.

If you have any Windows devices, MP3Tag (free, but with a donation requested) can add DiscNumbers to your flies relatively quickly, provided that the Album tags of the individual CDs identify the individual CDs as [Disc 1], [Disc 2] or whatever, to conform to MinimServer conventions for multi-disc albums.

In such a case, with MP3Tag, you can select as much of your folder structure as your computer can comfortably handle at one go (say all the folders related to a particular composer), and all the flies from subfolders of that branch of the system will then be accessible via the MP3Tag editing grid. You can then filter the display; for instance, in my example, filtering by "1]" (or "Album HAS 1]" in Mp3Tag speak) will (hopefully) limit the display to those files having "[Disc 1]" in their Album tags. At that point, you can select all the files in the grid, set the DiscNumber to 1, save, and then do the same thing for all the other [Disc n] files in the selection. Finally, if you need to, you can filter by and then select all files which have blank disc numbers, set their DiscNumber tags to 1, and save. In a few minutes, you can have tagged hundreds or perhaps even thousands of files with their correct DiscNumbers

Another trick is to use MP3Tag to batch rename your files from disc and track numbers and Title tags. I do this for all my multi-disc albums (typically CD rips rather than downloads) where I separately identify the individual discs. In this scheme, file names will be of the form:

104 Brandenburg Concerto No. 1 in F major, BWV 1046 4. Menuetto - Trio I - Polacca - Trio II

where 1 is the Disc number, 04 is the track number and the rest of the file name is formed from the Title tag. This allows all the files in a multi-disc album to be moved into single, combined folders and still retain their proper order and disc number identity. Whenever you find yourself breaching the 65,000 folder limit, you can combine perhaps 20 or 50 folders in this way to give yourself some headroom. That approach could keep things workable until Simon gets around to lifting or increasing the folder limit.

David

[simbun]

Or if you don't have 'Disc X' in the tags and you wanted to base it on the folder the files are in (you mentioned you put multi-discs in CD1, CD2, CDX directories) you could simply use the following in the MassTagger component of foobar to format the value of discnumber:

Code:

$puts(curDir,$directory(%path%,1))                                      get the current folder and put into variable curDir
$iif( $strcmp($left($get(curDir),2),'CD'),                              If starts with 'CD'
     $ifequal( $div($substr($get(curDir),3,$len($get(curDir))),1), 0,     If the remainder can't be divided by 1 (result is 0 so not numeric)
               '1',                                                          set to discnumber 1
               $substr($get(curDir),3,$len($get(curDir)))                    else set to relevant number
             ),
     '1'                                                                else default to '1'
   )

I had to put an extra i in the if condition to stop the parser from refusing to post, and if you did want to use this, the function needs to be all on one line with comments and spaces removed.

Well that killed an hour or so, what to do now... :-)

[DavidHB]

(20-04-2020 14:50)simbun Wrote:  Or if you don't have 'Disc X' in the tags and you wanted to base it on the folder the files are in (you mentioned you put multi-discs in CD1, CD2, CDX directories) you could simply use the following in the MassTagger component of foobar to format the value of discnumber ...
Well that killed an hour or so, what to do now... :-)

Showoff!

David

[jarome]

(05-09-2019 08:17)simoncn Wrote:  The 65000 folder limit is based on memory resource issues. When MinimServer was first released in 2012, a key feature was its ability to run on low-end devices, including some NAS devices with only 128MB of RAM for the operating system and all applications. As MinimServer holds key data structures in RAM for good performance, these structures needed to be as compact as possible and (without going into too much technical detail) this led to the 65000 folder limit. As the design point was to handle a library of 100,000 files (not folders), this number seemed sufficient and indeed has been sufficient for all users until now.

Things have moved on since then, with all current NAS devices having at least 256MB of RAM and some MinimServer users having very large libraries, so I will look again at how I can redesign the internal structures of MinimServer to remove the 65000 folder limit. One option is to allow up to 65000 folders for each content directory. As MinimServer can have up to 8 content directories, this would increase the limit to over 500,000 folders. Another option (which would consume more memory for small and medium-sized libraries) is to change the internal data structres of MinimServer to allow an unlimited number of folders.

This change will not happen immediately, so you need a workaround for now. Your suggestion of using a second Mac with a different hostname would work well. With this setup, there is no need to change port numbers because the MinimServer instances are differentiated by hostname, unlike the situation where multiple instances are running on the same machine.

I have over 65000 albums, and over 100000 tracks. Splitting servers is not a good option since I am using my DCS Bartok to listen to my music, and it is a pain to switch servers.

[9dIe2Hd0VGy3Pw0zYPYs]

Hi Simon,

is there something new for the 65k folder issue?

Regards

[simoncn]

Not at present. I will post here when there is a solution.