+- MinimServer Forum (https://forum.minimserver.com)
+-- Forum: MinimServer (/forumdisplay.php?fid=1)
+--- Forum: Support (/forumdisplay.php?fid=4)
+--- Thread: autoupdate is always getting triggered (/showthread.php?tid=6142)
autoupdate is always getting triggered - music23084 - 30-03-2021 05:15
I have a huge music library ~450 GB in flac mounted via rclone.
I have switched the startupScan=false, but still it is scanning (re-reading the files instead of just listing the files) the whole library from the beginning every time.
I might need to scan the whole library in stages or it should automatically skip files which are already scanned.
Attach is the debug log (zip compressed)
Also included the sample rclone log which shows the chunks being read on every start.
RE: autoupdate is always getting triggered - simoncn - 30-03-2021 21:39
The log shows that MinimServer starts scanning your files at 09:28:03. Part way through this scan (at 09:37:48), an 'exit' command is issued. This terminates the scan before MinimServer can complete the scan and write its cache file.
You need to let the initial scan complete so that MinimServer can write its cache file. This might take a long time. After this, MinimServer will read from its cache file when started, which will be much quicker.
RE: autoupdate is always getting triggered - music23084 - 01-04-2021 04:05
For a large music library, it should be good to write the contents of the scanned data to cache before exit (in case the cache has been updated)
Continuously scanning the library could take 2-3 days and I have to keep the system on specifically for this purpose. Whereas if it supports incremental learning, the scanning could be running in background while I am doing other tasks.
Do you think is it good candidate of enhancement or bug?
Thanks for all the hard work.
RE: autoupdate is always getting triggered - stefano_mbp - 01-04-2021 06:36
If can help this is my experience.
I’m running Minimserver on a Intel Nuc8i3/16GB ram/Ubuntu 20.04.1 LTS,
my library is on an usb attached 8TB hdd drive and is more or less 12000 albums.
The first full scan takes about 2 hours and the following ones just over 1 minute
RE: autoupdate is always getting triggered - simoncn - 02-04-2021 19:47
The initial scan cannot be incremental because MinimServer needs a complete scan of the library before it can present its top-level index menu.
Subsequent scans use the MinimServer cache but still require looking at each file in the library to check whether it has changed.
Your log shows that reading each file from the cloud via rclone is taking about 3 to 4 seconds. This is about 17 files per minute or 1000 files per hour. On a local filesystem, a speed of at least 500 files per minute (30 times faster) would be expected.
MinimServer is designed for reading and serving files on the device running MinimServer or on another device on the same local network. I don't think it will be possible to get acceptable performance for using it to read and serve files from a cloud library.
RE: autoupdate is always getting triggered - music23084 - 05-04-2021 16:18
Hi Simoncn,
If during incremental learning the server is exited normally. Will it flush the cache before exit?
Thanks
RE: autoupdate is always getting triggered - simoncn - 05-04-2021 19:15
The incomplete learning will be discarded and the previous complete cache will be retained.