Post Reply 
 
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Feature Request: Re-Scan/inotify
26-03-2012, 11:30
Post: #1
Feature Request: Re-Scan/inotify
Hi

finally somebody who makes a upnp server for big music collections "the right away". great project! thank you very much!

are you planing to release the source code under an open source license? so that i could make changes or add features on my own? or that others could help fixing bugs etc ?

one thing that i am missing at the moment is a re-scan feature. when i add new files to my library, i have to delete the cache file of minimserver and restart it, otherwise the new files don't get added to the catalog. or am i missing some configuration option? a great thing would be to use the inotify to make it possible to add new files in real-time to the library without having to continously re-scan all the files.

greets
KoS
Find all posts by this user
Quote this message in a reply
26-03-2012, 12:19
Post: #2
RE: Feature Request: Re-Scan/inotify
(26-03-2012 11:30)KoS Wrote:  Hi

finally somebody who makes a upnp server for big music collections "the right away". great project! thank you very much!

are you planing to release the source code under an open source license? so that i could make changes or add features on my own? or that others could help fixing bugs etc ?

Thanks for the kind words!

No, I'm not planning to release open source code for MinimServer.

MinimServer uses a Java component framework named jMinim, and I intend to release this part as open source code at some stage. This won't happen in the near future, because at the moment I'm putting all my time into working on MinimServer.

Quote:one thing that i am missing at the moment is a re-scan feature. when i add new files to my library, i have to delete the cache file of minimserver and restart it, otherwise the new files don't get added to the catalog. or am i missing some configuration option? a great thing would be to use the inotify to make it possible to add new files in real-time to the library without having to continously re-scan all the files.

greets
KoS

You should be able to add new files by selecting Restart from the tray icon. You don't need to stop MinimServer first, and you don't need to delete the cache files. Deleting the cache files is a bad idea because it makes MinimServer start much more slowly. You can also continue to play music while the MinimServer restart is in progress. So the recommended sequence is:
1) Add the new files
2) Start some music playing (but not the new music just added)
3) Press Restart
4) When the icon changes from yellow to green, refresh the display in the control point by browsing the top-level container, or restart the control point
5) Your new music should now appear in the control point display

If the above isn't working for you, please provide more details of what happens when you go through this sequence. It would also be useful to look at the minimserver.log file to see if there are any error or warnng messages there.

It would be possible to support inotify in MinimServer, but this would require adding a dependency on Java 7, and I'm very reluctant to do this. It would also need quite a major change to the MinimServer design. I don't see much benefit from doing this, as the user has to know that an inotify update has happened so that they can refresh or restart the control point to update the information displayed there.
Find all posts by this user
Quote this message in a reply
26-03-2012, 13:57
Post: #3
RE: Feature Request: Re-Scan/inotify
(26-03-2012 12:19)simoncn Wrote:  MinimServer uses a Java component framework named jMinim, and I intend to release this part as open source code at some stage. This won't happen in the near future, because at the moment I'm putting all my time into working on MinimServer.
okay thanks for the info.

Quote:You should be able to add new files by selecting Restart from the tray icon. You don't need to stop MinimServer first, and you don't need to delete the cache files. Deleting the cache files is a bad idea because it makes MinimServer start much more slowly. You can also continue to play music while the MinimServer restart is in progress. So the recommended sequence is:
1) Add the new files
2) Start some music playing (but not the new music just added)
3) Press Restart
4) When the icon changes from yellow to green, refresh the display in the control point by browsing the top-level container, or restart the control point
5) Your new music should now appear in the control point display

If the above isn't working for you, please provide more details of what happens when you go through this sequence. It would also be useful to look at the minimserver.log file to see if there are any error or warnng messages there.
i think i found a bug. first i thought the files weren't being rescan upon restarting minim (what is the best way to restart the minimserver without minimwatch on a headless machine?), but as soon as i changed the name of the directory containting the new files, the appeared after a restart. the problem is that the directory is not scanned if the name includes some german umlauts (äöü). i don't get any more information in the log file.

Quote:It would be possible to support inotify in MinimServer, but this would require adding a dependency on Java 7, and I'm very reluctant to do this. It would also need quite a major change to the MinimServer design. I don't see much benefit from doing this, as the user has to know that an inotify update has happened so that they can refresh or restart the control point to update the information displayed there.
until now i haven't had on any control point (hardware or software) the problem that i wouldn't see new content. the devices don't cache the hierarchy (if they cache something, then the thumbnails), so as soon as i navigate through the hierarchy it is always loaded from the upnp server and "up-to-date". so the benefit would be that after copying new music to the library, the user doesn't have to manually force a rescan or wait for the next "automatic" rescan, but instead the "view" on the control point would always be consistent with the one of the real/existing files.
Find all posts by this user
Quote this message in a reply
26-03-2012, 17:51
Post: #4
RE: Feature Request: Re-Scan/inotify
(26-03-2012 13:57)KoS Wrote:  i think i found a bug. first i thought the files weren't being rescan upon restarting minim (what is the best way to restart the minimserver without minimwatch on a headless machine?), but as soon as i changed the name of the directory containting the new files, the appeared after a restart. the problem is that the directory is not scanned if the name includes some german umlauts (äöü). i don't get any more information in the log file.

You can use MinimWatch on a headless machine. This is described in the User guide on the MinimServer website. See the sections "Running MinimWatch" and "Using MinimWatch" for details.

The problem with directory names is caused by an incorrect locale setting on the machine that's running MinimServer. If you can give me more details of what this machine is, and how the files were copied to that machine, I can give you specific information about how to resolve the problem.

Unfortunately it's not possible for MinimServer to detect this problem. There's a limitation in Java that prevents MinimServer from knowing that these directories exist. The same applies to filenames with characters that aren't valid in the current locale.

Quote:until now i haven't had on any control point (hardware or software) the problem that i wouldn't see new content. the devices don't cache the hierarchy (if they cache something, then the thumbnails), so as soon as i navigate through the hierarchy it is always loaded from the upnp server and "up-to-date". so the benefit would be that after copying new music to the library, the user doesn't have to manually force a rescan or wait for the next "automatic" rescan, but instead the "view" on the control point would always be consistent with the one of the real/existing files.

I'd be interested to know which combination of server and control point works in this way. Most control points cache some or all of the browse tree after downloading it from the server.
Find all posts by this user
Quote this message in a reply
24-04-2012, 12:03
Post: #5
RE: Feature Request: Re-Scan/inotify
So i'm back from my holidays :-)

(26-03-2012 17:51)simoncn Wrote:  You can use MinimWatch on a headless machine. This is described in the User guide on the MinimServer website. See the sections "Running MinimWatch" and "Using MinimWatch" for details.
that works as expected. but is there a way to restart the server directly from command line, in a non-interactive way, e.g. used by a cron script?

i tried the following which obviously doesn't work:
Code:
echo 'restart' | java -jar /opt/minimserver/lib/minimwatch.jar
MinimWatch 0.31, Copyright (c) 2012 Simon Nash. All rights reserved.
Enter command (? for help), or null to exit:
>Command not available while waiting for media server: restart
>command input not available
Remote media server application is active
MinimServer is running

Quote:The problem with directory names is caused by an incorrect locale setting on the machine that's running MinimServer. If you can give me more details of what this machine is, and how the files were copied to that machine, I can give you specific information about how to resolve the problem.
thanks for the hint. it was a locale problem on the virtual machine that i created for the minim server. works now.

Quote:
(26-03-2012 13:57)KoS Wrote:  until now i haven't had on any control point (hardware or software) the problem that i wouldn't see new content. the devices don't cache the hierarchy (if they cache something, then the thumbnails), so as soon as i navigate through the hierarchy it is always loaded from the upnp server and "up-to-date". so the benefit would be that after copying new music to the library, the user doesn't have to manually force a rescan or wait for the next "automatic" rescan, but instead the "view" on the control point would always be consistent with the one of the real/existing files.

I'd be interested to know which combination of server and control point works in this way. Most control points cache some or all of the browse tree after downloading it from the server.
e.g. Kinksy on Windows & iOS
Find all posts by this user
Quote this message in a reply
24-04-2012, 14:54
Post: #6
RE: Feature Request: Re-Scan/inotify
(24-04-2012 12:03)KoS Wrote:  So i'm back from my holidays :-)

I hope you had a great time!

Quote:that works as expected. but is there a way to restart the server directly from command line, in a non-interactive way, e.g. used by a cron script?

i tried the following which obviously doesn't work:
Code:
echo 'restart' | java -jar /opt/minimserver/lib/minimwatch.jar
MinimWatch 0.31, Copyright (c) 2012 Simon Nash. All rights reserved.
Enter command (? for help), or null to exit:
>Command not available while waiting for media server: restart
>command input not available
Remote media server application is active
MinimServer is running

This is caused by the "restart" command arriving before MinimWatch has located the MinimServer instance that it's controlling.

A simple solution would be to add a new MinimWatch command "sleep <n>" to sleep for <n> seconds. You could include this in the echo string to provide the necessary delay. Would this be suitable?

Quote:e.g. Kinksy on Windows & iOS

Kinsky works as you say on Windows. On iOS, if you browse up the hierarchy, Kinsky doesn't refresh what's displayed. So if you're looking at the contents of an album, and you browse up to the list of all albums, you'll see an old list of albums instead of a current list. If you browse down the hierarchy, Kinsky iOS does refresh the contents.

I've been thinking about auto-refresh, and I can see that this would be a useful addition to MinimServer (when using a control point that does the same thing). Unfortunately it would be a lot of work to implement this because of some aspects of the design of MinimServer, so I can't say if or when it would be supported.
Find all posts by this user
Quote this message in a reply
24-04-2012, 17:55
Post: #7
RE: Feature Request: Re-Scan/inotify
(24-04-2012 14:54)simoncn Wrote:  I hope you had a great time!
sailing in the carribean, it was great. haven't done any sailing before so it was really exciting!

Quote:A simple solution would be to add a new MinimWatch command "sleep <n>" to sleep for <n> seconds. You could include this in the echo string to provide the necessary delay. Would this be suitable?
i thought also about that, but somehow i don't know how to combine the two so that i works like it should:

Code:
sleep 5; echo 'restart' | java -jar /opt/minimserver/lib/minimwatch.jar
the sleep here delays the whole execution/start of minimwatch and not only the echo command :-(
if there are multiple minimserver on the network, will then always the local one be choosen as the first? so that the restart command restart the local one?
or would it be possible to make it like for other unix daemons where a SIGHUP signal to the minimserver would force it to reload, or rescan, the media library?

Quote:Kinsky works as you say on Windows. On iOS, if you browse up the hierarchy, Kinsky doesn't refresh what's displayed. So if you're looking at the contents of an album, and you browse up to the list of all albums, you'll see an old list of albums instead of a current list. If you browse down the hierarchy, Kinsky iOS does refresh the contents.
hmm.. strange, i'll test it again on my setup here with the iPad because i was sure it was refreshing the whole list.

Quote:I've been thinking about auto-refresh, and I can see that this would be a useful addition to MinimServer (when using a control point that does the same thing). Unfortunately it would be a lot of work to implement this because of some aspects of the design of MinimServer, so I can't say if or when it would be supported.
i think i can live with it if it's possible to force the rescan by a script. so i can write an inotify-script that then forces the rescan.

i'll add some bug reports & feature requests as separate threads, i hope thats fine?

thanks for your help!
Find all posts by this user
Quote this message in a reply
24-04-2012, 19:14
Post: #8
RE: Feature Request: Re-Scan/inotify
(24-04-2012 17:55)KoS Wrote:  
Quote:A simple solution would be to add a new MinimWatch command "sleep <n>" to sleep for <n> seconds. You could include this in the echo string to provide the necessary delay. Would this be suitable?
i thought also about that, but somehow i don't know how to combine the two so that i works like it should:

Code:
sleep 5; echo 'restart' | java -jar /opt/minimserver/lib/minimwatch.jar
the sleep here delays the whole execution/start of minimwatch and not only the echo command :-(

You would do the following:
Code:
echo -e "sleep 5\nrestart\nsleep 10\nexit" | java -jar /opt/minimserver/lib/minimwatch.jar

MinimWatch will start and process the "sleep 5" command immediately. It will wait 5 seconds (while MinimServer is discovered), then it will process the "restart" command. It will wait 10 seconds (while MinimServer restarts), then it will process the "exit" command. You can adjust the two sleep times based on how long things take in your setup.

I've written some prototype code to test this, and it works very well. Smile

Quote:if there are multiple minimserver on the network, will then always the local one be choosen as the first? so that the restart command restart the local one?

No, the selection of a MinimServer instance will be random, based on which MinimServer instance responds first to the M-SEARCH discovery message. So this approach doesn't work if you'll be running multiple MinimServer instances.

It would be possible to extend MinimWatch to allow it to select a specific MinimServer instance (based on the displayName). With this, you'd need to use a command string like:
Code:
echo -e "sleep 5\nselect server1\nrestart\nsleep 10\nexit" | java -jar /opt/minimserver/lib/minimwatch.jar

Quote:or would it be possible to make it like for other unix daemons where a SIGHUP signal to the minimserver would force it to reload, or rescan, the media library?

There's a problem with this, because some NAS versions of Linux send a SIGHUP to MinimServer on exit from the shell that started MinimServer (even if MinimServer was started as a background process). To prevent this SIGHUP from ending the MinimServer process, I've added a --nohup option to MinimServer that makes it ignore SIGHUP. This is equivalent to using the nohup program to launch MinimServer.

I presume that a Linux daemon that uses SIGHUP to reload its configuration would also reload its configuration when the shell that started the daemon exits. This doesn't seem ideal. How do these Linux daemons handle this?

Quote:i think i can live with it if it's possible to force the rescan by a script. so i can write an inotify-script that then forces the rescan.

OK, that makes sense.

Quote:i'll add some bug reports & feature requests as separate threads, i hope thats fine?

thanks for your help!

Please go ahead and do that. It's definitely a good idea to use a separate thread for each topic.
Find all posts by this user
Quote this message in a reply
26-04-2012, 16:29 (This post was last modified: 26-04-2012 16:46 by KoS.)
Post: #9
RE: Feature Request: Re-Scan/inotify
(24-04-2012 19:14)simoncn Wrote:  
(24-04-2012 17:55)KoS Wrote:  
Quote:A simple solution would be to add a new MinimWatch command "sleep <n>" to sleep for <n> seconds. You could include this in the echo string to provide the necessary delay. Would this be suitable?
i thought also about that, but somehow i don't know how to combine the two so that i works like it should:

Code:
sleep 5; echo 'restart' | java -jar /opt/minimserver/lib/minimwatch.jar
the sleep here delays the whole execution/start of minimwatch and not only the echo command :-(

You would do the following:
Code:
echo -e "sleep 5\nrestart\nsleep 10\nexit" | java -jar /opt/minimserver/lib/minimwatch.jar

MinimWatch will start and process the "sleep 5" command immediately. It will wait 5 seconds (while MinimServer is discovered), then it will process the "restart" command. It will wait 10 seconds (while MinimServer restarts), then it will process the "exit" command. You can adjust the two sleep times based on how long things take in your setup.

I've written some prototype code to test this, and it works very well. Smile
in the meantime i've now used a small expect script to do the restart :-)
Code:
#!/usr/bin/expect
spawn java -jar /opt/minimserver/lib/minimwatch.jar
sleep 5
send "restart\n"
sleep 3
send "exit\n"


Quote:It would be possible to extend MinimWatch to allow it to select a specific MinimServer instance (based on the displayName). With this, you'd need to use a command string like:
Code:
echo -e "sleep 5\nselect server1\nrestart\nsleep 10\nexit" | java -jar /opt/minimserver/lib/minimwatch.jar
that would be the perfect combination/extension. or something like "select local1" or so... if somebody has an minim server running on the network too, i wouldn't be happy to restart his one ;-)

Quote:
Quote:or would it be possible to make it like for other unix daemons where a SIGHUP signal to the minimserver would force it to reload, or rescan, the media library?

There's a problem with this, because some NAS versions of Linux send a SIGHUP to MinimServer on exit from the shell that started MinimServer (even if MinimServer was started as a background process). To prevent this SIGHUP from ending the MinimServer process, I've added a --nohup option to MinimServer that makes it ignore SIGHUP. This is equivalent to using the nohup program to launch MinimServer.

I presume that a Linux daemon that uses SIGHUP to reload its configuration would also reload its configuration when the shell that started the daemon exits. This doesn't seem ideal. How do these Linux daemons handle this?
hmm.. haven't realized the problem with the SIGHUP when exiting from a shell. you could use instead another signal for the restart of the daemon, e.g. SIGUSR1 or SIGUSR2 ?



(24-04-2012 14:54)simoncn Wrote:  Kinsky works as you say on Windows. On iOS, if you browse up the hierarchy, Kinsky doesn't refresh what's displayed. So if you're looking at the contents of an album, and you browse up to the list of all albums, you'll see an old list of albums instead of a current list. If you browse down the hierarchy, Kinsky iOS does refresh the contents.
just tested it and you're right. sorry for the des-information :-( so they are missing a refresh button :-) the ChorusDSHD app on iOS has the sample problem with refreshing.. but at least it has an refresh button.
Find all posts by this user
Quote this message in a reply
26-04-2012, 17:00
Post: #10
RE: Feature Request: Re-Scan/inotify
(26-04-2012 16:29)KoS Wrote:  in the meantime i've now used a small expect script to do the restart :-)
Code:
#!/usr/bin/expect
spawn java -jar /opt/minimserver/lib/minimwatch.jar
sleep 5
send "restart\n"
sleep 3
send "exit\n"

Thanks for the pointer to 'expect'. I wan't aware of this. Unfortunately it's not available on my NAS. Sad

Quote:
Quote:It would be possible to extend MinimWatch to allow it to select a specific MinimServer instance (based on the displayName). With this, you'd need to use a command string like:
Code:
echo -e "sleep 5\nselect server1\nrestart\nsleep 10\nexit" | java -jar /opt/minimserver/lib/minimwatch.jar
that would be the perfect combination/extension. or something like "select local1" or so... if somebody has an minim server running on the network too, i wouldn't be happy to restart his one ;-)

In this example, "server1" was an example of a displayName. You can set a different displayName for each instance of MinimServer, which is displayed by the control point. You'd need to use the same displayName that you assigned to your local server, and make sure that there isn't another instance of MinimServer with that displayName running on your network.

It would be possible to so something special for local instances of MinimServer, but this would have to cope with multiple local instances (each with their own displayName), so I prefer the simple approach that uses displayName for both the local and remote case.

Quote:hmm.. haven't realized the problem with the SIGHUP when exiting from a shell. you could use instead another signal for the restart of the daemon, e.g. SIGUSR1 or SIGUSR2 ?

That's a good idea, though I'd need to be careful not to confict with the use of these signals for other purposes, such as controlling LinuxThreads. I think the best approach would be to make the choice of 'reload' signal configurable, so that people could choose a signal that doesn't conflict with other usage in their environment.
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump:


User(s) browsing this thread: 1 Guest(s)