JAVA restart necessary to fix inactive Minimserver?

[hvaleton]

(07-06-2013 13:17)simoncn Wrote:  

(07-06-2013 10:05)hvaleton Wrote:  192.168.1.57 is my Galaxy Nexus smartphone

Thanks for all the very detailed information. What control point or other audio software are you running on the Galaxy Nexus?

BubbleDS
MediaHouse
Sonos (but that AP does not 'see' the MinimServer)

(07-06-2013 13:17)simoncn Wrote:  In the latest netstat output, this device has 14 established HTTP connections to Minimserver. I can't think of any good reason for this, and I think it's likely to be the cause of the descriptor leakage problem.

Very strange behaviour: after switching off my Nexus the netstat command output still showed the 14 established connections to 192.168.1.57, same as before the switch-off.

Could it be that it takes some time for these connections to disappear from the netstat output?
Should I perhaps restart the MinimServer? Or even the Qnap?

I ran netstat from my Nexus and have enclosed the output as an attachment.
It does not show any connections with the Qnap whose ip is 192.168.1.44.

I'm completely puzzled now...

[simoncn]

(07-06-2013 15:08)hvaleton Wrote:  Very strange behaviour: after switching off my Nexus the netstat command output still showed the 14 established connections to 192.168.1.57, same as before the switch-off.

Could it be that it takes some time for these connections to disappear from the netstat output?
Should I perhaps restart the MinimServer? Or even the Qnap?

I ran netstat from my Nexus and have enclosed the output as an attachment.
It does not show any connections with the Qnap whose ip is 192.168.1.44.

I'm completely puzzled now...

I will try BubbleDS and MediaHouse to see if I can reproduce this behaviour. My guess is that these connections are used by the control point to download album art files, and the control point is leaving them open indefinitely after the album art download has completed. The correct behaviour would be for the control point to close these connections after a reasonable period such as 5 or 10 minutes.

Also, it seems that the QNAP is failing to close and release socket connections that have been "abandoned" by a powered-off client. My recollection is that Linux has a very long default timeout (120 minutes) before it starts sending keep-alive probes to see if the client is still there.

[simoncn]

(07-06-2013 18:29)simoncn Wrote:  I will try BubbleDS and MediaHouse to see if I can reproduce this behaviour. My guess is that these connections are used by the control point to download album art files, and the control point is leaving them open indefinitely after the album art download has completed. The correct behaviour would be for the control point to close these connections after a reasonable period such as 5 or 10 minutes.

Also, it seems that the QNAP is failing to close and release socket connections that have been "abandoned" by a powered-off client. My recollection is that Linux has a very long default timeout (120 minutes) before it starts sending keep-alive probes to see if the client is still there.

I don't think these connections from 192.168.1.57 are causing the large number of leaked sockets in the lsof output. I can see these connections in lsof as well as the leaked sockets (marked as "can't identify protocol"). From doing some web searching, it seems the leaked sockets are "half closed" and don't show up in the output from netstat.

I've tried using BubbleDS and MediaHouse, and neither of these seems to cause any leaked sockets to be created. I'll continue to investigate what might be creating the leaked sockets.

[rb73]

I'm keeping an eye on this, as I had a similar issue a while (18 days to be precise) back. From one day to the next Minimserver was no longer available in my control points. As I had visitors coming to listen to some music, I didn't have time to figure out what might be wrong, so just rebooted the Qnap.

If you want, I can get you the minimserver-diag.tar.gz?

JW

[simoncn]

(09-06-2013 10:16)rb73 Wrote:  I'm keeping an eye on this, as I had a similar issue a while (18 days to be precise) back. From one day to the next Minimserver was no longer available in my control points. As I had visitors coming to listen to some music, I didn't have time to figure out what might be wrong, so just rebooted the Qnap.

If you want, I can get you the minimserver-diag.tar.gz?

JW

Did you have this issue with MinimServer 0.72 or an earlier version?

If you had this with 0.72, I would be interested to see the lsof output for the MinimServer process. This output isn't included in minimserver-diag.tar.gz. To get it, you need to run lsof from an ssh console prompt.

[rb73]

(09-06-2013 21:20)simoncn Wrote:  Did you have this issue with MinimServer 0.72 or an earlier version?

If you had this with 0.72, I would be interested to see the lsof output for the MinimServer process. This output isn't included in minimserver-diag.tar.gz. To get it, you need to run lsof from an ssh console prompt.

This was after the upgrade to 0.72.

I've attached the output of lsof.

[simoncn]

(10-06-2013 08:37)rb73 Wrote:  This was after the upgrade to 0.72.

I've attached the output of lsof.

Thanks for sending ths. It doesn't show the descriptor leakage pattern that hvaleton was getting. If you have a minimserver-diag.tar.gz file that was captured after the previous crash, please post it here. Also, if the problem happens again, please disable MinimServer to capture the minimserver-diag.tar.gz output and post it here. Many thanks!

[rb73]

(10-06-2013 08:55)simoncn Wrote:  If you have a minimserver-diag.tar.gz file that was captured after the previous crash, please post it here. Also, if the problem happens again, please disable MinimServer to capture the minimserver-diag.tar.gz output and post it here. Many thanks!

No worries. I'll keep an eye out, and should it happen again I'll post the diagnostic files here.

The .tar.gz is too big to attach, but you can download it from my dropbox here.

I'm pretty sure I didn't disable MinimServer after the crash, so it should be the one captured after the crash.

[simoncn]

(10-06-2013 09:18)rb73 Wrote:  No worries. I'll keep an eye out, and should it happen again I'll post the diagnostic files here.

The .tar.gz is too big to attach, but you can download it from my dropbox here.

I'm pretty sure I didn't disable MinimServer after the crash, so it should be the one captured after the crash.

Thanks very much for sending this. It shows that the problem is caused by a race condition in MinimServer that means some connection threads aren't being destroyed when they should be. This might be the cause of the other problem as well, because the connection socket won't be released while the thread is still alive.

I'll put a fix for this in the next release. Many thanks for your help with finding this problem!

[rb73]

(10-06-2013 12:53)simoncn Wrote:  I'll put a fix for this in the next release. Many thanks for your help with finding this problem!

You're welcome, glad I could help