Minimserver 0.63 in QNAP 419+ (Arm)

[pmfortune]

Since I upgraded to 0.63 my server seems to hang about once a week and it is not possible to restart it using Minimwatch. I didn't have this issue before the upgrade.

Is this just my setup or has anyone else noticed this?

Thanks

Peter-Marc

[simoncn]

(14-01-2013 23:29)pmfortune Wrote:  Since I upgraded to 0.63 my server seems to hang about once a week and it is not possible to restart it using Minimwatch. I didn't have this issue before the upgrade.

Is this just my setup or has anyone else noticed this?

Thanks

Peter-Marc

This sounds like a bug in MinimServer. Are you familiar with using SSH to access the QNAP, or are you willing to try this? If so, I'll post some instructions for how to get diagnostic information that will help me identify the problem.

Edit: As a first step, you can look in two files when the problem occurs to see if there's any information there. One file is /share/Public/minimserver.log and the other file is /tmp/minimserver-out-nnnn.log, where nnnn is the process ID of the MinimServer process. You should be able obtain both these files from a remote FTP/SFTP client such as WinSCP, without needing to login via SSH. There are also some additional steps that would require an SSH login.

[pmfortune]

(15-01-2013 07:09)simoncn Wrote:  

(14-01-2013 23:29)pmfortune Wrote:  Since I upgraded to 0.63 my server seems to hang about once a week and it is not possible to restart it using Minimwatch. I didn't have this issue before the upgrade.

Is this just my setup or has anyone else noticed this?

Thanks

Peter-Marc

This sounds like a bug in MinimServer. Are you familiar with using SSH to access the QNAP, or are you willing to try this? If so, I'll post some instructions for how to get diagnostic information that will help me identify the problem.

Edit: As a first step, you can look in two files when the problem occurs to see if there's any information there. One file is /share/Public/minimserver.log and the other file is /tmp/minimserver-out-nnnn.log, where nnnn is the process ID of the MinimServer process. You should be able obtain both these files from a remote FTP/SFTP client such as WinSCP, without needing to login via SSH. There are also some additional steps that would require an SSH login.

Sorry not familiar with SSH - but very willing to help if you give me full instructions.

[simoncn]

(15-01-2013 08:58)pmfortune Wrote:  Sorry not familiar with SSH - but very willing to help if you give me full instructions.

OK, here goes:

1) Login using an SSH client, such as Putty on Windows. Enter the same userID and password that you use to login to the QNAP admin console.

2) Type the command:
ps

3) Somewhere near the bottom of the output, you'll see a line like this:
19396 admin 29924 S /usr/local/jre/bin/java -jar ../lib/minimserver.jar --noconsole --nohup
The first number on this line (19396) is the process ID (PID) for MinimServer. In the rest of these instructions, we'll use this number to represent the PID. You need to change this number to the actual number you see on your QNAP.

4) If you don't see a line like this, it mean the MinimServer process has crashed. You can find the MinimServer PID by typing the command:
cat /tmp/minimserver.pid
Skip steps 5 and 6 and go straight to step 7.

5) Type the command:
kill -quit 19396

6) You won't see any output from the 'kill' command, but it should have written output to the file /tmp/minimserver-out-19396.log.

7) Now it's time to copy MinimServer's log files to another computer. These files are:
/share/Public/minimserver.log
/tmp/minimserver-out-19396.log
Copying the first of these should be easy, because it's visible in the Public share of the QNAP. For the second, you can copy it to the Public share by typing the command:
cp -p /tmp/minimserver-out-19396.log /share/Public
This will copy the minimserver-out-19396.log file to the /share/Public directory of the QNAP, which will make it visible in the Public share.

8) Type the command:
exit
to close the SSH console window.

9) Zip the minimserver.log and minimserver-out-19396.log files and attach them here.

Many thanks!

[pmfortune]

(15-01-2013 10:59)simoncn Wrote:  

(15-01-2013 08:58)pmfortune Wrote:  Sorry not familiar with SSH - but very willing to help if you give me full instructions.

OK, here goes:

1) Login using an SSH client, such as Putty on Windows. Enter the same userID and password that you use to login to the QNAP admin console.

2) Type the command:
ps

3) Somewhere near the bottom of the output, you'll see a line like this:
19396 admin 29924 S /usr/local/jre/bin/java -jar ../lib/minimserver.jar --noconsole --nohup
The first number on this line (19396) is the process ID (PID) for MinimServer. In the rest of these instructions, we'll use this number to represent the PID. You need to change this number to the actual number you see on your QNAP.

4) If you don't see a line like this, it mean the MinimServer process has crashed. You can find the MinimServer PID by typing the command:
cat /tmp/minimserver.pid
Skip steps 5 and 6 and go straight to step 7.

5) Type the command:
kill -quit 19396

6) You won't see any output from the 'kill' command, but it should have written output to the file /tmp/minimserver-out-19396.log.

7) Now it's time to copy MinimServer's log files to another computer. These files are:
/share/Public/minimserver.log
/tmp/minimserver-out-19396.log
Copying the first of these should be easy, because it's visible in the Public share of the QNAP. For the second, you can copy it to the Public share by typing the command:
cp -p /tmp/minimserver-out-19396.log /share/Public
This will copy the minimserver-out-19396.log file to the /share/Public directory of the QNAP, which will make it visible in the Public share.

8) Type the command:
exit
to close the SSH console window.

9) Zip the minimserver.log and minimserver-out-19396.log files and attach them here.

Many thanks!

No problem - but I don't have a PC - I'm totally Mac based. Before I search out a Mac SSH client can you recommend one?

Thanks

Peter-Marc

[simoncn]

(15-01-2013 18:40)pmfortune Wrote:  No problem - but I don't have a PC - I'm totally Mac based. Before I search out a Mac SSH client can you recommend one?

Thanks

Peter-Marc

You can open a Terminal window on the Mac and type:

ssh admin@xx.xx.xx.xx

where xx.xx.xx.xx is the IP address of the QNAP.

[pmfortune]

(16-01-2013 00:14)simoncn Wrote:  

(15-01-2013 18:40)pmfortune Wrote:  No problem - but I don't have a PC - I'm totally Mac based. Before I search out a Mac SSH client can you recommend one?

Thanks

Peter-Marc

You can open a Terminal window on the Mac and type:

ssh admin@xx.xx.xx.xx

where xx.xx.xx.xx is the IP address of the QNAP.

OK - done

There were also a number of crash logs - wasn't sure if they would be useful - attached just in case!

[pmfortune]

A couple more crash logs!

[simoncn]

Thanks very much for doing this and posting the results!

The crash logs are caused by a known problem that will be fixed in the next release.

The interesting file is minimserver-out-29864.log. I don't see anything here that would cause MinimServer to be unresponsive to MinimWatch. Was MinimServer in this unresponsive state when you did the kill -quit command?

However, there is something in minimserver-out-29864.log that indicates a problem. Many threads are blocked waiting for a socket write to complete. I've been working on a fix for this, and it will be available in the next release. It's possible that the large number of blocked threads might be causing a resource issue that is making MinimServer unresponsive.

[pmfortune]

(19-01-2013 13:43)simoncn Wrote:  Thanks very much for doing this and posting the results!

The crash logs are caused by a known problem that will be fixed in the next release.

The interesting file is minimserver-out-29864.log. I don't see anything here that would cause MinimServer to be unresponsive to MinimWatch. Was MinimServer in this unresponsive state when you did the kill -quit command?

However, there is something in minimserver-out-29864.log that indicates a problem. Many threads are blocked waiting for a socket write to complete. I've been working on a fix for this, and it will be available in the next release. It's possible that the large number of blocked threads might be causing a resource issue that is making MinimServer unresponsive.

There was actually something playing when I entered the 'kill quit' command - it completed that track and then was unresponsive to Kinsky. However after a reboot (using Minimwatch) it was fine.

(NB most of the hangs I need to restart Minimserver via the QNAP control interface as it stops talking to Minimwatch as well.)

Thanks

Peter-Marc