Sound quality differences between NAS units

[winxi]

Thank you for your efforts in doing these tests, but I hope you understand that such quick tests are not significant in a scientific/statistical sense.

A significant test would require a controlled setting, for example:

Preparation phase: There are two NAS types and two persons, the test person and the instructor. The test person would be ideally somebody who claims to hear a difference between NAS types. The test person chooses a track for which he/she claims to hear a clear difference and he/she is allowed to listen to this track as often as he/she likes, while knowing which NAS type is used.

Test phase: The test person and the instructor are in different rooms. In this phase it is very important that there is no kind of communication (verbal or non-verbal) from the instructor to the test person. Now the test starts:
1) The instructor flips a coin to choose a NAS, notes the result of the coin flip on a sheet of paper and starts playing the track from this NAS on a renderer in the room of the test person.
2) The test person listens to the track and tries to decide which NAS was used to play the track.
3) When the decision of the test person is ready, the test person notes the result of the listening test on a sheet of paper.
4) The test person informs the instructor that the decision is ready (claps with hands, for example).
5) The instructor stops playback and starts over at step 1)
These steps should be repeated N times, where N >= 20. The higher N is, the more significant the result will be.

Evaluation phase: To evaluate the results, the matches of the test person have to be counted. In terms of statistical testing, there would be two hypothesis: the null hypothesis H0 and the alternative H1.
H0: There is no audible difference between two NAS types (p = 0.5, where p is the probability to decide for the right NAS type in a single listening test).
H1: There is a audible difference (p > 0.5)
When, for example, N would be 20 and there were 15 matches of the test person, H0 could be rejected with a probability of error of pe ~ 2.07%. If there were more matches, pe would be even lower. pe is the probability that H0 is rejected although H0 is true: pe = P(H1 is accepted | H0 is true).
This means that 15 or more matches (for N = 20) would indicate a significant difference in sound quality between the two NAS types.

I'm aware that thoroughly carrying out such a test requires a lot of time and effort. I think, however, the result would be very interesting and if such a test showed a significant result, it may be able to convince even me that the NAS has an influence on the sound quality