Question about results

[asher-stern]

Hi.
Thank you for sharing your code.
I tried it. Technically it works fine. However, I am puzzled when examining its results.

I've generated two score files, almost identical. Only 4 scores out of 10000 are different, and the difference is very small. Nevertheless, when comparing alg2.txt to alg1.txt (alg2 is the first argument), I get very small values of epsilon. This is the case for various values of alpha, big and small ones. This means that the comparison software concludes that in a very high confidence alg2 is better than alg1. However, the reasonable conclusion should be that there is no difference and no edge to any of the score files.

Similarly, when I compare a score file to itself (say alg1 to alg1), it prints "The denominator is 0" but returns a result, and that result is that epsilon is very small for the first algorithm. Again, this makes no sense since it compares the same score-file to itself. Epsilon should be 0.5 in such a case.

The files are attached here (see below).

Thank you very much in advance.

alg1.txt
alg2.txt

[helboukkouri]

Hi @asher-stern ,

I don't know if this will help you but you may want to try the fix in #2 and check if it solves your issue :)

Hicham