Reconstruction benchmark

[ahojnnes]

Example output when invoked with evaluate.py --datasets eth3d --categories dslr ...:

=====scenes===== ======AUC @ X deg (%)====== ===images=== =components=
                  0.5     1      5      10      reg   all  num largest

==============================eth3d=dslr==============================
botanical_garden  81.55  90.66  98.12  99.06     30    30    2      30
boulders          80.29  89.84  97.90  98.95     26    26    1      26
bridge            83.77  91.75  98.34  99.17    110   110    1     110
courtyard         70.72  84.52  96.86  98.43     38    38    1      38
delivery_area     71.67  77.05  81.37  81.91     40    44    1      40
door              83.92  92.05  98.41  99.21      7     7    1       7
electro           68.45  74.89  81.07  81.90     41    45    1      41
exhibition_hall   23.75  33.77  43.75  45.35     68    68    2      68
facade            82.30  90.91  98.17  99.09     76    76    1      76
kicker            74.24  83.54  91.50  92.53     30    31    1      30
lecture_room      68.92  82.28  96.13  98.07     23    23    2      23
living_room       73.11  85.43  96.84  98.42     65    65    1      65
lounge             0.00   0.59   1.88   2.05      2    10    1       2
meadow            39.35  54.77  87.51  93.61     15    15    2      15
observatory       30.65  38.77  46.23  47.19     27    27    2      14
office            34.72  45.38  59.94  62.28     21    26    1      21
old_computer      30.83  39.51  47.94  49.05     54    54    1      54
pipes             78.57  89.27  97.85  98.93     14    14    1      14
playground        58.45  64.35  69.28  69.92     32    38    2      32
relief            46.40  48.04  49.35  49.51     31    31    2      18
relief_2          47.88  50.23  52.20  52.44     31    31    2      20
statue            97.36  98.68  99.74  99.87     11    11    1      11
terrace           83.09  91.51  98.30  99.15     23    23    1      23
terrace_2         81.98  90.33  97.96  98.98     13    13    1      13
terrains          69.29  84.10  96.66  98.33     42    42    1      42
----------------------------------------------------------------------
overall           66.37  75.22  83.24  84.33    870   898   33     833
----------------------------------------------------------------------
average           62.45  70.89  79.33  80.54     35    36    1      33

[sarlinpe]

Great work! I left mostly cosmetic comments that shouldn't change the results.

[ahojnnes]

Great work! I left mostly cosmetic comments that shouldn't change the results.

Thanks for the review. Did a bunch of refactoring and should have addressed all the comments.

[ahojnnes]

Now also added an A/B comparison tool between different runs.