I notice that the default row of dataFrame of launch_gradio_widget is set to 2. Since the gradio does not provide delete_row option in the webui interface, the calculation of some web demos (e.g., exact_match, I don't test all) includes the blank row, which may confuse the user.
For example, if we as the users only hope to calculate the exact match score of one sample, thus we only fill the first row in the webui and left the second unremovable row blank, the returned calculation results is always wrong since it counts the second blank text row as a 1.0 score (exactly match! because they are both 0 length '' str).
The above example is shown in this figure.

Actually, I have posted a discussion in huggingface web demo nearly one month ago and no one responsed.
Here are some potential useful suggestions to fix this issue:
-
Considering the upstream package gradio is not tending to add delete a row option (related issue discussed in here), a quick fix is set the default row to 1 instead of 2 in evaluate/utils/gradio.py/launch_gradio_widget/LINE 115.
-
Check the user's input data and discard these null inputs (blank text input), which could be done in evaluate/utils/gradio.py/parse_test_cases function.
And I would like to have a PR for fixing this if the the community welcome.😁
I notice that the default row of
dataFrameoflaunch_gradio_widgetis set to2. Since the gradio does not providedelete_rowoption in the webui interface, the calculation of some web demos (e.g., exact_match, I don't test all) includes the blank row, which may confuse the user.For example, if we as the users only hope to calculate the exact match score of one sample, thus we only fill the first row in the webui and left the second unremovable row blank, the returned calculation results is always wrong since it counts the second blank text row as a
1.0score (exactly match! because they are both 0 length '' str).The above example is shown in this figure.
Actually, I have posted a discussion in huggingface web demo nearly one month ago and no one responsed.
Here are some potential useful suggestions to fix this issue:
Considering the upstream package gradio is not tending to add
delete a rowoption (related issue discussed in here), a quick fix is set the default row to1instead of2inevaluate/utils/gradio.py/launch_gradio_widget/LINE 115.Check the user's input data and discard these null inputs (blank text input), which could be done in
evaluate/utils/gradio.py/parse_test_casesfunction.And I would like to have a PR for fixing this if the the community welcome.😁