Adding the skin_mnist task to the default tasks

[walidabn]

• In the context of our dogfooding, we added a simple classification task, skin_mnist, that classifies skin diseases among 7 categories. The data is taken from the HAM10000 dataset, with data that is public on Kaggle / Harvard Dataverse.

[s314cy]

to be merged after #578 has been reviewed

[martinjaggi]

can we get more documentation on this new task (and a level higher how this did or not go easily with the documentation how to add new tasks)

docu needed on this task:

• reference the dataset more precisely for full reproducibility. a paper would be best but links can work too (problem of permanence)
• more details on model and train/test split
• are there published accuracy numbers on this dataset in the literature, and how does it compared to yours here?

[walidabn]

Task documentation

• **The Dataset : ** We trained our model on the HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. The dataset is also sometimes refered as "Skin MNIST" in practice. The dataset and its documentation are available at this link on the Harvard Dataverse, with this following Bibtex citation.
  @DaTa{DVN/DBW86T_2018,
  author = {Tschandl, Philipp},
  publisher = {Harvard Dataverse},
  title = {{The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions}},
  UNF = {UNF:6:KCZFcBLiFE5ObWcTc2ZBOA==},
  year = {2018},
  version = {V4},
  doi = {10.7910/DVN/DBW86T},
  url = {https://doi.org/10.7910/DVN/DBW86T}
  }

The dataset is also thoroughly described in this paper

• Model : We first tried to use AutoKeras to determine which model would be a good fit for this image classification problem. AutoKeras can, given a simple problem with a small dataset (here, an image classification task), try out a set of different models, and train/test them for a certain amount of epochs (here, 50), and returns the model which achives the highest test accuracy.
  This results in a convolutional neural network with 3 Conv2D layers with relu activation, respectively of size 256, 128 and 64, with (3,3) filters. Each layer is followed by a (2,2) max pooling layer, and a dropout layer with a 0.3 dropout probability. After the third convolutional layer, the input goes through a 32-Dense layer, before undergoing a softmax layer with 7 outputs, since the task is a 7-class classification problem. We split the train set equally between 3 clients in a federated setup. Each client has the same label distribution (i.e. each peer has the same number of examples for each of the 7 labels).

• Results : We compared the results obtained in this task with the same model and the same hyperparameters in a single setting case, where the single participant pooling all the data (the global case), which achieves 69.4% accuracy. In our federated setup with 3 clients, we reach 56% accuracy. State of the art models, such as RegNetY-320 reaches 91% accuracy on the task. Previous results, from AlexNet and InceptionV3 reached 85% accuracy. The current model we use is a simple convolutional neural network, with only 3 layers, without any kind of preprocessing, which explains the gap between state of the art models and our model.