Introduction

In the midst of all of the bad news in the COVID era, there are good news reports of the important work done by data engineers and data scientists to get the data of all COVID cases around the world together and provide useful insights that will be useful for pan professionals.

The objective of this article is to focus on a use case that demonstrates the integration between daily changing Source, Azure Databricks, and Power BI to deliver insights and data visualizations using a publicly available COVID-19 dataset. While Azure Databricks provides the distributed computing power to process and transform complex datasets, Power BI is a fitting recipient of the transformed dataset that surfaces these insights to business users.

Source

The latest available public data on the geographic distribution of COVID-19 cases worldwide from the European Center for Disease Prevention and Control (ECDC). Each row/entry contains the number of new cases reported per day and per country or region.

Before we start with our exercise, we will need to have the following prerequisites:

1. You need to have an active Azure Subscription.
2. Azure Databricks — You need to set up both Databricks service and cluster in Azure, you can go over the steps in this article. As shown in this article, we have created a Databricks service.
3. Power BI Subscription

Let’s go ahead and demonstrate the data load into Databricks using Python notebooks from the source mentioned above.

STEP 1: Create Azure Databricks

The next step is to create a Databricks Workspace. You can think of the workspace like an application that you are installing within Azure, where you will access all of your Databricks assets. Follow these steps to create a workspace:

• On the Azure home screen, click ‘Create a Resource’.
• In the ‘Search the Marketplace’ search bar, type ‘Databricks’ and you should see ‘Azure Databricks‘ pop up as an option. Click that option.
• Click ‘Create’ to begin creating your workspace.

Use the same resource group you created or selected earlier. Then, enter a workspace name. Remember to always stick to naming standards when creating Azure resources, but for now, enter whatever you would like. You can keep the location as whatever comes default or switch it to a region closer to you. For the pricing tier, select ‘Trial’. Finally, select ‘Review and Create‘. We can skip-networking and tags for now which are for more advanced set-ups.

• This should bring you to a validation page where you can click ‘create’ to deploy your workspace. This will bring you to a deployment page and the creation of the workspace should only take a couple of minutes

• Once the deployment is complete, click ‘Go to resource‘ and then click ‘Launch Workspace’ to get into the Databricks workspace.

• Create a Workspace (Notebook) with Python or your choice of language for performing analysis.

STEP 2: Start Scripting in Notebook.

• Import relevant libraries and fetch data from the source according to its data type.

• Digging the data to get useful and relevant analytical data

1. Top Countries with cases and deaths.

2. Calculating total cases, deaths, and death rates around the globe.

3. Visualizing total cases and deaths with the timeline.

4. Calculating fatality rate with top 8 countries.

5. Calculating the global change rate per day.

5.1 Global Rate Change for cases per day.

5.2 Global Rate Change for deaths per day.

6. Calculating Change rate for top countries only.

6.1 Top countries Rate Change for cases.

6.2 Top countries Rate Change for deaths.

7. Fetching Country particular data (e.g. INDIA)

8. Fetching Continent particular data.

9. Storing credentials from data bricks clusters. Save SERVER HOSTNAME & HTTP PATH.

Step 3: Data brick Job Scheduling

The very important scope of this article is to capture the daily changing data, then we have to run every other command for every day when new data will be in our source to avoid this, we will automate the whole process and schedule a job to run our notebook with an interval of a day, so it will check for a new file and apply all the commands accordingly.

Click on (calendar-like symbol), on the top-right corner of your notebook.

• Click on the ‘+New’ button.

• Schedule a job according to your need and then click ’ok’.

Step 4: Connecting Power Bi to your Databricks.

• Click on’ Get Data’ and search for ‘Azure databricks’ to add databricks credentials that we get from previous steps.

• Click on ‘DATA’ to get a view and validation for the tables in Power BI.

• Start making insightful reports by applying different filters and visuals, a sample report is here to get an overview.

Here you can select Individual countries or groups of them to assign them with a particular date with the help of slicers to view analytics accordingly.

This is the end of this article, we successfully automate the process of Loading, Transforming, and Writing the data process from a web source and then migrating the processed data to Power BI, it will also read new data and apply all the transformations on it with an interval that was defined by the developer.

Thank you and feel free to try out some different transformations and create some new visuals in the Power BI Report.

Neuroergonomics is the study of the human brain in relation to human factors and ergonomics. Specifically, it looks to understand and improve performance through biological measures. Currently, neuroergonomics is an emerging method of research within HF and generally considered outside of standard HF research, but the idea of it being a distinct form of research will not be around forever.

Just like cognitive neuroscience was once an emerging field within cognitive psychology, cognitive psychology now is deeply reliant on neuroscience. By today’s standards, if cognitive psychology doesn’t use neuroscience techniques it seems outdated. Similarly, I believe that as technology advances HF practitioners will more often use, and rely on, neuroergonomic approaches and just like cognitive psychology, there will come a day when omitting neuroergonomics will make HF research seem dated.

Why is neuroergonomics needed?

By studying the brain, we are able to collect data that otherwise would have been impossible to collect. During user studies, most of the data we collect from users (specifically the human factors team at Cambridge Consultants, though this is fairly standard) falls into either observable behaviour or subjective feedback, with the occasional body measurement (height, reach, wingspan, grip strength, etc.). Observable behaviour is a great indicator of task performance, but we also strive to understand what participants are thinking as they complete a task. Subjective feedback allows us to understand what participants are thinking, however, most actions and thoughts are unconscious meaning people do not actually know what stimuli initiated a behaviour. Additionally, people are astoundingly bad at describing their thought processes objectively (For more on how cognitive biases affect self-report measures, see this article I wrote).

Neuroergonomics has immeasurable potential because it adds an additional type of data we can collect, allowing us to gain a greater understanding of the underlying biological and neurological processes driving our actions. Furthermore, it allows us to use biological indicators to measure and assess performance.

Like traditional data, we can collect and analyze this to better understand operators as they interact with a system. However, unlike traditional data, some of the data collection methods allow us to use the data in real-time, creating adaptive safety measure and the next generation of smart devices.

I will discuss two major topics being researched in neuroergonomics: mental workload, and human error. In each section, I will provide an overview of the topic, an explanation of how it is beneficial to the HF field and ways the research can practically be implemented in the (near) future.

Mental workload

One of the most discussed and important topics in HF research is mental workload. Ensuring operators are cognitively able to perform a task is essential for safe and efficient behaviour, regardless of whether the operator is a surgeon or a truck driver. Until recently, the mental workload has been studied using subjective tests like the NASA TLX and secondary task paradigms. However, both of these methods have flaws that limit their applicability and utility.

While the NASA TLX is fast, cheap, and can be administered anywhere, it is subjective in nature. On the other hand, multitasking studies attempt to be realistic and naturalistic, but often force participants to complete tasks within specific parameters, which can’t always model real-world usage.

Using a neuroergonomic approach, there are multitudes of different methods to track cognitive workload and multitasking. An fMRI can trace the blood flow in the brain, indicating what region is being used. Causse, Dehais, Péran, Sabatini, & Pastor used fMRI to study pilots’ decision making while landing planes. The researchers assessed whether priming pilots to land quickly due to economic concerns (e.g. fuel costs and trip delays) altered their decision making during non-optimal landing conditions. They concluded that there was a shift in the parts of the brain used once economic concerns entered the equation. The fMRI indicated a shift from the dorsolateral prefrontal cortex to the ventromedial prefrontal cortex, a shift from using the brain area involved in reasoning and logic to a region involved in emotional processing.

If this study was done using standard human factors methods, it is entirely possible that there would be no difference in observable behaviour, and the results would be the same for both conditions. In real life, maybe there is no difference in outcomes for 99.9% of the time. However, knowing that economic pressure causes the emotional processing region to be activated, it is possible a pilot under preexisting emotional stress when landing can cause compromised outcomes due to higher mental workload. This study allowed researchers to discover safety-critical information that otherwise would have stayed hidden using classic HF techniques.

Another tool to measure workload is EEG readings. Researchers found that the amount of theta activity, brain activity in the 4–7.5 Hz range, in the frontal lobe is positively correlated to a higher mental workload (Gevins & Smith 2003). Since EEG machines are portable and minimally intrusive, this can be an integral method to accurately judge workload in naturalistic settings.

Furthermore, there has also been a wealth of research indicating that mental workload can accurately be assessed using heart rate variability. Schnell, Postnikov, and Hamel(2011) tested the mental workload of pilots using a low fidelity fighter-jet simulator and pilots flying a real fighter jet. Using EEG and heart rate monitors, they recorded the data and compared it to the estimated cognitive demands each task required. They found that the pilot’s heart rate was an efficient, reliable, and simple method to determine mental workload. This is another example of a cheap and non-intrusive method that can bolster the current methods of assessing mental workload.

Due to the structure of the FDA guidance on human factors, we often focus on ensuring devices are safe by minimizing risk, as well as intuitive and enjoyable to use. Those are all important, but we rarely, if ever consider is what is going on behind the scenes (i.e. the biological basis of behaviour) For example, how can we really understand what surgeons are experiencing during a robotics surgery without a neuroergonomic approach to mental workload? Is the device presenting an appropriate level of information that surgeons can effectively concentrate for a 2 hour, 6 hours, or 10-hour surgery? Is the device presenting so much unnecessary information that the surgeon is less cognizant of the necessary information? Does the common practice of playing music during surgery increase the mental workload of the surgeon, requiring adapting the UI for the actual use environment?

These are all important questions to consider when designing and evaluating all types of systems, not just surgical robotics, where operator performance is crucial to outcomes and would otherwise be hidden without neuroergonomic methods.

Human Error

The next topic looks at creating systems that are less sensitive to human error, and, if an error occurs, are able to minimize the potential effects retroactively. Historically, human error has had catastrophic effects as seen in cases like the Bhopal disaster, the Three Mile Island accident, and the 1971 Iraq poison grain disaster. Even more recently, events like the Costa Concordia grounding and the Durban train collision remind us how dangerous errors can be even with our advance systems and technological safeguards in place. Each of these catastrophic events was the result of human error and potentially could have been avoided with better system designs.

Costa Concordia grounding

Unfortunately, human error isn’t just the cause of rare calamitous events, but something that plagues us daily. Doctors and nurses, some of the few people we entrust our lives to, are notoriously bad at mitigating human error. An estimated 400,000 hospital patients die prematurely each year due to preventable harms. Equally concerning, an estimated 4–8 million patients per year experience serious harm as a result of preventable error.

Every task has room for error to occur, which is why this is such an integral field of study. Under some circumstances when an error is about to be performed or an error is detected, regardless of who or what made the error, the brain produces a stereotyped response called error-related negativity (ERN)(Bates, Patel & Liddle, 2005). Using an EEG, ERN signals can be recorded and used to monitor for errors in order to make corrections in real-time (Parasuraman, 2011).

While we currently can’t attach an EEG to every person to monitor for errors, knowing that the brain monitors and reacts to errors has far-reaching implications. If we had a headband that had EEG capabilities and could sync with our smartphone, we would be able to monitor for errors in real-time and get a notification any time an error is detected.

EEG headband currently on the market

Not only could this improve outcomes in every field by allowing you to correct a mistake before it is too late, but by realizing your mistakes in real-time, you can learn from your mistakes and, over time, potentially decrease the rate of errors.

The applications for this technology get really interesting when you couple it with existing technologies such as Near Field Communication (NFC) and Bluetooth. Imagine you were working in the ER and accidentally hit the wrong button as you are inputting an order into a Pyxis machine. The EEG could detect the ERN was detected and send an alert to your smartphone. The smartphone, with NFC technology, could know that you were using the Pyxis machine and would instruct the machine to cancel the order so you could start again.

Pyxis storage cabinet

Alternatively, if a nurse accidentally set up the wrong dose or titration schedule on an infusion pump, the EEG could detect an ERN and alert the user to an error.

The applications for this type of connected devices are essentially endless. The combination of devices interacting with ERN signals could form the next generation of “smart’ products, enabling us to use our unconscious error detection mechanisms to create intelligent machines with built-in safeguards and require less user input.

Where does this leave us?

While right now none of these applications is practical in industry, hopefully, this shed some light on some of the potential advantages to integrating neuroergonomics into human factors. As these topics continue to be researched and technology advances, there will be a point in time that these tools move from idealistic to attainable and commonplace. When that day comes, it will be interesting to see how these new tools will alter human factors research and implementation of safety measures.

If you enjoyed the post, please share this article or click the thumbs up icon below to let me know!

Objective

Deep neural networks have been successfully used in diverse emerging domains to solve real-world complex problems with many more deep learning (DL) architectures, being developed to date. To achieve these state-of-the-art performances, the DL architectures use activation functions (AFs), to perform diverse computations between the hidden layers and the output layers of any given DL architecture. Here, we introduce common types of non-linear activation functions that are alternative to the well known sigmoid function and then evaluate their characteristics. Moreover deeper neural networks will be analyzed because they positively influence the final performances compared to shallower networks. They also strictly depend on the weight initialization hence the effect of drawing weights from Gaussian and uniform distribution will be analyzed making particular attention on how the number of incoming and outgoing connection to a node influences the whole network.

Introduction

The main activation function that was widely used is the Sigmoid function, however, when the Rectifier Linear Unit (ReLU) (Nair & Hinton, 2010) was introduced, it soon became a better replacement for the Sigmoid function due to its positive impact on the different machine learning tasks. After that, different variants of the ReLU activation function have been introduced and this experiment explores them and their impact on the Forest Cover Type dataset. The forest cover type (the predominant kind of tree cover) from strictly cartographic variables (as opposed to remotely sensed data). The actual forest cover type for a given 30 x 30-meter cell was determined from US Forest Service (USFS) Region 2 Resource Information System data. Independent variables were then derived from data obtained from the US Geological Survey and USFS. The data is in raw form (not scaled) and contains binary columns of data for qualitative independent variables such as wilderness areas and soil type.

This study area includes four wilderness areas located in the Roosevelt National Forest of northern Colorado. These areas represent forests with minimal human-caused disturbances, so that existing forest cover types are more a result of ecological processes rather than forest management practices.

· Introduction to the data set

The study area includes four wilderness areas located in the Roosevelt National Forest of northern Colorado. Each observation is a 30m x 30m patch. You are asked to predict an integer classification for the forest cover type. The seven types are:

1 — Spruce/Fir 5 — Aspen
2 — Lodgepole Pine 6 — Douglas-fir
3 — Ponderosa Pine 7 — Krummholz
4 — Cottonwood/Willow

The training set (15120 observations) contains both features and the Cover_Type. The test set contains only the features. You must predict the Cover_Type for every row in the test set (565892 observations).

o Data Fields

· Elevation — Elevation in meters

· Aspect — Aspect in degrees azimuth

· Slope — Slope in degrees

· Horizontal_Distance_To_Hydrology — Horz Dist to nearest surface water features

· Vertical_Distance_To_Hydrology — Vert Dist to nearest surface water features

· Horizontal_Distance_To_Roadways — Horz Dist to the nearest roadway

· Hillshade_9am (0 to 255 index) — Hillshade index at 9am, summer solstice

· Hillshade_Noon (0 to 255 index) — Hillshade index at noon, summer solstice

· Hillshade_3pm (0 to 255 index) — Hillshade index at 3pm, summer solstice

· Hillshade_3pm (0 to 255 index) — Hillshade index at 3pm, summer solstice

· Hillshade_3pm (0 to 255 index) — Hillshade index at 3pm, summer solstice

· Horizontal_Distance_To_Fire_Points — Horz Dist to nearest wildfire ignition points

· Wilderness_Area (4 binary columns, 0 = absence or 1 = presence) — Wilderness area designation

· Soil_Type (40 binary columns, 0 = absence or 1 = presence) — Soil Type designation

· Cover_Type (7 types, integers 1 to 7) — Forest Cover Type designation

o The wilderness areas are:

· Rawah Wilderness Area

· Neota Wilderness Area

· Comanche Peak Wilderness Area

· Cache la Poudre Wilderness Area

o The soil types are:

1) Cathedral family — Rock outcrop complex, extremely stony.
2) Vanet — Retake families complex, very stony.
3) Haploborolis — Rock outcrop complex, rubbly.
4) Ratake family — Rock outcrop complex, rubbly.
5) Vanet family — Rock outcrop complex, rubbly.
6) Vanet — Wetmore families — Rock outcrop complex, stony.
7) Gothic family.
8) Supervisor — Limber families complex.
9) Troutville family, very stony.
10) Bullwark — Catamount families — Rock outcrop complex, rubbly.
11) Bullwark — Catamount families — Rock land complex, rubbly.
12) Legault family — Rock land complex, stony.
13) Catamount family — Rock land — Bullwark family complex, rubbly.
14) Pachic Argiborolis — Aquolis complex.
15) Unspecified in the USFS Soil and ELU Survey.
16) Cryaquolis — Cryoborolis complex.
17) Gateview family — Cryaquolis complex.
18) Rogert family, very stony.
19) Typic Cryaquolis — Borohemists complex.
20) Typic Cryaquepts — Typic Cryaquolls complex.
21) Typic Cryaquolls — Leighcan family, till substratum complex.
22) Leighcan family, till substratum, extremely bouldery.
23) Leighcan family, till substratum — Typic Cryaquolls complex.
24) Leighcan family, extremely stony.
25) Leigh can family, warm, extremely stony.
26) Granile — Catamount families’ complex, very stony.
27) Leigh can family, warm — Rock outcrop complex, extremely stony.
28) Leigh can family — Rock outcrop complex, extremely stony.
29) Como — Legault families’ complex, extremely stony.
30) Como family — Rock land — Legault family complex, extremely stony.
31) Leigh can — Catamount families’ complex, extremely stony.
32) Catamount family — Rock outcrop — Leigh can family complex, extremely stony.
33) Leigh can — Catamount families — Rock outcrop complex, extremely stony.
34) Cryorthents — Rock land complex, extremely stony.
35) Cryumbrepts — Rock outcrop — Cryaquepts complex.
36) Bross family — Rock land — Cryumbrepts complex, extremely stony.
37) Rock outcrop — Cryumbrepts — Cryorthents complex, extremely stony.
38) Leigh can — Moran families — Cryaquolls complex, extremely stony.
39) Moran family — Cryorthents — Leigh can family complex, extremely stony.
40) Moran family — Cryorthents — Rock land complex, extremely stony.

An exploratory study on feature selection

Feature engineering is a process of transforming the given data into a form which is easier to interpret. Here, we are interested in making it more transparent for an ML & DL model, but some features can be generated so that the data visualization prepared for people without a data-related background can be more digestible. However, the concept of transparency for the learning models is a complicated thing as different models often require different approaches for the different kinds of data.

· LOADING OF DATASET

This dataset includes information on tree type, shadow coverage, distance to nearby landmarks (roads etcetera), soil type, and local topography. Here, are loading our data set and importing all the python packages that will be useful in exploration, visualization of our data and predicting our model.

· Data Statistics

1. Head: This is another python function head () which gives the overview of our dataset with 5 initial rows along with all the columns available.

2. Shape: This is a python function shape () which determines the size of the data frame, as we can see 581012 instances are having 54 attributes. This also gives an evident confirmation that our dataset is successfully loaded as it matches with the data description.

3. Data types: Exploring data types through the function info () which gives a lot of other insights about our dataset.

· There are 581012 entries in each column

· We can see all columns has been inferred as int64.

· There are no null values, so it’s no sense to use any missing value techniques (e.g. Imputer).

· Total memory usage of our dataset is 243.8 Mb.

4. Statistical description: This can be explored through describe () function which computes the summary of statistics pertaining to the Data-Frame columns. This function gives the mean, std and IQR( interquartile range) values. This function excludes the character columns (although our dataset doesn’t have any) and given summary about numeric columns.

· No attribute is missing as the count is 581012 for all attributes. Hence, all rows can be used.

· Negative value(s) present in Vertical_Distance_To_Hydrology. Hence, some tests such as chi-sq can’t be used.

· Wilderness_Area and Soil_type are one-hot encoded. Hence, they could be converted back for some analysis

· All attributes have some distinct values and no constant attributes, so there is no sense in removing any columns.

· Scales are not the same for all, hence, rescaling and standardization may be necessary for some algorithm.

5. Skew: In probability theory and statistics, skewness is a measure of the asymmetry of the probability distribution of a real-valued random variable about its mean. The skewness value can be positive or negative, or undefined. Skewness is asymmetry in a statistical distribution, in which the curve appears distorted or skewed either to the left or to the right. Skewness can be quantified to define the extent to which a distribution differs from a normal distribution.

· Values close to 0 show less skew

· Several attributes in Soil_Type show a large skew. Hence, some algos may benefit if the skew is corrected.

6. Class Distribution: Class distribution is the way to determine whether all the classes are having the same number of instances or not. Imbalanced class distribution, it’s a scenario where the number of observations belonging to one class is significantly lower than those belonging to the other classes.

Learning algorithms tend to produce unsatisfactory classifiers when faced with imbalanced datasets. For any imbalanced data set, if the event to be predicted belongs to the minority class and the event rate is less than 5%, it is usually referred to as a rare event.

Class 4 (having lowest appearances of a class in ‘Cover_type’): 2747

Total no. of observation: 581,102

% of Class 4: 0.47

· Class 4 is having the <1 percentage of presence.

· Classes 3, 4, 5, 6, 7 are having a very low presence in comparison to Classes 1 & 2 in ‘Cover_type’.

Here, we distributed each class with the same number of rows & columns to make imbalance class distribution ineffective.

Exploratory Data Analysis

· PCA ( Principal Component Analysis)

One of the common problems in the analysis of complex data comes from a large number of variables, which requires a large amount of memory and computation power. This is where Principal Component Analysis (PCA) comes in. It is a technique to reduce the dimension of the feature space by feature extraction.

· Data Integration

1. Correlation:

· Correlation tells the relation between two attributes.

· Correlation requires continuous data. Hence, ignore Wilderness_Area and Soil_Type as they are binary.

2. Scatter Plot (pair plot): Scatter plot is extremely intuitive yet powerful. Just plot the vertical coordinate and horizontal coordinate of each data point in the sample to get its scatter plot. If the relationship is non-linear, or there may be the presence of an outlier, these targets will be visible in the scatter plot. In the case of many features i.e. dimensions, a scatter-plot matrix can be used.

· The plots show to which class a point belongs to. The class distribution overlaps in the plots.

· Hill shade patterns give nice ellipsoid patterns with each other.

· Aspect and Hill shades attribute to form a sigmoid pattern.

· Horizontal and vertical distance to hydrology give an almost linear pattern.

· Data Visualization

1. HEAT MAP: Heat map will feature variables as row headers and column headers, and the variable vs. itself on the diagonal — is an extremely powerful way to visualize relationships between variables in high dimensional space.

2. BOX PLOT: t displays essential stats about distribution in a concise visual form. Aka candlestick plot. Also popular in finance. This is known as the box and whisker graph too. It’s popular among statisticians. Used to visualize range. It can be drawn horizontally.

· Max, 3rd Quartile, Median, 1st Quartile, min are the feature can be easily detectable through box plot.

· Box whisker plot displays outliers as a dot!

3. PAIR PLOT: If we want 4d visualization for scatter plot, this is when Pair plot from seaborn package comes into play. Let’s say we have n number of features in a data, Pair plot will create us a (n x n) figure where the diagonal plots will be histogram plot of the feature corresponding to that row and rest of the plots are the combination of feature from each row in y-axis and feature from each column in x-axis.

4. LM PLOT: Every plot in Seaborn has a set of fixed parameters. We have three mandatory parameters and the rest are optional that we may use as per our requirements. These 3 parameters are values for X-axis, values for Y-axis and reference to the dataset. These 3 are pre-dominantly visible in almost all of Seaborn plots and besides, there is an optional parameter which I would like you to memorize as it comes in very handy. This is a hue parameter and it takes in categorical columns and kind of helps us to group our data plot as per hue parameter values.

· Horizontal_Distance_To_Hydrology & Vertical_Distance_To_Hydrology with Soil_Type2.

· Horizontal_Distance_To_Hydrology & Vertical_Distance_To_Hydrology with Wilderness_Area1.

5. VIOLIN PLOT: The violin plots can be inferred as a combination of Box plot at the middle and distribution /density plots(Kernel Density Estimation ) on both side of the data. This can give us the details of distribution like whether the distribution is multimodal, skewness etc. It also gives us the useful info like 95% confidence interval.

· Elevation is has a separate distribution for most classes. Highly correlated with the target and hence an important attribute.

· Aspect contains a couple of normal distribution for several classes

· Horizontal distance to road and hydrology have a similar distribution.

· Hillshade 9 am and 12 pm display left skew.

· Hillshade 3 pm is normal.

· Lots of 0s in the vertical distance to hydrology.

· Wilderness_Area3 gives no class distinction. As values are not present, others give some scope to distinguish.

· Soil_Type, 1,5,8,9,12,14,18–22, 25–30 and 35–40 offer class distinction as values are not present for many classes.

Grouping of one hot encoded attributes: For categorical variables where no such ordinal relationship exists, the integer encoding is not enough. In this case, a one-hot encoding can be applied to the integer representation. This is where the integer encoded variable is removed and a new binary variable is added for each unique integer value.

· WildernessArea_4 has a lot of presence for cover_type 4. Good class distinction.

· WildernessArea_3 has not much class distinction.

· SoilType 1–6,10–14,17, 22–23, 29–33,35,38–40 offer lot of class distinction as counts for some are very high.

· Data Cleaning

Removing Unnecessary Columns: There are maybe some columns in our data set which will not help in any prediction because they don’t have any specific distinct values to make them even considerable and just creating some noises here. This can be determined by calculating the standard deviation of each column, which we already did through the statistical description.

· Add constant columns as they don’t help in the prediction process.

· Removing the ones which have the standard deviation equals to zero.

· No columns have that standard, so there is no need to remove any column.

· Data Preparation

1. Standardization(Standard Scalar): The idea behind Standard Scalar is that it will transform your data such that its distribution will have a mean value 0 and standard deviation of 1. Given the distribution of the data, each value in the dataset will have the sample mean value subtracted and then divided by the standard deviation of the whole dataset.

2. Normalization: Normalization is one of the methods used in data science to bring features in a dataset to the same scale. When you normalize a feature, all feature values will be in the range of 0 to 1.

· Select numerical columns which need to be normalized.

· Normalize Training Data.

· Converting numpy array to a data frame.

· Normalize Testing Data by using the mean and SD of the training set.

Evaluation, prediction, and analysis

This study aims to analyze the performance of generalized MLP architectures which has a back-propagation algorithm using various activation functions for the neurons of hidden and output layers. For experimental comparisons Sigmoid, Tanh, ReLU & Softmax.

The activation function used to transform the activation level of a unit (neuron) into an output signal. There are several common activation functions in use with artificial neural networks (ANN). The most common choice of activation functions for multi-layered perceptron (MLP) is used as transfer functions in research and engineering. Among the reasons for this popularity is its boundedness in the unit interval, the function’s and it’s derivative’s fast computability and several amenable mathematical properties in the realm of approximation theory. However, considering the huge variety of problem domains MLP is applied in, it is intriguing to suspect that specific problems call for single or a set of specific activation functions.

· Here, we can’t use the sigmoid activation function. Our dataset output layer has 7 different classes to determine, which means it’s a multi-classification task but sigmoid is a binary classification algorithm as it just transforms the output to either 0 or 1.

· The sigmoid and hyperbolic tangent activation functions cannot be used in networks with many layers due to the vanishing gradient problem.

· Instead of sigmoid activation, we can use softmax activation function which significantly great in building a multi classifier as it a form of logistic regression that normalizes an input value into a vector of values that follows a probability distribution whose total sums up to 1.

· It should be noted that softmax is not ideally used as an activation function like Sigmoid, Tanh & ReLU (Rectified Linear Units) between layers which may be multiple or just a single one but as an output layer.

1. Validating Data Through ReLU (Rectified Linear Units): The solution of gradient descent is to use the rectified linear activation function, or ReL for short, the main idea is to let the gradient be non zero and recover during training eventually. A node or unit that implements this activation function is referred to as a rectified linear activation unit or ReLU for short. Often, networks that use the rectifier function for the hidden layers are referred to as rectified networks. ReLU is less computationally expensive than tanh and sigmoid because it involves simpler mathematical operations. That is a good point to consider when we are designing deep neural nets. The rectified linear activation function is a simple calculation that returns the value provided as input directly, or the value 0.0 if the input is 0.0 or less.

· Softmax activation is great to use as output layer (as it normalizes an input value into a vector of values that follows a probability distribution whose total sums up to 1) in neural networks with ReLU as the middle layer.

· As Y variable is multi-class categorical variable, hence using softmax as activation function and sparse-categorical cross-entropy as the loss function.

· Using 2 layers with 64 neurons on both hidden layers and 8 neurons on the output layer.

· We use 26 epochs, one forward passes and one backward pass of all the training examples.

· Here we take batch_size of 60 for this model, the number of training examples in one forward/backwards pass. The higher the batch size, the more memory space you’ll need.

Visualize Training History

2. Validating Data Through Tanh: The Hyperbolic Tangent Activation Function. This is due in part to the fact that if a strongly-negative input is provided to the logistic sigmoid, it outputs values very near zero. Like the logistic sigmoid, the tanh function is also sigmoid (“s”-shaped), but instead outputs values that range. Its outputs range from 0 to 1 and are often interpreted as probabilities (in, say, logistic regression). The tanh function, a.k.a. hyperbolic tangent function, is a rescaling of the logistic sigmoid, such that its outputs range from -1 to 1.

The advantage is that the negative inputs will be mapped strongly negative and the zero inputs will be mapped near zero in the tanh graph.

· The function is differentiable.

· The function is monotonic while its derivative is not monotonic.

· We use 54 neurons on the input layer as the total columns are of the same size.

· Here 100 neurons are placed with 4 hidden layers & softmax function is used in the output layer.

· We used dropout function with a value of 0.2 to deal regularization in the deep learning model.

· The tanh function is mainly used classification between two classes.

No. Of Epochs, Loss on validation, Accuracy on validation, Actual loss on Model & Actual accuracy on Model.

Training and Validation Accuracy

Model Accuracy

Conclusion

In this study, we have used 3 conventional differentiable and monotonic activation functions for the evolution of MLP architecture along with Generalized Delta rule learning. These proposed well-known and effective activation functions are TanH, ReLU & Softmax. Having compared their performances, simulation results show that ReLU(rectified linear unit) function performs better recognition accuracy than those of the other functions. In other words, the neural network computed good results when “ReLU-Softmax” combination of activation functions was used for neurons (or nodes) of hidden and output layers. The use softmax layer as a hidden layer is not possible as we have to keep all our nodes (hidden variables) linearly dependent which may result in many problems and poor generalization.

• ReLu is less computationally expensive than tanh because it involves simpler mathematical operations.
• Softmax were used in output layers instead of the hidden layer.
• ‘Tanh-Softmax ‘combination gives good results instead of ‘Tanh-Tanh’ combination with an accuracy of 61%.
• ‘ReLU-Softmax’ combination gives the best results with perfect accuracy of 100%.
• Normalization and Standardization help the algorithm to achieve these good accuracies.

Comparison of non-linear activation functions for deep neural networks on multi-classification task was originally published in Analytics Vidhya on Medium, where people are continuing the conversation by highlighting and responding to this story.