{"id":19559,"date":"2021-06-30T08:01:05","date_gmt":"2021-06-30T08:01:05","guid":{"rendered":"https:\/\/www.askpython.com\/?p=19559"},"modified":"2021-06-30T08:01:07","modified_gmt":"2021-06-30T08:01:07","slug":"activation-functions-python","status":"publish","type":"post","link":"https:\/\/www.askpython.com\/python\/examples\/activation-functions-python","title":{"rendered":"4 Activation Functions in Python to know!"},"content":{"rendered":"\n

Hello, readers! In this article, we will be focusing on Python Activation functions<\/strong>, in detail.<\/p>\n\n\n\n

So, let us get started!! \ud83d\ude42<\/p>\n\n\n\n

\n\n\n\n

What is an Activation function?<\/h2>\n\n\n\n

In the world of Neural networking and deep learning with convolutional models, Python has been playing a significant role when it comes to modeling and analysis of data. <\/p>\n\n\n\n

Activation functions are the mathematical base model that enables us to control the output of the neural network model. That is, it helps us analyze and estimate whether a neuron contributing to the enablement of the model is to be kept within or removed (fired).<\/p>\n\n\n\n

Some of the prominent Activation functions–<\/p>\n\n\n\n

  1. ReLu function<\/a><\/strong><\/li>
  2. Leaky ReLu function<\/a><\/strong><\/li>
  3. Sigmoid function<\/a><\/strong><\/li>
  4. Softmax function<\/a><\/strong><\/li>
  5. Linear function<\/a><\/strong>, etc.<\/li><\/ol>\n\n\n\n

    Having understood about Activation function, let us now have a look at the above activation functions in the upcoming section.<\/p>\n\n\n\n

    \n\n\n\n

    1. ReLu function<\/h2>\n\n\n\n

    ReLu function is a type of Activation function that enables us to improvise the convolutional picture of the neural network. It detects the state of the neural network in terms of the model results. <\/p>\n\n\n\n

    The ReLu function states that when the input is negative, return zero. Else for a non-negative input, it returns one.<\/p>\n\n\n\n

    Example<\/strong>:<\/p>\n\n\n\n

    Here, we have implemented a user-defined function to inculcate the ReLu condition using max() function<\/a> in Python.<\/p>\n\n\n

    \ndef ReLu(ar):\n    return max(0.0,ar)\nar = 1.0\nprint(ReLu(ar))\nar1= -1.0\nprint(ReLu(ar1))\n<\/pre><\/div>\n\n\n
    Output–<\/strong><\/p>\n\n\n
    \n1.0\n0.0\n<\/pre><\/div>\n\n\n
    \n\n\n\n
    2. Leaky ReLu function<\/h2>\n\n\n\n
    The gradient score i.e. the derivative value for the non-zero input passed to the ReLu function was found to be zero. Which basically stated that the weights are not being updated properly by the learning function. <\/p>\n\n\n\n
    To overcome this Gradient issue of ReLu function, we have been introduced to Leaky ReLu function.<\/p>\n\n\n\n
    Leaky ReLu function attaches a small linear component(constant value) to the negative (non-zero) input weight being passed to the function. By this, the gradient score for these non-zero input weights turned out to be a non-zero value.<\/p>\n\n\n\n
    Example<\/strong>:<\/p>\n\n\n
    \ndef ReLu(x):\n  if x>0 :\n    return x\n  else :\n    return 0.001*x\n\nx = -1.0\nprint(ReLu(x))\n<\/pre><\/div>\n\n\n
    Outpu<\/strong>t:<\/p>\n\n\n
    \n-0.001\n<\/pre><\/div>\n\n\n
    \n\n\n\n
    3. Sigmoid function<\/h2>\n\n\n\n
    The Sigmoid Activation function is simply based on the below sigmoid mathematical formula–<\/p>\n\n\n\n
    \"Sigmoid
    Sigmoid formula<\/strong><\/figcaption><\/figure><\/div>\n\n\n\n
    As the denominator is always greater than one, thus the output of this activation function is always between 0 and 1.<\/p>\n\n\n\n
    Example<\/strong>:<\/p>\n\n\n
    \nimport numpy as np \ndef sigmoid(num):\n return 1\/(1 + np.exp(-num))\nnum = -1.0\nprint(sigmoid(num))\n \n<\/pre><\/div>\n\n\n
    Output<\/strong>:<\/p>\n\n\n
    \n0.2689414213699951\n<\/pre><\/div>\n\n\n
    \n\n\n\n
    4. Softmax function<\/h2>\n\n\n\n
    The softmax activation function can be termed as a mathematical model that accepts a vector of numeric data variables as input and then normalizes the data.<\/p>\n\n\n\n
    That is, it normalizes<\/a> (scales the data values) to a probability distribution wherein the probability of every data value is proportional to the scale of every value present in the vector.<\/p>\n\n\n\n
    As a result, all the data values will be in the range of 0 – 1. Also, the summation of all the data values would be equal to 1 as they are being interpreted as probabilities.<\/p>\n\n\n\n
    \n\n\n\n
    Conclusion<\/h2>\n\n\n\n
    By this, we have come to the end of this topic. Feel free to comment below, in case you come across any question.<\/p>\n\n\n\n
    For more such posts related to Python programming, Stay tuned with us.<\/p>\n\n\n\n
    Till then, Happy Learning!! \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"
    Hello, readers! In this article, we will be focusing on Python Activation functions, in detail. So, let us get started!! \ud83d\ude42 What is an Activation function? In the world of Neural networking and deep learning with convolutional models, Python has been playing a significant role when it comes to modeling and analysis of data. Activation […]<\/p>\n","protected":false},"author":4,"featured_media":19566,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[9],"tags":[],"class_list":["post-19559","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-examples"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/19559","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=19559"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/19559\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/19566"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=19559"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=19559"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=19559"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}