{"id":19602,"date":"2021-06-27T19:28:00","date_gmt":"2021-06-27T19:28:00","guid":{"rendered":"https:\/\/www.askpython.com\/?p=19602"},"modified":"2021-07-05T19:28:11","modified_gmt":"2021-07-05T19:28:11","slug":"bleu-score","status":"publish","type":"post","link":"https:\/\/www.askpython.com\/python\/bleu-score","title":{"rendered":"BLEU score in Python – Beginners Overview"},"content":{"rendered":"\n

Hello, readers! In this article, we will be focusing on the implementation of BLEU score in Python<\/strong>.<\/p>\n\n\n\n

So, let us get started! \ud83d\ude42<\/p>\n\n\n\n

Also read: Custom datasets in Python<\/a><\/strong><\/em><\/p>\n\n\n\n

\n\n\n\n

What is BLEU score?<\/h2>\n\n\n\n

In the domain of Machine Learning modeling, deep learning, and natural language processing, we need certain error metrics that enable us to evaluate the built model over the string input.<\/p>\n\n\n\n

BLEU score is one such metric that enables us to estimate the efficiency of the Machine Translation models or systems. Today, this has been widely used by Natural language processing models and applications altogether.<\/p>\n\n\n\n

Behind the scene, the BLEU score on terms compares the candidate sentence against the reference sentences and then estimates how well the candidate sentence is blended in accordance with the reference sentences. In this way, it rates the score between the range of 0 – 1, respectively.<\/p>\n\n\n\n

\n\n\n\n

Calculation of BLEU score in Python<\/h2>\n\n\n\n

To implement the BLEU score, we’ll use the NLTK module<\/a> which consists of sentence_bleu() function. It enables us to pass the reference sentences and a candidate sentence. Then, it checks the candidate sentence against the reference sentences. <\/p>\n\n\n\n

If a perfect match is found, it returns 1 as the BLEU score. If no match at all, it returns 0. For a partial match, the BLUE score will be between 0 and 1. <\/p>\n\n\n\n

Implementing BLEU Score<\/strong><\/h2>\n\n\n\n

In the below example, <\/p>\n\n\n\n

  1. We have imported the NLTK library and the sentence_bleu submodule.<\/li>
  2. Further, we generate a list of reference statements and point them through the object ref<\/strong>.<\/li>
  3. Then we create a test<\/strong> sentence and use sentence_bleu() to test it against ref<\/strong>.<\/li>
  4. As a result, it gives an approximate output as 1.<\/li>
  5. The next time, we create a test01<\/strong> statement and pass it to the function. <\/li>
  6. As the statement consists moonlight<\/strong> which is a part of the reference statements but not exactly a match for reference statements, thus it returns an approximate value close to 0.<\/li><\/ol>\n\n\n
    \nfrom nltk.translate.bleu_score import sentence_bleu\nref = [\n    'this is moonlight'.split(),\n    'Look, this is moonlight'.split(),\n    'moonlight it is'.split()\n]\ntest = 'it is moonlight'.split()\nprint('BLEU score for test-> {}'.format(sentence_bleu(ref, test)))\n\ntest01 = 'it is cat and moonlight'.split()\nprint('BLEU score for test01-> {}'.format(sentence_bleu(ref, test01)))\n<\/pre><\/div>\n\n\n
    Output–<\/strong><\/p>\n\n\n
    \nBLEU score for test-> 1.491668146240062e-154\nBLEU score for test01-> 9.283142785759642e-155\n<\/pre><\/div>\n\n\n
    \n\n\n\n
    Implementing N-gram score in Python<\/h2>\n\n\n\n
    As seen above, by default, the sentence_bleu() function searches for 1 word in the reference statements for a match. We can have multiple words in the queue to be searched against the reference statements. This is known as N-gram.<\/p>\n\n\n\n
    • 1-gram: 1 word<\/strong><\/li>
    • 2-gram: pairs of words<\/strong><\/li>
    • 3-gram: triplets<\/strong>, etc<\/li><\/ul>\n\n\n\n
      For the same, we can pass the below parameters to the sentence_bleu() function for implementation of N-gram:<\/p>\n\n\n
      \n1-gram: (1, 0, 0, 0)\n2-gram: (0, 1, 0, 0) \n3-gram: (1, 0, 1, 0)\n4-gram: (0, 0, 0, 1)\n<\/pre><\/div>\n\n\n
      Example<\/strong>:<\/p>\n\n\n\n
      In the below example, we have calculated the 2-gram BLEU score for the candidate sentence test01<\/strong> using the reference statements ref<\/strong> as mentioned below using sentence_bleu() function, passing the weights for 2-gram score i.e. (0,1,0,0).<\/p>\n\n\n
      \nfrom nltk.translate.bleu_score import sentence_bleu\nref = [\n    'this is moonlight'.split(),\n    'Look, this is moonlight'.split(),\n    'moonlight it is'.split()\n]\ntest01 = 'it is cat and moonlight'.split()\nprint('2-gram:' sentence_bleu(ref, test01, weights=(0, 1, 0, 0)))\n<\/pre><\/div>\n\n\n
      Output<\/strong>:<\/p>\n\n\n
      \n2-gram: 0.25\n<\/pre><\/div>\n\n\n
      \n\n\n\n
      Conclusion<\/h2>\n\n\n\n
      By this, we have come to the end of this topic. Feel free to comment below, in case you come across any question.<\/p>\n\n\n\n
      For more such posts related to Python programming, Stay tuned with us.<\/p>\n\n\n\n
      Till then, Happy learning!! \ud83d\ude42<\/p>\n","protected":false},"excerpt":{"rendered":"
      Hello, readers! In this article, we will be focusing on the implementation of BLEU score in Python. So, let us get started! \ud83d\ude42 Also read: Custom datasets in Python What is BLEU score? In the domain of Machine Learning modeling, deep learning, and natural language processing, we need certain error metrics that enable us to […]<\/p>\n","protected":false},"author":4,"featured_media":19668,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-19602","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-python"],"blocksy_meta":[],"_links":{"self":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/19602","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/comments?post=19602"}],"version-history":[{"count":0,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/posts\/19602\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media\/19668"}],"wp:attachment":[{"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/media?parent=19602"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/categories?post=19602"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.askpython.com\/wp-json\/wp\/v2\/tags?post=19602"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}