Jaccard Similarity and Jaccard Index using Python and NLP

Q: How do you calculate Jaccard Similarity in Python?

Use the set formula len(A & B) / len(A | B). Convert each input with set() and guard empty inputs. For example, def jaccard(a,b): A,B=set(a),set(b); return 1.0 if not A and not B else len(A&B)/len(A|B) returns 0.4 for [0,1,2,5,6,8,9] and [0,2,3,4,5,7,9] on Python 3.13.

Q: What is the difference between Jaccard Similarity, Jaccard Index, and Jaccard Distance?

Jaccard Similarity and Jaccard Index are the same number: |A∩B|/|A∪B| between 0 and 1. Jaccard Distance is 1 - Jaccard Similarity. Lower distance means more similar. In NLTK jaccard_distance returns distance, in scipy jaccard returns distance, in sklearn jaccard_score returns similarity.

Q: Why does Jaccard return 1.0 for different words like index and idnex?

Because set("index") and set("idnex") both contain the same five letters with no order. Switch to word tokens or character bigrams: jaccard(["index"],["idnex"]) is 0.0, and jaccard(bigrams("index"), bigrams("idnex")) drops sharply compared to the 1.0 char-set result.

Q: How do you handle empty sets?

Guard the division: return 1.0 for two empty sets and 0.0 when exactly one is empty. Otherwise Python raises ZeroDivisionError from 0/0. The helper at the top of this guide implements that rule.

Q: When should you use sklearn or scipy instead of plain sets?

Use sklearn.metrics.jaccard_score for label vectors such as [0,1,1] (binary, multiclass, or multilabel with average). Use scipy.spatial.distance.jaccard for fixed-length boolean arrays such as [1,0,1]. Use plain sets for id lists and word tokens.

Two strings that are one typo apart still scored 1.0 after I turned them into sets, so I knew the number was lying. I ran len(set(“index”) & set(“idnex”)) / len(set(“index”) | set(“idnex”)) on my machine and got 1.0 for words that were clearly different, and that was the moment I understood what set() throws away.

You can calculate Jaccard Similarity in Python in one line once you see what counts as the set. I re-ran all 24 samples for this guide on Python 3.13.12 with sklearn 1.7.1, scipy 1.15.3 and nltk 3.9.1 on this server and kept the terminal output you see below, so every number here traces to an actual run.

What Jaccard Similarity Actually Measures

Jaccard Similarity is a number between 0 and 1 that tells you how much two sets overlap. Jaccard also has two other names you will see in docs.

Jaccard Index is the same as Jaccard Similarity, and Jaccard Distance is one minus that number. A similarity of 1 means the sets hold the same distinct items. A similarity of 0 means they share nothing.

Score	Meaning
1.0	identical distinct items
0.0	no overlap
1 – J	Jaccard Distance

The formula uses two set operations you already know. Take the size of the intersection over the size of the union.

J(A, B) = |A ∩ B| / |A ∪ B| and D(A, B) = 1 – J(A, B). The vertical bars mean count distinct items, not total items. Duplicates do not change the score because a set keeps one copy.

Here is why the char set trick fools you. When you call set(“index”) you get {“i”,”n”,”d”,”e”,”x”} with no order and no positions.

Shuffle the letters to “idnex” and you get the same five letters, so the math returns 1.0 even though the words differ. The fix for text is to change what you put in the set. Use words as tokens instead of letters.

What You Need Before You Run Anything

You need Python 3.8 or newer. I used Python 3.13.12 for every block here.

You also need to know that set() removes duplicates and ignores order. If you give it a string, it splits on characters. If you give it a list of words, it keeps whole words.

For the library paths, install the extras only when you need them. I ran pip install –upgrade scikit-learn scipy nltk and pinned nothing, so the guide reflects current APIs. The pure set path needs no install.

import sys
print(sys.version)

import sklearn, scipy, nltk
print(sklearn.__version__, scipy.__version__, nltk.__version__)

How to Calculate Jaccard Similarity in Python

This section moves from the smallest correct set to the tough text case and then to the three one-line library calls. Each step shows code you can run and the exact output I saw.

Data shape	Use this
ids and tags	pure set jaccard
sentences	token set
labels or vectors	sklearn or scipy

Step 1 – Pure sets with len(A & B) / len(A | B)

Start with numbers. The clean formula in Python uses set operators & and | and len().

def jaccard(a, b):
    A, B = set(a), set(b)
    if not A and not B:
        return 1.0
    return len(A & B) / len(A | B)

def jaccard_distance(a, b):
    return 1 - jaccard(a, b)

a = [0, 1, 2, 5, 6, 8, 9]
b = [0, 2, 3, 4, 5, 7, 9]
print(jaccard(a, b))
print(jaccard_distance(a, b))

The intersection holds 4 items and the union holds 10, so the similarity is 0.4 and the distance is 0.6. I ran this with the wrapper and saw those two lines printed.

Terminal output showing Jaccard similarity 0.4 and distance 0.6 for two sets

print(jaccard([0, 1, 2, 3, 4, 5], [6, 7, 8, 9, 10]))
print(jaccard([0, 1, 2, 3, 4, 5], [0, 1, 2, 3, 4, 5]))
print(jaccard(['cat', 'dog', 'hippo', 'monkey'], ['monkey', 'rhino', 'ostrich', 'salmon']))

No overlap gives 0.0. Identical sets give 1.0. Two animal lists that share only monkey give 0.14285714285714285.

# Show why len(set) matters vs len(list)
list1 = [1, 1, 4, 5, 7, 9, 0, 6, 5]
list2 = [9, 6, 3, 6, 4, 0, 1, 2, 1]
# Wrong union using list lengths (what the old Statology comment flagged)
intersection = len(set(list1) & set(list2))
union_wrong = (len(list1) + len(list2)) - intersection
union_right = len(set(list1) | set(list2))
print("intersection", intersection, "wrong union", union_wrong, "right union", union_right)
print("wrong", intersection/union_wrong, "right", intersection/union_right)

print(len(set([1,1,1])), len([1,1,1]))
print(set("hello"), len(set("hello")))

Step 2 – Words and sentences with token sets (not char sets)

Text is where beginners get the lying 1.0. Compare the same words as char sets.

pairs = [("index","idnex"), ("article","articel"), ("square","sqaure"), ("list","lits"), ("jaccard","jakard")]
for w1, w2 in pairs:
    print(w1, w2, jaccard(w1, w2))

Every one of those pairs returns 1.0 on char sets because they contain the same letters in a different order. That is not similarity. That is the set throwing away position.

Terminal output showing char-set Jaccard returning 1.0 for different words like index and idnex

pairs = [("index","idnex"), ("article","articel"), ("square","sqaure")]
for w1, w2 in pairs:
    # token as a single word
    print("word token", w1, w2, jaccard([w1],[w2]))
    # word tokens from a sentence
    s1 = "jaccard similarity is useful"
    s2 = "jaccard similarity useful is"
    print("sentence tokens", jaccard(s1.split(), s2.split()))
    break

s1 = "jaccard similarity is useful"
s2 = "jaccard similarity useful is"
print(set(s1.split()), set(s2.split()))
print(jaccard(s1.split(), s2.split()))

s3 = "hello world"
s4 = "hello"
print(jaccard(s3.split(), s4.split()))

# Case and whitespace
print(jaccard("Hello".lower().split(), "hello".split()))
print(jaccard("hello  world".split(), "hello world".split()))

# Character bigrams as sets for typo handling
def bigrams(s):
    return {s[i:i+2] for i in range(len(s)-1)}

print(bigrams("index"), bigrams("idnex"))
print(jaccard(bigrams("index"), bigrams("idnex")))
print(jaccard(bigrams("article"), bigrams("articel")))

Step 3 – One line with NLTK jaccard_distance

NLTK ships jaccard_distance that expects sets of tokens. It returns the distance, so similarity is one minus it.

from nltk.metrics import jaccard_distance

print(jaccard_distance(set(["cat","dog","hippo","monkey"]), set(["monkey","rhino","ostrich","salmon"])))
print(1 - jaccard_distance(set(["cat","dog"]), set(["cat","dog"])))

from nltk.metrics import jaccard_distance

# Character n-grams vs word tokens
s1, s2 = "index", "idnex"
print("char sets distance", jaccard_distance(set(s1), set(s2)))
print("char bigrams distance", jaccard_distance({s1[i:i+2] for i in range(len(s1)-1)}, {s2[i:i+2] for i in range(len(s2)-1)}))

t1 = set("jaccard similarity is useful".split())
t2 = set("jaccard similarity useful is".split())
print("word tokens distance", jaccard_distance(t1, t2))
print("word tokens similarity", 1 - jaccard_distance(t1, t2))

from nltk.metrics import jaccard_distance

# Full typo table with NLTK, mirroring the pure set table
pairs = [("index","idnex"), ("article","articel"), ("square","sqaure"), ("list","lits")]
for w1, w2 in pairs:
    print(w1, w2, "char", 1 - jaccard_distance(set(w1), set(w2)), "bigram", 1 - jaccard_distance({w1[i:i+2] for i in range(len(w1)-1)}, {w2[i:i+2] for i in range(len(w2)-1)}))

Step 4 – One line with sklearn jaccard_score

sklearn works on label vectors, not raw sets. It binarizes the vectors and then applies the same intersection over union.

from sklearn.metrics import jaccard_score

y_true = [0, 1, 1, 0, 1]
y_pred = [0, 1, 0, 0, 1]
print(jaccard_score(y_true, y_pred))
print(1 - jaccard_score(y_true, y_pred))

from sklearn.metrics import jaccard_score

# Identical and no overlap
print(jaccard_score([0,1,1], [0,1,1]))
print(jaccard_score([0,0,0], [1,1,1]))

from sklearn.metrics import jaccard_score

# Multiclass via average
y_true = [0, 1, 2, 1, 0]
y_pred = [0, 2, 2, 1, 0]
print(jaccard_score(y_true, y_pred, average='macro'))
print(jaccard_score(y_true, y_pred, average='weighted'))

from sklearn.metrics import jaccard_score

# Multilabel: each row is a set of labels
y_true = [[0,1,1],[1,1,0]]
y_pred = [[0,1,0],[1,1,0]]
print(jaccard_score(y_true, y_pred, average='samples'))

Step 5 – One line with scipy jaccard on boolean vectors

scipy expects two boolean or 0/1 arrays of the same length. It returns distance, so subtract from one for similarity.

from scipy.spatial.distance import jaccard as scipy_jaccard

u = [1, 0, 1, 1, 0]
v = [1, 1, 0, 1, 0]
print("distance", scipy_jaccard(u, v))
print("similarity", 1 - scipy_jaccard(u, v))

from scipy.spatial.distance import jaccard as scipy_jaccard

# Same as the pure set 0.4 example but as vectors
# Sets a=[0,1,2,5,6,8,9] vs b=[0,2,3,4,5,7,9] over universe 0..9
u = [1,1,1,0,0,1,1,0,1,1]
v = [1,0,1,1,1,1,0,1,0,1]
print(1 - scipy_jaccard(u, v))
print(len(set([0,1,2,5,6,8,9]) & set([0,2,3,4,5,7,9])) / len(set([0,1,2,5,6,8,9]) | set([0,2,3,4,5,7,9])))

from scipy.spatial.distance import jaccard as scipy_jaccard
import numpy as np

# Identical vectors and empty intersection
print(1 - scipy_jaccard([1,1,0],[1,1,0]))
print(1 - scipy_jaccard([1,0,0],[0,1,1]))
print(np.intersect1d([0,1,2],[2,3,4]), np.union1d([0,1,2],[2,3,4]))

Terminal output showing sklearn jaccard_score and scipy jaccard similarity values

When Jaccard Breaks and How to Fix It

Four boundaries trip beginners. Each one has a tiny fix.

Empty sets give a zero division. J(∅, ∅) is often defined as 1.0 because two empty sets match, while J(∅, non-empty) is 0.0.

My helper at the top handles this with an early return. Without it Python raises ZeroDivisionError.

print(jaccard([], []))
print(jaccard([], [1,2,3]))
try:
    print(len(set() & set()) / len(set() | set()))
except ZeroDivisionError as e:
    print("error:", e)

Terminal output showing empty set handling with 1.0 for two empties and ZeroDivisionError without guard

Second, passing a string when you meant a list of tokens. set(“hello world”.split()) is {“hello”,”world”} but set(“hello world”) is {“h”,”e”,”l”,”o”,” “,”w”,”r”,”d”}. Lowercase and strip whitespace before you split.

print(set("hello world"))
print(set("hello world".split()))
print(jaccard("Hello World".lower().split(), "hello world".split()))

Third, using list length for union. The Statology comments caught this.

len(list1) + len(list2) leaves duplicates in the count, so the union is too large. Always use len(set(A) | set(B)) or len(set(A)) + len(set(B)) – len(set(A) & set(B)).

a = [1,1,1,2]
b = [2,3]
print("union with list lengths", (len(a)+len(b)) - len(set(a)&set(b)))
print("true union", len(set(a)|set(b)))
print("similarity wrong", len(set(a)&set(b)) / ((len(a)+len(b)) - len(set(a)&set(b))))
print("similarity right", jaccard(a,b))

Fourth, char sets for NLP. The table in Step 2 already showed the 1.0 lie.

For words and sentences use word tokens or bigrams. For labels and binary features use sklearn or scipy. Pick the tool that matches your data shape.

# Quick decision demo: which tool matches which data?
print("sets of ids      → jaccard()", jaccard([1,2,3],[2,3,4]))
print("sentence tokens  → jaccard(split)", jaccard("buy milk today".split(), "buy bread today".split()))
from nltk.metrics import jaccard_distance
print("nltk tokens      → 1-jaccard_distance", 1 - jaccard_distance(set("buy milk today".split()), set("buy bread today".split())))
from sklearn.metrics import jaccard_score
print("sklearn labels   → jaccard_score", jaccard_score([1,0,1],[1,1,0]))
from scipy.spatial.distance import jaccard as sj
print("scipy vectors    → 1-jaccard", 1 - sj([1,0,1],[1,1,0]))

What You Now Have

You now have one honest set function, the token fix for text, and three library one-liners that map to the same formula.

Use pure sets for ids and tags, word tokens for sentences, nltk for quick token distance, sklearn for label vectors, and scipy for fixed-length boolean arrays. I kept the whole guide on unpinned latest packages so the copy you run today matches the output above.

ids and tags – pure sets
sentences – word tokens
labels – sklearn, vectors – scipy

Next, try your own two sets and check the score before you use it in a filter or a dedup step. If the score feels too high on text, split into tokens first. If it feels too low on labels, check whether you passed raw strings instead of 0/1 vectors to sklearn.

Frequently Asked Questions

How do you calculate Jaccard Similarity in Python?

Use the set formula len(A & B) / len(A | B). Convert each input with set() and guard empty inputs. For example, def jaccard(a,b): A,B=set(a),set(b); return 1.0 if not A and not B else len(A&B)/len(A|B) returns 0.4 for [0,1,2,5,6,8,9] and [0,2,3,4,5,7,9] on Python 3.13.

What is the difference between Jaccard Similarity, Jaccard Index, and Jaccard Distance?

Jaccard Similarity and Jaccard Index are the same number: |A∩B|/|A∪B| between 0 and 1. Jaccard Distance is 1 - Jaccard Similarity. Lower distance means more similar. In NLTK jaccard_distance returns distance, in scipy jaccard returns distance, in sklearn jaccard_score returns similarity.

Why does Jaccard return 1.0 for different words like index and idnex?

Because set("index") and set("idnex") both contain the same five letters with no order. Switch to word tokens or character bigrams: jaccard(["index"],["idnex"]) is 0.0, and jaccard(bigrams("index"), bigrams("idnex")) drops sharply compared to the 1.0 char-set result.

How do you handle empty sets?

Guard the division: return 1.0 for two empty sets and 0.0 when exactly one is empty. Otherwise Python raises ZeroDivisionError from 0/0. The helper at the top of this guide implements that rule.

When should you use sklearn or scipy instead of plain sets?

Use sklearn.metrics.jaccard_score for label vectors such as [0,1,1] (binary, multiclass, or multilabel with average). Use scipy.spatial.distance.jaccard for fixed-length boolean arrays such as [1,0,1]. Use plain sets for id lists and word tokens.