NLP & Text Classification

Teaching Machines to Read

Unstructured text—reviews, emails, support tickets—contains some of the most valuable signals in business. For **Netflix**, NLP is how they analyze millions of reviews to understand user sentiment. For **Visa**, it's how they automatically route fraud complaints to the right team. This chapter introduces the core of **Natural Language Processing (NLP)**.

We cover **Tokenization**, **TF-IDF** (Term Frequency–Inverse Document Frequency), **Word Embeddings**, and the **Transformer Architecture** that powers GPT, BERT, and every modern language model.

Tokenization & Text Preprocessing

Before a machine can "read," text must be broken into **Tokens** (words or subwords). For **Netflix**, this means taking a review like "Absolutely loved Stranger Things!" and converting it into numerical tokens: [1042, 856, 2391, 5]. Stop words like "the" and "a" are removed, and words are **lowercased** and **stemmed**.

PythonRuns entirely in your browser — nothing is sent to a server.

# Netflix Review Processing: Tokenization
class TextPreprocessor:
    def __init__(self, stop_words):
        self.stop_words = set(stop_words)

    def tokenize(self, text):
        print("INPUT: '{0}'".format(text))
        # Step 1: Lowercase
        text = text.lower()
        # Step 2: Remove punctuation (simplified)
        clean = "".join(c for c in text if c.isalnum() or c == " ")
        # Step 3: Split into tokens
        tokens = clean.split()
        # Step 4: Remove stop words
        filtered = [t for t in tokens if t not in self.stop_words]
        
        print("LOWERCASE     : {0}".format(text))
        print("CLEANED       : {0}".format(clean))
        print("TOKENS        : {0}".format(tokens))
        print("AFTER STOPWORD: {0}".format(filtered))
        return filtered

# Scenario: Processing a Netflix review
preprocessor = TextPreprocessor(stop_words=["the", "a", "is", "it", "was", "and"])
preprocessor.tokenize("The show was absolutely AMAZING and thrilling!")

# Netflix Review Processing: Tokenization
class TextPreprocessor:
    def __init__(self, stop_words):
        self.stop_words = set(stop_words)

    def tokenize(self, text):
        print("INPUT: '{0}'".format(text))
        # Step 1: Lowercase
        text = text.lower()
        # Step 2: Remove punctuation (simplified)
        clean = "".join(c for c in text if c.isalnum() or c == " ")
        # Step 3: Split into tokens
        tokens = clean.split()
        # Step 4: Remove stop words
        filtered = [t for t in tokens if t not in self.stop_words]
        
        print("LOWERCASE     : {0}".format(text))
        print("CLEANED       : {0}".format(clean))
        print("TOKENS        : {0}".format(tokens))
        print("AFTER STOPWORD: {0}".format(filtered))
        return filtered

# Scenario: Processing a Netflix review
preprocessor = TextPreprocessor(stop_words=["the", "a", "is", "it", "was", "and"])
preprocessor.tokenize("The show was absolutely AMAZING and thrilling!")

TF-IDF: Finding Important Words

**TF-IDF** measures how important a word is to a document relative to an entire collection. For **Visa**, TF-IDF is used to analyze fraud complaint text—words like "unauthorized" and "foreign" score high because they're rare in normal complaints but common in fraud reports. This allows automatic ticket routing.

PythonRuns entirely in your browser — nothing is sent to a server.

# Visa Complaint Router: TF-IDF Scoring
import math

class TFIDFCalculator:
    def __init__(self, documents):
        self.docs = documents
        self.n_docs = len(documents)

    def compute_tfidf(self, target_word, doc_idx):
        doc = self.docs[doc_idx]
        words = doc.lower().split()
        # TF: frequency in this document
        tf = words.count(target_word.lower()) / len(words)
        # IDF: log(total_docs / docs_containing_word)
        containing = sum(1 for d in self.docs if target_word.lower() in d.lower())
        idf = math.log(self.n_docs / (containing + 1))
        tfidf = round(tf * idf, 4)
        
        print("WORD: '{0}' in Doc {1}".format(target_word, doc_idx + 1))
        print(" - TF  = {0:.4f} (frequency in this doc)".format(tf))
        print(" - IDF = {0:.4f} (rarity across all docs)".format(idf))
        print(" - TF-IDF = {0} (importance score)".format(tfidf))
        return tfidf

# Scenario: Visa complaint analysis
complaints = [
    "My card was used for an unauthorized foreign transaction",
    "I need to update my billing address please",
    "Unauthorized purchase detected on my account from foreign country"
]
calc = TFIDFCalculator(complaints)
calc.compute_tfidf("unauthorized", 0)
print("---")
calc.compute_tfidf("unauthorized", 1)

# Visa Complaint Router: TF-IDF Scoring
import math

class TFIDFCalculator:
    def __init__(self, documents):
        self.docs = documents
        self.n_docs = len(documents)

    def compute_tfidf(self, target_word, doc_idx):
        doc = self.docs[doc_idx]
        words = doc.lower().split()
        # TF: frequency in this document
        tf = words.count(target_word.lower()) / len(words)
        # IDF: log(total_docs / docs_containing_word)
        containing = sum(1 for d in self.docs if target_word.lower() in d.lower())
        idf = math.log(self.n_docs / (containing + 1))
        tfidf = round(tf * idf, 4)
        
        print("WORD: '{0}' in Doc {1}".format(target_word, doc_idx + 1))
        print(" - TF  = {0:.4f} (frequency in this doc)".format(tf))
        print(" - IDF = {0:.4f} (rarity across all docs)".format(idf))
        print(" - TF-IDF = {0} (importance score)".format(tfidf))
        return tfidf

# Scenario: Visa complaint analysis
complaints = [
    "My card was used for an unauthorized foreign transaction",
    "I need to update my billing address please",
    "Unauthorized purchase detected on my account from foreign country"
]
calc = TFIDFCalculator(complaints)
calc.compute_tfidf("unauthorized", 0)
print("---")
calc.compute_tfidf("unauthorized", 1)

The Transformer: Attention Is All You Need

The **Transformer Architecture** revolutionized NLP. Unlike older models that read text sequentially, Transformers use **Self-Attention** to consider ALL words simultaneously, understanding context and relationships. GPT, BERT, and every modern AI language model is built on Transformers. For **Netflix**, Transformers power their multi-language content recommendation engine.

PythonRuns entirely in your browser — nothing is sent to a server.

# Transformer Architecture: Self-Attention Simulation
class SelfAttention:
    def __init__(self, sentence):
        self.words = sentence.split()

    def compute_attention(self, target_idx):
        target = self.words[target_idx]
        print("SELF-ATTENTION for '{0}':".format(target))
        print("Sentence: {0}".format(" ".join(self.words)))
        
        # Simulate attention scores (how much each word relates to the target)
        scores = []
        for i, word in enumerate(self.words):
            if i == target_idx:
                score = 1.0
            elif abs(i - target_idx) <= 2:
                score = round(0.8 - abs(i - target_idx) * 0.2, 2)
            else:
                score = 0.1
            scores.append(score)
            print(" - '{0}' -> Attention: {1:.2f}".format(word, score))
        
        max_idx = scores.index(max(s for j, s in enumerate(scores) if j != target_idx))
        print("\nSTRONGEST LINK: '{0}' <-> '{1}'".format(target, self.words[max_idx]))

# Scenario: Netflix content understanding
attn = SelfAttention("The thrilling show had amazing visual effects")
attn.compute_attention(target_idx=2)

# Transformer Architecture: Self-Attention Simulation
class SelfAttention:
    def __init__(self, sentence):
        self.words = sentence.split()

    def compute_attention(self, target_idx):
        target = self.words[target_idx]
        print("SELF-ATTENTION for '{0}':".format(target))
        print("Sentence: {0}".format(" ".join(self.words)))
        
        # Simulate attention scores (how much each word relates to the target)
        scores = []
        for i, word in enumerate(self.words):
            if i == target_idx:
                score = 1.0
            elif abs(i - target_idx) <= 2:
                score = round(0.8 - abs(i - target_idx) * 0.2, 2)
            else:
                score = 0.1
            scores.append(score)
            print(" - '{0}' -> Attention: {1:.2f}".format(word, score))
        
        max_idx = scores.index(max(s for j, s in enumerate(scores) if j != target_idx))
        print("\nSTRONGEST LINK: '{0}' <-> '{1}'".format(target, self.words[max_idx]))

# Scenario: Netflix content understanding
attn = SelfAttention("The thrilling show had amazing visual effects")
attn.compute_attention(target_idx=2)

Practice Questions

Question 1

Why is TF-IDF more useful than simple word count for Visa's complaint routing?

It uses less memory
It weighs words by both their frequency in the document AND their rarity across all documents, surfacing truly distinctive terms
It only works with English
It is faster to compute

Question 2

What is the key innovation of the Transformer's Self-Attention mechanism?

It reads text faster by skipping words
It allows the model to consider relationships between ALL words in a sentence simultaneously, regardless of distance
It only works with short sentences
It requires labeled data to function