Deep Learning & Advanced Forecasting

Neural Networks, LSTMs, and Time-Series Prediction

When traditional ML hits its ceiling, **Deep Learning** takes over. Neural networks with millions of parameters can capture patterns that simpler models miss. For **Walmart**, **LSTM (Long Short-Term Memory)** networks predict demand weeks ahead by learning seasonal patterns. For **Netflix**, deep learning powers the recommendation engine that serves 230 million users.

In this chapter, we explore **Neural Network fundamentals** (layers, activation functions, backpropagation), **LSTMs** for time-series forecasting, and **CNNs** (Convolutional Neural Networks) for pattern recognition in Visa's transaction data.

Neural Networks: Layers & Activation Functions

A **Neural Network** is a series of layers, each containing "neurons" that apply a mathematical transformation. The **Input Layer** takes the raw data, **Hidden Layers** extract features, and the **Output Layer** produces the prediction. **Activation Functions** (like ReLU and Sigmoid) introduce non-linearity, allowing networks to learn complex patterns.

PythonRuns entirely in your browser — nothing is sent to a server.

# Neural Network: Forward Pass Simulation
class NeuralNetwork:
    def __init__(self, layer_sizes):
        self.layers = layer_sizes

    def relu(self, x):
        return max(0, x)

    def sigmoid(self, x):
        import math
        return round(1 / (1 + math.exp(-x)), 4)

    def forward_pass(self, inputs):
        print("NEURAL NETWORK: {0}".format(" -> ".join(str(s) for s in self.layers)))
        print("Input: {0}".format(inputs))
        
        # Simulate passing through layers
        current = inputs
        for i in range(1, len(self.layers)):
            # Simplified weighted sum
            weighted = [round(v * 0.5 + 0.1, 3) for v in current]
            activated = [self.relu(v) for v in weighted]
            print("Layer {0} ({1} neurons): {2} -> ReLU -> {3}".format(
                i, self.layers[i], [round(w, 2) for w in weighted[:3]], activated[:3]))
            current = activated[:self.layers[i]]
        
        output = self.sigmoid(sum(current))
        print("Output (Sigmoid): {0}".format(output))
        return output

# Scenario: Visa fraud classification network
nn = NeuralNetwork(layer_sizes=[3, 4, 2, 1])
nn.forward_pass(inputs=[750, 1, 0])  # [amount, is_foreign, is_night]

# Neural Network: Forward Pass Simulation
class NeuralNetwork:
    def __init__(self, layer_sizes):
        self.layers = layer_sizes

    def relu(self, x):
        return max(0, x)

    def sigmoid(self, x):
        import math
        return round(1 / (1 + math.exp(-x)), 4)

    def forward_pass(self, inputs):
        print("NEURAL NETWORK: {0}".format(" -> ".join(str(s) for s in self.layers)))
        print("Input: {0}".format(inputs))
        
        # Simulate passing through layers
        current = inputs
        for i in range(1, len(self.layers)):
            # Simplified weighted sum
            weighted = [round(v * 0.5 + 0.1, 3) for v in current]
            activated = [self.relu(v) for v in weighted]
            print("Layer {0} ({1} neurons): {2} -> ReLU -> {3}".format(
                i, self.layers[i], [round(w, 2) for w in weighted[:3]], activated[:3]))
            current = activated[:self.layers[i]]
        
        output = self.sigmoid(sum(current))
        print("Output (Sigmoid): {0}".format(output))
        return output

# Scenario: Visa fraud classification network
nn = NeuralNetwork(layer_sizes=[3, 4, 2, 1])
nn.forward_pass(inputs=[750, 1, 0])  # [amount, is_foreign, is_night]

LSTM: Time-Series Demand Forecasting

**LSTM (Long Short-Term Memory)** networks are a special type of RNN designed to remember patterns over long sequences. For **Walmart**, LSTMs learn that "sales spike 3 weeks before Diwali, dip on Mondays, and surge during monsoon." This allows them to forecast demand weeks ahead, factoring in seasonality that linear models miss.

PythonRuns entirely in your browser — nothing is sent to a server.

# Walmart Demand: LSTM Forecasting Simulation
class LSTMForecaster:
    def __init__(self, history):
        self.history = history
        self.memory = 0 # LSTM's "cell state"

    def predict_next(self, n_steps):
        print("LSTM FORECASTING ({0} steps ahead):".format(n_steps))
        print("Historical Data: {0}".format(self.history))
        
        # Simulate LSTM memory gates
        predictions = []
        for step in range(n_steps):
            # Forget gate: partially forget old memory
            forget = 0.7
            self.memory = self.memory * forget
            
            # Input gate: incorporate recent trend
            recent_avg = sum(self.history[-3:]) / 3
            trend = self.history[-1] - self.history[-2]
            self.memory += recent_avg * 0.3
            
            prediction = round(recent_avg + trend * (step + 1) * 0.5, 0)
            predictions.append(int(prediction))
            print(" - Step {0}: Memory={1:.1f}, Trend={2}, Forecast={3}".format(
                step + 1, self.memory, trend, prediction))
        
        print("\nFORECAST: {0}".format(predictions))

# Scenario: Walmart weekly demand (units)
weekly_demand = [1000, 1200, 1100, 1400, 1350, 1600]
lstm = LSTMForecaster(weekly_demand)
lstm.predict_next(n_steps=3)

# Walmart Demand: LSTM Forecasting Simulation
class LSTMForecaster:
    def __init__(self, history):
        self.history = history
        self.memory = 0 # LSTM's "cell state"

    def predict_next(self, n_steps):
        print("LSTM FORECASTING ({0} steps ahead):".format(n_steps))
        print("Historical Data: {0}".format(self.history))
        
        # Simulate LSTM memory gates
        predictions = []
        for step in range(n_steps):
            # Forget gate: partially forget old memory
            forget = 0.7
            self.memory = self.memory * forget
            
            # Input gate: incorporate recent trend
            recent_avg = sum(self.history[-3:]) / 3
            trend = self.history[-1] - self.history[-2]
            self.memory += recent_avg * 0.3
            
            prediction = round(recent_avg + trend * (step + 1) * 0.5, 0)
            predictions.append(int(prediction))
            print(" - Step {0}: Memory={1:.1f}, Trend={2}, Forecast={3}".format(
                step + 1, self.memory, trend, prediction))
        
        print("\nFORECAST: {0}".format(predictions))

# Scenario: Walmart weekly demand (units)
weekly_demand = [1000, 1200, 1100, 1400, 1350, 1600]
lstm = LSTMForecaster(weekly_demand)
lstm.predict_next(n_steps=3)

CNN: Pattern Recognition in Transaction Data

**Convolutional Neural Networks (CNNs)** are famous for image recognition, but they also excel at detecting local patterns in sequential data. For **Visa**, CNNs scan transaction sequences looking for "shapes" that indicate fraud—like a burst of small transactions followed by one large withdrawal.

PythonRuns entirely in your browser — nothing is sent to a server.

# Visa Fraud: CNN Pattern Detection Simulation
class CNNPatternDetector:
    def __init__(self, kernel_size):
        self.kernel = kernel_size

    def detect_patterns(self, transaction_sequence):
        print("CNN PATTERN SCAN (Kernel={0}):".format(self.kernel))
        print("Transaction Sequence: {0}".format(transaction_sequence))
        
        # Slide a kernel window across the sequence
        patterns = []
        for i in range(len(transaction_sequence) - self.kernel + 1):
            window = transaction_sequence[i:i + self.kernel]
            # Detect "burst" pattern: small amounts followed by large
            is_burst = all(w < 50 for w in window[:-1]) and window[-1] > 500
            
            if is_burst:
                patterns.append(("BURST", i, window))
                print(" - Window [{0}:{1}]: {2} -> BURST PATTERN DETECTED".format(i, i + self.kernel, window))
            else:
                print(" - Window [{0}:{1}]: {2} -> Normal".format(i, i + self.kernel, window))
        
        if patterns:
            print("\nALERT: {0} suspicious burst pattern(s) found!".format(len(patterns)))
        else:
            print("\nAll patterns normal.")

# Scenario: Scanning a cardholder's recent transactions
transactions = [10, 15, 12, 800, 50, 20]
cnn = CNNPatternDetector(kernel_size=4)
cnn.detect_patterns(transactions)

# Visa Fraud: CNN Pattern Detection Simulation
class CNNPatternDetector:
    def __init__(self, kernel_size):
        self.kernel = kernel_size

    def detect_patterns(self, transaction_sequence):
        print("CNN PATTERN SCAN (Kernel={0}):".format(self.kernel))
        print("Transaction Sequence: {0}".format(transaction_sequence))
        
        # Slide a kernel window across the sequence
        patterns = []
        for i in range(len(transaction_sequence) - self.kernel + 1):
            window = transaction_sequence[i:i + self.kernel]
            # Detect "burst" pattern: small amounts followed by large
            is_burst = all(w < 50 for w in window[:-1]) and window[-1] > 500
            
            if is_burst:
                patterns.append(("BURST", i, window))
                print(" - Window [{0}:{1}]: {2} -> BURST PATTERN DETECTED".format(i, i + self.kernel, window))
            else:
                print(" - Window [{0}:{1}]: {2} -> Normal".format(i, i + self.kernel, window))
        
        if patterns:
            print("\nALERT: {0} suspicious burst pattern(s) found!".format(len(patterns)))
        else:
            print("\nAll patterns normal.")

# Scenario: Scanning a cardholder's recent transactions
transactions = [10, 15, 12, 800, 50, 20]
cnn = CNNPatternDetector(kernel_size=4)
cnn.detect_patterns(transactions)

Practice Questions

Question 1

Why are LSTMs better than standard linear models for Walmart's demand forecasting?

They use more electricity
They can learn long-term seasonal patterns and non-linear trends that linear models cannot capture
They only work with Walmart data
They are simpler to implement

Question 2

How does a CNN detect fraud patterns in Visa's transaction sequences?

It reads the entire history at once
It slides a small window (kernel) across the sequence, detecting local patterns like bursts of small transactions followed by a large withdrawal
It only works with images
It compares transactions to a fixed template