Machine learning

Machine learning is the process of learning a function from data instead of programming every rule explicitly.

Given a dataset

\[ D=\{(x_i,y_i)\}_{i=1}^{N}, \]

where

• \(x_i\) = input (features)
• \(y_i\) = output (label or target)

the goal is to learn a function

\[ f_\theta(x) \approx y, \]

where \(\theta\) represents the model parameters learned from the data.

For a new input \(x_{\text{new}}\), the prediction is

\[ \hat{y}=f_\theta(x_{\text{new}}). \]

---

Types of Machine Learning

1. Supervised Learning

The training data contains both inputs and correct outputs.

Training dataset:

\[ D=\{(x_i,y_i)\}_{i=1}^{N}. \]

The objective is to find model parameters that minimize prediction error.

A common objective is

\[ \theta^*=\arg\min_{\theta} \sum_{i=1}^{N} L\!\left(y_i,f_\theta(x_i)\right), \]

where

• \(L(\cdot)\) is the loss function.
• \(\theta^*\) is the optimal parameter set.

---

2. Unsupervised Learning

Only inputs are available.

Dataset:

\[ D=\{x_i\}_{i=1}^{N}. \]

The objective is to discover hidden structure in the data.

Mathematically,

\[ x_i \longrightarrow C_j, \]

where \(C_j\) represents the cluster assigned to sample \(x_i\).

---

3. Reinforcement Learning

An agent interacts with an environment.

At time \(t\),

• State: \(s_t\)
• Action: \(a_t\)
• Reward: \(r_t\)

The objective is to maximize the expected cumulative reward

\[ G_t=\sum_{k=0}^{\infty}\gamma^k r_{t+k+1}, \]

where

• \(\gamma\in[0,1]\) is the discount factor.

---

How Machine Learning Works

Step 1: Data Input

Collect observations

\[ X= \begin{bmatrix} x_1\\ x_2\\ \vdots\\ x_N \end{bmatrix}, \qquad Y= \begin{bmatrix} y_1\\ y_2\\ \vdots\\ y_N \end{bmatrix}. \]

Each sample contains several features

\[ x_i=[x_{i1},x_{i2},\ldots,x_{id}]^T. \]

---

Step 2: Finding Patterns

The learning algorithm estimates model parameters

\[ \theta=f(D). \]

The goal is

\[ \text{Error} \rightarrow \min. \]

For many models,

\[ \hat{y}=f_\theta(x). \]

---

Step 3: Making Predictions

For an unseen example

\[ x_{\text{new}}, \]

the prediction is

\[ \hat{y}=f_\theta(x_{\text{new}}). \]

---

Common Machine Learning Algorithms

1. Linear Regression (Supervised Learning)

Linear regression models the relationship between input and output using a straight line.

Model

For one feature,

\[ \hat{y}=wx+b, \]

where

• \(w\) = slope
• \(b\) = intercept

For multiple features,

\[ \hat{y}=w^Tx+b. \]

The parameters are learned by minimizing the Mean Squared Error (MSE):

\[ J(w,b)=\frac{1}{N} \sum_{i=1}^{N} (y_i-\hat{y}_i)^2. \]

The optimization problem is

\[ (w,b)= \arg\min J(w,b). \]

Example: Predicting house price from house size.

---

2. K-Means Clustering (Unsupervised Learning)

K-Means partitions the data into \(K\) clusters.

Each cluster has a centroid

\[ \mu_1,\mu_2,\ldots,\mu_K. \]

Each point is assigned to its nearest centroid:

\[ C_i= \arg\min_j \|x_i-\mu_j\|^2. \]

The centroid is updated as

\[ \mu_j= \frac{1}{|C_j|} \sum_{x_i\in C_j} x_i. \]

The objective is to minimize the total within-cluster variance:

\[ J= \sum_{j=1}^{K} \sum_{x_i\in C_j} \|x_i-\mu_j\|^2. \]

Example: Grouping clothes by color without predefined labels.

---

3. Q-Learning (Reinforcement Learning)

Q-Learning estimates the value of taking action \(a\) in state \(s\).

The Q-value is

\[ Q(s,a). \]

The update rule is

\[ Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s',a') - Q(s,a) \right]. \]

where

• \(\alpha\) = learning rate
• \(\gamma\) = discount factor
• \(r\) = immediate reward
• \(s'\) = next state

The agent chooses the action with the highest Q-value:

\[ a^* = \arg\max_a Q(s,a). \]

Example: Teaching a pet a trick—correct actions receive rewards, increasing their Q-values, while incorrect actions receive little or no reward.

---

Summary

Concept	Mathematical Expression
Learning from data	\(f_\theta(x)\approx y\)
Supervised learning	\(\theta^*=\arg\min_\theta \sum_i L(y_i,f_\theta(x_i))\)
Unsupervised learning	\(x_i \rightarrow C_j\)
Reinforcement learning	\(G_t=\sum_{k=0}^{\infty}\gamma^k r_{t+k+1}\)
Linear regression	\(\hat{y}=wx+b\)
Linear regression loss	\(J=\frac{1}{N}\sum (y-\hat{y})^2\)
K-Means assignment	\(C_i=\arg\min_j\|x_i-\mu_j\|^2\)
K-Means objective	\(\sum_{j=1}^{K}\sum_{x_i\in C_j}\|x_i-\mu_j\|^2\)
Q-Learning update	\(Q\leftarrow Q+\alpha[r+\gamma\max Q(s',a')-Q]\)
Prediction	\(\hat{y}=f_\theta(x_{\text{new}})\)

These equations provide the mathematical foundation for the concepts described in the original text while keeping the connection between the intuition and the underlying ML algorithms clear.

---

---

---

Deep Learning

Deep learning is a branch of machine learning that uses artificial neural networks with multiple layers to automatically learn hierarchical representations from data. Unlike traditional machine learning, deep learning learns both the features and the mapping from input to output directly from the data.

Deep learning has achieved state-of-the-art performance in:

• Image recognition
• Speech recognition
• Natural language processing (NLP)
• Scientific computing
• Drug discovery
• Materials science

---

1. Mathematical Formulation

Suppose we have a dataset

\[ D=\left\{(x_i,y_i)\right\}_{i=1}^{N}, \]

where

• \(x_i\in\mathbb{R}^{d}\) : input feature vector
• \(y_i\) : corresponding target/output

The goal is to learn a function

\[ f_\theta:\mathbb{R}^{d}\rightarrow\mathbb{R}^{k}, \]

such that

\[ f_\theta(x)\approx y, \]

where

• \(\theta\) represents the trainable parameters
• \(d\) is the input dimension
• \(k\) is the output dimension

---

2. A Single Neural Network Layer

A fully-connected (dense) layer computes

\[ z=W x+b, \]

where

• \(W\) = weight matrix
• \(b\) = bias vector
• \(z\) = linear output

A nonlinear activation function is then applied

\[ a=\sigma(z), \]

where

\[ \sigma(\cdot) \]

may be

• ReLU
• Sigmoid
• tanh
• GELU
• Softplus

---

3. Deep Neural Network

A deep neural network stacks several layers together.

\[ a^{(0)}=x \]

For layer \(l\)

\[ z^{(l)}=W^{(l)}a^{(l-1)}+b^{(l)} \]

\[ a^{(l)}=\sigma\left(z^{(l)}\right) \]

Finally,

\[ \hat y=a^{(L)} \]

where

• \(L\) = total number of layers

Network Architecture

Input
  │
  ▼
┌─────────────┐
│ Hidden Layer│
└─────────────┘
        │
        ▼
┌─────────────┐
│ Hidden Layer│
└─────────────┘
        │
        ▼
┌─────────────┐
│ Hidden Layer│
└─────────────┘
        │
        ▼
     Output

---

4. Activation Functions

ReLU

\[ \mathrm{ReLU}(x)=\max(0,x) \]

Advantages

• Very fast
• Avoids vanishing gradients
• Most commonly used

---

Sigmoid

\[ \sigma(x)=\frac{1}{1+e^{-x}} \]

Output range

\[ 0<\sigma(x)<1 \]

Often used for binary classification.

---

tanh

\[ \tanh(x)=\frac{e^x-e^{-x}}{e^x+e^{-x}} \]

Output range

\[ -1<\tanh(x)<1 \]

Zero-centered activation.

---

5. Loss Function

For regression problems we commonly use the Mean Squared Error (MSE)

\[ L=\frac1N\sum_{i=1}^{N}\left(\hat y_i-y_i\right)^2 \]

where

• \(\hat y_i\) = prediction
• \(y_i\) = target

---

6. Learning the Parameters

Training aims to minimize the loss

\[ \theta^*=\arg\min_{\theta}L(\theta) \]

The gradient of the loss is computed using backpropagation

\[ \frac{\partial L}{\partial\theta} \]

Parameters are updated using Gradient Descent

\[ \theta\leftarrow \theta-\eta\nabla_\theta L \]

where

• \(\eta\) is the learning rate.

---

Backpropagation Diagram

Forward Pass

Input
  │
  ▼
Layer 1
  │
  ▼
Layer 2
  │
  ▼
Prediction
  │
  ▼
Loss

Backward Pass

Loss
 ▲
 │
Layer 2
 ▲
 │
Layer 1
 ▲
 │
Input

---

7. Why Deep Learning Works

Deep networks learn increasingly abstract features.

Input Image

      │

Edges

      │

Corners

      │

Textures

      │

Object Parts

      │

Complete Object

Examples

• Edge detection
• Texture learning
• Shape recognition
• Object identification

---

8. PyTorch Example

The following example learns the function

\[ y=\sin(x) \]

using a simple feedforward neural network.

import torch
import torch.nn as nn
import torch.optim as optim
import matplotlib.pyplot as plt

torch.manual_seed(0)

# Generate data
N = 200

x = torch.linspace(-3.14, 3.14, N).view(-1,1)
y = torch.sin(x)

# Define model
model = nn.Sequential(
    nn.Linear(1,64),
    nn.ReLU(),
    nn.Linear(64,64),
    nn.ReLU(),
    nn.Linear(64,1)
)

criterion = nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=1e-3)

epochs = 2000

losses = []

for epoch in range(epochs):

    optimizer.zero_grad()

    pred = model(x)

    loss = criterion(pred, y)

    loss.backward()

    optimizer.step()

    losses.append(loss.item())

    if (epoch+1) % 200 == 0:
        print(f"Epoch {epoch+1:4d} Loss = {loss.item():.6f}")

---

Prediction

with torch.no_grad():
    prediction = model(x)

plt.figure(figsize=(8,4))
plt.scatter(x.numpy(), y.numpy(), s=10, label="Training Data")
plt.plot(x.numpy(), prediction.numpy(), color="red", lw=2, label="Prediction")
plt.legend()
plt.xlabel("x")
plt.ylabel("y")
plt.show()

---

Training Workflow

Training Data
      │
      ▼
Neural Network
      │
      ▼
Prediction
      │
      ▼
Compute Loss
      │
      ▼
Backpropagation
      │
      ▼
Update Weights
      │
      ▼
Repeat

---

Key Takeaways

• Deep learning uses multiple layers to learn hierarchical features.
• Each layer performs a linear transformation followed by a nonlinear activation.
• Training minimizes a loss function using gradient-based optimization.
• Backpropagation computes gradients efficiently using the chain rule.
• PyTorch automates differentiation with autograd, making model development straightforward.