# Computer Vision with Transformers: Beyond CNNs

> Vision Transformers (ViT) split images into patches and use attention for global context. Build, train, and compare vs CNNs.

Published: 2026-03-01  
Canonical: https://aimastermindlab.com/tutorials/computer-vision-transformers

---
Vision Transformers (ViT) replaced Convolutional Neural Networks (CNNs) in many cutting-edge architectures by treating image patches like words in a sentence.

## Why Transformers for Vision?
CNNs use local filters, meaning they have a very localized "receptive field." A pixel in the corner doesn't easily talk to a pixel in the opposite corner until deep in the network.
Transformers use **Self-Attention**, allowing every patch of the image to attend to every other patch immediately in the first layer, establishing deep global context.

## 1. Patch Imbedding
Instead of feeding raw pixels into a convolution, we split the image into a grid of 16x16 patches.
We flatten these patches, project them via a linear layer (creating the embedding), and add **Positional Encodings** (so the model knows where the patch sits).

```python
import torch
import torch.nn as nn

class PatchEmbedding(nn.Module):
    def __init__(self, in_channels=3, patch_size=16, emb_size=768):
        super().__init__()
        self.patch_size = patch_size
        self.proj = nn.Conv2d(in_channels, emb_size, kernel_size=patch_size, stride=patch_size)

    def forward(self, x):
        # x is (Batch, Channels, Height, Width)
        x = self.proj(x)  # (Batch, EmbSize, H/16, W/16)
        x = x.flatten(2)  # (Batch, EmbSize, NumPatches)
        x = x.transpose(1, 2) # (Batch, NumPatches, EmbSize)
        return x
```

## 2. The Transformer Encoder
The patches are fed through standard Transformer encoder blocks containing Multi-Head Self Attention (MHSA) and Multi-Layer Perceptrons (MLPs).

## 3. The Class Token
A special `<CLS>` token is prepended to the sequence of patches. This token gathers information from all the other patches via self-attention. At the end of the transformer stack, we pass *only* the `<CLS>` token to a standard linear layer to make our final image classification prediction!

## CNNs vs ViTs in Production
- **Small Datasets**: CNNs still win. They have a strong "inductive bias" (they inherently understand edges and textures).
- **Large Datasets**: ViTs dominate. Given massive amounts of data, the Transformer figures out visual structures globally and scales far better than deep CNNs.
---

Source: https://aimastermindlab.com/tutorials/computer-vision-transformers — AI Mastermind Lab (https://aimastermindlab.com)