This project visualizes the loss landscape of a trained GAN to understand how its optimization surface behaves around the learned parameters.
A loss landscape is a surface that shows how the model’s loss changes when its parameters are slightly perturbed in different directions.
It’s like looking at the “terrain” your optimizer has to walk over — valleys, hills, and basins represent how the model learns.
-
Flat regions → Stable minima, better generalization
-
Sharp regions → Sensitive minima, potential overfitting
f(α,β)=Loss(θ + α·d₁ + β·d₂)
where θ is the model’s parameters, and α, β are step sizes in two random directions (d₁, d₂).
The code visualizes the loss landscape of a GAN by slightly perturbing its learned parameters and measuring how the Generator’s loss changes. It helps you understand how “flat” or “sharp” the minima of the GAN are in parameter space, which is useful for analyzing training stability and generalization. Loads a subset of real images for loss evaluation.
- Loads a pretrained GAN (Generator & Discriminator).
- Generates two random directions in parameter space for perturbation.
- Computes the Generator loss at every point on a 2D grid (size defined by GRID_STEPS) in parameter space.
- Creates visualizations:
- 2D Topographic Map
- 2D Heatmap
- 3D Surface Plot
- 3D Mesh Object (.PLY) for interactive exploration.
- 1D Slice Plot
- 1D dual-axis loss landscape plots
- Allows adjustable resolution (GRID_STEPS = 10, 50, 100, etc) to see the difference between coarse and fine-grained loss landscapes.
- Uses memory-saving techniques to perturb only the final layer and clear unused memory during computation.
The parameter GRID_STEPS defines how many points we sample on the loss surface.
| GRID_STEPS | Description | Computation | Visual Smoothness |
|---|---|---|---|
| 10 | Coarse grid | Fast | Rough surface |
| 50 | Moderate grid | Slower | Smooth & detailed |
| 100 | High resolution | Heavy | Very smooth |
| 200 | Ultra fine | Very heavy | Highly detailed surface |
Increasing GRID_STEPS → smoother, more accurate surfaces
But computation cost increases roughly with GRID_STEPS²
Shows how the Generator Loss (blue) and Success Rate (red) change when the model’s parameters are slightly perturbed. Left Y-axis = Loss, Right Y-axis = Success %. Helps visualize both stability (flat vs. sharp minima) and discriminator response around trained weights. Solid and dashed lines represent different slices through the loss surface.
Shows the "top-down view" of loss values — like a heatmap of valleys and peaks.
Lower (darker) regions = smaller loss → better minima.
Shows a mountain-like terrain — visually represents how loss changes in nearby parameter space.
Helps understand if your GAN found a flat, wide minimum (good) or sharp, unstable one (bad).
-
Clone the Repository
git clone https://github.com/AravKataria/visualizing-loss-landscape-at-different-resolutions-of-a-GAN-.git cd visualizing-loss-landscape-at-different-resolutions-of-a-GAN- -
Install requirements
pip install -r requirements.txt -
Prepare Your Files Generator checkpoint: generator.pth Discriminator checkpoint: discriminator.pth Images folder: Folder with real images for evaluation
-
Set the Resolution
GRID_STEPS = 11 # Fast, coarse grid GRID_STEPS = 50 # Moderate grid (recommended) GRID_STEPS = 100 # High-resolution, slower GRID_STEPS = 200 # highest-resolution, very slow -
Run the Script
python visualize_loss_landscape.py