Pattern Recognition

This is the collection of notebooks for the course FYS-3012 - Pattern Recognition, taught at UiT The Arctic University of Norway. You can view and execute the notebooks by clicking on the buttons below.

📑 Content

1. Introduction to pattern recognition

   • What is pattern recognition?
   • Applications of pattern recognition
   • Practical information

   

2. Bayes decision theory

   • Motivation
   • Naïve Bayes assumption
   • Bayes classifier
   • Probablity of error
   • Dealing with risk

   

3. Density estimation I

   • Parametric density estimation
   • Likelihood function
   • Maximum-likelihood estimation
   • Non-parametric density estimation
   • Histogram approach
   • Parzen windowing

   

4. Density estimation II

   • K nearest neighbor
   • Mixture models

   

5. Linear classifiers I

   • Mean squared error (MSE) classifier
   • Least sum of squares classifier
   • Widrow-Hoff algorithm
   • Perceptron

   

6. Linear classifiers II

   • Support vector machines

   

7. Non-linear classifiers I

   • Multilayer Perceptron
   • Backpropagation
   • Gradient descent

   

8. Non-linear classifiers II

   • Activation functions
   • The vanishing gradient problem
   • Gradient descent with momentum
   • Multiclass classification with neural networks
   • Softmax function
   • Cross-entropy loss
   • Weight initialization

   

9. Non-linear classifiers III

   • Neural networks as representation learning
   • A vector perspective on backpropagation
   • Connecting neural networks and support vector machines

   

10. Non-linear classifiers IV

• Function approximation for non-linear classification
• Radial basis function (RBF) networks
• Prototypical learning

11. Non-linear classifiers V

• Kernel methods
• Mercer's theorem
• Kernels
• Non-linear SVMs

12. Data transformations and dimensionality reduction I

• Fisher discriminant analysis
• Principal component analysis (PCA)

13. Data transformations and dimensionality reduction II

• Kernel PCA
• Laplacian Eigenmaps

14. Ranking

• Ranking
• PageRank algorithm

15. Clustering I

• Sequential clustering

16. Clustering II

• Hierarchical

17. Clustering III

• K-means
• spectral clustering

💻 How to code locally

To run the notebooks locally the recommended steps are the following:

1. Download and install Miniconda.

2. Download the env.yml file.

3. Open the shell and navigate to the location with the yml file you just downloaded.

   • If you are on Windows, open the Miniconda shell.

4. Install the environment with

   > conda env create -f env.yml
   

5. Activate your environment:

   > conda activate pytsa
   

6. Go to the folder with the notebooks

7. Launch Jupyter Lab with the command

   > jupyter lab
   

🎥 Notebook format and slides

The notebooks are structured as a sequence of slides to be presented using RISE. If you open a notebook you will see the following structure:

The top-right button indicates the type of slide, which is stored in the metadata of the cell. To enable the visualization of the slide type you must first install RISE and then on the top menu select View -> Cell Toolbar -> Slieshow. Also, to split the cells like in the example, you must enable Split Cells Notebook from the nbextensions.

By pressing the Enter\Exit RISE Slideshow button at the top you can enter the slideshow presentation.

See the RISE documentation for more info.

📝 Citation

If you are using this material in your courses or in your research, please consider citing it as follows:

@misc{wick2024prbook,
  author       = {Kristoffer Wickstrøm},
  title        = {Pattern Recognition Course},
  year         = {2025},
  howpublished = {Online},
  url          = {https://github.com/Wickstrom/pattern-recognition-handbook}
}

This repository is heavily based on Time Series Analysis with Python by my collegue at UiT Filippo Maria Bianchi. If you find my repository useful, please head over there and give him a star.

@misc{bianchi2024tsbook,
  author       = {Filippo Maria Bianchi},
  title        = {Time Series Analysis with Python},
  year         = {2024},
  howpublished = {Online},
  url          = {https://github.com/FilippoMB/python-time-series-handbook}
}