CustomLM

CustomLM is a from-scratch implementation of a transformer-based Language Model (GPT) designed for academic exploration and experimentation. Developed as part of a course project, it focuses on tokenization strategies, architecture design, and hyperparameter analysis.

📌 Features

• Custom GPT Architecture
  Manual implementation of a transformer model inspired by GPT, built with PyTorch.

• Flexible Tokenization
  Models trained on character-level, syllable-level, and word-level representations.

• Training and Evaluation
  In-depth analysis of training/validation loss and time across hyperparameter configurations.

• Text Generation
  Sequence generation with top-performing model variants.

🚀 Technologies

• PyTorch, NLTK, datasets
• Includes a custom syllable tokenizer and manual tokenization logic.
• Runs in Kaggle (GPU-enabled) for training efficiency.

👥 Authors

Filippo Lucchesi, Francesco Pio Crispino, Martina Speciale