Join our 12-week Meetup Events held every Monday from 17/02/2025 to 12/05/2025:

• In-Person: Visit us at the ANU School of Computing
• Online: Join our Discord Server for chats

It's only March, but the future looks bright! We have an incredible lineup of events, guest speakers, and workshops ahead. This is your chance to connect, learn, and grow with a vibrant community of like-minded people. Whether you're attending in person or joining online, there's a place for you here. Let's make Semester 1, 2025 the best one yet!

• 🏃‍♀️ Speed run some basic knowledge

  • Play and visualise LLMs with LLM Visualization created by Brendan Bycroft.
  • Enjoy transformer videos made by 3Blue1Brown:
    • But what is a GPT? Visual intro to transformers | Chapter 5, Deep Learning
    • Attention in transformers, visually explained | Chapter 6, Deep Learning
  • Read these awesome articles from real human intelligence 📜
    • LLM Basics: Embedding Spaces - Transformer Token Vectors Are Not Points in Space from Nicky Pochinkov.
    • A Gentle Introduction to Positional Encoding in Transformer Models, Part 1 from Jason Brownlee.

• 🛠️ Build one from scratch

  • Follow one of tutorial videos from Andrej Karpathy (former OpenAI research scientist):
    • Let's reproduce GPT-2 (124M)
    • Let's build the GPT Tokenizer
    • Let's build GPT: from scratch, in code, spelled out.

• 📜 Read some simple yet functional repos

  • minGPT: A small, clean, interpretable and educational GPT re-implementated in PyTorch.
  • nanoGPT: The simplest, fastest repository for training/finetuning medium-sized GPTs. A rewrite of minGPT.
  • build-nanogpt: Walk through step-by-step and clean GitHub commits to slowly build a nanoGPT.
  • nano-llama31: A minimal, dependency-free implementation of the Llama 3.1 architecture.

• ⚔️ Compare performance of the latest LLMs
  • LLM Leaderboard from vellum
  • Independent analysis of AI models and API providers from Artifical Analysis
  • The LLM Index from Sapling
  • open-llm-leaderboard from Open LLM Leaderboard
  • The Big Benchmarks Collection from Open LLM Leaderboard

• 🎮 Good visualisation is all you need
  • WizMap from Polo Club of Data Science @ Georgia Tech for visualising large-scale token embeddings.
  • Dodrio from Polo Club of Data Science @ Georgia Tech for attention head summarization and semantic and syntactic knowledge contexts from transformer models.

• 📦 Interesting topics and other stuffs
  • ChatGPT: 30 Year History | How AI Learned to Talk from Art of the Problem on YouTube.
  • The moment we stopped understanding AI [AlexNet] from Welch Lab on YouTube.
  • CNN Explainer from Polo Club of Data Science @ Georgia Tech for helping non-experts learn about Convolutional Neural Networks (CNNs).
  • NeuroCartography and Summit from Polo Club of Data Science @ Georgia Tech for visualising image embeddings from ImageNet.

• Data Source Contributor 🕵️‍♀️
  • Identify and provide access to Australia-related data sources.
  • Collaborate with other contributors to ensure data quality and relevance.
• Data Collecting, Crawling and Scraping 👩‍🌾
  • Develop scripts and tools to collect data from various sources.
  • (Optional) Have experience with web scraping tools (e.g., BeautifulSoup, Scrapy).
• Data Cleaning 👩‍⚕️
  • Clean and preprocess datasets to ensure they are ready for analysis and modeling.
  • (Optional) Have experience with data manipulation libraries (e.g., Pandas, NumPy).
• Model Building, Training and Tuning 👩‍💻
  • Develop and train LLMs to solve with our datasets.
  • Have experience with machine learning frameworks (e.g., TensorFlow, PyTorch).
• GitHub Organising 👩‍🔧
  • Manage the GitHub repository by organizing files, documentation, and issues.
  • (Optional) Have proficiency in using Git and GitHub.
• Hugging Face Organising 👩‍🏭
  • Manage and organize model versions and datasets.
  • Ensure proper documentation and metadata for each model and dataset.
• Social Media Organising 👩‍💼
  • Promote the project and its updates on social media platforms (e.g., Discord, Meetup).
  • Engage with the community to increase project visibility and collaboration.