self-attention-implementation

Here are 8 public repositories matching this topic...

ranfysvalle02 / ai-self-attention

This repository provides a basic implementation of self-attention. The code demonstrates how attention mechanisms work in predicting the next word in a sequence. It's a basic implementation that demonstrates the core concept of attention but lacks the complexity of more advanced models like Transformers.

python machine-learning ml python3 rnn attention-mechanism rnn-model attention-is-all-you-need self-attentive-rnn self-attention transformer-architecture self-attention-implementation

Updated Sep 23, 2024
Python

bhattbhavesh91 / self-attention-python

Sponsor

Star

This repository will guide you to implement a simple self-attention mechanism using the Python's NumPy library

python numpy attention-mechanism self-attention self-attention-implementation

Updated Jan 19, 2023
Jupyter Notebook

dariush-bahrami / mytransformers

Star

My implementation of transformers

nlp deep-learning neural-network transformers attention-mechanism attention-is-all-you-need self-attention pytorch-implementation self-attention-implementation

Updated Feb 20, 2024
Python

khwajawisal / Attention-augmented-Convolutional-Neural-Networks

Star

self-attention-implementation attention-in-neural-networks

Updated Dec 10, 2019
Python

sumony2j / SeedGPT

Star

SeedGPT is a minimal decoder-only LLM from scratch, inspired by andrej karpathy's tutorials, generating text token-by-token based on input word prompts.