- Project Description
- Repository Structure
- Instructions
- What's next for GinkgoGreen
- Accomplishments that we're proud of
- What we learned
Music often reflects our emotional state—sometimes more honestly than we can express ourselves. While many people intuitively recognize this connection, those navigating mental health challenges like depression, anxiety, or bipolar disorder may struggle with emotional awareness.
Music Mood Mirror was born from the idea that our listening habits—especially lyrics and genres—can act as passive signals for emotional trends. By tapping into the music people already love, we aim to offer meaningful emotional insights without the burden of manual tracking or journaling.
Music Mood Mirror is directly aligned with the healthcare theme of this hackathon, focusing on mental wellness and emotional self-awareness.
- For individuals with conditions like depression, anxiety, or bipolar disorder, emotional self-tracking can be inconsistent or burdensome.
- Our tool provides a non-intrusive, passive method for emotional monitoring, supplementing traditional mood journaling.
- Insights from the app could potentially be shared with therapists, psychiatrists, or wellness platforms to offer early signals of emotional changes or relapses.
- The use of LLMs for emotional signal extraction from natural media (music) adds a novel angle to digital mental health support.
Music Mood Mirror is a web app that passively analyzes users’ music listening history to track emotional trends over time. It leverages Large Language Models (LLMs) to extract sentiment, positivity scores, and recurring emotional themes from song lyrics.
-
Input:
- User-uploaded listening history (song titles, artists, timestamps)
-
LLM-Powered Analysis:
- Sentiment and thematic analysis of lyrics
- Emotion tagging and positivity scoring
-
Output:
- Weekly stats: play duration, reason, completion %, repeat count
- Mood tags and scores (when user enables “Analyze Mood with AI”)
- Visualizations of emotional themes by week
Although designed with mental health support in mind, Music Mood Mirror is equally valuable for anyone curious about how their music reflects their mood—turning everyday playlists into a mirror for emotional self-awareness. Let me know if you’d like a visual diagram or flowchart to include in your submission, or if you need a shorter version for slide decks!
music-mood-mirror/
├── web_app/ # Main Flask web application source code
│ ├── static/ # Static files (CSS, JS, images)
│ ├── templates/ # HTML templates for the web UI
│ ├── app.py # Entry point for the web server
│ ├── install_llama_cpp.sh # Script to install llama-cpp-python and dependencies
│ ├── requirements.txt # Python dependencies for the project
│ └── ... # Other backend modules and utilities
├── notebooks/ # Sample datasets and processed data
│ ├── run_first.sh # Installs all required Python libraries and dependencies
│ ├── main_Music-mood Mirror.ipynb # Main notebook for end-to-end lyric analysis
│ ├── ground_truth.py # Script to generate or collect ground truth data
│ ├── verify_results.ipynb # Compares analysis results from local LLMs with grount truth
│ ├── RegisterLLM2MLflow.ipynb # Registers Gemma-2B model with MLflow in AIStudio
│ └── ...
├── data/ # Sample datasets and processed data
│ ├── sample/ # Example listening history files for testing
│ └── stats/ # Analysis results and statistics (Excel, CSV)
├── figures/ # Screenshots and images for documentation
├── analysis/ # CSV files and result charts from model benchmarking
├── FEEDBACKS.md # Detailed feedback on AI Studio
└── README.md # Project documentation (this file)
We strongly recommend setting up and running the web app of this project in WSL2. We tested this project on Ubuntu 22.04; other distributions may also work but we haven't tested them.
Clone the music-mood-mirror Github repo and navigate to
the music-mood-mirror/web_app folder
git clone https://github.com/excelle08/music-mood-mirror.git
cd music-mood-mirror/web_appWe recommend using Python's virtual environment (venv):
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activateThis project uses llama-cpp-python to execute the Gemma-2B model
for lyrics emotion inference, so we need to first build and install
this package by running the script:
bash install_llama_cpp.shNext, let's install the Python packages required to run the web app:
pip install -r requirements.txtpython app.pyThis command will first load the Gemma-2B LLM model, test it by asking it
"What is MLflow" and then start the web server. During the first run it will
also download the model into your ~/models folder, which could take a few
more minutes depending on the internet connectivity. If the LLM model did
not load successfully, the web app will still start but the features that
require AI inference will not be available.
If you see the following output in your terminal, your web app is started and ready to use:
Model file gemma-2b-it.Q4_K_M.gguf is ready at /home/<username>/models/gemma-2b-it.Q4_K_M.gguf.
Testing the model with prompt: What is MLflow?
Model output: MLflow is an open-source platform for managing the entire machine learning lifecycle, from data preparation to model deployment. It provides a centralized repository ......
......
**Benefits of using MLflow:**
* **Improved Model Reproducibility:** MLflow makes it easier to reproduce the model's results, reducing the risk
Starting the web app...
* Debugger is active!
* Debugger PIN: 866-921-639
Then open http://127.0.0.1:8081 in your browser, you will see a welcome page which gives a brief introduction to this project and provides access points to the core features via the sidebar:
Click on "Register" at the bottom of the left sidebar to register a user account
After registering the account, you will be directed to the login page to sign in
After logging in successfully, the web app will automatically direct you to upload data for analysis.
At this step, you can upload the sample data we provided in the data/sample
folder. We provide two datasets:
processed_with_weeklyfreq/2017.json: A full year of music listening history by someone in 2017, with lyrics filled and behavioral metrics (e.g. weekly repeat frequency) pre-processed. This dataset allows for quick import and view.raw_spotify/2018_week_1_and_2.json: Two weeks of music listening history directly exported from Spotify, which means it does not contain lyrics or behavioral data and requires lyrics filling and analysis by this web app.
We recommend you upload the first dataset and then the next one. Of course, you can try your own Spotify listening history, but be aware that it could take longer time to search for lyrics on the internet.
After you choose the dataset and click "Upload", you will be able to see the upload and import progress. Uploading pre-processed data is faster than uploading raw Spotify history data, because the latter needs searching for lyrics.
When upload finishes, you can click on the "View My History" button to go to your uploaded playlist or click on anywhere outside the modal window to close it and upload another dataset.
After uploading the history dataset, you can view and process the history data in the "My Playlist" page:
It will show a table of your listening history that you just uploaded. For songs with lyrics populated, you can click on the "View" button to see their lyrics.
Now, it's time to click on "Analyze History" to analyze the weekly behavior metrics. After it finishes the last four columns of the table will populate.
Click on "Analyze Mood with AI" button, the web server will start picking significant songs in your playlist and analyze your mood during that time with the help of AI. This will take some time, so please be patient. You can monitor the progress in the progress bar modal window like this:
In this section we will discuss the dataset we use and analysis methodologies we applied.
We accept the standard Spotify listening history JSON files that you can export from your Spotify account. The JSON file should contain a list of entries, with each entry representing a music play and includes these fields that we care about:
master_metadata_track_name- The title of the songmaster_metadata_album_album_name- Album namemaster_metadata_album_artist_name- Artist namems_played- How long the song was played, in msreason_start- Reason for the song started playingreason_end- Reason for the song endedts- Date and time when the song was played, in ISO datetime formatshuffle- Was the song played in random shuffle mode?skipped- Was the song skipped?
We also accept pre-processed JSON files which may have lyrics, behavioral data and mood data pre-filled. You can take a look at the sample dataset we provide in this repo.
To ensure satisfactory data quality and focus on songs that the user truely wanted to listen and are more likely to genuinely reflect the listener's emotion, we filtered out tracks with short durations or low completion rates.
Based on our analysis on distribution in music completion rate and music play duration among over 18,000 history entries, we decided to filter out songs that were played for less than 90 seconds or less than 50% of their full length. We also filtered out songs that we could not find corresponding lyrics.
In the first step of data analysis, we focus on
the repeat behavior in each week. We processed (1) how many times the song has
been repeated during the week (repeats_this_week) and (2) how many times the
song has been repeated in the rolling next 7 days (repeats_next_7d). In
further analysis, we decided to focus on repeats_this_week.
Because inference with AI can be costly and time consuming, we only have the AI read the top 10 significant songs that the user played in each caleandar week. Our criteria for top significant songs are as follows:
- Repeated for the highest number of times (sorting
repeats_this_week) - The user actively chose to listen (selecting songs with
reason_startofclickrow,playbtnorbackbtn).
We asked the AI to answer 3 emotional tags among a list of given tags that can best summarize the emotions conveyed by the song's lyrics. Each tag is previously assigned with a "positivity score" of 1 (most negative) to 5 (most positive) depending on the meaning, so the "positivity score" of the song will be the average of the "positivity scores" of the three tags answered by the AI.
The prompt we used and the positivity scores assigned to tags are described in Sample Prompt and Positivity Score Mapping of the next section.
The weekly mood score will be calculated by weighted average "positivity score"
among the scores of significant songs the user played in the week, with weights
being the number of repeats_this_week.
Weekly mood tags are aggregation of tags associated with the weekly significant
songs. Tags are also weighted by repeats_this_week so they are displayed in
different sizes in the weekly mood cloud to reflect which tags are more significant.
List all pre-trained or custom models used, with details on their sources, architectures, and the methods applied for training, fine-tuning, or inference.
We used the Gemma-2B model, running locally via the llama-cpp-python framework, as our LLM inference engine to analyze mood tags for each selected song. For each song, the model predicts three emotional tags from a predefined list. Each tag is mapped to a positivity score (1–5) using a dictionary, and the song's overall positivity score is the average of its tags' scores.
Below is an example of the prompt used to instruct the LLM to infer mood tags from song lyrics:
You are an expert in analyzing song lyrics to determine the emotions they convey.
Analyze the following song lyrics and return exactly 3 emotion tags that best summarize the emotions conveyed by the song. Only output the tags, in this format: #tag1 #tag2 #tag3.
The tags must be adjectives and strictly chosen from the following list: Joyful, Melancholic, Hopeful, Angry, Romantic, Nostalgic, Sad, Energetic, Passionate, Lonely, Uplifting, Bittersweet, Empowering, Heartbroken, Reflective, Playful, Dark, Calm, Longing, Triumphant
'''{insert song lyrics here}'''
The following dictionary maps each emotional tag to its corresponding positivity score (1–5):
positivity_dict = {
'Joyful': 5,
'Melancholic': 2,
'Hopeful': 5,
'Angry': 1,
'Romantic': 4,
'Nostalgic': 3,
'Sad': 1,
'Energetic': 4,
'Passionate': 4,
'Lonely': 1,
'Uplifting': 5,
'Bittersweet': 3,
'Empowering': 5,
'Heartbroken': 1,
'Reflective': 3,
'Playful': 4,
'Dark': 1,
'Calm': 4,
'Longing': 2,
'Triumphant': 5
}To optimize efficiency, we only analyze the top 10 most frequently played unique songs per week, as determined by repeat count. The weekly mood score is then calculated as a weighted average of these songs' positivity scores, weighted by their repeat counts.
For benchmarking, we also used the o3-mini model via OpenAI's API to analyze 20 randomly sampled songs, serving as a ground truth for comparison with our local LLM. The chart below illustrates the comparison of positivity scores between Gemma-2B and o3-mini:
-
Enhancing Personalization
We plan to introduce mood trend summaries and intelligent suggestions such as:“You've been listening to more melancholic ballads this week, which may suggest a lower mood. Want to try a calming playlist or talk to someone?”
This will help users reflect on their emotional patterns and take proactive steps when needed. -
Improving Model Accuracy and Efficiency
We'll continue refining our LLM pipeline through techniques like fine-tuning, smarter data sampling, and domain-specific adjustments to improve the quality and responsiveness of Music Mood Mirror. -
Growing as a Team
Looking ahead, we’re excited to deepen both our product thinking and technical capabilities. Our goal is to keep building meaningful, AI-powered tools that positively impact people’s lives—starting with music and mental wellness.
One of the most meaningful accomplishments for us goes beyond the code itself. This project marks a full-circle moment in our friendship — we first teamed up for a college entrepreneur competition nearly 10 years ago but weren’t able to complete it due to coursework demands. Now, a decade later, we’ve grown into a data scientist/ML engineer and a software engineer, finally bringing a shared idea to life with Music Mood Mirror.
We're proud of building an end-to-end prototype within a short timeframe, combining LLM-based emotional analysis, a functional backend pipeline, and a responsive web dashboard. This wasn’t just a technical win — it was a personal one, too.
Through this hackathon, we learned how powerful it can be to blend friendship, creativity, and technical skill into something with the potential to make a real impact. We deepened our experience with LLMs, learned how to process and visualize emotional data from lyrics, and sharpened our ability to iterate quickly across the full stack — from ideation to deployment.
Most importantly, we were reminded that collaboration grounded in mutual respect and shared history can be a unique superpower. This project reinforced our belief in using technology with empathy — and we’re excited to keep building.