Realtime Database Benchmarking

This project compares different types of databases for real-time performance. The implementation is written in C++ for core database logic and Python for visualisation of performance metrics.

Overview

The project evaluates 4 database implementations, each with different data structures and threading strategies:

1. Naive:
   • Uses a std::vector for database entries
   • Not thread-safe
2. Locked:
   • Uses a std::vector for database entries
   • Thread-safe via a single mutex shared between all read and write operations
3. Efficient:
   • Uses std::unordered_map (hash map) for database entries
   • Not thread-safe but provides improved access times
4. Scalable:
   • Uses std::unordered_map for database entries
   • Thread-safe using a second std::unordered_map for mutexes
     • each database entry has its own mutex

Benchmarking Approach

• A fixed number of random keys are generated per database.
• The wall time is measured for an increasing number of access (read/write) operations.

Getting Started

Prerequisites

• A UNIX OS or Git Bash
• C++17 or later
• Python 3

Installation and Usage

1. Clone the repository

   git clone https://github.com/simran-ss-sandhu/Realtime-Database-Benchmarking.git

2. Navigate to the project directory

   cd Realtime-Database-Benchmarking

3. Build and Run the project

   make clean all

Performance Results

Test Setup

• Entries: 100
• Access operations: Ranged from 1 to 500,000
• Threading tests: Single-threaded vs Multi-threaded when applicable
• OS: MacOS_Sonoma14.0
• CPU: Apple M1 Pro

Observations

• Naive and Locked (1 thread): almost identical performance
  
• Locked (4 threads): around 2x slower than the single-threaded instance due to contention
  
• Efficient (1 thread): around 4x faster than Locked (4 threads), 2x faster than Naive and Locked (1 thread)
  
• Scalable (1 thread): around 25% slower than Efficient (1 thread)
  
• Scalable (4 threads): around 3x faster than Scalable (1 thread)
  

Multi-threading impact

• A final test measured performance against increasing thread counts with a fixed number of accesses (500,000)
• The results showed a non-linear decrease in wall time, with a 10-threaded instance being around 8.5 times faster than the single-thread one.
•