This project compares the performance of various Large Language Models (LLMs) integrated into a healthcare-focused app. It applies software engineering principles to evaluate LLMs across multiple metrics and platforms.
The app allows users to input healthcare-related questions, which are processed by both cloud-based and local LLMs, enabling a detailed performance comparison.
- Build an interactive, secure app for healthcare-related chat with LLMs.
- Include cloud-based APIs (e.g., OpenAI, Gemini, Claude, Grok) and local models.
- Compare models from:
- Cloud-based: OpenAI (ChatGPT), Gemini, Claude, Grok.
- Local: bioGPT, LLama-3.2 (1B).
- Evaluate performance on mobile, laptop, cloud VMs, and edge devices.
- Simulate varied network conditions to test reliability and latency.
- ⏱ Response Time
- 🎯 Accuracy & Relevance
- 🧠 Resource Usage (CPU, RAM, GPU)
- 🚧 Latency & Delay Analysis
- Collect and analyze results using charts, graphs, and tables.
- TypeScript
- Visual Studio Code (VSCode)
- React Native – UI development
- Zustand – State management
- Ky – HTTP requests
- Gluestick – UI component library
- Firebase Auth – Authentication
- Express.js – API and service handling
- Mongoose (ORM) – MongoDB interaction
- Axios – Internal and external HTTP requests
- MongoDB
- Express.js – Microservice framework
- LLM SDKs/APIs – Official packages from OpenAI, Anthropic, Google, etc.
- Sign in with your credentials via Firebase.
- Fill in patient details (Name, Age, Height, Weight, Symptoms).
- Tap Start Chatting.
- Choose your preferred LLM from the dropdown.
- Begin the conversation and compare results across models.
🔐 If a signed-out user tries to access any protected page (like the form or chat), the app redirects them to the Sign-In screen.
- 🧾 Fetch & resume previous chat sessions.
- 🧠 Compare different model responses on identical queries.
- 📈 View performance stats and model efficiency insights.
- In-depth analysis of cloud vs. local LLMs in a real-world app.
- Software engineering insights into LLM integration.
- Visualization dashboards to present the performance metrics.
✅ Updated versions of the following diagrams are required:
- Class Diagrams (Class & Class2)
- Activity Diagram
- State Diagram
- Network Architecture Diagram
- Sequence Diagram (with correct tools and tech stack)
- User opens the app → lands on homepage.
- Clicks button to go to form.
- If not signed in → auto redirected to Sign-In page.
-
Expand into other industries like education, finance, legal.
-
Add support for new LLMs as they are released.
-
Enhance the benchmarking engine for deeper analysis and automation.
This project is licensed under the MIT License.
- Adheil Gupta (23BDS002)
- Arnav Gupta (23BDS009)
- Atharva Agrawal (23BDS010)
- SuryaNarayan Rao (23BDS025)
Follow these steps to set up and run the project locally:
git clone <repo-url>
cd <repo-directory>cd frontend- Replace
<your-IP>in the project files with the IP address of the backend server. - Open the
firebaseConfig.js, GoogleService-Info.plist, google-services.jsonfile and populate it with your Firebase project configuration.
npx install-expo-modules@latest
npm installnpx expo startcd models- Create a
.envfile. - Follow the format provided in
.env.example. - Populate the keys using your model files and credentials.
npm installnpm startcd backend- Create a
.envfile. - Follow the format provided in
.env.example. - Populate the keys using your own configuration values.
npm installnpm start