Information Convergence Platform

The Information Convergence Platform is a data modeling project designed to integrate and query data from various sources, including SQL databases, XML files, and spreadsheets. It provides a unified interface to upload schemas, execute queries, and view results in multiple formats.

Features

• Schema Upload: Upload and manage XML schemas to define data structures.
• Query Execution: Run complex queries across multiple data sources.
• Data Integration: Merge and resolve data from SQL, XML, and spreadsheet sources.
• Result Presentation: Display query results in HTML tables, and export to JSON, CSV, and TXT formats.
• Web Interface: User-friendly frontend built with Flask for seamless interaction.

Project Structure

InformationConvergencePlatform/
├── Datasources/
├── Output/
├── Queries/
├── Schemas/
├── file_dump/
├── templates/
│   ├── index.html
│   └── result.html
├── .gitignore
├── app.py
├── app_helper.py
├── conditional_filtering.py
├── execute_query.py
├── execution_helper.py
├── markdown_queries.py
└── relational_queries.py

• Datasources/: Contains data files from various sources.
• Output/: Stores the output files generated after query execution.
• Queries/: JSON files defining the queries to be executed.
• Schemas/: XML schema files defining the data structures.
• templates/: HTML templates for the web interface.
• app.py: Main Flask application.
• app_helper.py: Helper functions for the Flask app.
• conditional_filtering.py: Handles conditional query filtering.
• execute_query.py: Core logic for executing queries.
• execution_helper.py: Utility functions for query execution.
• markdown_queries.py: Functions related to XML/Markdown queries.
• relational_queries.py: Functions related to SQL and spreadsheet queries.

Setup Instructions

1. Clone the Repository

   git clone https://github.com/nathanmathewv/InformationConvergencePlatform.git
   cd InformationConvergencePlatform

2. Create a Virtual Environment (Optional but Recommended)

   python3 -m venv venv
   source venv/bin/activate  # On Windows: venv\Scripts\activate

3. Install Dependencies

   pip install -r requirements.txt

4. Run the Application

   python app.py

   The application will start on http://127.0.0.1:5000/.

Usage

1. Upload Schema

   • Navigate to the "Upload Schema" section.
   • Enter the desired schema file name.
   • Paste the XML content of your schema.
   • Click "Upload" to save the schema.

2. Run Query

   • Navigate to the "Run Query" section.
   • Enter the name of the schema file you uploaded.
   • Paste your query in JSON format.
   • Click "Run Query" to execute.

3. View Results

   • After execution, results will be displayed in a tabular format.
   • Options to download results in JSON, CSV, or TXT formats are available.

License

This project is licensed under the MIT License. See the LICENSE file for details.