doc updated

shubhvjain · shubhvjain · commit 457ca269f98c · 2025-03-11T18:29:19.000+01:00
diff --git a/docs/_static/custom.css b/docs/_static/custom.css
@@ -0,0 +1,5 @@
+div.highlight pre {
+  white-space: pre-wrap !important; /* Allows text wrapping */
+  word-wrap: break-word !important; /* Ensures words break if needed */
+  overflow-x: auto; /* Enables scrolling only if necessary */
+}
diff --git a/docs/api.rst b/docs/api.rst
@@ -17,6 +17,9 @@ The core package contains 2 main module :
 `tools` module
 ---------------
 
+  Methods vary depending on the type of input (e.g, country name vs energy data) and the output (e.g single value vs time series DataFrame). Most tools  depend on the data from the `data` sub package.
+  As a convention, methods that primarily accept DataFrame as an input (along with other parameters) and return  a DataFrame are prefixed with `_df`. 
+
 .. automodule:: codegreen_core.tools
    :members:
 
diff --git a/docs/conf.py b/docs/conf.py
@@ -13,7 +13,7 @@
 # https://www.sphinx-doc.org/en/master/usage/configuration.html#project-information
 
 project = "codegreen_core"
-copyright = "2024, Dr. Anne Hartebrodt"
+copyright = "2025, Dr. Anne Hartebrodt"
 author = "Dr. Anne Hartebrodt"
 
 # -- General configuration ---------------------------------------------------
@@ -46,6 +46,7 @@
 
 html_theme = "alabaster"
 html_static_path = ["_static"]
+html_css_files = ["custom.css"]
 
 html_theme_options = {
     "github_user": "codegreen-framework",
diff --git a/docs/features.rst b/docs/features.rst
@@ -0,0 +1,58 @@
+Features of `codegreen-core`
+=============================
+
+**Gather energy production data**
+
+Energy is produced from multiple sources, some of these sources are non-renewable, such as fossil fuels and coal, and others are renewable, like wind and solar. The combination of energy produced from different sources (or the energy mix) varies from country to country and season to season. The amount of carbon emissions generated from energy production depends on this energy mix. When more energy is derived from renewable sources, carbon emissions are lower, making the energy more 'green.' (and vice versa). This relationship is measured by carbon intensity, which quantifies the amount of carbon dioxide emitted per unit of energy produced.
+
+The codegreen-core package offers tools to collect time series data of the energy mix for various countries, which can also be used to calculate the carbon intensity of energy.
+
+.. image:: _static/DE_1.png
+   :alt: Germany example 1
+   :width: 600px  
+   :align: center  
+
+The figure above shows the percentage of energy generated from renewable (green) and non renewable (red) sources in Germany on 1st and 2nd June 2024.
+
+This data helps analyze the energy production in a country over a period of time to  identify patterns and compare energy data for  multiple countries. 
+
+.. image:: _static/multiple_2.png
+   :alt: Multiple example 1
+   :width: 800px  
+   :align: center 
+
+The figure above shows the percentage of energy generated from renewable sources in four countries (Germany, France, Italy, and Spain) from June 1 to June 24, 2024.
+
+An interesting observation is that the amount of green energy changes almost every hour. This presents two approaches to reducing carbon emissions:
+
+- **Time Shifting**: Adjusting the timing of computations to align with periods of greater availability of green energy.
+- **Location Shifting**: Moving computational tasks to locations that utilize more green energy.
+
+
+**Calculating carbon emission of a computational task**
+
+Given the time taken by a computational task, the location where it was performed, and the hardware specifications (number of cores/GPUs used and size of memory), the codegreen-core package provides an estimate of the carbon emissions produced by the task
+
+
+.. image:: _static/CE_DE_1.png
+   :alt: CE DE example 1
+   :width: 600px  
+   :align: center 
+
+The figure above shows the carbon emissions produced by a 12-hour computational task performed on 124 cores with 64 GB of memory in Germany
+
+
+**Predicting the optimal time/location to start the computational task**
+
+Given the approximate run time of the task, the hardware specifications, the location, and a specified criteria,  `codegreen-core`  predicts an optimal time based on energy production forecast data. It is designed to be fault-tolerant, and if no optimal time exists, the current time is returned. The user provides a criteria, such as the minimum percentage of renewable energy for the entire duration, which is used to predict the optimal time. 
+
+.. image:: _static/optimal_it_1.png
+   :alt: optimal eg 1
+   :width: 800px  
+   :align: center 
+
+The figure above shows the carbon emissions produced by an 8-hour computational task performed on 124 cores with 64 GB of memory in Italy, along with the potential savings in carbon emissions when the computation is started at suggested times using three different criteria (values of percentage renewable energy).
+
+..
+  // Challenges and future plans 
+  // One of the main challenges is the availability of time series of energy produced using renewable and non renewable sources for different coutnreis. The current energy forecasts are also limited to the next 24 hours which limits the optimal time decitions within the next 24 hours.  In future, we plan to integrate data for Non EU counties as well as train predication models that can generate time series forecasts of for longer periods of time. 
diff --git a/docs/index.rst b/docs/index.rst
@@ -11,11 +11,11 @@ Welcome to codegreen_core's documentation!
    :caption: Contents:
 
    introduction
-   methodology
-   getting_started
-   api
+   features
+   installation
+   setup
+   api 
    status
+   methodology
    references
-   version_history
-
-
+   version_history
diff --git a/docs/installation.rst b/docs/installation.rst
@@ -0,0 +1,15 @@
+Installation 
+==============
+
+You can install the package using pip : 
+
+.. code-block:: python
+
+  pip install codegreen_core
+
+Alternatively, you can clone the Git repository and install the package manually:  
+
+.. code-block:: bash
+
+  git clone https://github.com/codegreen-framework/codegreen-core.git
+  pip install -e  . 
diff --git a/docs/introduction.rst b/docs/introduction.rst
@@ -11,68 +11,3 @@ The carbon footprint of a computer depends on two main factors : its hardware sp
 
 The `codegreen-core` package is a comprehensive tool that enables users to calculate the carbon emissions of a computational task and provides strategies for minimizing those emissions. 
 
-Features of `codegreen-core`
------------------------------
-
-**Gather energy production data**
-
-Energy is produced from multiple sources, some of these sources are non-renewable, such as fossil fuels and coal, and others are renewable, like wind and solar. The combination of energy produced from different sources (or the energy mix) varies from country to country and season to season. The amount of carbon emissions generated from energy production depends on this energy mix. When more energy is derived from renewable sources, carbon emissions are lower, making the energy more 'green.' (and vice versa). This relationship is measured by carbon intensity, which quantifies the amount of carbon dioxide emitted per unit of energy produced.
-
-The codegreen-core package offers tools to collect time series data of the energy mix for various countries, which can also be used to calculate the carbon intensity of energy.
-
-.. image:: _static/DE_1.png
-   :alt: Germany example 1
-   :width: 600px  
-   :align: center  
-
-The figure above shows the percentage of energy generated from renewable (green) and non renewable (red) sources in Germany on 1st and 2nd June 2024.
-
-This data helps analyze the energy production in a country over a period of time to  identify patterns and compare energy data for  multiple countries. 
-
-.. image:: _static/multiple_2.png
-   :alt: Multiple example 1
-   :width: 800px  
-   :align: center 
-
-The figure above shows the percentage of energy generated from renewable sources in four countries (Germany, France, Italy, and Spain) from June 1 to June 24, 2024.
-
-An interesting observation is that the amount of green energy changes almost every hour. This presents two approaches to reducing carbon emissions:
-
-- **Time Shifting**: Adjusting the timing of computations to align with periods of greater availability of green energy.
-- **Location Shifting**: Moving computational tasks to locations that utilize more green energy.
-
-
-**Calculating carbon emission of a computational task**
-
-Given the time taken by a computational task, the location where it was performed, and the hardware specifications (number of cores/GPUs used and size of memory), the codegreen-core package provides an estimate of the carbon emissions produced by the task
-
-
-.. image:: _static/CE_DE_1.png
-   :alt: CE DE example 1
-   :width: 600px  
-   :align: center 
-
-The figure above shows the carbon emissions produced by a 12-hour computational task performed on 124 cores with 64 GB of memory in Germany
-
-
-**Predicting the optimal time/location to start the computational task**
-
-Given the approximate run time of the task, the hardware specifications, the location, and a specified criteria,  `codegreen-core`  predicts an optimal time based on energy production forecast data. It is designed to be fault-tolerant, and if no optimal time exists, the current time is returned. The user provides a criteria, such as the minimum percentage of renewable energy for the entire duration, which is used to predict the optimal time. 
-
-.. image:: _static/optimal_it_1.png
-   :alt: optimal eg 1
-   :width: 800px  
-   :align: center 
-
-The figure above shows the carbon emissions produced by an 8-hour computational task performed on 124 cores with 64 GB of memory in Italy, along with the potential savings in carbon emissions when the computation is started at suggested times using three different criteria (values of percentage renewable energy).
-
-
-
-..
-  // Challenges and future plans 
-  // One of the main challenges is the availablilty of time series of energy produced using renewable and non renewable sources for different coutnreis. The current energy forecasts are also limited to the next 24 hours which limits the optimal time decitions within the next 24 hours.  In future, we plan to integrate data for Non EU counties as well as train predication models that can generate time series forecasts of for longer periods of time. 
-
-Next step
-----------
-
-See the  Getting started guide for installation and setup.
diff --git a/docs/methodology.rst b/docs/methodology.rst
@@ -1,23 +1,7 @@
 Methodology
 ============
 
-Here we describe how we calcualte stuff
-
-
-
-``tools`` Module
-------------------
-
-This subpackage  provides tools and methods for tasks like calculating the carbon intensity of energy production and calculating the emissions produced due to a computation. 
-
-Each tool is implemented in a separate module and must be imported individually (See below). 
-
-..
-  Methods vary depending on the type of input (e.g, country name vs energy data) and the output (e.g single value vs time series DataFrame). Most tools  depend on the data from the `data` sub package.
-  As a convention, methods that primarily accept DataFrame as an input (along with other parameters) and return  a DataFrame are prefixed with `_df`. 
-
-
-
+This page describes the working of various tools and how they compute values. 
 
 Carbon Intensity of Energy
 ---------------------------
diff --git a/docs/setup.rst b/docs/setup.rst
@@ -0,0 +1,157 @@
+Setting up the package
+=======================
+
+The package requires a configuration file where all settings are defined.  
+Create a new file named `.codegreencore.config` in your root directory or project root. 
+This file will contain all the configurations required to run the package successfully. 
+
+This section describes how to set up the package. 
+
+Configuration options available
+--------------------------------
+
+The table below summarizes all available configs : 
+
+.. list-table:: Available Configuration Options
+   :header-rows: 1
+   :widths: 20 50 10 20
+
+   * - Name
+     - Description
+     - Default 
+     - Possible Values
+   * - `ENTSOE_token`
+     - The token required to fetch data from the ENTSO-E portal. Follow the steps at https://transparency.entsoe.eu to create a free account and obtain an API token.
+     - None
+     - String
+   * - `default_energy_mode`
+     - Defines the source of energy forecasts to be used for making optimal time predictions
+     - public_data
+     - public_data / local_prediction
+   * - `enable_energy_caching`
+     - Enables or disables local caching of energy data
+     - false
+     - true/false
+   * - `energy_redis_path`
+     - Path to Redis instance for caching
+     - None
+     - String (Redis URL, redis://localhost:6379 )
+   * - `enable_offline_energy_generation`
+     - To enable storing and periodic update of historical energy in csv files
+     - false
+     - true/false
+   * - `offline_data_dir_path`
+     - Path to the folder where historical energy data will be stored
+     - None
+     - String 
+   * - `offline_data_start_date`
+     - The start date from which historical energy data must be downloaded and stored
+     - None
+     - String (`YYYY-mm-dd` format) 
+
+
+
+Basic Configuration 
+--------------------
+Below is the template for the basic configuration 
+
+.. code-block:: bash
+
+  [codegreen]
+    ENTSOE_token = token-here
+
+This configuration will allow you to fetch data online and use it.  
+It is recommended to start with the basic setup and explore the available APIs before making advanced customizations.  
+Please refer to the above table for details about the configuration options. 
+
+Using predication models for energy forecasts 
+-----------------------------------------------
+
+By default we use energy data that is available from public sources. While this data is very useful, it may sometimes by inadequate to make better decisions. 
+
+The codegreen_core package includes built in predication models that generate energy forecasts. One advantage of our models is that we have trained different models for each country. They also provide forecasts for more that 24 hours in the future. 
+
+To enable the use  prediction model where the package requires energy forecast data, set the `default_energy_mode` to `local_prediction`
+
+Note : This feature is still under development and models may not be available for all countries. If a model is unavailable, the system will fall back to using publicly available energy data.
+
+Setting up energy data caching
+---------------------------
+
+Some methods in the tools module rely of energy data provided by the data module. 
+By default, every time a method is called, data is fetched from online sources. 
+To avoid repeatedly fetching the same data, the package allows storing recent energy data in a Redis cache.
+
+**To enable this feature**  
+
+- Set `enable_energy_caching` to `true` 
+- Provide the path to Redis cache in `energy_redis_path` configuration
+- Ensure that the Redis cache is running ; otherwise an exception will be thrown.
+
+**How caching works** 
+
+Forecast data is synced in the cache whenever a method request data. The first request may take some time as it triggers the cache sync process.
+
+To sync energy generation data run the following piece of code (either manually or via a CRON Job):
+
+.. code-block:: python
+
+  from codegreen_core.data import sync_offline_data
+  sync_offline_data(cache=True)
+
+Cached data includes : 
+
+- Recent energy forecasts for all available countries (up to 24 hours ahead)
+- Recent energy production data ( last 72 hours up until 5 hours before the last sync time since different countries have different upload schedules) 
+- Recent forecast data generated by the predication models  (up to 72 hours ahead in time, if that option is enabled)
+
+
+Setting offline storage of energy data
+---------------------------------------
+
+If you work with energy generation data for longer periods , you have the option to store it offline for quick access. 
+The package  supports long term storage of generation data only.
+
+**To enable this feature**
+
+- Set `enable_offline_energy_generation` to `true`
+- Provide a folder path in `offline_data_dir_path` where data will be stored. 
+- Specify the start date from which  data should be stored using  `offline_data_start_date`  configuration in `YYYY-MM-DD` format. 
+
+After configuring these settings , manually start the initial sync using following code :
+
+.. code-block:: python
+
+  from codegreen_core.data import sync_offline_data
+  sync_offline_data(file=True)
+
+**How Offline Storage Works**
+
+This  setup  will create initial files for each available country. 
+Each  country will have two files : A CSV file with the data and JSON containing metadata for easier syncing. 
+Syncing may take time depending on the selected start date.
+
+If you want to back fill data from an earlier time, modify `offline_data_start_date` 
+
+
+**Keeping offline storage up-to-date**
+
+Date files needs to be updated with latest data regularly. You can update them manually (by running the above command) or set up a CRON job to call the sync method periodically 
+
+
+
+**Using Preprocessed Data for Faster Setup**
+
+Since initial setup can take a long time, you can also download preprocessed data from our Github repo and use it as a starting point. 
+
+Steps : 
+
+- Download the zip file
+- Extract the folder and place the data in the desired location 
+- Update the config file if required (`offline_data_dir_path` and `offline_data_start_date`)
+- Run the sync code to ensure the files are updated with the latest available data.
+
+Available preprocessed data: 
+
+- Data since Jan 1, 2025 : Link to the zip file   https://github.com/codegreen-framework/energy-data/raw/refs/heads/main/2025.zip
+- Data since Jan 1, 2020 : Under development
diff --git a/docs/use_codegreen.rst b/docs/use_codegreen.rst
