Skip to content

Performance test: check deep AST using sum() operator #34

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
May 17, 2024

Conversation

ianmnz
Copy link
Contributor

@ianmnz ianmnz commented Apr 29, 2024

Performance tests to check the possibility of dealing with deep AST.
In a first glance, using the .sum() operator seems to allow the handling of such ASTs

However, a bug with the parameters prevents the use of the sum operator:
For time-variable parameters (possibly for scenario-variable as well), it seems to ignore the vector and it only keeps one scalar. To dig further...

@ianmnz ianmnz requested review from sylvlecl and tbittar April 29, 2024 14:33
@ianmnz
Copy link
Contributor Author

ianmnz commented Apr 29, 2024

@tbittar I quickly discussed about the bug with @sylvlecl
We can discuss it when I get back

) -> TimeScenarioSeriesData:
X = truncnorm((lower - mean) / std, (upper - mean) / std, loc=mean, scale=std)

sample = X.rvs(horizon * scenarios, random_state=seed)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use tuple of ints for argument size ? This way, sample would have shape [horizon, scenarios] instead of [1, horizon * scenarios]

Suggested change
sample = X.rvs(horizon * scenarios, random_state=seed)
sample = X.rvs(size=(horizon, scenarios), random_state=seed)

Comment on lines 73 to 78
data = {}
for absolute_timestep in range(horizon):
for scenario in range(scenarios):
data[TimeScenarioIndex(absolute_timestep, scenario)] = sample[
scenario + absolute_timestep * scenarios
]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So you don't need this step

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How to avoid this step completely ? With your previous suggestion, the only difference I see is this:

Suggested change
data = {}
for absolute_timestep in range(horizon):
for scenario in range(scenarios):
data[TimeScenarioIndex(absolute_timestep, scenario)] = sample[
scenario + absolute_timestep * scenarios
]
data = {}
for absolute_timestep in range(horizon):
for scenario in range(scenarios):
data[TimeScenarioIndex(absolute_timestep, scenario)] = sample[absolute_timestep, scenario]
]

Comment on lines +217 to +218
network.connect(PortRef(demand, "balance_port"), PortRef(node, "balance_port"))
network.connect(PortRef(gen, "balance_port"), PortRef(node, "balance_port"))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How are port connections going to work in YAML files ? Defined in the model ? Defined in the study ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure. @sylvlecl or @vargastat would know it better

Comment on lines 201 to 203
database.add_data("D", "demand", generate_const_data(100, horizon, scenarios))

database.add_data("G", "p_max", ConstantData(100))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we have 2 classes/methods for constant data ?

Copy link
Contributor Author

@ianmnz ianmnz May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I misnamed the function for lack of creativity.
While the second is a scalar value, the first is a (horizon, scenarios) matrix with the same value on every entry.

I accept name suggestions for the function :)

Comment on lines 59 to 70
def generate_random_data(
mean: float,
std: float,
horizon: int,
scenarios: int,
*,
seed: Optional[int] = 2024,
upper: float = float("inf"),
lower: float = float("-inf")
) -> TimeScenarioSeriesData:
X = truncnorm((lower - mean) / std, (upper - mean) / std, loc=mean, scale=std)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of this is unused

Copy link
Contributor Author

@ianmnz ianmnz May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used it when I was testing more general things, but for the UT themselves, I chose a more controlled data generation.
It is much more a utility when developing the idea of the test than the test itself

Copy link
Collaborator

@tbittar tbittar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, just remove all unsued stuff in test_utils.py for now before merging

@ianmnz ianmnz marked this pull request as ready for review May 17, 2024 11:23
@ianmnz ianmnz requested a review from tbittar May 17, 2024 11:49
@ianmnz ianmnz merged commit 346d852 into main May 17, 2024
1 check passed
@ianmnz ianmnz deleted the test_performance_large_ast branch May 17, 2024 12:08
ianmnz added a commit that referenced this pull request Jul 24, 2024
commit 7fa89213801af1f57e340bcc40b0dac1eb6c4af7
Merge: 964297a 0074e94
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Wed Jul 24 11:43:38 2024 +0200

    Merge branch 'main' into feature/investment_pathway

commit 0074e94
Author: ianmnz <ian.menezes@rte-france.com>
Date:   Wed Jul 24 10:18:58 2024 +0200

    Add scenario benders test, bounds validation, and support for bool variables (#46)

    * Cherry-picked solver var name prefixes and suffixes
    * Added scenario tests for benders decomposed
    * Added BoolVar type
    * Added check for upper-lower variable bounds
    * Added index struct check for variable naming
    * Added check for inverted variable bounds

commit a7fab1b
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Mon Jun 17 17:02:47 2024 +0200

    Short term storage complex (#27)

    * test Quota CO2

    * reformatage du fichier test_quota_co2

    * Ajout de Short term storage complex dans les modeles standard

    * Ajout du test electrolyzer n inputs

    * fix taking in account the comments on the pull request

    * Ajout du isclose et modification des tests pour les electrolyzer n_inputs

    * Fixed test 3 and 4 of the electrolyzer_n_inputs

    * Model SHORT_TERM_STORAGE_COMPLEX avec le même test que le SIMPLE

    * fixed test short term storage

    * fixed generate data for short term storage complex

    * fixed import order

    * Formatting change

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>

commit 12ccfd7
Author: Louis Chouane <116257827+ChouaneLouis@users.noreply.github.com>
Date:   Mon Jun 3 10:56:05 2024 +0200

    Re-implement functional tests in yaml + multiple fixes (#35)

    Solved multiple issues:
    - parsing of shift expressions was wrong
    - integer variables were not correctly transferred to solver
    - improved some error messages at parsing time

    Co-authored-by: vargastat <tatiana.vargas@rte-france.com>

commit 346d852
Merge: 1c1bbd2 52b5b62
Author: ianmnz <ian.menezes@rte-france.com>
Date:   Fri May 17 14:08:37 2024 +0200

    Merge pull request #34 from AntaresSimulatorTeam/test_performance_large_ast

    Performance test for deep AST

commit 52b5b62
Merge: dc4940d 1c1bbd2
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri May 17 13:46:24 2024 +0200

    Merge branch 'main' into test_performance_large_ast

commit dc4940d
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri May 17 13:21:28 2024 +0200

    Add RecursionError raise check for tests

commit f85d56e
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Thu May 16 16:43:10 2024 +0200

    Remove unused generate_random_data function

commit ead9b99
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Thu May 16 16:19:38 2024 +0200

    Performance test: check large nb of port connections

commit 1c1bbd2
Author: vargastat <130074062+vargastat@users.noreply.github.com>
Date:   Mon May 13 13:59:08 2024 +0200

    Yaml format for components and TSV format for timeseries (#37)

commit acc77e8
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri Apr 26 11:43:55 2024 +0200

    Performance test: check deep AST using sum() operator

commit 1221880
Merge: 9f419cd ecd3e1d
Author: Florian Omnès <florian.omnes@rte-france.com>
Date:   Tue Apr 23 17:36:48 2024 +0200

    Merge pull request #32 from AntaresSimulatorTeam/feature/upgrade-ortools

    Upgrade ortools in requirements

commit ecd3e1d
Author: Thomas Bittar <thomas.bittar@rte-france.com>
Date:   Tue Apr 23 17:28:38 2024 +0200

    Remove unnecessary requirements

commit be47c07
Author: Thomas Bittar <thomas.bittar@rte-france.com>
Date:   Tue Apr 23 17:14:10 2024 +0200

    Upgrade ortools in requirements

commit 9f419cd
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Fri Apr 19 09:46:19 2024 +0200

    Models/add new model (#30)

    * Add variant for electrolyzer subject to renewable

    * Add model hydro valley

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit fb547dd
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Fri Apr 19 09:43:20 2024 +0200

    Model test yaml (#29)

    * test electrolyzer_n_inputs with yaml models

    * quota C02 test with yaml model

    * print to test the path

    * fixed path for the first n_input_electrolyzer_yaml test

    * fix of libs path and move of the tests to models

    * factorisation, removal of unused import and move of test files

    * reformat test

    * move tests of model to tests/model

    * Fix formatting

    * re-added libs_dir to model/conftest.py

    * added type declaration and removed unused argument

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>

commit 7c7d2d5
Author: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
Date:   Fri Mar 29 17:17:58 2024 +0100

    Test case for AC link and PST models  (#2)

    Addition of an ac.yml library, and associated test cases.

    Signed-off-by: Sylvain Leclerc <sylvain.leclerc@rte-france.com>

commit f9f9b3a
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Fri Mar 22 15:34:36 2024 +0100

    test Quota CO2 (#17)

    * test Quota CO2

    * reformatage du fichier test_quota_co2

    * Ajout de Short term storage complex dans les modeles standard

    * Ajout du test electrolyzer n inputs

    * fix taking in account the comments on the pull request

    * Ajout du isclose et modification des tests pour les electrolyzer n_inputs

    * Fixed test 3 and 4 of the electrolyzer_n_inputs

commit 94f13b1
Author: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
Date:   Wed Mar 20 17:24:33 2024 +0100

    Model library format parsing (#11)

    First implementation of library format parsing,
    including expressions parsing.

    Implementation notes:
     - ANTLR4 is used for expression parsing
     - pydantic together with PyYaml are used for yaml parsing
     - An additional layer of "unresolved" objects, that are the
       1-to-1 image of the input data, is introduced.
       Once everything is read, we can resolve all objects to
       actually use them for computation.

    Note:
    - Last additions for investment studies have not yet been included ("context" and separation between 2 objectives).

    ---------

    Signed-off-by: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
    Signed-off-by: vargastat <tatiana.vargas@rte-france.com>
    Co-authored-by: vargastat <tatiana.vargas@rte-france.com>

commit c4b0abd
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Tue Mar 19 11:34:37 2024 +0100

    Add flow-based models (#23)

    * Add flow based models

    * Add new model for flow based with net positions

    * Delete useless model

    * Correct errors

    * Rename nodes to real countries to clarify differences between models

    * Small corrections

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit dfc8871
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Tue Mar 19 11:34:16 2024 +0100

    Add models representing binding constrains (#25)

    * Correct error in stock model

    * Add model constraining total imports

    * Add nuclear modulation model

    * Add model with minimum electric vehicle load

    * Add hvdc model

    * Delete models because more efficient models are available with ports

    * Small corrections on names and temporal sums

    * Change model to take into account total import on a node

    * Small corrections

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit 2e2f8bb
Author: tbittar <thomas.bittar@gmail.com>
Date:   Fri Mar 15 16:40:17 2024 +0100

    Correct electrolyzer model typos (#26)

    * Fix sign mistakes

    * Fix sign and arrow direction

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>
tbittar added a commit that referenced this pull request Apr 11, 2025
* Define tree structure for time blocks and data

Define tree structure

* Improve test description

* Create class to store configuration of a resolution node

* Work in progress

* Design the construction of network on  decision tree

* Work in progress

* Add comments

* Fix rebase conflict

* Feature/network cloning (#28)

* Move pathway tests to functional folder

* Add replicate method to classes to be able to copy them to tree

* Remove integration test from functional xpansion test

* Modify DecisionTree

* Move decision_tree test to unit tests folder

* Ideas for the coupling_model

* Write subproblem with tree

* Add first pathway test

* Add three node test for pathway

* Add prob to tree nodes for expectation computation

* Remove test debug init

* Add prob to decision tree node

* Add ExpectedValueStrategy to Benders

* Retour of PR comments

* Squashed merged commit from 'main'

commit 7fa89213801af1f57e340bcc40b0dac1eb6c4af7
Merge: 964297a 0074e94
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Wed Jul 24 11:43:38 2024 +0200

    Merge branch 'main' into feature/investment_pathway

commit 0074e94
Author: ianmnz <ian.menezes@rte-france.com>
Date:   Wed Jul 24 10:18:58 2024 +0200

    Add scenario benders test, bounds validation, and support for bool variables (#46)

    * Cherry-picked solver var name prefixes and suffixes
    * Added scenario tests for benders decomposed
    * Added BoolVar type
    * Added check for upper-lower variable bounds
    * Added index struct check for variable naming
    * Added check for inverted variable bounds

commit a7fab1b
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Mon Jun 17 17:02:47 2024 +0200

    Short term storage complex (#27)

    * test Quota CO2

    * reformatage du fichier test_quota_co2

    * Ajout de Short term storage complex dans les modeles standard

    * Ajout du test electrolyzer n inputs

    * fix taking in account the comments on the pull request

    * Ajout du isclose et modification des tests pour les electrolyzer n_inputs

    * Fixed test 3 and 4 of the electrolyzer_n_inputs

    * Model SHORT_TERM_STORAGE_COMPLEX avec le même test que le SIMPLE

    * fixed test short term storage

    * fixed generate data for short term storage complex

    * fixed import order

    * Formatting change

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>

commit 12ccfd7
Author: Louis Chouane <116257827+ChouaneLouis@users.noreply.github.com>
Date:   Mon Jun 3 10:56:05 2024 +0200

    Re-implement functional tests in yaml + multiple fixes (#35)

    Solved multiple issues:
    - parsing of shift expressions was wrong
    - integer variables were not correctly transferred to solver
    - improved some error messages at parsing time

    Co-authored-by: vargastat <tatiana.vargas@rte-france.com>

commit 346d852
Merge: 1c1bbd2 52b5b62
Author: ianmnz <ian.menezes@rte-france.com>
Date:   Fri May 17 14:08:37 2024 +0200

    Merge pull request #34 from AntaresSimulatorTeam/test_performance_large_ast

    Performance test for deep AST

commit 52b5b62
Merge: dc4940d 1c1bbd2
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri May 17 13:46:24 2024 +0200

    Merge branch 'main' into test_performance_large_ast

commit dc4940d
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri May 17 13:21:28 2024 +0200

    Add RecursionError raise check for tests

commit f85d56e
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Thu May 16 16:43:10 2024 +0200

    Remove unused generate_random_data function

commit ead9b99
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Thu May 16 16:19:38 2024 +0200

    Performance test: check large nb of port connections

commit 1c1bbd2
Author: vargastat <130074062+vargastat@users.noreply.github.com>
Date:   Mon May 13 13:59:08 2024 +0200

    Yaml format for components and TSV format for timeseries (#37)

commit acc77e8
Author: Ian Menezes <ian.menezes@rte-france.com>
Date:   Fri Apr 26 11:43:55 2024 +0200

    Performance test: check deep AST using sum() operator

commit 1221880
Merge: 9f419cd ecd3e1d
Author: Florian Omnès <florian.omnes@rte-france.com>
Date:   Tue Apr 23 17:36:48 2024 +0200

    Merge pull request #32 from AntaresSimulatorTeam/feature/upgrade-ortools

    Upgrade ortools in requirements

commit ecd3e1d
Author: Thomas Bittar <thomas.bittar@rte-france.com>
Date:   Tue Apr 23 17:28:38 2024 +0200

    Remove unnecessary requirements

commit be47c07
Author: Thomas Bittar <thomas.bittar@rte-france.com>
Date:   Tue Apr 23 17:14:10 2024 +0200

    Upgrade ortools in requirements

commit 9f419cd
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Fri Apr 19 09:46:19 2024 +0200

    Models/add new model (#30)

    * Add variant for electrolyzer subject to renewable

    * Add model hydro valley

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit fb547dd
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Fri Apr 19 09:43:20 2024 +0200

    Model test yaml (#29)

    * test electrolyzer_n_inputs with yaml models

    * quota C02 test with yaml model

    * print to test the path

    * fixed path for the first n_input_electrolyzer_yaml test

    * fix of libs path and move of the tests to models

    * factorisation, removal of unused import and move of test files

    * reformat test

    * move tests of model to tests/model

    * Fix formatting

    * re-added libs_dir to model/conftest.py

    * added type declaration and removed unused argument

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>

commit 7c7d2d5
Author: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
Date:   Fri Mar 29 17:17:58 2024 +0100

    Test case for AC link and PST models  (#2)

    Addition of an ac.yml library, and associated test cases.

    Signed-off-by: Sylvain Leclerc <sylvain.leclerc@rte-france.com>

commit f9f9b3a
Author: Yann-Temudjin <160221141+Yann-Temudjin@users.noreply.github.com>
Date:   Fri Mar 22 15:34:36 2024 +0100

    test Quota CO2 (#17)

    * test Quota CO2

    * reformatage du fichier test_quota_co2

    * Ajout de Short term storage complex dans les modeles standard

    * Ajout du test electrolyzer n inputs

    * fix taking in account the comments on the pull request

    * Ajout du isclose et modification des tests pour les electrolyzer n_inputs

    * Fixed test 3 and 4 of the electrolyzer_n_inputs

commit 94f13b1
Author: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
Date:   Wed Mar 20 17:24:33 2024 +0100

    Model library format parsing (#11)

    First implementation of library format parsing,
    including expressions parsing.

    Implementation notes:
     - ANTLR4 is used for expression parsing
     - pydantic together with PyYaml are used for yaml parsing
     - An additional layer of "unresolved" objects, that are the
       1-to-1 image of the input data, is introduced.
       Once everything is read, we can resolve all objects to
       actually use them for computation.

    Note:
    - Last additions for investment studies have not yet been included ("context" and separation between 2 objectives).

    ---------

    Signed-off-by: Sylvain Leclerc <sylvain.leclerc@rte-france.com>
    Signed-off-by: vargastat <tatiana.vargas@rte-france.com>
    Co-authored-by: vargastat <tatiana.vargas@rte-france.com>

commit c4b0abd
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Tue Mar 19 11:34:37 2024 +0100

    Add flow-based models (#23)

    * Add flow based models

    * Add new model for flow based with net positions

    * Delete useless model

    * Correct errors

    * Rename nodes to real countries to clarify differences between models

    * Small corrections

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit dfc8871
Author: Juliette-Gerbaux <130555142+Juliette-Gerbaux@users.noreply.github.com>
Date:   Tue Mar 19 11:34:16 2024 +0100

    Add models representing binding constrains (#25)

    * Correct error in stock model

    * Add model constraining total imports

    * Add nuclear modulation model

    * Add model with minimum electric vehicle load

    * Add hvdc model

    * Delete models because more efficient models are available with ports

    * Small corrections on names and temporal sums

    * Change model to take into account total import on a node

    * Small corrections

    ---------

    Co-authored-by: Juliette-Gerbaux <juliette.gerbaux_externe@rte-france.com>

commit 2e2f8bb
Author: tbittar <thomas.bittar@gmail.com>
Date:   Fri Mar 15 16:40:17 2024 +0100

    Correct electrolyzer model typos (#26)

    * Fix sign mistakes

    * Fix sign and arrow direction

    ---------

    Co-authored-by: Thomas Bittar <thomas.bittar@rte-france.com>

* Fixed var name tree prefix

* Cherry-picked benders_decomposed debug mode

* Feature/automatic coupling model (#52)

* Refactored Port classes to port.py

* Add automatic decision tree node coupling

* Cherry-picked benders_decomposed debug mode

* Fixed var name tree prefix

* Move invest pathway model to test only

* Allow for more flexible expressions between tree nodes

* Reset to a more rigid type of coupling constraints

* Fixed unit test errors

* Fixed expected value risk management missing

* Fix antlr differences

* Fixed from PR comments

---------

Co-authored-by: ianmnz <ian.menezes@rte-france.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants