Exposing all DSS objects and uniform batches

[PMeira]

In DSS C-API v0.12.0, there are two sets of functions Obj_* and Batch_* to manipulate DSS properties directly, bypassing the textual parser and string conversion to read/write the properties.

Initially, my plan would be to generate code from the schema like in IObj.py (also WIP) -- for Python I added the "magic numbers" directly to avoid the lookup overhead, Julia could use enums/constants. Most of the code is very simple (mapping class id, property id and property type to the language types). I also plan to publish a C++ version of that in a few days.

Besides some general things like naming conventions (ideally we should try to use a default convention across all projects), does anyone have suggestions before I try this? We can always add an initial implementation and iterate later, as long as things work.

We should add more functions to get/manipulate the power flow state in DSS C-API v0.12.1, effectively moving away from the classic API based on the original COM implementation and the "Active..." concept. And dss-extensions/dss_capi#113 will allow us to return the matrix shapes.

EDIT: Some samples for Python available in test_obj.py and test_batch.py.

In Python, this is currently being developed in AltDSS-Python. See the docs at https://dss-extensions.org/AltDSS-Python/examples/GettingStarted.html
We should start porting it soon to Julia.

[PMeira]

Some initial code for this should in the next release, or the one right after. First we'll add the code for individual objects, then work on the batches later.

The life cycle of the objects still needs some thinking. The life cycle on the DSS engine:

• Buses can be added or invalidated (removed form the circuit) when objects are added, edited or disabled.
• Objects are always valid in the current circuit, but a clear command (or calling ClearAll()) will invalidate the pointers.

I'm not sure using weak refs in Julia is a good idea, but either way there is some tracking required; maybe adding a second pointer that indicates the circuit state is enough. The initial version doesn't need to handle this yet and we can mark the submodule as experimental.

[PMeira]

I'll try to push a test branch here and in DSS C-API this Monday.

Some notes on the current implementation:

• Modules: Nearly all of the code will be in AltDSS submodule, with a further submodule in that for each DSS class. Since the same version supports loading libs from EPRI's distribution, this should make it clear that the submodule is exclusive to the AltDSS engine. For some other languages, we're separating into "classic" and "alt" modules/namespaces etc.
• Types: I played a lot with the types but they can be updated to simplify the code in a lot of places. One especially ugly pattern I used is to include a few .jl with several common methods to keep them on each module. Using abstract bases (already in the code, e.g. AbstractCircuitElement) and reexporting the methods should work fine, but not sure how Documenter.jl handles that, nor how it affects performance. We just need some feedback to update this later.
  • Personally, I wouldn't mind keeping the base class methods completely separate, but from the years of questions I've got so far, a lot of people don't like that, hence the explicit repetition across modules.
• Pointer lifetimes: It's too much for me to handle at once, so I ignored the pointer lifetime issues. We can handle it in multiple ways. Any of them will require a check before using the object/bus pointers, thus equivalent. AltDSS provides callbacks we can use to track the circuit and bus lifetimes.
• Naming conventions: Following the original API here and what we adopted in other projects, the property names are left as close as possible with the DSS scripts. That is, besides unfortunate names like %LoadLoss (becomes pctLoadLoss), the property name is left as-is.
  • How about the other methods? Should we use more conventional names for Julia?
• Most of methods are trivial, very easy to autogen, but also to maintain incrementally. Example below. Here, _PropertyIndex contains all property numbers, and the aux methods obj_get_float64/obj_set_float64 handle everything including error checking. flags::SetterFlagsT is for advanced control of some side effects (e.g. skip YPrim updates).

"""
Base Frequency for ratings.

Name: `BaseFreq`
Units: Hz
(Getter)
"""
function BaseFreq(obj::Obj)::Float64
    obj_get_float64(obj, _PropertyIndex.BaseFreq)
end

"""
Base Frequency for ratings.

Name: `BaseFreq`
Units: Hz
(Setter)
"""
function BaseFreq(obj::Obj, value::Float64, flags::SetterFlagsT=SetterFlagsT(0))
    obj_set_float64(obj, _PropertyIndex.BaseFreq, value, flags)
end

• Batches: Since there is some overhead when mapping the pointers to structs in Julia, implementing batches as in AltDSS-Python (and altdss.hpp) is still in the plans. For the moment, I added a simple All function (might be renamed, but I like that it's short) for all modules, in two versions: one which handles all elements, and one that accepts a vector of elements. This seems enough to handle many use-cases.

• Tests: As always, we'll need a lot of tests. Besides the individual object properties, buses and energy meters were adapted to expose objects directly.

• We can create elements and populate them through the new modules (example below), but the New method can be extended to use keyword args to simplify the syntax. Currently, it can get too verbose quickly. It will be interesting to compare the performance of using keyword args in Julia to create a large circuit (bypassing the DSS parser) vs. running a .DSS script.

  • Besides the verbosity, currently each property edit opens/closes an edit context for the object. We should expose the begin_edit and end_edit methods to allow more control (avoiding repeated calcs), or just use those in the New (and a similar Edit) method.

using OpenDSSDirect: dss, AltDSS as alt
dss("new circuit.test")
l1 = alt.Load.New("load1")
alt.Load.Bus1(l1, "sourcebus")
alt.Load.kW(l1, 123.0)

To conclude, it's good to mention that several of the classic modules can be reused directly, we don't need to reimplement them. For the DSS objects, I think the new API is already a lot more convenient since we don't need to worry about what's active and so on.

using OpenDSSDirect: dss
using OpenDSSDirect.AltDSS

dss("redirect electricdss-tst/Version8/Distrib/IEEETestCases/13Bus/IEEE13Nodeckt.dss")

println(Load.ToJSON(Load.Get(1)))

a_load = Load.Get("671")
a_line = Line.Get("670671")
@show Load.Powers(a_load)
@show Line.Powers(a_line)

# Sum the total powers for all loads
@show sum(Load.All(Load.TotalPowers))

# Get all the PC elements connected to the bus 611, then all their powers
b = Bus.Get("611")
@show All(Bus.PCElements(b), CircuitElement.Powers)

Output:

{"Name":"671","Bus1":"671.1.2.3","Phases":3,"Conn":"delta","Model":1,"kV":4.1600000000000001E+000,"kW":1.1550000000000000E+003,"kvar":6.6000000000000000E+002}
Load.Powers(a_load) = ComplexF64[385.3551151898592 + 207.35015137183095im, 396.0391930015869 + 240.19800776502717im, 373.61959009465954 + 212.49007853944545im]
Line.Powers(a_line) = ComplexF64[1047.1026908775632 + 457.10058063300164im, 460.00958190179523 + 85.93416204102739im, 971.754152653354 + 275.83781107280834im, -1039.3647554533911 - 412.8814307408783im, -464.3760084944864 - 83.25386043876559im, -952.3976127410281 - 250.405378099729im]
sum(Load.All(Load.TotalPowers)) = ComplexF64[3454.676508394195 + 2094.6437370296226im]
All(Bus.PCElements(b), CircuitElement.Powers) = Vector{ComplexF64}[[163.45974879842822 + 76.92806636588504im, 0.0 + 0.0im], [-7.275957614183426e-15 - 92.45613844770335im, 0.0 + 0.0im]]