Improvement to Python wrapper

Some small changes to Compute graph API to define structured datatype.
Some small changes to take into account google colab new version numbers
for the packages.

Author: christophe0606

ComputeGraph/documentation/Generic.md

@@ -1,4 +1,4 @@
-# Generic and functions bodes
+# Generic and functions nodes
 
 The generic and function nodes are the basic nodes that you use to create other kind of nodes in the graph.
 
@@ -16,7 +16,7 @@ There are 3 other classes that can be used to create new nodes from functions:
 * `Binary`
 * `Dsp`
 
-## Generic Nodes
+# Generic Nodes
 
 Any new kind of node must inherit from one of those classes. Those classes are providing the methods `addInput` and/or `addOutput` to define new IOs.
 
@@ -38,7 +38,7 @@ class ProcessingNode(GenericNode):
 
 See the [simple](../examples/simple/README.md) example for more explanation about how to define a new node.
 
-### Methods
+## Methods
 
 The constructor of the node is using the `addInput` and/or `addOutput` to define new IOs.
 
@@ -66,16 +66,17 @@ def typeName(self):
 
 This method defines the name of the C++ class implementing the wrapper for this node.
 
-### Datatypes
+# Datatypes
 
 Datatypes for the IOs are inheriting from `CGStaticType`.
 
-Currently there are two classes defined:
+Currently there are 3 classes defined:
 
 * `CType` for the standard CMSIS-DSP types
 * `CStructType` for a C struct
+* `PythonClassType` to create structured datatype for the Python scheduler
 
-#### CType
+## CType
 
 You create such a type with `CType(id)` where `id` is one of the constant coming from the Python wrapper:
 
@@ -94,25 +95,28 @@ You create such a type with `CType(id)` where `id` is one of the constant coming
 
 For instance, to define a `float32_t` type for an IO you can use `CType(F32)`
 
-#### CStructType
+## CStructType
 
 The constructor has the following definition
 
 ```python
-def __init__(self,name,python_name,size_in_bytes): 
+def __init__(self,name,size_in_bytes): 
 ```
 
 * `name` is the name of the C struct
-* `python_name` is the name of the Python class implementing this type (when you generate a Python schedule)
-* `size_in_bytes` is the size of the struct. It should take into account padding. It is used in case of buffer sharing since the datatype of the shared buffer is `int8_t`. The Python script must be able to compute the size of those buffers and needs to know the size of the structure.
+* `size_in_bytes` is the size of the struct. It should take into account padding. It is used in case of buffer sharing since the datatype of the shared buffer is `int8_t`. The Python script must be able to compute the size of those buffers and needs to know the size of the structure including padding.
 
-In Python, there is no `struct`. This datatype is mapped to an object. Object have reference type. Compute graph FIFOs are assuming a value type semantic.
+## PythonClassType
 
-As consequence, in Python side you should never copy those structs since it would copy the reference. You should instead copy the members of the struct.
+```python
+def __init__(self,python_name)
+```
+
+In Python, there is no `struct`. This datatype is mapped to an object. Object have reference type. Compute graph FIFOs are assuming a value type semantic.
 
-If you don't plan on generating a Python scheduler, you can just use whatever name you want for the `python_name`. It will be ignored by the C++ code generation.
+As consequence, in Python side you should never copy those structs since it would copy the reference. You should instead copy the members of the struct and they should be scalar values.
 
-## Function and constant nodes
+# Function and constant nodes
 
 A Compute graph C++ wrapper is useful when the software components you use have a state that needs to be initialized in the C++ constructor, and preserved between successive calls to the `run` method of the wrapper.
 

ComputeGraph/documentation/PythonAPI.md

@@ -4,7 +4,7 @@ Python APIs to describe the nodes and graph and generate the C++, Python or Grap
 
 1. ## [Graph class](Graph.md)
 
-2. ## [Generic and function nodes](Generic.md)
+2. ## [Generic, function nodes and datatypes](Generic.md)
 
 3. ## Scheduler
 

ComputeGraph/examples/eventrecorder/main.cpp

@@ -9,6 +9,9 @@
 #include  CMSIS_device_header
 #include "cmsis_os2.h"
 
+#if defined(RTE_Compiler_EventRecorder)
+#include "EventRecorder.h"
+#endif 
 
 void app_main (void *argument)
 {

ComputeGraph/examples/example3/AppNodes.h

@@ -32,6 +32,8 @@ using namespace std;
 #include <cstdio>
 #include "arm_math.h"
 
+#include "cg_status.h"
+
 #include "host/FileSink.h"
 #include "host/FileSource.h"
 #include "CFFT.h"

ComputeGraph/examples/example8/README.md

@@ -11,11 +11,15 @@ This example is illustrating :
 
 It is  possible to use a custom datatype:
 
+For the C++ version of the scheduler it would be:
+
 ```python
-complexType=CStructType("complex","MyComplex",8)
+complexType=CStructType("MyComplex",8)
 ```
 
-This is defining a new datatype that is mapped to the type `complex` in C/C++ and the class `MyComplex` in Python. The last argument is the size in bytes of the struct in C.
+This is defining a new datatype that is mapped to the type `complex` in C/C++.
+
+The last argument is the size in bytes of the struct in C.
 
 The type complex may be defined with:
 
@@ -26,6 +30,15 @@ typedef struct {
 } complex;
 ```
 
+For the Python version of the scheduler:
+
+```python
+complexType=PythonClassType("MyComplex")
+```
+
+The class `MyComplex` is used in Python. 
+
+
 **Note that:**
 
 - The value **must have** value semantic in C/C++. So avoid classes 

ComputeGraph/examples/example8/appnodes.py

@@ -23,9 +23,9 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 ############################################
-from cmsisdsp.cg.static.nodes.simu import *
+from cmsisdsp.cg.nodes.simu import *
 from custom import *
-from cmsisdsp.cg.static.nodes.Duplicate import *
+from cmsisdsp.cg.nodes.Duplicate import *
 
 class Sink(GenericSink):
     def __init__(self,inputSize,fifoin):

ComputeGraph/examples/example8/graph.py

@@ -42,7 +42,13 @@ def typeName(self):
 # Modify the fields of the objects, or create a totally new
 # object.
 
-complexType=CStructType("complex","MyComplex",8)
+GEN_PYTHON = False 
+
+
+if GEN_PYTHON:
+   complexType=PythonClassType("MyComplex")
+else:
+   complexType=CStructType("complex",8)
 
 src=Source("source",complexType,5)
 b=ProcessingNode("filter",complexType,7,5)
@@ -71,7 +77,6 @@ def typeName(self):
 
 g.connect(b.ob,nd.i)
 
-GEN_PYTHON = False 
 
 print("Generate graphviz and code")
 

ComputeGraph/examples/example8/sched.py

@@ -10,7 +10,7 @@
 
 import numpy as np
 import cmsisdsp as dsp
-from cmsisdsp.cg.static.nodes.simu import *
+from cmsisdsp.cg.nodes.simu import *
 from appnodes import * 
 from custom import *
 
@@ -102,22 +102,19 @@ def scheduler(someVariable):
        cgStaticError = dup0.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sc.run()
+       cgStaticError = sa.run()
        if cgStaticError < 0:
           break
        cgStaticError = sb.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sa.run()
+       cgStaticError = sc.run()
        if cgStaticError < 0:
           break
        cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = filter.run()
-       if cgStaticError < 0:
-          break
-       cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = sd.run()
@@ -126,13 +123,16 @@ def scheduler(someVariable):
        cgStaticError = dup0.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sc.run()
+       cgStaticError = sa.run()
        if cgStaticError < 0:
           break
        cgStaticError = sb.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sa.run()
+       cgStaticError = sc.run()
+       if cgStaticError < 0:
+          break
+       cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = source.run()
@@ -147,22 +147,19 @@ def scheduler(someVariable):
        cgStaticError = dup0.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sc.run()
+       cgStaticError = sa.run()
        if cgStaticError < 0:
           break
        cgStaticError = sb.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sa.run()
+       cgStaticError = sc.run()
        if cgStaticError < 0:
           break
        cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = filter.run()
-       if cgStaticError < 0:
-          break
-       cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = sd.run()
@@ -171,13 +168,16 @@ def scheduler(someVariable):
        cgStaticError = dup0.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sc.run()
+       cgStaticError = sa.run()
        if cgStaticError < 0:
           break
        cgStaticError = sb.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sa.run()
+       cgStaticError = sc.run()
+       if cgStaticError < 0:
+          break
+       cgStaticError = source.run()
        if cgStaticError < 0:
           break
        cgStaticError = filter.run()
@@ -189,13 +189,13 @@ def scheduler(someVariable):
        cgStaticError = dup0.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sc.run()
+       cgStaticError = sa.run()
        if cgStaticError < 0:
           break
        cgStaticError = sb.run()
        if cgStaticError < 0:
           break
-       cgStaticError = sa.run()
+       cgStaticError = sc.run()
        if cgStaticError < 0:
           break
 

ComputeGraph/examples/example8/test.dot

@@ -64,38 +64,38 @@ source [label=<
 
 
 
-source:i -> filter:i [label="complex(11)"
+source:i -> filter:i [label="MyComplex(11)"
 ,headlabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >7</FONT>
 </TD></TR></TABLE>>
 ,taillabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>]
 
-filter:ob -> sd:i [label="complex(5)"
+filter:ob -> sd:i [label="MyComplex(5)"
 ,headlabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>
 ,taillabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>]
 
 filter:oa ->  
-dup0 [label="complex(5)"
+dup0 [label="MyComplex(5)"
 
 ,taillabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>]
 
 
-dup0 -> sa:i [label="complex(5)"
+dup0 -> sa:i [label="MyComplex(5)"
 ,headlabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>
 ]
 
 
-dup0 -> sb:i [label="complex(5)"
+dup0 -> sb:i [label="MyComplex(5)"
 ,headlabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>
 ]
 
 
-dup0 -> sc:i [label="complex(5)"
+dup0 -> sc:i [label="MyComplex(5)"
 ,headlabel=<<TABLE BORDER="0" CELLPADDING="2"><TR><TD><FONT COLOR="blue" POINT-SIZE="12.0" >5</FONT>
 </TD></TR></TABLE>>
 ]

ComputeGraph/examples/simple/generated/scheduler.cpp

@@ -13,6 +13,7 @@ The support classes and code is covered by CMSIS-DSP license.
 #include "AppNodes.h"
 #include "scheduler.h"
 
+
 #if !defined(CHECKERROR)
 #define CHECKERROR       if (cgStaticError < 0) \
        {\

ComputeGraph/examples/simple/generated/scheduler.h

@@ -16,6 +16,7 @@ extern "C"
 #endif
 
 
+
 extern uint32_t scheduler(int *error);
 
 #ifdef   __cplusplus

ComputeGraph/tests/EventRecorder.log

@@ -8,7 +8,7 @@ Some tests to validate some parts of the Compute graph. They all rely on the CMS
   * Create a complex graph containing all classes defined in `GenericNodes.h` (synchronous mode). Used to test that it builds and that there are no errors in the templates
   * `cbuild "cprj\syncgraph.CommandLine+VHT-Corstone-300.cprj" `
 * `create_async.py`
-  * Create a complex graph containing all classes defined in `GenericNodes.h` (ssynchronous mode). Used to test that it builds and that there are no errors in the templates
+  * Create a complex graph containing all classes defined in `GenericNodes.h` (asynchronous mode). Used to test that it builds and that there are no errors in the templates
   * `cbuild "cprj\asyncgraph.CommandLine+VHT-Corstone-300.cprj" `
 * `create_fifobench_sync.py`
   * Create a graph with FFT / IFFT : the graph is decomposing a signal and rebuilding it. It is used to test the performance of different FIFOs implementations (synchronous mode)

ComputeGraph/tests/README.md

@@ -113,11 +113,11 @@ int main(void)
   cycleMeasurementStart();
   nbSched=scheduler(&error,input_buffer,output_buffer);
   cycleMeasurementStop();
-  int32_t cycles = getCycles() - overhead;
+  int32_t cycles = (getCycles() - overhead)/nbSched;
 
   printf("Number of schedule iterations = %d\n\r",nbSched);
   printf("Error code = %d\n\r",error);
-  printf("Cycles = %d\n\r",cycles);
+  printf("Cycles per iteration = %d\n\r",cycles);
 
 
 #if defined(COMMAND_LINE)

ComputeGraph/tests/main_fifobench.cpp

@@ -2,4 +2,5 @@ rm -f  CMakeCache.txt
 rm -rf CMakeFiles
 rm -f Makefile
 rm -rf bin_dsp
-rm -f cmake_install.cmake
+rm -f cmake_install.cmake
+rm -rf dist

PythonWrapper/build_linux/clean.sh

@@ -27,12 +27,13 @@
  */
 #ifndef CMSISMODULE_H
 #define CMSISMODULE_H
-#define NPY_NO_DEPRECATED_API NPY_1_21_API_VERSION
+#define NPY_NO_DEPRECATED_API NPY_1_22_API_VERSION
 
 #include <numpy/numpyconfig.h>
 
 // API version used on google colab
-#if (NPY_API_VERSION != 0x0000000E )
+// List on https://github.com/numpy/numpy/blob/main/numpy/core/include/numpy/numpyconfig.h
+#if (NPY_API_VERSION != 0x0000000F )
 #error("Error building with wrong NumPy API version")
 #endif