Skip to content

Main next release #79

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 125 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
125 commits
Select commit Hold shift + click to select a range
7f5825d
snapshot
VigneshVSV Aug 15, 2024
27f6d0c
update dataklasses
VigneshVSV Aug 15, 2024
93dde35
updated messaging contract and RPC server
VigneshVSV Aug 18, 2024
205b996
changes continuing testing new messaging contract and RPC server - no…
VigneshVSV Sep 15, 2024
d955fa6
update test message broker with only one server and add comments
VigneshVSV Oct 26, 2024
efb3840
test until AsyncZMQServer complete
VigneshVSV Oct 27, 2024
5299cc1
minor updates to zmq message brokers
VigneshVSV Dec 2, 2024
a7e92a4
minor updates to zmq message brokers
VigneshVSV Dec 2, 2024
4450e25
update dataResponse
VigneshVSV Dec 2, 2024
b8d4495
updates
VigneshVSV Dec 13, 2024
16e9d0d
request message class added
VigneshVSV Dec 16, 2024
8f5933f
remove unnecessary lines in message contract
VigneshVSV Dec 19, 2024
6e117e2
fix indices
VigneshVSV Dec 19, 2024
e8a7340
moved request and response messages to their own object
VigneshVSV Dec 22, 2024
b9e8ffd
updates to messaging contract and some tests
VigneshVSV Dec 22, 2024
1533a3d
update sender and receiver ID in messaging contract header
VigneshVSV Dec 22, 2024
bd5b66d
bug fix brokers more and more tests
VigneshVSV Dec 22, 2024
88432a1
update other files - to correspond to ZMQ message broker improvements
VigneshVSV Dec 22, 2024
1d795f6
update with receiverID
VigneshVSV Dec 23, 2024
2a5786e
update RPC server logic
VigneshVSV Dec 23, 2024
2f7afcc
move rpc_server to protocols.zmq
VigneshVSV Dec 23, 2024
325f4e2
rearrange tests
VigneshVSV Dec 25, 2024
c00a80e
RPC server can execute with new messaging contract - incomplete though
VigneshVSV Dec 25, 2024
5d24d21
certain refactoring of HTTP server
VigneshVSV Dec 25, 2024
af79347
more refactoring - protocols moved to top folder
VigneshVSV Dec 28, 2024
7dde041
add server execution context test and more refactoring
VigneshVSV Dec 28, 2024
1dee4cd
added full fledged ZMQ server and some unit tests for it
VigneshVSV Dec 29, 2024
4796025
added thing execution context to action call and unit test
VigneshVSV Dec 29, 2024
890be25
test files refactoring
VigneshVSV Dec 29, 2024
8d1f1da
some updates
VigneshVSV Jan 1, 2025
c66ab3e
added pydantic for action argument validation
VigneshVSV Jan 8, 2025
eba034d
HTTP server update
VigneshVSV Jan 9, 2025
d2c2207
snapshot serializer customization on interaction-per-interaction basis
VigneshVSV Jan 9, 2025
e518be8
remove old docs
VigneshVSV Jan 10, 2025
24a336d
move TD to another top level folder
VigneshVSV Jan 10, 2025
2a1361c
completed serializer customization singleton class
VigneshVSV Jan 10, 2025
e43b6c6
fix bugs with Serializers singleton and resolve circular imports
VigneshVSV Jan 11, 2025
b4cb256
fix TD related imports and thereof more circular imports, also compos…
VigneshVSV Jan 12, 2025
f6f1c44
organised client abstractions base classes
VigneshVSV Jan 15, 2025
a46e6f0
straightening actions both client and server side
VigneshVSV Jan 15, 2025
441e252
Actions moved to descriptors
VigneshVSV Jan 16, 2025
e11c098
bug fixes classmethod for actions
VigneshVSV Jan 16, 2025
9c3d01d
bug fixes classmethod for actions
VigneshVSV Jan 16, 2025
0107c56
bug fixes classmethod for actions
VigneshVSV Jan 16, 2025
9230b80
bug fixes and improve tests for actions
VigneshVSV Jan 17, 2025
03f504a
bug fixes for action client from action affordance
VigneshVSV Jan 17, 2025
d79f753
bug fixes async actions
VigneshVSV Jan 18, 2025
6352af7
integration pydantic for action argument validation started
VigneshVSV Jan 18, 2025
1b8e7b8
added pydantic model validation for arguments
VigneshVSV Jan 19, 2025
81f791c
re-ran working tests and fixed imports
VigneshVSV Jan 19, 2025
0077e63
pydantic and JSON schema argument validation significantly improved a…
VigneshVSV Jan 20, 2025
4a34af5
scheduling of async and threaded actions now possible
VigneshVSV Jan 21, 2025
87e43d6
scheduler cleanups ongoing
VigneshVSV Jan 21, 2025
b71455b
moved meta class to another file and renamed server namespace to 'core'
VigneshVSV Jan 22, 2025
75dad93
descriptor registry organisation and tests
VigneshVSV Jan 24, 2025
c9fe9bf
bug fixes and renaming files
VigneshVSV Jan 26, 2025
07b2db8
bug fix instance binding for state machine
VigneshVSV Jan 27, 2025
1f0654f
add test for property registry
VigneshVSV Jan 27, 2025
12adea8
test read and write multiple properties
VigneshVSV Jan 27, 2025
e2165f3
added UML diagram for RPC server
VigneshVSV Jan 28, 2025
9fba797
moved schedulers to three types - async, threaded & queued
VigneshVSV Jan 28, 2025
22149cb
bug fix ZMQServer and transports argument to RPCServer
VigneshVSV Jan 28, 2025
78d1332
namespace for core zmq features moved to core. docs for RPC server co…
VigneshVSV Jan 29, 2025
f365c4b
state machine doc update
VigneshVSV Jan 29, 2025
1b1242b
improve test registry
VigneshVSV Feb 2, 2025
6e83a81
update test thing init with refactoring common tests for registries
VigneshVSV Feb 8, 2025
bda0b3b
refactor imports from core & add more docstrings
VigneshVSV Feb 9, 2025
3d52a00
added JSON schema object
VigneshVSV Feb 9, 2025
4006c8f
refactor TD namespace import and bug fix affordance generation to use…
VigneshVSV Feb 9, 2025
3870434
add TD dataschema tests
VigneshVSV Feb 9, 2025
d298ef5
finish test number, string & boolean data schema
VigneshVSV Feb 9, 2025
8919d4e
complete generate() for all three interaction affordances
VigneshVSV Apr 19, 2025
6d90855
minor improvements in tests
VigneshVSV Apr 19, 2025
f03557b
bind affordance object to descriptor instead of bound action
VigneshVSV Apr 19, 2025
37e3c11
generic improvements
VigneshVSV Apr 19, 2025
a6f14ad
Bug fix actions after generating TD (TM/without forms)
VigneshVSV Apr 19, 2025
d3e452d
move non working tests to own folder so that pipeline can run
VigneshVSV Apr 21, 2025
1e137c3
fix unique ID generation for client
VigneshVSV Apr 21, 2025
499a75f
complete rebase with main
VigneshVSV Apr 21, 2025
26684dc
bug fix to allow supplying an Action to action decorator
VigneshVSV Apr 21, 2025
3c6e753
bug fixes, load test suite correctly while unittesting from folder level
VigneshVSV Apr 21, 2025
7730fd3
remove exposed_actions test as it may not reliably work
VigneshVSV Apr 21, 2025
60c3ea4
remove uploading coverage report from testing next release
VigneshVSV Apr 21, 2025
d054bd7
bug fix actions for classmethod - dont allow rewrapping
VigneshVSV Apr 26, 2025
1f0e3d6
bug fix unknown payload for preserialized data, reintroduce test_rpc_…
VigneshVSV Apr 26, 2025
d38c3a7
provide serializer settings reset method
VigneshVSV Apr 26, 2025
e544e5c
reinstate local property tests
VigneshVSV Apr 26, 2025
c5d9b05
reinstate thing description tests
VigneshVSV Apr 26, 2025
1be19c8
remove storage logic (like JSON and database) to own folder - tests n…
VigneshVSV Apr 27, 2025
368b81d
reogranise consumed interactions - no protocols folder, only client/s…
VigneshVSV Apr 27, 2025
72b9f82
introduce messaging contract tests for message mapped client pool
VigneshVSV Apr 27, 2025
d59f459
add polling and exit tests for message mapped client pool
VigneshVSV Apr 27, 2025
6a77eae
reorder tests to check consumed properties and actions after testing …
VigneshVSV May 1, 2025
ccc5022
more reordering for RPC server tests
VigneshVSV May 1, 2025
0fde81c
bug fix response id in zmq message brokers for noblock messages
VigneshVSV May 3, 2025
fadf6a0
add test for all operations of properties except observe/unobserve
VigneshVSV May 3, 2025
e45485b
refactor ZMQ consumed affordances
VigneshVSV May 3, 2025
8d9e067
renaming, refactories in test RPC server
VigneshVSV May 3, 2025
4d7d186
add basic event instantiation tests and corresponding bug fixes and r…
VigneshVSV May 4, 2025
65745c8
set observable flag correctly in obversable events and properties tha…
VigneshVSV May 9, 2025
ab6d4e5
bug fixes for basic working events
VigneshVSV May 10, 2025
1e2945f
introduce event message formalism (hacked subclass of ResponseMessage…
VigneshVSV May 10, 2025
4d584f8
test async ZMQ event client
VigneshVSV May 10, 2025
9b45743
test if RPC server can be started by Thing class
VigneshVSV May 10, 2025
61006ef
add tunneling of events from INPROC to IPC and TCP along with tests
VigneshVSV May 10, 2025
24073ff
bug fix event publisher context termination
VigneshVSV May 10, 2025
cf1b70e
HTTP server init, run and stop basic test
VigneshVSV May 11, 2025
1142909
github actions attempt fix test discovery
VigneshVSV May 11, 2025
1dcf68c
addable HTTP handlers per interaction affordance
VigneshVSV May 11, 2025
30bc927
disable tests temporarily due to issue with coverage tool
VigneshVSV May 11, 2025
60ebbbb
refactoring and bug fixes to add a Thing to HTTP server
VigneshVSV May 15, 2025
780bf2f
refactor TD & TM code
VigneshVSV May 16, 2025
7f5deb9
various bug fixes for generating TD/TM
VigneshVSV May 16, 2025
90f3b86
test run_with_http_server and global zmq context
VigneshVSV May 16, 2025
a9d37d5
ObjectProxy cleanup ongoing
VigneshVSV May 16, 2025
f2d5e18
update Thing cls run()
VigneshVSV May 16, 2025
7311f2e
working proxy, some tests added, still needs to be improved
VigneshVSV May 17, 2025
b596c7e
add async read write property invoke action test cases for ObjectProxy
VigneshVSV May 17, 2025
dafbc20
test noblock replies for invoke_action
VigneshVSV May 17, 2025
02e9cc0
test RW multiple properties
VigneshVSV May 18, 2025
3daa402
can invoke action and property operations with HTTP server now again
VigneshVSV May 18, 2025
063b197
sync local changes
VigneshVSV Jun 1, 2025
7a6b9e6
remove setup.py file because we have a toml file now
VigneshVSV Jun 1, 2025
1690cba
update packages in toml file to include all folders and subfolders
VigneshVSV Jun 1, 2025
398c694
update README to reflect updated object API
VigneshVSV Jun 13, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 48 additions & 2 deletions .github/workflows/test-dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ on:
pull_request:
branches:
- main
- main-next-release
# - main-next-release
push:
branches:
- main
- main-next-release
# - main-next-release

jobs:

test:
strategy:
matrix:
Expand All @@ -30,6 +31,8 @@ jobs:

runs-on: ${{ matrix.os }}

if: github.ref == 'refs/heads/main'

steps:
- name: Checkout code
uses: actions/checkout@v4
Expand Down Expand Up @@ -59,3 +62,46 @@ jobs:
uses: codecov/codecov-action@v4.0.1
with:
token: ${{ secrets.CODECOV_TOKEN }}


test-next-release:
strategy:
matrix:
include:
# Define specific Python versions for each OS
- os: ubuntu-latest
python-version: 3.11
# - os: windows-latest
# python-version: 3.11
# - os: macos-latest
# python-version: 3.11
- os: ubuntu-latest
python-version: 3.12

runs-on: ${{ matrix.os }}

if: github.ref == 'refs/heads/main-next-release'

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v3
with:
python-version: ${{ matrix.python-version }}

- name: Install dependencies
run: pip install -r tests/requirements.txt

- name: Run unit tests and generate coverage report
run: |
pip install coverage
coverage run -m unittest discover -s tests -p 'test_*.py'
coverage report -m






4 changes: 2 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[submodule "examples"]
path = examples
url = https://github.com/VigneshVSV/hololinked-examples.git
url = https://github.com/hololinked-dev/examples.git
[submodule "doc"]
path = doc
url = https://github.com/VigneshVSV/hololinked-docs.git
url = https://github.com/hololinked-dev/docs-v2.git
45 changes: 21 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,11 @@ Or, clone the repository (main branch for latest codebase) and install `pip inst

For next-release code base, see [main-next-release](https://github.com/hololinked-dev/hololinked/tree/main-next-release) branch. Out of the many improvements, an attempt to align better with Web of Things is being made, along with a layer of protocol agnosticim.

### Main Next-Release

This branch is the main branch for the next release. The current release is 0.2 and the next release is 0.3.
Not all features may work, but unit tests need to pass while merging.

### Usage/Quickstart

`hololinked` is compatible with the [Web of Things](https://www.w3.org/WoT/) recommended pattern for developing hardware/instrumentation control software.
Expand Down Expand Up @@ -64,35 +69,30 @@ class OceanOpticsSpectrometer(Thing):

#### Instantiating properties

Say, we wish to make device serial number, integration time and the captured intensity as properties. There are certain predefined properties available like `String`, `Number`, `Boolean` etc.
or one may define one's own. To create properties:
Say, we wish to make device serial number, integration time and the captured intensity as properties. There are certain predefined properties available like `String`, `Number`, `Boolean` etc.
or one may define one's own using [pydantic or JSON schema](https://docs.staging.hololinked.dev/howto/articles/properties/#schema-constrained-property). To create properties:

```python

class OceanOpticsSpectrometer(Thing):
"""class doc"""

serial_number = String(default=None, allow_None=True, URL_path='/serial-number',
serial_number = String(default=None, allow_None=True,
doc="serial number of the spectrometer to connect/or connected",
http_method=("GET", "PUT"))
# GET and PUT is default for reading and writing the property respectively.
# So you can leave it out, especially if you are going to use ZMQ and dont understand HTTP

)

integration_time = Number(default=1000, bounds=(0.001, None), crop_to_bounds=True,
URL_path='/integration-time',
doc="integration time of measurement in milliseconds")

intensity = List(default=None, allow_None=True,
doc="captured intensity", readonly=True,
fget=lambda self: self._intensity)

def __init__(self, instance_name, serial_number, **kwargs):
super().__init__(instance_name=instance_name, serial_number=serial_number, **kwargs)
def __init__(self, id, serial_number, **kwargs):
super().__init__(id=id, serial_number=serial_number, **kwargs)

```

> There is an ongoing work to remove HTTP API from the property API and completely move them to the HTTP server

In non-expert terms, properties look like class attributes however their data containers are instantiated at object instance level by default.
For example, the `integration_time` property defined above as `Number`, whenever set/written, will be validated as a float or int, cropped to bounds and assigned as an attribute to each instance of the `OceanOpticsSpectrometer` class with an internally generated name. It is not necessary to know this internally generated name as the property value can be accessed again in any python logic, say, `print(self.integration_time)`.

Expand All @@ -102,7 +102,6 @@ To overload the get-set (or read-write) of properties, one may do the following:
class OceanOpticsSpectrometer(Thing):

integration_time = Number(default=1000, bounds=(0.001, None), crop_to_bounds=True,
URL_path='/integration-time',
doc="integration time of measurement in milliseconds")

@integration_time.setter # by default called on http PUT method
Expand Down Expand Up @@ -148,8 +147,7 @@ If you are <span style="text-decoration: underline">not familiar</span> with Web
what the property represents and how to interact with it from somewhere else. Such a JSON is both human-readable, yet consumable by any application that may use the property - say, a client provider to create a client object to interact with the property or a GUI application to autogenerate a suitable input field for this property.
For example, the Eclipse ThingWeb [node-wot](https://github.com/eclipse-thingweb/node-wot) supports this feature to produce a HTTP(s) client that can issue `readProperty("integration_time")` and `writeProperty("integration_time", 1000)` to read and write this property.

The URL path segment `../spectrometer/..` in href field is taken from the `instance_name` which was specified in the `__init__`.
This is a mandatory key word argument to the parent class `Thing` to generate a unique name/id for the instance. One should use URI compatible strings.
[Full Documentation](https://docs.staging.hololinked.dev/howto/articles/properties/)

#### Specify methods as actions

Expand All @@ -159,7 +157,7 @@ decorate with `action` decorator on a python method to claim it as a network acc

class OceanOpticsSpectrometer(Thing):

@action(URL_path='/connect', http_method="POST") # POST is default for actions
@action(input_schema={"type": "object", "properties": {"serial_number": {"type": "string"}}})
def connect(self, serial_number = None):
"""connect to spectrometer with given serial number"""
if serial_number is not None:
Expand Down Expand Up @@ -206,6 +204,8 @@ and how to interact with it):

> input and output schema ("input" field above which describes the argument type `serial_number`) are optional and will be discussed in docs

[Full Documentation](https://docs.staging.hololinked.dev/howto/articles/actions/)

#### Defining and pushing events

create a named event using `Event` object that can push any arbitrary data:
Expand All @@ -215,7 +215,6 @@ class OceanOpticsSpectrometer(Thing):

# only GET HTTP method possible for events
intensity_measurement_event = Event(name='intensity-measurement-event',
URL_path='/intensity/measurement-event',
doc="""event generated on measurement of intensity,
max 30 per second even if measurement is faster.""",
schema=intensity_event_schema)
Expand All @@ -240,14 +239,14 @@ class OceanOpticsSpectrometer(Thing):
})
last_time = time.time()

@action(URL_path='/acquisition/start', http_method="POST")
@action()
def start_acquisition(self):
if self._acquisition_thread is not None and self._acquisition_thread.is_alive():
return
self._acquisition_thread = threading.Thread(target=self.capture)
self._acquisition_thread.start()

@action(URL_path='/acquisition/stop', http_method="POST")
@action()
def stop_acquisition(self):
self._run = False
```
Expand Down Expand Up @@ -289,6 +288,8 @@ what the event represents and how to subscribe to it) with subprotocol SSE (HTTP

> data schema ("data" field above which describes the event payload) are optional and discussed later

[Full Documentation](https://docs.staging.hololinked.dev/howto/articles/events/)

Events follow a pub-sub model with '1 publisher to N subscribers' per `Event` object, both through ZMQ and HTTP SSE.

To start the Thing, a configurable HTTP Server is already available (from `hololinked.server.HTTPServer`) which redirects HTTP requests to the object:
Expand All @@ -304,7 +305,7 @@ if __name__ == '__main__':
keyfile = f'assets{os.sep}security{os.sep}key.pem')

O = OceanOpticsSpectrometer(
instance_name='spectrometer',
id='spectrometer',
serial_number='S14155',
log_level=logging.DEBUG
)
Expand All @@ -314,10 +315,6 @@ if __name__ == '__main__':
# both interprocess communication & TCP, no HTTP
```

> There is an ongoing work to remove HTTP API from the API of all of properties, actions and events and completely move them to the HTTP server for a more accurate syntax. The functionality will not change though.

Here one can see the use of `instance_name` and why it turns up in the URL path. See the detailed example of the above code [here](https://gitlab.com/hololinked-examples/oceanoptics-spectrometer/-/blob/simple/oceanoptics_spectrometer/device.py?ref_type=heads).

##### NOTE - The package is under active development. Contributors welcome, please check CONTRIBUTING.md and the open issues. Some issues can also be independently dealt without much knowledge of this package.

- [examples repository](https://github.com/hololinked-dev/examples) - detailed examples for both clients and servers
Expand Down
2 changes: 1 addition & 1 deletion doc
Submodule doc updated from d4e965 to 644876
2 changes: 1 addition & 1 deletion hololinked/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "0.2.12"
__version__ = "0.3.1"
Loading