ROS2-Side Updates (#94)

* Merge Dev to main for release 0.4.0 (#74)

* Add logging dependency (#32)

* Changing yamato host flavor to large (#33)

* Amanda/read chunks (#36)

* s.send to s.sendall

* Increase default timeout

* Timeout as argument on instantiation

* Update version number (#38)

* Laurie/topic list (#41)

* Updating Contribution Content (#44)

* Contributing guidelines

* Issue templates

* PR feedback

* Updated per later changes (#45)

* Adding PR template (#46)

* One persistent connection for all messages sent to Unity

* make UnityService accessible from ros_tcp_endpoint (#49)

* Don't keep retrying on success (#50)

* Resolve conflicts for merging dev to main

* Queue module was renamed in Python 3 (#53)

* Update package.xml (#54)

* RosConnection 2.0 (#57)

* Remove outdated handshake references

* Fix sending response to unity service (#59)

* add changelog (#60)

* Forum links to README, config.yml (#61)

* Send empty message on connection start (#58)

* add test coverage for python scripts (#62)

* fix the bug (#64)

* Minor bug fixes (#65)

* Add Tests and Linter (#63)

* Add tests for tcp sender

* Add linter and pre-commit hooks

* Reformat with python black

* Add tests

* Add tests for server, client, publisher, subscriber, ros_service, unity_service, thread_pauser

* Fix pre-commit-config file name; Add client tests

* Add github workflow check

* Formatting

* Resolve conflicts (#66)

* Fix bug (#68)

* Fix bug

* Temporarily remove publisher test

Co-authored-by: Shuo Diao <shuo@unity3d.com>

* update version to 0.4.0 (#69)

Co-authored-by: Devin Miller (Unity) <mrpropellers@users.noreply.github.com>
Co-authored-by: Amanda <31416491+at669@users.noreply.github.com>
Co-authored-by: LaurieCheers-unity <73140792+LaurieCheers-unity@users.noreply.github.com>
Co-authored-by: Hamid Younesy <hyounesy@users.noreply.github.com>
Co-authored-by: peifeng-unity <56408141+peifeng-unity@users.noreply.github.com>

* ROS2 support (#77)

Co-authored-by: Tiffany Yau <tiffany.yau@unity3d.com>

* update changelog (#83)

* Merge v0.5.0 (#84)

* Merge main to dev (#85)

* Stale issue handler (#87)

* Initial commit: stale handler

* Remove latest tag from yamato config

* Updated dates, changelog

* PR Feedback: cron schedule

* Exposing publisher queue_size and latch (#88)

* Working to improve the ROS TCP connector by adding queues and latching.

* Exposing the latch parameter for ROS publishers.

* Updating the CHANGELOG

* Improving style for publish method definition.

* Improving the performance of the read_message by not splitting the da… (#90)

* Improving the performance of the read_message by not splitting the data into 1024 chunks and stitching them together as you go, instead receiving the entire message all at once.

* Updating the changelog

* Publisher signature

* Publisher constructor

* Don't mess around with packets, just read the whole message

* source_destination_dict shatters into 4 separate tables

* Update tests

* 1 more test fix

* Updated server, setup dir

Co-authored-by: Shuo Diao <shuo@unity3d.com>
Co-authored-by: Devin Miller (Unity) <mrpropellers@users.noreply.github.com>
Co-authored-by: LaurieCheers-unity <73140792+LaurieCheers-unity@users.noreply.github.com>
Co-authored-by: Hamid Younesy <hyounesy@users.noreply.github.com>
Co-authored-by: peifeng-unity <56408141+peifeng-unity@users.noreply.github.com>
Co-authored-by: Tiffany Yau <tiffany.yau@unity3d.com>
Co-authored-by: Peter Smith <thisisntanemailaccount@gmail.com>

Author: 8 people

.github/workflows/stale.yaml

@@ -0,0 +1,27 @@
+name: 'Stale issue handler'
+on:
+  workflow_dispatch:
+  schedule:
+    - cron: '0 17 * * *' # 17:00 UTC; 10:00 PDT
+
+permissions:
+  issues: write
+
+jobs:
+  stale:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/stale@v4.0.0
+        id: stale
+        with:
+          stale-issue-label: 'stale'
+          stale-issue-message: 'This issue has been marked stale because it has been open for 14 days with no activity. Please remove the stale label or comment on this issue, or the issue will be automatically closed in the next 14 days.'
+          days-before-stale: 14
+          days-before-pr-stale: -1
+          days-before-close: 14
+          days-before-pr-close: -1
+          exempt-issue-labels: 'blocked,must,should,keep,pinned,work-in-progress,request,announcement'
+          close-issue-message: 'This issue has been marked stale for 14 days and will now be closed. If this issue is still valid, please ping a maintainer.'
+      - name: Print outputs
+        run: echo ${{ join(steps.stale.outputs.*, ',') }}
+        

.yamato/yamato-config.yml

@@ -1,7 +1,7 @@
 name: Endpoint Unit Tests
 agent:
     type: Unity::VM
-    image: robotics/ci-ubuntu20:latest
+    image: robotics/ci-ubuntu20:v0.1.0-795910
     flavor: i1.large
 commands:
     # run unit tests and save test results in /home/bokken/build/output/Unity-Technologies/ROS-TCP-Endpoint

CHANGELOG.md

@@ -6,12 +6,34 @@ The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/) a
 
 ## Unreleased
 
+Add the [Close Stale Issues](https://github.com/marketplace/actions/close-stale-issues) action
+
 ### Upgrade Notes
 
 ### Known Issues
 
 ### Added
 
+Support for queue_size and latch for publishers. (https://github.com/Unity-Technologies/ROS-TCP-Endpoint/issues/82)
+
+### Changed
+
+### Deprecated
+
+### Removed
+
+### Fixed
+
+## [0.5.0] - 2021-07-15
+
+### Upgrade Notes
+
+Upgrade the ROS communication to support ROS2 with Unity
+
+### Known Issues
+
+### Added
+
 ### Changed
 
 ### Deprecated
@@ -38,6 +60,8 @@ Add linter, unit tests, and test coverage reporting
 
 ### Changed
 
+Improving the performance of the read_message in client.py, This is done by receiving the entire message all at once instead of reading 1024 byte chunks and stitching them together as you go.
+
 ### Deprecated
 
 ### Removed

package.xml

@@ -2,8 +2,10 @@
 <?xml-model href="http://download.ros.org/schema/package_format3.xsd" schematypens="http://www.w3.org/2001/XMLSchema"?>
 <package format="3">
   <name>ros_tcp_endpoint</name>
-  <version>0.0.1</version>
-  <description>ROS TCP Endpoint Unity Integration (ROS2 version)</description>
+  <version>0.5.0</version>
+  <description>ROS TCP Endpoint Unity Integration (ROS2 version)
+  Acts as the bridge between Unity messages sent via TCP socket and ROS messages.
+  </description>
   <maintainer email="unity-robotics@unity3d.com">Unity Robotics</maintainer>
   <license>Apache 2.0</license>
 

ros_tcp_endpoint/client.py

@@ -35,7 +35,9 @@ def __init__(self, conn, tcp_server, incoming_ip, incoming_port):
         Set class variables
         Args:
             conn:
-            source_destination_dict: dictionary of destination name to RosCommunicator class
+            tcp_server: server object
+            incoming_ip: connected from this IP address
+            incoming_port: connected from this port
         """
         self.conn = conn
         self.tcp_server = tcp_server
@@ -96,16 +98,7 @@ def read_message(self, conn):
         destination = self.read_string()
         full_message_size = ClientThread.read_int32(conn)
 
-        while len(data) < full_message_size:
-            # Only grabs max of 1024 bytes TODO: change to TCPServer's buffer_size
-            grab = 1024 if full_message_size - len(data) > 1024 else full_message_size - len(data)
-            packet = ClientThread.recvall(conn, grab)
-
-            if not packet:
-                self.logerr("No packets...")
-                break
-
-            data += packet
+        data = ClientThread.recvall(conn, full_message_size)
 
         if full_message_size > 0 and not data:
             self.logerr("No data for a message size of {}, breaking!".format(full_message_size))
@@ -150,16 +143,16 @@ def serialize_command(command, params):
         return cmd_info + json_info
 
     def send_ros_service_request(self, srv_id, destination, data):
-        if destination not in self.tcp_server.source_destination_dict.keys():
-            error_msg = "Service destination '{}' is not registered! Known topics are: {} ".format(
-                destination, self.tcp_server.source_destination_dict.keys()
+        if destination not in self.tcp_server.ros_services_table.keys():
+            error_msg = "Service destination '{}' is not registered! Known services are: {} ".format(
+                destination, self.tcp_server.ros_services_table.keys()
             )
             self.tcp_server.send_unity_error(error_msg)
             self.logerr(error_msg)
             # TODO: send a response to Unity anyway?
             return
         else:
-            ros_communicator = self.tcp_server.source_destination_dict[destination]
+            ros_communicator = self.tcp_server.ros_services_table[destination]
             service_thread = threading.Thread(
                 target=self.service_call_thread, args=(srv_id, destination, data, ros_communicator)
             )
@@ -186,8 +179,8 @@ def logerr(self, text):
 
     def run(self):
         """
-        Read a message and determine where to send it based on the source_destination_dict
-         and destination string. Then send the read message.
+        Receive a message from Unity and determine where to send it based on the publishers table
+         and topic string. Then send the read message.
 
         If there is a response after sending the serialized data, assume it is a
         ROS service response.
@@ -206,6 +199,7 @@ def run(self):
             while not halt_event.is_set():
                 destination, data = self.read_message(self.conn)
 
+                # Process this message that was sent from Unity
                 if self.tcp_server.pending_srv_id is not None:
                     # if we've been told that the next message will be a service request/response, process it as such
                     if self.tcp_server.pending_srv_is_request:
@@ -223,12 +217,12 @@ def run(self):
                 elif destination.startswith("__"):
                     # handle a system command, such as registering new topics
                     self.tcp_server.handle_syscommand(destination, data)
-                elif destination in self.tcp_server.source_destination_dict:
-                    ros_communicator = self.tcp_server.source_destination_dict[destination]
-                    response = ros_communicator.send(data)
+                elif destination in self.tcp_server.publishers_table:
+                    ros_communicator = self.tcp_server.publishers_table[destination]
+                    ros_communicator.send(data)
                 else:
-                    error_msg = "Topic '{}' is not registered! Known topics are: {} ".format(
-                        destination, self.tcp_server.source_destination_dict.keys()
+                    error_msg = "Not registered to publish topic '{}'! Valid publish topics are: {} ".format(
+                        destination, self.tcp_server.publishers_table.keys()
                     )
                     self.tcp_server.send_unity_error(error_msg)
                     self.logerr(error_msg)

ros_tcp_endpoint/publisher.py

@@ -24,17 +24,18 @@ class RosPublisher(RosSender):
     """
     Class to publish messages to a ROS topic
     """
+
     # TODO: surface latch functionality
-    def __init__(self, topic, message_class, queue_size=10):
+    def __init__(self, topic, message_class, queue_size=10, latch=False):
         """
 
         Args:
             topic:         Topic name to publish messages to
             message_class: The message class in catkin workspace
             queue_size:    Max number of entries to maintain in an outgoing queue
         """
-        strippedTopic = re.sub('[^A-Za-z0-9_]+', '', topic)
-        node_name = f'{strippedTopic}_RosPublisher'
+        strippedTopic = re.sub("[^A-Za-z0-9_]+", "", topic)
+        node_name = f"{strippedTopic}_RosPublisher"
         RosSender.__init__(self, node_name)
         self.msg = message_class()
         self.pub = self.create_publisher(message_class, topic, queue_size)
@@ -50,8 +51,8 @@ def send(self, data):
         Returns:
             None: Explicitly return None so behaviour can be
         """
-        #message_type = type(self.msg)
-        #message = deserialize_message(data, message_type)
+        # message_type = type(self.msg)
+        # message = deserialize_message(data, message_type)
 
         self.pub.publish(data)