RTK2go fail with the parser

[CompuMakEngineering]

Hi ,

I am using RTKtogo to receive the RTK correction, below is an example of what I receive, however the script fail in parsing this data, the start byte doesn't match, however If I change the format to compare with \xd3 it matches, however everything else fail downstream,

Also I noticed that there is character like '>' and letter N and ? , I am not sure if I am using the correct RTCM or NTRIP version as well, can you please advise

I am using ROS melodic with Python 2

('Data received ', '\xd3\x00\x95>\xc0\x00N\xd4\xb9\xa2\x90u\x9b"d\x03\xd9\xdf\xd2$\x00+B\x90\xeb\xfd \xc0\xa1\xee \x07\t\xfe\xa10\x01\x17\x14\xbf\xe6 \x19\x98\x94\xff\xea\x9f\xf4\xc9\xc0\x060\xa9\xe7\xff0\xca)\xa3\xa0\x1b\xda\xbf\xa1X\x00O\x85\xe5\x0f\xfa\xc3\x18\x8c"\xe0\xec%\xfd\x1a\xc0\x01\x08<r?\xd6\x16\x90\xa8F\x03;\xff\xe9\x95\x00\x08aY\x1d\xfe\x91\x84\xab=\x8f\xfc9\xceP\x84\x010\x0e\xe2\x8f\xf3\x0f\xad\x031\xbf\xbd\xeb\xaa\x9d@\x00\xc9\xa4Gr\x8cq\x89\xa7\xaa\x10\xf3\xbf\xd2\xab\x00\x03@\xcf_\xfd@\x85\xad\x16\xd3\x00\x8a?@\x00\x9c\xd3\xf3\x14\x01c\xc6\xce)\x00\xa5\x8b\xfa:C\xf9\xaf\xe8|\xed\x10H\xb6@8\xb0\x18\xd8\xfe\x8e@\xffd1\xc9\xff\xc4&R\x1a\xed\x17\xee\xfc\x1a$l>f\xfc\xa4\x0e\x81\x06\xdc\xd6H\xfe\x8do\xe9\xe3\x00-\x800\xfas\xe0$&H\x8f\xc0b3\xfaY\x03\xf1\x88\x00\x00t\xb8P\x14\x1az\x10:\x85\xfe\x820\x80\x02\x00\x00?\x80(NH\xb4J\x00\x00?\xa1d?{\xff\x90\x9f\xf0\x04\x1a\x00\x00\x00\x80\x00\x0c\xc0\x00\x08\x00 \x00\x03\xf9\xc0\x8c\xa2\xa3\xd3\x00\xf4CP\x00N\xd4\xb9\xa2\x00\x00\x02\x18\x00\xcd\x80\x00\x00\x00 \x00@\x00\x7f\xff\xea\t\x88\xca\x08II\ni\x80\x00\x00\x00\x00\x16\xd1\x1b};6n<\xc6Lr\x84p\x01\x88MA\xdd\x01O\xf7g\xde\x1f\x1b\x046OrE('\xa2JZ7:1\xb3\x03-\xbf\xe3\xbc~\x90\xe9\xa9\xe2\x1a\xca\x903i\xc3:>T"Erj\x16XAlh0\x15#\x1d\x0e'\xdb\x86)\xd7\xb5\xee\xf8'\xf6\xa2\x98%\xb1\xb2\x0b!R\x0c[\x93\xfa\xcb\x1c\t@o\xd5\x0eb I\xa8\x02Jj)\xe7^\x11\x16\x1a$\xbc\xc4\x19B\xb6\x149j\t\xc8\x11*\xca\xd1\xb4{9NM\xad\t1\xcc\x12T\x93\x13\xc6\x0f[\xcf4\xcd\x00\x04\xc10T\x14\x85\x81XB\x13\x05\x81XV\x15\x04\x81HT\x11\x84\xe10}\r\xfa\xa1\xca\xb7\xa4\x7fr\xfe\xa0\xff#\xfeT\xff\xb7\x02G\xd4\xe3\xbd\xe9\x08\xc1\xd3\x1e\r<\xe2\t\xfe\x16\x94a\xc9\xc6\xd3\x00\xc8C\xf0\x00s\x9a~b\x00\x00\xf0\x18\x00\x00\x00\x00\x00 \x80\x00\x00{\xdf\xa5''#\xa0\xa3\xa4!\xc1\xea\x83\xd4\xd5\xde\xed/\x1d4\x07\xa4\xb7\x84\x83\xa1\xfayA{\x82K\xf6p^_\xa2eS\x9eD\xf0\xecL\xae\xd1\x82\x82g\x86\xbdx@88\xc4\xff5O\xec9\x8c\x89\x90\xab\xedcAN*\xbe\x80-\xcc\x883\xc8\x02D\x16\xf5\xa7\xf3\x00\x89\xb9\x84c\xbb\x02\xe4\xd4\xff\xb2\x81\x84C\xae\x06,B\x7fQ\\x00\x00\x1cxmL\xfb\xe7\xc4H\xcc\xbc4\xda\xc1\x93\x8c\xf7,DOc\x14\x00RP\x10\x00\x12\x02\xe1\x18>\x07\x04\x81\x10F\x14\x84@\xd8:\x0c\x84\x01\xb9\xf7{\xf7\x11\x1d\xaa|\xb6\xb7\xbf\xf0,&\xb8\x1d\xe4\x16\xe8\x970\xde@\xd2\x7fs\x80\x99\xe9z\xd3\x00\xf4D\x90\x00N\xd4\xb9\xa2\x00\x00\n\x05\x14\x14 \x00\x00\x00\x04\x00\x80\x00\x7f\xff\xea\xebj\xea\xca\x89\xca\xab\xab\xc0\x00\x00\x00\x00\x7fgs\xfb\xa3\xe8ga\x89\xf6\xcf\x01\x03\xfd\x17\xdd[\x85J\x06\x18K\xe1\xa3\x84\xfaW\xa9\xa5\xe3\xb1\xc7.\xbc\xfc\xba4\x1bc\xc7\x93\xd2\xc3\x1d\xb0L?I\xc4p#\x82:\x98\x99\x89\xc8.\x1c\xfc\xad\xcd\xfb\xfcW\xd0,\xad\xa3W\x1e ;\xce1(1\xe0m\xcb\xfd\xf6\x18\x04"\x9c\x11\x1e\xb7\xffF\x1d\xe7\x9c "\xa78\r{]\xef\xd3\x11\xdc\x02$\x12$\x93\xf0\xee\xbe\x15\x81\xb9\xec\x1b\xed\xf7X\x83\xf7\xdc\x1d,K\x11\x8cd\xe8E\xd2\x84\xa3@P\x13d\xddB\xd0\xf0:O1\xcc\x00\x05\x01hL\x14\x84aXN\x16\x04A@V\x17\x85!`*\r\x84\xa1H}\x1d\xf4\x05\xc3\x1fy\x97\xd3O\xb0\xa20\xc3\x03\x84\\x0f2 \x0cH\xafD\xee\x80\x9b\x98\x90\x1e\x8al\x11\x9a\xc0\xfcP\xd3\x00\x16E0\x00N\xd4\xb9\xa2\x00\x00\x00\x00\x00\x00\x00\x00\x00')

[pondersome]

Are you possibly being "sandboxed"? when using rtk2go for extended durations, the default server communication needs to be <=1hz or your connection will get "sandboxed" and you'll start receiving garbage correction data and ultimately be unable to connect. Search the error output for "sandbox" to see if this is the case.

I have a fork that allows you to configure the ntrip communications rate with a parameter here:
https://github.com/pondersome/RTK_GPS_NTRIP/tree/ntrip-rate-control

[CompuMakEngineering]

Thanks for your response, I am using ROS 1, do you have another branch with the fix for ROS1

Also I didn't find any error message containing the sandbox response,

here is some more details, if I use ROS 1 branch, the first error I got is
----------------<<<<<<<<<<<<<<<<
process[ntrip_client-2]: started with pid [13324]
Traceback (most recent call last):
File "/home/cmakav/catkin_ws/src/ntrip_client_org/scripts/ntrip_ros.py", line 20, in
if importlib.util.find_spec(_MAVROS_MSGS_NAME) is not None:
AttributeError: 'module' object has no attribute 'util'
-------------------->>>>>>>>>>>>>>

When I investigated further, I found that the util is not supported by python2, unfoutuantly I am using ROS 1 melodic which support python 2 only,
so I removed the util and hardcoded the message type to RTCM messages
----------------------------<<<<<<<<<<<<<
#if importlib.util.find_spec(_MAVROS_MSGS_NAME) is not None:
#have_mavros_msgs = True
#from mavros_msgs.msg import RTCM as mavros_msgs_RTCM
#if importlib.util.find_spec(_RTCM_MSGS_NAME) is not None:
have_rtcm_msgs = True
from rtcm_msgs.msg import Message as rtcm_msgs_RTCM
------------------------------>>>>>>>>>>>>

Now I am able to connect to the Rtk2go and get some responses, however the start byte didn't match (_RTCM_3_2_PREAMBLE = 0b11010011)

---

Now the raw data received is as below

'Data received ', '\xd3\x00\xa5>\xc0\x00]'+\x02\xa0\x1d\x0b\x08G\xfc\xea\xdf\xd3\xa6\x00

While when I look into the buffer I see the below , which contain some non hex value character , this is what makes me think that something is wrong , I was expecting to see all elements of the buffer contains hex numbers

'inside parser ------buffer len is ', 1154)
('inside parser while loop -- element number:', 0, ' is -> ', '\xd3')
('inside parser while loop -- element number:', 1, ' is -> ', '\x00')
('inside parser while loop -- element number:', 2, ' is -> ', '\xa5')
('inside parser while loop -- element number:', 3, ' is -> ', '>') <<<<<<<< is this correct ?
('inside parser while loop -- element number:', 4, ' is -> ', '\xc0')
('inside parser while loop -- element number:', 5, ' is -> ', '\x00')
('inside parser while loop -- element number:', 6, ' is -> ', ']') <<<<<<<<<<< is this correct?
('inside parser while loop -- element number:', 7, ' is -> ', "'") <<<<<<<<
('inside parser while loop -- element number:', 8, ' is -> ', '+') <<<<<<<<<
('inside parser while loop -- element number:', 9, ' is -> ', '\x02')
('inside parser while loop -- element number:', 10, ' is -> ', '\xa0')
('inside parser while loop -- element number:', 11, ' is -> ', '\x1d')
('inside parser while loop -- element number:', 12, ' is -> ', '\x0b')
('inside parser while loop -- element number:', 13, ' is -> ', '\x08')
('inside parser while loop -- element number:', 14, ' is -> ', 'G') <<<<<<<<<<<,

Thanks for your help

[CompuMakEngineering]

One more thing to add

If I change the _RTCM_3_2_PREAMBLE = '\xd3'

I was able to match the first byte successfully , however everything broke downstream

---

[DEBUG] [1729773627.778946]: Read 1200 bytes
('inside parser ------buffer len is ', 1200)
('inside parser while loop -- element number:', 0, ' is -> ', '\xd3')
inside parser after PREAMBLE ------
('message len ', '\x00')
('message len ', 0)
('message len ', 0, ' ', 165, ' ', 1023)
Exception in thread Thread-6:
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
self.run()
File "/opt/ros/melodic/lib/python2.7/dist-packages/rospy/timer.py", line 237, in run
self._callback(TimerEvent(last_expected, last_real, current_expected, current_real, last_duration))
File "/home/cmakav/catkin_ws/src/ntrip_client_mod/scripts/ntrip_ros.py", line 148, in publish_rtcm
print("response received Before",self._client.recv_rtcm())
File "/home/cmakav/catkin_ws/src/ntrip_client_mod/src/ntrip_client/ntrip_client.py", line 288, in recv_rtcm
return self.rtcm_parser.parse(data) if data else []
File "/home/cmakav/catkin_ws/src/ntrip_client_mod/src/ntrip_client/rtcm_parser.py", line 86, in parse
message_length = (combined_buffer[index + 1] << 8 | combined_buffer[index + 2]) & 0x03FF
TypeError: unsupported operand type(s) for <<: 'str' and 'int'

---

[pondersome]

I'm sorry but I missed that you were on ROS1. My fix only applies to the ROS2 branch. I don't have a ROS1 fix or guidance about how to backport my fix.

To be clear, my fix is specifically about resolving the triggering of sandbox mode where the rtcm data gets scrambled at rtk2go. Scrambled data would likely cause a parser fail. So that's why I suggested you might be getting sandboxed. If this is the case, nothing you do in the parser will make any difference. If this is not the case, then none of this applies to your issue.

Note that the ros2 branch I started with does not reveal the http response that shows the sandbox message:

200 Ok, its now sandbox time

To expose the http response you'd want to add something like:

self._logwarn(response) # show the response

in ntrip_client.py. at line 142

otherwise a packet sniffer could reveal the response code

If you are being sandboxed, then my branch might offer you hints on fixing the ROS1 implementation:

https://github.com/pondersome/ntrip_client/tree/feature/ros2_ntrip_rate_control

[CompuMakEngineering]

Thanks for the response

Does anyone tested/used the code from ROS 1 branch ?
Does the server response I posted earlier looks correct or invalid? Can you please try to use with ROS II and let me know if the server response looks the same

If RTK2go is not the best service, Do you have other recommendations for RTK correction servers ?

My end goal is to be able to apply the rtk corrections on the latitude and longitudinal, do you have any recommendations how to do that?

Thanks
Shico