VE.Direct: irregular and long periods without update #411
-
|
I've been using openDTU-OnBattery for three months now, and I've consistently encountered an issue with the Victron SmartSolar 100/30. It seems that the data sent by the Victron system is quite irregular. While the connected HM-600 inverter dutifully transmits data every 5 seconds, the data from the Victron system can have an age ranging from 3 to 600 seconds. Although there was some improvement after an update around June, the update interval remains unpredictable. I rely on the reported data to update inverter settings through my custom script, rather than using openDTU's built-in dynamic power limiter. However, with such lengthy update periods, it's becoming challenging to effectively manage the battery's charging and discharging cycles. I'm curious if others are facing a similar problem. If you are, I'd appreciate any insights into potential solutions. It's occurred to me that the irregularity might be linked to the binary data that the Victron charger intermittently sends amidst the text-based messages. Could it be that the parser needs some adjustments to handle these interruptions? |
Beta Was this translation helpful? Give feedback.
Replies: 9 comments 8 replies
-
|
Hi @cnitschkowski, which release are you using? A period of 3 to 600 seconds looks really strange. Normally it should be around 1 second (this is the frequency of the mppt). You are mention some binary data. We don't use the hex protocol, so there should no binary data on the TX pin. Ensure that your settings in the Victron App looks like:
Logs would be good. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the reply. I'll check the settings when I'm home but I'm quite sure it is already set to this setting. I'm using a build from commit ec67fe1 Where can I get logs from? The console in the web ui does not show any messages regarding VE.direct, only the debug statements from the hoymiles communication. |
Beta Was this translation helpful? Give feedback.
-
Please elaborate. What binary data? How did you "see" that data? What's your hardware setup? How long are the cables between the charge controller and the OpenDTU-OnBattery device? Are you using a level shifter or galvanic isolator like ADUM1201?
That is a good question. The VeDirectFrameHandler is not very talkative. It does not print anything on the web console at all, not even frame validity errors, which it does detect. Those only appear on the serial output. I argue that this component also needs a verbose logging switch, which when enabled dumps all data received through the serial interface, so we can at least make guesses about what goes wrong. I had a look at the implementation and I conclude that it is very well possible that the state machine that helps decode a VE.Direct message is not getting to the final stages and indicating that a whole frame was received, if the data is corrupt. The VeDirectFrameHandler needs some sort of timeout: If the state is not idle and the timeout expires, we cannot expect to decode a frame successfully and we should reset the state machine so it can try to decode the next frame correctly (rather than trying to decode the next frame while the state machine is trying to decode the last). That's all theory, though. |
Beta Was this translation helpful? Give feedback.
-
|
@cnitschkowski Please give this a go: https://github.com/schlimmchen/OpenDTU-OnBattery/releases/tag/2023.08.30-vedirect-debugging I bet we see something interesting in the web console (or the serial console/interface). Inspect the three latest commits on the respective branch to see what the linked release includes. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for providing me with this build. I had a look at the debug console and it's flooded with messages. I cut out messages regarding the communication with the inverter, so only messages starting with [VE.Direct] are left: As you can see there's a lot going on in just about one second. There are timeouts, buffer overruns and invalid checksums. Regarding the "binary data": I did not "see" any binary data as I didn't debug the software but if I call correctly, the Victron devices may send the same data as text and in a binary encoded format, interrupting the text transmission. Skip that thought if I didn't understand that properly. The Victron SmartSolar 100/30 is connected using approximately 30cm long cables over a logic level adapter. |
Beta Was this translation helpful? Give feedback.
-
As far as I can say, there is no hex mode data in the stream. I searched for "Checksum\t" in the dump:
Looks like there is data missing. No idea why. Loose or bad connected cable? |
Beta Was this translation helpful? Give feedback.
-
|
@schlimmchen I saw that you added a "reset state machine" in your debug code. This was called in the log. Still thinking, that no data is recieved for whatever reason. If we assume that the mppt is sending its data correctly and the esp32 is fast enough to catch the data (mine is), then there is a data lost on the connection (30cm does this might change the voltage on the level shifter?). |
Beta Was this translation helpful? Give feedback.
-
|
@cnitschkowski Thanks for trying the test-firmware!
Well, the debug buffer overrun should really not occur. I guess I forgot to reset the index in some cases. And no, the checksum is calculated in-flight, i.e., only based on the single bytes that are processed. The debug buffer and the overrun has nothing to do with it. Something about the timing isn't right... Unfortunately, I don't know if the timestamps from the web console can be trusted. A reset of the state machine should only happen after 500ms of inactivity on the serial line. That's clearly not the case before the state machine reset occurs. @helgeerbe You are right, thanks for looking for the "CHECKSUM" fields. Why is there such a small frame at all? Maybe the output cannot be trusted because I did not take enough care to set the debug buffer index...
I am really not sure if this can explain the behavior... If the baud rate would not match or if there are many random bitflips, possibly due to a defective level shifter or a missing GND connection, then I would expect to never find a large string such as "CHECKSUM" to be transmitted successfully at all. I will double-check my code and provide a new build so we test further. In the end, the most probable cause in indeed a wiring issue (interference, missing GND, defevtive level shifter), but I would still like to use this chance to prove that the code can be more robust. As I wrote above: If some data is corrupted, the state machine will currently not recover for a long time and only by chance, which is very undesirable. |
Beta Was this translation helpful? Give feedback.
-
|
After some more investigation I am convinced that the timestamps are unusable. Also, I have a new most probable theory that might explain the behavior: Some other non-random signal is interfering with the RX pin of the ESP32.
|
Beta Was this translation helpful? Give feedback.
-
|
I'll attach my pin mappings file. |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, sorry for capturing the thread but I have a similar problem. In my case there are no errors in the log but the communication to the MPPT isn't working continually either. It looks like that in the live view:
I already tried a different ESP32, another ADUM1201 and even connected the VE.Direct directly to the ESP32 with nothing else connected but RX and GND from Victron. I also removed all unused entrys in the pin_mapping.json: The behavior keeps exactly the same: working a few seconds, not working a few seconds, working a few seconds, and so on... What also makes me curios: why is there no firmware version transmitted by my MPPT? Is the communication behavior of the "big" MPPTs maybe a little bit different to the smaller ones (I'm using the SmartSolar MPPT 250/100)? I'm using the latest version of openDTU onBattery 2023.09.13 (4e489fe) and with the activated VE.Direct log the console output looks like this: Any ideas? |
Beta Was this translation helpful? Give feedback.
-
|
Only the messages starting with 0x0d 0x0a are text-encoded VE.Direct messages, and only those are supported by OpenDTU-OnBattery. In your case, they are only transmitted every three seconds, with intermittend HEX-encoded messages in between. I guess that's why the data age is reset every second, even though the data is invalid in between... That's a guess. Let me have another look at the code at some time. For me, there are also HEX-encoded messages, but they are blasted out right before a text-encoded message, and the text-encoded messages appear every second. You might want to try a firmware update on that MPPT (is that possible?). Or maybe don't, because it would be nice to fix this 😉 There seems to be valid text-encoded info in your trace. Did you make sure to not remove related VE.Direct messages (like "invalid checksum" or "complete frame received") from the log by accident? |
Beta Was this translation helpful? Give feedback.
-
|
Hi there, Since I am going to control everything externally in ioBroker I now just used this ESPHOME component to get the data out of the MPPT. This is working very well and data gets updated every second. |
Beta Was this translation helpful? Give feedback.
-
No. Once the device is running, the web console should contain the same info as the serial output. I think you found a bug, but I have to idea what it could be and unfortunately, due to my time being very limited, I'll have to let this one go. Sorry... What you could do is wait for the next release, which will include #452, which in turn adds more debug messages, i.e., a message for each field that is decoded. That might give us an idea. |
Beta Was this translation helpful? Give feedback.
-
|
Just installed the latest version 2023.10.04 (4324ae3). This is what the log looks like now: |
Beta Was this translation helpful? Give feedback.
-
|
I finally managed to upgrade my setup by using a PCB instead of loose wiring to connect the Victron MPPT and NRF-module to the ESP. After some initial struggles the problem described (#411 (comment)) went away. The update period is usually one second, sometimes up to two seconds. Therefore I guess @schlimmchen was right (#411 (comment)) and the problem was caused by a bad connection between ESP32 and Victron charger. Thanks for your help! |
Beta Was this translation helpful? Give feedback.
-
|
Thanks @cnitschkowski for getting back to this. Glad you solved the issue. |
Beta Was this translation helpful? Give feedback.
Please elaborate. What binary data? How did you "see" that data?
What's your hardware setup? How long are the cables between the charge controller and the OpenDTU-OnBattery device? Are you using a level shifter or galvanic isolator like ADUM1201?
That is a good question. The VeDirectFrameHandler is not very talkative. It does not print anything on the web console at all, not even frame validity errors, which it does detect. Those only appear on the serial output. I argue that this component also needs a ve…