Some insulin treatments not uploading to nightscout

[jk1993]

Hi,

For the last few months I've been using a NovoPen Echo Plus to record mealtime insulin and this has been syncing to nightscout which is great. However in the last couple of weeks some doses have not been uploading to nightscout. If I look in the event log in xdrip I can see the following error which must be related - "Exception uploading REST API treatments":

If I look in the nightscout docker logs I see this, which I think is related as it looks like it's happening in the "save" function in treatments.js:

Another reason I think it may be related is the thing that seems pretty consistent is that insulin doses under 10 units seem to upload fine, whereas any doses 10 or above seem to get excluded. I'm thinking maybe the value being 10 or above causes the data string being uploaded to nightscout to be too large or use too many characters or something.

Nightscout version: 15.0.2

xDrip version: 2024.11.17

Thanks

[Navid200]

Thanks for reporting this.

@tzachi-dar
What could be causing this?

[jk1993]

Any chance this could be looked into? I'm currently unable to analyse overall insulin usage as any doses of 10 units and above are missing from nightscout

[dbames]

Do you have the nightscout logs for the entry the failed to save? From a quick glance at the nightscout code, it looks to me that it's more likely it is the value of _id that would be the issue rather than the json payload in insulinInjections

[jk1993]

The second screenshot is from the nightscout logs, I couldn't see anything else in there at the time it tried to insert the treatment. I guess that makes sense looking at that uuid since it contains the number of units in it, so anything 10.0 or above would have an extra character compared to anything below, assuming this is what it uses to create the id on save.

The issue is still consistent in that all treatments of less than 10 units are fine and anything above 10 units doesn't upload to nightscout. I'm not sure why it's only started happening recently though. Any help would be greatly appreciated and happy to provide more detailed logs

[jk1993]

I've done some digging and had a look through my treatments data in nightscout to see exactly when this stopped working. It seems the treatments over 10 units stopped sending to nightscout on the 23rd November 2024. I had a look through my downloads folder on android to see when I last downloaded a new xdrip version and it looks like I downloaded version 20241117 on the 23rd November exactly. It seemed too much of a coincidence to be unrelated but I've tried installing the previous version and I'm still getting the same issue.

Is it possible to convert this post to an issue? I'd really really like to be able to have my insulin treatments syncing to nightscout again as it was incredibly useful to see the data over time.

Thanks

[Navid200]

No please. There is no reason to open an issue.
I just set up a test setup and entered 1 unit of insulin treatment in xDrip and it was uploaded to Nightscout.

Opening an issue does not increase the chance of something being fixed. But, opening too many issues will drown everything in an ocean of issues. That will make it worse for me when I need to see what I need to focus on next.

[alx-k]

Just wanted to add that I'm having exactly the same problem with exactly the same log output from xDrip and Nightscout.

[jk1993]

Thanks for confirming my issue! Just to add (not trying to complicate things) 9.5 units seems to upload fine so its any NovoPen NFC scan where units > 9.5 or units >= 10.0

[alx-k]

Good addition, I couldn't test that since my pen cannot do .5 increments.

[jk1993]

Makes sense. That also confirms that the issue persists across multiple NovoPen devices, rather than just my model (Echo Plus)

[Navid200]

I don't know what is going on with respect to novopen.

Frankly, xDrip does too much. No single person knows everything that xDrip can do. That makes it very hard to maintain. I personally don't record my insulin injections. So, I am not the right person to fix this. But, the problem is I am not sure anyone can fix it. I remember issues with some versions of the pen not working. I am not sure if we have stopped supporting the pens or not.

This is still labeled as a bug. So, it's not that we have forgotten about it.

[Crosus97]

I’m a sysadmin with basic development knowledge, and my partner has diabetes. We rely on Nightscout to sync NovoPen insulin injection data correctly. Since in your last message it seemed you were unsure where to go next, I decided to dig deeper and gather as much relevant information as possible so we can get this fixed.

I provided ChatGPT with the following:

• The NightscoutUploader.java file.
• The NightscoutTreatments.java file.
• Nightscout server logs from my instance.
• Test results from sending treatments to Nightscout via curl.
• Observations on how xDrip currently formats the insulinInjections field and how Nightscout logs it.

Based on this, ChatGPT helped me identify the likely cause and propose a patch that should make insulinInjections upload as a proper JSON array instead of a string, ensuring correct storage and processing in Nightscout.

PLEASE TAKE THIS AS A GUIDELINE TO HELP NARROW DOWN THE ISSUE, NOT SOMETHING TO BE FOLLOWED TO THE LETTER.

Summary
When xDrip uploads NovoPen treatments, the insulinInjections field is currently sent as a JSON string instead of a real JSON array. This makes Nightscout store it as plain text and can cause downstream processing issues. Sending a proper array fixes it.

Tests done
Against my Nightscout instance:

• With string → created, but stored as plain text.
• With array → created and parsed as an array; works as expected.

Root cause
In NightscoutUploader.java, the uploader places treatment.insulinJSON (a serialized JSON string) directly into the payload, instead of sending an actual array.

Proposed fix (xDrip upload)
Parse treatment.insulinJSON into a JSONArray and put that into the payload (so Nightscout receives a real array).

diff --git a/app/src/main/java/com/eveningoutpost/dexdrip/utilitymodels/NightscoutUploader.java b/app/src/main/java/com/eveningoutpost/dexdrip/utilitymodels/NightscoutUploader.java
@@ -XXXX,7 +XXXX,17 @@ private void populateV1APITreatmentEntry(org.json.JSONArray array, Treatments treatment) throws Exception {
-        if (treatment.insulinJSON != null) {
-            record.put("insulinInjections", treatment.insulinJSON);
-        }
+        if (treatment.insulinJSON != null) {
+            // Ensure insulinInjections is a real JSON array (not a quoted JSON string)
+            try {
+                record.put("insulinInjections", new org.json.JSONArray(treatment.insulinJSON));
+            } catch (org.json.JSONException e) {
+                // If it isn't valid JSON, omit the field rather than sending malformed data
+                // (or optionally wrap as an empty array)
+                // record.put("insulinInjections", new org.json.JSONArray());
+            }
+        }

Optional companion fix (download)
Make NightscoutTreatments.java tolerant of both array and string so round-trip remains consistent.

diff --git a/app/src/main/java/com/eveningoutpost/dexdrip/utilitymodels/NightscoutTreatments.java b/app/src/main/java/com/eveningoutpost/dexdrip/utilitymodels/NightscoutTreatments.java
@@ -XXXX,7 +XXXX,19 @@ public class NightscoutTreatments {
-            try {
-                injections = tr.getString("insulinInjections");
-            } catch (org.json.JSONException e) {
-                // ignore
-            }
+            try {
+                // Accept either a JSON array or a string
+                Object inj = tr.get("insulinInjections");
+                if (inj instanceof org.json.JSONArray) {
+                    injections = ((org.json.JSONArray) inj).toString();
+                } else if (inj instanceof String) {
+                    injections = (String) inj;
+                }
+            } catch (org.json.JSONException e) {
+                // ignore
+            }

Expected outcome
Nightscout receives insulinInjections as a real array, stores it as structured data, and downstream processing works correctly for NovoPen treatments.

---

I apologize if anyone feels uncomfortable or offended by the fact that I tried to resolve this issue with ChatGPT-5 (Plus user). My only intention is to help get this fixed so that NovoPen insulin data syncs properly for those who need it.