Delta calculations needs clean up

[sanderrenes]

After I found that my Deltas are displayed incorrectly (See issue #3688
I started prodding around in the code, with my little experience in Kotlin and less than zero in Android Studio, / gradle (still not sure what that does), this is challenging, but I did find some odd stuff in the calculations of the deltas in :

...\androidAPS\AndroidAPS\implementation\src\main\kotlin\app\aaps\implementation\iob\GlucoseStatusProviderImpl.kt

in particular the class starting at line 19 : class GlucoseStatusProviderImpl @Inject constructor(

I can roughly guess how this class developed over time, but right now the math does not appear to be mathing.
I have several fixes I would suggest (and could try to implement as soon as I understand how).

1. intermediate rounding in calculations of 5-min deltas
2. the boundary values in the if statements
3. the actual delta calculations
4. a possible source for the errors in issue #3688

1: the renormalization of the change in glucose levels to the 'change over 5 minutes'
Current implementation:

val now = data[0]
val then = data[i]
val nowDate = now.timestamp
val thenDate = then.timestamp
val minutesAgo = ((nowDate - thenDate) / (1000.0 * 60)).roundToLong()
              
// multiply by 5 to get the same units as delta, i.e. mg/dL/5m
change = now.recalculated - then.recalculated
val avgDel = change / minutesAgo * 5

This intermediate rounding of the minutesAgo causes unnecessary rounding errors.
Example: your glucose sensor reports a change of 0.2 (say 5 to 5.3 )
when the measures are 1:29 minutes apart--> AvgDelta= 0,2/15 =1
when the measures are 1:31 minutes apart--> AvgDelta= 0,2/25 = 0.5

2 seconds should not make this much of a difference.

suggested fix: make a normalization function that can be reused along these lines:

val now = data[0]
val then = data[i]
val nowDate = now.timestamp
val thenDate = then.timestamp
              
// time is in milliseconds, we want the average change over 5 minutes: 5*60*1000 milliseconds
//  to get the same units as delta, i.e. mg/dL/5m
change = now.recalculated - then.recalculated  // why do all the others have "val" in front, and this one does not?
val unitsTime5 = ((nowDate - thenDate) / (5*6*1000))
val avgDel = change / unitsTime5  

This should also help with newer sensors that give more measures per minute, as this calculation can run on any time frame without rounding.

2. the if statement boundary

if (2.5 < minutesAgo && minutesAgo < 17.5) {
...
}
} else if (17.5 < minutesAgo && minutesAgo < 42.5) {
..}

discards values with minutesAgo==17.5.
If we drop the rounding to whole minutes, this has to go too
A simple fix will do:

} else if (17.5 <= minutesAgo && minutesAgo < 42.5) {

3 the averageDelta is not equal to the average of the deltas

from the code:
the calculated avgDelta (see point 1) are binned by an if statement e.g.:

 } else if (17.5 < minutesAgo && minutesAgo < 42.5) {
                    longDeltas.add(avgDel)
                } 
...
longAvgDelta = average(longDeltas),

unfortunately, this is a problematic way of calculating a summary of change.
For a given glucose graph over time, the longAvgDelta and shortAvgDelta will differ if they are taken at different intervals (a measurement per minute, 2 minutes, or 5 minutes), or if values are missing in the longDeltas array.
I
think this should be the average 5-minute change over the last 17,5 minutes (shortAvgDelta) and the average 5 minutes change over the period from 17,5-42,5 minutes.

To calculate those, instead of making buckets of the avgDel, the buckets should contain the actual values:
data[i].recalculated

then the last value in the bucket <17,5 and the first value bucket 17,5<= are the glucose measures that are closest in time to 17,5. Take a weighted average between those two values to determine the best linear estimate of the glucose at 17,5.

Then, plug that value into the normalized 5-minute delta function of point 1 to get the desired delta.
Do the same for 42.5

4 a potential source of error 3688
the clearest error in 3688 are that the lower two values for the deltas are the same.
after the calculations, the last delta is set as such:

val delta = if (lastDeltas.isEmpty()) {
                shortAverageDelta
            } else {
                average(lastDeltas)
            }

if the lastDeltas.isEmpty() returns true, they should be the same.

which seems to suggest that the FSL2 readings somehow are not entered in the lastDeltas array
This if statement determines what goes into that array:

 if (data[i].value > 39 && !data[i].filledGap) {
..
}
 if (2.5 < minutesAgo && minutesAgo < 7.5) {
                            lastDeltas.add(avgDel)
                        }

So somehow that last if statement is not triggered, but the other if statements in the parent statement are.
I have no reason for it yet, might be timing of measurements, might be the !data[!].filledGap, might be something else all together

[sanderrenes]

Can someone who knows this code base a bit better than I do check that the suggested changes (1-3) do not break other things?
or tell me how to check?

any hints on 4?

[MilosKozak]

write simple tests, what you think it's correct and we can elaborate on this

[MilosKozak]

I created test example here ed2e36e

[sanderrenes]

I took a first stab at both some tests and a class that implements them.
Since I do not know Kotlin syntax, I asked Claude Sonnet to help me on that end. Unfortunately, it could not tell me how to run the tests on the class; it seems to have an incorrect idea of some bash/gradlew stuff; I hope this was not the case for the Kotlin code.

I added them to a fork of AAPS in my own account, since that seemed the easiest way to share them.

Class:
DeltaCalculator
tests:
DeltaCalculatorTest

[MilosKozak]

did you found where is the bug?

[sanderrenes]

I think i found the bug in delta 5 and 15 being zero or allways the same. See other issue. But regardless of the issues with fsl2 sensors and the delta, the implementation of the deltas is inconsistent between the different deltas and gives incorrect answers whenever there is more than one measurement in a bucket. When fsl3 reports per minute, this is going to be a problem Note to self, add tests with different time intervals Op do 10 apr 2025 16:32 schreef Milos Kozak ***@***.***>:

…

did you found where is the bug? — Reply to this email directly, view it on GitHub <#3933 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHC7OJCSA32UP3F2H2HFWPL2YZ6GJAVCNFSM6AAAAAB2Q2WBOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJTHAYDKNRQHA> . You are receiving this because you authored the thread.Message ID: ***@***.***> *MilosKozak* left a comment (nightscout/AndroidAPS#3933) <#3933 (comment)> did you found where is the bug? — Reply to this email directly, view it on GitHub <#3933 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AHC7OJCSA32UP3F2H2HFWPL2YZ6GJAVCNFSM6AAAAAB2Q2WBOOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOOJTHAYDKNRQHA> . You are receiving this because you authored the thread.Message ID: ***@***.***>