[jdbc] SQLite: Fix format when persisting item with timestamp

[lovery]

Description

Change the date and time format for time value which is inserted when persist method of an item is called.
Bellow is an example of a JSRule for testing the issue.

rules.JSRule({
    name: "Persist previous item state on change",
    triggers: triggers.ItemStateChangeTrigger("YOUR_ITEM_NAME"),
    execute: data => {
        oneSecondAgo = new Date(Date.now()); 
        oneSecondAgo.setSeconds(oneSecondAgo.getSeconds() - 1);
        items.getItem(data.itemName).persistence.persist(time.toZDT(oneSecondAgo), data.oldState);
    }
});

The default time format is '%Y-%m-%d %H:%M:%f' - check it here.
All other formats aren't supported by the HabPanel's graphics - the values don't appear on the graphic if the time is not in the upper format.

I tried one other solution, but since the format isn't specified anywhere, I didn't like it, because it could lead to the same problem as before:

         java.sql.Timestamp tsObj = new java.sql.Timestamp(date.toInstant().toEpochMilli());
         String timestamp = tsObj.toString();

Here is a link to a thread in OH community forum

Testing

Automatic built will be under the following folder:
https://openhab.jfrog.io/ui/native/libs-pullrequest-local/org/openhab/addons/bundles/

Here is also a link to a build that I made for OH 4.2 and 4.3

Correct existing data in SQL

In case someone wants to update their data in the SQLite, I am providing guidelines how to do that:

1. Create a copy of your database or at least of the tables you are going to edit

2. First you have to find the table in which the item is persisted in the SQLite with:
   SELECT ItemId, ItemName FROM items WHERE ItemName = 'XXXXXXXX';
   Check that you are getting the right id connected to your item.

3. Check that you have mixed time values: SELECT time, value FROM itemXXXX;
   Keep in mind that you may have to add leading zeros to the id you get, I had ItemId - 256 and I had to add one leading 0, your case could be different.

4. All my values started with 17, so I used this for in the where clause of my update statement like:
   UPDATE itemXXXX SET time = datetime(CAST(substr(time, 0, 11) as REAL), 'unixepoch', 'localtime') || '.' || substr(time, 11, 3) WHERE time LIKE '17%';

Fixes #18101

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/how-do-i-use-persistence-persit-in-javascript/161623/6

[jlaur]

@lovery - thank you for the fix, and sorry for the long wait! I'm trying to understand this issue. I've checked this and my understanding now is that we're actually storing a formatted string (please correct me if I'm wrong). So to understand the difference in behavior, we need to look at this:

openhab-addons/bundles/org.openhab.persistence.jdbc/src/main/java/org/openhab/persistence/jdbc/internal/db/JdbcSqliteDAO.java

Line 69 in 3b64968

sqlTypes.put("tablePrimaryValue", "strftime('%Y-%m-%d %H:%M:%f' , 'now' , 'localtime')");

vs.

openhab-addons/bundles/org.openhab.persistence.jdbc/src/main/java/org/openhab/persistence/jdbc/internal/db/JdbcSqliteDAO.java

Lines 134 to 135 in 3b64968

	java.sql.Timestamp timestamp = new java.sql.Timestamp(date.toInstant().toEpochMilli());
	Object[] params = { timestamp, storedVO.getValue() };

So as far as I can see, your fix will do the same formatting on Java side as done on SQL side for the doStoreItemValue overload without the ZonedDateTime parameter.

I'm wondering though if it would be safest to use the same strftime function in both cases? I'm worried about time-zone issues. The SQL version will convert a UTC timestamp to local time using the system time zone. I think your version will use the time zone provided in the ZonedDateTime? Can you try to run a small test setting your openHAB time zone in regional settings to a different time zone than your system time zone, and see if that will cause issues?

[lovery]

Hi @jlaur, you've understood is correctly, a formatted string is stored.
I've tested the current fix with one system time zone and other time zone selected in the OH settings and you are right the inserted time is NOT OK like this.

I re-wrote the code and used the strftime, but I had to use also the substr because the unixepoch modifier in the SQLite expects up to 10 signs, so it can't handle values with milliseconds.

[jlaur]

I re-wrote the code and used the strftime, but I had to use also the substr because the unixepoch modifier in the SQLite expects up to 10 signs, so it can't handle values with milliseconds.

Perhaps that's because it expects number of seconds rather than milliseconds?

The unixepoch() function returns a unix timestamp - the number of seconds since 1970-01-01 00:00:00 UTC.

Maybe you could use date.toInstant().getEpochSecond() directly as a query parameter for:

strftime('%Y-%m-%d %H:%M:%f', ?, 'unixepoch', 'localtime')

Do you think that would work?

[lovery]

Maybe you could use date.toInstant().getEpochSecond() directly as a query parameter for:

strftime('%Y-%m-%d %H:%M:%f', ?, 'unixepoch', 'localtime')

Do you think that would work?

@jlaur Changed to use getEpochSecond(), and also I added a method for getting the milliseconds, instead of using the substr.

[jlaur]

Changed to use getEpochSecond(), and also I added a method for getting the milliseconds, instead of using the substr.

Excellent, thanks for the simplification! We are almost there, just requesting one correction to comply with naming conventions (also reported by SAT). Please confirm that the new version is fully tested, and we are ready to merge. Thanks for your efforts!

[lovery]

@jlaur Fully tested the change, everything is good with it.

[jlaur]

LGTM