[core] Switch DateTimeType to Instant internally for consistent time-zone handling

[jlaur]

Proposal for switching DateTimeType to use Instant internally rather than ZonedDateTime.

This will effectively discard time-zone information from the source of the DateTimeType and leave it to consumers to apply time-zone at the moment of presentation, according to the configured time-zone in openHAB regional settings or in the browser.

Configured time-zone is now applied for:

• REST API
• SSE item state events
• Item UI registry (affecting e.g. sitemaps)

Consequential changes because DateTimeType is now "zone-less":

• Parameter timezone for TimestampOffsetProfile has been removed. Remove time-zone parameter from profile timestamp-offset openhab-docs#2403 has been prepared.
• Methods toZone, toLocaleZone and getZonedDateTime have been marked as deprecated and all usages have been refactored. I would suggest to remove them in 5.0 (or maybe even before when all bindings are refactored, see Simplify DateTimeType handling openhab-addons#17725).
• Method getZonedDateTime(ZoneId) has been introduced for convenience.

The following demonstrates the difference between old and new behavior.

REST API calls are for http://localhost:8080/rest/items/DateTimeTest and the returned state is shown in the comparison table:

{
    "link": "http://localhost:8080/rest/items/DateTimeTest",
    "state": "2024-10-31T21:00:00.000+0100",
    "stateDescription": {
        "pattern": "%1$tY-%1$tm-%1$td %1$tH:%1$tM:%1$tS",
        "readOnly": false,
        "options": []
    },
    "editable": true,
    "type": "DateTime",
    "name": "DateTimeTest",
    "label": "",
    "category": "",
    "tags": [],
    "groupNames": []
}

Updates are made in Karaf using commands like:

openhab:update DateTimeTest 2024-10-31T21:00:00+01:00

Step	Before	Now
System time-zone: Europe/Copenhagen
Set openHAB time-zone: Europe/Copenhagen
Update to: 2024-10-01T20:00:00Z
Observe REST API result	2024-10-01T20:00:00.000+0000	2024-10-01T22:00:00.000+0200
Observe /settings/items/	2024-10-01T20:00:00.000+0000	2024-10-01T22:00:00.000+0200
Observe /settings/items/DateTimeTest	2024-10-01 22:00:00	2024-10-01 22:00:00
Observe sitemap	2024-10-01 22:00:00	2024-10-01 22:00:00
Update to: 2024-10-31T20:00:00Z
Observe REST API result	2024-10-31T20:00:00.000+0000	2024-10-31T21:00:00.000+0100
Observe /settings/items/	2024-10-31T20:00:00.000+0000	2024-10-31T21:00:00.000+0100
Observe /settings/items/DateTimeTest	2024-10-31 21:00:00	2024-10-31 21:00:00
Observe sitemap	2024-10-31 21:00:00	2024-10-31 21:00:00
Update to: 2024-10-31T21:00:00+01:00
Observe REST API result	2024-10-31T21:00:00.000+0100	2024-10-31T21:00:00.000+0100
Observe /settings/items/	2024-10-31T21:00:00.000+0100	2024-10-31T21:00:00.000+0100
Observe /settings/items/DateTimeTest	2024-10-31 21:00:00	2024-10-31 21:00:00
Observe sitemap	2024-10-31 21:00:00	2024-10-31 21:00:00
Set time-zone: US/Alaska
Observe REST API result	2024-10-31T21:00:00.000+0100	2024-10-31T12:00:00.000-0800
Observe /settings/items/	2024-10-31T21:00:00.000+0100	2024-10-31T12:00:00.000-0800
Observe /settings/items/DateTimeTest	2024-10-31 21:00:00	2024-10-31 12:00:00
Observe sitemap	2024-10-31 21:00:00	2024-10-31 12:00:00
Update to: 2024-10-31T12:00:00-0800
Observe REST API result	2024-10-31T12:00:00.000-0800	2024-10-31T12:00:00.000-0800
Observe /settings/items/	2024-10-31T12:00:00.000-0800	2024-10-31T12:00:00.000-0800
Observe /settings/items/DateTimeTest	2024-10-31 21:00:00	2024-10-31 12:00:00
Observe sitemap	2024-10-31 21:00:00	2024-10-31 12:00:00

As it can be seen from the table, we now have consistent results.

Example DSL rule:

var dt = ZonedDateTime.of(2024, 4, 26, 0, 0, 0, 0, ZoneId.systemDefault())
DateTimeTest1.postUpdate(new DateTimeType(dt))
DateTimeTest2.postUpdate(new DateTimeType(dt.toInstant().atZone(ZoneId.of("UTC"))))
DateTimeTest3.postUpdate(new DateTimeType(dt.toInstant()))

Previous result with system time-zone CET/DST:

[openhab.event.ItemStateChangedEvent ] - Item 'DateTimeTest1' changed from NULL to 2024-04-26T00:00:00.000+0200
[openhab.event.ItemStateChangedEvent ] - Item 'DateTimeTest2' changed from NULL to 2024-04-25T22:00:00.000+0000

New result with same system time-zone:

[openhab.event.ItemStateChangedEvent ] - Item 'DateTimeTest1' changed from NULL to 2024-04-26T00:00:00.000+0200
[openhab.event.ItemStateChangedEvent ] - Item 'DateTimeTest2' changed from NULL to 2024-04-26T00:00:00.000+0200
[openhab.event.ItemStateChangedEvent ] - Item 'DateTimeTest3' changed from NULL to 2024-04-26T00:00:00.000+0200

Summary of benefits for developers:

• In cases where TimeZoneProvider is used only to generate a DateTimeType with the correct time-zone, this can now be completely removed. This simplifies the code. See Simplify DateTimeType handling openhab-addons#17725 for examples.
• The default DateTimeType constructor can now be used (without losing correct time-zone).
• In cases where the configured time-zone was not correctly applied, this will now work properly without any changes.
• The change is fully backwards compatible, although two methods have been deprecated: toZone and toLocaleZone. I will go through usages and remove them, since they no longer make sense. See Simplify DateTimeType handling openhab-addons#17725.
• Pull request reviews should be a bit smoother, since add-on maintainers will no longer have to assist contributors in constructing a proper DateTimeType using TimeZoneProvider.

And for users:

• There will now be consistent behavior across all bindings, where previously time-zone could differ between bindings and even within same binding: System time-zone or openHAB time-zone.
• There will now be consistent behavior across different parts of the UI, where previously some parts used the time-zone contained in the DateTimeType (which could be either depending on binding) and other parts used system time-zone.
• A change in time-zone will have immediate effect for the UI without requiring state updates for the items.

And finally limitations:

• It is no longer possible to send a DateTimeType command to a binding with a specified time-zone. The bindings will have to provide a time-zone if needed (e.g. time-zone from device, openHAB configuration or system time-zone). I have not found any bindings using this possibility, so I don't think this will cause any issues.

Resolves #2898

[jlaur]

I hit a wall with this when realizing that toFullString() and toString() should probably format as UTC for consistency. However, I believe one of these is used by Main UI for display purposes, so to improve the display with correct time-zone rather breaking it by raw UTC format, we would need UI changes - see also #2898 (comment).

I don't know if any @openhab/webui-maintainers can throw some light at what may be a possible way forward?

[jlaur]

@mherwege - seeing your work in #4169 I'm wondering if you can advise how to accomplish decoupling the time-zone + formatting of DateTimeType. The general idea is to reduce it to be only an Instant without time-zone information. Therefore the UI should no longer display the raw output of toString() / toFullString(). Instead it must use the configured time-zone from TimeZoneProvider for formatting the DateTime in local time.

Currently it's formatted like this: 2024-04-10T20:55:01.000+0200. The problem is that "+0200" is the time-zone provided when updating the value, not the current time-zone. I don't know how this formatting works in Main UI.

[mherwege]

@jlaur Main UI works with what it gets from the REST API.
On the Items screen (items list), it will show the state exactly as it is received from the REST API /items endpoint.
Here is an extract of such a response:

    {
        "link": "http://openhabian.local:8080/rest/items/PMD",
        "state": "2024-04-11T00:00:00.000+0200",
        "stateDescription": {
            "pattern": "%1$td-%1$tm-%1$tY",
            "readOnly": true,
            "options": []
        },
        "editable": true,
        "type": "DateTime",
        "name": "PMD",
        "label": "PMD",
        "category": "calendar",
        "tags": [],
        "groupNames": [
            "Afval"
        ]
    }

state is what gets shown in the items list.

The formatted state comes from the states event stream in the displayState field. It is a result of applying the stateDescription pattern, but that is done in core.
Event stream extract:

data: {"Afval":{"state":"2024-04-10T00:00:00.000+0200","displayState":"10-04-2024","type":"DateTime"},"Papier":{"state":"2024-04-10T00:00:00.000+0200","displayState":"10-04-2024","type":"DateTime"},"Restafval":{"state":"2024-04-18T00:00:00.000+0200","displayState":"18-04-2024","type":"DateTime"},"PMD":{"state":"2024-04-11T00:00:00.000+0200","displayState":"11-04-2024","type":"DateTime"}}

displayState gets used in widgets when available.

So, to understand where state and displayState come from, one needs to look at the REST API code. According to

openhab-core/bundles/org.openhab.core.io.rest.core/src/main/java/org/openhab/core/io/rest/core/internal/item/ItemResource.java

Line 953 in cbb458e

private Collection<Item> getItems(@Nullable String type, @Nullable String tags) {

state is added through the EnrichedItemDTOMapper here:

openhab-core/bundles/org.openhab.core.io.rest.core/src/main/java/org/openhab/core/io/rest/core/item/EnrichedItemDTOMapper.java

Line 81 in cbb458e

String state = item.getState().toFullString();

So it is simply using the toFullString() method on the state of the item.

[jlaur]

@mherwege - many thanks for your insights. I will have a closer look at this soon.

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/use-instant-in-historicitem-gettimestamp-instead-of-zoneddatetime/158302/3

[jlaur]

@openhab/core-maintainers - I don't have permission to remove the "work in progress" label since I'm not in the contributors team, so perhaps someone can do that for me? I consider this PR ready for review now after finally fixing the returned state in the REST API. In parallel I will run some more tests and also see if it is possible to use TimeZoneProvider in more places rather than converting to system time-zone.

@mherwege - perhaps you'd also like to have a look?

[lolodomo]

What about bindings setting a DateTimeType state with a particular time zone?
Is it really backward compatible with all bindings?

[jlaur]

What about bindings setting a DateTimeType state with a particular time zone?

They can still do that, but the time-zone will be stripped, so it's really not needed anymore.

Is it really backward compatible with all bindings?

It should be, but there can be unit tests failing if they expect a certain time-zone:

DateTimeType dt = new DateTimeType(ZonedDateTime.now());
ZonedDateTime zdt = dt.getZonedDateTime();

Previously this would return a ZonedDateTime with the same zone as provided when creating the object. But since the zone is now stripped, it will return a ZonedDateTime with system time-zone. Generally getInstant() should be preferred over getZonedDateTime() now.

[jlaur]

@joerg1985 - perhaps you would also like to have a look? I saw some of your recent related comments and enhancements, e.g. #4384.

[jlaur]

This will call DateTimeType.toFullString:

openhab-core/bundles/org.openhab.core/src/main/java/org/openhab/core/items/events/ItemEventFactory.java

Line 300 in 3bc9ae3

ItemEventPayloadBean bean = new ItemEventPayloadBean(getStateType(state), state.toFullString());

which is probably the reason for system time-zone being used in some places. I don't know how to improve this, since TimeZoneProvider is required for properly formatting this in the configured local time-zone. One solution might be to change the implementation of DateTimeType.toFullString to format in UTC as Instant. However, this would require all receivers to reformat by applying local time-zone from TimeZoneProvider, so I think it would be a bigger change. Any suggestions will be much appreciated here.

Keeping it "as is", in current version of the code I have provided, will be correct in all cases where system time-zone is the same as the time-zone configured in openHAB. When they are different and a state is provided using the correct time-zone from TimeZoneProvider (i.e. from bindings correctly implementing this), it might have been correctly displayed in local time-zone previously, but will now be in system time-zone. However, I have not been able to reproduce this. As it can be seen in the PR description table, even providing a different time-zone when updating items, resulted in system time-zone being used for formatting in UI. So perhaps "as is" keeps status quo.

[jlaur]

@J-N-K - thanks for picking this up for review. I have tried to address all your review comments. I have force-pushed after rebasing towards main, but all today's changes are in the last two commits.

[kaikreuzer]

Many thanks!

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/the-method-getzoneddatetime-from-the-type-datetimetype-is-deprecated/160763/5

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/datetime-operations/160794/6

[openhab-bot]

This pull request has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/solved-habpanel-sluggish/81756/219