URL encode filters, error handling and consistency with HTTP Tool

[Eonfge]

When implementing certain filters, I ran into an inconsistency related to URL Encoding. Take these three queries as an example:

No results, datetime not url encoded

v1.1/Sensors?$filter=properties/serial-number eq '123456' and properties/date-modified eq '2025-04-17T14:33:30.831439663+02:00' and ( Datastreams/any(d: d/Thing/name eq 'fe07afec-6eda-53e1-aa44-e1e97845d7e9') or MultiDatastreams/any(d: d/Thing/name eq 'fe07afec-6eda-53e1-aa44-e1e97845d7e9') )

1 result, datetime url encoded

v1.1/Sensors?$filter=properties/serial-number eq '123456' and properties/date-modified eq '2025-04-17T14%3A33%3A30.831439663%2B02%3A00' and ( Datastreams/any(d: d/Thing/name eq 'fe07afec-6eda-53e1-aa44-e1e97845d7e9') or MultiDatastreams/any(d: d/Thing/name eq 'fe07afec-6eda-53e1-aa44-e1e97845d7e9') )

1 result, entire filter encoded

v1.1/Sensors?$filter=properties%2Fserial-number%20eq%20%27123456%27%20and%20properties%2Fdate-modified%20eq%20%272025-04-17T14%3A33%3A30.831439663%2B02%3A00%27%20and%20%28%20Datastreams%2Fany%28d%3A%20d%2FThing%2Fname%20eq%20%27fe07afec-6eda-53e1-aa44-e1e97845d7e9%27%29%20or%20MultiDatastreams%2Fany%28d%3A%20d%2FThing%2Fname%20eq%20%27fe07afec-6eda-53e1-aa44-e1e97845d7e9%27%29%20%29

This brings a simple discussion to mind, why not encode the entire filter object?

Background

• Docker image: fraunhoferiosb/frost-server:2.5
• Ubuntu 24.04

All in the manual

https://fraunhoferiosb.github.io/FROST-Server/sensorthingsapi/requestingData/STA-Filtering.html

The difference that encoding the datetime makes, is not obvious: Only with testing and deduction can one discover that the eq comparison fails due to formatting, instead of actual mismatch.

• Could there be error for such an edge case?

It is mentioned in the documentation that one should encode a datetime, but it's the only reference to URL-encoding. No other parameter has this mentioned.

• Would it not be better to encourage URL encoding at all times?
• Would that be something to add to the manual, or would it be better to change the logic in the (web-)client to account for this?

It also adds to the confusion since the Java client does encode the filter-parameters.

[hylkevds]

Good point. The HTTP tool is not really designed for GET requests. You can make those without any tools, by just putting the request in the URL bar of the browser. The tool can't simply URL-Encode the entire URL, since that would break the path part, so it would have to analyse the URL and only URL-Encode the query part. But only if it is not already URL-Encoded, and detecting that may be tricky.

The problem with the + sign is that it has a special meaning in URLs: it's the short-hand encoding for a blank space.
In your case you don't notice it, since you're putting your date-time in a string. There is no way to detect this, since a space is a valid character in a string. But date times don't need to be quoted, and when you don't, you'll get an error, since a date-time can't contain a space.

The query-part of a URL can always be fully URL-Encoded. The only disadvantage is that it makes the query string longer. One does need to be careful to not double-encode by accident, which is why the HTTP tool doesn't even try and assumes the user knows best.

The Java client can better deal with this problem, since it constructs the query from parts and doesn't have to deal with a full query provided by the user.