Skip to content

bigquery timestamp #51

@tiboun

Description

@tiboun

I create this issue in order to discuss about timestamp handling in bigquery. Even if output is similar to bigquery in unit tests, the java timestamp doesn't seems to be in phase between Bigquery's timestamp and duckdb's timestamp.

Bigquery java timestamp is:
image

Whereas duckdb's timestamp is:
image

We can notice that the timezone generated by the toString method of Timestamp is different.

If we format the timestamp as UTC:

![image](https://github.com/user-attachments/assets/1b877895-8bb8-42ee-8823-e1dafcaac0a2)

We get the expected output.

If we apply the same to DuckDB's output, we get:
image

Please check the example below:

SELECT GENERATE_TIMESTAMP_ARRAY(start_timestamp, end_timestamp, INTERVAL 1 HOUR)
           AS timestamp_array
FROM
    (SELECT
         TIMESTAMP '2016-10-05 00:00:00' AS start_timestamp,
         TIMESTAMP '2016-10-05 02:00:00' AS end_timestamp
     UNION ALL
     SELECT
         TIMESTAMP '2016-10-05 12:00:00' AS start_timestamp,
         TIMESTAMP '2016-10-05 14:00:00' AS end_timestamp
     UNION ALL
     SELECT
         TIMESTAMP '2016-10-05 23:59:00' AS start_timestamp,
         TIMESTAMP '2016-10-06 01:59:00' AS end_timestamp);

Obtained:

SELECT
	Generate_Series(start_timestamp::TIMESTAMP,
	end_timestamp::TIMESTAMP,
	INTERVAL 1 HOUR::INTERVAL) AS timestamp_array
FROM
	(
	SELECT
		TIMESTAMP '2016-10-05 00:00:00' AS start_timestamp,
		TIMESTAMP '2016-10-05 02:00:00' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMP '2016-10-05 12:00:00' AS start_timestamp,
		TIMESTAMP '2016-10-05 14:00:00' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMP '2016-10-05 23:59:00' AS start_timestamp,
		TIMESTAMP '2016-10-06 01:59:00' AS end_timestamp)

But expected one of :

SELECT
	Generate_Series(start_timestamp,
	end_timestamp,
	INTERVAL 1 HOUR::INTERVAL) AS timestamp_array
FROM
	(
	SELECT
		TIMESTAMPTZ '2016-10-05 00:00:00+00' AS start_timestamp,
		TIMESTAMPTZ '2016-10-05 02:00:00+00' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMPTZ '2016-10-05 12:00:00+00' AS start_timestamp,
		TIMESTAMPTZ '2016-10-05 14:00:00+00' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMPTZ '2016-10-05 23:59:00+00' AS start_timestamp,
		TIMESTAMPTZ '2016-10-06 01:59:00+00' AS end_timestamp);
SELECT
	Generate_Series(start_timestamp,
	end_timestamp,
	INTERVAL 1 HOUR::INTERVAL) AS timestamp_array
FROM
	(
	SELECT
		TIMESTAMP '2016-10-05 00:00:00' AT TIME ZONE 'UTC' AS start_timestamp,
		TIMESTAMP '2016-10-05 02:00:00' AT TIME ZONE 'UTC' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMP '2016-10-05 12:00:00' AT TIME ZONE 'UTC' AS start_timestamp,
		TIMESTAMP '2016-10-05 14:00:00' AT TIME ZONE 'UTC' AS end_timestamp
UNION ALL
	SELECT
		TIMESTAMP '2016-10-05 23:59:00' AT TIME ZONE 'UTC' AS start_timestamp,
		TIMESTAMP '2016-10-06 01:59:00' AT TIME ZONE 'UTC' AS end_timestamp)

This is because bigquery timestamp is with timezone and when not defined, it sets to UTC.
Getting epoch from timestamp results in a different output value when no timestamp is specified.

Serializing

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions