Clarification: Why does time_series table allow duplicate (cluster_id, time, count_index, value) entries?

[minshao]

I’ve been looking into the current design of the time_series table. Right now, it allows multiple rows with the same combination of:

(cluster_id, time, count_index, value)

From the code, it seems like this behavior is expected, but I’d like to clarify the reasoning.

My concerns/questions

• Intuitively, I would expect (cluster_id, time, count_index, value) to be unique, with the count column being accumulated (or updated) instead of inserting duplicates.
• Allowing duplicates makes it harder to query and aggregate later on
• I’m not fully familiar with the shape and semantics of time_series data, so I might be missing the original design intent.

Discussion points

• Is there a strong reason why we need to allow duplicates with the same (cluster_id, time, count_index, value)?
• Would enforcing uniqueness at the DB level (and updating count) be more efficient or safer?
• Or, is the current design intentional to allow for multiple independent insertions that are only aggregated at query time?
• If so, should we document this explicitly, so future contributors know not to enforce uniqueness?

[sehkone]

@minshao, as you pointed out, (cluster_id, time, count_index, value) must be unique. I think the lack of such a restriction on the table might have been a mistake. Since duplicates wouldn’t be inserted unless the same raw data was clustered repeatedly—which virtually never happens—we might not have realized the mistake existed. I appreciate that you brought this up.

[minshao]

Thanks for the help! Appreciate it! I'll incorporate the restriction when I move the table to RocksDB.