Optimize ClickHouse uploader hot path

[jessejlt]

Summary

• cache per-column converters alongside each Schema so the inner ingest loop avoids type-switch churn
  reuse a preallocated []any row buffer and pipe batches through the new convertRecordInto helper to eliminate per-row allocations
• add BenchmarkProcessRecords to track hot-path regressions moving forward

Benchmarks

• BenchmarkProcessRecords-16: 35,069 ns → 28,393 ns (≈19% faster)
• Allocations: 640 → 512 per op (20% reduction)
• Memory: 15,360 B → 7,168 B per op (53% reduction)

Tests

go test ./ingestor/clickhouse
go test -bench=ProcessRecords -benchmem ./ingestor/clickhouse

[Copilot]

Pull Request Overview

This PR optimizes the ClickHouse uploader hot path by caching per-column converters alongside each Schema and reusing a preallocated row buffer to eliminate per-row allocations during the inner ingest loop.

Key changes:

• Cache value converters per Schema to avoid repeated type-switch overhead
• Use preallocated row buffer and convertRecordInto helper to reduce allocations
• Add benchmark test to track performance regressions

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
storage/store_test.go	Add cleanup to prevent resource leaks in test
ingestor/clickhouse/uploader_bench_test.go	New benchmark test for measuring hot path performance
ingestor/clickhouse/uploader.go	Optimize record processing with cached converters and buffer reuse
ingestor/clickhouse/schema.go	Add converter caching and new convertRecordInto function
cmd/collector/config/config_test.go	Fix test case interval value
cmd/collector/config/config.go	Simplify WAL flush interval validation

[Copilot]

This variable shadowing pattern is used to capture the loop variable for the closure. Consider using a more explicit pattern like columnName := column.Name and referencing it directly in the error message to make the intent clearer.

[Copilot]

The error message says 'must be greater than 0' but the condition allows 0 (only checks < 0). The message should be 'must be greater than or equal to 0' to match the actual validation logic.

Suggested change

	return errors.New("wal-flush-interval must be greater than 0")
	return errors.New("wal-flush-interval must be greater than or equal to 0")

[Copilot]

This test case change from 100 to 0 should include a comment explaining why zero is now a valid interval value, especially since this appears to be testing a boundary condition.

Suggested change

	{
	{
	// Zero is now considered a valid WAL flush interval value, meaning "no periodic flush".