DRAFT: Update arrow/parquet to 56.0.0 #16690

alamb · 2025-07-05T11:51:36Z

Which issue does this PR close?

Related to Release arrow-rs / parquet Major version 56.0.0 (July 2025) arrow-rs#7395

Rationale for this change

There are several non trivial changes in arrow 56 so I want to start testing soon

Also, I would like a stable base to test new parquet pushdown code from @XiangpengHao

Parquet filter pushdown v4 arrow-rs#7850

What changes are included in this PR?

Update to use pre-release version of arrow
Update tests / APIs as necessary

Are these changes tested?

Are there any user-facing changes?

alamb · 2025-07-05T13:19:16Z

I see some failures in the row_group_pruning tests

$ cargo test --package datafusion --test parquet_config parquet::row_group_pruning

failures:
    parquet::row_group_pruning::prune_binary_eq_match
    parquet::row_group_pruning::prune_binary_lt
    parquet::row_group_pruning::prune_binary_neq
    parquet::row_group_pruning::prune_string_eq_match
    parquet::row_group_pruning::prune_string_lt
    parquet::row_group_pruning::prune_string_neq

~~I am tracking it down~~

Update it is: Incorrect inlined string view comparison after " Add prefix compare for inlined" arrow-rs#7874

Here is an example failure

thread 'parquet::row_group_pruning::prune_binary_eq_match' panicked at datafusion/core/tests/parquet/row_group_pruning.rs:146:9:
assertion `left == right` failed: Expected 2 rows, got 1: Input:
+---------------+----------------+------------------------------+-------------------+------------------------------+
| name          | service_string | service_binary               | service_fixedsize | service_large_binary         |
+---------------+----------------+------------------------------+-------------------+------------------------------+
| all frontends | frontend one   | 66726f6e74656e64206f6e65     | 666531            | 66726f6e74656e64206f6e65     |
| all frontends | frontend two   | 66726f6e74656e642074776f     | 666532            | 66726f6e74656e642074776f     |
| all frontends | frontend three | 66726f6e74656e64207468726565 | 666533            | 66726f6e74656e64207468726565 |
| all frontends | frontend seven | 66726f6e74656e6420736576656e | 666537            | 66726f6e74656e6420736576656e |
| all frontends | frontend five  | 66726f6e74656e642066697665   | 666535            | 66726f6e74656e642066697665   |
| mixed         | frontend six   | 66726f6e74656e6420736978     | 666536            | 66726f6e74656e6420736978     |
| mixed         | frontend four  | 66726f6e74656e6420666f7572   | 666534            | 66726f6e74656e6420666f7572   |
| mixed         | backend one    | 6261636b656e64206f6e65       | 626531            | 6261636b656e64206f6e65       |
| mixed         | backend two    | 6261636b656e642074776f       | 626532            | 6261636b656e642074776f       |
| mixed         | backend three  | 6261636b656e64207468726565   | 626533            | 6261636b656e64207468726565   |
| all backends  | backend four   | 6261636b656e6420666f7572     | 626534            | 6261636b656e6420666f7572     |
| all backends  | backend five   | 6261636b656e642066697665     | 626535            | 6261636b656e642066697665     |
| all backends  | backend six    | 6261636b656e6420736978       | 626536            | 6261636b656e6420736978       |
| all backends  | backend seven  | 6261636b656e6420736576656e   | 626537            | 6261636b656e6420736576656e   |
| all backends  | backend eight  | 6261636b656e64206569676874   | 626538            | 6261636b656e64206569676874   |
+---------------+----------------+------------------------------+-------------------+------------------------------+
Query:
SELECT name, service_binary FROM t WHERE service_binary = CAST('backend one' AS bytea)
Output:
+-------+------------------------+
| name  | service_binary         |
+-------+------------------------+
| mixed | 6261636b656e64206f6e65 |
| mixed | 6261636b656e642074776f |
+-------+------------------------+
Metrics:
time_elapsed_opening{partition=0}=51.348875ms, time_elapsed_scanning_until_data{partition=0}=179.834µs, time_elapsed_scanning_total{partition=0}=223.75µs, time_elapsed_processing{partition=0}=46.345583ms, file_open_errors{partition=0}=0, file_scan_errors{partition=0}=0, start_timestamp{partition=0}=2025-07-05 14:32:15.489561 UTC, end_timestamp{partition=0}=2025-07-05 14:32:15.541149 UTC, elapsed_compute{partition=0}=NOT RECORDED, output_rows{partition=0}=5, predicate_evaluation_errors{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, row_groups_pruned_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, row_groups_matched_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=2, row_groups_pruned_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=1, bytes_scanned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_pushdown_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, statistics_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=59.625µs, bloom_filter_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=46.298417ms, page_index_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=5, page_index_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=46.333µs, metadata_load_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=4.802583ms, files_ranges_pruned_statistics{partition=0}=0, predicate_evaluation_errors{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_pruned_bloom_filter{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_matched_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_groups_pruned_statistics{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, bytes_scanned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=2098163, pushdown_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, pushdown_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, row_pushdown_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, statistics_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, bloom_filter_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, page_index_rows_pruned{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_rows_matched{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=0, page_index_eval_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, metadata_load_time{partition=0, filename=var/folders/1l/tg68jc6550gg8xqf1hr4mlwr0000gn/T/parquet_pruningn7W55n.parquet}=NOT RECORDED, files_ranges_pruned_statistics{partition=0}=0, num_predicate_creation_errors=0
  left: 2
 right: 1

alamb · 2025-07-07T21:18:54Z

ok, I now have a clean run!

zhuqi-lucas · 2025-07-08T02:52:56Z

Thank you @alamb , I am curious about the benchmark result comparing the main branch, because we will include the apache/arrow-rs#7850 for this PR.

And some improvement part of the improvement we have ported to datafusion, but we will also benefit from the dependency changes from the arrow side, such as the sort phase(the merge compare we have ported)/compare, etc.

Could we trigger the benchmark for this PR, thanks!

alamb · 2025-07-08T10:43:00Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (76bc1b2) to ebb8e95 diff using: tpch_mem clickbench_partitioned clickbench_extended
Results will be posted here when complete

alamb · 2025-07-08T10:43:22Z

Could we trigger the benchmark for this PR, thanks!

Done!

alamb · 2025-07-08T11:15:47Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  1968.78 ms │              1943.30 ms │    no change │
│ QQuery 1     │   671.98 ms │               752.37 ms │ 1.12x slower │
│ QQuery 2     │  1322.34 ms │              1460.81 ms │ 1.10x slower │
│ QQuery 3     │   679.14 ms │               685.65 ms │    no change │
│ QQuery 4     │  1379.83 ms │              1357.79 ms │    no change │
│ QQuery 5     │ 15240.77 ms │             15204.35 ms │    no change │
│ QQuery 6     │  2055.68 ms │              2095.06 ms │    no change │
│ QQuery 7     │  1832.29 ms │              1860.29 ms │    no change │
└──────────────┴─────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 25150.80ms │
│ Total Time (alamb_update_arrow_56.0)   │ 25359.62ms │
│ Average Time (HEAD)                    │  3143.85ms │
│ Average Time (alamb_update_arrow_56.0) │  3169.95ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │          6 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.50 ms │                 2.22 ms │ +1.13x faster │
│ QQuery 1     │    34.30 ms │                34.87 ms │     no change │
│ QQuery 2     │    80.44 ms │                82.83 ms │     no change │
│ QQuery 3     │    99.02 ms │                98.66 ms │     no change │
│ QQuery 4     │   604.00 ms │               591.53 ms │     no change │
│ QQuery 5     │   865.09 ms │               853.85 ms │     no change │
│ QQuery 6     │     2.22 ms │                 2.35 ms │  1.06x slower │
│ QQuery 7     │    38.66 ms │                39.53 ms │     no change │
│ QQuery 8     │   863.08 ms │               865.34 ms │     no change │
│ QQuery 9     │  1170.02 ms │              1198.50 ms │     no change │
│ QQuery 10    │   256.55 ms │               235.52 ms │ +1.09x faster │
│ QQuery 11    │   282.16 ms │               273.37 ms │     no change │
│ QQuery 12    │   856.42 ms │               879.42 ms │     no change │
│ QQuery 13    │  1251.28 ms │              1228.51 ms │     no change │
│ QQuery 14    │   810.79 ms │               801.54 ms │     no change │
│ QQuery 15    │   792.79 ms │               780.00 ms │     no change │
│ QQuery 16    │  1598.21 ms │              1612.46 ms │     no change │
│ QQuery 17    │  1600.07 ms │              1600.00 ms │     no change │
│ QQuery 18    │  2922.80 ms │              2859.57 ms │     no change │
│ QQuery 19    │    91.60 ms │                88.65 ms │     no change │
│ QQuery 20    │  1156.47 ms │              1178.00 ms │     no change │
│ QQuery 21    │  1291.31 ms │              1323.34 ms │     no change │
│ QQuery 22    │  2100.41 ms │              2199.67 ms │     no change │
│ QQuery 23    │  7358.55 ms │              7637.05 ms │     no change │
│ QQuery 24    │   438.16 ms │               415.34 ms │ +1.05x faster │
│ QQuery 25    │   299.70 ms │               287.79 ms │     no change │
│ QQuery 26    │   446.98 ms │               415.91 ms │ +1.07x faster │
│ QQuery 27    │  1534.24 ms │              1572.63 ms │     no change │
│ QQuery 28    │ 12754.23 ms │             11991.17 ms │ +1.06x faster │
│ QQuery 29    │   537.38 ms │               518.72 ms │     no change │
│ QQuery 30    │   777.26 ms │               778.63 ms │     no change │
│ QQuery 31    │   805.02 ms │               758.54 ms │ +1.06x faster │
│ QQuery 32    │  2408.21 ms │              2360.51 ms │     no change │
│ QQuery 33    │  3161.89 ms │              3169.50 ms │     no change │
│ QQuery 34    │  3181.61 ms │              3197.43 ms │     no change │
│ QQuery 35    │  1255.44 ms │              1256.22 ms │     no change │
│ QQuery 36    │   119.55 ms │               119.48 ms │     no change │
│ QQuery 37    │    49.89 ms │                50.74 ms │     no change │
│ QQuery 38    │   120.47 ms │               121.13 ms │     no change │
│ QQuery 39    │   199.39 ms │               200.24 ms │     no change │
│ QQuery 40    │    41.00 ms │                41.43 ms │     no change │
│ QQuery 41    │    38.10 ms │                38.31 ms │     no change │
│ QQuery 42    │    32.07 ms │                31.78 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 54329.35ms │
│ Total Time (alamb_update_arrow_56.0)   │ 53792.25ms │
│ Average Time (HEAD)                    │  1263.47ms │
│ Average Time (alamb_update_arrow_56.0) │  1250.98ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         36 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  98.26 ms │                95.79 ms │     no change │
│ QQuery 2     │  19.98 ms │                20.60 ms │     no change │
│ QQuery 3     │  32.60 ms │                32.88 ms │     no change │
│ QQuery 4     │  18.59 ms │                18.52 ms │     no change │
│ QQuery 5     │  48.91 ms │                49.15 ms │     no change │
│ QQuery 6     │  11.62 ms │                11.80 ms │     no change │
│ QQuery 7     │  90.21 ms │                87.97 ms │     no change │
│ QQuery 8     │  23.40 ms │                25.01 ms │  1.07x slower │
│ QQuery 9     │  53.14 ms │                54.06 ms │     no change │
│ QQuery 10    │  42.78 ms │                40.78 ms │     no change │
│ QQuery 11    │  11.41 ms │                11.28 ms │     no change │
│ QQuery 12    │  34.36 ms │                31.87 ms │ +1.08x faster │
│ QQuery 13    │  26.55 ms │                26.12 ms │     no change │
│ QQuery 14    │   9.73 ms │                 9.52 ms │     no change │
│ QQuery 15    │  18.69 ms │                18.55 ms │     no change │
│ QQuery 16    │  17.99 ms │                18.22 ms │     no change │
│ QQuery 17    │  96.57 ms │                95.77 ms │     no change │
│ QQuery 18    │ 186.92 ms │               198.21 ms │  1.06x slower │
│ QQuery 19    │  24.15 ms │                23.11 ms │     no change │
│ QQuery 20    │  31.03 ms │                31.32 ms │     no change │
│ QQuery 21    │ 142.83 ms │               144.34 ms │     no change │
│ QQuery 22    │  14.82 ms │                14.18 ms │     no change │
└──────────────┴───────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 1054.52ms │
│ Total Time (alamb_update_arrow_56.0)   │ 1059.05ms │
│ Average Time (HEAD)                    │   47.93ms │
│ Average Time (alamb_update_arrow_56.0) │   48.14ms │
│ Queries Faster                         │         1 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │        19 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

zhuqi-lucas · 2025-07-08T11:21:00Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_extended.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ QQuery 0     │  1968.78 ms │              1943.30 ms │    no change │
│ QQuery 1     │   671.98 ms │               752.37 ms │ 1.12x slower │
│ QQuery 2     │  1322.34 ms │              1460.81 ms │ 1.10x slower │
│ QQuery 3     │   679.14 ms │               685.65 ms │    no change │
│ QQuery 4     │  1379.83 ms │              1357.79 ms │    no change │
│ QQuery 5     │ 15240.77 ms │             15204.35 ms │    no change │
│ QQuery 6     │  2055.68 ms │              2095.06 ms │    no change │
│ QQuery 7     │  1832.29 ms │              1860.29 ms │    no change │
└──────────────┴─────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 25150.80ms │
│ Total Time (alamb_update_arrow_56.0)   │ 25359.62ms │
│ Average Time (HEAD)                    │  3143.85ms │
│ Average Time (alamb_update_arrow_56.0) │  3169.95ms │
│ Queries Faster                         │          0 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │          6 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark clickbench_partitioned.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.50 ms │                 2.22 ms │ +1.13x faster │
│ QQuery 1     │    34.30 ms │                34.87 ms │     no change │
│ QQuery 2     │    80.44 ms │                82.83 ms │     no change │
│ QQuery 3     │    99.02 ms │                98.66 ms │     no change │
│ QQuery 4     │   604.00 ms │               591.53 ms │     no change │
│ QQuery 5     │   865.09 ms │               853.85 ms │     no change │
│ QQuery 6     │     2.22 ms │                 2.35 ms │  1.06x slower │
│ QQuery 7     │    38.66 ms │                39.53 ms │     no change │
│ QQuery 8     │   863.08 ms │               865.34 ms │     no change │
│ QQuery 9     │  1170.02 ms │              1198.50 ms │     no change │
│ QQuery 10    │   256.55 ms │               235.52 ms │ +1.09x faster │
│ QQuery 11    │   282.16 ms │               273.37 ms │     no change │
│ QQuery 12    │   856.42 ms │               879.42 ms │     no change │
│ QQuery 13    │  1251.28 ms │              1228.51 ms │     no change │
│ QQuery 14    │   810.79 ms │               801.54 ms │     no change │
│ QQuery 15    │   792.79 ms │               780.00 ms │     no change │
│ QQuery 16    │  1598.21 ms │              1612.46 ms │     no change │
│ QQuery 17    │  1600.07 ms │              1600.00 ms │     no change │
│ QQuery 18    │  2922.80 ms │              2859.57 ms │     no change │
│ QQuery 19    │    91.60 ms │                88.65 ms │     no change │
│ QQuery 20    │  1156.47 ms │              1178.00 ms │     no change │
│ QQuery 21    │  1291.31 ms │              1323.34 ms │     no change │
│ QQuery 22    │  2100.41 ms │              2199.67 ms │     no change │
│ QQuery 23    │  7358.55 ms │              7637.05 ms │     no change │
│ QQuery 24    │   438.16 ms │               415.34 ms │ +1.05x faster │
│ QQuery 25    │   299.70 ms │               287.79 ms │     no change │
│ QQuery 26    │   446.98 ms │               415.91 ms │ +1.07x faster │
│ QQuery 27    │  1534.24 ms │              1572.63 ms │     no change │
│ QQuery 28    │ 12754.23 ms │             11991.17 ms │ +1.06x faster │
│ QQuery 29    │   537.38 ms │               518.72 ms │     no change │
│ QQuery 30    │   777.26 ms │               778.63 ms │     no change │
│ QQuery 31    │   805.02 ms │               758.54 ms │ +1.06x faster │
│ QQuery 32    │  2408.21 ms │              2360.51 ms │     no change │
│ QQuery 33    │  3161.89 ms │              3169.50 ms │     no change │
│ QQuery 34    │  3181.61 ms │              3197.43 ms │     no change │
│ QQuery 35    │  1255.44 ms │              1256.22 ms │     no change │
│ QQuery 36    │   119.55 ms │               119.48 ms │     no change │
│ QQuery 37    │    49.89 ms │                50.74 ms │     no change │
│ QQuery 38    │   120.47 ms │               121.13 ms │     no change │
│ QQuery 39    │   199.39 ms │               200.24 ms │     no change │
│ QQuery 40    │    41.00 ms │                41.43 ms │     no change │
│ QQuery 41    │    38.10 ms │                38.31 ms │     no change │
│ QQuery 42    │    32.07 ms │                31.78 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 54329.35ms │
│ Total Time (alamb_update_arrow_56.0)   │ 53792.25ms │
│ Average Time (HEAD)                    │  1263.47ms │
│ Average Time (alamb_update_arrow_56.0) │  1250.98ms │
│ Queries Faster                         │          6 │
│ Queries Slower                         │          1 │
│ Queries with No Change                 │         36 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘
--------------------
Benchmark tpch_mem_sf1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃      HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 1     │  98.26 ms │                95.79 ms │     no change │
│ QQuery 2     │  19.98 ms │                20.60 ms │     no change │
│ QQuery 3     │  32.60 ms │                32.88 ms │     no change │
│ QQuery 4     │  18.59 ms │                18.52 ms │     no change │
│ QQuery 5     │  48.91 ms │                49.15 ms │     no change │
│ QQuery 6     │  11.62 ms │                11.80 ms │     no change │
│ QQuery 7     │  90.21 ms │                87.97 ms │     no change │
│ QQuery 8     │  23.40 ms │                25.01 ms │  1.07x slower │
│ QQuery 9     │  53.14 ms │                54.06 ms │     no change │
│ QQuery 10    │  42.78 ms │                40.78 ms │     no change │
│ QQuery 11    │  11.41 ms │                11.28 ms │     no change │
│ QQuery 12    │  34.36 ms │                31.87 ms │ +1.08x faster │
│ QQuery 13    │  26.55 ms │                26.12 ms │     no change │
│ QQuery 14    │   9.73 ms │                 9.52 ms │     no change │
│ QQuery 15    │  18.69 ms │                18.55 ms │     no change │
│ QQuery 16    │  17.99 ms │                18.22 ms │     no change │
│ QQuery 17    │  96.57 ms │                95.77 ms │     no change │
│ QQuery 18    │ 186.92 ms │               198.21 ms │  1.06x slower │
│ QQuery 19    │  24.15 ms │                23.11 ms │     no change │
│ QQuery 20    │  31.03 ms │                31.32 ms │     no change │
│ QQuery 21    │ 142.83 ms │               144.34 ms │     no change │
│ QQuery 22    │  14.82 ms │                14.18 ms │     no change │
└──────────────┴───────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 1054.52ms │
│ Total Time (alamb_update_arrow_56.0)   │ 1059.05ms │
│ Average Time (HEAD)                    │   47.93ms │
│ Average Time (alamb_update_arrow_56.0) │   48.14ms │
│ Queries Faster                         │         1 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │        19 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

Thank you @alamb, it seems we have some improvement for clickbench. Not too much because we gain for sort string view mostly which is not in clickbench but in sort_tpch.

alamb · 2025-07-08T12:36:59Z

Thank you @alamb, it seems we have some improvement for clickbench. Not too much because we gain for sort string view mostly which is not in clickbench but in sort_tpch.

I will start those as well

alamb · 2025-07-08T12:37:49Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (76bc1b2) to ebb8e95 diff using: sort_tpch
Results will be posted here when complete

alamb · 2025-07-08T12:39:02Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ Q1           │  330.29 ms │               350.13 ms │ 1.06x slower │
│ Q2           │  308.58 ms │               312.59 ms │    no change │
│ Q3           │ 1051.46 ms │              1076.53 ms │    no change │
│ Q4           │  444.92 ms │               462.29 ms │    no change │
│ Q5           │  410.39 ms │               412.36 ms │    no change │
│ Q6           │  449.16 ms │               452.76 ms │    no change │
│ Q7           │  783.43 ms │               833.85 ms │ 1.06x slower │
│ Q8           │  696.74 ms │               712.94 ms │    no change │
│ Q9           │  732.56 ms │               732.64 ms │    no change │
│ Q10          │ 1077.29 ms │              1090.28 ms │    no change │
│ Q11          │  547.35 ms │               563.97 ms │    no change │
└──────────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6832.16ms │
│ Total Time (alamb_update_arrow_56.0)   │ 7000.33ms │
│ Average Time (HEAD)                    │  621.11ms │
│ Average Time (alamb_update_arrow_56.0) │  636.39ms │
│ Queries Faster                         │         0 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │         9 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

zhuqi-lucas · 2025-07-08T12:47:58Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ Q1           │  330.29 ms │               350.13 ms │ 1.06x slower │
│ Q2           │  308.58 ms │               312.59 ms │    no change │
│ Q3           │ 1051.46 ms │              1076.53 ms │    no change │
│ Q4           │  444.92 ms │               462.29 ms │    no change │
│ Q5           │  410.39 ms │               412.36 ms │    no change │
│ Q6           │  449.16 ms │               452.76 ms │    no change │
│ Q7           │  783.43 ms │               833.85 ms │ 1.06x slower │
│ Q8           │  696.74 ms │               712.94 ms │    no change │
│ Q9           │  732.56 ms │               732.64 ms │    no change │
│ Q10          │ 1077.29 ms │              1090.28 ms │    no change │
│ Q11          │  547.35 ms │               563.97 ms │    no change │
└──────────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6832.16ms │
│ Total Time (alamb_update_arrow_56.0)   │ 7000.33ms │
│ Average Time (HEAD)                    │  621.11ms │
│ Average Time (alamb_update_arrow_56.0) │  636.39ms │
│ Queries Faster                         │         0 │
│ Queries Slower                         │         2 │
│ Queries with No Change                 │         9 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

Thank you @alamb , it seems no improvement for sort dependencies(So it means the merge will occupied most time), so we gain most for the ported PRs to merge phase from:

#16509
#16630

alamb · 2025-07-09T10:09:04Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (76bc1b2) to ebb8e95 diff using: sort_tpch
Results will be posted here when complete

alamb · 2025-07-09T10:20:30Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark sort_tpch1.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Query        ┃       HEAD ┃ alamb_update_arrow_56.0 ┃       Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ Q1           │  327.61 ms │               348.24 ms │ 1.06x slower │
│ Q2           │  297.95 ms │               300.94 ms │    no change │
│ Q3           │ 1038.69 ms │              1036.65 ms │    no change │
│ Q4           │  453.62 ms │               445.95 ms │    no change │
│ Q5           │  410.20 ms │               414.40 ms │    no change │
│ Q6           │  446.72 ms │               453.75 ms │    no change │
│ Q7           │  798.24 ms │               793.45 ms │    no change │
│ Q8           │  700.70 ms │               709.04 ms │    no change │
│ Q9           │  745.09 ms │               730.99 ms │    no change │
│ Q10          │ 1063.65 ms │              1052.35 ms │    no change │
│ Q11          │  547.28 ms │               545.21 ms │    no change │
└──────────────┴────────────┴─────────────────────────┴──────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃           ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 6829.73ms │
│ Total Time (alamb_update_arrow_56.0)   │ 6830.98ms │
│ Average Time (HEAD)                    │  620.88ms │
│ Average Time (alamb_update_arrow_56.0) │  621.00ms │
│ Queries Faster                         │         0 │
│ Queries Slower                         │         1 │
│ Queries with No Change                 │        10 │
│ Queries with Failure                   │         0 │
└────────────────────────────────────────┴───────────┘

zhuqi-lucas · 2025-07-09T13:44:00Z

Thank you @alamb @Dandandan , we may also try sort_tpch10 benchmark , but it may also not too much improvement, the ported PR already has 1.4x faster for sort_tpch Q11(inlined string view sort).

Dandandan · 2025-07-09T21:23:14Z

I wonder if we should focus on more parallelisation in SortPreservingMerge (besides improving the hot paths like gc interleave, comparisons, etc.)

I think there might be two areas:

Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.
Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

zhuqi-lucas · 2025-07-10T01:15:07Z

I wonder if we should focus on more parallelisation in SortPreservingMerge (besides improving the hot paths like gc interleave, comparisons, etc.)

I think there might be two areas:

Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.

Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

Thank you @Dandandan , interesting idea, i agree if we can improve this point, it will help not only stringview both also other type, so it will benefit more cases, i will investigate!

zhuqi-lucas · 2025-07-10T14:49:47Z

@alamb @Dandandan

I submit a PR based this PR, try to see if fast_gc can also help for sort_tpch or sort_tpch10 benchmark:

#16739

May be we can run sort_tpch for above PR to see the result, thanks!

alamb · 2025-07-11T20:33:28Z

Doing (some of the) merge algorithm itself in parallel - I am not sure what would the best way forward here, but it seems it could give the largest gains, as merging is currently done in a single thread.

Here is a related idea;

Improve performance of large sorts with Cascaded merge / tree #7181

Doing conversion to Row format in parallel - it seems currently this is done in a single thread, it should give some (smaller) speedup

This is an interesting idea and I think it may make a significant difference -- the core merge is a single threaded operation, so the less time that thread has to wait (for example, to fetch the next cursor of rows) will contribute directly to the overall speed of the query

alamb · 2025-07-15T18:09:18Z

🤖 ./gh_compare_branch.sh Benchmark Script Running
Linux aal-dev 6.11.0-1016-gcp #16~24.04.1-Ubuntu SMP Wed May 28 02:40:52 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
Comparing alamb/update_arrow_56.0.0 (aa4b8cb) to 18a30ce diff using: clickbench_pushdown
Results will be posted here when complete

alamb · 2025-07-15T18:29:48Z

🤖: Benchmark completed

Details

Comparing HEAD and alamb_update_arrow_56.0.0
--------------------
Benchmark clickbench_pushdown.json
--------------------
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┓
┃ Query        ┃        HEAD ┃ alamb_update_arrow_56.0 ┃        Change ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━┩
│ QQuery 0     │     2.30 ms │                 2.36 ms │     no change │
│ QQuery 1     │    37.12 ms │                34.62 ms │ +1.07x faster │
│ QQuery 2     │    85.20 ms │                81.35 ms │     no change │
│ QQuery 3     │   110.66 ms │                99.00 ms │ +1.12x faster │
│ QQuery 4     │   636.12 ms │               683.39 ms │  1.07x slower │
│ QQuery 5     │   875.44 ms │               867.55 ms │     no change │
│ QQuery 6     │     2.20 ms │                 2.22 ms │     no change │
│ QQuery 7     │    60.25 ms │                54.50 ms │ +1.11x faster │
│ QQuery 8     │   877.35 ms │               880.46 ms │     no change │
│ QQuery 9     │  1187.59 ms │              1197.53 ms │     no change │
│ QQuery 10    │   274.37 ms │               249.11 ms │ +1.10x faster │
│ QQuery 11    │   311.65 ms │               293.56 ms │ +1.06x faster │
│ QQuery 12    │  1074.89 ms │              1055.93 ms │     no change │
│ QQuery 13    │  1528.92 ms │              1483.48 ms │     no change │
│ QQuery 14    │  1128.46 ms │              1078.40 ms │     no change │
│ QQuery 15    │   732.11 ms │               788.13 ms │  1.08x slower │
│ QQuery 16    │  1563.70 ms │              1589.78 ms │     no change │
│ QQuery 17    │  1546.79 ms │              1572.03 ms │     no change │
│ QQuery 18    │  3014.31 ms │              2853.48 ms │ +1.06x faster │
│ QQuery 19    │    86.03 ms │                84.12 ms │     no change │
│ QQuery 20    │  1163.65 ms │              1128.18 ms │     no change │
│ QQuery 21    │  1375.45 ms │              1276.21 ms │ +1.08x faster │
│ QQuery 22    │  2624.92 ms │              2407.28 ms │ +1.09x faster │
│ QQuery 23    │   985.77 ms │               966.30 ms │     no change │
│ QQuery 24    │   238.60 ms │               217.41 ms │ +1.10x faster │
│ QQuery 25    │   594.65 ms │               522.32 ms │ +1.14x faster │
│ QQuery 26    │   398.97 ms │               371.70 ms │ +1.07x faster │
│ QQuery 27    │  2345.01 ms │              2213.13 ms │ +1.06x faster │
│ QQuery 28    │ 12730.34 ms │             12615.99 ms │     no change │
│ QQuery 29    │   523.53 ms │               529.91 ms │     no change │
│ QQuery 30    │  1231.07 ms │              1183.07 ms │     no change │
│ QQuery 31    │  1193.39 ms │              1176.06 ms │     no change │
│ QQuery 32    │  2504.08 ms │              2464.79 ms │     no change │
│ QQuery 33    │  3273.28 ms │              3201.70 ms │     no change │
│ QQuery 34    │  3398.36 ms │              3270.08 ms │     no change │
│ QQuery 35    │  1282.25 ms │              1290.68 ms │     no change │
│ QQuery 36    │    27.41 ms │                27.90 ms │     no change │
│ QQuery 37    │    27.08 ms │                27.77 ms │     no change │
│ QQuery 38    │    27.03 ms │                27.02 ms │     no change │
│ QQuery 39    │    27.12 ms │                27.54 ms │     no change │
│ QQuery 40    │    27.23 ms │                26.92 ms │     no change │
│ QQuery 41    │    26.99 ms │                27.28 ms │     no change │
│ QQuery 42    │    27.16 ms │                27.30 ms │     no change │
└──────────────┴─────────────┴─────────────────────────┴───────────────┘
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━┓
┃ Benchmark Summary                      ┃            ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━┩
│ Total Time (HEAD)                      │ 51188.78ms │
│ Total Time (alamb_update_arrow_56.0)   │ 49977.54ms │
│ Average Time (HEAD)                    │  1190.44ms │
│ Average Time (alamb_update_arrow_56.0) │  1162.27ms │
│ Queries Faster                         │         12 │
│ Queries Slower                         │          2 │
│ Queries with No Change                 │         29 │
│ Queries with Failure                   │          0 │
└────────────────────────────────────────┴────────────┘

github-actions bot added the common Related to common crate label Jul 5, 2025

alamb marked this pull request as draft July 5, 2025 11:57

alamb force-pushed the alamb/update_arrow_56.0.0 branch from 6698376 to 1747605 Compare July 5, 2025 12:20

github-actions bot added logical-expr Logical plan and expressions proto Related to proto crate labels Jul 5, 2025

alamb force-pushed the alamb/update_arrow_56.0.0 branch from 1747605 to fc5bd79 Compare July 5, 2025 13:06

github-actions bot added documentation Improvements or additions to documentation sql SQL Planner core Core DataFusion crate sqllogictest SQL Logic Tests (.slt) substrait Changes to the substrait crate datasource Changes to the datasource crate labels Jul 5, 2025

This was referenced Jul 5, 2025

Incorrect inlined string view comparison after " Add prefix compare for inlined" apache/arrow-rs#7874

Closed

Parquet filter pushdown v4 apache/arrow-rs#7850

Open

alamb force-pushed the alamb/update_arrow_56.0.0 branch from 1c82d05 to 76bc1b2 Compare July 7, 2025 18:47

alamb mentioned this pull request Jul 7, 2025

POC: Test DataFusion with experimental Parquet Filter Pushdown (try 4) #16711

Draft

alamb added 2 commits July 15, 2025 13:53

Update arrow/parquet to 56.0.0, Update pyo3

5106df5

Update for new APIs

aa4b8cb

alamb force-pushed the alamb/update_arrow_56.0.0 branch from 76bc1b2 to aa4b8cb Compare July 15, 2025 17:57

DRAFT: Update arrow/parquet to 56.0.0 #16690

Are you sure you want to change the base?

DRAFT: Update arrow/parquet to 56.0.0 #16690

Uh oh!

Conversation

alamb commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Which issue does this PR close?

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

alamb commented Jul 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Jul 7, 2025

Uh oh!

zhuqi-lucas commented Jul 8, 2025

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

zhuqi-lucas commented Jul 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

alamb commented Jul 8, 2025

Uh oh!

zhuqi-lucas commented Jul 8, 2025

Uh oh!

alamb commented Jul 9, 2025

Uh oh!

alamb commented Jul 9, 2025

Uh oh!

zhuqi-lucas commented Jul 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Dandandan commented Jul 9, 2025

Uh oh!

zhuqi-lucas commented Jul 10, 2025

Uh oh!

zhuqi-lucas commented Jul 10, 2025

Uh oh!

alamb commented Jul 11, 2025

Uh oh!

alamb commented Jul 15, 2025

Uh oh!

alamb commented Jul 15, 2025

Uh oh!

Uh oh!

alamb commented Jul 5, 2025 •

edited

Loading

alamb commented Jul 5, 2025 •

edited

Loading

zhuqi-lucas commented Jul 8, 2025 •

edited

Loading

zhuqi-lucas commented Jul 9, 2025 •

edited

Loading