Skip to content

Conversation

@andygrove
Copy link
Member

@andygrove andygrove commented Dec 27, 2025

Which issue does this PR close?

Closes #2998

Rationale for this change

Format Baseline Optimized Improvement
YEAR 142 µs 70 µs 51% faster
QUARTER 145 µs 114 µs 21% faster
MONTH 94 µs 76 µs 19% faster
WEEK 127 µs 78 µs 39% faster

What changes are included in this PR?

How are these changes tested?

@andygrove andygrove changed the title feat: Improve performance of date truncate perf: Improve performance of date truncate Dec 27, 2025
@codecov-commenter
Copy link

codecov-commenter commented Dec 27, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 59.58%. Comparing base (f09f8af) to head (cddee7b).
⚠️ Report is 803 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2997      +/-   ##
============================================
+ Coverage     56.12%   59.58%   +3.46%     
- Complexity      976     1377     +401     
============================================
  Files           119      167      +48     
  Lines         11743    15493    +3750     
  Branches       2251     2569     +318     
============================================
+ Hits           6591     9232    +2641     
- Misses         4012     4962     +950     
- Partials       1140     1299     +159     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@sqlbenchmark
Copy link

Comet Microbenchmark Results: CometDatetimeExpressionBenchmark

Commit: cddee7b - feat: Improve performance of date truncate

Benchmark Results

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - YEAR:                    Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                38             40           2         27.9          35.8       1.0X
Comet (Scan)                                         37             39           1         28.6          35.0       1.0X
Comet (Scan + Exec)                                  34             36           1         31.0          32.3       1.1X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - YYYY:                    Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                33             36           2         31.7          31.6       1.0X
Comet (Scan)                                         34             35           1         30.8          32.5       1.0X
Comet (Scan + Exec)                                  33             34           1         32.1          31.2       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - YY:                      Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                31             33           1         33.5          29.8       1.0X
Comet (Scan)                                         33             34           3         32.1          31.1       1.0X
Comet (Scan + Exec)                                  32             33           1         32.5          30.8       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - MON:                     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                33             34           1         31.8          31.4       1.0X
Comet (Scan)                                         34             35           1         30.9          32.4       1.0X
Comet (Scan + Exec)                                  33             34           1         32.0          31.2       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - MONTH:                   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                33             34           1         31.9          31.4       1.0X
Comet (Scan)                                         33             35           1         31.3          31.9       1.0X
Comet (Scan + Exec)                                  32             33           1         33.3          30.1       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate  - MM:                      Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                33             34           1         31.8          31.5       1.0X
Comet (Scan)                                         33             34           1         31.7          31.5       1.0X
Comet (Scan + Exec)                                  31             33           1         33.4          30.0       1.1X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - YEAR:        Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                31             33           1         34.1          29.3       1.0X
Comet (Scan)                                         33             34           1         31.9          31.3       0.9X
Comet (Scan + Exec)                                  33             34           1         32.1          31.1       0.9X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - YYYY:        Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                31             32           1         34.2          29.2       1.0X
Comet (Scan)                                         32             34           1         32.5          30.8       1.0X
Comet (Scan + Exec)                                  31             33           1         33.4          30.0       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - YY:          Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                30             32           1         34.4          29.1       1.0X
Comet (Scan)                                         32             33           1         32.7          30.6       1.0X
Comet (Scan + Exec)                                  32             33           1         33.1          30.2       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - MON:         Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                32             34           1         32.4          30.8       1.0X
Comet (Scan)                                         32             34           3         32.6          30.7       1.0X
Comet (Scan + Exec)                                  31             32           1         33.3          30.0       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - MONTH:       Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                33             34           1         31.4          31.9       1.0X
Comet (Scan)                                         32             33           1         32.7          30.6       1.0X
Comet (Scan + Exec)                                  31             33           1         33.3          30.0       1.1X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Date Truncate (Dictionary) - MM:          Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                32             33           1         32.3          30.9       1.0X
Comet (Scan)                                         32             33           1         32.8          30.5       1.0X
Comet (Scan + Exec)                                  32             33           1         33.1          30.2       1.0X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - YEAR:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               105            105           1         10.0          99.7       1.0X
Comet (Scan)                                        101            102           0         10.3          96.7       1.0X
Comet (Scan + Exec)                                 314            315           1          3.3         299.2       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - YYYY:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               104            105           1         10.0          99.5       1.0X
Comet (Scan)                                        101            102           1         10.4          96.4       1.0X
Comet (Scan + Exec)                                 315            318           2          3.3         300.4       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - YY:                 Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               103            105           1         10.1          98.6       1.0X
Comet (Scan)                                        101            102           1         10.4          96.4       1.0X
Comet (Scan + Exec)                                 313            314           1          3.4         298.5       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - MON:                Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               104            105           1         10.1          99.4       1.0X
Comet (Scan)                                        102            103           1         10.3          96.9       1.0X
Comet (Scan + Exec)                                 264            265           1          4.0         251.7       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - MONTH:              Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               104            105           0         10.1          99.4       1.0X
Comet (Scan)                                        102            103           1         10.3          97.5       1.0X
Comet (Scan + Exec)                                 269            270           1          3.9         256.4       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - MM:                 Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               104            105           1         10.1          99.4       1.0X
Comet (Scan)                                        105            106           1         10.0         100.5       1.0X
Comet (Scan + Exec)                                 276            277           1          3.8         263.3       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - DAY:                Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                88             89           1         11.9          84.1       1.0X
Comet (Scan)                                         85             86           1         12.3          81.2       1.0X
Comet (Scan + Exec)                                 224            225           1          4.7         214.1       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - DD:                 Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                87             89           1         12.0          83.0       1.0X
Comet (Scan)                                         85             86           1         12.3          81.4       1.0X
Comet (Scan + Exec)                                 224            225           1          4.7         213.9       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - HOUR:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                94             95           1         11.2          89.5       1.0X
Comet (Scan)                                         92             93           1         11.5          87.3       1.0X
Comet (Scan + Exec)                                 186            188           1          5.6         177.4       0.5X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - MINUTE:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                80             81           1         13.2          76.0       1.0X
Comet (Scan)                                         77             78           1         13.7          73.3       1.0X
Comet (Scan + Exec)                                 143            144           1          7.3         136.1       0.6X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - SECOND:             Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                31             32           1         33.7          29.7       1.0X
Comet (Scan)                                         28             29           1         37.1          26.9       1.1X
Comet (Scan + Exec)                                 101            102           1         10.4          95.9       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - WEEK:               Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98             99           1         10.7          93.5       1.0X
Comet (Scan)                                         96             98           2         10.9          91.7       1.0X
Comet (Scan + Exec)                                 252            257           2          4.2         240.5       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate  - QUARTER:            Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                               122            123           1          8.6         116.4       1.0X
Comet (Scan)                                        117            119           2          9.0         111.4       1.0X
Comet (Scan + Exec)                                 314            315           1          3.3         299.4       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - YEAR:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98             98           1         10.7          93.2       1.0X
Comet (Scan)                                         95             95           0         11.1          90.2       1.0X
Comet (Scan + Exec)                                 323            325           1          3.2         308.4       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - YYYY:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98             99           1         10.7          93.1       1.0X
Comet (Scan)                                         95             96           1         11.1          90.4       1.0X
Comet (Scan + Exec)                                 324            325           1          3.2         308.9       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - YY:     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                97            101           9         10.8          92.8       1.0X
Comet (Scan)                                         94             95           0         11.1          89.9       1.0X
Comet (Scan + Exec)                                 324            325           1          3.2         309.5       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - MON:    Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98            101           9         10.7          93.5       1.0X
Comet (Scan)                                         95             96           0         11.0          90.8       1.0X
Comet (Scan + Exec)                                 278            279           1          3.8         265.4       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - MONTH:  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98            100           2         10.7          93.7       1.0X
Comet (Scan)                                         95             96           1         11.0          91.0       1.0X
Comet (Scan + Exec)                                 277            278           1          3.8         264.4       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - MM:     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                98             99           1         10.7          93.8       1.0X
Comet (Scan)                                         95             96           0         11.0          91.1       1.0X
Comet (Scan + Exec)                                 278            279           1          3.8         265.4       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - DAY:    Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                74             75           1         14.2          70.6       1.0X
Comet (Scan)                                         71             72           1         14.7          67.8       1.0X
Comet (Scan + Exec)                                 232            233           1          4.5         221.4       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - DD:     Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                74             75           1         14.1          70.8       1.0X
Comet (Scan)                                         71             72           0         14.8          67.7       1.0X
Comet (Scan + Exec)                                 233            234           1          4.5         222.0       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - HOUR:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                72             75           5         14.5          69.0       1.0X
Comet (Scan)                                         70             71           1         15.0          66.5       1.0X
Comet (Scan + Exec)                                 191            193           3          5.5         181.9       0.4X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - MINUTE:  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                73             73           1         14.4          69.3       1.0X
Comet (Scan)                                         70             71           1         15.0          66.7       1.0X
Comet (Scan + Exec)                                 146            147           1          7.2         139.5       0.5X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - SECOND:  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                26             28           1         39.6          25.3       1.0X
Comet (Scan)                                         23             24           1         44.9          22.3       1.1X
Comet (Scan + Exec)                                 100            100           1         10.5          95.0       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - WEEK:   Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
------------------------------------------------------------------------------------------------------------------------
Spark                                                92             93           1         11.4          87.9       1.0X
Comet (Scan)                                         90             90           0         11.7          85.7       1.0X
Comet (Scan + Exec)                                 264            265           1          4.0         251.6       0.3X

OpenJDK 64-Bit Server VM 17.0.17+10 on Linux 6.8.0-57-generic
AMD Ryzen 9 7950X 16-Core Processor
Timestamp Truncate (Dictionary) - QUARTER:  Best Time(ms)   Avg Time(ms)   Stdev(ms)    Rate(M/s)   Per Row(ns)   Relative
-------------------------------------------------------------------------------------------------------------------------
Spark                                                118            121           6          8.9         112.7       1.0X
Comet (Scan)                                         115            117           2          9.1         109.5       1.0X
Comet (Scan + Exec)                                  330            332           2          3.2         314.9       0.4X


Automated benchmark run by dfbench

@apache apache deleted a comment from sqlbenchmark Jan 2, 2026
@apache apache deleted a comment from sqlbenchmark Jan 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve performance of date truncate

3 participants