Skip to content

Conversation

@alamb
Copy link

@alamb alamb commented Nov 24, 2025

Update DataFusion results for the DataFusion 51 release (TODO add blog URL here when published)

I followed the directions in

Changes

  • Update some readme contents to remove outdated contents
  • Fix lukewarm-code tagging (the scripts in this repository run datafusion-cli from scratch each time, so there are no caches maintained from query to query) - Tag runs as lukewarm #692 (comment)
  • Add scripts to convert from csv --> json result format

Variants:

  • DataFusion parquet
  • DataFusion parquet-partitioned

Note I did not include datafusion with vortex
(TBD ping SpiralDB)

Results included

  • c6a.4xlarge
  • c6a.2xlarge
  • c6a.xlarge
  • c6a.large

Not sure

  • c8g.4xlarge
  • t3a.small
@CLAassistant
Copy link

CLAassistant commented Nov 24, 2025

CLA assistant check
All committers have signed the CLA.

@alamb alamb force-pushed the alamb/update_datafusion branch from 0c9ff10 to f9c2654 Compare November 24, 2025 15:17
@rschu1ze rschu1ze self-assigned this Nov 24, 2025
@rschu1ze
Copy link
Member

@alamb Please ping me when this PR is ready for review - thanks.

@alamb
Copy link
Author

alamb commented Nov 24, 2025

Thank you @rschu1ze -- I am still doing some performance analysis (you can see details here if you care apache/datafusion#18909). I will let you know when it is ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

3 participants