Skip to content

[CORE] Refine the test with specified spark version#9274

Merged
zhztheplayer merged 1 commit into
apache:mainfrom
yikf:simple-testWithSpecifiedSparkVersion
Apr 10, 2025
Merged

[CORE] Refine the test with specified spark version#9274
zhztheplayer merged 1 commit into
apache:mainfrom
yikf:simple-testWithSpecifiedSparkVersion

Conversation

@yikf

@yikf yikf commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This PR aims to refine the test with specified spark version.

for example, before, testWithSpecifiedSparkVersion(" XXX ", Some (" 3.3 ")) can let dev think only in 3.3 test, but the reality is above 3.3 is tested.

How was this patch tested?

GA
@github-actions

github-actions Bot commented Apr 9, 2025

Copy link
Copy Markdown

Thanks for opening a pull request!

Could you open an issue for this pull request on Github Issues?

https://github.com/apache/incubator-gluten/issues

Then could you also rename commit message and pull request title in the following format?

[GLUTEN-${ISSUES_ID}][COMPONENT]feat/fix: ${detailed message}

See also:
@github-actions

github-actions Bot commented Apr 9, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
Yohahaha
Yohahaha approved these changes Apr 9, 2025
View reviewed changes

@Yohahaha Yohahaha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice refactoring, thank you!
@Yohahaha

Yohahaha commented Apr 9, 2025

Copy link
Copy Markdown
Contributor

may need to verify the total count of passed UT does not decreased.
@yikf

yikf commented Apr 9, 2025

Copy link
Copy Markdown
Contributor Author

@Yohahaha Thanks for your review. I checked the,

  1. The number UTs of the JsonFunctionsValidateSuite at Spark 3.5, for testAtLeastSparkVersion since it is most likely to cause decrease ut.
  2. Reviewed the refactor function and call sides.

It works fine as expected.
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from acdfab1 to f14772d Compare April 9, 2025 12:58
@github-actions

github-actions Bot commented Apr 9, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
zhztheplayer
zhztheplayer reviewed Apr 9, 2025
View reviewed changes

@zhztheplayer zhztheplayer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % CH CI is failing
jackylee-ch
jackylee-ch reviewed Apr 10, 2025
View reviewed changes
Comment thread gluten-hudi/src/test/scala/org/apache/gluten/execution/HudiSuite.scala Outdated
Comment thread gluten-substrait/src/test/scala/org/apache/spark/sql/GlutenQueryTest.scala Outdated
Comment thread gluten-iceberg/src/test/scala/org/apache/gluten/execution/IcebergSuite.scala Outdated
Comment thread gluten-iceberg/src/test/scala/org/apache/gluten/execution/IcebergSuite.scala Outdated
Comment thread backends-velox/src/test/scala/org/apache/spark/sql/execution/VeloxParquetReadSuite.scala Outdated
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from f14772d to ceb368a Compare April 10, 2025 02:39
@github-actions

github-actions Bot commented Apr 10, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from ceb368a to 00a9939 Compare April 10, 2025 02:52
@github-actions

github-actions Bot commented Apr 10, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
jackylee-ch
jackylee-ch reviewed Apr 10, 2025
View reviewed changes
Comment thread ...ouse/src-iceberg/test/scala/org/apache/gluten/execution/iceberg/ClickHouseIcebergSuite.scala Outdated

@jackylee-ch jackylee-ch Apr 10, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can also be changed to testWithMinSparkVersion

@yikf yikf Apr 10, 2025

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, is should be all.
image
Comment thread ...ouse/src-iceberg/test/scala/org/apache/gluten/execution/iceberg/ClickHouseIcebergSuite.scala Outdated

@jackylee-ch jackylee-ch Apr 10, 2025

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this can also be changed to testWithMinSparkVersion
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from 00a9939 to f567653 Compare April 10, 2025 06:07
@github-actions

github-actions Bot commented Apr 10, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
@yikf yikf force-pushed the simple-testWithSpecifiedSparkVersion branch from f567653 to 4ebf0c4 Compare April 10, 2025 10:57
@github-actions

github-actions Bot commented Apr 10, 2025

Copy link
Copy Markdown

Run Gluten Clickhouse CI on x86
@zhztheplayer zhztheplayer merged commit 118b114 into apache:main Apr 10, 2025
@zhouyuan zhouyuan mentioned this pull request Apr 10, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment