Remove Spark 2.x #2316
Remove Spark 2.x #2316
Conversation
There was a problem hiding this comment.
LGTM pending comments addressed. I think we'll need to make sure that we are logging a deprecation warning when Spark 2.x is used in 8.x. We may also want to discuss bumping the default scala supported version to 2.13 instead of leaving it on 2.12 considering how long these major releases live for, but that can be a follow up after more discussion.
| [[removals-9.0]] | ||
| ==== Removal of Spark 2.x | ||
|
|
||
| Support for the Spark 2.x has been removed in {eh} 9.0, Spark 3.x is still supported. |
There was a problem hiding this comment.
Lets instead say that Spark 3.x is the new default support version
| @@ -10,12 +10,10 @@ apply plugin: 'spark.variants' | |||
| sparkVariants { | |||
| capabilityGroup 'org.elasticsearch.spark.variant' | |||
|
|
|||
| // Changing the formatting of these lines could break .buildkite/pipeline.py, it uses regex to parse the `spark20scala212` part | |||
| // Changing the formatting of these lines could break .buildkite/pipeline.py, it uses regex to parse the `spark30scala212` part | |||
There was a problem hiding this comment.
We should check in with delivery about this line to see if anything needs to be updated in CI for this change
There was a problem hiding this comment.
Sure, I also found this old PR of yours that removes the old Spark from release manager.
+1 on making Scala 2.13 the default, I can try doing that as a follow up. |
I added deprecation warning in #2305 and backported to 8.x, do you think we want to add more? |
After Spark 2.x deprecation in 8.18 #2305, this change removes support for it in 9.0.