[core][rest] support schema validation and infer for external paimon table #6501
+682
−73
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
[core][rest] Add schema validation and inference for REST catalog external tables
Purpose
Currently, REST catalog supported external tables but did not validate schema consistency between the filesystem and server-side schema. External tables always required explicit schema definition even when schema already existed in the filesystem.
This PR
Enhances REST catalog external table creation with schema inference and validation:
in the location
Examples:
CREATE TABLE t2 (id INT, name STRING) USING paimon LOCATION 'path' (explicit schema)CREATE TABLE t2 USING paimon LOCATION 'path' (schema inference, newly supported)other fix about external table
sys.tablesto avoid failure caused by external paimon tableTests
PaimonExternalTableTest
RESTCatalogTest.testCreateExternalTableWithSchemaInference
RESTCatalogTest.testReadSystemTablesWithExternalTable
API and Format
Documentation