Skip to content

Conversation

@XiaoHongbo-Hope
Copy link
Contributor

@XiaoHongbo-Hope XiaoHongbo-Hope commented Oct 31, 2025

[core][rest] Add schema validation and inference for REST catalog external tables

Purpose

Currently, REST catalog supported external tables but did not validate schema consistency between the filesystem and server-side schema. External tables always required explicit schema definition even when schema already existed in the filesystem.

This PR

Enhances REST catalog external table creation with schema inference and validation:

  • Validates client-provided schema against filesystem schema before creating table metadata
  • Enables creating external tables without explicit schema when schema exists
    in the location

Examples:
CREATE TABLE t2 (id INT, name STRING) USING paimon LOCATION 'path' (explicit schema)
CREATE TABLE t2 USING paimon LOCATION 'path' (schema inference, newly supported)

other fix about external table

  • Ignore NotImplementedException exception when show all tables by load sys.tables to avoid failure caused by external paimon table

Tests

PaimonExternalTableTest
RESTCatalogTest.testCreateExternalTableWithSchemaInference
RESTCatalogTest.testReadSystemTablesWithExternalTable

API and Format

Documentation

@XiaoHongbo-Hope XiaoHongbo-Hope marked this pull request as draft October 31, 2025 13:52
@XiaoHongbo-Hope XiaoHongbo-Hope changed the title support schema validation and infer for external paimon table Oct 31, 2025
@XiaoHongbo-Hope XiaoHongbo-Hope marked this pull request as ready for review November 2, 2025 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

1 participant