Skip to content

Find a better way to special-case hudi, remove dependency from presto-hive #17164

Open
@cocozianu

Description

@cocozianu

Having hudi special cased in presto-hive,is a recipe for trouble. Example here:

directoryListerOverride = Optional.of(new HudiDirectoryLister(configuration, session, table));

This created problems, for example the commit here 070c995 brought us hudi-presto-bundle-0.10.0 which packages classes outside of hudi scope that will trigger conflicts at runtime and/or build time with other installation of presto.

Even presto-delta inside this repository had to add an to exclude hudi-presto-bundle, unfortunately, because hudi inserts itself in critical places inside apache hive, such an approach can lead to unexpected runtime failures.

The way forward it seems to me is have a proper plugin (sub-project) for hudi's version of hive, in the meantime we'll need to revert the commit that brought us hudi-presto-bundle 0.10.0.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions