Provided and maintained by 🦄 UnicoLab

Transform your raw data into ML-ready features with just a few lines of code!

KDP provides a state-of-the-art preprocessing system built on TensorFlow Keras. It handles everything from feature normalization to advanced embedding techniques, making your ML pipelines faster, more robust, and easier to maintain. Built with ❤️ by 🦄 UnicoLab, it provides a clean, efficient, and extensible foundation for building sophisticated machine learning models for enterprise AI applications.

• 🚀 Efficient Single-Pass Processing: Process all features in one go, dramatically faster than alternatives
• 🧠 Distribution-Aware Encoding: Automatically detects and optimally handles different data distributions
• 👁️ Tabular Attention: Captures complex feature interactions for better model performance
• 🔍 Feature Selection: Automatically identifies and focuses on the most important features
• 🔄 Feature-wise Mixture of Experts: Specialized processing for different feature types
• 📦 Production-Ready: Deploy your preprocessing along with your model as a single unit

# Using pip
pip install kdp

# Using Poetry
poetry add kdp

from kdp import PreprocessingModel, FeatureType

# Define your features
features_specs = {
    "age": FeatureType.FLOAT_NORMALIZED,
    "income": FeatureType.FLOAT_RESCALED,
    "occupation": FeatureType.STRING_CATEGORICAL,
    "description": FeatureType.TEXT
}

# Create and build the preprocessor
preprocessor = PreprocessingModel(
    path_data="data/my_data.csv",
    features_specs=features_specs,
    # Enable advanced features
    use_distribution_aware=True,
    tabular_attention=True
)
result = preprocessor.build_preprocessor()
model = result["model"]

# Use the preprocessor with your data
processed_features = model(input_data)

We've built an extensive documentation system to help you get the most from KDP:

• 🚀 Quick Start Guide - Get up and running in minutes
• 📊 Feature Processing - Learn about all supported feature types
• 🧙‍♂️ Auto-Configuration - Let KDP configure itself for your data

• 📈 Distribution-Aware Encoding - Smart handling of different distributions
• 👁️ Tabular Attention - Capture complex feature interactions
• 🔢 Advanced Numerical Embeddings - Rich representations for numbers
• 🤖 Transformer Blocks - Apply transformer architecture to tabular data
• 🎯 Feature Selection - Focus on what matters in your data
• 🧠 Feature-wise Mixture of Experts - Specialized processing per feature

• 🔗 Integration Guide - Use KDP with existing ML pipelines
• 🚀 Tabular Optimization - Supercharge your preprocessing
• 📈 Performance Tips - Handling large datasets efficiently

• 💡 Motivation - Why we built KDP
• 🤝 Contributing - Help improve KDP

Your preprocessing pipeline is built as a Keras model that can be used independently or as the first layer of any model:

KDP outperforms alternative preprocessing approaches, especially as data size increases:

We welcome contributions! Please check out our Contributing Guide for guidelines on how to proceed.

Have questions or want to connect with other KDP users? Join us on Discord:

KDP includes tools to help developers:

• Documentation Generation: Automatically generate API docs from docstrings
• Model Diagram Generation: Visualize model architectures with make generate_doc_content or run:

  python scripts/generate_model_diagrams.py

  This creates diagram images in docs/features/imgs/models/ for all feature types and configurations.

This project is licensed under the MIT License - see the LICENSE file for details.

• Built with TensorFlow and Keras
• Inspired by modern deep learning research
• Community-driven development
• All contributors who help make KDP better

Built with ❤️ for the ML community by 🦄 UnicoLab.ai