Skip to content

feat(cdk): port QA checks from airbyte-ci to airbyte-python-cdk #526

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

devin-ai-integration[bot]
Copy link
Contributor

Description

This PR ports the quality assurance (QA) checks from airbyte-ci to the airbyte-python-cdk, creating a new module at airbyte_cdk.qa with a CLI command airbyte-cdk connector pre-release-check.

Changes

  • Created a new module airbyte_cdk.qa with the following components:
    • Core abstractions in models.py for Check, CheckResult, and Report classes
    • Simplified Connector class in connector.py with minimal dependencies
    • Ported checks for assets, documentation, metadata, packaging, security, and testing
    • Added CLI implementation for running pre-release checks
  • Added comprehensive docstrings to the module describing the checks, configuration options, and usage
  • Added dependencies for semver and pydash to support the QA checks
  • Added unit tests for the Connector and CheckResult classes

How to test

Run the CLI command:

airbyte-cdk connector pre-release-check --connector-name <connector-name>

Requested by

Aaron ("AJ") Steers (aj@airbyte.io)

Link to Devin run

https://app.devin.ai/sessions/765c11efdefa43578eb78b1308e7b89a

Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Copy link
Contributor Author

Original prompt from Aaron:

@Devin - Can you help me create a plan to refactor or port the "quality assurance" or "qa" checks for connectors, from airbyte-ci into either the airbyte python cdk, or else into a standalone app or script? Can you tell me how portable the code is as of now, and which specific code files contain the logic? Also let me know if it relies on dagger/docker at all
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring
@github-actions github-actions bot added the enhancement New feature or request label May 1, 2025
devin-ai-integration bot and others added 7 commits May 1, 2025 04:03
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
…ecture

Co-Authored-By: Aaron <AJ> Steers <aj@airbyte.io>
Comment on lines +107 to +115
config.addinivalue_line("markers", "qa_check: mark a test as a QA check")
config.addinivalue_line("markers", "check_category(category): mark a test with a check category")
config.addinivalue_line("markers", "connector_language(languages): mark a test as applicable to specific connector languages")
config.addinivalue_line("markers", "connector_type(types): mark a test as applicable to specific connector types")
config.addinivalue_line("markers", "connector_support_level(levels): mark a test as applicable to specific connector support levels")
config.addinivalue_line("markers", "connector_cloud_usage(usages): mark a test as applicable to specific connector cloud usages")
config.addinivalue_line("markers", "connector_ab_internal_sl(sl): mark a test as applicable to connectors with a specific ab_internal_sl")
config.addinivalue_line("markers", "requires_metadata: mark a test as requiring metadata")
config.addinivalue_line("markers", "runs_on_released_connectors: mark a test as running on released connectors")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need all these?

Copy link
Contributor Author

The markers are used to filter tests based on connector properties. We can simplify this by keeping only the essential ones:

Essential markers:

  • qa_check: Identifies tests that are QA checks
  • check_category: Categorizes checks (metadata, security, etc.)
  • requires_metadata: Indicates if a check needs metadata.yaml
  • connector_language: Filters by connector language (Python, Java, etc.)

Optional markers that could be removed:

  • connector_type, connector_support_level, connector_cloud_usage: These are more granular filters
  • connector_ab_internal_sl: Very specific to internal support levels
  • runs_on_released_connectors: Could be simplified

I can reduce these to just the essential markers if preferred.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
1 participant