Skip to content

PSM Interop: simplify Kokoro buildscripts #2729

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
May 6, 2024
Merged

PSM Interop: simplify Kokoro buildscripts #2729

merged 1 commit into from
May 6, 2024

Conversation

@sergiitk
Copy link
Member

@sergiitk sergiitk commented Apr 26, 2024 •
edited
Loading

Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73.

Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to psm_interop_kokoro_lib.sh:

  1. The list of tests in the each test suite
  2. Per-test-suite flag customization
  3. run_test methods
  4. build_docker_images_if_needed methods
  5. Generic build_test_app_docker_images methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow.

Now all PSM Interop jobs share the same buildscripts by all test suites:

  1. buildscript that invokes the test: psm-interop-test-{language}.sh (configured as build_file in the build cfg)
  2. buildscript that builds the xDS test client/server and publishes them as a Docker image: psm-interop-build-{language}.sh (conventional name called from psm_interop_kokoro_lib.sh)

psm-interop-test-{language}.sh:

  1. Sets GRPC_LANGUAGE, BUILD_SCRIPT_DIR environment variables.
  2. Downloads the shared psm_interop_kokoro_lib.sh from the main branch of the psm-interop repo.
  3. Sources psm-interop-build-{language}.sh
  4. Calls psm::run "${PSM_TEST_SUITE}" (PSM_TEST_SUITE configured in the cfg file).

psm-interop-build-{language}.sh:

  1. Defines psm::lang::build_docker_images which is called from psm_interop_kokoro_lib.sh.
  2. Invokes any repo-specific logic.
  3. May use psm::build::docker_images_generic for generic Docker build, tag, push, or provide implement its own build/publish method.

References:
@sergiitk
Copy link
Member Author

sergiitk commented Apr 26, 2024 •
edited
Loading

Using grpc/psm-interop#73 / grpc/psm-interop@1c0f169:

upd: failed
@sergiitk
Copy link
Member Author

sergiitk commented Apr 26, 2024 •
edited
Loading

Using grpc/psm-interop#73 / grpc/psm-interop@ce43c7d:
@sergiitk
Copy link
Member Author

sergiitk commented Apr 27, 2024 •
edited
Loading

Using grpc/psm-interop#73 / grpc/psm-interop@8f28619:
@sergiitk
Copy link
Member Author

sergiitk commented Apr 27, 2024 •
edited
Loading

Using grpc/psm-interop#73 / grpc/psm-interop@5aeb13f:

Full LB test run.
@sergiitk sergiitk mentioned this pull request Apr 30, 2024
Merged
sergiitk added a commit to grpc/psm-interop that referenced this pull request May 1, 2024
@sergiitk
Move all common buildscript functionality to psm_interop_kokoro_lib.sh (
6ffc132 
#73)

Major rework of the shared Kokoro PSM Interop install library.

Nearly all common functionality was moved from per-language/per-branch
PSM Interop build scripts to psm_interop_kokoro_lib.sh:
1. The list of tests in the each test suite 
2. Per-test-suite flag customization
3. `run_test` methods
4. `build_docker_images_if_needed` methods
5. Generic `build_test_app_docker_images` methods (simple docker build +
docker push + docker tag). grpc-java is one exception, as it doesn't run
docker directly, but a cloudbuild flow.

Added new options to control PSM interop test driver execution via
environment variables:
```sh
# Overrides the list of test to run. A whitespace-separated string. Example:
# PSM_TESTS="app_net_test baseline_test"
readonly PSM_TESTS="${PSM_TESTS:-}"

# A space-separated string with extra flags to append to the test driver arguments.
# Can be used to execute a run and test a new flag value, f.e.:
# PSM_EXTRA_FLAGS="--noenable_workload_identity --td_bootstrap_image=us-docker.pkg.dev/new-image..."
# In addition, can be used to run a single test in a suite. 
# F.e. to run only test_mtls in security_test:
# PSM_TESTS="security_test" PSM_EXTRA_FLAGS="SecurityTest.test_mtls"
readonly PSM_EXTRA_FLAGS="${PSM_EXTRA_FLAGS:-}"
```

Improved logging:
1. Buildscript test logs are now prepended with the date:
   - when a shell command passed to `psm::tools::run_verbose`
   - `psm::tools::log` is used in place of `echo`
2. Individual install and build steps partially or fully redirected to
the dedicated log files:
  - `install-apt.log`
  - `install-pip.log`
  - `build-docker.log`
3. Various other log improvements: printing tests in the suite, printing
flags passed to the frameworks, etc

Corresponding per-language buildscript PRs:
1. java - grpc/grpc-java#11121
2. cpp - grpc/grpc#36450
3. go - grpc/grpc-go#7171
4. node - grpc/grpc-node#2729

Notes for the reviewer:
1. The new script is backward-compatible with the old-style
buildscripts.
2. Per-language buildscript PRs will be backported to all supported
branches, and this should be the very last backport chore we'll ever
need.
3. A minor change to the subsetting test was needed to solve [a one-off
exception](https://github.com/grpc/grpc/blob/6e981d7460af561230d455c3623774e76e18cbfe/tools/internal_ci/linux/grpc_xds_k8s_lb.sh#L123-L127)
from the normal `run_test` flow.

ref b/288578634
@sergiitk
Copy link
Member Author

sergiitk commented May 2, 2024 •
edited
Loading

The last check using the install script from main:
@sergiitk sergiitk changed the title PSM Interop: move test suite lists to psm_interop_kokoro_lib.sh May 2, 2024
@sergiitk
Copy link
Member Author

sergiitk commented May 2, 2024

Please merge it yet. I'd like to wait until I got approvals on the same change from the rest of the repos. So we don't have to do another set of backports if anything comes up.
@sergiitk
Copy link
Member Author

sergiitk commented May 3, 2024 •
edited
Loading

Using grpc/psm-interop@2cbdfd02:
@sergiitk
Copy link
Member Author

sergiitk commented May 3, 2024 •
edited
Loading

@sergiitk sergiitk force-pushed the psm-interop-common-prod-tests branch from 6d6e535 to 1495b02 Compare May 6, 2024 21:06
@sergiitk
Copy link
Member Author

sergiitk commented May 6, 2024 •
edited
Loading

Just to stay on the safe side, ad-hoc run after squashing all commits and rebasing against master:
@murgatroid99 murgatroid99 merged commit 15b7d4d into grpc:master May 6, 2024
@sergiitk sergiitk deleted the psm-interop-common-prod-tests branch May 6, 2024 22:21
sergiitk pushed a commit to sergiitk/grpc-node that referenced this pull request May 6, 2024
@murgatroid99 @sergiitk
Merge pull request grpc#2729 from sergiitk/psm-interop-common-prod-tests
fec135a 
PSM Interop: simplify Kokoro buildscripts
@sergiitk sergiitk mentioned this pull request May 6, 2024
Merged
sergiitk pushed a commit to sergiitk/grpc-node that referenced this pull request May 6, 2024
@murgatroid99 @sergiitk
Merge pull request grpc#2729 from sergiitk/psm-interop-common-prod-tests
cf14020 
PSM Interop: simplify Kokoro buildscripts
@sergiitk sergiitk mentioned this pull request May 6, 2024
Merged
sergiitk pushed a commit to sergiitk/grpc-node that referenced this pull request May 6, 2024
@murgatroid99 @sergiitk
Merge pull request grpc#2729 from sergiitk/psm-interop-common-prod-tests
00f348c 
PSM Interop: simplify Kokoro buildscripts
sergiitk pushed a commit to sergiitk/grpc-node that referenced this pull request May 6, 2024
@murgatroid99 @sergiitk
Merge pull request grpc#2729 from sergiitk/psm-interop-common-prod-tests
0040f6b 
PSM Interop: simplify Kokoro buildscripts
@sergiitk sergiitk mentioned this pull request May 6, 2024
Merged
@sergiitk sergiitk mentioned this pull request May 6, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

4 participants

@sergiitk @gnossen @murgatroid99 @kokoro-team