Skip to content

Conversation

@sergiitk
Copy link
Member

@sergiitk sergiitk commented Apr 26, 2024

Integrates the new features of the the Kokoro PSM Interop install library introduced in grpc/psm-interop#73.

Nearly all common functionality was moved from per-language/per-branch PSM Interop build scripts to psm_interop_kokoro_lib.sh:

  1. The list of tests in the each test suite
  2. Per-test-suite flag customization
  3. run_test methods
  4. build_docker_images_if_needed methods
  5. Generic build_test_app_docker_images methods (simple docker build + docker push + docker tag). grpc-java is one exception, as it doesn't run docker directly, but a cloudbuild flow.

Now all PSM Interop jobs share the same buildscripts by all test suites:

  1. buildscript that invokes the test: psm-interop-test-{language}.sh (configured as build_file in the build cfg)
  2. buildscript that builds the xDS test client/server and publishes them as a Docker image: psm-interop-build-{language}.sh (conventional name called from psm_interop_kokoro_lib.sh)

psm-interop-test-{language}.sh:

  1. Sets GRPC_LANGUAGE, BUILD_SCRIPT_DIR environment variables.
  2. Downloads the shared psm_interop_kokoro_lib.sh from the main branch of the psm-interop repo.
  3. Sources psm-interop-build-{language}.sh
  4. Calls psm::run "${PSM_TEST_SUITE}" (PSM_TEST_SUITE configured in the cfg file).

psm-interop-build-{language}.sh:

  1. Defines psm::lang::build_docker_images which is called from psm_interop_kokoro_lib.sh.
  2. Invokes any repo-specific logic.
  3. May use psm::build::docker_images_generic for generic Docker build, tag, push, or provide implement its own build/publish method.

References:

RELEASE NOTES: n/a

@codecov
Copy link

codecov bot commented Apr 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.54%. Comparing base (adf976b) to head (15a33a7).
Report is 35 commits behind head on master.

❗ Current head 15a33a7 differs from pull request most recent head 113ae98. Consider uploading reports for the commit 113ae98 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7171      +/-   ##
==========================================
- Coverage   81.24%   80.54%   -0.70%     
==========================================
  Files         345      347       +2     
  Lines       33941    33991      +50     
==========================================
- Hits        27574    27379     -195     
- Misses       5202     5423     +221     
- Partials     1165     1189      +24     

see 25 files with indirect coverage changes

@sergiitk
Copy link
Member Author

sergiitk commented Apr 27, 2024

@arvindbr8
Copy link
Contributor

Seems like the tests pass. @sergiitk - Looks good to me. I can approve when you send the PR out for review.

@sergiitk
Copy link
Member Author

@arvindbr8 Thank you. This depends on grpc/psm-interop#73. Once it's merged, I'll promote this PR from a draft.

sergiitk added a commit to grpc/psm-interop that referenced this pull request May 1, 2024
#73)

Major rework of the shared Kokoro PSM Interop install library.

Nearly all common functionality was moved from per-language/per-branch
PSM Interop build scripts to psm_interop_kokoro_lib.sh:
1. The list of tests in the each test suite 
2. Per-test-suite flag customization
3. `run_test` methods
4. `build_docker_images_if_needed` methods
5. Generic `build_test_app_docker_images` methods (simple docker build +
docker push + docker tag). grpc-java is one exception, as it doesn't run
docker directly, but a cloudbuild flow.

Added new options to control PSM interop test driver execution via
environment variables:
```sh
# Overrides the list of test to run. A whitespace-separated string. Example:
# PSM_TESTS="app_net_test baseline_test"
readonly PSM_TESTS="${PSM_TESTS:-}"

# A space-separated string with extra flags to append to the test driver arguments.
# Can be used to execute a run and test a new flag value, f.e.:
# PSM_EXTRA_FLAGS="--noenable_workload_identity --td_bootstrap_image=us-docker.pkg.dev/new-image..."
# In addition, can be used to run a single test in a suite. 
# F.e. to run only test_mtls in security_test:
# PSM_TESTS="security_test" PSM_EXTRA_FLAGS="SecurityTest.test_mtls"
readonly PSM_EXTRA_FLAGS="${PSM_EXTRA_FLAGS:-}"
```

Improved logging:
1. Buildscript test logs are now prepended with the date:
   - when a shell command passed to `psm::tools::run_verbose`
   - `psm::tools::log` is used in place of `echo`
2. Individual install and build steps partially or fully redirected to
the dedicated log files:
  - `install-apt.log`
  - `install-pip.log`
  - `build-docker.log`
3. Various other log improvements: printing tests in the suite, printing
flags passed to the frameworks, etc

Corresponding per-language buildscript PRs:
1. java - grpc/grpc-java#11121
2. cpp - grpc/grpc#36450
3. go - grpc/grpc-go#7171
4. node - grpc/grpc-node#2729

Notes for the reviewer:
1. The new script is backward-compatible with the old-style
buildscripts.
2. Per-language buildscript PRs will be backported to all supported
branches, and this should be the very last backport chore we'll ever
need.
3. A minor change to the subsetting test was needed to solve [a one-off
exception](https://github.com/grpc/grpc/blob/6e981d7460af561230d455c3623774e76e18cbfe/tools/internal_ci/linux/grpc_xds_k8s_lb.sh#L123-L127)
from the normal `run_test` flow.

ref b/288578634
@sergiitk
Copy link
Member Author

sergiitk commented May 2, 2024

@sergiitk sergiitk marked this pull request as ready for review May 2, 2024 22:46
@sergiitk
Copy link
Member Author

sergiitk commented May 2, 2024

@arvindbr8 please review, but don't merge it yet. I'd like to wait until I got approvals on the same change from the rest of the repos. So we don't have to do another set of backports if anything comes up.

@arvindbr8
Copy link
Contributor

Sure will do once the tests pass. Do you mind re-assigning this to me when you think it's ready?

@sergiitk sergiitk changed the title test/kokoro: move test suite lists to psm_interop_kokoro_lib.sh May 2, 2024
@sergiitk
Copy link
Member Author

sergiitk commented May 3, 2024

@arvindbr8 arvindbr8 added this to the 1.64 Release milestone May 4, 2024
@arvindbr8 arvindbr8 merged commit f2d6421 into grpc:master May 4, 2024
1 check passed
@sergiitk sergiitk deleted the psm-interop-common-prod-tests branch May 6, 2024 19:19
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
sergiitk added a commit to sergiitk/grpc-go that referenced this pull request May 6, 2024
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 3, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

2 participants