fix: native-image support feature class by suztomo · Pull Request #682 · googleapis/java-compute

suztomo · 2022-04-21T21:42:20Z

Fixes #685

I didn't see the ending of the error message approach, where I update the list and wait 30 min to get error and repeat. It already reached ~500 classes (0425a99). So I try registering all classes in the proto directory for reflection:

suztomo · 2022-04-22T01:44:00Z

java.lang.AssertionError: Zone us-central1-a was not found in zones list.
	at com.google.cloud.compute.v1.integration.ITPaginationTest.testPaginationIterating(ITPaginationTest.java:122)

suztomo · 2022-04-22T20:53:20Z

https://source.cloud.google.com/results/invocations/8c0474b8-e199-4faf-b2a4-683ef232f65f/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fjava-compute%2Fpresubmit%2Fgraalvm-native/log this seems true positive:

Failures (121):
  JUnit Vintage:RegionInstanceGroupManagersClientTest
    ClassSource [className = 'com.google.cloud.compute.v1.RegionInstanceGroupManagersClientTest', filePosition = null]
    => org.opentest4j.MultipleFailuresError: Multiple Failures (2 failures)
	java.lang.ExceptionInInitializerError: <no message>
	java.lang.NullPointerException: <no message>
       org.junit.vintage.engine.execution.TestRun.getStoredResultOrSuccessful(TestRun.java:196)
       org.junit.vintage.engine.execution.RunListenerAdapter.fireExecutionFinished(RunListenerAdapter.java:226)
       org.junit.vintage.engine.execution.RunListenerAdapter.testRunFinished(RunListenerAdapter.java:93)
       org.junit.runner.notification.SynchronizedRunListener.testRunFinished(SynchronizedRunListener.java:42)
       org.junit.runner.notification.RunNotifier$2.notifyListener(RunNotifier.java:103)
       [...]
       Suppressed: java.lang.ExceptionInInitializerError
         com.google.cloud.compute.v1.Compute.<clinit>(Compute.java:18809)
         com.google.cloud.compute.v1.Operation.getDescriptor(Operation.java:291)
         com.google.cloud.compute.v1.stub.HttpJsonRegionInstanceGroupManagersStub.<clinit>(HttpJsonRegionInstanceGroupManagersStub.java:84)
         com.google.cloud.compute.v1.RegionInstanceGroupManagersClientTest.startStaticServer(RegionInstanceGroupManagersClientTest.java:58)
         java.lang.reflect.Method.invoke(Method.java:566)
         [...]
       Caused by: java.lang.RuntimeException: Generated message class "com.google.api.FieldBehavior" missing method "valueOf".
         com.google.protobuf.GeneratedMessage.getMethodOrDie(GeneratedMessage.java:1973)
         com.google.protobuf.GeneratedMessage.access$1100(GeneratedMessage.java:61)
         com.google.protobuf.GeneratedMessage$GeneratedExtension.<init>(GeneratedMessage.java:1789)
         com.google.protobuf.GeneratedMessage.newFileScopedGeneratedExtension(GeneratedMessage.java:1643)
         com.google.api.FieldBehaviorProto.<clinit>(FieldBehaviorProto.java:55)
         [...]
       Caused by: java.lang.NoSuchMethodException: com.google.api.FieldBehavior.valueOf(com.google.protobuf.Descriptors$EnumValueDescriptor)
         java.lang.Class.getMethod(DynamicHub.java:1114)
         com.google.protobuf.GeneratedMessage.getMethodOrDie(GeneratedMessage.java:1970)
         [...]
       Suppressed: java.lang.NullPointerException
         com.google.cloud.compute.v1.RegionInstanceGroupManagersClientTest.stopServer(RegionInstanceGroupManagersClientTest.java:73)
         java.lang.reflect.Method.invoke(Method.java:566)
         org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
         org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
         org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
         [...]

       Suppressed: java.lang.NoClassDefFoundError: Could not initialize class com.google.cloud.compute.v1.Compute
         com.google.cloud.compute.v1.Operation.getDescriptor(Operation.java:291)
         com.google.cloud.compute.v1.stub.HttpJsonRoutersStub.<clinit>(HttpJsonRoutersStub.java:75)
         com.google.cloud.compute.v1.RoutersClientTest.startStaticServer(RoutersClientTest.java:59)
         java.lang.reflect.Method.invoke(Method.java:566)
         org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
         [...]
       Suppressed: java.lang.NullPointerException
         com.google.cloud.compute.v1.RoutersClientTest.stopServer(RoutersClientTest.java:73)
         java.lang.reflect.Method.invoke(Method.java:566)
         org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
         org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
         org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
         [...]

Retrying.

suztomo · 2022-04-25T13:13:28Z

ITPaginationTest failed: https://source.cloud.google.com/results/invocations/7a915acd-4f68-45f6-9791-9782fb824d3e/targets/github%2Fjava-compute%2Fgoogle-cloud-compute%2Ftarget%2Fsurefire-reports%2Fgraalvm-native%2Fcom.google.cloud.compute.v1.integration.ITPaginationTest/tests;failed=false

java.lang.AssertionError: Zone us-central1-a was not found in zones list.
	at com.google.cloud.compute.v1.integration.ITPaginationTest.testPaginationIterating(ITPaginationTest.java:122)

RegionInstanceGroupManagersClientTest is not part of the tests this time. Why?

protected static final String DEFAULT_ZONE = "us-central1-a"; is defined in BaseTest.java.

Retrying.

It turned out that when it got "Zone us-central1-a was not found in zones list" error, it failed at maven-surefire-plugin:2.22.2:test (default-test) @ google-cloud-compute (normal JVM invocation).

When it got NoSuchMethodException and NoClassDefFoundError, it failed at native-maven-plugin:0.9.11:test (test-native) @ google-cloud-compute (GraalVM native-image invocation).

suztomo · 2022-04-26T12:53:17Z

"Kokoro - Test: Java GraalVM Native Image" succeeded.

It took 1 hr 30 min.

suztomo · 2022-04-27T15:21:13Z

All checks have passed

mpeddada1 · 2022-04-27T16:02:55Z

+        "com.google.cloud.compute.v1.TestIamPermissionsLicenseCodeRequest$Builder");
+    computeClassesBuilder.add("com.google.cloud.compute.v1.TestIamPermissionsLicenseRequest");
+    computeClassesBuilder.add(
+        "com.google.cloud.compute.v1.TestIamPermissionsLicenseRequest$Builder");
+    computeC


Out of curiosity, what happens when we remove these lines?

You get the errors such as:

Suppressed: java.lang.ExceptionInInitializerError com.google.cloud.compute.v1.Compute.<clinit>(Compute.java:18809) com.google.cloud.compute.v1.Operation.getDescriptor(Operation.java:291) com.google.cloud.compute.v1.stub.HttpJsonRegionInstanceGroupManagersStub.<clinit>(HttpJsonRegionInstanceGroupManagersStub.java:84) com.google.cloud.compute.v1.RegionInstanceGroupManagersClientTest.startStaticServer(RegionInstanceGroupManagersClientTest.java:58) java.lang.reflect.Method.invoke(Method.java:566) [...] Caused by: java.lang.RuntimeException: Generated message class "com.google.api.FieldBehavior" missing method "valueOf". com.google.protobuf.GeneratedMessage.getMethodOrDie(GeneratedMessage.java:1973) com.google.protobuf.GeneratedMessage.access$1100(GeneratedMessage.java:61) com.google.protobuf.GeneratedMessage$GeneratedExtension.<init>(GeneratedMessage.java:1789) com.google.protobuf.GeneratedMessage.newFileScopedGeneratedExtension(GeneratedMessage.java:1643) com.google.api.FieldBehaviorProto.<clinit>(FieldBehaviorProto.java:55) [...] Caused by: java.lang.NoSuchMethodException: com.google.api.FieldBehavior.valueOf(com.google.protobuf.Descriptors$EnumValueDescriptor) java.lang.Class.getMethod(DynamicHub.java:1114) com.google.protobuf.GeneratedMessage.getMethodOrDie(GeneratedMessage.java:1970) [...]

#685

mpeddada1 · 2022-04-27T16:44:04Z

+        "com.google.cloud.compute.v1.TestIamPermissionsLicenseRequest$Builder");
+    computeC


We can probably shorten this and similar classes above by making using of registerClassHierarchyForReflection("com.google.cloud.compute.v1.XpnHostList") which will register nested non-private classes.

With that approach, we need to exclude the inner classes. It's an additional step in listing classes. I think it's easier to maintain the class list if we just register all classes regardless of nested or not.

What do you think?

mpeddada1 · 2022-04-27T17:48:06Z

+    computeClassesBuilder.add(
+        "com.google.cloud.compute.v1.AbandonInstancesRegionInstanceGroupManagerRequest");
+    computeClassesBuilder.add(
+        "com.google.cloud.compute.v1.AbandonInstancesRegionInstanceGroupManagerRequest$Builder");


Could you point me to the tests that fail if we don't include these com.google.cloud.compute.v1 classes? I've experienced situations where I had to add a ton a classes for reflection to make the test work but it was just one class that was needed in the end, so I'm secretly hoping that we'll run into same scenario here.

https://source.cloud.google.com/results/invocations/81073aee-a5fa-4581-af46-462972101862/targets/cloud-devrel%2Fclient-libraries%2Fjava%2Fjava-compute%2Fpresubmit%2Fgraalvm-native/log

Here is an example:

JUnit Vintage:NetworkFirewallPoliciesClientTest:getTest MethodSource [className = 'com.google.cloud.compute.v1.NetworkFirewallPoliciesClientTest', methodName = 'getTest', methodParameterTypes = ''] => com.google.api.gax.rpc.UnknownException: java.lang.RuntimeException: Generated message class "com.google.cloud.compute.v1.FirewallPolicy" missing method "getAssociationsList". com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:119) com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41) com.google.api.gax.httpjson.HttpJsonApiExceptionFactory.create(HttpJsonApiExceptionFactory.java:67) com.google.api.gax.httpjson.HttpJsonExceptionCallable$ExceptionTransformingFuture.onFailure(HttpJsonExceptionCallable.java:97) com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67) [...] Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57) com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112) com.google.cloud.compute.v1.NetworkFirewallPoliciesClient.get(NetworkFirewallPoliciesClient.java:687) com.google.cloud.compute.v1.NetworkFirewallPoliciesClient.get(NetworkFirewallPoliciesClient.java:662) com.google.cloud.compute.v1.NetworkFirewallPoliciesClientTest.getTest(NetworkFirewallPoliciesClientTest.java:388) [...] Caused by: java.lang.RuntimeException: Generated message class "com.google.cloud.compute.v1.FirewallPolicy" missing method "getAssociationsList". com.google.protobuf.GeneratedMessageV3.getMethodOrDie(GeneratedMessageV3.java:1847) com.google.protobuf.GeneratedMessageV3.access$1000(GeneratedMessageV3.java:79) com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$RepeatedFieldAccessor$ReflectionInvoker.<init>(GeneratedMessageV3.java:2409) com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$RepeatedFieldAccessor.<init>(GeneratedMessageV3.java:2479) com.google.protobuf.GeneratedMessageV3$FieldAccessorTable$RepeatedMessageFieldAccessor.<init>(GeneratedMessageV3.java:3004) [...] Caused by: java.lang.NoSuchMethodException: com.google.cloud.compute.v1.FirewallPolicy.getAssociationsList() java.lang.Class.getMethod(DynamicHub.java:1114) com.google.protobuf.GeneratedMessageV3.getMethodOrDie(GeneratedMessageV3.java:1844) [...]

Be careful. GraalVM native test runs 1309 test cases across 90 test files.

gcf-merge-on-green · 2022-04-27T23:02:11Z

Merge-on-green attempted to merge your PR for 6 hours, but it was not mergeable because either one of your required status checks failed, one of your required reviews was not approved, or there is a do not merge label. Learn more about your required status checks here: https://help.github.com/en/github/administering-a-repository/enabling-required-status-checks. You can remove and reapply the label to re-run the bot.

mpeddada1 · 2022-04-29T14:53:52Z

@suztomo It looks like the tests were failing because the configurations in GAX weren't getting picked up by the tests. Here is draft PR that worked locally: https://github.com/googleapis/java-compute/pull/688/files. The ProtobufMessageFeature which is present in gax-grpc allows for registration of reflective usages of the GRPC Protobuf libraries. This error message helped me understand what was happening:

Caused by: java.lang.RuntimeException: Generated message class "com.google.cloud.compute.v1.FirewallPolicy" missing method "getAssociationsList".
       com.google.protobuf.GeneratedMessageV3.getMethodOrDie(GeneratedMessageV3.java:1847)
       com.google.protobuf.GeneratedMessageV3.access$1000(GeneratedMessageV3.java:79)

suztomo · 2022-04-29T16:01:41Z

Fantastic. I didn't notice that we can leverage com.google.api.client.json.GenericJson.

ci: pull request to invoke native image test

f03becf

product-auto-label Bot added size: u Pull request is empty. api: compute Issues related to the googleapis/java-compute API. labels Apr 21, 2022

suztomo added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 22, 2022

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 22, 2022

suztomo added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 22, 2022

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 22, 2022

suztomo added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 25, 2022

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 25, 2022

ci: debugging zone list failure

8e0c4e5

product-auto-label Bot added size: xs Pull request size is extra small. and removed size: u Pull request is empty. labels Apr 25, 2022

suztomo mentioned this pull request Apr 25, 2022

GraalVM native-image compilation errors #685

Closed

feat: reflection-config.json for FieldBehavior

28d2178

product-auto-label Bot added size: s Pull request size is small. and removed size: xs Pull request size is extra small. labels Apr 25, 2022

suztomo added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 25, 2022

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 25, 2022

suztomo added 6 commits April 25, 2022 11:30

Merge remote-tracking branch 'origin/main' into graalvm_test

9928304

feat: reflection-config.json for OperationResponseMapping

3b45484

feat: reflection config for DescriptorProtos$FieldOptions

fea706b

feat: reflection config for DescriptorProtos$FieldOptions$Builder

c019334

feat: reflection config for DescriptorProtos$FieldOptions$CType

214a83f

feat: reflection config for DescriptorProtos$FieldOptions$JSType

64d6098

product-auto-label Bot added size: m Pull request size is medium. and removed size: s Pull request size is small. labels Apr 25, 2022

feat: computeFeature rather than YAML

8da7e54

suztomo force-pushed the graalvm_test branch from 026fa22 to 3f25b2a Compare April 25, 2022 23:35

suztomo added 5 commits April 25, 2022 19:50

deps: graal-sdk

47f9a62

feat: adding more classes for reflection

343d479

feat: more reflection configurations

d9e0e72

feat: more reflection configurations

0425a99

product-auto-label Bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Apr 26, 2022

feat: comment for how to generate the list

7e7b53b

suztomo requested a review from mpeddada1 April 26, 2022 12:53

suztomo changed the title ~~ci: pull request to invoke native image test~~ Apr 26, 2022

suztomo added the owlbot:run Add this label to trigger the Owlbot post processor. label Apr 26, 2022

gcf-owl-bot Bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label Apr 26, 2022

suztomo marked this pull request as ready for review April 26, 2022 14:20

suztomo requested a review from a team April 26, 2022 14:20

suztomo added the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 26, 2022

yoshi-kokoro removed the kokoro:force-run Add this label to force Kokoro to re-run the tests. label Apr 26, 2022

mpeddada1 reviewed Apr 27, 2022

View reviewed changes

suztomo added the automerge Merge the pull request once unit tests and other checks pass. label Apr 27, 2022

suztomo requested a review from mpeddada1 April 27, 2022 17:01

mpeddada1 reviewed Apr 27, 2022

View reviewed changes

suztomo requested a review from mpeddada1 April 27, 2022 20:00

gcf-merge-on-green Bot removed the automerge Merge the pull request once unit tests and other checks pass. label Apr 27, 2022

suztomo closed this Apr 29, 2022

emmileaf mentioned this pull request Feb 17, 2023

IT Failure(google-cloud-compute): ITPaginationTest.testPaginationIterating zone not found googleapis/google-cloud-java#9134

Open

Navigation Menu

Repository navigation

fix: native-image support feature class - #682

fix: native-image support feature class#682
suztomo wants to merge 17 commits into
googleapis:mainfrom
suztomo:graalvm_test

suztomo commented Apr 21, 2022 •

edited

Loading

suztomo commented Apr 22, 2022

suztomo commented Apr 22, 2022

suztomo commented Apr 25, 2022 •

edited

Loading

suztomo commented Apr 26, 2022 •

edited

Loading

suztomo commented Apr 27, 2022

mpeddada1 Apr 27, 2022

suztomo Apr 27, 2022

mpeddada1 Apr 27, 2022

suztomo Apr 27, 2022

mpeddada1 Apr 27, 2022

suztomo Apr 27, 2022 •

edited

Loading

gcf-merge-on-green Bot commented Apr 27, 2022

mpeddada1 commented Apr 29, 2022

suztomo commented Apr 29, 2022

Uh oh!

Labels

3 participants

		"com.google.cloud.compute.v1.TestIamPermissionsLicenseRequest$Builder");
		computeC

Conversation

suztomo commented Apr 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

suztomo commented Apr 22, 2022

suztomo commented Apr 22, 2022

suztomo commented Apr 25, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

suztomo commented Apr 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

suztomo commented Apr 27, 2022

mpeddada1 Apr 27, 2022

Choose a reason for hiding this comment

suztomo Apr 27, 2022

Choose a reason for hiding this comment

mpeddada1 Apr 27, 2022

Choose a reason for hiding this comment

suztomo Apr 27, 2022

Choose a reason for hiding this comment

mpeddada1 Apr 27, 2022

Choose a reason for hiding this comment

suztomo Apr 27, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

gcf-merge-on-green Bot commented Apr 27, 2022

mpeddada1 commented Apr 29, 2022

suztomo commented Apr 29, 2022

Uh oh!

Labels

3 participants

suztomo commented Apr 21, 2022 •

edited

Loading

suztomo commented Apr 25, 2022 •

edited

Loading

suztomo commented Apr 26, 2022 •

edited

Loading

suztomo Apr 27, 2022 •

edited

Loading