ServiceOptions.getDefaultProjectId incorrectly defaults to GCE metadata before service account

[danielnorberg]

When running on a GCE host, ServiceOptions.getDefaultProjectId() incorrectly returns the project id from the GCE metadata server instead of defaulting to the project ID specified in the JSON credentials file pointed by the GOOGLE_APPLICATION_CREDENTIALS environment variable.

This seems to be due to getAppEngineProjectId() returning a project id even when running outside GAE, causing getDefaultProjectId() to not call getServiceAccountProjectId().

This seems to have been introduced in #3413 and resulted in the behavior changing from google-cloud-core 1.3.0 to 1.4.0.

Documentation:

• https://github.com/GoogleCloudPlatform/google-cloud-java#specifying-a-project-id
• https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/google-cloud-clients/google-cloud-core/src/main/java/com/google/cloud/ServiceOptions.java#L321

Repro: https://github.com/danielnorberg/google-cloud-java-default-project-bug-repro

[yihanzhen]

Compute Engine Project ID is looked up before the service account. We should adjust the documentation.

[danielnorberg]

The behavior used to match the documentation. And looking at the code, the current behavior seems non-intentional. Wouldn’t it make more sense to restore the previous behavior?

We have a fair amount of code in production relying on the documented behavior.

[yihanzhen]

Hi @danielnorberg ,

Sorry to see the problem is bothering you. However 1.4.0 was released almost a year ago and this part of the code has stayed unchanged since then. It wouldn't have been introduced by a rather recent change.

The behavior is actually intentional since the metadata server is the only way to determine the Project ID for GAE Flex and GAE 8. This behavior is pretty inseparable for those and GCE. We should have adjusted the documentations however, which we do apologize.

Alternatively, you could override the default project id but explicitly setting the project id using setProjectId and getProjectId.

[danielnorberg]

But then effectively when running on GCP, ServiceOptions.getDefaultProjectId() will never return the project id of the service account credentials. Is that desirable?

I understand that I can override this behavior in code that I have control over, but in my case I do not have full control over the code. I'm part of an infrastructure team operating a platform running code written by many teams across our organization, and this kind of breaking change is tricky for us to deal with.

[yihanzhen]

GAE/GCE credentials and project id would be considered default since that would make authentication work more seamlessly in most cases. This behavior is shared among libraries in multiple languages.