Instantiating a client with credentials from a service account credentials file does not use the right project name.

[tomthorogood]

Steps to reproduce

1. Using the vended service_account.Credentials.from_service_account_file, create a Credentials object that defines a different project than the configured environment. (As in "Someone from Project X gave me credentials so that I could access their datastore from my Project Y).
2. Instantiate a client with datastore.Client(credentials=credentials), where credentials is the output from Step chore: add split repo templates #1
3. assert client.project == credentials.project_id # AssertionError

However, this is only a part of the problem. I am able to construct and run read-only queries against datastore just fine with this mismatch. The issue strikes when I need to manipulate records, because the Batch object is performing a check to make sure that Batch.project == entity.key.project.

When I manipulate records that I fetched using a mis-configured client, the entities are correct, but the later-initialized Batch is then incorrect (and throws the error, Key must be from same project as batch) because the client is creating a Batch from its own configuration, and Batch performs a check that self.project == entity.key.project.

Code example

# From a GKE workload running on Project X, trying to manipulate a datastore on project Y
credentials = service_account.Credentials.from_service_account_file('/path/to/project-y-credentials.json')
client = datastore.Client(credentials=credentials)
print(client.project) # "project-x"
print(credentials.project_id) # "project-y"

This can easily be worked around by explicitly providing the project:

client = datastore.Cilent(credentials=credentials, project=credentials.project_id)

but is a step that nobody should have to take, and finding out that was the problem was time-consuming.

---

It would be great if, when provided a credentials object that defines a project, the datastore client could use credentials.project_id when determining the project name, so that distributed credentials properly configure their clients.

Thank you!

[HemangChothani]

@tomthorogood I am able to reproduce the scenario only if i follow these steps :
1- set the GOOGLE_APPLICATION_CREDENTIALS environment variable
2- use another file for creating credential : credentials = service_account.Credentials.from_service_account_file('/path/to/project-y-credentials.json

If you haven't set GOOGLE_APPLICATION_CREDENTIALS and try to create a client with credentials without passing project explicitly like client = datastore.Client(credentials=credentials) , it will throw an error.

Recommended:

1. If want to use from_service_account_file method then unset GOOGLE_APPLICATION_CREDENTIALS environment variable and create a client with credential and pass project id like client = datastore.Cilent(credentials=credentials, project=project_id)

For the issue Creating a client from explicit credentials requires passing project, the feature request has already open here googleapis/python-cloud-core#8

[tomthorogood]

I think it's fair to just make it a required parameter. That way at least folks won't go down too deep a rabbit hole before figuring it out. Thank you!

[HemangChothani]

Closing this issue as issue has opened in common lib.