How to use Workload identity to access ESP in the Google Kubernetes Engine with the Google Cloud .NET SDK?

Solution for How to use Workload identity to access ESP in the Google Kubernetes Engine with the Google Cloud .NET SDK?
is Given Below:

Background

On the Google Kubernetes Engine we’ve been using Cloud Endpoints, and the Extensible Service Proxy (v2) for service-to-service authentication.

The services authenticate themselves by including the bearer JWT token in the Authorization header of the HTTP requests.

The identity of the services has been maintained with GCP Service Accounts, and during deployment, the Json Service Account key is mounted to the container at a predefined location, and that location is set as the value of the GOOGLE_APPLICATION_CREDENTIALS env var.

The services are implemented in C# with ASP.NET Core, and to generate the actual JWT token, we use the Google Cloud SDK (https://github.com/googleapis/google-cloud-dotnet, and https://github.com/googleapis/google-api-dotnet-client), where we call the following method:

var credentials = GoogleCredential.GetApplicationDefault();

If the GOOGLE_APPLICATION_CREDENTIALS is correctly set to the path of the Service Account key, then this returns a ServiceAccountCredential object, on which we can call the GetAccessTokenForRequestAsync() method, which returns the actual JWT token.

var jwtToken = await credentials.GetAccessTokenForRequestAsync("https://other-service.example.com/");
var authHeader = $"Bearer {jwtToken}";

This process has been working correctly without any issues.

The situation is that we are in the process of migrating from using the manually maintained Service Account keys to using Workload Identity instead, and I cannot figure out how to correctly use the Google Cloud SDK to generate the necessary JWT tokens in this case.

The problem

When we enable Workload Identity in the container, and don’t mount the Service Account key file, nor set the GOOGLE_APPLICATION_CREDENTIALS env var, then the GoogleCredential.GetApplicationDefault() call returns a ComputeCredential instead of a ServiceAccountCredential.
And if we call the GetAccessTokenForRequestAsync() method, that returns a token which is not in the JWT format.

I checked the implementation, and the token seems to be retrieved from the Metadata server, of which the expected response format seems to be the standard OAuth 2.0 model (represented in this model class):

{
  "access_token": "foo",
  "id_token": "bar",
  "token_type": "Bearer",
  ...
}

And the GetAccessTokenForRequestAsync() method returns the value of access_token. But as far as I understand, that’s not a JWT token, and indeed when I tried using it to authenticate against ESP, it responded with

{
 "code": 16,
 "message": "JWT validation failed: Bad JWT format: Invalid JSON in header",
 ..
}

As far as I understand, normally the id_token contains the JWT token, which should be accessible via the IdToken property of the TokenResponse object, which is also accessible via the SDK, I tried accessing it like this:

var jwtToken = ((ComputeCredential)creds.UnderlyingCredential).Token.IdToken;

But this returns null, so apparently the metadata server does not return anything in the id_token field.

Question

What would be the correct way to get the JWT token with the .NET Google Cloud SDK for accessing ESP, when using Workload Identity in GKE?

To get an IdToken for the attached service account, you can use GoogleCredential.GetApplicationDefault().GetOidcTokenAsync(...).