GitHub provider retrieves organization and team membership even when not included in claims

Hi there --

I'm trying to use GitHub as an OAuth2 provider, and only need the user's email address as a claim, so the user's orgs don't all have to grant consent before they can use my app. Setting "user:email" as the sole claim works, right up to this point:

https://github.com/oauth2-proxy/oauth2-proxy/blob/10b68716e53644a8fa0cbbaf156bf67ef93a017d/providers/github.go#L133

...where the provider attempts to retrieve the org and team memberships even if they're not in the requested claims.

The expected behaviour would be to either fail gracefully if the enriched fields were unreadable, or even better, to only enrich the fields that are addressed in the claims.