The Flyte platform consists of multiple components. Securing communication between each component is crucial to ensure
the integrity of the overall system.
The following diagram summarizes the components and their interactions as part of Flyte’s auth implementation:
In summary, there are two main resources required for a complete auth flow in Flyte:
An identity layer
Using an implementation of the Open ID Connect (OIDC) specification, it enables clients to verify the identity of the end user based on the authentication performed by an Authorization Server. For this flow to work, you must first register Flyte as a new client (app) to the Identity Provider (IdP).
An authorization server
As defined by IETF’s RFC #6749, the authorization server’s role is to issue access tokens to the client after successfully authenticating the resource owner and obtaining authorization. In this context, the resource owner is the end user of Flyte; and the client is the tool or component that intends to interact with flyteadmin : flytepropeller, flyteconsole or any of the CLI tools.
There are two supported options to use an authorization server in Flyte:
Internal authorization server: It comes pre-installed with Flyte and it is a suitable choice for quick start and testing purposes.
External (custom) authorization server: This a service provided by one of the supported IdPs and is the recommended option if your organization needs to retain control over scope definitions and grants, token expiration policies and other advanced security controls.
Note
Regardless of the type of authorization server to use, you will still need an IdP to provide identity through OIDC.
In this section, you can find canonical examples of how to set up OIDC on some of the supported IdPs; enabling users to authenticate in the
browser.
Create an OAuth2 Client Credential following the official documentation and take note of the client_id and client_secret
In the Authorized redirect URIs field, add http://localhost:30081/callback for sandbox deployments, or https://<your-deployment-URL>/callback for other methods of deployment.
If you don’t already have an Okta account, sign up for one here.
Create an app integration, with OIDC - OpenID Connect as the sign-on method and Web Application as the app type.
Add sign-in redirect URIs:
http://localhost:30081/callback for sandbox or https://<yourdeploymenturl>/callback for other Flyte deployment types.
Optional - Add logout redirect URIs:
http://localhost:30081/logout for sandbox, https://<yourdeploymenturl>/callback for other Flyte deployment types).
Take note of the Client ID and Client Secret
If you don’t have a Keycloak installation, you can use this which provides a quick way to deploy Keycloak cluster on AWS.
Create an OIDC client with client secret and note them down. Use the following instructions
Add Login redirect URIs:
http://localhost:30081/callback for sandbox or https://<yourdeploymenturl>/callback for other Flyte deployment types.
From the Azure homepage go to Azure Active Directory
From the Ovierview page, take note of the Tenant ID
Go to App registrations
Create a New registration
Give it a descriptive name
For the Supported account types select the option that matches your organization’s security policy
In the Redirect URI section select:
Web platform
Add http://localhost:30081/callback for sandbox or https://<yourdeploymenturl>/callback for other Flyte deployment types
Click on Register
Once created, click on the registered app and go to the Certificates and secrets section
Go to Client secrets and create a New client secret
Enter a description and an expiration policy
Take note of the secret Value as it will be used in the Helm chart
For further reference, check out the official Azure AD Docs on how to configure the IdP for OpenIDConnect.
Note
Make sure the app is registered without additional claims.
The OpenIDConnect authentication will not work otherwise, please refer to this GitHub Issue and Azure AD Docs for more information.
Go to your values file and locate the auth section and replace values accordingly:
auth:enabled:trueoidc:# baseUrl: https://accounts.google.com # Uncomment for Google# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta use the Issuer URI from Okta's default auth serverbaseUrl:https://dev-<org-id>.okta.com/oauth2/default# Replace with the client ID and secret created for Flyte in your IdPclientId:<client_ID>clientSecret:<client_secret>internal:clientSecret:'<your-random-password>'# Use the output of step #2 (only the content inside of '')clientSecretHash:<your-hashed-password>authorizedUris:-https://<your-flyte-deployment-URL>
Where flyte-namespace is the Kubernetes namespace where you have installed Flyte.
Add a new key under stringData:
apiVersion:v1# Add from herestringData:oidc_client_secret:<client_secret from the previous step># End heredata:...
Save and close your editor.
Go to your Helm values file and verify that the configmap section include the following, replacing the content where indicated:
configmap:adminServer:server:httpPort:8088grpcPort:8089security:secure:falseuseAuth:trueallowCors:trueallowedOrigins:# Accepting all domains for Sandbox installation-"*"allowedHeaders:-"Content-Type"auth:appAuth:thirdPartyConfig:flyteClient:clientId:flytectlredirectUri:http://localhost:53593/callbackscopes:-offline-allselfAuthServer:staticClients:flyte-cli:id:flyte-cliredirect_uris:-http://localhost:53593/callback-http://localhost:12345/callbackgrant_types:-refresh_token-authorization_coderesponse_types:-code-tokenscopes:-all-offline-access_tokenpublic:trueflytectl:id:flytectlredirect_uris:-http://localhost:53593/callback-http://localhost:12345/callbackgrant_types:-refresh_token-authorization_coderesponse_types:-code-tokenscopes:-all-offline-access_tokenpublic:trueflytepropeller:id:flytepropeller# Use the bcrypt hash generated for your random passwordclient_secret:"<bcrypt-hash>"redirect_uris:-http://localhost:3846/callbackgrant_types:-refresh_token-client_credentialsresponse_types:-tokenscopes:-all-offline-access_tokenpublic:falseauthorizedUris:# Use the public URL of flyteadmin (a DNS record pointing to your Ingress resource)-https://<your-flyte-deployment-URL>-http://flyteadmin:80-http://flyteadmin.flyte.svc.cluster.local:80userAuth:openId:# baseUrl: https://accounts.google.com # Uncomment for Google# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the default auth serverbaseUrl:https://dev-<org-id>.okta.com/oauth2/default# Use the client ID generated by your IdPclientId:<client_ID>scopes:-profile-openid
Additionally, outside the configmap section, add the following block and replace the necessary information:
secrets:adminOauthClientCredentials:# -- If enabled is true, helm will create and manage `flyte-secret-auth` and populate it with `clientSecret`.# If enabled is false, it's up to the user to create `flyte-secret-auth`enabled:true# Use the non-encoded version of the random passwordclientSecret:"<your-random-password>"clientId:flytepropeller
Save and exit your editor.
Restart flyteadmin for the changes to take effect:
kubectlrolloutrestartdeployment/flyteadmin-nflyte
Restart flytepropeller to start using authenticated requests:
For flytectl/pyflyte, make sure that your local config file ($HOME/.flyte/config.yaml) includes the following option:
admin:...authType:Pkce#change from the default `clientCred` to enable client auth without using shared secrets...
Note
Congratulations!
It should now be possible to go to Flyte UI and be prompted for authentication. Flytectl should automatically pickup the change and start prompting for authentication as well.
If you want to use an external OAuth2 provider for App authentication, please continue reading into the next section.
As mentioned previously, Flyte ships with an internal authorization server; hence setting up an external Authorization Server is optional and dependent on your organization’s security requirements.
In this section, you will find instructions on how to setup an OAuth2 Authorization Server in the different IdPs supported by Flyte:
Note
Google IdP
Google IdP does not offer an OAuth2 Authorization Server that could be used to protect external services (For example Flyte). In this case, Google offers a separate Cloud Product called Google Cloud Identity.
Configuration for Cloud Identity is not included in this guide. If unavailable, setup can stop here and FlyteAdmin BuiltIn OAuth2 Authorization Server can be used instead.
Okta’s custom authorization servers are available through an add-on license. The free developer accounts do include access, which you can use to test before rolling out the configuration more broadly.
From the left-hand menu, go to Security > API
Click on Add Authorization Server.
Assign an informative name and set the audience to the public URL of FlyteAdmin (e.g. https://example.foobar.com).
Note
The audience must exactly match one of the URIs in the authorizedUris section above
Note down the Issuer URI; this will be used for all the baseUrl settings in the Flyte config.
Go to Scopes and click Add Scope.
Set the name to all (required) and check Required under the User consent option.
Uncheck the Block services from requesting this scope option and save your changes.
Add another scope, named offline. Check both the Required and Include in public metadata options.
Uncheck the Block services from requesting this scope option.
Click Save.
Go to Access Policies, click Add New Access Policy. Enter a name and description and enable Assign to - Allclients.
Add a rule to the policy with the default settings (you can fine-tune these later).
Navigate back to the Applications section.
Create an integration for flytectl; it should be created with the OIDC - OpenID Connect sign-on method, and the Native Application type.
Add http://localhost:53593/callback to the sign-in redirect URIs. The other options can remain as default.
Assign this integration to any Okta users or groups who should be able to use the flytectl tool.
Note down the Client ID; there will not be a secret.
Create an integration for flytepropeller; it should be created with the OIDC - OpenID Connect sign-on method and Web Application type.
Check the ClientCredentials option under Client acting on behalf of itself.
This app does not need a specific redirect URI; nor does it need to be assigned to any users.
Note down the Client ID and Client secret; you will need these later.
Take note of the Issuer URI for your Authorization Server. It will be used as the baseURL parameter in the Helm chart
You should have three integrations total - one for the web interface (flyteconsole), one for flytectl, and one for flytepropeller.
If you don’t have a Keycloak installation, you can use this which provides quick way to deploy Keycloak cluster on AWS.
Create a realm in keycloak installation using its admin console
Under Client Scopes, click Add Create inside the admin console.
Create two clients (for flytectl and flytepropeller) to enable these clients to communicate with the service.
flytectl should be created with Access Type Public and standard flow enabled.
flytePropeller should be created as an Access Type Confidential, enabling the standard flow
Take note of the client ID and client Secrets provided.
Navigate to tab Overview, obtain <clientid> and <tenantid>
Navigate to tab Authentication, click +Addaplatform
Add Web for flyteconsole and flytepropeller, Mobile and desktop applications for flytectl.
Add URL https://<console-url>/callback as the callback for Web
Add URL http://localhost:53593/callback as the callback for flytectl
In Advanced settings, set Enablethefollowingmobileanddesktopflows to Yes to enable deviceflow
Navigate to tab Certificates & secrets, click +Newclientsecret to create <clientsecret>
Navigate to tab Token configuration, click +Addoptionalclaim and create email claims for both ID and Access Token
Navigate to tab API permissions, add email, offline_access, openid, profile, User.Read
Navigate to tab Expose an API, Click +Addascope and +Addaclientapplication to create <customscope>
Follow the steps in this section to configure flyteadmin to use an external auth server. This section assumes that you have already completed and applied the configuration for the OIDC Identity Layer.
Go to the values YAML file you used to install Flyte using a Helm chart
Find the auth section and follow the inline comments to insert your configuration:
auth:enabled:trueoidc:# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the custom auth server:baseUrl:https://dev-<org-id>.okta.com/oauth2/<auth-server-id># Use the client ID and secret generated by your IdP for the first OIDC registration in the "Identity Management layer : OIDC" section of this guideclientId:<oidc-clientId>clientSecret:<oidc-clientSecret>internal:# Use the clientID generated by your IdP for the flytepropeller app registrationclientId:<flytepropeller-client-id>#Use the secret generated by your IdP for flytepropellerclientSecret:'<flytepropeller-client-secret-non-encoded>'# Use the bcrypt hash for the clientSecretclientSecretHash:<-flytepropeller-secret-bcrypt-hash>authorizedUris:# Use here the exact same value used for 'audience' when the Authorization server was configured-https://<your-flyte-deployment-URL>
Find the inline section of the values file and add the following content, replacing where needed:
inline:auth:appAuth:authServerType:ExternalexternalAuthServer:# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the custom auth server:baseUrl:https://dev-<org-id>.okta.com/oauth2/<auth-server-id>metadataUrl:.well-known/oauth-authorization-serverthirdPartyConfig:flyteClient:# Use the clientID generated by your IdP for the `flytectl` app registrationclientId:<flytectl-client-id>redirectUri:http://localhost:53593/callbackscopes:-offline-alluserAuth:openId:# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the custom auth server:baseUrl:https://dev-<org-id>.okta.com/oauth2/<auth-server-id>scopes:-profile-openid# - offline_access # Uncomment if your IdP supports issuing refresh tokens (optional)# Use the client ID and secret generated by your IdP for the first OIDC registration in the "Identity Management layer : OIDC" section of this guideclientId:<oidc-clientId>
Save your changes
Upgrade your Helm release with the new configuration:
Find the auth section in your Helm values file, and replace the necessary data:
Note
If you were previously using the internal auth server, make sure to delete all the selfAuthServer section from your values file
configmap:auth:appAuth:authServerType:External# 2. Optional: Set external auth server baseUrl if different from OpenId baseUrl.externalAuthServer:# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the custom auth server:baseUrl:https://dev-<org-id>.okta.com/oauth2/<auth-server-id>metadataUrl:.well-known/openid-configurationthirdPartyConfig:flyteClient:# 3. Replace with a new Native/Public Client ID provisioned in the custom authorization server.clientId:flytectl# This should not changeredirectUri:http://localhost:53593/callback# 4. "all" is a required scope and must be configured in the custom authorization server.scopes:-offline-alluserAuth:openId:# baseUrl: https://<keycloak-url>/auth/realms/<keycloak-realm> # Uncomment for Keycloak and update with your installation host and realm name# baseUrl: https://login.microsoftonline.com/<tenant-id>/oauth2/v2.0/authorize # Uncomment for Azure AD# For Okta, use the Issuer URI of the custom auth server:baseUrl:https://dev-<org-id>.okta.com/oauth2/<auth-server-id>scopes:-profile-openid# - offline_access # Uncomment if OIdC supports issuing refresh tokens.clientId:<client id>secrets:adminOauthClientCredentials:enabled:true# see the section "Disable Helm secret management" if you require to do so# Replace with the client_secret provided by your IdP for flytepropeller.clientSecret:<client_secret># Replace with the client_id provided by provided by your IdP for flytepropeller.clientId:<client_id>
Save your changes
Upgrade your Helm release with the new configuration:
At this point, every interaction with Flyte components -be it in the UI or CLI- should require a successful login to your IdP, where your security policies are maintained and enforced.
Alternatively, you can instruct Helm not to create and manage the secret for flytepropeller. In that case, you’ll have to create it following these steps:
Disable Helm secrets management in your values file
secrets:adminOauthClientCredentials:enabled:false#set to false# Replace with the client_id provided by provided by your IdP for flytepropeller.clientId:<client_id>
Create a secret declaratively:
apiVersion:v1kind:Secretmetadata:name:flyte-secret-authnamespace:flytetype:OpaquestringData:# Replace with the client_secret provided by your IdP for flytepropeller.client_secret:<client_secret>
If your organization does any automated registration, then you’ll need to authenticate with the client credentials flow. After retrieving an access token from the IDP, you can send it along to flyteadmin` as usual.
admin:# Update with the Flyte's ingress endpoint (e.g. flyteIngressIP for sandbox or example.foobar.com)# You must keep the 3 forward-slashes after dns:endpoint:dns:///<Flyte ingress url># Update auth type to `Pkce` or `ClientSecret`authType:Pkce# Set to the clientId (will be used for both Pkce and ClientSecret flows)# Leave empty to use the value discovered through flyteAdmin's Auth discovery endpoint.clientId:<Id># Set to the location where the client secret is mounted.# Only needed/used for `ClientSecret` flow.clientSecretLocation:</some/path/to/key># If required, set the scopes needed here. Otherwise, flytectl will discover scopes required for OpenID# Connect through flyteAdmin's Auth discovery endpoint.# scopes: [ "scope1", "scope2" ]
To read further about the available config options, please
visit here
Flytekit configuration variables are automatically designed to look up values from relevant environment variables.
Important
However, to aid with continuous integration use-cases, Flytekit configuration can also reference other environment
variables.
For instance, if your CI system is not capable of setting custom environment variables like
FLYTE_CREDENTIALS_CLIENT_SECRET but does set the necessary settings under a different variable, you may use
exportFLYTE_CREDENTIALS_CLIENT_SECRET_FROM_ENV_VAR=OTHER_ENV_VARIABLE to redirect the lookup. A
FLYTE_CREDENTIALS_CLIENT_SECRET_FROM_FILE redirect is available as well, where the value should be the full
path to the file containing the value for the configuration setting, in this case, the client secret. We found
this redirect behavior necessary when setting up registration within our own CI pipelines.
The following is a listing of the Flytekit configuration values we set in CI, along with a brief explanation.
# When using OAuth2 service auth, this is the username and password.exportFLYTE_CREDENTIALS_CLIENT_ID=<client_id>
exportFLYTE_CREDENTIALS_CLIENT_SECRET=<client_secret>
# This tells the SDK to use basic authentication. If not set, Flytekit will assume you want to use the# standard OAuth based three-legged flow.exportFLYTE_CREDENTIALS_AUTH_MODE=basic
# This value should be set to conform to this# `header config <https://github.com/flyteorg/flyteadmin/blob/12d6aa0a419ccec81b4c8289fd172e70a2ded525/auth/config/config.go#L124-L128>`_# on the Admin side.exportFLYTE_CREDENTIALS_AUTHORIZATION_METADATA_KEY=<headername>
# When using basic authentication, you'll need to specify a scope to the IDP (instead of ``openid``, which is# only for OAuth). Set that here.exportFLYTE_CREDENTIALS_OAUTH_SCOPES=<idpdefinedscopes>
# Set this to force Flytekit to use authentication, even if not required by Admin. This is useful as you're# rolling out the requirement.exportFLYTE_PLATFORM_AUTH=True