Skip to main content

Service accounts

A service account is a directory identity for a workload instead of a person. An agent, job, or service running in Kubernetes calls the AI Gateway and the Connector Gateway as its service account by presenting the projected ServiceAccount token Kubernetes gives its pod. There's no key to issue, store, or rotate. The account has its own groups, model set grants, and budget, and its spend is attributed to it.

Use a service account when a Kubernetes workload should act independently of any user. Use a virtual API key when a script or CI job should act as a person: the key carries its owner's access and budget, and the client stores it.

How a workload is recognized​

A workload's token signs it in when three conditions hold:

  • A trusted issuer tells Stacklok Enterprise which cluster's tokens to accept: the cluster's issuer URL, where its public signing keys are, and the audience workloads request their token for.
  • A binding maps the workload's Kubernetes ServiceAccount, by namespace and name and optionally its UID, to one service account.
  • The service account is Active.

After sign-in, the account's model access and budgets decide which models it can call, and its groups decide which connectors it reaches, as they do for a user. See What an account can reach.

Prerequisites​

  • Each gateway the workload calls, set up for workloads. For the AI Gateway, a cluster administrator completes Accept service account workloads once per installation. The Connector Gateway accepts workloads by default when it connects to the Enterprise Manager over TLS.
  • The platform admin grant, which covers directory administration.
  • The Kubernetes ServiceAccount the workload runs as. Stacklok Enterprise doesn't create it. Note its namespace and name.

In the console, select Administration under Experience in the account menu, then open Service Accounts.

Trust the cluster's issuer​

Each cluster whose workloads call the gateways needs one trusted issuer. You can also add an issuer from the Bindings step when you create an account.

  1. Open the Trusted issuers tab and select Add issuer.

  2. Choose where the issuer is:

    • This cluster trusts the workloads in the cluster that runs Stacklok Enterprise. Choose it only when every gateway also runs in this cluster. Each gateway reads the issuer's keys from the Kubernetes API of its own cluster, so a gateway in another cluster refuses this cluster's workloads. When the two clusters share an issuer URL, that gateway also accepts its own cluster's tokens under this issuer. For Issuer URL, enter the issuer this command prints, which on managed clusters such as Amazon EKS is a public URL:

      kubectl get --raw /.well-known/openid-configuration | jq -r .issuer

    • Another issuer trusts a cluster by its public Issuer URL and its public keys, given as a JWKS URL or with Upload JWKS. Use it for every other cluster, and also for the cluster that runs Stacklok Enterprise when gateways run in more than one cluster.

    The directory holds one trusted issuer per issuer URL, so each cluster needs a distinct issuer URL. Amazon EKS and Google Kubernetes Engine clusters already have one. For a self-managed cluster that keeps a default issuer URL, add a distinct value as the first --service-account-issuer flag of its API server and keep the old value after it, so tokens that pods already hold stay valid.

  3. Enter an Audience that only Stacklok Enterprise accepts, such as stacklok-enterprise-platform. Workloads request their token for this audience, so share it with the teams that deploy them.

  4. Select Add issuer.

Both gateways pick up a new or changed issuer on their next poll of the directory, every 30 seconds by default.

The default Signing algorithms, under Advanced, accept RS256. Change them only when the cluster signs its tokens with another type of key.

Disabling an issuer stops its workloads from signing in and keeps their bindings, so enabling it again lets them back in. You can delete an issuer only when no binding uses it.

Rotate an issuer's signing keys​

For an issuer with a JWKS URL or an uploaded JWKS, rotate its keys in this order so both gateways keep accepting the workloads' tokens. The cluster's administrator changes the signing key, so plan these steps with them:

  1. Add the new public key to the document at the JWKS URL, or upload a JWKS that holds both the new and the current key.
  2. Wait at least 15 minutes before signing tokens with the new key. A gateway that can't refresh an issuer's keys keeps using its cached copy for up to 15 minutes.
  3. Keep the retired key in the JWKS for at least 60 minutes after the last token signed with it, the longest token lifetime the gateways accept.

Create a service account​

On the Accounts tab, select New service account. The wizard walks through five steps, and every step after Details can be skipped and completed later from the account's tabs.

  1. Details: enter a Name and a description that says which workload uses the account.

  2. Connectors: add the account to groups, or create one, and choose each group's connectors. The account gets everything a group confers: its connectors, budget pool, and model set grants. A change to a group's connectors applies to every member of the group. The list leaves out groups managed through SCIM and connectors that can't serve a service account.

  3. Model Sets: grant model sets directly, and optionally set a Budget amount and period. The All Users model set and the organization default budget never apply to a service account. A workload that calls the AI Gateway therefore needs model set grants and a budget from this step or from its groups, unless the gateway turns those controls off, as described in What an account can reach. A workload that only calls connectors needs neither.

  4. Bindings: choose the Issuer URL, then enter the Namespace and ServiceAccount name of the workload's ServiceAccount. To pin the binding to that exact ServiceAccount, also enter its UID:

    kubectl get serviceaccount <SERVICE_ACCOUNT_NAME> -n <NAMESPACE> \
      -o jsonpath='{.metadata.uid}'

  5. Review: check the summary and select Create and activate service account.

Without a binding, the button reads Create service account and the account is created Suspended. A service account authenticates only while it's Active, and activating it requires at least one binding to an enabled issuer. Add a binding on the account's Bindings tab, then select Activate in the alert on its page.

Once the account is active, the team that deploys the workload configures it as described in Call the AI Gateway from Kubernetes workloads and Call the Connector Gateway from Kubernetes workloads. One token serves both gateways.

Choose between a pinned and a name-only binding​

A name-only binding matches any ServiceAccount with its namespace and name. That includes a ServiceAccount deleted and created again under the same name, and one created by anyone who can create ServiceAccounts in that namespace, which the built-in Kubernetes edit role allows. Pin the UID for any account whose grants or budget you wouldn't give to everyone who can create a ServiceAccount with that name in the namespace.

A pinned binding still admits every pod that runs as its ServiceAccount, and anyone who can create pods in the namespace, which the edit role also allows, can run a pod as that ServiceAccount and get its token. For an account with sensitive grants, also restrict who can create pods in that namespace.

A pinned binding takes precedence over a name-only binding for its UID. When another account already reaches the same ServiceAccount by name, the console asks you to confirm with Reassign. The new account takes over the ServiceAccount with that UID, and the other account keeps its name-only binding for any other ServiceAccount with that name, such as one created again later.

To change a binding, remove it from the account's Bindings tab and add a new one. Each change applies on the workload's next request.

Revoke a workload's access​

To stop a workload at once, suspend its account from the account's actions menu. The gateways refuse the workload's next request, and requests already admitted finish.

Removing the account's binding also stops the workload, unless another account has a name-only binding for the same ServiceAccount, as after a Reassign. That account then takes the workload over on its next request, so remove its binding too, or suspend the account instead.

Deleting the Kubernetes ServiceAccount doesn't revoke access right away. The gateways verify tokens by their signature, so tokens already issued stay valid until they expire, up to 60 minutes.

An account that loses its last binding, or whose issuers are all disabled, stays Active but its workloads are refused. Its page shows Active, but its workloads can't authenticate with a link to fix it.

Deleting a service account is permanent. It removes the account's bindings, group memberships, and model set grants, and deletes its own budget. The account is kept with the status Deleted, so its past spend remains attributed to it. The Accounts tab shows deleted accounts when you filter Status by Deleted.

What an account can reach​

A trusted issuer accepts tokens from every namespace in its cluster, and the gateways refuse any workload without an active, bound account.

An account reaches a connector only through a group the connector is granted to, and only when the connector uses None, Bearer token, or Header injection authentication. The other authentication types act as the calling person, and a service account has no person behind it. An account has no connections to make, so the gateway serves it every connector it reaches. The account's Connectors tab lists the connectors granted to its groups and the group each one comes through, marks the ones it can't use as Not available to service accounts, and lets you change its groups and their connectors. Connectors governed by a Cedar policy aren't listed there. A connector's Activity tab shows the account's tool calls under its name, with a Service account badge.

The AI Gateway's model access and budget settings decide which models the account can call:

Model accessBudgetsAn active, bound account can call
OnOnThe models its grants cover, within budget
OnOffThe models its grants cover, unmetered
OffOnAny model, within budget
OffOffAny model, unmetered

Users get the same treatment, so a gateway with neither model access nor budgets admits every bound workload as freely as every signed-in user. A cluster administrator sets both in the gateway's chart values. See Control model access and Enable the AI Gateway.

Next steps​

Troubleshooting​

Activate is unavailable

The account has no binding, or every issuer its bindings use is disabled. Add a binding on the account's Bindings tab, or enable an issuer on the Trusted issuers tab.

A workload is refused with 401 or 403

See Handle errors for the AI Gateway or the Connector Gateway for what each response means and where to find the reason.