Contact Us 1-800-596-4880
  1. Homepage
  2. Flex Gateway (1.12)
  3. LLM Proxy Overview

  4. LLM Proxy Policies

LLM Proxy Policies

By default, LLM Proxy applies these policies:

  • Client ID Enforcement

  • LLM Proxy Core Policy

  • Model Based Routing Policy or Semantic Routing Policy (policy name dependent on embedded service provider)

You don’t need to modify these policies.

LLM Proxy supports most included policies, but doesn’t support outbound policies.

LLM Proxy doesn’t support Rate Limiting: SLA-Based Policy.

These policies are specfic and useful for LLM Proxies:

Apply Policies to LLM Proxies

  1. From API Manager, click LLM Proxies.

  2. Click the name of the LLM Proxy you want to apply a policy to.

  3. Click AI Policies.

  4. Click + Add inbound policy.

  5. Select the policy to apply.

  6. Configure the required parameters.

    For policy configuration parameters, see Inbound Policies Directory.

  7. If necessary, configure Advanced options.

  8. Click Apply.

LLM Proxy Authentication Policy

By default, the LLM Proxy has the Client ID Enforcement policy applied. This is required because the Client ID Enforcement populates the Authentication.clientName variable in the Authentication object that is used as a unique identifier for LLM Metrics.

To remove the Client ID Enforcement policy, ensure that you either:

See Also