AWS Bedrock

Description

The gateway translates the OpenAI chat-completions format to the AWS Bedrock InvokeModel API (the /model/<id>/invoke endpoint) and signs every request with AWS Signature Version 4 (SigV4). Calling applications send a normal OpenAI-format request; the gateway handles the conversion, the HMAC-SHA256 signing, and the upstream call.

Because each Bedrock model family expects a different request body, the gateway applies a per-family translation selected from the model identifier prefix: the Anthropic Messages body for anthropic., a Llama prompt string for meta., the Amazon Nova messages body for amazon., and a Mistral prompt string for mistral.. The cohere. prefix is recognised but is currently forwarded as a raw OpenAI body with no dedicated translation, so Cohere-on-Bedrock requests may be rejected by the upstream service. Other prefixes are routed as unknown and forwarded unchanged.

⚠️ Caution: Streaming is not implemented for AWS Bedrock. The Bedrock streaming response uses a binary event-stream protocol that the gateway does not yet decode. Requests with stream: true are silently downgraded — the upstream call is non-streaming and the response is returned as a single message.

Feature	Limitation
Chat completions through the InvokeModel API	—
Streaming responses	Not implemented; silently downgraded to non-streaming.
Permanent IAM credentials	—
Temporary STS credentials with session token	—
Per-family body translation	Anthropic, Llama (`meta.`), Amazon Nova, and Mistral only. `cohere.` is forwarded as a raw OpenAI body and may be rejected upstream.
Vision input	Depends on the underlying model.
Tool use	Not supported. No Bedrock family body builder serializes `tools`, so the tool loop never fires — the model answers in text even when tools are requested.

Required gateway configuration

Key	Type	Default	Description
`bedrock_region`	string	`us-east-1`	AWS region used to construct the Bedrock URL `https://bedrock-runtime.<REGION>.amazonaws.com`.

EU data residency. On a gateway with eu_region_routing: true, bedrock_region must be an EU-member region (eu-central-1, eu-west-1, eu-west-3, eu-north-1, eu-south-1, eu-south-2) — otherwise the request is rejected 403 data_residency_blocked and never reaches a US endpoint. eu-west-2 (London) and eu-central-2 (Zurich) are not EU-member and are rejected. See Data residency.

Under enforcement, only a bare on-demand model id (e.g. anthropic.claude-3-haiku-20240307-v1:0) is vouched — it stays in the one configured EU-27 region. AWS cross-region inference profiles (eu.anthropic.*, us.anthropic.*, global.anthropic.*) are refused: their destination set spans multiple regions and is not provably EU-27 (AWS's "EU" geography for Claude even routes to Zurich/CH). The gateway still serializes a profile id correctly (so a non-EU-enforced gateway may use one), but an EU-enforced gateway rejects it before any network call. Newer Claude ids that AWS offers only via a profile are therefore unavailable on Bedrock under strict EU-27 — use Vertex Gemini (europe-west*) for a big-vendor EU-27 route.

The bedrock_region key is not exposed in the Edit Gateway dialog of the SPA. Set it through the admin API:

curl -X PATCH "https://<your-gateway-host>/admin/v1/gateways/<ID>" \
  -H "Content-Type: application/json" \
  -d '{"config": {"bedrock_region": "us-west-2"}}'

BYOK key format

The BYOK value is the AWS access-key pair, in one of two forms.

For permanent credentials:

ACCESS_KEY_ID:SECRET_ACCESS_KEY

For temporary credentials issued by AWS Security Token Service (STS):

ACCESS_KEY_ID:SECRET_ACCESS_KEY:SESSION_TOKEN

The gateway parses the colon-separated string at request time and signs the upstream call with the credentials.

IAM requirements

The IAM principal associated with the credentials must have the bedrock:InvokeModel permission on the model ARNs the gateway calls:

{
  "Effect": "Allow",
  "Action": "bedrock:InvokeModel",
  "Resource": "arn:aws:bedrock:<REGION>::foundation-model/*"
}

Restrict the Resource to specific model ARNs for tighter access control:

"Resource": [
  "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0",
  "arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0"
]

💡 Note: Prefer short-lived STS credentials in production. Rotate credentials on a regular schedule.

Adding the key

The procedure for storing a BYOK key is the same for every provider. See Provider keys (BYOK). Select bedrock in the Provider drop-down list of the Add Model dialog and paste the colon-separated AWS credentials in the API Key field.

Through the API:

# Permanent credentials
curl -X POST "https://<your-gateway-host>/admin/v1/gateways/<ID>/keys" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "bedrock",
    "alias": "default",
    "key": "AKIAIOSFODNN7EXAMPLE:wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY"
  }'

# Temporary STS credentials
curl -X POST "https://<your-gateway-host>/admin/v1/gateways/<ID>/keys" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "bedrock",
    "alias": "sts",
    "key": "ASIA...EXAMPLE:secret...key:FwoGZXIvYXdzE..."
  }'

Inference endpoint

POST /v1/<TENANT>/<GATEWAY>/bedrock/chat/completions

The model field in the request body must be the full Bedrock model identifier, including the family prefix and version, for example anthropic.claude-3-5-sonnet-20241022-v2:0 or amazon.nova-pro-v1:0.

Request example

curl -X POST \
  "https://<your-gateway-host>/v1/myapp/production/bedrock/chat/completions" \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer <TOKEN>" \
  -d '{
    "model": "anthropic.claude-3-5-sonnet-20241022-v2:0",
    "messages": [
      {"role": "system", "content": "You are a helpful assistant."},
      {"role": "user",   "content": "What is Amazon Bedrock?"}
    ],
    "max_tokens": 512,
    "temperature": 0.7
  }'