Amazon Bedrock exposes several runtime API families. For a multi-turn chat application, AWS recommends Converse when the chosen model supports it because the message format stays consistent across model providers.
1. Verify Region, model, and API compatibility
Choose an AWS Region first, then verify the exact model ID or inference profile available there. Do not copy a model ID from another Region without checking support. Confirm that the model supports Converse, streaming, tool use, and the modalities your application needs.
For an initial smoke test, this guide uses Amazon Nova Micro. Replace the model ID with a supported model or inference profile from your target Region.
2. Grant least-privilege inference access
Converse is authorized through the underlying model-invocation actions. Scope permissions to the model or inference-profile resources the application actually uses.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"bedrock:InvokeModel",
"bedrock:InvokeModelWithResponseStream"
],
"Resource": "YOUR_MODEL_OR_INFERENCE_PROFILE_ARN"
}
]
}
Use an IAM role supplied by the workload environment. Do not store long-lived AWS access keys in application source code.
3. Send a Converse request with Boto3
Install and configure the AWS SDK for Python, then create a Bedrock Runtime client:
import boto3
from botocore.exceptions import ClientError
client = boto3.client("bedrock-runtime", region_name="us-east-1")
model_id = "amazon.nova-micro-v1:0"
messages = [
{
"role": "user",
"content": [{"text": "Explain this alert in one sentence."}],
}
]
try:
response = client.converse(
modelId=model_id,
messages=messages,
inferenceConfig={
"maxTokens": 300,
"temperature": 0.2,
"topP": 0.9,
},
requestMetadata={
"application": "alert-assistant",
"environment": "staging",
},
)
except ClientError as error:
# Log the AWS error code and request ID without logging sensitive prompts.
raise RuntimeError("Bedrock Converse request failed") from error
message = response["output"]["message"]
text = "".join(block.get("text", "") for block in message["content"])
print(text)
print(response.get("usage", {}))
print(response.get("metrics", {}))
Pass previous user and assistant messages back on the next call when your application owns conversation history. Bound that history by tokens, privacy rules, and retention policy.
4. Add streaming only when the UX needs it
Use converse_stream for incremental output. Check model streaming support first. Consume stream events by type, assemble text deltas by content-block index, and record the final stop reason, token usage, and latency metadata.
Streaming changes error handling: a request can fail before the stream opens or partway through consumption. Test both paths and make partial-output behavior explicit in the UI.
5. Handle production failure modes
- Retry throttling and transient service errors with capped exponential backoff and jitter.
- Do not blindly retry validation errors, access-denied errors, or unsupported model/API combinations.
- Set application deadlines and cancellation behavior around the SDK call.
- Validate tool-call inputs against your own schema and authorization policy before executing tools.
- Log model ID, Region, stop reason, token usage, latency, request ID, and a privacy-safe request correlation ID.
- Use
requestMetadatafor dimensions such as application and environment when model invocation logging is configured.
6. Add guardrails and evaluation deliberately
Bedrock Guardrails, prompt caching, service tiers, tool use, and model-specific request fields are not interchangeable across every model. Add each capability only after verifying support for the chosen model and testing both allowed and blocked cases.
Create a regression set for correctness, refusals, tool calls, latency, and cost. Compare Amazon Nova and other candidate models using the same requests before standardizing.