A team runs a Resilience Hub assessment. They get a score. Maybe they even fix a few findings. And then nothing happens. No aggregation. No cadence. No program. Just a snapshot that lives in one account, attached to one application, reviewed once.

I’ve seen this pattern enough times in financial services to know it’s not a tooling problem. It’s an organizational one. And if you want resilience that actually means something, you have to confront it directly.

Resilience Doesn’t Live in One Account

Modern financial systems don’t live in one place. They span line-of-business accounts, shared platform layers, data environments, and third-party dependencies. The critical user journeys — payments, trades, claims — cut across all of them.

Yet Resilience Hub is typically deployed against one app, in one account, with one assessment. That’s not resilience. That’s local optimization pretending to be a program.

As outlined in the Building Resilient Financial Services whitepaper, resilience must be measurable and demonstrable across interconnected services, not evaluated in isolation. A payment flow that touches API Gateway in one account, a processing Lambda in a shared services account, and DynamoDB in a data account doesn’t care that each of those components passed its individual resilience check. What matters is whether the flow as a whole can survive failure.

This is the gap Resilience Hub doesn’t close on its own. It was designed to evaluate applications, not programs. If you want the latter, you have to build it.

The Pattern That Actually Works: Hub-and-Spoke

If you want to scale resilience, you need to treat it like a distributed measurement system. Each account owns its applications and runs its own assessments. But the program — the aggregation, the scoring, the trend analysis — lives centrally.

This isn’t complicated architecture. It’s a standard IAM role in each spoke account that a central account can assume, a scheduled Lambda or Step Functions workflow that pulls assessment data across accounts and normalizes the structure, a data store in S3 or DynamoDB that tracks history (not just current state), and a dashboard layer — QuickSight or whatever your organization already uses — tied to critical services and business impact.

The hard part isn’t building it. The hard part is getting the organizational agreement that resilience data should flow centrally in the first place.

Resilience Hub Is a Data Source, Not a Dashboard

This is where I think most teams go wrong. They use Resilience Hub as a dashboard — open the console, look at the score, maybe screenshot it for a quarterly review. That’s underutilizing it significantly.

Resilience Hub is a modeling engine, a policy evaluator, and a scoring system. It is not your enterprise view, your governance layer, or your program. The real value emerges when you stop looking at the UI and start treating it as a data source.

The API surface is small and that’s a good thing:

client.list_apps()
client.list_app_assessments()
client.describe_app_assessment()

From those three calls, you can build enterprise-wide resilience scoring, trend analysis over time, drift detection across environments, and audit evidence tied to real systems. The UI tells you what happened once. The API lets you prove improvement over time — and that’s what regulators actually care about.

Where This Connects to Program Cadence

This is the part most organizations completely miss. Resilience is not a tool, a dashboard, or a quarterly checkbox. It’s a cadence.

I keep coming back to this when working with financial services clients: the difference between organizations that have a resilience program and organizations that have a resilience report is whether they review on a regular cadence and actually act on what they find.

Monthly reviews. Quarterly scenario testing. Continuous improvement loops. If you’re not doing these, you don’t have a program — you have a document that says you do.

From Assessment to Evidence

Regulators don’t care about your architecture diagrams. They care about whether services stay within tolerance, whether failures are tested, and whether you improve over time. These aren’t unreasonable asks, but they require evidence that most organizations can’t produce because they never built the infrastructure to collect it.

A multi-account Resilience Hub strategy gives you centralized visibility across distributed systems, consistent measurement aligned to business services, and — critically — evidence you can actually defend when someone asks how you know your systems are resilient.

Most organizations never make this leap. They run assessments, fix findings, and move on. But resilience doesn’t scale that way. It scales when you treat it as a system — measured continuously, aggregated centrally, and reviewed on a cadence.

AWS Resilience Hub gives you the raw signal. What you build around it determines whether you have a tool or a capability.

The AI project stalls. Not because the technology isn’t ready — because the governance layer isn’t there.

AWS Bedrock Guardrails is that governance layer. And in regulated environments like banking, insurance, and healthcare, it’s not optional — it’s the prerequisite for going to production. This post walks through what Guardrails actually does, how it works under the hood, why it matters specifically in financial services, and how to implement it with code you can actually deploy.

What Guardrails Solves

Let’s be direct about the problem. Large language models have four failure modes that matter most in regulated industries:

Harmful content generation: Even well-prompted models can produce hate speech, violent content, or guidance on misconduct if pushed in the right direction — especially in customer-facing contexts where you can’t predict every input.

Prompt injection and jailbreaks: Sophisticated users will attempt to override your system prompt, bypass your application logic, or extract information from your context window that they shouldn’t have. This isn’t theoretical — it’s the first thing a red team tests.

PII leakage: In a RAG system where your model has access to customer records, there’s a real risk of the model surfacing one customer’s information in another customer’s session, or including SSNs and account numbers in a response that gets logged, cached, or screenshotted.

Hallucination: For a general-purpose chatbot, hallucination is annoying. For a compliance assistant answering questions about regulatory requirements, it’s a liability.

Bedrock Guardrails addresses all four — as a managed layer that sits between your application and the foundation model, evaluating both input and output independently.

How It Works

The core mental model is simple: guardrails wrap the model invocation, not the model itself. You define a set of policies once, attach them to your Bedrock calls, and every prompt and every response gets evaluated against those policies before anything reaches the end user.

There are two evaluation passes:

• Input evaluation runs before the prompt reaches the foundation model. If the user’s message violates a policy, the model never sees it — you get a blocked message back immediately.
• Output evaluation runs after the model generates a response. The model might have produced something that passes input filters but fails on output — hallucinated content that contradicts your source documents, or a response that inadvertently includes PII from the retrieved context.

If either pass blocks, you get back a configurable message. The model response is never surfaced to the user.

The Six Policy Types

1. Content Filters

Detect and filter harmful content across six categories: Hate, Insults, Sexual, Violence, Misconduct, and Prompt Attack. Each category has an adjustable filter strength — Low, Medium, or High — so you can calibrate based on your use case. A customer service chatbot for a brokerage doesn’t need the same thresholds as an internal developer tool.

AWS extended content filtering to code-related content in 2025, which matters for any application where users can submit or request code. Harmful content in comments, variable names, and string literals is now caught at the same level as prose.

2. Prompt Attack Detection

This sits inside content filters but deserves its own callout. Jailbreaks and prompt injections are the most common adversarial inputs your application will face once it’s live. Guardrails detects both and gives you the option to block or log them — useful for incident response when your security team wants to know who tried what.

3. Denied Topics

Define topics that are off-limits in the context of your application. For a retail banking chatbot, this might be investment advice (FINRA), cryptocurrency recommendations, or competitor product comparisons. You describe the topic in plain language; AWS uses that description to classify user inputs and model responses.

This is one of the more powerful policy types for financial services, because it lets you draw a hard line around regulatory risk without having to enumerate every possible phrasing of a question.

4. Sensitive Information Filters (PII Redaction)

Bedrock Guardrails uses probabilistic ML detection to identify PII in both inputs and outputs. Predefined entity types include: SSN, Date of Birth, phone numbers, email addresses, credit card numbers, driver’s license numbers, bank account numbers, and more.

For anything not on the predefined list — like account routing numbers in a proprietary format, or internal employee IDs — you can add custom regex patterns.

When PII is detected, you have two options: block the entire message, or mask the sensitive fields and allow the rest through. Masking is useful for logging and audit scenarios where you want to retain the conversation structure without storing raw PII.

5. Contextual Grounding Checks

This is the hallucination filter, and it’s the most technically interesting policy type for RAG applications.

Contextual grounding checks compare the model’s response against two things: the source documents retrieved from your knowledge base, and the user’s original query. It generates two scores:

• Grounding score: How factually consistent is the response with the source material?
• Relevance score: Does the response actually answer what was asked?

You set a threshold between 0 and 0.99 for each. A response below either threshold gets blocked. AWS recommends starting around 0.7 for both and adjusting based on testing. In practice, this means if your compliance knowledge base says “employees must complete annual AML training,” and the model responds “employees should complete AML training within 90 days of hire” — that’s a grounding failure. The content is plausible; it’s just not what your source says. Contextual grounding catches it.

6. Automated Reasoning Checks

This is the newest and most powerful capability for factual accuracy. Where contextual grounding uses ML scoring, Automated Reasoning uses formal logic — encoding your organization’s policies as structured logical rules, then verifying model responses against those rules mathematically.

The practical implication: Automated Reasoning doesn’t just score a response; it can explain why a response is incorrect and what correction would make it valid. For HR policy bots, compliance Q&A systems, and any use case where you need to be able to show your work to an auditor, this is the capability that changes the conversation.

Implementation

Let’s make this concrete. Here’s a Terraform module that creates a guardrail configured for a financial services knowledge assistant:

resource "aws_bedrock_guardrail" "finserv_assistant" {
  name                      = "finserv-knowledge-assistant"
  description               = "Guardrails for retail banking knowledge assistant"
  blocked_input_messaging   = "I'm not able to help with that request. Please contact your relationship manager for assistance."
  blocked_outputs_messaging = "I wasn't able to generate a response that meets our quality standards. Please rephrase your question."

  # PII: block sensitive data in inputs, mask in outputs
  sensitive_information_policy_config {
    pii_entities_config {
      type   = "SSN"
      action = "BLOCK"
    }
    pii_entities_config {
      type   = "US_BANK_ACCOUNT_NUMBER"
      action = "ANONYMIZE"
    }
    pii_entities_config {
      type   = "CREDIT_DEBIT_CARD_NUMBER"
      action = "ANONYMIZE"
    }
    pii_entities_config {
      type   = "US_PASSPORT_NUMBER"
      action = "BLOCK"
    }
    # Custom regex for internal account IDs
    regexes_config {
      name        = "internal-account-id"
      description = "Internal account reference numbers"
      pattern     = "ACC-[0-9]{8}"
      action      = "ANONYMIZE"
    }
  }

  # Block investment advice topics — FINRA risk mitigation
  topic_policy_config {
    topics_config {
      name       = "investment-advice"
      definition = "Specific recommendations to buy, sell, or hold financial securities, stocks, bonds, mutual funds, ETFs, or other investment products."
      type       = "DENY"
      examples   = [
        "Should I buy more Apple stock?",
        "What funds should I put my 401k into?",
        "Is now a good time to sell my bonds?"
      ]
    }
    topics_config {
      name       = "competitor-products"
      definition = "Comparisons or recommendations involving competing financial institutions or their products."
      type       = "DENY"
    }
  }

  # Content filters — calibrated for customer-facing context
  content_policy_config {
    filters_config {
      type            = "HATE"
      input_strength  = "HIGH"
      output_strength = "HIGH"
    }
    filters_config {
      type            = "INSULTS"
      input_strength  = "MEDIUM"
      output_strength = "MEDIUM"
    }
    filters_config {
      type            = "PROMPT_ATTACK"
      input_strength  = "HIGH"
      output_strength = "NONE"
    }
    filters_config {
      type            = "VIOLENCE"
      input_strength  = "HIGH"
      output_strength = "HIGH"
    }
  }

  # Contextual grounding — prevent hallucination in RAG responses
  contextual_grounding_policy_config {
    filters_config {
      type      = "GROUNDING"
      threshold = 0.75
    }
    filters_config {
      type      = "RELEVANCE"
      threshold = 0.70
    }
  }

  # Encrypt guardrail configuration with customer-managed key
  kms_key_arn = aws_kms_key.bedrock_guardrail_key.arn

  tags = {
    Environment = "production"
    Application = "finserv-knowledge-assistant"
    Compliance  = "FFIEC"
  }
}

Now attach it to your Bedrock invocation:

import boto3
import json

bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")

def invoke_with_guardrails(prompt: str, source_documents: list[str]) -> dict:
    """
    Invoke a Bedrock model with Guardrails applied to both input and output.
    source_documents: list of text chunks retrieved from your knowledge base
    """
    guardrail_id = "your-guardrail-id"
    guardrail_version = "DRAFT"  # Use a pinned version in production

    # Format the request with grounding source for contextual checks
    grounding_source = "\n\n".join(source_documents)

    response = bedrock.invoke_model(
        modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
        guardrailIdentifier=guardrail_id,
        guardrailVersion=guardrail_version,
        body=json.dumps({
            "anthropic_version": "bedrock-2023-05-31",
            "max_tokens": 1024,
            "system": f'''You are a helpful assistant for retail banking customers.
Answer questions based only on the provided documentation.
If the answer is not in the documentation, say so clearly.

Documentation:
{grounding_source}''',
            "messages": [
                {
                    "role": "user",
                    "content": [
                        {
                            "type": "text",
                            "text": prompt,
                            "guardContent": {
                                "text": {
                                    "qualifiers": ["query"]
                                }
                            }
                        }
                    ]
                }
            ]
        }),
        contentType="application/json",
        accept="application/json"
    )

    result = json.loads(response["body"].read())

    # Check if guardrails intervened
    if response.get("amazon-bedrock-guardrailAction") == "INTERVENED":
        guardrail_trace = response.get("amazon-bedrock-trace", {})
        return {
            "blocked": True,
            "reason": guardrail_trace.get("guardrail", {}).get("actionReason", "Policy violation"),
            "response": None
        }

    return {
        "blocked": False,
        "reason": None,
        "response": result["content"][0]["text"]
    }

One thing worth calling out: the guardContent qualifier on the user message tells Guardrails which part of the prompt to evaluate for the relevance check. Without it, Guardrails would try to evaluate your entire system prompt (including the retrieved documents) as if it were the user query — which produces noisy results.

The Audit Trail

A guardrail that blocks requests is only half the picture. The other half is knowing what it blocked, when, and why.

Every Guardrail invocation emits metrics to Amazon CloudWatch:

• GuardrailInvocations — total count
• GuardrailInterventions — how many were blocked
• GuardrailIntervention[PolicyType] — breakdowns by policy

Set up a CloudWatch alarm on GuardrailInterventions spiking above your baseline and you have an early warning system for adversarial use or misconfigured prompts.

For a more complete audit trail — the kind that satisfies an FFIEC examiner or a SOC 2 auditor — route blocked events through EventBridge to an S3 bucket and query them with Athena. The pattern looks like this:

# Lambda function triggered by EventBridge rule on Bedrock Guardrail events
import boto3
import json
from datetime import datetime

s3 = boto3.client("s3")
AUDIT_BUCKET = "your-ai-audit-logs-bucket"

def handler(event, context):
    """Log guardrail interventions to immutable S3 audit trail."""
    
    audit_record = {
        "timestamp": datetime.utcnow().isoformat(),
        "guardrail_id": event.get("guardrailId"),
        "policy_triggered": event.get("policyType"),
        "action": event.get("action"),  # BLOCKED or ANONYMIZED
        "session_id": event.get("sessionId"),
        # Do NOT log the raw prompt — PII may not be fully redacted at this point
        "prompt_token_count": event.get("inputTokenCount"),
        "region": event.get("awsRegion"),
        "model_id": event.get("modelId")
    }

    key = f"guardrail-interventions/{datetime.utcnow().strftime('%Y/%m/%d')}/{context.aws_request_id}.json"
    
    s3.put_object(
        Bucket=AUDIT_BUCKET,
        Key=key,
        Body=json.dumps(audit_record),
        ServerSideEncryption="aws:kms",
        BucketKeyEnabled=True
    )

    return {"statusCode": 200}

This gives you an immutable, KMS-encrypted log of every guardrail intervention — queryable by date, policy type, model, or session ID without ever storing the raw prompt content.

What This Actually Changes for Banks

I keep coming back to one question in these conversations: what does it take to get an AI project from a successful proof-of-concept to a production system a compliance officer will sign off on?

The answer usually involves four things: data isolation, access controls, auditability, and behavioral controls. The first three are solved problems on AWS — VPC endpoints, IAM, CloudTrail. The fourth one — actually constraining what the model says and does — has historically required custom application logic that’s brittle, hard to test, and invisible to your governance team.

Bedrock Guardrails changes that. It gives you behavioral controls that are:

• Centralized. One guardrail definition applied consistently across every invocation, every session, every user.
• Versioned. You can pin a guardrail version to your production deployment and test changes in a draft version before promoting.
• Auditable. Every intervention is observable through CloudWatch metrics and loggable through EventBridge.
• Model-agnostic. The ApplyGuardrail API works independently of the foundation model — you can apply your guardrail to Claude, Titan, Llama, and even third-party models outside of Bedrock through the standalone API.

That last point matters more than it sounds. Most banks aren’t going to standardize on a single foundation model. As the model landscape evolves, your safety policies shouldn’t have to be rewritten every time you swap out the underlying model.

Getting Started

The fastest way to get a guardrail running is through the AWS console — there’s a test playground in the Guardrails UI where you can paste prompts and verify your policies before deploying. Start there, calibrate your contextual grounding thresholds against real examples from your knowledge base, then export the configuration to Terraform or CloudFormation for your production deployment. A few things to validate before go-live:

• Test your PII detection against real data samples (anonymized). The predefined entity types work well for standard formats; you’ll discover gaps quickly with actual data.
• Set your contextual grounding thresholds conservatively at first (0.7/0.7) and monitor your block rate. Too many false positives means end users get frustrated; too few means you’re letting hallucinations through.
• Verify your denied topics by trying to phrase a restricted question a dozen different ways. The topic detection is robust, but your definition matters — vague definitions lead to both over-blocking and under-blocking.

If you’re building in a regulated environment and you’re not running Guardrails, you’re carrying a liability that grows every day your AI system is in production. The capability exists. The question is whether you implement it before something goes wrong, or after.

On August 1, 2012, Knight Capital Group—one of the largest market makers on the New York Stock Exchange—lost $440 million in 45 minutes due to a software deployment failure. The incident nearly bankrupted the firm and sent shockwaves through financial markets. While the technical details are fascinating, the real lesson lies in what wasn’t there: an effective, centralized mechanism to stop runaway automation before catastrophic losses occurred.

This post explores how modern streaming architectures using Apache Kafka and Apache Spark can implement the kind of real-time risk controls that regulations now require—and that Knight Capital desperately needed. We’ll connect the dots between a historic trading disaster, regulatory requirements, and a hands-on demo you can deploy yourself.

The Knight Capital Incident: What Happened?

On that August morning, Knight Capital deployed new trading software to eight servers. Due to an operational error, one server retained old code that had been repurposed. When the market opened, this server began executing a dormant algorithm called “Power Peg” that was never meant to run in production.

The result was catastrophic:

• The algorithm sent millions of unintended orders to the market
• Knight accumulated massive, unwanted positions in 154 stocks
• The firm lost $440 million in approximately 45 minutes
• Knight Capital required a $400 million emergency bailout to survive

The Core Failure Modes

Several factors contributed to the disaster:

1. Partial Deployment: Not all servers received the correct code update
2. Lack of Centralized Control: No single point could halt all trading activity
3. Insufficient Pre-Trade Controls: Orders weren’t validated against risk limits before execution
4. Delayed Detection: The problem wasn’t identified and stopped quickly enough

The Knight Capital incident wasn’t just a software bug—it was a systems design failure. The firm lacked the architectural patterns needed to maintain “direct and exclusive control” over its market access, a concept that would soon become central to regulatory requirements.

Enter SEC Rule 15c3-5: The Market Access Rule

In response to concerns about the risks posed by direct market access and algorithmic trading, the SEC adopted Rule 15c3-5 in November 2010 (before the Knight incident, though Knight’s failure validated the rule’s necessity).

What is Market Access?

Market access refers to the ability to send orders directly to exchanges or alternative trading systems. Broker-dealers that provide market access—whether for their own trading or for customers—act as gatekeepers to the markets.

What the Rule Requires

SEC Rule 15c3-5, formally titled “Risk Management Controls for Brokers or Dealers with Market Access,” requires broker-dealers to:

1. Implement Risk Management Controls: Establish, document, and maintain a system of risk management controls and supervisory procedures reasonably designed to manage the financial, regulatory, and other risks of market access.

2. Pre-Trade Controls: Implement controls that prevent the entry of orders that exceed appropriate pre-set credit or capital thresholds, or that appear to be erroneous.

3. Direct and Exclusive Control: Broker-dealers must have “direct and exclusive control” over the technology that provides market access. This means they cannot delegate control to customers or third parties—they must retain the ability to stop trading immediately.

4. Regular Review: Controls must be reviewed and tested regularly to ensure they’re working as intended.

The “Direct and Exclusive Control” Concept

This phrase is critical. It means:

• The broker-dealer must be able to disable or limit market access immediately
• Control cannot be delegated to customers or outsourced
• There must be a centralized mechanism to enforce risk limits
• The firm must maintain supervisory procedures over all market access

The rule doesn’t prescribe specific technologies (it doesn’t say “you must use Kafka”), but it does mandate capabilities that modern streaming architectures are well-suited to provide.

Regulatory Text

From the SEC’s adopting release:

“The rule requires a broker-dealer with market access to establish, document, and maintain a system of risk management controls and supervisory procedures reasonably designed to manage the financial, regulatory, and other risks of this business activity.”

The rule specifically addresses:

• Financial risk management (credit and capital thresholds)
• Regulatory risk management (compliance with regulatory requirements)
• Erroneous order controls (preventing clearly erroneous orders from reaching the market)

Connecting Regulation to Architecture

Let’s translate regulatory requirements into architectural patterns:

The key insight: Separation of concerns between detection and enforcement.

• Detection (Spark): Analyzes order patterns, computes risk signals, may suggest kill actions
• Enforcement (Router): Makes the final decision on every order based on authoritative state
• Control Plane (Kill Switch): Maintains single source of truth for kill status
• Audit (Kafka + DynamoDB): Immutable record of all decisions

This separation ensures that even if detection fails or is delayed, enforcement remains consistent. The kill switch state is authoritative and replayable.

Why Kafka Compaction for Kill State?

One of the most interesting architectural choices in our demo is using a Kafka compacted topic for kill switch state. Here’s why:

The Problem

We need a “configuration” or “state” that:

• Is the single source of truth
• Can be updated in real-time
• Is immediately available to all consumers
• Has a complete audit trail
• Can be replayed to bootstrap new services

The Solution: Log Compaction

Kafka’s log compaction retains the latest value for each key while preserving the full history of changes. For kill switch state:

Compaction Example:

Commands Topic (Full History):
ACCOUNT:12345 KILL   t=100
ACCOUNT:12345 UNKILL t=200
ACCOUNT:12345 KILL   t=300

State Topic (Compacted - Latest Only):
ACCOUNT:12345 KILL   t=300

How it works:

• Key: Scope (e.g., “ACCOUNT:12345”, “SYMBOL:AAPL”, “GLOBAL”)
• Value: Current status (KILLED or ACTIVE) with metadata
• Compaction: Kafka automatically retains only the latest state per scope
• Replayability: New consumers can read the entire topic to bootstrap current state

This gives us:

1. Single source of truth: The compacted topic is authoritative
2. Fast bootstrap: New routers can quickly load all current kill states
3. Audit trail: The commands topic retains full history
4. Distributed config: No need for external config store

State Compaction Process

The following sequence diagram illustrates how Kafka’s log compaction maintains the latest state per scope:

sequenceDiagram
    participant K1 as Kafka<br/>killswitch.commands.v1<br/>(Full History)
    participant KSA as Kill Switch<br/>Aggregator
    participant K2 as Kafka<br/>killswitch.state.v1<br/>(Compacted)
    participant KC as Kafka<br/>Compaction Process
    participant OR as Order Router<br/>(New Instance)

    Note over K1,OR: State Evolution Over Time
    
    Note over K1: t=100
    K1->>KSA: KILL command<br/>ACCOUNT:12345
    KSA->>K2: Publish state<br/>Key: ACCOUNT:12345<br/>Value: KILLED (t=100)
    
    Note over K1: t=200
    K1->>KSA: UNKILL command<br/>ACCOUNT:12345
    KSA->>K2: Publish state<br/>Key: ACCOUNT:12345<br/>Value: ACTIVE (t=200)
    
    Note over K1: t=300
    K1->>KSA: KILL command<br/>ACCOUNT:12345
    KSA->>K2: Publish state<br/>Key: ACCOUNT:12345<br/>Value: KILLED (t=300)
    
    Note over K2: Before Compaction:<br/>ACCOUNT:12345 KILLED (t=100)<br/>ACCOUNT:12345 ACTIVE (t=200)<br/>ACCOUNT:12345 KILLED (t=300)
    
    K2->>KC: Compaction triggered<br/>(based on segment.ms<br/>and dirty ratio)
    
    KC->>KC: Retain latest value<br/>per key
    
    Note over K2: After Compaction:<br/>ACCOUNT:12345 KILLED (t=300)<br/>(older values removed)
    
    Note over OR: New router starts up
    OR->>K2: Read from beginning
    K2-->>OR: ACCOUNT:12345 = KILLED (t=300)
    
    Note over OR: Router bootstrapped<br/>with current state<br/>(fast, no history to read)
    
    Note over K1: Commands topic still has<br/>full history for audit

Compaction Configuration

cleanup.policy=compact
min.cleanable.dirty.ratio=0.01  # Compact frequently
segment.ms=60000                # Small segments for faster compaction

These settings ensure kill state updates propagate quickly while maintaining the full history in the commands topic.

Demo Architecture Walkthrough

Our demo implements these patterns using serverless AWS services:

Key architectural decisions mapped to regulatory requirements:

Normal Order Flow

The following sequence diagram illustrates how orders flow through the system when no kill switches are active:

sequenceDiagram
    participant OG as Order Generator<br/>(Lambda)
    participant K1 as Kafka<br/>orders.v1
    participant S as Spark<br/>Risk Detector
    participant K2 as Kafka<br/>risk_signals.v1
    participant OR as Order Router<br/>(Lambda)
    participant KS as Kafka<br/>killswitch.state.v1
    participant K3 as Kafka<br/>orders.gated.v1
    participant K4 as Kafka<br/>audit.v1
    participant DDB as DynamoDB<br/>Audit Index

    Note over OG,DDB: Normal Operation - No Kill Switches Active
    
    OG->>K1: Publish order<br/>(5 orders/sec)
    Note right of K1: Key: account_id<br/>Partition by account
    
    K1->>S: Consume orders
    S->>S: Compute 60s window<br/>order_count = 50<br/>notional = $500K
    Note right of S: Below thresholds:<br/>order_rate < 100<br/>notional < $1M
    S->>K2: Publish risk signal<br/>(metrics only, no alert)
    
    K1->>OR: Consume order
    OR->>KS: Check kill state<br/>for ACCOUNT:12345
    KS-->>OR: No kill state found<br/>(ACTIVE by default)
    
    Note over OR: Decision: ALLOW
    
    OR->>K3: Forward order<br/>to gated topic
    OR->>K4: Publish audit event<br/>decision=ALLOW
    OR->>DDB: Write audit record<br/>(async, best effort)
    
    Note over OG,DDB: Order successfully routed

Topic Flow

1. orders.v1: Raw orders from generator
2. risk_signals.v1: Windowed aggregations from Spark (order rate, notional, concentration)
3. killswitch.commands.v1: Kill/unkill commands (from Spark or operator)
4. killswitch.state.v1: Authoritative kill state (compacted)
5. orders.gated.v1: Orders that passed kill switch check
6. audit.v1: Immutable audit trail of all routing decisions

Order Router Enforcement

The following diagram shows the detailed logic of how the order router enforces kill switches:

sequenceDiagram
    participant K1 as Kafka<br/>orders.v1
    participant OR as Order Router<br/>(Lambda)
    participant Cache as In-Memory<br/>Kill State Cache
    participant K2 as Kafka<br/>killswitch.state.v1
    participant K3 as Kafka<br/>orders.gated.v1
    participant K4 as Kafka<br/>audit.v1
    participant DDB as DynamoDB<br/>Audit Index

    Note over K1,DDB: Order Router Processing Logic
    
    K1->>OR: Consume order<br/>account_id: 12345<br/>symbol: AAPL
    
    OR->>Cache: Check kill state<br/>for scopes
    
    Note over Cache: Check hierarchy:<br/>1. GLOBAL<br/>2. ACCOUNT:12345<br/>3. SYMBOL:AAPL
    
    alt GLOBAL kill active
        Cache-->>OR: GLOBAL = KILLED
        Note over OR: Decision: DROP<br/>Reason: Global kill
    else ACCOUNT kill active
        Cache-->>OR: ACCOUNT:12345 = KILLED
        Note over OR: Decision: DROP<br/>Reason: Account kill
    else SYMBOL kill active
        Cache-->>OR: SYMBOL:AAPL = KILLED
        Note over OR: Decision: DROP<br/>Reason: Symbol kill
    else No kills active
        Cache-->>OR: All scopes ACTIVE
        Note over OR: Decision: ALLOW
        OR->>K3: Forward order
    end
    
    OR->>K4: Publish audit event<br/>decision: ALLOW/DROP<br/>scope_matches: [...]<br/>corr_id: uuid-789
    
    OR->>DDB: Write audit record<br/>(async, best effort)
    
    Note over K2,OR: State updates arrive
    K2->>OR: New state update
    OR->>Cache: Update in-memory cache
    
    Note over Cache: Cache always reflects<br/>latest compacted state

Latency Considerations

This design introduces additional latency relative to in-process risk checks. However, it provides centralized, authoritative enforcement and replayable state — properties essential for satisfying the “direct and exclusive control” requirement of SEC Rule 15c3-5.

In most retail and DMA (Direct Market Access) environments, the added milliseconds (approximately 50ms at most) are an acceptable tradeoff for deterministic control and auditability. This approach is not intended to reflect the architecture of any former employer but rather examines how brokerages can solve these regulatory challenges in a robust, scalable way.

Kill Switch Activation Sequence

Here’s what happens when Spark detects a threshold breach:

sequenceDiagram
    participant OG as Order Generator<br/>(Lambda)
    participant K1 as Kafka<br/>orders.v1
    participant S as Spark<br/>Risk Detector
    participant K2 as Kafka<br/>risk_signals.v1
    participant K3 as Kafka<br/>killswitch.commands.v1
    participant KSA as Kill Switch<br/>Aggregator (Lambda)
    participant K4 as Kafka<br/>killswitch.state.v1
    participant DDB as DynamoDB<br/>State Cache
    participant OR as Order Router<br/>(Lambda)
    participant K5 as Kafka<br/>audit.v1

    Note over OG,K5: Panic Mode Triggered
    
    OG->>K1: Publish orders<br/>(50 orders/sec)
    Note right of K1: High rate for<br/>ACCOUNT:12345
    
    K1->>S: Consume orders
    S->>S: Compute 60s window<br/>order_count = 150<br/>notional = $2.5M
    
    Note over S: BREACH DETECTED!<br/>order_count > 100
    
    S->>K2: Publish risk signal<br/>with breach flag
    S->>K3: Publish KILL command<br/>scope: ACCOUNT:12345<br/>reason: "Order rate breach"<br/>corr_id: uuid-123
    
    Note over K3: Commands topic<br/>(full history retained)
    
    K3->>KSA: Consume KILL command
    KSA->>KSA: Process command<br/>Create state record
    
    KSA->>K4: Publish state<br/>Key: ACCOUNT:12345<br/>Value: KILLED<br/>corr_id: uuid-123
    Note right of K4: Compacted topic<br/>(latest state per key)
    
    KSA->>DDB: Update state cache<br/>(optional, for fast lookup)
    
    Note over K4,OR: State propagates to all routers
    
    K4->>OR: Router reads state update
    OR->>OR: Update in-memory cache<br/>ACCOUNT:12345 = KILLED
    
    K1->>OR: New order from 12345
    OR->>OR: Check kill state<br/>ACCOUNT:12345 = KILLED
    
    Note over OR: Decision: DROP
    
    OR->>K5: Publish audit event<br/>decision=DROP<br/>reason: "Kill switch active"<br/>corr_id: uuid-123
    
    Note over OG,K5: Order blocked - Kill switch active

Key observations:

• Detection (Spark) is decoupled from enforcement (Router)
• State updates flow through compacted topic (single source of truth)
• Every decision is audited with correlation IDs
• Manual override capability (operator can unkill)

Spark SQL for Risk Detection

The Spark job uses Spark SQL for windowed aggregations:

SELECT
    window(event_time, '60 seconds') as window,
    account_id,
    COUNT(*) as order_count,
    SUM(qty * price) as total_notional,
    COUNT(DISTINCT symbol) as unique_symbols
FROM orders
GROUP BY window(event_time, '60 seconds'), account_id

When thresholds are breached, Spark emits a kill command:

{
  "cmd_id": "uuid",
  "scope": "ACCOUNT:12345",
  "action": "KILL",
  "reason": "Order rate breach: 150 orders in 60s",
  "triggered_by": "spark",
  "metric": "order_rate_60s",
  "value": 150
}

Enforcement Logic

The order router maintains an in-memory cache of kill state (bootstrapped from the compacted topic) and checks every order:

def check_kill_status(order):
    scopes = [
        'GLOBAL',
        f'ACCOUNT:{order["account_id"]}',
        f'SYMBOL:{order["symbol"]}'
    ]
    
    for scope in scopes:
        if scope in kill_state and kill_state[scope]['status'] == 'KILLED':
            return True, scope, kill_state[scope]['reason']
    
    return False, None, None

Every decision is audited with correlation IDs for traceability.

Alternative Approaches for High-Frequency Trading

For high-frequency trading environments, the architecture described above would introduce unacceptable latency. In these ultra-low-latency scenarios, the pre-trade risk gate would be embedded directly in the order handling process—potentially implemented in hardware (e.g., FPGA)—to ensure deterministic, microsecond-level enforcement without introducing network or broker latency.

Key differences in HFT implementations:

• Embedded Controls: Risk checks directly in the order path, not as external services
• Hardware Acceleration: FPGAs or dedicated ASICs for microsecond-level checks
• Local State: State maintained in local memory with minimal or no network calls
• Minimal Serialization: Custom binary protocols instead of JSON
• Deterministic Performance: Bounded, predictable latency for all operations

In such environments, you wouldn’t add Kafka to the hot path, as even the most optimized message broker would introduce unacceptable latency. Instead, while still maintaining the regulatory requirements for “direct and exclusive control,” risk configurations would be loaded at startup and updated via side channels, with enforcement happening directly within the order processing pipeline.

Why This Matters for Students

This demo teaches several critical concepts:

1. Event-Driven Architecture: Using Kafka as the backbone for real-time systems
2. Stream Processing: Spark Structured Streaming for windowed aggregations
3. Separation of Concerns: Detection vs. enforcement vs. control
4. Operational Patterns: Compaction, idempotency, correlation IDs
5. Regulatory Thinking: How compliance requirements shape architecture
6. Serverless at Scale: Building production-grade systems without managing servers

Most importantly, it connects abstract concepts (regulations, risk management) to concrete implementations you can deploy and experiment with.

Try It Yourself

The complete demo is available in the repository. You can:

1. Deploy to AWS: Full serverless stack with Terraform
2. Run locally: Docker Compose for quick iteration
3. Experiment: Change thresholds, add new scopes, implement throttling
4. Learn: Detailed workshop docs with exercises

See the repository for step-by-step instructions.

Suggested Exercises

1. Add a SYMBOL-level kill switch that triggers on concentration
2. Implement throttling (rate limiting) instead of binary kill/allow
3. Add deduplication to prevent duplicate order IDs
4. Build a dashboard to visualize risk signals in real-time
5. Implement automatic unkill after a cooldown period

Conclusion

The Knight Capital incident taught the industry a painful lesson about the importance of centralized control and pre-trade risk management. SEC Rule 15c3-5 codified these lessons into regulatory requirements that all broker-dealers must follow.

Modern streaming architectures using Kafka and Spark provide elegant solutions to these requirements:

• Kafka’s compacted topics give us authoritative, replayable state
• Spark’s streaming SQL enables real-time risk detection
• Separation of detection and enforcement ensures consistent control
• Immutable audit trails provide full traceability

While this demo uses synthetic data and simplified logic, the architectural patterns are production-grade. Real broker-dealers use similar approaches to maintain the “direct and exclusive control” that regulations require and that Knight Capital lacked.

The next time you hear about a trading glitch or market disruption, ask: “Where was the kill switch?”

Sources and Further Reading

Primary Regulatory Sources

1. SEC Rule 15c3-5 Final Adopting Release
   Securities and Exchange Commission, Release No. 34-63241 (November 3, 2010)
   https://www.sec.gov/files/rules/final/2010/34-63241.pdf

2. SEC Small Entity Compliance Guide for Rule 15c3-5
   https://www.sec.gov/files/rules/final/2010/34-63241-secg.htm

3. Code of Federal Regulations: 17 CFR § 240.15c3-5
   https://www.law.cornell.edu/cfr/text/17/240.15c3-5

Knight Capital Incident

4. SEC Administrative Proceeding Against Knight Capital
   File No. 3-15570 (October 16, 2013)
   Details the regulatory findings and penalties related to the incident.

5. Nanex Research: Knight Capital’s Trading Glitch
   Technical analysis of the order flow during the incident (secondary source).

Technical Resources

6. Apache Kafka Documentation: Log Compaction
   https://kafka.apache.org/documentation/#compaction

7. Apache Spark Structured Streaming Guide
   https://spark.apache.org/docs/latest/structured-streaming-programming-guide.html

Disclaimer: This blog post and associated demo are for educational purposes only. They do not constitute trading advice, legal advice, or compliance guidance. The architecture described does not represent any former employer’s actual systems or implementations. The demo uses synthetic data and simplified logic to illustrate concepts rather than real production implementations. Actual production trading systems require extensive additional controls, testing, and regulatory review. Always consult with legal and compliance professionals when implementing market access systems.

This post demonstrates how to build an automated agent that analyzes AWS costs and generates actionable recommendations to reduce cloud spend. By combining AWS Bedrock’s analytical capabilities with Cost Explorer data, the system identifies cost outliers and provides specific optimization steps that go beyond basic visualizations to deliver meaningful insights.

What we’re building

A cost optimization agent that:

1. Runs on a schedule (weekly or monthly)
2. Fetches detailed cost data from AWS Cost Explorer
3. Analyzes spending patterns and identifies optimization opportunities
4. Generates a markdown report with specific recommendations
5. Sends a summary to Slack or email
6. Tracks recommendations and their potential savings

This solution goes beyond basic cost visualization by providing specific, actionable steps to optimize AWS spending—essentially turning data into decisions.

Real-World Applications

This solution addresses cost management challenges across different contexts:

Enterprise FinOps Teams: In larger organizations, the agent provides consistent, ongoing cost analysis that augments the FinOps team’s capabilities, ensuring no optimization opportunity goes unnoticed even as the infrastructure grows in complexity.

Startups and Small Teams: For organizations without dedicated cloud financial analysts, the agent provides expert-level cost optimization recommendations that would otherwise require specialized knowledge or expensive consultants.

Managed Service Providers: MSPs can deploy the agent across client environments, standardizing cost optimization practices while customizing thresholds and priorities for each client’s specific needs.

Development Environments: The agent can enforce stricter cost controls in non-production environments, identifying development and testing resources that can be safely downsized, scheduled, or terminated to reduce costs without affecting production workloads.

Multi-Cloud Strategies: While this implementation focuses on AWS, the architecture pattern can be extended to analyze costs across multiple cloud providers, giving organizations a unified view of optimization opportunities.

Architecture overview

Here’s the high-level architecture:

EventBridge (scheduled) → Lambda → Bedrock Agent with Action Groups → S3 (report) → SNS (notifications)

The key components:

• EventBridge: Triggers the agent on a schedule
• Lambda: Initializes and coordinates the analysis process
• Bedrock Agent: Orchestrates the data gathering and analysis
• Action Groups: Custom Lambda functions for specific tasks
• S3: Stores the generated reports
• SNS: Sends notifications with the summary
• DynamoDB: Tracks recommendations and their implementation status

This event-driven architecture ensures the cost optimization process runs automatically on schedule, eliminating the need for manual intervention and ensuring consistent analysis.

What you’ll need

• AWS Account with Bedrock and Cost Explorer access
• IAM role with appropriate permissions
• S3 bucket for storing reports
• SNS topic or email for notifications
• Basic understanding of AWS services

Step 1: Create the Action Group Lambda functions

First, let’s create the Lambda functions our Bedrock Agent will use as action groups:

1. Cost Data Retrieval Lambda

This function fetches comprehensive cost data from AWS Cost Explorer:

import json
import os
import boto3
import datetime
from dateutil.relativedelta import relativedelta

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        time_period = payload.get('time_period', 'MONTH')
        services = payload.get('services', [])
        
        # Configure time period based on request
        end_date = datetime.datetime.now()
        
        if time_period == 'MONTH':
            start_date = end_date - relativedelta(months=1)
            granularity = 'DAILY'
        elif time_period == 'QUARTER':
            start_date = end_date - relativedelta(months=3)
            granularity = 'MONTHLY'
        elif time_period == 'WEEK':
            start_date = end_date - relativedelta(weeks=1)
            granularity = 'DAILY'
        else:
            # Default to monthly
            start_date = end_date - relativedelta(months=1)
            granularity = 'DAILY'
        
        # Format dates for Cost Explorer
        start_str = start_date.strftime('%Y-%m-%d')
        end_str = end_date.strftime('%Y-%m-%d')
        
        # Initialize Cost Explorer client
        ce_client = boto3.client('ce')
        
        # Get cost data, anomalies, and recommendations
        response = get_cost_data(ce_client, start_str, end_str, granularity)
        anomalies = get_anomalies(ce_client, start_str, end_str)
        reservation_recs = get_reservation_recommendations(ce_client)
        savings_plans_recs = get_savings_plans_recommendations(ce_client)
        
        # Return the compiled data
        return {
            'statusCode': 200,
            'body': json.dumps({
                'cost_data': response,
                'anomalies': anomalies,
                'reservation_recommendations': reservation_recs,
                'savings_plans_recommendations': savings_plans_recs
            })
        }
        
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

# Helper functions (implementation details omitted for brevity)
def get_cost_data(ce_client, start_str, end_str, granularity):
    # Implementation details omitted
    return {}

def get_anomalies(ce_client, start_str, end_str):
    # Implementation details omitted
    return []

def get_reservation_recommendations(ce_client):
    # Implementation details omitted
    return []

def get_savings_plans_recommendations(ce_client):
    # Implementation details omitted
    return []

2. Resource Analysis Lambda

This function analyzes AWS resources for optimization opportunities:

import json
import boto3
import datetime

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        resource_types = payload.get('resource_types', ['ec2', 'rds', 'ebs', 'lambda'])
        
        results = {}
        
        # Analyze each resource type
        if 'ec2' in resource_types:
            results['ec2'] = analyze_ec2_instances()
        
        if 'rds' in resource_types:
            results['rds'] = analyze_rds_instances()
        
        if 'ebs' in resource_types:
            results['ebs'] = analyze_ebs_volumes()
        
        if 'lambda' in resource_types:
            results['lambda'] = analyze_lambda_functions()
        
        return {
            'statusCode': 200,
            'body': json.dumps(results)
        }
        
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

def analyze_ec2_instances():
    ec2_client = boto3.client('ec2')
    cloudwatch = boto3.client('cloudwatch')
    
    # Get instances and identify optimization opportunities
    instances = get_all_instances(ec2_client)
    low_utilization = find_low_utilization_instances(instances, cloudwatch)
    potential_downsizing = find_downsizing_opportunities(instances, cloudwatch)
    
    return {
        'total_instances': len(instances),
        'running_instances': count_running_instances(instances),
        'stopped_instances': count_stopped_instances(instances),
        'low_utilization_instances': low_utilization,
        'potential_downsizing': potential_downsizing
    }

# Helper functions (implementation details omitted for brevity)
def get_all_instances(ec2_client):
    # Implementation details omitted
    return []

def find_low_utilization_instances(instances, cloudwatch):
    # Implementation details omitted
    return []

def find_downsizing_opportunities(instances, cloudwatch):
    # Implementation details omitted
    return []

def count_running_instances(instances):
    # Implementation details omitted
    return 0

def count_stopped_instances(instances):
    # Implementation details omitted
    return 0

def analyze_rds_instances():
    # Implementation details omitted
    return {}

def analyze_ebs_volumes():
    # Implementation details omitted
    return {}

def analyze_lambda_functions():
    # Implementation details omitted
    return {}

3. Report Generation Lambda

This function generates cost optimization reports and sends notifications:

import json
import boto3
import os
import time
from datetime import datetime

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        cost_data = payload.get('cost_data', {})
        resource_analysis = payload.get('resource_analysis', {})
        destination = payload.get('destination', 'S3')
        
        # Generate detailed markdown report
        report_content = generate_markdown_report(cost_data, resource_analysis)
        
        # Create a unique filename with timestamp
        timestamp = datetime.now().strftime('%Y-%m-%d-%H-%M-%S')
        filename = f"cost-optimization-report-{timestamp}.md"
        
        results = {}
        
        # Save to S3 if requested
        if destination in ['S3', 'BOTH']:
            s3_url = save_to_s3(report_content, filename)
            results['s3_url'] = s3_url
        
        # Send notification if requested
        if destination in ['SNS', 'BOTH']:
            summary = generate_summary(cost_data, resource_analysis)
            if 's3_url' in results:
                summary += f"\n\nDetailed report: {results['s3_url']}"
            
            send_notification(summary, timestamp)
            results['sns_notification'] = 'Sent'
        
        # Store recommendations in DynamoDB for tracking
        store_recommendations(cost_data, resource_analysis)
        
        return {
            'statusCode': 200,
            'body': json.dumps(results)
        }
        
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

# Helper functions (implementation details omitted for brevity)
def generate_markdown_report(cost_data, resource_analysis):
    # Implementation details omitted
    return ""

def save_to_s3(report_content, filename):
    # Implementation details omitted
    return ""

def generate_summary(cost_data, resource_analysis):
    # Implementation details omitted
    return ""

def send_notification(summary, timestamp):
    # Implementation details omitted
    pass

def store_recommendations(cost_data, resource_analysis):
    # Implementation details omitted
    pass

Step 2: Set up DynamoDB for tracking recommendations

You’ll need DynamoDB tables to track reports and recommendations. In production, you’d define these in your infrastructure-as-code using Terraform or CloudFormation. The tables need:

1. CostOptimizationRecommendations table:

   • Partition key: recommendation_id (String)
   • PAY_PER_REQUEST billing mode for cost efficiency

2. CostOptimizationReports table:

   • Partition key: report_id (String)
   • PAY_PER_REQUEST billing mode for cost efficiency

The first table stores individual recommendations with their implementation status, while the second table tracks metadata about generated reports.

Step 3: Create the Bedrock Agent

Now let’s create the agent that will orchestrate the entire analysis process:

1. In the Bedrock console, go to “Agents” → “Create agent”

2. Name it “CostOptimizationAgent”

3. Select Claude 3.5 Sonnet for the foundation model

4. Create three action groups:

   a. GetCostData

   {
     "actionGroupName": "GetCostData",
     "description": "Retrieves cost and usage data from AWS Cost Explorer",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: Cost Explorer API\n  version: 1.0.0\npaths:\n  /getCostData:\n    post:\n      summary: Get cost and usage data from AWS Cost Explorer\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                time_period:\n                  type: string\n                  description: The time period to analyze (WEEK, MONTH, QUARTER)\n                services:\n                  type: array\n                  items:\n                    type: string\n                  description: Optional filter for specific AWS services\n      responses:\n        200:\n          description: Successful response with cost data"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-COST-DATA-LAMBDA-ARN]"
     }
   }
   

   b. AnalyzeResources

   {
     "actionGroupName": "AnalyzeResources",
     "description": "Analyzes AWS resources for optimization opportunities",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: Resource Analysis API\n  version: 1.0.0\npaths:\n  /analyzeResources:\n    post:\n      summary: Analyze AWS resources for optimization opportunities\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              properties:\n                resource_types:\n                  type: array\n                  items:\n                    type: string\n                  description: Resource types to analyze, e.g., 'ec2', 'rds', 'ebs', 'lambda'\n      responses:\n        200:\n          description: Successful analysis of resources"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-RESOURCE-ANALYSIS-LAMBDA-ARN]"
     }
   }
   

   c. GenerateReport

   {
     "actionGroupName": "GenerateReport",
     "description": "Generates a cost optimization report and sends notifications",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: Report Generation API\n  version: 1.0.0\npaths:\n  /generateReport:\n    post:\n      summary: Generate a cost optimization report and send notifications\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              required:\n                - cost_data\n                - resource_analysis\n              properties:\n                cost_data:\n                  type: object\n                  description: Cost and usage data from Cost Explorer\n                resource_analysis:\n                  type: object\n                  description: Results of resource analysis\n                destination:\n                  type: string\n                  description: Where to send the report (S3, SNS, or BOTH)\n      responses:\n        200:\n          description: Successful report generation"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-REPORT-GENERATION-LAMBDA-ARN]"
     }
   }
   

5. Configure the agent’s instructions:

   You are a helpful cost optimization agent for AWS. Your purpose is to analyze AWS costs and resource usage to identify potential savings opportunities.
   
   When asked to analyze costs:
   1. Get cost and usage data using GetCostData action
   2. Analyze resources for optimization opportunities using AnalyzeResources action
   3. Generate a report of findings and recommendations using GenerateReport action
   
   Your recommendations should be practical and actionable, focusing on:
   - EC2 instance optimization (rightsizing, stopping idle instances)
   - Unattached or underutilized EBS volumes
   - Rarely used Lambda functions
   - Reserved Instance or Savings Plans opportunities
   - Multi-AZ configurations that might not be needed for non-production
   
   Present findings clearly in order of potential savings, with the highest-impact opportunities first.
   

These instructions are crucial as they define how the agent will behave when analyzing costs. The careful wording ensures it focuses on the most impactful optimization opportunities.

Step 4: Create the main Lambda function

Create the main Lambda function that will be triggered by the schedule:

import json
import os
import boto3
import logging
import time

# Initialize Bedrock Runtime client
bedrock_agent_runtime = boto3.client('bedrock-agent-runtime')
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    try:
        # Get parameters from the event
        time_period = event.get('time_period', 'MONTH')
        
        # Invoke the Bedrock agent
        response = bedrock_agent_runtime.invoke_agent(
            agentId=os.environ['BEDROCK_AGENT_ID'],
            agentAliasId=os.environ['BEDROCK_AGENT_ALIAS_ID'],
            sessionId=f"cost-analysis-{int(time.time())}",
            inputText=f"Analyze AWS costs for the past {time_period.lower()}, look for optimization opportunities, and generate a comprehensive report with specific actionable recommendations."
        )
        
        # Process agent response
        completion = ''
        for event in response.get('completion', []):
            chunk = json.loads(event['chunk']['bytes'].decode())
            if chunk['type'] == 'message':
                completion += chunk['message']['content'][0]['text']
        
        return {
            'statusCode': 200,
            'body': json.dumps({'status': 'Cost analysis completed'})
        }
    
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

Step 5: Set up the EventBridge rule

Create an EventBridge rule to run the analysis on a schedule. In a production environment, you’d define this in your infrastructure-as-code using Terraform or CloudFormation, setting parameters like:

• Rule name: “WeeklyCostOptimizationAnalysis”
• Schedule expression: “cron(0 8 ? * MON *)” (runs every Monday at 8 AM)
• Target: Your main Lambda function

EventBridge ensures the cost analysis runs automatically at your chosen interval without manual intervention.

Step 6: Set up SNS for notifications

Set up an SNS topic for notifications by creating a topic and adding subscribers. In production, you’d define this in your infrastructure-as-code, configuring:

• Topic name: “CostOptimizationAlerts”
• Protocol: Email, SMS, or webhook (based on your preferred notification channel)
• Subscribers: Finance team, cloud administrators, or a Slack webhook

The notification system ensures key stakeholders are informed of optimization opportunities as they’re identified.

Analysis Capabilities

The agent can identify several types of cost optimization opportunities:

1. EC2 Instance Optimization

• Idle Instances: Identifies running instances with CPU utilization consistently below 5%
• Rightsizing Opportunities: Finds instances that could be downsized based on utilization patterns
• Stopped Instances: Locates instances that have been stopped for extended periods
• Instance Family Upgrades: Suggests moving to newer generation instance families
• Reserved Instance Coverage: Identifies on-demand instances that should be covered by RIs

2. Storage Optimization

• Unattached EBS Volumes: Finds volumes not attached to instances
• Old Snapshots: Identifies EBS snapshots older than 6 months
• Underutilized Volumes: Locates volumes with consistently low I/O patterns
• Storage Class Transitions: Recommends moving infrequently accessed data to lower-cost storage tiers

3. Database Optimization

• Overprovisioned RDS Instances: Identifies oversized database instances
• Multi-AZ in Development: Flags multi-AZ deployments in non-production environments
• Idle Databases: Finds database instances with minimal connection counts
• Reserved Instance Opportunities: Suggests RIs for stable database workloads

4. Serverless Optimization

• Overallocated Memory: Identifies Lambda functions with excessive memory allocation
• Rarely Used Functions: Finds functions that are rarely invoked but consume resources
• Long-Running Functions: Suggests optimizations for functions that consistently run long

Cost considerations

This solution is cost-efficient:

• Lambda costs: Most usage will fall under the free tier
• EventBridge: No additional cost for scheduled rules
• Bedrock API: ~$0.015 per 1,000 tokens with Claude Sonnet
• S3: Minimal storage costs for reports
• DynamoDB: Pay-per-request pricing keeps costs very low
• SNS: Practically free for email notifications

For a weekly execution schedule, the infrastructure costs typically remain under $5 per month for most organizations due to the minimal compute resources required.

Extending the solution

Here are some ways to enhance your cost optimization agent:

Multi-account analysis: Extend to analyze costs across an AWS Organization. This offers a comprehensive view of spending across your entire cloud estate.

Implementation tracking: Track which recommendations were implemented and their actual savings. This helps quantify the ROI of the optimization agent.

Automated remediation: Add capability to automatically implement low-risk optimizations like removing unattached EBS volumes. The agent could implement changes automatically during off-hours.

Slack integration: Send reports directly to Slack channels, enabling team discussions around cost optimization opportunities and tagging responsible teams.

Tagging compliance: Check for resources without proper cost allocation tags, ensuring your organization maintains visibility into spend by department, team, or project.

Budget alerts integration: Combine cost optimization with proactive budget alerts, automatically triggering more aggressive analysis when a budget threshold is approaching.

Custom thresholds: Allow different teams or environments to set custom thresholds for what constitutes underutilization based on their specific workload patterns.

Conclusion

This solution leverages several AWS services to create an intelligent cost optimization system that analyzes cloud spending and provides specific recommendations for reducing costs.

Key advantages of this approach include:

1. Automation: Regular, scheduled analysis without manual intervention
2. Actionable insights: Specific recommendations rather than just data visualization
3. Comprehensive coverage: Analysis across multiple resource types (EC2, RDS, Lambda, etc.)
4. Prioritization: Recommendations sorted by potential impact
5. Integration: Works with existing AWS services and notification systems

The architecture combines the data collection capabilities of AWS Cost Explorer with the analytical power of Amazon Bedrock to generate insights similar to those from a cloud financial analyst. By implementing this solution, organizations can transform cost management from a periodic, manual exercise into an ongoing, automated process.

The system is particularly effective at identifying unused resources, rightsizing opportunities, and reservation/Savings Plans recommendations - areas that often yield significant savings when properly optimized.

An AI assistant can address this challenge by analyzing pull requests before human reviewers, catching common issues and allowing the team to focus on more complex aspects of the code review. This approach doesn’t replace human judgment but enhances it, ensuring that routine issues are caught consistently while freeing up developer time for deeper analysis.

This post demonstrates how to build an automated PR reviewer using AWS Bedrock Agents that analyzes code changes and provides feedback directly in GitHub.

What we’re building

A PR review agent that:

1. Gets triggered automatically when a new PR is opened or updated
2. Fetches the PR diff from GitHub
3. Analyzes the changes using AWS Bedrock
4. Posts a detailed review comment with suggestions
5. Tracks review history in DynamoDB

The goal isn’t to replace human reviewers, but to complement them by identifying common issues, style violations, and potential bugs before human review begins.

Real-World Applications

This solution addresses common development challenges across different contexts:

Enterprise Development Teams: In large organizations with strict coding standards, the agent ensures consistency across hundreds of developers, reducing the burden on senior engineers who often shoulder the bulk of review responsibilities.

Open Source Projects: Maintainers can use the PR reviewer to handle the initial assessment of community contributions, ensuring they meet project guidelines before dedicating their limited time to review.

Educational Settings: Computer science programs can deploy the agent to provide students with immediate feedback on their code submissions, helping them learn best practices without requiring instructor intervention for every issue.

Continuous Integration Pipelines: The PR reviewer can become part of a broader CI/CD strategy, providing code quality assessments alongside traditional test runs and builds.

Onboarding New Team Members: New developers on a project receive immediate feedback on their work that helps them understand team coding standards more quickly, accelerating their integration into the team.

Architecture overview

Here’s how the system works:

GitHub Webhook → EventBridge → Lambda → Bedrock Agent with Action Groups → GitHub API → PR Comments

The key components:

• GitHub Webhook: Triggers on PR events
• EventBridge: Routes events to Lambda
• Lambda: Processes GitHub events and invokes Bedrock Agent
• Bedrock Agent: Coordinates the review process
• Action Groups: Custom Lambda functions that the agent can call
• DynamoDB: Tracks review history and status

This event-driven architecture ensures the review process begins automatically whenever a pull request is opened or updated, without requiring any manual intervention.

What you’ll need

• AWS Account with Bedrock access
• GitHub repository where you want to enable reviews
• GitHub Personal Access Token with repo permissions
• Basic familiarity with AWS and GitHub

Step 1: Create the Action Group Lambda functions

First, let’s create the Lambda functions our Bedrock Agent will use as action groups. We need three functions:

1. Get PR Diff Lambda

This function fetches the pull request details and diff from GitHub:

import json
import os
import boto3
import requests
from github import Github

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        repo_name = payload['repo_name']
        pr_number = payload['pr_number']
        
        # Initialize GitHub
        g = Github(os.environ['GITHUB_TOKEN'])
        repo = g.get_repo(repo_name)
        pr = repo.get_pull(int(pr_number))
        
        # Get PR details and diff
        pr_title = pr.title
        pr_description = pr.body or ""
        pr_author = pr.user.login
        pr_files = [f.filename for f in pr.get_files()]
        
        diff_url = f"https://github.com/{repo_name}/pull/{pr_number}.diff"
        headers = {'Authorization': f"token {os.environ['GITHUB_TOKEN']}"}
        diff_response = requests.get(diff_url, headers=headers)
        diff = diff_response.text
        
        # Return everything the agent needs
        return {
            'statusCode': 200,
            'body': json.dumps({
                'pr_title': pr_title,
                'pr_description': pr_description,
                'pr_author': pr_author,
                'pr_files': pr_files,
                'pr_diff': diff
            })
        }
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

2. Analyze Code Lambda

This function sends the code diff to Bedrock for analysis:

import json
import os
import boto3

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        diff = payload['diff']
        languages = payload.get('languages', [])
        
        # Create prompt for the LLM
        prompt = f"""
        Please review the following code diff and provide actionable feedback:
        
        {diff}
        
        In your analysis, please look for:
        1. Potential bugs or logic errors
        2. Security vulnerabilities
        3. Performance issues
        4. Code style and best practices
        5. Missing tests or documentation
        
        Format your response as:
        
        ## Summary
        (Brief overview of the changes and their purpose)
        
        ## Critical Issues
        (List any serious problems that must be fixed)
        
        ## Suggestions
        (List minor issues and improvements)
        
        ## Positive Notes
        (Highlight good practices in the code)
        """
        
        # Call Bedrock for analysis
        bedrock = boto3.client('bedrock-runtime')
        response = bedrock.invoke_model(
            modelId='anthropic.claude-3-sonnet-20240229-v1:0',
            contentType='application/json',
            accept='application/json',
            body=json.dumps({
                "anthropic_version": "bedrock-2023-05-31",
                "max_tokens": 2000,
                "messages": [{"role": "user", "content": prompt}]
            })
        )
        
        # Parse and return response
        response_body = json.loads(response['body'].read())
        analysis = response_body['content'][0]['text']
        
        return {
            'statusCode': 200,
            'body': json.dumps({'analysis': analysis})
        }
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

3. Post Comment Lambda

This function posts the review feedback to the GitHub PR and records the review in DynamoDB:

import json
import os
import boto3
import time
from github import Github

def lambda_handler(event, context):
    try:
        # Extract parameters from the event
        payload = json.loads(event['body'])
        repo_name = payload['repo_name']
        pr_number = payload['pr_number']
        comment = payload['comment']
        
        # Record review in DynamoDB
        dynamodb = boto3.resource('dynamodb')
        table = dynamodb.Table(os.environ['REVIEW_TABLE'])
        
        review_id = f"{repo_name}-{pr_number}-{int(time.time())}"
        table.put_item(
            Item={
                'review_id': review_id,
                'repo_name': repo_name,
                'pr_number': pr_number,
                'timestamp': int(time.time()),
                'comment': comment
            }
        )
        
        # Post comment to GitHub
        g = Github(os.environ['GITHUB_TOKEN'])
        repo = g.get_repo(repo_name)
        pr = repo.get_pull(int(pr_number))
        
        # Format the comment for GitHub
        formatted_comment = f"""
## AI Code Review 🤖

{comment}

---
*This review was automatically generated by the PR Review Agent. [Learn more](https://example.com/pr-agent)*
        """
        
        pr.create_issue_comment(formatted_comment)
        
        return {
            'statusCode': 200,
            'body': json.dumps({
                'review_id': review_id,
                'status': 'Comment posted successfully'
            })
        }
    except Exception as e:
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

Step 2: Set up DynamoDB for tracking reviews

You’ll need a DynamoDB table to track review history. In production, you’d define this in your infrastructure-as-code using Terraform or CloudFormation. The table needs:

• Partition key: review_id (String)
• PAY_PER_REQUEST billing mode for cost efficiency

This table will store metadata about each review, including repository, PR number, timestamp, and the content of the review comment.

Step 3: Create the Bedrock Agent

Now let’s create the agent that will orchestrate the entire review process:

1. In the Bedrock console, go to “Agents” → “Create agent”

2. Name it “PRReviewAgent”

3. Select Claude 3.5 Sonnet for the foundation model

4. Create three action groups:

   a. GetPRDiff

   {
     "actionGroupName": "GetPRDiff",
     "description": "Retrieves the diff for a GitHub pull request",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: GitHub PR Diff API\n  version: 1.0.0\npaths:\n  /getPRDiff:\n    post:\n      summary: Get the diff for a GitHub pull request\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              required:\n                - repo_name\n                - pr_number\n              properties:\n                repo_name:\n                  type: string\n                  description: The repository in format 'owner/repo'\n                pr_number:\n                  type: integer\n                  description: The pull request number\n      responses:\n        200:\n          description: Successful response\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  pr_title:\n                    type: string\n                  pr_description:\n                    type: string\n                  pr_author:\n                    type: string\n                  pr_files:\n                    type: array\n                    items:\n                      type: string\n                  pr_diff:\n                    type: string"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-GET-PR-DIFF-LAMBDA-ARN]"
     }
   }
   

   b. AnalyzeCode

   {
     "actionGroupName": "AnalyzeCode",
     "description": "Analyzes code for potential issues and improvements",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: Code Analysis API\n  version: 1.0.0\npaths:\n  /analyzeCode:\n    post:\n      summary: Analyze code diff for issues and suggestions\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              required:\n                - diff\n              properties:\n                diff:\n                  type: string\n                  description: The code diff to analyze\n                languages:\n                  type: array\n                  items:\n                    type: string\n                  description: Programming languages in the diff (optional)\n      responses:\n        200:\n          description: Successful analysis\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  analysis:\n                    type: string"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-ANALYZE-CODE-LAMBDA-ARN]"
     }
   }
   

   c. PostComment

   {
     "actionGroupName": "PostComment",
     "description": "Posts a review comment to a GitHub pull request",
     "apiSchema": {
       "type": "openapi",
       "payload": "openapi: 3.0.0\ninfo:\n  title: GitHub Comment API\n  version: 1.0.0\npaths:\n  /postComment:\n    post:\n      summary: Post a comment to a GitHub pull request\n      requestBody:\n        required: true\n        content:\n          application/json:\n            schema:\n              type: object\n              required:\n                - repo_name\n                - pr_number\n                - comment\n              properties:\n                repo_name:\n                  type: string\n                  description: The repository in format 'owner/repo'\n                pr_number:\n                  type: integer\n                  description: The pull request number\n                comment:\n                  type: string\n                  description: The comment text to post\n      responses:\n        200:\n          description: Successful response\n          content:\n            application/json:\n              schema:\n                type: object\n                properties:\n                  review_id:\n                    type: string\n                  status:\n                    type: string"
     },
     "actionGroupExecutor": {
       "type": "lambda",
       "lambdaArn": "[YOUR-POST-COMMENT-LAMBDA-ARN]"
     }
   }
   

5. Configure the agent’s instructions:

   You are a helpful GitHub Pull Request reviewer designed to analyze code changes and provide constructive feedback. Your goal is to help developers improve their code by identifying issues and suggesting improvements.
   
   When a PR is submitted:
   1. Get the PR diff using the GetPRDiff action
   2. Analyze the code for issues using the AnalyzeCode action
   3. Post a helpful review comment using the PostComment action
   
   Your reviews should be constructive and educational. Focus on:
   - Potential bugs or logic errors
   - Security vulnerabilities
   - Performance issues
   - Code style and best practices
   - Missing tests or documentation
   
   Use a professional tone and be concise but thorough in your feedback.
   

These instructions are crucial as they define how the agent will behave when reviewing code. The careful wording encourages constructive feedback while maintaining a professional tone.

Step 4: Create the main Lambda function

Now create the main Lambda function that will be triggered by GitHub events and coordinate the review process:

import json
import os
import boto3
import logging

# Initialize Bedrock Runtime client
bedrock_agent_runtime = boto3.client('bedrock-agent-runtime')
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    try:
        # Parse the GitHub webhook event
        github_event = json.loads(event['body'])
        headers = event.get('headers', {})
        
        # Check if it's a pull request event we care about
        if headers.get('X-GitHub-Event') == 'pull_request':
            action = github_event.get('action')
            
            # Process only opened or synchronized (updated) PRs
            if action in ('opened', 'synchronize'):
                pr = github_event['pull_request']
                repo_name = github_event['repository']['full_name']
                pr_number = pr['number']
                
                # Invoke the Bedrock agent to perform the review
                response = bedrock_agent_runtime.invoke_agent(
                    agentId=os.environ['BEDROCK_AGENT_ID'],
                    agentAliasId=os.environ['BEDROCK_AGENT_ALIAS_ID'],
                    sessionId=f"{repo_name}-{pr_number}",
                    inputText=f"Review pull request #{pr_number} in repository {repo_name}"
                )
                
                # Process agent response
                completion = ''
                for event in response.get('completion', []):
                    chunk = json.loads(event['chunk']['bytes'].decode())
                    if chunk['type'] == 'message':
                        completion += chunk['message']['content'][0]['text']
                
                return {
                    'statusCode': 200,
                    'body': json.dumps({'status': 'Agent invoked successfully'})
                }
            
            return {'statusCode': 200, 'body': json.dumps({'status': 'Ignored PR action'})}
        
        return {'statusCode': 200, 'body': json.dumps({'status': 'Ignored webhook event'})}
    
    except Exception as e:
        logger.error(f"Error: {str(e)}")
        return {'statusCode': 500, 'body': json.dumps({'error': str(e)})}

Step 5: Create the API Gateway

1. Create a new REST API in API Gateway
2. Add a POST method for the root resource
3. Set the integration type to Lambda Function and select your main function
4. Deploy the API to a stage and note the URL

The API Gateway serves as the entry point for GitHub webhooks, receiving events when pull requests are opened or updated.

Step 6: Set up the GitHub webhook

1. Go to your GitHub repository
2. Navigate to Settings → Webhooks → Add webhook
3. Set the Payload URL to your API Gateway URL
4. Set Content type to application/json
5. Select “Let me select individual events” and check “Pull requests”
6. Click “Add webhook”

With the webhook in place, GitHub will now notify your system whenever pull requests are created or updated, triggering the automated review process.

Testing the system

To test your PR review agent:

1. Create a simple PR with some code changes
2. Check that the webhook is triggered (visible in GitHub webhook settings)
3. Verify your Lambda is invoked (check CloudWatch logs)
4. Check that a comment appears on the PR with the review

You should see a comprehensive review comment that identifies potential issues, makes suggestions for improvement, and highlights positive aspects of the code changes.

Enhancing the agent

Here are some ways to improve your PR reviewer:

Language-specific rules: Extend the AnalyzeCode lambda to apply language-specific linters or static analysis tools for more precise feedback based on the programming language.

Contextual awareness: Include repository history, architecture documentation, or team standards in the analysis to provide more relevant and contextual suggestions.

Review customization: Allow teams to set review focus areas through configuration. For example, some teams might prioritize security checks while others emphasize performance.

Learning from feedback: Track which suggestions developers implement versus ignore to improve future recommendations and reduce false positives.

PR metadata analysis: Consider PR size, files changed, and complexity when determining the review strategy. Large PRs might receive different handling than small, focused changes.

Inline comments: Enhance the agent to post specific comments on individual lines in the diff rather than just a summary comment.

Pre-commit integration: Offer developers the ability to run the same analysis locally before submitting their PR, using pre-commit hooks.

Cost considerations

This architecture is cost-efficient for most development teams:

• Lambda costs: Typically covered by the free tier for small to medium teams
• API Gateway: Approximately $1 per million requests
• Bedrock API: Around $0.015 per 1,000 tokens with Claude Sonnet
• DynamoDB: Pay-per-request pricing with minimal storage needs

The total cost will vary based on your team’s PR volume and the size of code changes being reviewed, but for most teams, it remains quite affordable compared to the developer time saved.

Security considerations

When implementing this:

• Store API tokens in AWS Secrets Manager
• Set IAM permissions using least privilege
• Consider the sensitivity of code being reviewed
• Remember that agents aren’t perfect at security analysis

While the PR reviewer can identify many common security issues, it should not be your only security control. Critical security reviews should still be performed by security experts, especially for sensitive components.

Conclusion

By combining AWS Bedrock Agents with Action Groups, this architecture creates an intelligent PR review system that helps maintain code quality while reducing the time developers spend on repetitive aspects of code reviews.

The solution provides several key benefits:

1. Consistency: Every PR receives the same baseline level of review
2. Early detection: Catches common issues before human reviewers see the code
3. Educational feedback: Provides context and explanations for suggested improvements
4. Focus: Allows human reviewers to concentrate on architecture and business logic
5. Historical tracking: Stores reviews for future reference and improvement

While this implementation uses specific AWS services, the architectural patterns can be adapted to other cloud providers or integrated with different version control systems beyond GitHub.

This approach to automated code review represents a practical application of generative AI that delivers immediate value to development teams: better code quality, faster reviews, and more time for developers to focus on creative and complex challenges rather than routine feedback.

In this post, I’ll show you how to build a simple yet powerful Q&A bot for Slack that leverages your company’s documentation to provide accurate, contextual answers. The best part? It runs entirely on AWS managed services, minimizing operational overhead while delivering immediate value to your organization.

Real-World Applications

This solution addresses documentation challenges across different departments:

HR and People Teams: Employees constantly ask about benefits, PTO policies, and workplace guidelines. An AI bot can instantly answer “How many vacation days do I have?” or “What’s our parental leave policy?” by citing the exact paragraph from your handbook.

Engineering Teams: Technical documentation grows exponentially with your codebase. When an engineer asks “How do I set up the development environment?” or “What’s our database migration process?”, the bot can provide step-by-step instructions from your wiki.

Product and Sales Teams: Sales representatives need quick access to product specifications, pricing details, and competitive positioning. A knowledge bot can answer “What are the enterprise tier limits?” during a client call without disrupting other team members.

Customer Support: Support teams juggle hundreds of internal processes. When an agent needs to know “What’s our escalation policy?” or “How do I process a refund?”, immediate answers improve customer response times.

New Employee Onboarding: The first weeks at a new job involve absorbing massive amounts of information. A knowledge bot gives new hires an accessible way to ask questions without feeling like they’re bothering colleagues.

What We’re Building

A Slack bot that:

1. Receives questions from users in a channel or DM
2. Uses AWS Bedrock Knowledge Base to search through your company documentation
3. Generates accurate answers with citations to source documents
4. Handles follow-up questions with conversation history

The system uses Retrieval-Augmented Generation (RAG), combining the reasoning capabilities of large language models with retrieval from your own data sources—giving you the benefits of generative AI while keeping your data within your AWS account.

Understanding Bedrock Knowledge Bases

AWS Bedrock Knowledge Bases represents a significant advancement in enterprise knowledge management. Let’s explore how it works and why it’s superior to traditional search or direct LLM prompting.

The RAG Architecture

Retrieval-Augmented Generation (RAG) addresses a fundamental limitation of LLMs: they have no knowledge of your internal documents. RAG works by:

1. Document Processing: Your documents are divided into chunks of an appropriate size for retrieval
2. Vector Embedding: Each chunk is converted into a numerical vector representation using an embedding model
3. Vector Storage: These embeddings are stored in a vector database (OpenSearch Serverless in Bedrock’s case)
4. Semantic Search: When a question arrives, it’s converted to the same vector space and semantically similar chunks are retrieved
5. Context Augmentation: Retrieved chunks are injected as context into the prompt sent to the LLM
6. Answer Generation: The LLM generates an answer based on this context, citing the relevant sources

This approach dramatically improves accuracy by giving the model direct access to your internal knowledge, while maintaining the reasoning capabilities of foundation models.

Supported Document Types

Bedrock Knowledge Bases supports a wide range of document formats:

• PDF files (text and scanned documents with OCR)
• Microsoft Office (Word, PowerPoint, Excel)
• Text and Markdown files
• HTML and web pages
• CSV and JSON data

This versatility means you can ingest existing documentation without reformatting.

Synchronization and Updates

A key feature is automatic synchronization. When documents in your S3 bucket are updated, Bedrock Knowledge Bases can automatically detect these changes and update the vector store, ensuring your bot always has the latest information.

Semantic vs. Keyword Search

Traditional search systems match keywords, but Bedrock Knowledge Bases understands concepts. If someone asks about “time off,” it can retrieve documents about “vacation,” “PTO,” and “leave of absence” because it understands these concepts are related—even if they don’t share exact keywords.

Architecture Overview

Here’s how the solution components work together:

When a user asks a question in Slack, the message triggers a webhook to API Gateway. This request is processed by a Lambda function that maintains conversation context in DynamoDB and communicates with Bedrock. The Bedrock Agent uses the Knowledge Base to search your documentation, retrieves relevant information, and formulates a response that’s sent back to the user through Slack.

Each component serves a specific purpose in this flow:

• Slack API handles the user interface, making the experience seamless within your existing communication platform.
• API Gateway provides a secure endpoint for Slack to send events to.
• Lambda orchestrates the process, maintaining conversation history and managing the interaction between Slack and Bedrock.
• Bedrock Agent uses Claude to interpret questions, retrieve information, and generate natural-sounding responses.
• Bedrock Knowledge Base indexes and searches your company documentation, finding the most relevant information for each question.
• DynamoDB stores conversation history so the bot can understand follow-up questions in context.

This serverless architecture scales automatically with usage and requires minimal maintenance once deployed.

Prerequisites

Before starting, make sure you have:

• AWS Account with Bedrock access (you’ll need quota for Claude models)
• Slack workspace with permissions to create apps
• Company documentation organized in a folder structure
• Basic familiarity with AWS services and Terraform (or CloudFormation)

Step 1: Create the Knowledge Base

First, we’ll create a Knowledge Base to store and index your company documentation:

1. Upload documents to an S3 bucket:

   aws s3 mb s3://your-company-docs
   aws s3 sync ./docs s3://your-company-docs/
   

   Organize your documents logically—folders like HR, Engineering, and Sales help the system understand document context.

2. Create the Knowledge Base in Bedrock:

   Navigate to AWS Bedrock in the console, select “Knowledge bases” → “Create knowledge base,” and follow the wizard:

   • Name: “CompanyDocs”
   • Data source: Select your S3 bucket
   • Vector store: “Create new Amazon OpenSearch Serverless vector store”
   • Embedding model: “Titan Embeddings G1” (offers excellent performance for most use cases)
   • Enable automatic synchronization to keep your knowledge base updated

The initial data synchronization process will take several minutes depending on the volume of your documents. During this time, Bedrock is analyzing your documents, chunking them appropriately, and converting them into vector embeddings for semantic search.

Step 2: Create the Bedrock Agent

Now let’s create an agent that will use our Knowledge Base:

1. In the Bedrock console, go to “Agents” → “Create agent”

2. Name it “CompanyDocsAssistant” and select Claude 3.5 Sonnet for the foundation model

3. In the “Action groups” section, add a Knowledge Base action group and select the “CompanyDocs” knowledge base we created

4. Configure the agent’s instructions with detailed guidance:

   You are a helpful assistant that answers questions about company documentation, policies, and procedures.
   
   When answering:
   1. Be concise but thorough
   2. Always cite sources by document name when you provide information
   3. If you don't know or can't find relevant information, say so clearly
   4. For follow-up questions, maintain context from previous exchanges
   5. Format responses with appropriate Slack formatting (bullets, bold, etc.) where helpful
   6. Present step-by-step procedures in numbered lists when applicable
   

5. For the IAM role, create a new service role with the necessary permissions to access your Knowledge Base

The detailed instructions are crucial—they set the tone and behavior of your assistant, determining how it will respond to various types of questions.

Step 3: Create the Lambda Function

Next, create a Lambda function to handle Slack events and communicate with our Bedrock agent:

import json
import os
import boto3
import logging
import urllib.request
import time
from boto3.dynamodb.conditions import Key

# Initialize clients
bedrock_agent_runtime = boto3.client('bedrock-agent-runtime')
dynamodb = boto3.resource('dynamodb')
conversation_table = dynamodb.Table(os.environ['CONVERSATION_TABLE'])
logger = logging.getLogger()
logger.setLevel(logging.INFO)

def lambda_handler(event, context):
    # Parse the incoming event from Slack
    body = json.loads(event['body'])
    
    # Handle URL verification challenge
    if body.get('type') == 'url_verification':
        return {'statusCode': 200, 'body': json.dumps({'challenge': body['challenge']})}
    
    # Process message events (app_mention or direct message)
    if body.get('event', {}).get('type') == 'app_mention' or \
       (body.get('event', {}).get('type') == 'message' and 
        body.get('event', {}).get('channel_type') == 'im'):
        
        event_data = body['event']
        user_id = event_data['user']
        channel_id = event_data['channel']
        text = event_data.get('text', '').replace(f"<@{os.environ['BOT_USER_ID']}>", '').strip()
        
        # Get conversation history and invoke Bedrock agent
        conversation_id = f"{user_id}:{channel_id}"
        history = get_conversation_history(conversation_id)
        response = invoke_bedrock_agent(text, history, conversation_id)
        
        # Send response back to Slack
        send_slack_message(channel_id, response)
        return {'statusCode': 200, 'body': json.dumps({'status': 'ok'})}
    
    return {'statusCode': 200, 'body': json.dumps({'status': 'ignored'})}

def invoke_bedrock_agent(question, history, conversation_id):
    try:
        # Format history for Bedrock and add the current question
        messages = format_conversation_history(history)
        messages.append({
            'role': 'user',
            'content': [{'text': question}]
        })
        
        # Invoke the Bedrock agent
        response = bedrock_agent_runtime.invoke_agent(
            agentId=os.environ['BEDROCK_AGENT_ID'],
            agentAliasId=os.environ['BEDROCK_AGENT_ALIAS_ID'],
            sessionId=conversation_id,
            inputText=question,
            enableTrace=True
        )
        
        # Extract and process the response
        completion = process_agent_response(response)
        
        # Store conversation in DynamoDB for history
        store_conversation_entry(conversation_id, 'user', question)
        store_conversation_entry(conversation_id, 'assistant', completion)
        
        return completion
    except Exception as e:
        logger.error(f"Error invoking Bedrock agent: {str(e)}")
        return f"I'm having trouble answering that right now. Technical details: {str(e)}"

# Additional helper functions for conversation history, messaging, etc.
# (implementation details omitted for brevity)

This Lambda function handles:

• Receiving events from Slack
• Maintaining conversation context
• Communicating with the Bedrock agent
• Sending responses back to the user

The actual implementation includes additional helpers for conversation history management, message formatting, and error handling that we’ve omitted here for brevity.

Step 4: Set up the DynamoDB Table

You’ll need a DynamoDB table to track conversation history. In production, you’d define this in your infrastructure-as-code using Terraform or CloudFormation. The table needs:

• Partition key: conversation_id (String)
• Sort key: timestamp (Number)
• PAY_PER_REQUEST billing mode for cost efficiency

This table enables the bot to understand follow-up questions by maintaining context from previous exchanges.

Step 5: Create the Slack App

1. Go to api.slack.com/apps and create a new app “From scratch”
2. Under “OAuth & Permissions,” add these scopes:
   • app_mentions:read
   • chat:write
   • im:history
   • im:read
3. Under “Event Subscriptions”:
   • Enable Events and set the Request URL to your API Gateway endpoint
   • Subscribe to bot events: app_mention and message.im
4. Install the app to your workspace and copy the Bot User OAuth Token

The Slack app configuration establishes the permissions and event subscriptions needed for the bot to receive messages and respond to users.

Step 6: Deploy the API Gateway

Create an API Gateway to receive events from Slack:

1. Create a new HTTP API with a POST route that integrates with your Lambda function
2. Deploy the API and note the URL
3. Update your Slack app’s Event Subscriptions URL with this endpoint
4. Add environment variables to your Lambda function:
   • SLACK_BOT_TOKEN: The OAuth token from your Slack app
   • BOT_USER_ID: The user ID of your Slack bot
   • BEDROCK_AGENT_ID: The ID of your Bedrock agent
   • BEDROCK_AGENT_ALIAS_ID: The alias ID of your agent
   • CONVERSATION_TABLE: Your DynamoDB table name

Cost Optimization

This solution is cost-effective, but there are a few considerations:

• Bedrock API calls: ~$0.015 per 1,000 tokens with Claude Sonnet
• Knowledge Base storage: ~$0.023/GB for S3 + OpenSearch Serverless vector storage
• Lambda: Free tier likely covers most usage patterns
• DynamoDB: Pay-per-request pricing keeps costs low
• API Gateway: ~$1 per million requests

For a team of 20 people asking 10 questions per day, expect costs around $30-50 per month. You can implement usage tracking to monitor and control costs as adoption grows.

Extending the Solution

Here are some ways to enhance this basic implementation:

• Multi-channel support: Monitor multiple Slack channels with channel-specific knowledge bases
• Document syncing: Set up automatic synchronization with your documentation systems
• Permissions: Implement access controls based on Slack user groups
• Analytics: Track common questions to identify gaps in your documentation
• Multi-model support: Use a simpler model for basic questions and Claude for complex ones
• Conversation summarization: Periodically summarize long conversations for better context management

Conclusion

With just a few AWS services, you’ve built an intelligent assistant that makes your company’s documentation accessible via Slack. No more hunting through SharePoint or Confluence—just ask the bot and get instant answers with citations to the source material.

The real power here is that your data remains within your AWS account, the system only has access to approved documents, and it continuously improves as you add more documentation. As AWS enhances Bedrock’s capabilities, your bot will automatically benefit from these improvements without any changes to your architecture.

This solution demonstrates how easily companies can now deploy practical AI applications using managed services. What used to require a specialized ML team and months of development can now be built in days using serverless components.

What documentation would you connect to your knowledge bot first? Let me know on LinkedIn!

Everyone can demo generative AI. Almost no one can run it safely in production.

Enterprises in finance, healthcare, and the public sector aren’t blocked by technology capabilities—they’re blocked by governance requirements that today’s AI implementations rarely satisfy.

These organizations face three critical blockers:

• Data leakage risk: Sensitive information, from PII to trade secrets, flowing through public model APIs
• Lack of auditability: No reliable record of prompts, responses, or who accessed what information
• Unclear ownership: Ambiguous rights over prompt engineering IP, training data, and generated outputs

AWS customers don’t want AI that behaves like a chatbot toy. They need AI that behaves like enterprise infrastructure: secured, monitored, audited, governed, and compliant with their existing security posture.

Design Goals for Enterprise-Ready GenAI

When designing generative AI systems for regulated environments, your architecture must satisfy these non-negotiable requirements:

• No public internet exposure for sensitive data
• No training on customer data without explicit permission
• Full audit trail of all prompts and responses
• IAM-first access control integrated with enterprise identity
• Serverless and scalable by default

This checklist maps directly to AWS Well-Architected Framework principles, particularly in security and operational excellence.

Reference Architecture Overview

Here’s a reference architecture that meets these requirements using AWS services:

Walking the Architecture: Building for Security and Scale

1️⃣ Edge & Entry: CloudFront + API Gateway

The edge layer serves as your first line of defense:

• Global edge protection through CloudFront
• Request validation and throttling via API Gateway
• Clear API contract for AI access
• WAF rules to block suspicious patterns

This approach frames AI as just another AWS workload, not an exception to your security rules. Your existing infrastructure and compliance controls extend naturally to your AI services.

2️⃣ Prompt Handling: Lambda

The Prompt Handler Lambda is where policy meets AI:

• Sanitizes inputs to prevent prompt injection
• Injects system prompts to enforce guardrails
• Enforces token limits (cost control)
• Attaches request metadata (user ID, application, purpose)

This layer ensures all model interactions are appropriately structured and traced. Every prompt includes context about who sent it, why, and what constraints apply.

3️⃣ Private Model Access: Bedrock via VPC Endpoint

The model interaction layer guarantees data privacy:

• No public internet egress
• No customer-managed model hosting
• No fine-tuning on customer prompts
• VPC integration with existing security controls

The model is consumed like a managed AWS service—not an external API. This distinction is critical for security teams evaluating AI adoption.

4️⃣ Response Filtering: Post-processing Lambda

The response handler implements safety guardrails:

• Content moderation (PII, offensive content)
• Output validation against schema
• Optional redaction of sensitive information
• Confidence scoring and hallucination detection

This layer acknowledges and mitigates hallucination risk without fear-mongering, providing mechanisms to validate and filter model outputs.

5️⃣ Audit & Evidence: DynamoDB / S3

The audit layer addresses compliance requirements:

• Persistent storage of prompt hashes
• Model ID and version tracking
• Timestamped responses
• Immutable audit logs

This creates a defensible evidentiary trail that satisfies governance requirements for regulated industries.

Why This Works for Regulated Industries

This architecture succeeds where most AI implementations fail because it addresses the key requirements that matter to enterprise stakeholders:

Security: IAM-based access control, VPC endpoints, and private networking eliminate public exposure risks. The system operates entirely within your security perimeter, following the principle of “default deny” with explicit allow policies.

Compliance: Complete prompt/response traceability enables regulatory reporting and satisfies audit requirements. You can demonstrate who used the system, when, how, and what results they received—critical for SOC2, HIPAA, and FedRAMP.

Cost control: Serverless scaling plus token limits provide predictable, manageable costs. Unlike self-hosted options, you’re not paying for idle infrastructure, and unlike public APIs, you have fine-grained control over usage patterns.

Operational clarity: The system is observable, debuggable, and auditable using the same tools you already use for the rest of your AWS infrastructure. There’s no AI-specific monitoring to implement.

This approach works particularly well for financial services (handling sensitive financial data), healthcare (maintaining PHI compliance), and public sector (satisfying FedRAMP requirements)—precisely the industries with the most to gain from AI and the most stringent security requirements.

Implementation Considerations

When implementing this pattern in production, several practical considerations emerge:

IAM roles and boundaries: Create specific IAM roles for each component with least-privilege access. The prompt handler needs Bedrock access but not S3 write access; the response filter needs DynamoDB write access but not Bedrock APIs. Use service control policies (SCPs) to enforce guardrails.

VPC design: Depending on your existing network topology, you may need to adjust the VPC design. For large enterprises with transit gateways, consider routing AI traffic through dedicated VPCs with specific security monitoring.

Cost management: Monitor token usage carefully. Implement token quotas at the API Gateway layer and consider using smaller context window models for initial responses, reserving larger context models for specific use cases.

Scaling characteristics: Lambda’s concurrency model handles traffic spikes well, but Bedrock has model-specific quotas and SLAs. Request quota increases proactively if you anticipate high volume. Consider implementing queue-based architectures for asynchronous workloads.

Cross-account patterns: For large organizations, implement a hub-and-spoke model where a central AI governance account hosts the Bedrock endpoint, with workload accounts accessing it through cross-account roles. This centralizes auditing while enabling distributed usage.

Conclusion

The gap between AI demos and AI in production isn’t primarily a technical gap—it’s a governance gap.

This reference architecture bridges that gap by treating generative AI as enterprise infrastructure rather than a standalone tool. It integrates with existing security controls, creates auditability, and provides the governance hooks necessary for regulated environments.

The result? AI that can safely navigate the journey from prompt to production, enabling organizations to capture AI’s business value without compromising on security and compliance requirements.

In regulated environments, the future of AI isn’t about building fancy demos—it’s about building trust. By architecting generative AI systems that behave like proper enterprise infrastructure—secured, monitored, audited, and governed—we allow organizations to focus on business value rather than security firefighting.

Remember that this is a reference architecture, not a one-size-fits-all solution. Your specific implementation should be tailored to your compliance requirements, existing infrastructure, and risk profile. But the principles outlined here—isolation, auditability, IAM-first access, and metadata enrichment—remain universal best practices for any enterprise AI deployment.

The Model Context Protocol (MCP) is an open standard for connecting AI agents to external systems. Think of it as a universal adapter that lets any AI agent talk to any tool or data source without custom integration code.

Anthropic announced MCP in November 2024 and donated it to the Linux Foundation’s Agentic AI Foundation about a year later, in December 2025. The adoption has been swift: OpenAI integrated it into ChatGPT, Google DeepMind uses it for Gemini agents, AWS built AgentCore around it, and development tools like Zed, Sourcegraph, Replit, and Codeium all support it. In just a few months, the community has built thousands of MCP servers. The protocol has become the de-facto standard for agent-to-tool communication.

If you’ve read my previous articles on AWS DevOps Agent, Security Agent, and Kiro, you’ve seen what these frontier agents do. This article explains how they actually work—the protocol layer that makes integration possible. More importantly, it shows how you can extend these agents to work with your proprietary systems.

The N×M integration problem

Here’s why MCP matters: You’ve built an amazing AI agent. It reasons brilliantly, writes elegant code, debugs complex issues. But it can’t access your company’s data. Your customer records are in Salesforce. Your code is in GitHub. Your metrics are in Datadog. Your tickets are in Jira.

The traditional approach is building custom integrations. One for each pairing. Want your agent to read Salesforce? Build a Salesforce connector. Want it to access GitHub? Build a GitHub connector. Want it to work with both? Build both connectors. Want to switch LLM providers? Rebuild everything.

This is the N×M integration problem: N agents times M data sources equals N×M custom integrations. As your ecosystem grows, the complexity becomes unmanageable. Three agents talking to four services means twelve custom connectors. Ten agents and fifty systems means five hundred integrations to build and maintain.

Each connector requires understanding the target system’s API, building authentication flows, handling rate limits and retries, writing serialization and deserialization logic, maintaining the connector as APIs change, and rebuilding for each new agent or LLM provider. This doesn’t scale.

With MCP, you build the Salesforce MCP server once. After that, DevOps Agent, Security Agent, Kiro, Claude Desktop—any MCP client—can access Salesforce. No custom integration needed. This is why every major AI company adopted MCP within months of its announcement. It solves an existential scaling problem.

How MCP works: The architecture

MCP follows a client-server architecture using JSON-RPC 2.0 for communication. The protocol exposes three core primitives that servers can implement.

Resources are like RESTful GET endpoints. They load information into LLM context—files, database records, API responses. When you ask an agent to retrieve a customer’s support ticket history, it’s accessing a resource.

Tools are like RESTful POST endpoints. They execute code and produce side effects—creating Jira tickets, deploying code, sending Slack messages. When you tell an agent to create a high-priority ticket for a customer, it’s using a tool.

Prompts are reusable templates that guide interactions with structured patterns. Think “Analyze this code for security vulnerabilities” or “Generate release notes from commits.” They standardize how agents approach common tasks.

The communication flow

Here’s how an agent actually communicates with an MCP server:

sequenceDiagram
    participant Agent as MCP Client (Agent)
    participant Server as MCP Server (Tool)
    
    Agent->>Server: 1. Initialize Connection
    Agent->>Server: 2. List Available Tools
    Server->>Agent: 3. Tool Definitions (JSON Schema)
    Note over Agent: Agent decides<br/>which tool to call
    Agent->>Server: 4. Call Tool (with parameters)
    Note over Server: Validates params<br/>Executes logic<br/>Queries external API
    Server->>Agent: 5. Tool Result
    Note over Agent: Agent uses result<br/>in response

The agent initializes a connection, requests the list of available tools, receives their JSON Schema definitions, decides which tool to call based on the user’s request, sends the tool invocation with parameters, waits for the server to validate inputs and execute the logic, receives the result, and uses it to generate the final response. It’s a clean request-response pattern.

Transport mechanisms

MCP supports two primary transport methods. Standard Input/Output (stdio) is designed for local integration—the server runs as a subprocess and communicates via stdin/stdout. This is what Claude Desktop uses to run local MCP servers. It’s simple, fast, and synchronous, but only works locally.

Server-Sent Events (SSE) is designed for remote integration. It uses HTTP-based streaming where the server pushes updates to the client. This is what you’d use for cloud-hosted MCP servers and enterprise integrations. It’s network-capable and supports real-time updates, but requires HTTP server infrastructure.

The protocol layer

MCP messages are structured as JSON-RPC 2.0 calls. Here’s what a tool call looks like: Request from client to server:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "name": "get_customer_info",
    "arguments": {
      "customer_id": "cust_12345"
    }
  }
}

Response from server to client:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "{\"name\": \"Acme Corp\", \"tier\": \"Enterprise\", \"health\": \"green\"}"
      }
    ]
  }
}

The beauty of this: you rarely write this JSON by hand. MCP SDKs for Python, TypeScript, Java, C#, and Kotlin abstract it away completely. Which brings us to FastMCP.

Building with FastMCP

Building MCP servers from scratch means handling JSON-RPC protocol details, managing connections, writing serialization boilerplate. It’s tedious work that distracts from the actual business logic you want to implement.

FastMCP is the solution—a decorator-based Python framework that makes building MCP servers feel like writing FastAPI applications. Think of it as FastAPI for AI agents with the same elegant decorator patterns, Flask for tool integration with minimal boilerplate, or Express.js for MCP with simple and intuitive design.

Here’s a complete, working MCP server:

from fastmcp import FastMCP

# Initialize server
mcp = FastMCP("My First Server")

# Define a tool
@mcp.tool()
def add_numbers(a: int, b: int) -> int:
    """Add two numbers together"""
    return a + b

# Run it
if __name__ == "__main__":
    mcp.run()

That’s it. You now have an MCP server that exposes a tool, automatically generates JSON Schema from type hints, handles serialization and deserialization, provides error handling, supports both stdio and SSE transports, and works with any MCP client. No boilerplate. No protocol details. Just your logic.

A real example: GitHub integration

Let’s build something practical—an MCP server that lets agents interact with GitHub. This demonstrates how FastMCP handles real-world integrations with external APIs, authentication, and error handling.

from fastmcp import FastMCP
from github import Github
import os

# Initialize server
mcp = FastMCP(
    name="GitHub Integration",
    instructions="Use this server to interact with GitHub repositories"
)

# Initialize GitHub client
github_token = os.getenv("GITHUB_TOKEN")
gh = Github(github_token)

@mcp.tool()
def get_repository_info(owner: str, repo: str) -> dict:
    """
    Get information about a GitHub repository.
    
    Args:
        owner: Repository owner (username or organization)
        repo: Repository name
    
    Returns:
        Repository details including stars, forks, issues
    """
    repository = gh.get_repo(f"{owner}/{repo}")
    
    return {
        "name": repository.name,
        "description": repository.description,
        "stars": repository.stargazers_count,
        "forks": repository.forks_count,
        "open_issues": repository.open_issues_count,
        "language": repository.language,
        "created_at": repository.created_at.isoformat(),
        "updated_at": repository.updated_at.isoformat()
    }

@mcp.tool()
def create_issue(
    owner: str,
    repo: str,
    title: str,
    body: str,
    labels: list[str] = None
) -> dict:
    """
    Create a new issue in a GitHub repository.
    
    Args:
        owner: Repository owner
        repo: Repository name
        title: Issue title
        body: Issue description
        labels: Optional list of labels to apply
    
    Returns:
        Created issue details
    """
    repository = gh.get_repo(f"{owner}/{repo}")
    issue = repository.create_issue(
        title=title,
        body=body,
        labels=labels or []
    )
    
    return {
        "number": issue.number,
        "url": issue.html_url,
        "state": issue.state,
        "created_at": issue.created_at.isoformat()
    }

@mcp.tool()
def list_pull_requests(
    owner: str,
    repo: str,
    state: str = "open"
) -> list[dict]:
    """
    List pull requests in a repository.
    
    Args:
        owner: Repository owner
        repo: Repository name
        state: PR state ('open', 'closed', 'all')
    
    Returns:
        List of pull requests
    """
    repository = gh.get_repo(f"{owner}/{repo}")
    pulls = repository.get_pulls(state=state)
    
    return [
        {
            "number": pr.number,
            "title": pr.title,
            "author": pr.user.login,
            "state": pr.state,
            "created_at": pr.created_at.isoformat(),
            "url": pr.html_url
        }
        for pr in pulls[:10]  # Limit to 10 for brevity
    ]

@mcp.resource("repo://{owner}/{repo}/README")
def get_readme(owner: str, repo: str) -> str:
    """
    Get the README content for a repository.
    
    This is exposed as a resource (not a tool) because it's
    primarily for loading context into the LLM.
    """
    repository = gh.get_repo(f"{owner}/{repo}")
    readme = repository.get_readme()
    return readme.decoded_content.decode('utf-8')

if __name__ == "__main__":
    # Run with stdio (for local Claude Desktop)
    mcp.run(transport="stdio")
    
    # Or run with SSE (for remote access)
    # mcp.run(transport="sse", port=8000)

FastMCP handled type validation—ensuring owner and repo are strings—error handling so GitHub API failures return proper MCP errors, schema generation from docstrings and type hints, authentication flow using the GitHub token from environment variables, and transport abstraction so the same code works with stdio or SSE.

You focused on your business logic—what the tool actually does—and clear documentation where docstrings become tool descriptions. The framework handles everything else.

Advanced FastMCP features

FastMCP isn’t just decorators—it’s a full-featured framework with capabilities that become important as your integrations grow more sophisticated.

Context injection lets you access MCP context and capabilities within your tools. This is useful for long-running operations where you want to send progress updates to the client:

from fastmcp import FastMCP, Context
from mcp.server.session import ServerSession

mcp = FastMCP("Progress Example")

@mcp.tool()
async def long_running_task(
    task_name: str,
    ctx: Context[ServerSession, None],
    steps: int = 5
) -> str:
    """Execute a task with progress updates."""
    
    # Send progress notifications to client
    await ctx.info(f"Starting: {task_name}")
    
    for i in range(steps):
        progress = (i + 1) / steps * 100
        await ctx.progress(progress, f"Step {i+1}/{steps}")
        
        # Do actual work here
        await asyncio.sleep(1)
    
    await ctx.info(f"Completed: {task_name}")
    return f"Task {task_name} completed successfully"

Sampling lets your server request LLM completions. This is powerful for agentic workflows where the server orchestrates LLM calls without needing API keys:

@mcp.tool()
async def generate_commit_message(
    diff: str,
    ctx: Context[ServerSession, None]
) -> str:
    """Generate a commit message from a git diff."""
    
    # Ask the client's LLM to generate the message
    result = await ctx.session.create_message(
        messages=[{
            "role": "user",
            "content": f"Generate a concise commit message for this diff:\n\n{diff}"
        }],
        max_tokens=100
    )
    
    return result.content[0].text

Elicitation lets servers request additional information mid-operation. This is useful when the tool needs clarification from the user:

from mcp.types import ElicitRequestTextParams

@mcp.tool()
async def deploy_to_environment(
    service: str,
    ctx: Context[ServerSession, None]
) -> str:
    """Deploy a service to an environment."""
    
    # Ask user which environment
    result = await ctx.elicit(
        ElicitRequestTextParams(
            mode="text",
            message="Which environment? (staging/production)",
            placeholder="staging"
        )
    )
    
    environment = result.value
    
    # Validate input
    if environment not in ["staging", "production"]:
        raise ValueError("Environment must be 'staging' or 'production'")
    
    # Proceed with deployment
    return f"Deploying {service} to {environment}..."

Filesystem roots define security boundaries for which directories agents can access. FastMCP enforces that paths must be within configured roots that the client specifies when connecting:

@mcp.tool()
def read_file(path: str, ctx: Context[ServerSession, None]) -> str:
    """Read a file from the allowed workspace."""
    
    # FastMCP enforces that path must be within configured roots
    # Client specifies roots when connecting:
    #   roots=["file:///safe/workspace"]
    
    with open(path, 'r') as f:
        return f.read()

Case study: Vinyl collection chatbot

To see how easy FastMCP makes building real-world integrations, I built a serverless chatbot that answers questions about my vinyl collection. The data comes from a Discogs export—a CSV file containing my complete record collection that I store in S3.

Ask it “What Grimes records do I have?” and it queries the CSV, parses the Discogs data, and returns results. Ask it “What is vinyl?” and it just answers from general knowledge. The bot intelligently decides when to use tools and when to rely on its training data.

Here’s the complete implementation of the MCP server that queries the Discogs collection data:

from fastmcp import FastMCP
import boto3
import csv
from io import StringIO

# Initialize FastMCP server
mcp = FastMCP("vinyl-collection-server")

# S3 client for accessing Discogs export
s3_client = boto3.client('s3')
DATA_BUCKET = "my-vinyl-collection"

@mcp.tool()
def query_vinyl_collection(query_type: str, search_term: str, limit: int = 10) -> str:
    """
    Query Luke's vinyl record collection from Discogs export data.

    Args:
        query_type: One of: artist, label, year, title, all
        search_term: What to search for
        limit: Max results (default 10)
    """
    # Download Discogs CSV export from S3
    response = s3_client.get_object(Bucket=DATA_BUCKET, Key='discogs.csv')
    csv_content = response['Body'].read().decode('utf-8')
    
    # Parse CSV
    records = []
    csv_reader = csv.DictReader(StringIO(csv_content))
    for row in csv_reader:
        records.append(row)
    
    # Filter based on query type
    matches = []
    for record in records:
        if query_type == "artist" and search_term.lower() in record['Artist'].lower():
            matches.append(record)
        elif query_type == "label" and search_term.lower() in record['Label'].lower():
            matches.append(record)
        elif query_type == "year" and search_term in record['Released']:
            matches.append(record)
        elif query_type == "title" and search_term.lower() in record['Title'].lower():
            matches.append(record)
        elif query_type == "all":
            if (search_term.lower() in record['Artist'].lower() or 
                search_term.lower() in record['Title'].lower()):
                matches.append(record)
    
    # Format results
    results = []
    for record in matches[:limit]:
        results.append(
            f"{record['Artist']} - {record['Title']} "
            f"({record['Label']}, {record['Released']})"
        )
    
    return "\n".join(results) if results else "No matches found"

if __name__ == "__main__":
    mcp.run(transport="sse", port=8080)

That’s it. No JSON schema writing. No protocol handling. No tool registration boilerplate. FastMCP generates the schema from type hints and docstrings, handles the Model Context Protocol communication, converts everything to Bedrock’s format, and manages tool execution. You write the business logic for parsing Discogs data. FastMCP handles everything else.

The whole system—frontend, Lambda, Bedrock integration, S3 data access—deployed in under two minutes. This isn’t a toy example. It’s running in production, costs about fifteen dollars a month, and demonstrates why FastMCP is becoming the standard way to build tools for AI agents.

This is the full loop: user asks a question, Bedrock decides whether to use a tool, FastMCP executes it against the Discogs export data, and the result is returned—all serverless.

How AWS frontier agents use MCP

Now let’s connect this to the agents I’ve covered in previous articles. These patterns show how MCP enables coordination between specialized agents.

DevOps Agent scenario: A Lambda function starts timing out in production. DevOps Agent detects the anomaly through CloudWatch MCP Server where it reads metrics, logs, and traces. It queries the GitHub MCP Server to analyze recent code changes that might have introduced the issue. It creates an incident through the PagerDuty MCP Server and notifies the on-call engineer. Finally, it posts a summary to the incidents channel via the Slack MCP Server.

Without MCP, AWS would need custom connectors for every observability tool, ticketing system, and chat platform—an integration nightmare that doesn’t scale. With MCP, DevOps Agent uses a standard MCP client and any MCP-compatible tool works instantly.

Security Agent scenario: During a penetration test, Security Agent finds a SQL injection vulnerability. It reads the vulnerable code through the GitHub MCP Server, creates a security ticket via the Jira MCP Server, requests a code fix from Kiro through MCP, and once Kiro generates the fix, it creates a pull request back through GitHub’s MCP Server and notifies the security team via Slack’s MCP Server.

The agents coordinate through MCP without custom integration code. Security Agent discovers, Kiro fixes, DevOps Agent validates deployment—all using the same protocol.

Kiro scenario: A developer asks Kiro to “Add authentication to the user API endpoint.” Kiro reads the existing code through the Filesystem MCP Server, checks authentication patterns in other services via the GitHub MCP Server, queries your company’s authentication library documentation through an Internal Auth MCP Server, writes the updated code back through the Filesystem MCP Server, and creates a pull request via GitHub’s MCP Server.

Kiro doesn’t just generate code—it actively researches your codebase and standards through MCP servers, understanding context before making changes.

Building enterprise MCP servers: Real patterns

Here are three production-grade patterns that demonstrate how FastMCP handles enterprise integrations.

Salesforce MCP Server shows integration with a major CRM system:

from fastmcp import FastMCP
from simple_salesforce import Salesforce
import os

mcp = FastMCP("Salesforce CRM")

sf = Salesforce(
    username=os.getenv("SF_USERNAME"),
    password=os.getenv("SF_PASSWORD"),
    security_token=os.getenv("SF_SECURITY_TOKEN")
)

@mcp.tool()
def get_account(account_id: str) -> dict:
    """Retrieve Salesforce account details."""
    return sf.Account.get(account_id)

@mcp.tool()
def create_case(
    account_id: str,
    subject: str,
    description: str,
    priority: str = "Medium"
) -> dict:
    """Create a support case in Salesforce."""
    return sf.Case.create({
        'AccountId': account_id,
        'Subject': subject,
        'Description': description,
        'Priority': priority
    })

@mcp.tool()
def search_accounts(query: str, limit: int = 10) -> list[dict]:
    """Search Salesforce accounts by name."""
    soql = f"SELECT Id, Name, Industry, AnnualRevenue FROM Account WHERE Name LIKE '%{query}%' LIMIT {limit}"
    return sf.query(soql)['records']

With this server deployed, any agent can now access Salesforce—DevOps Agent, Security Agent, Kiro, or any future agent you build. No custom integration needed per agent. Build once, use everywhere.

Internal Wiki MCP Server demonstrates knowledge base integration:

from fastmcp import FastMCP
from elasticsearch import Elasticsearch
import os

mcp = FastMCP("Company Wiki")

es = Elasticsearch(
    os.getenv("ELASTICSEARCH_URL"),
    api_key=os.getenv("ELASTICSEARCH_API_KEY")
)

@mcp.resource("wiki://{article_id}")
def get_wiki_article(article_id: str) -> str:
    """Get wiki article by ID."""
    result = es.get(index="wiki", id=article_id)
    return result['_source']['content']

@mcp.tool()
def search_wiki(
    query: str,
    limit: int = 5
) -> list[dict]:
    """
    Search company wiki for relevant articles.
    
    Returns articles ranked by relevance.
    """
    response = es.search(
        index="wiki",
        body={
            "query": {
                "multi_match": {
                    "query": query,
                    "fields": ["title^2", "content", "tags"]
                }
            },
            "size": limit
        }
    )
    
    return [
        {
            "id": hit['_id'],
            "title": hit['_source']['title'],
            "excerpt": hit['_source']['content'][:200] + "...",
            "score": hit['_score'],
            "url": f"https://wiki.company.com/articles/{hit['_id']}"
        }
        for hit in response['hits']['hits']
    ]

This lets agents access institutional knowledge—Security Agent can read security policies, DevOps Agent can consult runbooks, and Kiro can reference coding standards, all from your internal wiki.

Multi-Tool Orchestration MCP Server aggregates data from multiple sources:

from fastmcp import FastMCP
from datadog_api_client import ApiClient, Configuration
from datadog_api_client.v1.api.metrics_api import MetricsApi

mcp = FastMCP("Operations Dashboard")

# Initialize multiple clients
datadog_config = Configuration()
datadog_config.api_key['apiKeyAuth'] = os.getenv("DD_API_KEY")
datadog_config.api_key['appKeyAuth'] = os.getenv("DD_APP_KEY")

@mcp.tool()
async def get_service_health(service_name: str) -> dict:
    """
    Get comprehensive health status for a service.
    
    Aggregates data from multiple monitoring systems.
    """
    # Get Datadog metrics
    with ApiClient(datadog_config) as api_client:
        api_instance = MetricsApi(api_client)
        metrics = api_instance.query_metrics(
            _from=int(time.time()) - 3600,
            to=int(time.time()),
            query=f"avg:system.cpu.user{{service:{service_name}}}"
        )
    
    # Get PagerDuty incidents (if integrated)
    # Get deployment status from CI/CD
    # Aggregate everything
    
    return {
        "service": service_name,
        "status": "healthy",  # or "degraded", "down"
        "cpu_usage": metrics['series'][0]['pointlist'][-1][1],
        "open_incidents": 0,
        "last_deployment": "2025-01-25T10:30:00Z",
        "error_rate": 0.02  # 2%
    }

This gives agents a unified view across disparate monitoring tools—one call returns comprehensive health status aggregated from Datadog, PagerDuty, and your CI/CD system.

Deploying MCP servers: Production patterns

Claude Desktop is the easiest way to test MCP servers during development. Create your MCP server using the patterns shown above, then configure Claude Desktop to use it. Edit the configuration file at ~/Library/Application Support/Claude/claude_desktop_config.json on Mac or %APPDATA%\Claude\claude_desktop_config.json on Windows:

{
  "mcpServers": {
    "github": {
      "command": "python",
      "args": ["/path/to/github_server.py"],
      "env": {
        "GITHUB_TOKEN": "your_token_here"
      }
    },
    "salesforce": {
      "command": "python",
      "args": ["/path/to/salesforce_server.py"],
      "env": {
        "SF_USERNAME": "your_username",
        "SF_PASSWORD": "your_password"
      }
    }
  }
}

Restart Claude Desktop and your agents will have access to these tools.

For production deployments, use remote MCP servers with SSE transport:

# server.py
from fastmcp import FastMCP

mcp = FastMCP("Production GitHub Server")

# ... (your tools here)

if __name__ == "__main__":
    # Run with SSE on custom port
    mcp.run(transport="sse", port=8080)

You can deploy this on AWS Lambda with a Lambda function URL, ECS or Fargate for containerized auto-scaling workloads, traditional EC2 instances, or AWS App Runner for fully managed hosting.

For AWS Bedrock AgentCore integration, configure the client like this:

from mcp import ClientSession, SseServerParameters
from mcp.client.sse import sse_client

server_params = SseServerParameters(
    url="https://mcp.company.com/github",
    headers={"Authorization": "Bearer YOUR_TOKEN"}
)

async with sse_client(server_params) as (read, write):
    async with ClientSession(read, write) as session:
        await session.initialize()
        tools = await session.list_tools()
        # Use tools...

AWS Bedrock AgentCore and MCP integration

This is where everything comes together. AgentCore natively supports MCP through its Gateway service.

graph TB
    subgraph AgentCore["AWS Bedrock AgentCore"]
        Runtime[AgentCore Runtime<br/>LangChain/LangGraph/Custom]
        Gateway[AgentCore Gateway<br/>Tool Discovery<br/>MCP Client<br/>Policy Enforcement<br/>Authentication]
        Runtime --> Gateway
    end
    
    Gateway --> GH[GitHub<br/>MCP Server]
    Gateway --> SF[Salesforce<br/>MCP Server]
    Gateway --> Custom[Your Custom<br/>MCP Server]

Integrating your FastMCP server with AgentCore follows three steps:

# Step 1: Build your MCP server with FastMCP
from fastmcp import FastMCP

mcp = FastMCP("Custom Enterprise Tool")

@mcp.tool()
def query_internal_database(sql: str) -> list[dict]:
    """Query internal PostgreSQL database."""
    # Your implementation
    pass

# Deploy to AWS (Lambda, ECS, etc.)
mcp.run(transport="sse", port=8080)

# Step 2: Register with AgentCore Gateway
# In AgentCore Console or via SDK:
import boto3

agentcore = boto3.client('bedrock-agentcore')

agentcore.register_mcp_server(
    name="internal-database",
    url="https://internal-mcp.company.com",
    authentication={
        'type': 'bearer',
        'tokenSecret': 'arn:aws:secretsmanager:...'
    }
)

# Step 3: Use in your agent
from langchain.agents import create_react_agent
from bedrock_agentcore import BedrockAgentCoreApp

app = BedrockAgentCoreApp()

@app.entrypoint
def my_agent(request):
    # Your agent automatically has access to all
    # MCP servers registered in AgentCore Gateway
    agent = create_react_agent(model, tools)
    return agent.invoke(request)

AgentCore Gateway gives you automatic tool discovery where it lists all registered MCP servers, policy enforcement so you can define which agents can use which tools, authentication handling through AgentCore Identity for OAuth and API keys, observability with every MCP call logged to CloudWatch, and scaling managed entirely by AgentCore infrastructure.

Security and best practices

Input validation is critical. Always validate and sanitize inputs before execution:

@mcp.tool()
def query_database(sql: str) -> list[dict]:
    """Execute a SQL query (SELECT only)."""
    
    # ✅ GOOD: Validate before execution
    dangerous_keywords = ['DROP', 'DELETE', 'UPDATE', 'INSERT', 'ALTER']
    if any(keyword in sql.upper() for keyword in dangerous_keywords):
        raise ValueError("Only SELECT queries allowed")
    
    # Additional validation
    if ';' in sql:
        raise ValueError("Multiple statements not allowed")
    
    return execute_query(sql)

Apply the principle of least privilege by granting minimal permissions. Instead of using full admin access tokens, use read-only or scoped tokens:

# BAD: Full admin access
github_token = os.getenv("GITHUB_ADMIN_TOKEN")

# GOOD: Read-only or scoped token
github_token = os.getenv("GITHUB_READONLY_TOKEN")

Rate limiting protects your backend systems from abuse:

from functools import lru_cache
from time import time

@lru_cache(maxsize=128)
def rate_limit_key(user_id: str) -> int:
    return int(time() / 60)  # 1-minute windows

@mcp.tool()
def expensive_operation(user_id: str, query: str) -> dict:
    """Rate-limited operation."""
    
    # Simple rate limiting (use Redis in production)
    key = f"{user_id}:{rate_limit_key(user_id)}"
    
    if request_count(key) > 10:
        raise Exception("Rate limit exceeded: 10 requests per minute")
    
    increment_count(key)
    return perform_operation(query)

Error handling should provide helpful errors without leaking internal details:

from mcp.types import ToolError

@mcp.tool()
def get_sensitive_data(resource_id: str) -> dict:
    """Access protected resource."""
    
    try:
        return fetch_resource(resource_id)
    except PermissionError:
        # GOOD: Generic error message
        raise ToolError("Access denied: insufficient permissions")
    except Exception as e:
        # BAD: Would leak internal details
        # raise ToolError(f"Database error: {str(e)}")
        
        # GOOD: Log internally, return generic error
        logger.error(f"Internal error: {e}", exc_info=True)
        raise ToolError("An internal error occurred")

The future of MCP

Based on current trajectory and industry patterns, MCP is evolving rapidly. In the near term—the next six to twelve months—expect broader ecosystem adoption with every major AI platform supporting MCP (OpenAI, Google, Microsoft, AWS), IDE integration in VS Code, JetBrains, and Zed, and browser extensions for Arc and Chrome. Enhanced security will standardize OAuth 2.0 flows, introduce fine-grained permission models, and establish audit logging specifications. Performance optimizations will add caching strategies, batch operations, and streaming support for large results.

The medium term—one to two years out—will bring enterprise features like an MCP server marketplace similar to AWS Marketplace, a certified and verified server registry, and SLA guarantees with monitoring. Advanced capabilities will enable multi-step workflows that chain tools across servers, state management for sessions and transactions, and push notifications where servers can alert clients. Developer experience improvements will include visual server builders for low-code MCP development, testing frameworks like pytest for MCP, and observability SDKs with OpenTelemetry integration.

Long term—two to five years—MCP becomes the HTTP of agentic AI. Every enterprise system will provide an MCP interface, legacy systems will get wrapped in MCP adapters, and universal agent communication becomes standard. Economic models will emerge with pay-per-call MCP servers, SaaS subscriptions for premium MCP servers, and even agent-to-agent commerce. The platform shift means developers stop building custom integrations entirely, “MCP-first” becomes the default architecture, and agents compose tools like developers compose libraries today.

Next steps

If you’re building agents, start by learning FastMCP basics in about thirty minutes—install it with pip, build the hello world server, and test with Claude Desktop. Then build an MCP server for your most-used tool in a couple hours—GitHub, Jira, Salesforce, or your internal API—following the patterns in this article and deploying locally to test thoroughly. Finally, integrate with your agents in about an hour by adding it to your Claude Desktop config or integrating with AgentCore Gateway, then verify agents can discover and use the tools.

If you’re at an enterprise, inventory your tool landscape over a day—list all systems agents need to access, identify which have existing MCP servers, and plan which you need to build custom. Build two to three pilot MCP servers in a week, starting with high-value, low-risk systems, using FastMCP for rapid development, and deploying to staging to test with real agents. Establish governance over two weeks by defining which agents can use which tools, implementing policy enforcement through AgentCore Policy, and setting up observability and monitoring. Then scale gradually over time, adding MCP servers incrementally, monitoring usage and costs, and iterating based on agent behavior.

Why this matters

Here’s the reality: The value of AI agents isn’t in the agents themselves—it’s in what they can access. An agent that can reason brilliantly but can’t touch your systems is a toy. An agent that can access your systems through MCP is a tool. A fleet of specialized agents, coordinating through MCP, is a transformation.

DevOps Agent is powerful. Security Agent is powerful. Kiro is powerful. But they’re only as powerful as the tools you give them through MCP.

That’s why understanding MCP isn’t optional—it’s foundational. It’s the difference between agents that demo well versus agents that actually work, prototypes in staging versus production deployments, vendor lock-in versus true interoperability.

FastMCP makes building these integrations trivial. The protocol is stable. The ecosystem is growing rapidly. The question isn’t “Should we adopt MCP?” It’s “Which systems do we connect first?”

Resources

For further exploration, start with the MCP Official Specification at modelcontextprotocol.io, the FastMCP GitHub repository at github.com/jlowin/fastmcp, and FastMCP Documentation at gofastmcp.com. The MCP Python SDK is available at github.com/modelcontextprotocol/python-sdk. AWS AgentCore Gateway documentation can be found in the AWS Bedrock AgentCore Documentation, and the MCP Server Registry is at github.com/modelcontextprotocol/servers.

If you’re building agents or integrating AI into enterprise systems, MCP is infrastructure you need to understand. Not as a nice-to-have, but as a prerequisite. Start with FastMCP. Build a server for your most-used tool. See how quickly you can give agents superpowers. The agents are here. The protocol is stable. The ecosystem is exploding. Time to plug in.

Not because they don’t care about security. Because security can’t keep up.

Over 60% of organizations update their web applications weekly or more frequently. Nearly 75% test those applications for security monthly or less. The math doesn’t work. The gap between development velocity and security validation grows wider every sprint.

At re:Invent 2025, AWS CEO Matt Garman announced AWS Security Agent—not as another security scanning tool to add to the pile, but as a fundamentally different approach to the problem.

Security Agent is a frontier agent that operates autonomously throughout the development lifecycle, conducting design reviews, analyzing code, and executing penetration tests on-demand, matching the pace of modern development instead of being its bottleneck.

You can watch the AWS Security Agent announcement here: https://www.youtube.com/watch?v=oMY0tUDEhtY

This is AWS’s bet that security doesn’t scale through more manual reviews—it scales through intelligent automation that understands your applications, your standards, and your threats.

What makes an agent “frontier-class”

AWS uses the term “frontier agent” to mean something specific. It’s not just GPT-4 with security tools.

1. Autonomous goal-directed behavior

Traditional security: “Run this SAST scan and give me findings.”
Frontier agent: “Validate this application meets our security requirements” → agent figures out how

You give it an objective, it decomposes the problem, forms hypotheses, collects evidence, and executes—without asking you for step-by-step guidance.

2. Multi-agent coordination

Security Agent doesn’t work alone. When conducting a penetration test, it spawns specialized sub-agents—one for authentication bypass, another for authorization flaws, a third for injection attacks. These agents work concurrently, investigating multiple attack vectors simultaneously and coordinating across findings.

3. Long-running and context-aware

Here’s the paradigm shift: Security Agent maintains persistent understanding of your applications.

Traditional security tools forget everything between scans. Security Agent learns:

• Your organizational security requirements
• Your application architecture and data flows
• Your code patterns and common vulnerabilities
• Your historical findings and remediation approaches

When testing your API, it doesn’t just throw generic payloads. It understands your authentication mechanism, maps your business logic, and targets application-specific vulnerabilities.

How it actually works

Security Agent operates across three phases of the development lifecycle, each with different capabilities:

Phase 1: Design Security Review

Before code exists, upload design documents, architecture diagrams, and threat models. Security Agent analyzes against:

• AWS security best practices
• Your organization’s security requirements
• Common architectural vulnerabilities
• Threat modeling patterns

Output: Security risk analysis with specific remediation guidance, in minutes instead of days.

Phase 2: Code Security Review

During development, GitHub integration provides automated security feedback on every pull request. Security Agent validates:

• Organizational security requirements (approved libraries, logging standards, data policies)
• OWASP Top 10 vulnerabilities
• Code-level security anti-patterns
• Compliance with your defined security standards

Developers get immediate feedback in their workflow—no context switching required.

Phase 3: On-Demand Penetration Testing

Whenever needed—pre-deployment, post-change, or on a schedule—Security Agent conducts comprehensive penetration testing.

Unlike traditional scanners, it:

• Builds understanding from your source code and architecture
• Creates customized attack plans based on your specific stack
• Executes multi-step attack chains (not just single-payload scans)
• Tests business logic vulnerabilities
• Validates findings to eliminate false positives
• Generates pull requests with remediation code

The penetration testing loop

When you trigger a pentest, here’s what happens:

graph TB
    Input[Target URLs + Code + Docs] --> Context[Build Application Understanding]
    
    Context --> Discovery[Discover Attack Surface<br/>Map endpoints, APIs, flows]
    
    Discovery --> Planning[Planning Agent<br/>Create customized attack plan]
    
    Planning --> Testing[Specialized Testing Agents]
    
    subgraph Testing[" "]
        Auth[Auth Bypass]
        Authz[Authorization]
        Inject[Injection]
        Logic[Business Logic]
    end
    
    Testing --> Validate[Validator Agent<br/>Eliminate false positives]
    
    Validate --> Remediate[Remediation Agent<br/>Generate code fixes]
    
    Remediate --> PR[Pull Request with Fix]

The clever part is the context-aware testing. Security Agent analyzes your source code to understand:

• Which endpoints are public vs. internal
• What authentication patterns you use
• How data flows through your system
• What your actual threat model looks like

When it sees JWT tokens, it doesn’t just test for SQL injection—it focuses on JWT-specific attacks like algorithm confusion, token tampering, and replay attacks.

Agent Spaces and organizational requirements

Everything starts with an Agent Space—the workspace where Security Agent operates and the security boundary for what it can access.

You might structure Agent Spaces as:

• Per-application: One space for your customer portal, another for your admin dashboard
• Per-team: One space per development team managing their portfolio
• Per-environment: Separate spaces for staging vs. production testing

The powerful part: you define your organization’s security standards once, centrally:

Authentication Requirements:
- OAuth 2.0 for all API endpoints
- JWT tokens with 15-minute expiration
- MFA required for admin functions

Data Protection Requirements:
- PII encrypted at rest (KMS)
- TLS 1.3 for data in transit
- No credit card data in logs

Logging Requirements:
- Correlation IDs on all requests
- Auth failures logged with IP
- No PII in application logs

These requirements automatically apply to all Agent Spaces, enforced during both design reviews and code reviews. Consistent enforcement across the organization—no more “this team follows standards, that team doesn’t.”

Deploying it (practical walkthrough)

AWS provides console-based setup. Here’s the flow:

Step 1: Create Agent Space

AWS Console → Security Agent → Create Agent Space
- Name: "production-security"
- Agent role: Auto-created

Step 2: Define Security Requirements

Security Requirements → Add requirements:
- Authentication standards
- Authorization patterns  
- Data protection policies
- Logging requirements
- Compliance frameworks

Step 3: Configure Penetration Testing

Agent Space → Enable penetration testing
- Add target domains (verify ownership)
- Configure CloudWatch logging
- Set up VPC access (for private apps)
- Add credentials to Secrets Manager

Step 4: Integrate with GitHub

Install AWS Security Agent GitHub App
- Authorize repository access
- Enable code review for Agent Space
- Auto-review triggered on PRs

Step 5: Execute Penetration Test

Security Agent Web App → Create pentest
- Target: https://staging.app.example.com
- Authentication: From Secrets Manager
- Attach: Source code + design docs
- Enable automatic remediation PRs

Watch as Security Agent:

• Discovers attack surface
• Executes targeted scenarios
• Validates findings
• Creates PRs with fixes

Testing it

AWS provides real test scenarios. Run these before connecting production:

Test 1: API authentication bypass

Deploy an API with intentionally weak JWT validation:

# Weak JWT verification
def verify_token(token):
    # Missing signature validation
    payload = jwt.decode(token, verify=False)
    return payload['user_id']

Trigger Security Agent pentest, watch it:

• Identify JWT usage
• Test algorithm confusion
• Attempt signature bypass
• Generate exploit proof
• Create PR with proper validation

Test 2: SQL injection in query

Deploy an endpoint with SQL injection:

# Vulnerable query
def get_user(user_id):
    query = f"SELECT * FROM users WHERE id = {user_id}"
    return db.execute(query)

Security Agent should:

• Detect SQL construction
• Test injection vectors
• Confirm exploitability
• Recommend parameterized queries

What’s not ready yet (the honest limitations)

1. us-east-1 only

Security Agent is currently only available in us-east-1.

If you have data residency requirements (GDPR, finance, healthcare), this is a blocker. Applications in other regions must be accessible from us-east-1.

Mitigation: Test staging/dev environments. AWS will expand regions post-GA.

2. GitHub-only code review

Currently only GitHub is supported for automated code review.

What’s missing:

• GitLab
• Bitbucket
• AWS CodeCommit
• Azure DevOps

Workaround: You can still use penetration testing by providing code via S3.

3. Not a replacement for professional pentesting

Security Agent is powerful but not guaranteed to discover all vulnerabilities. It’s best used for continuous testing at development velocity.

AWS’s position: “AWS Security Agent is not a professional penetration testing service, and we encourage users to integrate AWS Security Agent into their security review workflow.”

The right mental model: Security Agent is your continuous validation layer. Professional pentesters are your comprehensive pre-launch audit.

4. False positive management

While Validator Agents significantly reduce false positives, AI-powered testing will never be 100% perfect.

What AWS does:

• Only reports high/medium confidence findings
• Hides unverified findings by default
• Provides reproducible exploit paths

What you should do:

• Review findings with security expertise
• Validate critical findings independently
• Use CloudWatch logs to understand agent reasoning

5. Learning curve

Security Agent builds topology understanding over time. Early investigations might be less accurate.

Mitigation:

• Run test assessments to let it learn
• Tag resources consistently
• Document dependencies explicitly

Should you actually use this?

Use it if:

• Your development velocity is outpacing security capacity
• You deploy weekly but test security monthly
• You knowingly ship vulnerable code to meet deadlines
• You want to scale security across your entire portfolio
• You’re AWS-native (tight integration benefits)
• You’re comfortable with preview-phase tech

Wait if:

• You need multi-region support now
• You use source control other than GitHub (for code review)
• Your security process already keeps pace with development
• You need production SLAs (preview = no SLAs)
• You require deterministic pricing

Key insight: Security Agent amplifies good practices and exposes bad ones. If your infrastructure is poorly tagged, deployments aren’t tracked, and requirements are scattered, it’ll struggle. But if you have solid foundations, it can be transformative.

My honest take

This is the future of application security. Not because AI replaces security engineers, but because it handles undifferentiated heavy lifting.

The traditional model: Security is a gate. Development builds features, security reviews them, findings go back to development, repeat until deadlines force compromise.

The agentic model: Security is embedded. AI agents continuously validate security throughout development, provide real-time guidance, and scale security expertise to match development velocity.

Security Agent doesn’t eliminate the need for security expertise—it amplifies what your security team can accomplish. One senior security engineer using Security Agent can cover more applications than a team of five without it.

The question isn’t whether agentic security is coming—it’s here. The question is whether you’re ready when GA drops.

If you’re experimenting with this or have questions, reach out on LinkedIn. The technology is moving fast, and we’re all figuring it out together.

Resources:

• AWS Security Agent User Guide
• AWS Security Agent Product Page
• AWS Frontier Agents Overview

Why Add Authentication?

Right now, anyone can vote, and anyone can reset the entire survey. In a real-world application, we need to control access. Authentication allows us to:

• Prevent abuse: Stop users from voting multiple times.
• Secure administrative functions: Only allow authorized users to reset the survey.
• Personalize the user experience: (Future enhancement) Show users their past votes.

Exploring the Options

When adding authentication to a serverless app on AWS, there are a few common patterns:

1. API Keys: The simplest approach. We could generate an API key and require it for certain API endpoints.

   • Pros: Easy to implement.
   • Cons: Not true user authentication. Keys can be shared or leaked. Doesn’t scale for managing individual users.

2. Lambda Authorizers (Custom Authorizers): We can write a custom Lambda function that is triggered by API Gateway before the main handler. This function would be responsible for validating a token (e.g., a session token you manage yourself).

   • Pros: Full control over the authentication logic.
   • Cons: You have to build and manage the entire user lifecycle yourself (sign-up, sign-in, password reset, etc.). This is a lot of work and easy to get wrong.

3. AWS Cognito: A fully managed user identity and authentication service. Cognito handles all the heavy lifting of user management, including sign-up, sign-in, password recovery, and multi-factor authentication (MFA). It integrates seamlessly with API Gateway.

   • Pros: Secure, scalable, and feature-rich. Offloads the undifferentiated heavy lifting of authentication.
   • Cons: Can seem complex at first due to the number of features.

For our application, AWS Cognito is the clear winner. It provides the best balance of security, features, and ease of integration.

Understanding the Current Architecture

Before we add authentication, let’s visualize how our serverless application currently works:

Right now, anyone can call our API endpoints. There’s no way to verify who’s making the request or prevent abuse.

The Plan: Integrating Cognito

We’ll use a Cognito User Pool to manage our users. Think of it as a user database with built-in authentication logic. Here’s the high-level plan:

1. Infrastructure (Terraform):

   • Create a Cognito User Pool to store and manage users.
   • Create a Cognito User Pool Client, which allows our frontend application to interact with the User Pool.
   • Add an authorizer to our API Gateway to protect our endpoints.

2. Frontend (HTML/JavaScript):

   • Create a login page (login.html).
   • Add sign-up and sign-in forms.
   • Use the Amazon Cognito Identity SDK for JavaScript to communicate with Cognito.
   • On successful sign-in, store the JWT (JSON Web Token) from Cognito in local storage.
   • Include the JWT in the Authorization header of all subsequent API requests.
   • Add a “Logout” button.

3. Backend (Lambda):

   • Update our Lambda functions to expect and validate the JWT from Cognito. API Gateway will do most of the validation for us.
   • We can use the claims inside the JWT to identify the user. For instance, we can use the user’s sub (subject) claim instead of the sessionId to track votes.

Here’s what the authenticated architecture will look like:

Let’s Get Building!

Step 1: Beefing up our Terraform

First, we need to add the Cognito resources to our terraform/main.tf file.

# --- Cognito User Pool ---
resource "aws_cognito_user_pool" "survey_user_pool" {
  name = "SurveyUserPool"
  auto_verified_attributes = ["email"]
}

resource "aws_cognito_user_pool_client" "survey_user_pool_client" {
  name = "SurveyUserPoolClient"
  user_pool_id = aws_cognito_user_pool.survey_user_pool.id
  generate_secret = false # This is a public client
  explicit_auth_flows = ["ALLOW_USER_PASSWORD_AUTH", "ALLOW_REFRESH_TOKEN_AUTH"]
}

We also need to update our API Gateway to use a Cognito authorizer.

# --- API Gateway ---
resource "aws_apigatewayv2_api" "survey_api" {
  name          = "CrackingTheCloudSurveyAPI"
  protocol_type = "HTTP"
  cors_configuration {
    allow_origins = ["*"]
    allow_methods = ["POST", "GET", "OPTIONS"]
    allow_headers = ["*"]
  }
}

resource "aws_apigatewayv2_authorizer" "cognito_authorizer" {
  api_id           = aws_apigatewayv2_api.survey_api.id
  authorizer_type  = "JWT"
  identity_sources = ["$request.header.Authorization"]
  name             = "CognitoAuthorizer"
  jwt_configuration {
    audience = [aws_cognito_user_pool_client.survey_user_pool_client.id]
    issuer   = "https://${aws_cognito_user_pool.survey_user_pool.endpoint}"
  }
}

# Update the routes to use the authorizer
resource "aws_apigatewayv2_route" "vote_route" {
  api_id    = aws_apigatewayv2_api.survey_api.id
  route_key = "POST /vote"
  target    = "integrations/${aws_apigatewayv2_integration.vote_integration.id}"
  authorization_type = "JWT"
  authorizer_id = aws_apigatewayv2_authorizer.cognito_authorizer.id
}

resource "aws_apigatewayv2_route" "reset_route" {
  api_id    = aws_apigatewayv2_api.survey_api.id
  route_key = "POST /reset"
  target    = "integrations/${aws_apigatewayv2_integration.reset_integration.id}"
  # Note: we might want more fine-grained control here in a real app
  authorization_type = "JWT"
  authorizer_id = aws_apigatewayv2_authorizer.cognito_authorizer.id
}

Step 2: The Frontend - Where the Magic Happens

This is where we’ll see the biggest changes. We need a way for users to sign up and sign in.

First, let’s create a new login.html page. This will be a simple page with forms for sign-up and sign-in.

We’ll also need to include the AWS Cognito Identity SDK in our project. We can either download it and host it ourselves, or use a CDN. For simplicity, we’ll use a CDN in our HTML files.

<!-- In login.html, index.html, etc. -->
<script src="https://cdn.jsdelivr.net/npm/amazon-cognito-identity-js@5.2.10/dist/amazon-cognito-identity.min.js"></script>

Our main.js will need a significant update. Here are the key parts:

// Add these variables at the top of main.js
const API_URL = '${API_URL}';
const COGNITO_USER_POOL_ID = '${COGNITO_USER_POOL_ID}';
const COGNITO_CLIENT_ID = '${COGNITO_CLIENT_ID}';

const poolData = {
    UserPoolId: COGNITO_USER_POOL_ID,
    ClientId: COGNITO_CLIENT_ID
};
const userPool = new AmazonCognitoIdentity.CognitoUserPool(poolData);

// On page load, check if the user is logged in
window.onload = function() {
    const idToken = localStorage.getItem('idToken');
    if (!idToken && !window.location.href.endsWith('login.html')) {
        window.location.href = 'login.html';
    } else if (idToken && window.location.href.endsWith('index.html')) {
        addSignOutButton();
    }
};

function addSignOutButton() {
    const container = document.getElementById('signout-container');
    if (container) {
        const signOutButton = document.createElement('button');
        signOutButton.textContent = 'Sign Out';
        signOutButton.onclick = signOutUser;
        container.appendChild(signOutButton);
    }
}

function signUpUser() {
    const email = document.getElementById('signUpEmail').value;
    const password = document.getElementById('signUpPassword').value;
    const messageDiv = document.getElementById('message');

    const attributeList = [];
    const dataEmail = {
        Name: 'email',
        Value: email,
    };
    const attributeEmail = new AmazonCognitoIdentity.CognitoUserAttribute(dataEmail);
    attributeList.push(attributeEmail);

    userPool.signUp(email, password, attributeList, null, function(err, result){
        if (err) {
            messageDiv.textContent = err.message || JSON.stringify(err);
            return;
        }
        messageDiv.textContent = 'Sign up successful! Please check your email for a verification code.';
        document.getElementById('confirm-container').style.display = 'block';
    });
}

function confirmSignUpUser() {
    const email = document.getElementById('signUpEmail').value;
    const code = document.getElementById('confirmationCode').value;
    const messageDiv = document.getElementById('message');

    const userData = {
        Username: email,
        Pool: userPool
    };

    const cognitoUser = new AmazonCognitoIdentity.CognitoUser(userData);
    cognitoUser.confirmRegistration(code, true, function(err, result) {
        if (err) {
            messageDiv.textContent = err.message || JSON.stringify(err);
            return;
        }
        messageDiv.textContent = 'Confirmation successful! You can now sign in.';
        document.getElementById('confirm-container').style.display = 'none';
    });
}

function signInUser() {
    const email = document.getElementById('signInEmail').value;
    const password = document.getElementById('signInPassword').value;
    const messageDiv = document.getElementById('message');

    const authenticationData = {
        Username: email,
        Password: password,
    };
    const authenticationDetails = new AmazonCognitoIdentity.AuthenticationDetails(authenticationData);
    const userData = {
        Username: email,
        Pool: userPool
    };
    const cognitoUser = new AmazonCognitoIdentity.CognitoUser(userData);

    cognitoUser.authenticateUser(authenticationDetails, {
        onSuccess: function (result) {
            const idToken = result.getIdToken().getJwtToken();
            localStorage.setItem('idToken', idToken);
            window.location.href = 'index.html';
        },
        onFailure: function(err) {
            messageDiv.textContent = err.message || JSON.stringify(err);
        },
    });
}

function signOutUser() {
    localStorage.removeItem('idToken');
    const cognitoUser = userPool.getCurrentUser();
    if (cognitoUser) {
        cognitoUser.signOut();
    }
    window.location.href = 'login.html';
}

async function vote(option) {
    const idToken = localStorage.getItem('idToken');
    if (!idToken) {
        window.location.href = 'login.html';
        return;
    }

    const messageDiv = document.getElementById('message');
    messageDiv.textContent = 'Submitting your vote...';

    try {
        const response = await fetch(`${API_URL}vote`, {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'Authorization': `Bearer ${idToken}`
            },
            body: JSON.stringify({ vote: option }),
        });

        if (!response.ok) {
            throw new Error(`HTTP error! status: $${response.status}`);
        }

        messageDiv.textContent = 'Thank you for voting!';
        document.querySelectorAll('.vote-btn').forEach(button => {
            button.disabled = true;
        });
        
    } catch (error) {
        console.error('Error submitting vote:', error);
        messageDiv.textContent = 'Sorry, there was an error submitting your vote.';
    }
}

Step 3: Backend Adjustments

Our backend Lambda functions need a small change. Since we are now identifying users by their JWT, we can remove the sessionId logic. The vote.py function will now get the user’s unique ID from the JWT claims that API Gateway passes along.

Here’s how the Lambda function processes an authenticated request:

# In backend/vote.py

def handler(event, context):
    try:
        body = json.loads(event.get('body', '{}'))
        vote_option = body.get('vote')

        # Get user id from the authorizer context
        # API Gateway extracts this from the JWT and passes it to Lambda
        user_id = event['requestContext']['authorizer']['jwt']['claims']['sub']

        if not user_id:
            return {'statusCode': 400, ...}

        if vote_option not in ['no', 'aws', 'other']:
            return {'statusCode': 400, ...}

        item = {
            'id': user_id,      # Use the cognito user id as the primary key
            'vote': vote_option
        }
        table.put_item(Item=item)

        return {'statusCode': 200, ...}
    except Exception as e:
        # ...