How to Build RAG on Your S3 Documents with Bedrock

If your documents already sit in Amazon S3 and you want to ask questions of them, the shortest path on AWS is Amazon Bedrock Managed Knowledge Base: point it at the bucket, and it handles parsing, chunking, embeddings, vector storage and retrieval. There is no vector database to choose, provision or scale, and the managed embedding model and reranker are included at no extra cost. Generally available since June 2026, it is the option to start with unless you have a specific reason to run your own retrieval pipeline.

Pick the Right Kind of Knowledge Base First

Bedrock offers three, and choosing wrongly is the most common way this gets harder than it needs to be.

• Managed Knowledge Base — fully managed connectors, parsing, vector store and embedding model. No infrastructure.
• Customer-managed (unstructured) — you build the retrieval pipeline and bring your own vector store, for full control over index storage and retrieval.
• Customer-managed (structured) — for querying structured data in Amazon Redshift, converting natural language to SQL.

For documents in S3, that is a choice between the first two. The differences that actually decide it:

	Managed	Customer-managed
Vector store	Auto-scaling, managed entirely by Bedrock	You choose, provision, scale and update it
Embedding model	Built-in managed model, no extra cost	You choose and pay for one
Reranking	Built-in semantic reranker, no extra cost	You choose your own reranker
Agentic retrieval	Supported	Not supported
Search	Agentic and semantic hybrid retrieval	Whatever you build
Connectors	7 — S3, SharePoint, Confluence, Google Drive, OneDrive, Web Crawler, Custom	S3 and Custom
Parsing	Built-in multimodal parser	Default, Foundation Model, or Bedrock Data Automation
AgentCore Gateway	Supported	Not supported
Infrastructure	None	You provision and maintain the vector DB

The managed embedding model and reranker being free is the part most comparisons miss. On the customer-managed path both are line items you configure and pay for.

Go customer-managed when you need direct access to the index, a specific vector store, or a retrieval strategy of your own design. Otherwise start managed — you can always build the bespoke pipeline later, and you will know more about your retrieval quality by then.

How It Works

Two phases, and it is worth knowing which one your problems live in.

Pre-processing — your data is converted to text and split into chunks. Chunks are converted to embeddings and stored in a vector store. Images, audio and video are processed according to the parser and embedding model you chose.

Runtime execution — your query is converted to a vector, compared against stored embeddings to find semantically similar content, and the retrieved material augments the foundation model’s response.

Retrieval quality problems almost always originate in the first phase. That matters because of what comes next.

Setting It Up

Prerequisites

• The bucket must be in the same Region as the knowledge base. Not negotiable, and the easiest thing to get wrong if your data lives where it was first created.
• Note the bucket URI, ARN, and the owner’s AWS account ID — cross-account buckets are supported, and the owner ID is how you point at one.
• An IAM role with permission to reach the data source. If you create the knowledge base in the console, the role with all required permissions can be created for you, which is the fastest way to a working setup.

Creating the data source

Managed and customer-managed knowledge bases use different configuration shapes for the same bucket, which is a common source of confusion when following mixed documentation. Managed uses MANAGED_KNOWLEDGE_BASE_CONNECTOR; customer-managed uses s3Configuration.

response = bedrock_agent.create_data_source(
    knowledgeBaseId=kb_id,
    name="my-s3-data-source",
    description="Product documentation from S3",
    dataSourceConfiguration={
        "type": "MANAGED_KNOWLEDGE_BASE_CONNECTOR",
        "managedKnowledgeBaseConnectorConfiguration": {
            "connectorParameters": {
                "type": "S3",
                "version": "1",
                "connectionConfiguration": {
                    "bucketName": "your-bucket-name",
                    "bucketOwnerAccountId": "123456789012"
                },
                "filterConfiguration": {
                    "inclusionPrefixes": ["documents/"]
                }
            }
        }
    },
    vectorIngestionConfiguration={
        "parsingConfiguration": {"parsingStrategy": "SMART_PARSING"}
    }
)

Two things in there worth dwelling on.

inclusionPrefixes scopes what gets crawled. This is the argument for organising the bucket so that text destined for retrieval lives under one prefix, separate from binaries and anything else. Get that right when you lay the bucket out and enabling retrieval is a config change; get it wrong and it is a migration.

CreateDataSource is asynchronous for managed knowledge bases. Status goes CREATING → AVAILABLE, typically in two to five minutes. Do not start ingestion until it reads AVAILABLE — a script that creates and immediately ingests will fail intermittently, in a way that looks like a permissions problem.

The decisions you cannot undo

Parsing and chunking cannot be modified after creation. Changing your mind means creating a new data source and re-ingesting everything.

For parsing, managed knowledge bases use a built-in multimodal parser with SMART_PARSING. On the customer-managed path you choose between the Bedrock default parser (text-only, no additional cost), Bedrock Data Automation (PDFs, images, audio, video), or a foundation model as parser (images, tables, visually rich documents).

For chunking, managed gives you built-in (default) or fixed-size. The customer-managed path opens up more: standard fixed-size for uniform documents, hierarchical for documents with clear section structure, semantic for splitting on topic boundaries, or custom chunking via a Lambda function.

If you are unsure, take the built-in default. It is tuned for mixed content, and the honest way to choose between chunking strategies is to measure retrieval quality on your own corpus — which you cannot do before you have ingested anything.

Embedding model choice is similarly sticky. Managed knowledge bases include an optimised model at no cost; if you supply your own it must be a Bedrock embedding model with float32 and 1024 dimensions. Changing embedding models later means re-embedding the corpus.

Syncing

The connector crawls new, modified and deleted content on each sync. The first sync crawls everything; subsequent syncs are incremental. Trigger one with StartIngestionJob, or Sync in the console under the data source.

New documents in the bucket are not retrievable until a sync runs. If your corpus changes daily, that sync is a scheduled job you need to own — it is not automatic on upload.

Querying It

Four API operations, and picking the right one shapes how much of RAG you are actually building.

• Retrieve — returns the most relevant source chunks as an array. You do the generation yourself. Use this when you want to control the prompt, post-process results, or feed retrieval into something other than a chat response.
• RetrieveAndGenerate — combines Retrieve with InvokeModel and returns a natural language answer with citations to specific source chunks. This is the whole RAG loop in a single call. A streaming variant, RetrieveAndGenerateStream, exists for token-by-token responses.
• GenerateQuery — converts natural language into a query suitable for a structured data store.
• AgenticRetrieveStream — uses a foundation model to decompose complex queries into sub-queries, retrieves iteratively, and evaluates whether results are sufficient. Returns deduplicated chunks plus trace events for observability. This is the multi-hop path, and it is managed knowledge bases only.

For conversational use, RetrieveAndGenerate returns a sessionId. Reuse it on subsequent requests to maintain context. You cannot set it yourself — Bedrock generates it on the first call.

Citations are worth building on rather than treating as decoration. Source attribution is what lets a user check an answer, and what gives you an audit trail when someone asks why the system said something.

The Security Behaviour to Understand Before You Sync

This one deserves a paragraph of its own, because it is easy to miss and awkward to undo:

All data that you sync from your data source becomes available to anyone with bedrock:Retrieve permissions to retrieve the data. This can also include any data with controlled data source permissions.

Syncing flattens source-level permissions by default. A document that only three people could read in the source system becomes retrievable by anyone who can call Retrieve against that knowledge base. Point a knowledge base at a bucket holding HR files, contracts and board material, and you have built a way to ask questions of all of it.

Two mechanisms address this. Managed knowledge bases support Access Control List awareness, which carries source permissions through. And RetrieveAndGenerate accepts a userContext parameter that filters results to documents the requesting user is authorised to see.

Neither is on by default. Decide access model before the first sync, not after someone demonstrates the problem.

Metadata filtering is the related tool for narrowing retrieval generally — attach attributes to documents and pre-filter the vector store before searching. It reduces noise and improves accuracy even where access control is not the concern.

Check Region Availability Early

At general availability in June 2026, Managed Knowledge Base was offered in US East (N. Virginia), US West (Oregon), Asia Pacific (Sydney and Tokyo), Europe (Dublin, Frankfurt and London), and AWS GovCloud (US-West).

Asia Pacific (Mumbai) was not in that list, which matters if you have data residency requirements in India. Note that this is a different list from the one for knowledge bases with structured data stores, which does include Mumbai — the two are easy to conflate when scanning the docs. Regional coverage expands, so check the supported Regions page rather than trusting any list in a blog post, this one included.

Because the bucket must sit in the same Region as the knowledge base, region availability and data residency are the same question here.

When to Go Customer-Managed Instead

The managed path stops being right when you need something specific from the index itself: a particular vector store, direct access to it, a retrieval strategy you have designed, or cost behaviour tuned to an unusual query pattern.

At that point the vector store becomes your decision rather than Bedrock’s, and the trade-offs are worth understanding before you make it — S3 Vectors vs OpenSearch Serverless for RAG covers that choice.

It is also worth asking whether you need retrieval at all. A corpus of a few hundred documents often fits inside a single model context window, and at that size chunking splits arguments into fragments where reading the whole document preserves the reasoning.

Summary

• Start with Managed Knowledge Base. Point it at the bucket; parsing, chunking, embeddings, vector storage and retrieval are handled, and the managed embedding model and reranker cost nothing extra.
• The bucket must be in the same Region as the knowledge base.
• Parsing, chunking and embedding model are effectively immutable. Changing them means re-ingesting. Take the built-in defaults unless you have measured a reason not to.
• CreateDataSource is asynchronous — wait for AVAILABLE, typically two to five minutes, before ingesting.
• Scope the crawl with inclusionPrefixes, and lay the bucket out so retrievable text sits under its own prefix.
• Syncing flattens source permissions. Enable ACL awareness or pass userContext, and decide this before the first sync.
• RetrieveAndGenerate is the whole loop in one call; Retrieve gives you the chunks if you want control; AgenticRetrieveStream handles multi-hop and is managed-only.
• Syncs are not automatic on upload — schedule StartIngestionJob if the corpus changes.

Service limits, Region availability and pricing move. Verify against the Bedrock Knowledge Bases documentation before committing to a design.