OCI RAG Agent Blueprint

Retrieval-Augmented Generation becomes useful in production only when it is treated as an engineered system: grounded retrieval, explicit runtime contracts, repeatable deployment, and testable client behavior.

This repository is a version 1.0 blueprint for building and deploying a RAG agent on OCI Enterprise AI, using OCI Vector Store for retrieval and the OpenAI-compatible Responses API for generation.

Version 1.0 has been validated with OCI Hosted Deployments in both non-streaming and streaming request modes. The Python CLI client and the Next.js reference UI both support the Hosted Application invoke gateway behavior where SSE data: frames are preserved but explicit event: names may be stripped.

What Can I Build With This?

Use this blueprint when you need a working starting point for a production-style RAG assistant on OCI, not just a minimal API sample.

Typical use cases include:

• An internal knowledge assistant that answers questions from company documentation, policies, runbooks, or onboarding material.
• A support assistant that retrieves product, troubleshooting, or service documentation before generating an answer.
• A technical documentation chatbot for engineering teams, field teams, or customers who need grounded answers with references.
• A policy, compliance, or procedure assistant where answers must stay connected to a controlled document collection.
• A workshop or proof-of-concept environment that demonstrates OCI Vector Store, the Responses API, streaming responses, short-term conversation memory, and a deployable reference UI.
• A reusable backend foundation for custom applications that need a /responses API compatible with both local development and OCI Hosted Application invoke endpoints.

The repository gives you the pieces needed to go from synchronized documents in OCI Vector Store to a local or hosted chat experience: backend agent, request and response contracts, streaming behavior, command-line validation, reference UI, deployment guidance, and a guided Agent Factory workflow.

Start Here

Use these guides depending on what you want to do:

• Quickstart: end-to-end path from OCI resources to a working local and hosted RAG demo.
• Agent API Usage: exact endpoints, request payloads, non-streaming examples, streaming examples, and Hosted Application invoke notes.
• Agent Factory: guided web UI and API for building and deploying the backend container to OCI Enterprise AI Hosted Applications.
• Environment Variables: complete runtime configuration reference.
• OCI Enterprise AI Deployment Guide: detailed hosted deployment procedure.
• Document Loading: upload local documents to Object Storage and trigger Vector Store connector synchronization.
• Troubleshooting FAQ: recurring operational issues and fixes.
• Possible Future Improvements: numbered roadmap proposals for future blueprint enhancements.

What You Get

The blueprint includes:

• A FastAPI backend agent built around the Responses API.
• OCI Vector Store file search integration.
• Short-term conversation management through Responses API conversations.
• Streaming and non-streaming /responses request paths.
• A Python CLI test client for local and hosted endpoint validation.
• A Next.js reference chatbot UI with Markdown rendering and streaming support.
• An Agent Factory application for guided OCI Hosted Application deployment.
• Docker Compose local development for the backend and reference UI.
• JSON request and response schemas.
• Specs, tests, linting, and coverage rules that keep behavior reviewable.

Runtime API

The agent exposes:

GET /health
POST /responses

For local development:

http://localhost:8080/health
http://localhost:8080/responses

For OCI Hosted Application invoke:

https://inference.generativeai.<region>.oci.oraclecloud.com/20251112/hostedApplications/<hosted-application-ocid>/actions/invoke/health
https://inference.generativeai.<region>.oci.oraclecloud.com/20251112/hostedApplications/<hosted-application-ocid>/actions/invoke/responses

Use stream: false for one JSON response and stream: true for Server-Sent Events.

See Agent API Usage for complete payload examples and curl commands.

Local Demo

Before starting the demo, create a root .env file from .env.sample and fill in the required OCI Enterprise AI values:

cp .env.sample .env

Start both local services:

./start_demo.sh

Build images and then start both services:

./start_demo.sh --build

The local deployment starts:

• rag-agent on http://localhost:8080
• rag-ui on http://localhost:3000

Open:

http://localhost:3000

Stop the demo:

./stop_demo.sh

Run The UI Without Docker

You can run the Next.js reference UI directly on your workstation:

cd ui
npm install
npm run dev

Then open:

http://localhost:3000

Set the UI backend URL to either the local /responses endpoint or the Hosted Application invoke /responses endpoint.

JWT authentication is disabled by default in the reference UI. Keep it disabled for the local Docker Compose backend. When the backend URL points to a Hosted Application protected with IDCS_AUTH_CONFIG, enable JWT authentication in the sidebar. Use the values from a configured OCI IAM confidential application: Identity Domain URL, Client ID, Client secret, and concatenated IDCS token request scope. Then use Test health to validate access to the protected /health endpoint. The UI requests the IDCS access token through its server-side Next.js route and sends Authorization: Bearer <token> on protected /responses calls only while JWT authentication is enabled.

Python CLI

From the repository root:

python -m clients.agent_cli \
  --endpoint "http://localhost:8080/responses" \
  --create-conversation true \
  --stream false \
  "Answer with only: ok"

Streaming:

python -m clients.agent_cli \
  --endpoint "http://localhost:8080/responses" \
  --create-conversation true \
  --stream true \
  "Answer with only: ok"

For Hosted Application validation, replace the endpoint with:

https://inference.generativeai.<region>.oci.oraclecloud.com/20251112/hostedApplications/<hosted-application-ocid>/actions/invoke/responses

For a Hosted Application protected with IDCS_AUTH_CONFIG, you must first have a correctly configured confidential application in OCI IAM Identity Domains. The confidential application must allow the OAuth Client credentials grant and you must have its Client ID and Client secret. Oracle documents this setup in Adding a Confidential Application.

Then set the client-side IDCS values in the root .env file:

IDENTITY_DOMAIN_URL=https://idcs-example.identity.oraclecloud.com
CONFIDENTIAL_APPLICATION_ID=replace-with-confidential-application-id
CONFIDENTIAL_APPLICATION_SECRET=replace-with-confidential-application-secret
IDCS_SCOPE=replace-with-primary-audience-plus-scope

Then run the Python client with --auth idcs:

python -m clients.agent_cli \
  --endpoint "https://inference.generativeai.<region>.oci.oraclecloud.com/20251112/hostedApplications/<hosted-application-ocid>/actions/invoke/responses" \
  --auth idcs \
  --create-conversation true \
  --stream true \
  "Answer with only: ok"

For OCI IAM IDCS auth, the Hosted Application keeps audience and scope separate, while the client token request uses the concatenated IDCS_SCOPE value. For example, Hosted Application audience=hello_world and scope=invoke means client IDCS_SCOPE=hello_worldinvoke. See OCI IAM IDCS Audience And Scope.

For a complete hosted diagnostic run, edit and execute:

./test_hosted_application.sh

Agent Factory

The agent-factory/ application provides a guided deployment workflow for the backend container:

• Resolve OCI resource names and OCIDs.
• Validate OCIR credentials.
• Build and push the backend image.
• Create or reuse a Hosted Application.
• Create a Hosted Deployment from the container artifact.
• Track command output and deployment status.
• Show the final Hosted Application invoke, health, and /responses URLs.

See Agent Factory for setup and usage.

Development Approach

This repository follows spec-driven development.

Every new capability starts with a specification under specs/. Code is written after the expected behavior, acceptance criteria, and test expectations are documented.

This keeps the project aligned around a simple rule: implementation must conform to the specification, not the other way around.

Quality Standards

Python code in this repository must follow these standards:

• Source files include the required project header.
• Code is formatted with black.
• Code is checked with pylint.
• Unit tests are written with pytest.
• New functionality targets more than 80% test coverage.
• Work is considered done only when formatting, linting, tests, and related fixes are complete.

Next.js UI changes must pass:

cd ui
npm run test
npm run lint
npm run build

See AGENTS.md for the full working guidelines.

Repository Structure

.
├── AGENTS.md
├── CHANGELOG.md
├── QUICKSTART.md
├── TROUBLESHOOTING.md
├── agent/
├── agent-factory/
├── clients/
├── docs/
├── schemas/
├── specs/
├── tests/
└── ui/

Current Status

Version 1.0 is ready for use as a working OCI Enterprise AI RAG blueprint. It has been tested with:

• Local backend and reference UI.
• OCI Hosted Application health checks.
• Hosted Deployment non-streaming /responses requests.
• Hosted Deployment streaming /responses requests.
• Python CLI streaming and non-streaming clients.
• Python CLI authenticated requests using OCI IAM IDCS JWT Bearer tokens.
• Next.js reference UI authenticated requests using OCI IAM IDCS JWT Bearer tokens, including a protected /health test action.
• Hosted Applications protected with IDCS_AUTH_CONFIG.
• Hosted Application self-test covering token acquisition, JWT diagnostics, /health, non-streaming /responses, and streaming /responses.
• Agent Factory guided deployment, OCIR credential validation, and IDCS token validation.
• Next.js UI streaming against Hosted Application invoke.