From Static Script to Thinking Agent: How I Built a LangGraph-Orchestrated RAG System

 

Turning a secure API into a stateful, tool-using AI agent with PostgreSQL + pgvector

---

In my last post, I laid out an ambitious roadmap — six architectural patterns that would transform the AI Triage Agent from a secure API endpoint into a production-grade agentic system: LangGraph orchestration, Planner-Orchestrator-Executor, structured memory, router agents, reflection loops, and multi-agent topology.

I'm not going to pretend I delivered all six in one week. That would be dishonest engineering storytelling, and the best engineers I know have a healthy respect for how long it actually takes to build things properly.

Here's what did happen: I built the foundational layer that makes all six patterns possible. The LangGraph state machine, the PostgreSQL + pgvector RAG engine, and the tool dispatch layer. And I learned exactly what it takes to close the gap to each of the remaining patterns — which I'll share transparently in this post.

---

The Architecture: A Stateful Agent with Real RAG

The current system has four layers, each with a distinct responsibility:

User Message
    │
    ▼
┌──────────────────────────────────┐
│  Zero Trust Security Pipeline    │  ← Week 4
│  (sanitizer → guard → output)    │
└──────────────┬───────────────────┘
               │
               ▼
┌──────────────────────────────────┐
│  LangGraph Agent                 │  ← Week 5
│  classifier → tool_runner →      │
│  responder / escalate            │
└──────────────┬───────────────────┘
               │
               ▼
┌──────────────────────────────────┐
│  PostgreSQL + pgvector RAG       │  ← Week 5
│  exact SQL  +  semantic search   │
└──────────────────────────────────┘

The security pipeline from Week 4 wraps the entire stack. Every message is sanitized, screened for prompt injection, and every response is filtered for PII before reaching the user. The agent operates inside that security boundary — not outside it.

Let me walk through the two new layers in detail.

---

Part 1: LangGraph — From Linear to Stateful

The Problem with Linear Code

In Week 3, the triage flow looked like this in main.py:

# Linear, stateless, no branching
result = classify(message)
response = INTENT_RESPONSES.get(result.intent, "...")

One call, one response. No branching, no tool execution, no conditional logic. This works for a demo. It doesn't work for production.

Enter LangGraph

LangGraph turns agents from single LLM calls into stateful, cyclic graphs — where each node is a specialized function and edges define the flow of control and data. This is the orchestration layer I promised in Week 4, and it's the single most important structural change to the codebase.

I built a 3-node StateGraph:

                    ┌──────────────┐
                    │  classifier  │  Classify intent via LLM
                    └──────┬───────┘
                           │
                           ▼
                    ┌──────────────┐
                    │  tool_runner │  Execute tool based on intent
                    └──────┬───────┘
                           │
                    ┌──────┴──────┐
                    │             │
              resolved?      needs escalation?
                    │             │
              ┌─────┘             └─────┐
              ▼                         ▼
       ┌──────────┐             ┌────────────┐
       │responder │             │ escalation │
       └────┬─────┘             └─────┬──────┘
            │                         │
            └──────────┬──────────────┘
                       ▼
                      END

Each node has a narrow, well-defined responsibility:

1. classifier_node — calls the existing LLM classifier, returns intent + confidence + escalation flag
2. tool_runner_node — dispatches to the appropriate tool based on the classified intent
3. responder_node / escalation_node — terminal nodes that format the final response

The routing is handled by a conditional edge:

def route_after_tool(state: AgentState) -> Literal["responder", "escalation"]:
    if state.get("resolved") and not state.get("needs_escalation"):
        return "responder"
    elif state.get("needs_escalation"):
        return "escalation"
    else:
        return "responder"

The state is a typed dictionary that flows through every node:

class AgentState(TypedDict):
    message: str              # original user message
    intent: str               # classified intent
    confidence: float         # classification confidence
    needs_escalation: bool    # from classifier
    tool_output: str          # output from tool execution
    resolved: bool            # whether tool resolved the issue
    response_text: str        # final response

Key insight: This AgentState is a working memory tier. It's not yet a full long-term memory system, but it gives the agent conversational awareness within a single triage interaction — significantly more than the stateless request-response model we had before.

What Changed

Component	Before	After
main.py	Direct classify() call	triage_agent.invoke()
New file	—	app/agent_graph.py (129 lines)
New file	—	app/tools.py (75 lines)
Classifier	Embedded in main.py	Extracted to app/classifier.py
Routing	Hardcoded if/else	LangGraph conditional edges
Test files	2 test files	3 test files (7 new agent tests)

The agent graph was the structural change. But the tools were still using hardcoded data — a Python dictionary. That's where the RAG layer came in.

---

Part 2: PostgreSQL + pgvector — From Mock to Real RAG

The Problem with Hardcoded Data

The first version of tools.py used a dictionary:

faqs = {
    "password_reset": "To reset your password: 1. Go to...",
    "billing": "Your current billing info...",
    "technical_support": "Common troubleshooting steps...",
}

This works for a demo. Not for production.

The Database Stack

PostgreSQL + pgvector gives you 80% of the capability with zero extra infrastructure. No second database to manage, no data sync, no new query language. Just add a VECTOR(384) column and use the <=> operator for cosine distance.

The schema is simple — two tables:

CREATE TABLE faq (
    id SERIAL PRIMARY KEY,
    intent VARCHAR(50) NOT NULL,
    question TEXT NOT NULL,
    content TEXT NOT NULL,
    embedding VECTOR(384),         -- all-MiniLM-L6-v2
    created_at TIMESTAMP DEFAULT NOW()
);

CREATE TABLE tickets (
    id SERIAL PRIMARY KEY,
    intent VARCHAR(50) NOT NULL,
    customer_message TEXT NOT NULL,
    resolution TEXT NOT NULL,
    embedding VECTOR(384),
    resolved_at TIMESTAMP DEFAULT NOW()
);

CREATE INDEX idx_faq_embedding
    ON faq USING ivfflat (embedding vector_cosine_ops);
CREATE INDEX idx_tickets_embedding
    ON tickets USING ivfflat (embedding vector_cosine_ops);

I used all-MiniLM-L6-v2 from sentence-transformers for embeddings — 384 dimensions, small enough to run locally, accurate enough for semantic search.

The Dual Retrieval Pattern

def faq_lookup(intent: str, user_message: str) -> ToolResult:
    # 1. Exact FAQ match by intent
    faq_rows = query_faq(intent)
    # 2. Semantic search across past tickets
    user_embedding = embed(user_message)
    similar = query_similar_tickets(intent, user_embedding, limit=3)
    combined = format_faq(faq_rows) + format_cases(similar)
    return ToolResult(success=True, data=combined, resolved=True)

Exact SQL query — fast, deterministic, always returns canonical documentation. Vector similarity search — embeds the user's actual message and finds semantically similar resolved tickets. The agent doesn't just look up password_reset — it finds the specific past case that matches the user's situation.

The "Account Locked" Test: pgvector in Action

When I tested with the query "I cannot log in for 3 days now!", the exact FAQ returned generic password reset instructions. But the vector search correctly identified that "3 days of being locked out" is semantically closer to "account locked after 10 failed attempts".

Similar Past Cases:
Case 1 (similarity: 87%)
User said: "I've been locked out for 5 days,
tried resetting 10 times, nothing works"
Resolution: Account was locked after 10 failed
attempts. IT manually unlocked the account and
reset the password.

Retrieval Quality

87% semantic similarity — vector search correctly matched "locked out" experience over generic password reset.

That's the difference between search and understanding.

---

Honoring the Roadmap: What I Promised vs. What I Built

In Week 4, I outlined six architectural patterns for agentic AI. Let me be transparent about where each one stands, what I learned, and when it will ship.

Pattern	Status	What Happened
LangGraph: The Orchestration Layer	DELIVERED	The 3-node StateGraph is live — classifier → tool_runner → responder/escalation with conditional routing. This is the structural foundation everything else builds on.
Memory in Agentic AI	PARTIAL	AgentState provides working memory within a single triage interaction (short-term). Long-term memory (user preferences across sessions, fact persistence) is not yet implemented. The pgvector store is a natural long-term memory backend — it just needs the retrieval layer to be wired for conversation history. Planned for Week 7.
The Router Agent Pattern	PARTIAL	The conditional route_after_tool() function is a basic router — it decides between respond and escalate based on state. A true router agent would delegate to specialist sub-agents (billing, tech support, etc.). That requires multi-agent topology. Planned for Week 8.
Planner-Orchestrator-Executor	DEFERRED	This pattern requires an agent that can decompose tasks, execute them in sequence, and replan on failure. The current agent is a single-pass pipeline. Multi-step planning needs an LLM-as-planner node in the graph and a feedback loop for error recovery. Planned for Week 8-9.
Reflection Pattern	DEFERRED	The Guard Classifier from Week 4 is a primitive reflection mechanism (pre-response safety check). A full reflection loop — where the agent critiques its own response and revises it — requires adding a critique node after the responder. Planned for Week 7 alongside observability, since you need metrics to know if reflection is improving quality.
Multi-Agent Patterns	DEFERRED	The supervisor-multi-agent topology is the endgame. Each specialist sub-agent needs its own graph, restricted tool access, and memory boundary. This is the most complex pattern — it requires everything below it to be stable first. Planned for Week 9+.

Why not all six at once? Building agentic AI systems requires depth, not breadth. A reflection loop without observability is blind. A multi-agent system without a reliable orchestration layer is fragile. Each pattern I deferred has a prerequisite I hadn't built yet — and rushing them would produce brittle, untrustworthy agents. That's not what this project is about.

The Revised Roadmap

Here's the updated plan with realistic weeks:

Week	Focus	Deliverable
5 DONE	LangGraph + pgvector RAG	Stateful agent, dual retrieval, tool dispatch
6	Observability	LangFuse traces on every DB query, retrieval quality metrics (hit rate, MRR, latency)
7	Memory + Reflection	Long-term memory via pgvector, constitutional reflection loop, self-critique
8	Planner + Router Agent	Task decomposition, multi-step execution, specialist routing
9+	Multi-Agent Supervisor	Sub-agent graphs, least-agency tool scoping, supervisor topology

Every week delivers a working, tested, blogged-about increment. No skipped steps.

---

Why This Matters for Saudi Arabia's AI Future

For the Saudi market, this week's work matters for three reasons:

1. Data Sovereignty

PostgreSQL runs in your own infrastructure. No data leaving your network for a third-party vector database API. For government services and regulated industries — a significant portion of Saudi AI adoption — this is a compliance requirement under PDPL.

2. Localized RAG

The architecture supports swapping in any sentence-transformers model. For the GCC market, the next evolution is Arabic sentence embeddings — enabling the agent to understand and respond in Arabic with the same semantic precision. "ما قدرت أسجل دخولي من ٣ أيام" should get the same "Account Locked" resolution as the English query. Few teams in the region are building this.

3. Methodical Engineering as a National Competency

Vision 2030 calls for AI to transform government services. But transformation doesn't come from demos that claim to deliver six architectural patterns at once. It comes from teams that build one solid layer at a time, test it, write about it, and only then move to the next one. That's what this project demonstrates — and it's a methodology, not just a codebase.

---

What's Next: Closing the Gap

The honest answer to "what's next" is different from what I said last post, because now I have data instead of speculation.

Week 6 — Observability. Before I add more patterns, I need to measure what I have. LangFuse traces on every query_faq() and query_similar_tickets() call. Hit rate, MRR, latency. You can't improve what you don't measure.

Week 7 — Memory + Reflection. With observability in place, I can add a reflection loop that critiques responses and a long-term memory layer using the pgvector store we already built. The data from Week 6 will tell me if these changes actually improve quality.

Weeks 8-9 — Planner, Router, Multi-Agent. Once the foundation is instrumented and reliable, I'll add task decomposition, specialist sub-agents, and finally the supervisor topology.

The best AI system isn't the one with the most patterns. It's the one built on a foundation you can trust — and that means building one layer at a time, measuring each one before moving to the next.

---

The journey from BA to AI Engineer Architect is about building systems that are both safe and capable. Week 4 gave us safety. Week 5 gave us the orchestration and retrieval foundation. Weeks 6-9 will close the gap to the full vision — one measured increment at a time.

Built in Riyadh, Saudi Arabia as part of the AI Triage Agent project. Week 5 of a 12-week engineering journey from BA to AI Engineer Architect.