RAG Query Pipeline

This diagram illustrates how BlueRobin processes user questions using the RAG (Retrieval Augmented Generation) pattern, combining semantic search with LLM generation.

Sequence Diagram

sequenceDiagram
    autonumber
    participant User
    participant Blazor as Blazor Web
(UI)
    participant API as Archives API
(FastEndpoints)
    participant RAG as RagService
(Application)
    participant QP as QueryPreprocessor
    participant Ollama as Ollama
(Embeddings)
    participant Qdrant as Qdrant
(Vector DB)
    participant MinIO as MinIO
(Storage)
    participant LLM as Ollama/OpenAI
(Generation)

    User->>Blazor: Ask question about documents
    Blazor->>API: POST /api/rag/ask
{ question, options }
    
    rect rgb(238, 233, 245)
        Note over API,RAG: Query Preprocessing Phase
        API->>RAG: AskAsync(userId, question, options)
        RAG->>QP: PreprocessAsync(question)
        QP->>LLM: Rewrite query for semantic search
        LLM-->>QP: Enhanced query
        QP-->>RAG: RewrittenQuery
    end

    rect rgb(253, 248, 234)
        Note over RAG,Qdrant: Retrieval Phase
        RAG->>Ollama: Generate query embedding
(nomic-embed-text)
        Ollama-->>RAG: Query vector [1024d]
        RAG->>Qdrant: Vector similarity search
(top_k=20, threshold=0.7)
        Qdrant-->>RAG: Matching chunks with scores
    end

    rect rgb(237, 245, 246)
        Note over RAG,MinIO: Context Building Phase
        RAG->>MinIO: Fetch full document content
processed/{docId}/content.md
        MinIO-->>RAG: Document text
        RAG->>RAG: Build context with
document headers
    end

    rect rgb(248, 237, 237)
        Note over RAG,LLM: Relevance Filtering Phase
        RAG->>LLM: FilterRelevantChunksAsync()
"Which chunks answer this question?"
        LLM-->>RAG: Filtered relevant chunks
    end

    rect rgb(238, 233, 245)
        Note over RAG,LLM: Generation Phase
        RAG->>LLM: Generate answer with context
(structured output format)
        
        alt Streaming Response
            loop Token by token
                LLM-->>RAG: Partial response
                RAG-->>API: Stream chunk
                API-->>Blazor: SSE event
            end
        else Non-Streaming
            LLM-->>RAG: Complete response
        end
    end

    RAG-->>API: RagResponse { answer, citations,
confidence, searchTimeMs }
    API-->>Blazor: JSON response
    Blazor-->>User: Display answer with citations

Component Responsibilities

Key Design Decisions

1. Query Preprocessing: Rewrites user questions for better semantic matching
2. Multi-phase Retrieval: Embed → Search → Filter → Generate
3. Chunk-level Relevance Filtering: LLM validates which chunks actually answer the question
4. Streaming Support: Real-time response display for better UX
5. Citation Tracking: Links answers back to source documents