Problem Statement

• AI integrations are tightly coupled to specific providers
• Switching OpenAI → Ollama → Azure = rewriting code
• RAG apps need: embeddings + vector store + LLM + evaluation
• How do you build provider-agnostic AI apps in .NET?

.NET AI Building Blocks

The core abstractions for provider-agnostic AI

The Four Building Blocks

IChatClient — Universal Chat Abstraction

// Register with any provider — one line change to swap
builder.AddOllamaApiClient("ollama-llama3-1").AddChatClient();

// Consume via DI — completely provider-agnostic
IChatClient chatClient = sp.GetRequiredService<IChatClient>();
var response = await chatClient.GetResponseAsync(messages);

IEmbeddingGenerator — Vector Embeddings

// Register embedding provider
builder.AddOllamaApiClient("ollama-all-minilm").AddEmbeddingGenerator();

// Provider-agnostic embedding generation
IEmbeddingGenerator<string, Embedding<float>> embedder = ...;
var embeddings = await embedder.GenerateAsync(["some text"]);
// → Embedding<float>[384]

DelegatingChatClient — Middleware Pattern

sealed class LoggingChatClient(IChatClient inner, ILogger logger)
    : DelegatingChatClient(inner)
{
    public override async Task<ChatResponse> GetResponseAsync(
        IEnumerable<ChatMessage> messages,
        ChatOptions? options = null,
        CancellationToken ct = default)
    {
        logger.LogInformation("Chat request: {Count} messages", messages.Count());
        var response = await base.GetResponseAsync(messages, options, ct);
        logger.LogInformation("Chat response: {Tokens} tokens",
            response.Usage?.TotalTokenCount);

        return response;
    }
}

DelegatingChatClient — Composable Pipeline

// Built-in middleware: logging, telemetry, caching

builder.Services.AddChatClient(chatClient)
    .UseOpenTelemetry()
    .UseLogging()
    .UseFunctionInvocation();

VectorStore — Attribute-Driven Data Model

public sealed class DocumentRecord
{
    [VectorStoreKey]
    public Guid Id { get; set; } = Guid.NewGuid();

    [VectorStoreData]
    public string Text { get; set; } = "";

    [VectorStoreData(IsIndexed = true)]
    public string Source { get; set; } = "";

    [VectorStoreVector(Dimensions: 384)]
    public string Embedding => Text;  // auto-embedded!
}

VectorStore — Provider Registration

builder.Services.AddSingleton<VectorStoreCollection<Guid, DocumentRecord>>(sp =>
{
    var embedder = sp.GetRequiredService<IEmbeddingGenerator<string, Embedding<float>>>();
    var vectorStore = new QdrantVectorStore(
        sp.GetRequiredService<QdrantClient>(),
        new QdrantVectorStoreOptions { EmbeddingGenerator = embedder });

    return vectorStore
        .GetCollection<Guid, DocumentRecord>("documents");
});

Building a RAG Application

PDF → Chunks → Vectors → Grounded Answers

Solution Architecture

Phase 1: INGEST                       Phase 2: QUERY
───────────────────────────           ────────────────────────
Upload PDF document                   "What does the report say about X?"
  → PdfPig extracts text                → Vector search (Qdrant)
  → Recursive chunking (128 tokens)     → Retrieve top-5 chunks
  → Embed (all-minilm, 384-dim)         → LLM generates grounded answer
  → Upsert to Qdrant                    → Citations from stored knowledge

PDF Extraction → Chunking

// PdfPig — pure .NET, no native deps
var extraction = PdfTextExtractor.Extract(pdfStream);
// → Text with "## Page N" markers, page count

// Recursive chunking with overlap
var chunks = TextChunker.Chunk(extraction.Text);
// Target: 128 tokens | Max: 200 | Overlap: 25
// Separators: paragraphs → lines → sentences → words

Ingestion Pipeline

var records = chunks.Select((chunk, i) =>
    new DocumentRecord
    {
        Text = chunk,
        FileName = fileName,
        ChunkIndex = i,
        Source = source,
    });

await collection.UpsertAsync(records, ct);
// Embedding generated automatically via VectorStoreVector!

Document Ingestion

Vector Search Adapter

public static Func<string, CancellationToken,
    Task<IEnumerable<TextSearchResult>>>
    Create(VectorStoreCollection<Guid, DocumentRecord> collection,
           int top = 5) =>
    async (query, ct) =>
    {
        var results = new List<TextSearchResult>();
        await foreach (var result in
            collection.SearchAsync(query, top: top, ct))
        {
            results.Add(new TextSearchResult
            {
                Text = result.Record.Text,
                SourceName = FormatSourceName(...),
            });
        }
        return results;
    };

Agentic RAG — TextSearchProvider

var searchFunc = VectorSearchAdapter.Create(collection, top: 5);

var chatAgent = chatClient.AsAIAgent(
    new ChatClientAgentOptions
    {
        Name = "agentic_chat",
        ChatOptions = new ChatOptions { Instructions = SystemInstructions },
        AIContextProviders = [
            new TextSearchProvider(searchFunc, ragOptions)
        ],
    });

app.MapAGUI("/", chatAgent);

Chat — Q&A with Citations

.NET Aspire Orchestration

One command to start everything

AppHost — Full Stack Orchestration

var ollama = builder.AddOllama("ollama")
    .WithDataVolume()
    .WithLifetime(ContainerLifetime.Persistent);
var llama = ollama.AddModel("llama3.1");
var embedModel = ollama.AddModel("all-minilm");

var qdrant = builder.AddQdrant("qdrant")
    .WithLifetime(ContainerLifetime.Persistent);

var api = builder.AddProject<Projects.CompanyIntel_Api>("api")
    .WithReference(llama).WithReference(embedModel)
    .WithReference(qdrant).WithReference(sqlite);

builder.AddJavaScriptApp("ui", "../CompanyIntel.UI", "dev")
    .WithEnvironment("AGENT_URL", api.GetEndpoint("http"));

RAG Evaluation

Measuring quality with LLM-as-Judge

Why Evaluate RAG?

• RAG quality is invisible without measurement
• Retrieval failures → hallucinated or irrelevant answers
• Need to catch regressions before users do
• Four dimensions of quality:

Microsoft.Extensions.AI.Evaluation

IEvaluator[] evaluators = [
    new RelevanceEvaluator(),
    new CoherenceEvaluator(),
    new GroundednessEvaluator(),
    new RetrievalEvaluator(),
];

• Scoring: 1–5 scale via LLM-as-Judge
• Judge LLM reads (question, answer, context) and scores each dimension
• Returns NumericMetric with value, rating, and reasoning
• Part of Microsoft.Extensions.AI.Evaluation.Quality package

Relevance (1–5)

Does the response address the user's question?

• Judge reads (question, answer) — no context needed
• Pure question-answer alignment check
• Catches: wrong topic, incomplete answers, tangential responses

Coherence (1–5)

Is the answer well-organized and readable?

• Evaluates the answer in isolation — no context needed
• Catches: repetitive text, broken formatting, contradictions
• Important for user trust and readability

Groundedness (1–5)

Is every claim supported by the retrieved context?

Retrieval (1–5)

Are the retrieved chunks useful for answering?

• Evaluates the retrieval pipeline, not the LLM answer
• Requires RetrievalEvaluatorContext with chunk texts
• Low score → fix chunking, embedding model, or search params

Testing with Aspire

public class AspireAppFixture : IAsyncLifetime
{
    public HttpClient ApiClient { get; private set; }
    public Uri OllamaEndpoint { get; private set; }

    public async ValueTask InitializeAsync()
    {
        var appHost = await DistributedApplicationTestingBuilder
            .CreateAsync<Projects.CompanyIntel_AppHost>();
        _app = await appHost.BuildAsync();

        await _app.StartAsync();
        await _app.ResourceNotifications.WaitForResourceHealthyAsync("api");
        
        ApiClient = _app.CreateHttpClient("api", "http");
    }
}

Evaluation Test

// 1. Call RAG endpoint
var result = await client.PostAsJsonAsync("/api/chat",
    new { message = question });

// 2. Configure LLM judge
IChatClient judge = new OllamaApiClient(ollamaEndpoint, "llama3.1");
var chatConfig = new ChatConfiguration(judge);

// 3. Provide grounding context
var context = new EvaluationContext[] {
    new GroundednessEvaluatorContext(groundingContext),
    new RetrievalEvaluatorContext(retrievedChunks),
};

// 4. Score each dimension
foreach (var evaluator in evaluators) {
    var evalResult = await evaluator.EvaluateAsync(
        messages, modelResponse, chatConfig, context, ct);
    var metric = evalResult.Get<NumericMetric>(metricName);
    Assert.True(metric.Value >= 3);
}

End-to-End Evaluation Flow

dotnet test

  → Aspire starts Ollama + Qdrant + API (ephemeral)
  → Upload test PDF (contoso-corp.pdf)
  → For each eval scenario:
      → POST /api/chat with question
      → Get (answer, context chunks)
      → Judge LLM scores 4 metrics (1–5)
      → Assert all metrics ≥ 3

  → Aspire tears down everything

Key Takeaways

1. MEAI abstractions — IChatClient, IEmbeddingGenerator. Write once, swap providers

2. VectorData attributes — [VectorStoreKey/Data/Vector] + automatic embedding = minimal boilerplate

3. DelegatingChatClient — middleware for AI calls. Logging, retry, telemetry as pipeline behaviors

4. Aspire for testing — spin up full RAG stack per test run. Ephemeral containers = clean, reproducible eval

5. LLM-as-Judge — measure Relevance, Coherence, Groundedness, Retrieval before shipping

Resources

• .NET AI essentials — the core building blocks
• Vector Data in .NET — building blocks part 2
• GitHub: company-intel-dotnet
• Microsoft.Extensions.AI docs

Thank You

Questions?