Cosmos DB Persistence Library

Overview

The Cosmos DB Persistence Library provides a structured abstraction for interacting with Azure Cosmos DB. It includes command handlers, query handlers, pagination utilities, and container providers to streamline data operations.

Features

• CRUD Operations: Create, update, replace, and delete items efficiently.
• Optimized Queries: Supports point reads, SQL-based queries, and LINQ expressions.
• Pagination Support: Fetch large datasets in manageable pages.
• Encapsulated Client Access: Centralized management of Cosmos DB containers and client interactions.
• Efficient Data Streaming: Convert LINQ queries into FeedIterator for optimized batch processing.

Installation

NuGet Package Management

GitHub Packages is a private package registry integrated with GitHub, allowing developers to store and distribute NuGet packages securely.

In order to consume the lates NuGet Package version from GitHub Packages the following steps should be observed:

Steps to Install NuGet Packages from GitHub Packages

1. Authenticate with GitHub

• Generate a GitHub Personal Access Token (PAT) with the read:packages scope.

• Store the token securely, as you'll use it for authentication.

2. Configure NuGet Source

• Add GitHub Packages as a source in your NuGet.config file:

<configuration>
  <packageSources>
    <add key="GitHub" value="https://nuget.pkg.github.com/YOUR_GITHUB_USERNAME/index.json" />
  </packageSources>
  <packageSourceCredentials>
    <GitHub>
      <add key="Username" value="YOUR_GITHUB_USERNAME" />
      <add key="ClearTextPassword" value="YOUR_GITHUB_PERSONAL_ACCESS_TOKEN" />
    </GitHub>
  </packageSourceCredentials>
</configuration>

3. Install Package

Run the following command to install a package from GitHub:

dotnet add package Dfe.Data.Common.Infrastructure.Persistence.CosmosDb --version 1.X.X

Note: Available package versions can be found at the followoing location:

https://github.com/DFE-Digital/infrastructure-persistence-cosmosdb/pkgs/nuget/Dfe.Data.Common.Infrastructure.Persistence.CosmosDb

Dependency Registration (Composition Root) Register Cosmos DB dependencies in your application's startup configuration:

using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Configuration;

var serviceCollection = new ServiceCollection();
serviceCollection.AddCosmosDbDependencies();
var serviceProvider = serviceCollection.BuildServiceProvider();

Configuration Setup In order to use the handlers to access specific repositories it is essential to add the required configuration repository options to the application settings file, specifying essential connection details, authentication, and container configurations. The following steps cover the setup process:

1. RepositoryOptions Configuration

This section defines global settings for your Cosmos DB connection:

"RepositoryOptions": {
  "EndpointUri": "ADD_ENPOINT_URI_HERE",
  "PrimaryKey": "ADD_PRIMARY_KEY_HERE",
  "DatabaseId": "ADD_DATABASE_ID_HERE",

• EndpointUri: The Cosmos DB instance URL, used to connect from applications.

• PrimaryKey: Authentication key—should be kept secure using environment variables or a secret vault (like Azure Key Vault).

• DatabaseId: Defines which Cosmos DB database is being used (giapsearch).

2. Cosmos DB Containers Setup

Each Cosmos DB container is specified with its name and partition key:

"Containers": [
  {
    "container-name-example-1": {
      "ContainerName": "ADD_UNIQUE_CONTAINER_NAME_HERE",
      "PartitionKey": "ADD_PARTITION_KEY_HERE"
    }
  },
  {
    "container-name-example-2": {
      "ContainerName": "ADD_UNIQUE_CONTAINER_NAME_HERE",
      "PartitionKey": "ADD_PARTITION_KEY_HERE"
    }
  }
]

• ContainerName: Defines specific data storage areas in Cosmos DB.

• PartitionKey: Helps optimize query performance.

Example Usage

Command Handler (Create, Update, Delete)

using Microsoft.Azure.Cosmos;
using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Handlers.Command;

ICosmosDbCommandHandler commandHandler = new CosmosDbCommandHandler(containerProvider);

// Create an item
var item = new MyItem { Id = "123", Name = "Sample Item" };
await commandHandler.CreateItemAsync(item, "myContainer", "partitionKeyValue");
// Update or insert an item
await commandHandler.UpsertItemAsync(item, "myContainer", "partitionKeyValue");
// Replace an item
await commandHandler.ReplaceItemAsync(item, "123", "myContainer", "partitionKeyValue");
// Delete an item
await commandHandler.DeleteItemAsync<MyItem>("123", "myContainer", "partitionKeyValue");

Query Handler (Point Reads & SQL Queries)

using Microsoft.Azure.Cosmos;
using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Handlers.Query;

ICosmosDbQueryHandler queryHandler = new CosmosDbQueryHandler(containerProvider);

// Perform a point read (single-item lookup)
var item = await queryHandler.ReadItemByIdAsync<MyItem>(
    id: "123",
    containerKey: "myContainer",
    partitionKeyValue: "partitionKeyValue");

// Query using SQL
string sqlQuery = "SELECT * FROM c WHERE c.Category = 'Electronics'";
IEnumerable<MyItem> items = await queryHandler.ReadItemsAsync<MyItem>("myContainer", sqlQuery);

LINQ-Based Query

Expression<Func<MyItem, MyItem>> selector = item => new MyItem { Id = item.Id, Name = item.Name };
Expression<Func<MyItem, bool>> predicate = item => item.Category == "Electronics";

IEnumerable<MyItem> items = await queryHandler.ReadItemsAsync("myContainer", selector, predicate);

Paginated Query Handler

using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Handlers.Query;

IPaginatedCosmosDbQueryHandler paginatedQueryHandler = new PaginatedCosmosDbQueryHandler(containerProvider);

IEnumerable<MyItem> pagedItems = await paginatedQueryHandler.ReadPaginatedItemsAsync(
    "myContainer", selector, predicate, pageNumber: 1, pageSize: 20);

// Get total count of matching items
int itemCount = await paginatedQueryHandler.GetItemCountAsync<MyItem>("myContainer", predicate);

LINQ to FeedIterator Conversion

using Microsoft.Azure.Cosmos;
using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Handlers.Query;

IQueryableToFeedIterator converter = new QueryableToFeedIterator();

IQueryable<MyItem> query = cosmosContainer.GetItemLinqQueryable<MyItem>()
    .Where(item => item.Category == "Electronics");

FeedIterator<MyItem> iterator = converter.GetFeedIterator(query);

Container Provider (Retrieve Cosmos DB Containers)

using Microsoft.Azure.Cosmos;
using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Providers;

ICosmosDbContainerProvider containerProvider = new CosmosDbContainerProvider();
Container container = await containerProvider.GetContainerAsync("myContainer");

Client Provider (Execute Cosmos DB Operations)

using Microsoft.Azure.Cosmos;
using Dfe.Data.Common.Infrastructure.Persistence.CosmosDb.Providers;

ICosmosDbClientProvider clientProvider = new CosmosDbClientProvider();

var database = await clientProvider.InvokeCosmosClientAsync(async client =>
{
    return await client.CreateDatabaseIfNotExistsAsync("MyDatabase");
});

Dependencies

• Microsoft.Azure.Cosmos
• .NET 6 or later recommended