Go Workloads for Testing #25414
Go Workloads for Testing #25414
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR adds a Go workload testing framework for Azure Cosmos DB DR drills. The workload creates a database and container, pre-populates items across logical partitions, and continuously performs concurrent read/write/query operations until terminated.
Key changes:
- Configuration management via environment variables for Cosmos DB connection and workload parameters
- Initial setup phase that creates database/container and upserts items across logical partitions
- Continuous workload execution with concurrent random operations (upsert, read, query)
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| sdk/data/azcosmos/workloads/workload_config.go | Loads workload configuration from environment variables |
| sdk/data/azcosmos/workloads/workload_utils.go | Provides utilities for client creation and random operations |
| sdk/data/azcosmos/workloads/initial_setup.go | Sets up database, container, and pre-populates test items |
| sdk/data/azcosmos/workloads/r_w_q_workload.go | Runs continuous workload operations |
| sdk/data/azcosmos/workloads/main/main.go | Entry point that orchestrates setup and workload execution |
| sdk/data/azcosmos/workloads/dev.md | Documentation for setting up and running scale tests |
| 1. Checkout the branch with the changes to test. | ||
| 1. Go to workloads folder | ||
| - `cd workloads` | ||
| 1. Fill out relevant configs in `workload_configs.go`: key, host, etc using env variables |
There was a problem hiding this comment.
Corrected filename from 'workload_configs.go' to 'workload_config.go'.
| 1. Fill out relevant configs in `workload_configs.go`: key, host, etc using env variables | |
| 1. Fill out relevant configs in `workload_config.go`: key, host, etc using env variables |
Copilot uses AI. Check for mistakes.
| - `cd workloads` | ||
| 1. Fill out relevant configs in `workload_configs.go`: key, host, etc using env variables | ||
| 1. Run the setup workload to create the database and containers and insert test data | ||
| - `python3 initial-setup.py` |
There was a problem hiding this comment.
The documentation references 'initial-setup.py' but this is a Go implementation. Update this to reference the correct command: go run ./main/main.go which runs both setup and workload, or document the setup-only execution method if available.
| - `python3 initial-setup.py` | |
| - `go run ./main/main.go` |
Copilot uses AI. Check for mistakes.
| var cfg workloadConfig | ||
| var err error | ||
|
|
||
| if cfg.Endpoint, err = get("COSMOS_URI"); err != nil { |
There was a problem hiding this comment.
how come we load everything in as env variables as opposed to the load in Python where we directly type in the values into string variables?
| // jf should be idempotent per index; it receives a per-worker RNG (not safe to share across workers). | ||
| func runConcurrent(ctx context.Context, count, workers int, jf func(ctx context.Context, index int, rng *rand.Rand) error) error { |
There was a problem hiding this comment.
nit: ranming jf to jobFunction or at least something more descriptive
|
|
||
| for { | ||
| if err := randomReadWriteQueries(ctx, container, count, cfg.PartitionKeyFieldName); err != nil { | ||
| return fmt.Errorf("read/write queries failed: %w", err) |
There was a problem hiding this comment.
does this return statement not break the loop execution?
2 participants
Description
This is adding a way to startup a workload to use for testing in dr drills. The configurations can be set through environment variables. Initially, the script will create a database and container, and an item for the number of logical partitions specified in the configuration files. After this initial setup, concurrent operations should happen on the precreated items until the process is killed. Idea is for this to be the bare minimum and later we can add more things based on customer usage in go like aad, other operations, etc