Conflex

Conflex (pronounced /ˈkɒnflɛks/) is a powerful and versatile configuration management package for Go that simplifies handling application settings across different environments and formats.

Conflex is designed to help Go applications follow best practices for configuration management as recommended by the Twelve-Factor App methodology, especially Factor III: Config.

Features

Easy Integration: Simple and intuitive API.
Flexible Sources: Load from files, environment variables (with custom prefixes), Consul, and easily extend with custom sources.
Format Agnostic: Supports JSON, YAML, TOML, and other formats via extensible codecs.
Type Casting: Built-in caster codecs for automatic type conversion (bool, int, float, time, duration, etc.).
Hierarchical Merging: Configurations from multiple sources are merged, with later sources overriding earlier ones.
Struct Binding: Automatically map configuration data to Go structs.
Built-in Validation: Validate configuration using struct methods, JSON Schemas, or custom functions.
Dot Notation Access: Navigate nested configuration easily (e.g., config.GetString("database.host")).
Type-Safe Retrieval: Get values as specific types (string, int, bool, etc.), with error-returning options for robust handling.
Configuration Dumping: Save the effective configuration to files or other custom destinations.
Clear Error Handling: Provides comprehensive error information for easier debugging.
Thread-Safe: Safe for concurrent access and configuration loading in multi-goroutine applications.
Nil-Safe Operations: All getter methods handle nil Conflex instances gracefully, returning appropriate zero values or errors.
Consistent Return Types: Error versions of getter methods return empty types (empty slices, maps) instead of nil for missing keys.

Installation

go get go.companyinfo.dev/conflex

Quick Start

Here's a minimal example to get you started:

package main

import (
    "go.companyinfo.dev/conflex"
    "go.companyinfo.dev/conflex/codec"
    "context"
    "log"
)

func main() {
    cfg, err := conflex.New(
        conflex.WithFileSource("config.yaml", codec.TypeYAML),
        conflex.WithFileSource("config.json", codec.TypeJSON),
        // Optionally add remote sources, e.g. Consul
        conflex.WithConsulSource("staging/service", codec.TypeJSON),
    )
    if err != nil {
        log.Fatalf("failed to create configuration: %v", err)
    }

    if err := cfg.Load(context.Background()); err != nil {
        log.Fatalf("failed to load configuration: %v", err)
    }

    // Access configuration values
    port := cfg.GetInt("server.port")
    host := cfg.GetString("server.host")
    log.Printf("Server is running on %s:%d", host, port)
}

How it works

Sources are loaded in order; later sources override earlier ones.
Dot notation allows deep access: cfg.Get("database.host").
Type-safe accessors: GetString, GetInt, GetBool, etc.
Context validation: Both Load() and Dump() methods validate that context is not nil.
Error handling: All methods return descriptive errors for easier debugging.

Built-in Codecs

Conflex comes with several built-in codecs:

JSON: codec.TypeJSON - Standard JSON format
YAML: codec.TypeYAML - YAML format
TOML: codec.TypeTOML - TOML format
Environment Variables: codec.TypeEnvVar - For environment variable parsing

Caster Codecs

Conflex also provides caster codecs for automatic type conversion:

Boolean: codec.TypeCasterBool - Converts to bool
Integer: codec.TypeCasterInt, codec.TypeCasterInt8, codec.TypeCasterInt16, codec.TypeCasterInt32, codec.TypeCasterInt64
Unsigned Integer: codec.TypeCasterUint, codec.TypeCasterUint8, codec.TypeCasterUint16, codec.TypeCasterUint32, codec.TypeCasterUint64
Float: codec.TypeCasterFloat32, codec.TypeCasterFloat64
String: codec.TypeCasterString - Converts to string
Time: codec.TypeCasterTime - Converts to time.Time
Duration: codec.TypeCasterDuration - Converts to time.Duration

Note: Environment variable codec (codec.TypeEnvVar) only supports decoding. Attempting to encode will return an error indicating that encoding to environment variables is not supported.

Error Handling

Conflex provides comprehensive error handling with detailed context information through the ConfigError type.

ConfigError Structure

type ConfigError struct {
    Source    string // The source where the error occurred (e.g., "source[0]", "json-schema", "binding")
    Field     string // The specific field where the error occurred (optional)
    Operation string // The operation being performed (e.g., "load", "validate", "bind", "merge")
    Err       error  // The underlying error
}

Error Examples

// Source loading error
err := cfg.Load(context.Background())
// Error: "config error in source[0] during load: failed to read file: no such file or directory"

// Validation error
err := cfg.Load(context.Background())
// Error: "config error in json-schema during validate: invalid schema"

// Binding error
err := cfg.Load(context.Background())
// Error: "config error in binding during bind: failed to decode configuration"

Getter Method Error Handling

Getter methods come in two variants:

Non-error versions: Return zero values for missing keys or nil instances

cfg.GetString("nonexistent") // Returns empty string
cfg.GetInt("nonexistent")    // Returns 0
cfg.GetBool("nonexistent")   // Returns false

Error versions: Return errors for missing keys or nil instances

cfg.GetStringE("nonexistent") // Returns ("", error)
cfg.GetIntE("nonexistent")    // Returns (0, error)
cfg.GetBoolE("nonexistent")   // Returns (false, error)

When called on a nil Conflex instance, error versions return "conflex instance is nil" error.

Advanced Usage

Struct Binding

Bind configuration directly to your own struct:

type Config struct {
    Port int    `conflex:"port"`
    Host string `conflex:"host"`
}

var c Config
cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithBinding(&c),
)
cfg.Load(context.Background())
// c.Port and c.Host are now populated

Environment Variable Naming Conventions

Conflex provides powerful environment variable support that automatically maps environment variables to nested configuration structures. This follows the Twelve-Factor App methodology for configuration management.

Basic Usage

cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithOSEnvVarSource("MYAPP_"), // Only env vars with prefix MYAPP_
)

Naming Convention Rules

Conflex uses a hierarchical naming convention where underscores (_) in environment variable names create nested configuration structures:

Environment variables are converted to lowercase
Underscores (_) create nested levels
Empty parts (consecutive underscores) are filtered out
Values are automatically trimmed of whitespace

Examples

Environment Variable	Configuration Path	Value
`MYAPP_SERVER_PORT`	`server.port`	`8080`
`MYAPP_DATABASE_HOST`	`database.host`	`localhost`
`MYAPP_DATABASE_USER_NAME`	`database.user.name`	`admin`
`MYAPP_FOO__BAR`	`foo.bar`	`value`
`MYAPP_A_B_C_D`	`a.b.c.d`	`nested`

Struct Field Mapping

When using struct binding, environment variables map directly to struct fields using the conflex tag:

type Config struct {
    Port     int    `conflex:"port"`
    Host     string `conflex:"host"`
    Database struct {
        Host     string `conflex:"host"`
        Port     int    `conflex:"port"`
        Username string `conflex:"username"`
        Password string `conflex:"password"`
    } `conflex:"database"`
}

Environment variables needed:

export MYAPP_PORT=8080
export MYAPP_HOST=localhost
export MYAPP_DATABASE_HOST=db.example.com
export MYAPP_DATABASE_PORT=5432
export MYAPP_DATABASE_USERNAME=admin
export MYAPP_DATABASE_PASSWORD=secret123

Advanced Examples

Complex Nested Configuration:

type AppConfig struct {
    Server struct {
        Host string `conflex:"host"`
        Port int    `conflex:"port"`
        TLS  struct {
            Enabled  bool   `conflex:"enabled"`
            CertFile string `conflex:"cert_file"`
            KeyFile  string `conflex:"key_file"`
        } `conflex:"tls"`
    } `conflex:"server"`
    Database struct {
        Primary struct {
            Host     string `conflex:"host"`
            Port     int    `conflex:"port"`
            Database string `conflex:"database"`
        } `conflex:"primary"`
        Replica struct {
            Host     string `conflex:"host"`
            Port     int    `conflex:"port"`
            Database string `conflex:"database"`
        } `conflex:"replica"`
    } `conflex:"database"`
}

Required environment variables:

export MYAPP_SERVER_HOST=0.0.0.0
export MYAPP_SERVER_PORT=8080
export MYAPP_SERVER_TLS_ENABLED=true
export MYAPP_SERVER_TLS_CERT_FILE=/etc/ssl/certs/server.crt
export MYAPP_SERVER_TLS_KEY_FILE=/etc/ssl/private/server.key
export MYAPP_DATABASE_PRIMARY_HOST=primary.db.example.com
export MYAPP_DATABASE_PRIMARY_PORT=5432
export MYAPP_DATABASE_PRIMARY_DATABASE=myapp
export MYAPP_DATABASE_REPLICA_HOST=replica.db.example.com
export MYAPP_DATABASE_REPLICA_PORT=5432
export MYAPP_DATABASE_REPLICA_DATABASE=myapp

Edge Cases and Special Handling

Consecutive Underscores:

MYAPP_FOO__BAR → foo.bar (empty parts filtered out)
MYAPP_A___B → a.b (multiple empty parts filtered)

Type Conflicts: If an environment variable creates a conflict between scalar and nested values, the nested structure takes precedence:

export MYAPP_FOO=scalar_value
export MYAPP_FOO_BAR=nested_value
# Result: foo.bar = "nested_value" (scalar "foo" is overwritten)

Whitespace Handling:

Keys and values are automatically trimmed of whitespace
MYAPP_KEY = value → key = "value"

Best Practices

Use Descriptive Prefixes: Always use application-specific prefixes to avoid conflicts

# Good
export MYAPP_DATABASE_HOST=localhost
export WEBAPP_DATABASE_HOST=localhost

# Avoid
export DATABASE_HOST=localhost  # Too generic

Consistent Naming: Use consistent naming patterns across your application

# Consistent pattern
export MYAPP_SERVER_HOST=localhost
export MYAPP_SERVER_PORT=8080
export MYAPP_SERVER_TIMEOUT=30s

Documentation: Document your environment variables in your application's README

# Required environment variables:
# MYAPP_SERVER_HOST - Server hostname (default: localhost)
# MYAPP_SERVER_PORT - Server port (default: 8080)
# MYAPP_DATABASE_HOST - Database hostname
# MYAPP_DATABASE_PORT - Database port (default: 5432)

Validation: Use struct validation to ensure required environment variables are set

func (c *Config) Validate() error {
    if c.Server.Host == "" {
        return errors.New("MYAPP_SERVER_HOST is required")
    }
    if c.Server.Port <= 0 {
        return errors.New("MYAPP_SERVER_PORT must be positive")
    }
    return nil
}

Merging and Precedence

Multiple sources are merged; later sources override earlier ones.
Environment variables can override file/remote config:

cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithOSEnvVarSource("MYAPP_"), // Only env vars with prefix MYAPP_
)

Content Sources

Load configuration from byte slices (useful for testing or dynamic configuration):

configData := []byte(`{"server": {"port": 8080, "host": "localhost"}}`)
cfg, _ := conflex.New(
    conflex.WithContentSource(configData, codec.TypeJSON),
)

Remote Sources (Consul)

cfg, _ := conflex.New(
    conflex.WithConsulSource("production/service", codec.TypeJSON),
)

Note: By default, the Consul source will use the Consul API client and automatically look up the CONSUL_HTTP_ADDR and CONSUL_HTTP_TOKEN environment variables for configuration. You can override these by setting the appropriate environment variables or configuring the Consul client manually.

Dumping Configuration

cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithFileDumper("out.yaml", codec.TypeYAML),
)
cfg.Load(context.Background())
cfg.Dump(context.Background()) // Writes merged config to out.yaml

Configurable File Permissions

You can customize file permissions when dumping configuration:

import "go.companyinfo.dev/conflex/dumper"

// Use default permissions (0644)
fileDumper := dumper.NewFile("config.yaml", encoder)

// Use custom permissions
fileDumper := dumper.NewFileWithPermissions("config.yaml", encoder, 0600)

The default file permissions are defined by the DefaultFilePermissions constant (0644).

Custom Codecs

Conflex allows you to extend configuration support to any format by registering your own codecs.

Implementing a Custom Codec

A codec must implement the following interface:

type Codec interface {
    Encode(v any) ([]byte, error)
    Decode(data []byte, v any) error
}

Example: Registering a Custom Codec

Suppose you want to support a custom format called mytype:

package mycodec

import (
    "go.companyinfo.dev/conflex/codec"
)

type MyCodec struct{}

func (MyCodec) Encode(v any) ([]byte, error) {
    // ... your encoding logic ...
}

func (MyCodec) Decode(data []byte, v any) error {
    // ... your decoding logic ...
}

func init() {
    codec.RegisterEncoder("mytype", MyCodec{})
    codec.RegisterDecoder("mytype", MyCodec{})
}

Then, in your application:

import (
    _ "yourmodule/mycodec" // ensure init() runs
    "go.companyinfo.dev/conflex"
    "go.companyinfo.dev/conflex/codec"
)

cfg, _ := conflex.New(
    conflex.WithFileSource("config.mytype", "mytype"),
)

Note: If you need type conversion functionality, consider using the built-in caster codecs (e.g., codec.TypeCasterInt, codec.TypeCasterBool) instead of creating a custom codec for simple type casting.

When to Use a Custom Codec

Supporting formats not built-in (e.g., XML, encrypted configs)
Integrating with legacy or proprietary configuration formats
Adding validation or transformation logic during encode/decode

Tip: If you build a useful codec, consider contributing it back to the community!

Validation

Conflex supports configuration validation to help catch errors early and ensure your application runs with correct settings.

1. Struct-Based Validation

If your binding struct implements the following interface, Conflex will call Validate() after binding:

type Validator interface {
    Validate() error
}

Example:

type MyConfig struct {
    Port int `conflex:"port"`
}

func (c *MyConfig) Validate() error {
    if c.Port <= 0 {
        return errors.New("port must be positive")
    }
    return nil
}

cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithBinding(&myConfig),
)
err := cfg.Load(context.Background()) // Will return error if validation fails

2. JSON Schema Validation (for Maps)

What is JSON Schema?
JSON Schema is a standard for describing the structure and validation rules of JSON data. It allows you to define required fields, data types, value constraints, and more, making it easy to validate configuration files and catch errors early.
Learn more at json-schema.org.

You can validate the loaded configuration map against a JSON Schema using github.com/santhosh-tekuri/jsonschema/v6:

Note: JSON Schema validation in Conflex is applied to the merged configuration map (map[string]any), not directly to Go structs. Schema validation happens before any struct binding. If you want to validate your struct, use the struct-based Validate() error method described above.

schemaBytes, err := os.ReadFile("schema.json")
if err != nil {
    log.Fatalf("failed to read schema: %v", err)
}
cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithJSONSchema(schemaBytes),
)

3. Custom Validation Functions

You can register a custom validation function for either the bound struct or the config map:

cfg, _ := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithValidator(func(cfg map[string]any) error {
        if cfg["port"].(int) <= 0 {
            return errors.New("port must be positive")
        }
        return nil
    }),
)

Summary Table

Validation Type	For Structs	For Maps	How to Use
Interface-based	`Validate() error`	—	Implement on struct
JSON Schema	—	Yes	`WithJSONSchema(schema)`
Custom Function	Yes	Yes	`WithValidator(func) error`

Tip: Validation helps prevent misconfiguration and makes your application more robust!

Real-World Example

This example demonstrates merging multiple sources (file, environment, Consul), using struct binding, validation, and a custom codec.

import (
    "context"
    "go.companyinfo.dev/conflex"
    "go.companyinfo.dev/conflex/codec"
    _ "yourmodule/mycodec" // Register your custom codec
)

type Config struct {
    Port int    `conflex:"port"`
    Host string `conflex:"host"`
}

func (c *Config) Validate() error {
    if c.Port <= 0 {
        return errors.New("port must be positive")
    }
    return nil
}

var c Config
cfg, err := conflex.New(
    conflex.WithFileSource("config.yaml", codec.TypeYAML),
    conflex.WithFileSource("config.json", codec.TypeJSON),
    conflex.WithOSEnvVarSource("MYAPP_"),
    conflex.WithConsulSource("production/service", codec.TypeJSON),
    conflex.WithFileSource("config.mytype", "mytype"), // custom codec
    conflex.WithBinding(&c),
    conflex.WithValidator(func(m map[string]any) error {
        if m["feature_enabled"] != true {
            return errors.New("feature_enabled must be true")
        }
        return nil
    }),
)
if err != nil {
    log.Fatalf("failed to create configuration: %v", err)
}
if err := cfg.Load(context.Background()); err != nil {
    log.Fatalf("failed to load configuration: %v", err)
}

Sample Configuration Files

YAML (config.yaml):

server:
  port: 8080
  host: localhost
feature_enabled: true

JSON (config.json):

{
  "server": {
    "port": 8080,
    "host": "localhost"
  },
  "feature_enabled": true
}

TOML (config.toml):

[server]
port = 8080
host = "localhost"

feature_enabled = true

Testing & Best Practices

Use the testify suite for unit and integration tests (see *_test.go files).
Mock sources and dumpers for isolated tests.
Always check errors from Load and Dump.
For concurrency, Conflex is thread-safe for Load and Get.

Troubleshooting & FAQ

Q: Why is my struct not being populated?

Make sure you pass a pointer to your struct to WithBinding.
Check your struct tags: use conflex:"fieldname" for the field name within its context.

Q: How do I override config with environment variables?

Use WithOSEnvVarSource("PREFIX_") and set env vars like PREFIX_SERVER_PORT=8080.
Environment variables follow the naming convention: PREFIX_SECTION_SUBSECTION_KEY=value
For nested structures, use underscores: PREFIX_DATABASE_USER_NAME=admin maps to database.user.name

Q: How do environment variables map to struct fields?

Environment variables are converted to lowercase and split by underscores
Use the conflex tag to map to the field name within its struct context: conflex:"port"
Example: MYAPP_SERVER_PORT=8080 with tag conflex:"port" in a struct tagged with conflex:"server" populates the struct field

Q: What happens with consecutive underscores in environment variable names?

Consecutive underscores are filtered out: MYAPP_FOO__BAR becomes foo.bar
This allows for cleaner environment variable names while maintaining the same configuration structure

Q: How do I add a new config source or dumper?

Implement the Source or Dumper interface and pass it to WithSource or WithDumper.

Q: How do I access nested values?

Use dot notation: cfg.Get("outer.inner.key").

Q: What happens if a source returns nil or an error?

If a source returns an error, Load will return it. If a source returns nil, it is skipped.

Q: What happens when I call getter methods on a nil Conflex instance?

Non-error versions return appropriate zero values (empty string, 0, false, etc.)
Error versions return "conflex instance is nil" error
This prevents panic and provides graceful degradation

Q: What do getter methods return for missing keys?

Non-error versions return zero values for the type (empty string, 0, false, empty slices/maps)
Error versions return the zero value plus an error describing the missing key
Error versions for slice/map types return empty slices/maps instead of nil for consistency

Roadmap and Future Plans

Contributing

Contributions are welcome! Here's how you can contribute:

Fork the repository
Create a new branch (git checkout -b feature/improvement)
Make your changes
Commit your changes (git commit -am 'Add new feature')
Push to the branch (git push origin feature/improvement)
Create a Pull Request

Please make sure to:

Follow the existing code style
Add tests if applicable
Update documentation as needed
Include a clear description of your changes in the PR

License

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Name		Name	Last commit message	Last commit date
Latest commit History 73 Commits
.github		.github
codec		codec
dumper		dumper
examples		examples
source		source
.gitignore		.gitignore
.revive.toml		.revive.toml
LICENSE		LICENSE
README.md		README.md
case_insensitive_test.go		case_insensitive_test.go
conflex.go		conflex.go
conflex_test.go		conflex_test.go
dumper.go		dumper.go
error.go		error.go
error_test.go		error_test.go
flake.lock		flake.lock
flake.nix		flake.nix
go.mod		go.mod
go.sum		go.sum
source.go		source.go

License

companyinfo/conflex

Folders and files

Latest commit

History

Repository files navigation

Conflex

Table of Contents

Features

Installation

Quick Start

How it works

Built-in Codecs

Caster Codecs

Error Handling

ConfigError Structure

Error Examples

Getter Method Error Handling

Advanced Usage

Struct Binding

Environment Variable Naming Conventions

Basic Usage

Naming Convention Rules

Examples

Struct Field Mapping

Advanced Examples

Edge Cases and Special Handling

Best Practices

Merging and Precedence

Content Sources

Remote Sources (Consul)

Dumping Configuration

Configurable File Permissions

Custom Codecs

Implementing a Custom Codec

Example: Registering a Custom Codec

When to Use a Custom Codec

Validation

1. Struct-Based Validation

2. JSON Schema Validation (for Maps)

3. Custom Validation Functions

Summary Table

Real-World Example

Sample Configuration Files

Testing & Best Practices

Troubleshooting & FAQ

Roadmap and Future Plans

Contributing

License

About

Topics

Resources

License

Uh oh!

Stars

Watchers

Forks

Uh oh!

Contributors 4

Uh oh!

Languages