Skip to content

on-the-ground/effect_ive_go

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation


“The Zen of the Effect-ive Gopher” – calm, centered, and side-effect free.

Worst Kind of Tests?

  • Tests tightly coupled with implementation    
    • Reproduce corner cases and subtle timing issues tied to the current implementation
    • Refactoring invalidates tests → Regression
    • Code becomes rigid, resistant to change
    • Maintaining tests becomes harder than maintaining code
  • How did we end up testing the implementation instead of the interface?    

Goroutines + Channels / Shared Memory

  • Complex entanglement of concurrency, synchronization, and ownership within the implementation      
    • Multiple goroutines with race conditions or timing issues around mutual exclusion and channels
    • Using mocks, spies, or exposing internal state -> debugging?
  • CSP is hard to test   
    • Preemptive goroutines
    • Implicit synchronization through shared channels
    • Non-deterministic select behavior
  • Is there a better alternative?  

Separation of Concerns

  • Divide & Conquer: M * N * L -> M + N + L
  • Pure functions: Tableizable     
    • Core data transformation logic
    • Easy to test, can be skipped if obvious
  • Side effects: Non-tableizable 
    • The devil lies in the effects: concurrency, synchronization, ownership, stream, state
    • Mixing pure logic with various side effects leads to the testing hell mentioned earlier
  • Is there a way to keep unavoidable side effects separated from pure logic?    
    • Effect Pattern

Cached Database

// http://github.com/on-the-ground/effect_ive_go/blob/main/examples/cached_database/main.go
...
	ctx, endOfDBHandler := state.WithEffectHandler[string, Person](
		false, // delegation == false
		state.NewCasStore(memDB),
		...
	)
	defer endOfDBHandler()
...
	ctx, endOfCacheHandler := state.WithEffectHandler[string, Person](
		true, // delegation == true
		state.NewSetStore(rist),
		...
	)
	defer endOfCacheHandler()
...
	ok, err := state.EffectInsertIfAbsent(ctx, key, person)
	log.Effect(ctx, log.LogInfo, "insert attempt", map[string]interface{}{
		"key":     key,
		"value":   person,
		"success": ok,
		"error":   err,
	})


Effect-ive Go != MagicBox

  • Effect-ive Go proposes the minimal idiomatic interface for delegating effects, staying true to Go
    • Uses context and teardown for idiomatic effect handler binding/unbinding
    • Effects declared with type and payload
    • Effect payloads are sent over channels to matching handlers found in context

Why Use Context to Find Handlers?

  • Explicit ≠ Clear: Does the function signature reveal the core logic?
// Typical function
func ValidateUser(ctx context.Context, db *sql.DB, logger *Logger, metrics *Metrics, config *AppConfig, tracer *Tracer, requestID string, featureFlags map[string]bool, ...) (bool, error)
// With injected handlers
func ValidateUser(ctx context.Context, stateHdl StateHandler, logHdl LogHandler, statHdl StatisticsHandler, configHdl ConfigHandler, obsrcHdl ObservationHandler,  ...) (bool, error)
// With scoped handlers
// Only core logic dependencies are exposed; auxiliary effects are delegated via context
func ValidateUser(ctx context.Context, whiteList, blackList []User, user User) bool
// Context abusing
// All core logic dependencies must be explicitly shown in the function signature
func ValidateUser(ctx context.Context, user User) bool

Effects on Effect-ive Go


  • Effect categories : Resumable / FireAndForget / Abortive(not suitable for Go)

  • Declaring an Effect:

    • Lookup handler by effect enum via context
      • Context is only for handler discovery
    • Pass payload via handler channel
    • Wait for result (Resumable), or send without waiting (FireAndForget)
  • Handler behavior:

    • Resumable: returns result via resume channel
    • Partitionable handlers process in parallel based on partitions

  • Lock free effects

    • Insert, Load, CompareAndSwap, CompareAndDelete effects
  • EventSourcingEffect

    • Subscribe to state command operations via prefix
  • TTL support

  • Multi-tier state using delegation


  • Stream operators stay alive until source is closed or context is canceled

    • EagerFilter, LazyFilter, Map, Merge, OrderBy, Pipe(bypass)
  • Arbiter provided to safely consume from source

    • Subscribe, Unsubscribe

  • Combines Stream and State handlers

  • External Semaphore

    • ResourceRegistration, Deregistration, Acquire, Release effects
  • TTL support

	// With numOwners == 1, this acts as a lock and expires after TTL
	ok, err := lease.ResourceRegistrationEffect(ctx, key, 1, ttl, pollInterval)

	ok, err = lease.AcquisitionEffect(ctx, key)

	/* Mutex zone */

	ok, err = lease.ReleaseEffect(ctx, key)

	ok, err = lease.ResourceDeregistrationEffect(ctx, key)

  • Provides a supervisor for managing goroutines within a scope

    • Parent and children never share context

    • Cancellation from parent is propagated by supervisor

    • All child goroutines are ensured to terminate when scope ends

  • AwaitAll allows waiting for multiple tasks simultaneously


  • Go does not distinguish between sync and async functions

    • CSP enables seamless concurrency

    • But without distinction, hangs may happen unpredictably

  • Task effect separates async invocation from result retrieval


  • State: for managing dynamic key-value data
  • Binding: for static, read-only key-value data
  • Used for config, environment lookup

Time?

  • Time requires precision → not delegable
  • Is nanosecond-level precision from the runtime meaningful?
  • Timespan:
    • Treat time not as a point but a range
    • Let the system ensure operations occur within the range
    • Allows for trust in the span(SoC)

Error?

  • A brief history of error handling:
    • Output as error: return -1
    • Clean output with exception: throw exp
    • Error as output: return output, error
  • In Go, errors are outputs
        • Attaching contextual messages to errors is part of domain logic

Once you've extracted all effects, is what remains truly pure?


Function as Table

  • Functions in practical languages can be impure at any time
    • It's essential to separately identify truly pure parts
  • A pure function is a lazy table
    • Same input → always the same output
    • Local reasoning, referential transparency, substitution?
    • => Tableizable, convertible to a table
fib = purefn.TableizeI1O1(func(n int) int {
	if n <= 1 {
		return n
	}
	return fib(n-1) + fib(n-2)
}, 32)

Tableize Implementation

func tableize[O any](
	pureFn func(...ComparableOrStringer) O,
	maxTableSize uint32,
) func(...ComparableOrStringer) O {
	memo := NewTrie[O](maxTableSize)
	return func(args ...ComparableOrStringer) O {
		keys := make([]ComparableOrString, len(args))
		for i, arg := range args {
			keys[i] = tableKey(arg)
		}
		v, ok := memo.Load(keys)
		if !ok {
			v = pureFn(args...)
			memo.Store(keys, v)
		}
		return v
	}
}

Benchmark

cpu: Intel(R) Core(TM) i7-14700

BenchmarkNaiveFib20-28                           55534       19869 ns/op       0 B/op      0 allocs/op

BenchmarkTableizedFib20-28                    24441051       71.02 ns/op      32 B/op      2 allocs/op

BenchmarkNaiveLevenshtein-28                    315432        3809 ns/op       0 B/op      0 allocs/op

BenchmarkTableizedLevenshtein/TrieSize_2-28    7247058       201.3 ns/op      96 B/op      4 allocs/op

BenchmarkTableizedLevenshtein/TrieSize_8-28    5805490       204.2 ns/op      96 B/op      4 allocs/op

BenchmarkTableizedLevenshtein/TrieSize_32-28   7311618       196.4 ns/op      96 B/op      4 allocs/op

BenchmarkNaiveDist-28                       1000000000      0.1012 ns/op       0 B/op      0 allocs/op

BenchmarkTableizedDist-28                      6820208       203.3 ns/op      96 B/op      4 allocs/op

PASS

coverage: 57.3% of statements

TableizeDebug

  • What if you tableized something assuming it was pure, but the output turns out to be unstable?
  • Use this for validation in CI, testbeds, or canary environments before production deployment
func tableizeDebug[O ComparableEquatable](
	pureFn func(...ComparableOrStringer) O,
	maxTableSize uint32,
) func(...ComparableOrStringer) O {
	memo := NewTrie[O](maxTableSize)
	return func(args ...ComparableOrStringer) O {
		...
		actual = pureFn(args...)
		loaded, ok := memo.Load(keys)
		if ok {
			if !Equals(actual, loaded) {
				panic("Do not tableize impure functions")
			}
		} else {
			memo.Store(keys, v)
		}
		return v
	}
}

Novelty of Effect-ive Programming

  • Focus on effects
    • Emphasize effects as a goal, not just a means
  • Be yourself, respect your runtime
    • Handle effects in a way that respects each language’s unique philosophy and idioms

PS: Haskell vs Effect-ive Programming

  • Haskell abstracts through mathematics
    • A function is a pure mapping with one input and one output: output = f(input)
    • Programs are primarily composed through pure function composition
  • Effects are essential in programs, but Haskell avoids executing them inside functions
    • Thus, effects are only allowed in specific impure zones like runX, main, etc.
  • Effects must be deferred lazily → monads
    • Encapsulate effects inside containers like Either[T]
    • How do you connect a pure input with an effectful output?
      • f1: func(T1) Either[T2], f2: func(T2) Either[T3]
    • Monad: an interface to connect containerized outputs with clean inputs
  • But problems arise when effects become deeply nested
    • Task[Either[T]] != Either[Task[T]]
    • StateReaderTaskEither[T]

PS: Haskell vs Effect-ive Programming

  • Let’s approach programming more like humans do
    • People struggle with reasoning about deeply deferred and nested operations
    • Resolve effects eagerly instead
    • But I don’t want to perform them myself, so let’s delegate them now to someone else
  • Effects are not core logic, so they can be patterned and handled mechanically
    • Assign handlers for each effect pattern and communicate with them
    • Where are those handlers? → Not DI, but IoC
  • We need a way to context-switch into a handler, perform the effect, and return
    • Low-level abstraction: Continuation Passing Style
    • High-level abstraction: Communicating Sequential Processes