Skip to content

Conversation

@Revolyssup
Copy link
Owner

Currently one cleanup go routine per key is created for deletion after expiration. This ensures that the keys are deleted immediately after the timer expires. But at the same time, this means a lot of goroutines. At scale this might be a problem theoretically if millions of keys accumulate.

So I want to move to a janitor based approach. The tradeoff here is that least recently used items will be evicted even if they are not expired when expired items exist. This reduces cache eviction accuracy.

Another solution on this PR can be to add a function makeSpace() which can be called instead of directly calling poptail. But this costs longer cleanup time.

So need a balance between Performance <-----> Accuracy.

func (lru *LRUCache[T]) makeSpace() {
    lru.mx.Lock()
    defer lru.mx.Unlock()
    
    if lru.removeExpiredItems(1) > 0 {
        return // Found and removed at least one expired item
    }
    
    // If no expired items, fall back to LRU eviction
    lru.poptail()
}

func (lru *LRUCache[T]) removeExpiredItems(max int) int {
    now := time.Now().UnixNano()
    removed := 0
    
    // Start from tail (least recently used) and remove expired items
    current := lru.Tail
    for current != nil && removed < max {
        next := current.Left // Save next before potential removal
        if current.expiration > 0 && now > current.expiration {
            lru.log.Debugf("Evicting expired key %s during space creation", current.key)
            lru.pop(current)
            delete(lru.m, current.key)
            removed++
        }
        current = next
    }
    return removed
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants