limit L1 cache item size

[lutzj7]

Hi Everybody

I'm searching for a way to limit the cache item size on L1 cache only. We have several instances/pods of our service and do not want to cache large item in memory because of the posible cost. Nevertheless the item should be stored on L2 cache anyway. The idea is to have a MaxL1CacheItemLimit as an configuration option.

One way is to have a specialized IMemoryCache implementation, but it feels better to have a native support from insight fusion cache...

Thx

[jodydonetti]

Hi @lutzj7

I'm searching for a way to limit the cache item size on L1 cache only. We have several instances/pods of our service and do not want to cache large item in memory because of the posible cost. Nevertheless the item should be stored on L2 cache anyway. The idea is to have a MaxL1CacheItemLimit as an configuration option.

One way is to have a specialized IMemoryCache implementation, but it feels better to have a native support from insight fusion cache...

No need for a specialized IMemoryCache, I think.

You can specify a lower Duration and a higher DistributedCacheDuration, so that you can keep items in memory for 10s but in the distributed cache for 30min (just as an example), and this should be more than enough.

Regarding instead the limit on the number of cache entries in L1, what you can do is create yourself a normal MemoryCache instance with a SizeLimit specified in the MemoryCacheOptions (see here), like this:

var memoryCache = new MemoryCache(new MemoryCacheOptions
{
  SizeLimit = 10
})

and then pass that to FusionCache, like this:

var fusionCache = new FusionCache(new FusionCacheOptions(), memoryCache);

If you do this you then also need to specify, for each cache entry, the size of that entry via the Size option: if you don't want to have a different size for each entry but are just interested in limiting the number of cache entries, you can just set the Size to 1 in the DefaultEntryOptions, so they will be used automatically for every call as a sort of "global default" (see here for the general mechanism, in cas you don't know it already).

Hope this helps, let me know.

[lutzj7]

Sorry for the late reply and of course many thanks for your answer.

It's a little bit of topic but maybe you can also help me with this question: How can I determine the size of an object, so that I can use Size and SizeLimit with the real size of the object?

My first thougt was to serialize the object, measure it (Length of the json string) do the size limiting stuff as needed and afterwards hand over it to the fusion cache. But the cache will than serialize it again... Can we disable that double serialization?

[jodydonetti]

It's a little bit of topic but maybe you can also help me with this question: How can I determine the size of an object, so that I can use Size and SizeLimit with the real size of the object?

Eh, it's basically not possible. See here for an issue from non other than Dan Moseley from 2017, still open.

There's the sizeof operator (see here), but it only works for basic types.

My first thougt was to serialize the object, measure it (Length of the json string) do the size limiting stuff as needed and afterwards hand over it to the fusion cache.

Yes, this is what most people end up doing.

A different approach may be to create a helper method that can do a best effort on a type-by-type basis, maybe summing up the length of each string properties/fields + the sizeof for basic types + maybe a baseline for the base object class they derive from etc.
All manual and potentially error prone, but kinda like when you override GetHashCode() and combine each field manually.

But the cache will than serialize it again... Can we disable that double serialization?

Correct, and no there's no way to serialize from the outside and pass the serialized string directly.

Hope this helps.

[lutzj7]

And a 2nd thing I just discovered. When one item initialy was not saved in L1 cache because of exceeding the SizeLimit it will not be saved back to L1 cache even if now is enough space again (other items are evicted..) Is there a way to solve that behavior?

[jodydonetti]

And a 2nd thing I just discovered. When one item initialy was not saved in L1 cache because of exceeding the SizeLimit it will not be saved back to L1 cache even if now is enough space again (other items are evicted..) Is there a way to solve that behavior?

This is strange, I'm not sure this works this way.
Can you provide an MRE?

Thanks.

[lutzj7]

Hi @jodydonetti

Incredible fast reply. Thanks

I will find some time maybe next week to build an MRE. But before that I will analyse it in deep to not waste your time if I've got something wrong yesterday.

Thank you also for all the other answers. I have made it configurable in my app on how the size should be calculated to leave it to the user if he like to spend the time of extra serialization over memory consumption.

Best