Introduce a function for caching static content

[falkoschindler]

Motivation

Inspired by PR #4796, this PR uses a simpler approach by caching individual strings, lists and dictionaries rather than whole elements. At the moment it is just a proof of concept showing the general idea. It can be tested as follows:

from nicegui import cache, ui

ui.label(cache('A label with a long text'))
ui.html(cache('An large <b>HTML</b> element'))
ui.table(rows=cache([{'name': 'Alice', 'age': 25}, {'name': 'Bob', 'age': 30}]))
ui.plotly(cache({'data': [{'type': 'scatter', 'x': [1, 2, 3], 'y': [1, 2, 3]}]}))

ui.run()

A temporary console.log shows cache misses on the JavaScript console. This should only happen when visiting the app for the first time or when altering content which is wrapped in a cache() call.

Implementation

The user wraps static content with the new cache() function, which converts str to CachedStr, list to CachedList and dict to CachedDict. This way Python can continue interacting with these objects like before, but props can later be augmented with cache information before sending them to the client. This is done in form of a string "CACHE_<hash_id><json>". In case the server knows that a specific hash ID is known to the client, it omits the "<json>" part, saving payload.

The client "unhashes" all props that start with "CACHE_". Unknown hashes are automatically added to a dictionary "__nicegui_data_store__" in localStorage and to a cookie "__nicegui_hash_keys__". When reloading the page, the server can use the information from this cookie to initialize its set of known hashes.

Overall this implementation is rather simple. Let's discuss if this feels nice or if it is missing anything important.

Progress

• I chose a meaningful title that completes the sentence: "If applied, this PR will..."
• The shared auto-index client probably needs more work.
• Make the "CACHE_" prefix more robust - or find a fundamentally better way to indicate cached data.
• Check which elements need special treatment for cached input.
• What happens when the __nicegui_data_store__ dictionary exceeds the maximum local storage size?
• Pytests have been added (or are not necessary).
• Documentation has been added (or is not necessary).

[falkoschindler]

@evnchn Before you continue with PR #4796, have a look at this PR #4900. It borrows some of your ideas, but keeps it simpler by caching individual values. This results in a simpler API and significantly less code. I'd love to hear what you think.

[evnchn]

Overall, this is a much more cleaner and concise implementation of the original client.fetch_xxx_from_browser_data_store methods which is proposed in #4796. Note that the PR has diverged from that implementation, and moved on to .cached for an element.

However, it shares the same issue: Should the element have a pipeline to convert the input, it is likely no longer a Cached instance (e.g. no longer CachedStr), and the caching will fail. Luckily it does not fail in a dramatic way, rather it falls-back gracefully since a CachedStr is still a str and so it gets processed faithfully by whatever pipeline the element may have.

Case in point:

ui.markdown(cache('''Cached **Markdown**'''))

...
"props":{"innerHTML":"<p>Cached <strong>Markdown</strong></p>\n"
...

---

Overall, I am assessing which way to go:

#4796 defines one and only one caching strategy per element based on the common-sense (which input is likely going to be the biggest, and one which if you want a different one you'll create a separate element).

• It likely works already, and you just need .cache to apply it
• To customize the caching strategy, you need to get dirty and write to:
  • .dynamic_keys: exclude element's outer-level keys from caching to avoid same-element conflicts, such as same element with different class then you do .dynamic_keys.add('class')
  • and .static_prop_keys: add element's props-level keys from caching, if the defaults aren't enough / the element is not cache-aware yet, such as tree.static_prop_keys.add('nodes') before I bake that into ui.tree
  • Requires clear knowledge of the NiceGUI JSON element definition, which is tough!
  • Logic-wise, dynamic_keys is opt-out whule static_prop_keys is opt-in, this is also very weird
• For adapting the element, we need to apply sensible defaults of static props keys using .static_prop_keys.add

#4900 allows the flexible definition of cache strategy, which is on the responsibility to call cache

• It will not work unless the user apply the cache manually
• To customize the caching strategy, it is very easy, just apply / not apply cache
• For adapting the element, we need to make all cachable inputs cache-aware by letting it respect the type and return Cached* if the input is also a Cached* instance

I think #4900 is worth it over #4796 since it is easier to use, despite not having one easy-to-call .cache function. That function can never be perfect since everyone's caching use case will be different, so despite being easy to call, it is not easy to use.

Check if you agree. If so, we can proceed with this PR more seriously. Thanks!

P.S.: It is clean since you leverage OOP, something which I should learn more.

[evnchn]

@falkoschindler 3 lines of change for making ui.markdown cache-capable

I think I like this over #4796 since this is more Pythonic. Even Pydantic got its SecretStr, so I see no problem with a CachedStr.

[evnchn]

• Should not .union into the known_hashes for auto-index client, since that client serves many browsers

[evnchn]

Caching the class goes something like this:

It is also very simple too. Nice!

[falkoschindler]

Check if you agree. If so, we can proceed with this PR more seriously.

I think so, yes. I'd rather start with an easy solution and see if we can come up with a .cache method later to cache whole elements. But I expect this feature to be used very selectively, for a large tree or a large Markdown text. Then it is fine to call cache on individual parameters.

3 lines of change for making ui.markdown cache-capable

This raises an important point: Elements don't automatically support cached input if, e.g., strings are transformed before being put into the props dictionary. I'll need to check how many elements are affected.

Should not .union into the known_hashes for auto-index client, since that client serves many browsers

Yes, this might be a solution. Alternatively the auto-index client could store hashes for individual connections, but this might require too much memory for long-running apps.

Caching the class goes something like this

I'm not sure if this is a relevant use case since class strings are rather short. I don't think replacing them with "CACHE_<32-digit-hash-id>" is worth the effort.

[rodja]

The original motivation for caching arose after introducing the tree based documentation hierarchy in #4732. By using ui.sub_pages to implement the documentation as a single page application, the tree is only loaded once anyway. Maybe this caching feature is not so pressing anymore?

[evnchn]

I don't think the two implementations are mutually exclusive. More, they can complement each other in select use cases. However, you can say it does make implementing this PR less urgent.

[rodja]

I don't think the two implementations are mutually exclusive.

True. Even if you have a SPA, you may want to speed up the page load when revisiting the SPA.