Non-embedded content for web?

[0pcom]

The contents which I'm attempting to use with cogentcore's content system are mostly products defined in a CSV file, except for just a few pages. I had hoped to avoid re-formatting the products from the CSV to individual files & embedding that.

I was actually able to use testing/fstest to create a filesystem at runtime, populate it with my desired content, and specify this filesystem to ct := content.NewContent(b).SetSource(myFS)

But, of course, this did not create the static HTML files needed for routing to a specific page.

So, I added logic on the server side of my application to create content files from the product CSV. That worked, however it is far from ideal as the time it takes to compile the cogentcore web application has increased substantially - core build web now takes at least 90 seconds, whereas before it was almost instant. This, despite the fact that the amount of data being embedded is relatively low. I believe this additional compile time is likely due to the number of files being embedded: 500 files representing each product, each category, and each subcategory.

$ time go run cogentcore.org/core@main build web
core build succeeded

real	1m20.200s
user	2m12.266s
sys	0m5.314s

$ time go run cogentcore.org/core@main build web
core build succeeded

real	1m15.947s
user	2m4.602s
sys	0m4.586s

An ideal solution to this issue might be if the files did not need to be embedded in the application in order for core build web to create the static html files - and perhaps additionally if core build web could be invoked with a flag or flags to direct it to create the html files without compiling the wasm binary.

If this were possible, I could go back to the method of creating the "testing/fstest" filesystem at runtime, and simply run something to the effect of core build web -content content -htmlonly or core build web -content content/* -htmlonly on the server side to generate the html files - without compiling the wasm.

Or the wasm could still be compiled in the same step, if a content directory on the disk could be specified.

If it's possible to allow users the option to create the html files explicitly without compiling the wasm or vice versa, that may be beneficial as well.

...

After further testing, I've learned a few things:

• it is possible to simply & explicitly compile the wasm as core build web would compile it.

• the wasm compilation including the embedded files is not the source of the excessive time that core build web takes to complete.

I can't tell what is actually making core build web take so long, but can only assume it's related to the creation of the other html resources. I will investigate this further as it does not make sense for it to take so long while producing so relatively little besides the wasm binary. The wasm binary is still the bulk of the work product.

...

the culprit does appear to be:

go run -tags offscreen,generatehtml .

Considering the fact that the topic of this discussion was based on the presumption that embedded files & the wasm compilation was the cause of the excessive time consumed by core build web - and that this is not actually the case; I think non-embedded content is not the correct way to address this.