Files & folders

[stevekrouse]

What is a val? In the beginning (july 2022), a val was called an "eval" and it stood for "an evaluation", or an occurrence of some code that has been evaluated (evaled), including it's logs and outputs. This concept was fun but it was confusing and over time the concept of a val got ever closer to a regular old JavaScript module (or "ECMASCript Module", ESM). It does beg the question - should we go all the way and remove the concept of a "val" and simply let vals be regular old "files"? In particular, if we want companies to build more and more of their core infra on us, I couple imagine it being difficult to convince them with our current primitives. There are tradeoffs in either direction:

Arguments in favor of files & folders

• File over app philosophy: ownership, portability, credible exit
• Interop with:
  • you existing beloved editor
  • local development (including offline mode)
  • git & github (deploy on push & pr preview deploys)
  • existing projects (ie vals could live in a subfolder like nextjs's api/ folder)
• Builds on users' existing intuitions
• Doesn't spend an innovation token - problems managing files are mostly solved
• All the problems of "organization" of files (folders) are solved and it's easy to imagine making full sites (even with different file types, ie images)

Arguments in favor of vals

• Hard to map all the metadata of vals into a file system (ie permissions, names, references)
• Lots of web-native creative tools don't cleanly map to files: notion, google docs, figma, twitter. You can export to files, but the mapping isn't one-to-one
• A lot of newer, younger computer users don't know about files and folders. They're more familiar with application-rich data structures.

This discussion was inspired by a chat with @wilg this weekend. Curious what @tmcw @andreterron @boucher @daniellevine would add to the analysis

[postpostscript]

For me, a large part of what is very cool about Val Town is the registry where you can convert back and forth between a val's name, HTTP endpoint, and import URL, and that there is just a single entrypoint to a subdomain. I have a lot of questions, but ultimately if any of the problems this brings up have to be solved in UI, I think that would make it harder to reason about web facing projects or to see at a glance how another person's is set up

That said, being able to officially work on "vals" in 3rd party editors would be a great feature! A folder system would also make it much easier to fork and tweak large projects, especially with relative imports. I think this is worth exploring, but from my hobbyist perspective, I'm just concerned that the site would lose something in the added complexity (or maturity!)

[pomdtr]

where you can convert back and forth between a val's name, HTTP endpoint, and import URL

This is a great point, and it might be the reason why val.town is the first online coding environment that clicked with me.

I also feel like static analysis might be an area to explore for val organization. When I use the VS Code extension, I always have a clear view on the val context thanks to the dependencies and references views:

It's a lot more powerful and intuitive than folders in my opinion, since a val can appear in multiple dependency tree at the same time.

[daniellevine]

Delighted to dig in more, will start with quick thoughts here.

tldr; I prefer vals. I love files, but slightly too constraining and have some PTSD from Dropbox feeling limited by the file metaphor.

I think intrinsically vals are both how you define them above, but also a living "service" (when used as APIs, but also more broadly). It's important to maintain flexibility to spend "innovation tokens" on interesting concepts that arise from the combination even if at the cost of running afoul of the traditional filetype.

An interesting example is that I don't think a val should be "in" multiple projects (up for debate and tradtional fs have flavors of this (symlinks)). I also think a public val includes all it's published versions as opposed to the latest, but that versions can be specified (git has flavors of this, but a traditional OS filesystem does not). I think the concept of "publishing" is novel for files as is collaborating/sharing/ACLs. All of these are brutal problems (and many that Git/GitHub have taken a crack at), but I wouldn't want to block off areas of exploration around those because we tell people we're like the files they already know. Lots of parallels to issues around package management, vendoring, monorepos, etc.

We would lose some things files give us, but I think it's important that we're honest with ourselves about those issues and accept them sooner rather than later. We will need to be very deliberate and thoughtful in accounting for those tradeoffs.

[Ed. I should note I love files and think we should be considerate about exportability, interoperation, and standards. I just think a Val is a combination of file, runtime, longitudinal data, collaboration data, etc. And we can dump an asset/artifact with all of that, but OS filesystems will break many of those things if we try too hard to interoperate.]

[stevekrouse]

Really well said! Thank you!

[boucher]

I think building a way to go from val.town -> files is good, even if it's a bit lossy. I think you should probably avoid making some kind of permanent decision on this front for a while. I'm philosophically aligned with this Files philosophy, but also sympathetic to Dan's arguments.

[wilg]

It might make sense to solidify the organizational model and best practices around vals before getting to into the weeds of how to interop with a traditional filesystem.

It seems already to be the case where working with multiple vals together is common. For example, Steve's dateme.directory. It links to the source code, but you can't easily find all the pieces besides with search. Presumably Steve has them in a "folder" in Val Town already, but I don't think we can see that. Each val is versioned independently, which is cool in some ways, but complicated in others. How do multiple people collaborate on a change? What if Steve makes an incompatible update to just one val? Or wants to make an interdependent update to multiple vals? Also, it would be pretty difficult to try to fork the entire website. I guess you'd have to fork each val independently and update the references. But which ones should be treated as libraries Steve will maintain versus parts of my own application? And since all the vals are homogenous, it raises some issues for dealing with non-code data. Is it better to write a static page in TSX wrapped in a val versus writing an HTML file? What if I have a logo and a bunch of favicons, or want to link to a PDF, etc.

If the goal is to make Val Town a great place for web applications, there will need to be a clear pattern for dealing with this stuff, independent of whether the backing store is files or a database or whatever.

I'm not super immersed in Val Town best practices or anything (yet!) but this is the stuff that I'm thinking about when evaluating whether to use it for something beyond a single endpoint.

[stevekrouse]

I stumbled across Retool Source Control. Looks like they had a similar set of problems and found a way to serialize their richer data structures into files that could be then source controlled on git, etc. Interesting touchpoint. Seems quite clunky.

[daniellevine]

I know way too much about files from Dropbox although some of this is probably outdated... there are lots of ways to do this. I'd say the two broadest schools of thought in use today are best represented by Microsoft and Apple.

Microsoft went with zip archives of collections of xml files (the docx, xlsx, etc format).

Apple has pushed bundles, which are essentially special folders. A lot of popular 3rd-party Mac apps have used them, my favorite example for a long time was 1Password and their .agilekeychain. You could double click it and I believe it would open 1Password, but you can also right-click it and inspect the contents of the "package".

Tradeoffs to both.

• Zips are a more readily usable format for various things (attaching files to an email), but can be easily polluted, hard to inspect, and have slow initial load times. They tend to behave as largely opague blobs (although some of that is probably at least semi-intentional from MSFT)
• Bundles can more easily have faster initial loads to the end user and are perhaps less opaque in certain aspects. But the implications for how they interoperate with other applications is brutal and I don't think it's really worth it. Apple broke many of their own apps around certain changes to bundles. Apple has also used bundles to push Apple specs/standards and create Apple-specific products that don't really make a ton of sense https://developer.apple.com/documentation/foundation/nsfilecoordinator/1411533-coordinate including by messing with their own prior standards https://developer.apple.com/documentation/coreservices/file_system_events I assume they've fixed the issues from a decade ago, but I have a lot of PTSD

[samuelstroschein]

Here are my thoughts:

• interop/files make val.town more attractive
• you could spare a lot of engineering effort around version control by storing *.val "files" in git
• your web app can abstract files and git for the user

https://www.loom.com/share/49f05ff77c6a4d73a34bc7e66d742bcd?sid=c380d64d-822a-42e3-9b7f-2254b1999e9c