Web server refactor

[thegridman]

Whilst the current web server code works, it is basically a POC to prove we could create a functional web server and handle basic requests in Ecstasy. There is still substantial work to do with refactoring to make better use of what Ecstasy can provide.

One particular area I wanted to kick of a discussion on is how to make the server more scalable, based on using services. Looking at other web servers, particularly in Java as that's what I have most experience with, they seem to follow a pattern of having a single thread that pulls the requests off the wire and then puts these requests into a queue where a thread pool handles them. I'm guessing in Ecstasy we can do something similar with a service that takes the requests of the wire and then a pool of "handler" service that process these requests. I could come up with code to do this, but is there already anything in Ecstasy that does anything similar, i.e. the service pool bit? I don't want to end up duplicating something that might already exist.

[ggleyzer]

We don't have any stock service pool component, but we do have some similar usage patterns at TxManager. It serves as a scalable "job router", orchestrating concurrent tasks across multiple object stores and using the "clientCache" circular array as a Client pool.

[markandrewfalco]

Should it even be a pool? Given that services are considerably lighter than threads would creating services on-demand but unreasonable?

[thegridman]

I did think about just creating a service per request but I wasn't sure if that's too excessive, maybe it isn't. For example if you look at Java frameworks like Jersey they typically create a new instance of the class handling the endpoint for every request (unless you say it is a singleton). In a server handling thousands of requests a second that is going to be a lot of short lived services. If that is ok from an Ecstacy runtime point of view then It probably simplifies the server design.

[ggleyzer]

In the current runtime implementation the memory/execution cost of a service is basically no different than any other object, but there is a big "but" at the moment. Unless the application explicitly calls "shutdown()" on a service, those service handles are held by the corresponding container forever. I have not invested enough time to figure out how to auto-clean them. Maybe Mark has some enlightening ideas? :)

[markandrewfalco]

I'll definitely give some thought to service shutdown. It seems like it would be essentially the same as what I've done in my LightWeightExecutorService work at WD, in that Services should be as GC'able as any other object, i.e. not GC roots, but that they should also live long enough to drain their queues.

As for the web server since Services seem to generally represent the ownership of some data it would seem that the user session should itself be a service and that requests be processed in that session service? Non-session based requests could run in transient services.

[ggleyzer]

I like that idea (service per user).

[cpurdy]

Pooling isn't just about avoiding allocating and initializing the object. One of the reasons to pool objects is because of the caches of useful data that those objects accrue over time. If the request-servicing objects (these services) are lazily building cached state over time, then it makes a lot of sense to pool them. Also, if there is a fixed maximum, and if these are viewed as "manageable", then it may make sense to pool them, so that their stats can be aggregated over time (stats managed by each individually).

My gut feel is that these make a lot of sense to pool, but that and $.50 isn't enough to get a coffee these days.

And our goal is definitely to make service creation/destruction almost as cheap as a typical object. Obviously, it will never be quite as cheap, but it should be close.