FetchableDevEnvironment API

[sheremet-va]

There is a growing need to standardise the API around environment communication. Even the default SSR environment currently doesn't expose any methods to run the code.

We have decided to implement RunnableDevEnvironment for the default SSR environment that exposes the runner and import directly to decrease the boilerplate (no need to run createModuleRunner manually):

const ssrEnvironment = server.environments.ssr
const entry = await ssrEnvironment.import('/entry-point.js')
await entry.handle(request)

const module = ssrEnvironment.runner.evaluatedModules.getModuleByUrl('/entry-point.js')

module.exports === entry

Taking this idea further, we think it would be good for the ecosystem to have a standard way to run the code via the Fetch API. This is already how most third-party tools operate, and we have seen more and more requests to support this out of the box. This is where the new FetchableDevEnvironment comes in.

FetchableDevEnvironment is a DevEnvironment that has a dispatchFetch method:

class FetchableDevEnvironment extends DevEnvironment {
  dispatchFetch(request: Request): Promise<Response>
}

FetchableDevEnvironment class is not exposed (only its type). The class is considered an implementation detail and instead, we will expose a createFetchableDevEnvironment function to create the environment like we already do with createServer. You can call it inside the environments.dev.createEnvironment config option:

import { createServer, createFetchableDevEnvironment } from 'vite'

const server = await createServer({
  environments: {
    edge: {
      createEnvironment(name, config) {
        return createFetchableDevEnvironment(name, config, {
          requestHandler(req: Request): Promise<Response> {
            // ... process request and return a response
          },
        })
      },
    },
  },
})

const response: Response = await server.environments.edge.dispatchFetch(
  new Request(convertToUrl('/entry-point.js')),
)

dispatchFetch is not called anywhere by Vite itself. The expectation is that frameworks can use this to run the code in the adapter's environment:

import { isFetchableDevEnvironment, isRunnableDevEnvironment } from 'vite'

function handleRequest(environment: DevEnvironment, request: Request) {
  if (isFetchableDevEnvironment(environment)) {
    return await environment.dispatchFetch(request)
  }
  else if (isRunnableDevEnvironment(environment)) {
    const entry = await environment.import('/entry-point.js')
    return await entry.default.fetch(request)
  }
  // ignore browser - fail on others
  else if (environment.consumer !== 'client') {
    throw new Error(`the environment "${environment.name}" doesn't support "dispatchEvent" or "import" API`)
  }
}

We are also thinking about providing utility functions to convert the http.IncommingMessage to Request and http.ServerResponse to Response. We might also need a function to combine Response + http.ServerResponse.

import { toFetchRequest, combineResponses } from 'vite' // name suggesions are welcome

server.middlewares.use(async (req, resp) => {
  const request = toFetchRequest(req)
  const response = await handleRequest(server.environments[condition], request)
  combineResponses(resp, response)
})

Fetchable Module Runner API

To make things easier on the module runner side, we also want to introduce a createFetchableModuleRunner function to create a runner that uses global fetch to fetch the module information. If global fetch is not available, the createFetchableModuleRunner will throw an error.

import { createFetchableModuleRunner } from 'vite/module-runner'

const runner = createFetchableModuleRunner({
  serverUrl: 'http://localhost:3000',
  environmentName: 'edge',
  hmr: createHmrConnection(), // custom function, no changes here
})

export default {
  async fetch(request) {
    const entry = await runner.import('/entry-point.js')
    return entry.handle(request)
  },
}

To achieve this, Vite implements a middleware that intercepts all requests to /@vite/import/{env}/{url} (the naming is based on already established /@vite/client and /@vite/env) and responds with the result of environment.fetchModule back to the runner:

// very simple example of an internal implementation, error handling omitted

server.middlewares.use(async (req, resp) => {
  if (req.url.startsWith('/@vite/import/') {
    const { environmentName, moduleUrl, importer } = parseUrl(req.url)
    const moduleInfo = await server.environments[environmentName].fetchModule(moduleUrl, importer)
    resp.status(200).set({ 'Content-Type': 'application/json' }).end(JSON.stringify(moduleInfo))
  }
})

Default SSR environment as FetchableDevEnvironment

To reduce boilerplate even further, we could also expose the dispatchFetch method on the default SSR environment. At the minimum, this would require:

• entryPoint option in the SSR section of the config
• standardised interface for the entry point (export default { fetch })

The internal implementation might look something like this:

const environment = new FetchableEnvironment(name, config, {
  async handleRequest(request) {
    const entry = await environment.import(environment.config.entryPoint)
    const response = await entry.default.fetch(request)
    validateResponse(response)
    return response
  },
})

await server.environments.ssr.dispatchFetch(
  new Request(convertToUrl('/entry-point.js'))
)

Currently, we don't have plans to do this, but any feedback regarding this is welcome.

Previous discussions:

• Environment API
• Runtime API PR
• Runtime API Feedback discussion
• Initial Docs PR
• Initial Implementation PR

[Rich-Harris]

This looks great! Especially the createFetchableModuleRunner stuff.

Is the hmr option required? If we can rely on fetch being available, then couldn't that be used to notify the client of updates (e.g. via a stream) and the server of errors (e.g. via a POST request)? I'm sure it's more complicated than that, just trying to understand.

The purist in me wants to avoid making environments.ssr 'special' by making it a RunnableDevEnvironment. It feels like being stricter about the separation between environment and runner would make the API easier to learn in the long run (otherwise the relationship is somewhat less clear), and avoid the footgun described in the docs:

One of the problems that was exposed with vite.ssrLoadModule is over-reliance on the server state inside the processed modules

I appreciate that such a restriction would make migrations from Vite 5 more awkward though, and possibly prevent other valid use cases.

[patak-dev]

The purist in me wants to avoid making environments.ssr 'special' by making it a RunnableDevEnvironment

I thought the same before, but once we have the concept of RunnableDevEnvironment, we can avoid the ssr environment being special by making every environment be a RunnableDevEnvironment by default. Before, if you wanted a new environment named server, you were forced to configure it with a dev: { createEnvironment } and create a ModuleRunner. Now just adding environments: { server: {} } would give us a ready-to-use environment. The simplicity of ssrLoadModule() helped quite a bit. I'm leaning towards having the default be as easy to use to end up helping more than it hurts.

[patak-dev]

@sheremet-va maybe we don't expose environment.import() and force users to be more explicit with environment.runner.import()?

[sheremet-va]

@sheremet-va maybe we don't expose environment.import() and force users to be more explicit with environment.runner.import()?

We can discuss this in a runnable PR 😄

But I would also prefer having a single function.

[sheremet-va]

If we can rely on fetch being available, then couldn't that be used to notify the client of updates (e.g. via a stream) and the server of errors (e.g. via a POST request)? I'm sure it's more complicated than that, just trying to understand.

I did not consider that at all, to be honest. We can always experiment with custom HMR implementations. I am just not sure if every runtime supports streams and such 🤔 Either way, I would love to provide a default HMR connection that just works.

[jacob-ebey]

How do I dispatch a http://localhost:5173/some/sub/route?a=a&b=b to /entry-point.js? convertToUrl('/entry-point.js', "/some/sub/route?a=a&b=b")?

[jacob-ebey]

I'll get bindings working if they aren't already. Should be simple in the setup I already have.

[jamesopstad]

Just sharing an update on this regarding what we've done in @cloudflare/vite-plugin. We've so far found the existing Vite middleware functionality to be sufficient for plugins to compose behaviour i.e. the plugin that creates an environment is also responsible for creating the middleware that forwards requests to the environment. Not ruling out incorporating the dispatchFetch API in the future and we'll do so if the need arises.

[ascorbic]

@jamesopstad I think it would be helpful if the Cloudflare environment did implement dispatchFetch, and the plugin allowed the request handling middleware to be disabled. This would allow frameworks to handle requests in a way more similar to the Vite docs, i.e. check if the ssr environment is runnable or fetchable and dispatch accordingly. Right now we would have to special-case Cloudflare so we don't dispatch the request again in our middleware. The nice thing about a standardised API like this is that we can just say that adapters can set a fetchable dev environment and we'll use it if it's available.

[jamesopstad]

@ascorbic I think one of the reasons this might be tricky is that we have multiple middleware that dispatch requests before and after assets, depending on the user's configuration. So while you might benefit from not special casing Cloudflare in one place you would then probably need to special case it elsewhere to replicate this platform specific behaviour. Middleware feels like a natural way to compose framework and platform behaviour.

I'd be curious to see how others are using the FetchableDevEnvironment though. @sheremet-va, are there examples out there that you're aware of?

[sheremet-va]

@sheremet-va, are there examples out there that you're aware of?

I've seen Nitro use the concept of fetchable environment: https://github.com/nitrojs/nitro/blob/acdda3a469024afde574eec932cc48695deff735/src/build/vite/dev.ts#L33

[benmccann]

fetch is sort of a confusing name to me. I guess the idea is that you can call it with the fetch API? But it's weird to name an API with the same name as the API that calls it. Doubly so when the request signatures differ. And this API is not actually fetching anything. Not to mention that you can call web APIs with wget/curl or any number of APIs besides fetch. Perhaps something like handleRequest might be a clearer name for the method than fetch

[sheremet-va]

The naming in the proposal is indeed based on the Fetch API because we rely on Request and Response from that API, so it makes sense to me personally.

[j4w8n]

I don't yet completely understand all of this, but I'm liking it. I'm currently building a plugin, which is a directory-based API router that should support multiple runtimes; and, since this is a fresh project, I'm using the v6 beta.

The proposed helper functions, to convert requests and responses, would be great. As of now, I've stolen SvelteKit's versions of these, via their getRequest and setResponse utilities (ref).

As for using a FetchableDevEnvironment for ssr by default: Seems like a decent idea, but I didn't understand why the example implementation shows using a config option to acquire the entry point file, but then the public dispatchFetch call still has to declare it. I'm probably just missing some context - I'm new:

const environment = new FetchableEnvironment(name, config, {
  async handleRequest(request) {
    const entry = await environment.import(environment.config.entryPoint)
    const response = await entry.default.fetch(request)
    validateResponse(response)
    return response
  },
})

await server.environments.ssr.dispatchFetch(
  new Request(convertToUrl('/entry-point.js'))
)

[sheremet-va]

convertToUrl is just an example. You would normally get the request from a server middleware.

[magne4000]

To add my 2 cents to the discussion, I think that this proposal would benefit from more standard middlewares.
Having Vite middlewares complying with Connect syntax, while also having other parts of Vite using the Web Fetch API feels a bit messy.

What I would like to be able to do:

server.middlewares.use(async (request: Request) => {
  // We can directly reuse `request` in `dispatchFetch`
  return new Response(...);
});

I created universal-middleware for this kind of purpose, where we could have support for both new and old syntax.
This lib allows one to write middlewares using this universal syntax, while also providing wrappers to directly target popular servers (like @universal-middleware/express, that would convert the middleware above to an express one).

The new syntax would internally be wrapped using @universal-middleware/express to avoid breaking changes. But it could transition to fully drop Connect in the future IMO.

[sheremet-va]

The first implementation of the API should be available as an experimental feature in 6.3. Any feedback is welcome. The fetchable runner is not part of the implementation yet, but the environment can be used by anyone.

[cyco130]

I know I'm late to the game but I agree with @magne4000's comment. Mixing node:http and fetch APIs can be a problem.

I'd expect most implementations of FetchableDevEnvironment to run an HTTP server and proxy requests to it. This puts an unnecessary burden on the framework authors because now we have to handle node:http API <=> fetch API conversions at the framework level. Having written such an adapter for Hattip, I know it's not trivial, especially in terms of performance.

Instead of FetchableDevEnvironment (or in addition to it), wouldn't an HttpServerDevEnvironment that just exposes a domain address (say http://localhost:3173) be much easier to implement for both environment authors and framework authors? Now the environment can just start a server in the target environment and return the address it's running on. And the framework author can just use a standard HTTP proxy middleware like http-proxy-3 which Vite already uses for it's proxy feature. It would also allow proxying websockets which is not possible with the fetch API.

Maybe one day we can even define a standardized two-way HTTP API to handle Vite <=> environment communications for environments that don't support websockets.

I implemented a toy (but mostly functional) Fastly Compute environment with the above idea. It is tiny which shows how awesome the environment API is already!

On the Vite side, apart from setting the environment config properly (noExternal, fastly:* modules etc.), we implement a /@vite/transport endpoint (similar to @sheremet-va's @vite/import idea) as a very simple one-way transport mechanism:

function configureServer(server) {
  const environment = server.environments.ssr;

  const ENDPOINT = `/@vite/transport`;

  server.middlewares.use((req, res, next) => {
    async function handleInvoke() {
      // Quick and dirty body parser
      const chunks: Buffer[] = [];
      for await (const chunk of req) {
        chunks.push(chunk);
      }

      const json = JSON.parse(Buffer.concat(chunks as Buffer[]).toString());

      if (
        json.type === "custom" &&
        json.event === "vite:invoke" &&
        json.data.name === "fetchModule"
      ) {
        const result = await (environment.fetchModule as any)(
          ...json.data.data
        );
        res.end(JSON.stringify({ result }));
        return;
      }

      console.error("Unknown invocation");
      console.error(json);

      res.statusCode = 500;
      res.end();
    }

    try {
      if (req.url === ENDPOINT && req.method === "POST") {
        handleInvoke().catch(next);
      } else {
        next();
      }
    } catch (error) {
      next(error);
    }
  });

  const proxy = createProxyServer();

  return () => {
    server.middlewares.use(async (req, res, next) => {
      try {
        console.log("Proxying", req.method, req.url);
        proxy.web(req, res, { target: "http://localhost:7676" });
      } catch (error) {
        next(error);
      }
    });
  };
}

and on the Fastly side, we use this endpoint as our one way transport mechanism:

/// <reference types="@fastly/js-compute" />
import { ModuleRunner } from "vite/module-runner";

const runner = new ModuleRunner({
  hmr: false,
  transport: {
    async invoke(data) {
      return fetch("http://localhost:5173/@vite/transport", {
        method: "POST",
        body: JSON.stringify(data),
      }).then((r) => {
        if (!r.ok) {
          return { error: new Error(`Transport error ${r.status}`) };
        }

        return r.json() as Promise<any>;
      });
    },
  },
});

addEventListener("fetch", (event) => event.respondWith(handleRequest(event)));

async function handleRequest(event: FetchEvent) {
  const module = await runner.import("/src/entry.fastly.ts");
  return await module.default(event);
}

It could be extended to support two-way communication by implementing a suitable endpoint on the Fastly side as well but I didn't bother since Fastly Compute recreates the entire JavaScript context for every request so HMR is not useful anyway. It refetches and reevaluates every module for every request but it's still much much faster and better than the very slow wasm rebuilds.

Just my two cents.