Stabilizing deps optimization at build time (and SSR)

[patak-dev]

Deps Optimization at build and SSR

• Added as opt-in from vite@3.0.0
• Original PR
• Rationale for making it opt-in here
• API to stabilize
  • optimizeDeps.disabled
  • ssr.optimizeDeps

Open questions:

• We could only move this feature out of experimental, or make it the default.
• We made this feature optional when we discover tree-shacking issues. After Vite 4.3, we think these issues should be resolved. We need feedback from real use cases here.
• The config options are hard to use. We used negation looking at deps optimization being enabled by default. We should evaluate using optimizeDeps.dev: boolean, optimizeDeps.build: boolean instead of optimizeDeps.disabled: 'dev' | 'build' | boolean
• During Dev SSR, we only optimize what is explicitly defined in the config. I think this is the best we can do for now, but it is a limitation.
• During Build, we have issues when workers have deps to optimize.

If you have feedback about this feature, let's use this discussion to gather it.

Resolution: #13839 (comment)

[windsonR]

I have two questions for this.

1. Looks like it must set build.rollupOptions.maxParallelFileOps to 100.
2. For me, I still face the problem of tree shaking.

use esbuild:

use rollup:

My solutions:

1. It's not easy to find the solution in document, and i searched in ISSUES for a long time. Finally found it here
2. Maybe I can config optimizeDeps.entries to avoid it.

[patak-dev]

This feature will remain experimental until these two issues are solved. Thanks for writing these down. About 2, it would be helpful to create a minimal reproduction and an issue in the repo.

A note: Given the trade-offs, I'm no longer sure that we should pursue this feature.

[windsonR]

I've report a issue

[hi-ogawa]

Hi. This is a feedback specifically for the use case of ssr.optimizeDeps during dev as my usage is based on Vitest's vite-node https://github.com/vitest-dev/vitest/tree/main/packages/vite-node.

Actually this issue on Vitest vitest-dev/vitest#4910 made me realize why ssr.optimizeDeps might not be working on my direct use case of vite-node hi-ogawa/vite-plugins#151.

As reported on Vitest's issue, it seems currently calling ViteDevServer.ssrLoadModule is required to trigger deps optimization for dev ssr. However, if I'm reading it correctly, Vitest/vite-node uses only other APIs such as transformRequest, resolveId, etc... since vite-node doesn't directly load module via ssrLoadModule API. Thus, for normal usage of vite-node, ssr deps optimization won't kick in.

On my use case, triggering ssrLoadModule during configureServer hook seems to work (my current workaround is here hi-ogawa/vite-plugins#152), which is something along the line of:

import { defineConfig } from 'vite';

export default defineConfig({
  plugins: [
    {
      name: 'force-ssr-optimize-deps',
      apply: "serve",
      configureServer(server) {
        return async () => {
          // load any module to trigger deps optimization
          await server.ssrLoadModule('/dummy-module.js');
        };
      },
    },
  ],
});

My concrete feedback/question would be:

• Is calling ssrLoadModule meant to work during configureServer hook like above?
• Would it make sense for Vite to provide some option to eagerly initialize ssr deps optimization without dummy ssrLoadModule call?
• Is there any alternative I'm missing?

On additional notes, noDiscovery and disabled options might be also relevant to how ssr deps optimization work. I needed noDiscovery: true and disabled: false (or "build") to make ssr deps optimization work for my usage, which is also what I copied from Vitest:
https://github.com/vitest-dev/vitest/blob/043b78f3257b266302cdd68849a76b8ed343bba1/packages/vitest/src/node/plugins/utils.ts#L46-L47

Maybe I should mentioned this earlier as a high level motivation of why I need ssr.optimizeDeps. The reason for this is that my experimental idea (which I called vite-node-miniflare) is to run Vite-based framework's SSR code during dev on Cloudflare Workerd by making use of vite-node's client/server architecture. For this purpose, I wanted to pre-bundle cjs-only dependency (such as react and react-dom) on Vite server side, then Vite-node client (which runs on Workerd) can execute only pre-bundled esm code. So, this use case assumes ssr.noExternal: true as well.

[patak-dev]

Thanks for the feedback! Let's move the init up, PR here: #15561

About noDiscovery: true, that should have no effect for dev ssr, only deps explicitely listed are optimized in dev ssr. disabled is needed as it is true by default.

By the way, we are looking into removing build time deps optimization altogether (probably in Vite 5.1)

• refactor: remove build time pre-bundling #15184

[hi-ogawa]

Nice, I'll keep an eye on your PR!

About noDiscovery: true, that should have no effect for dev ssr, only deps explicitely listed are optimized in dev ssr. disabled is needed as it is true by default.

Indeed, I just verified it works without noDiscovery. I think I mixed up something while flipping through flags. Thanks for pointing it out!

disabled is needed as it is true by default.

I see. I think I was confused by @default 'build' in jsdoc, but I supposed this default is only meant for top-level optimizeDeps but not for ssr.optimizeDeps.

vite/packages/vite/src/node/optimizer/index.ts

Lines 121 to 128 in a300867

	/**
	* Disables dependencies optimizations, true disables the optimizer during
	* build and dev. Pass 'build' or 'dev' to only disable the optimizer in
	* one of the modes. Deps optimization is enabled by default in dev only.
	* @default 'build'
	* @experimental
	*/
	disabled?: boolean | 'build' | 'dev'

[patak-dev]

Yes, disabled ended up being very confusing. I'm happy it will be gone as part of the removal of build time deps optimization. We're still discussing there if we need the flag at all (as you can get the same with noDiscovery: true + include: [] or undefined

[jthemphill]

Deps optimization with vite dev seems to break anything that relies on .wasm files produced locally and bundled via pnpm.

When I try to build the pnpm branch of my repo:

jhemphill@macbook-pro-4 www % pnpm run build:wasm-st && pnpm run dev

> htmf@0.5.0 build:wasm-st /Users/jhemphill/oss/htmf/wasm/www
> wasm-pack build --target web --profiling

[INFO]: 🎯  Checking for the Wasm target...
[INFO]: 🌀  Compiling to Wasm...
warning: unstable feature specified for `-Ctarget-feature`: `atomics`
  |
  = note: this feature is not stably supported; its behavior can change in the future

warning: unstable feature specified for `-Ctarget-feature`: `bulk-memory`
  |
  = note: this feature is not stably supported; its behavior can change in the future

warning: unstable feature specified for `-Ctarget-feature`: `mutable-globals`
  |
  = note: this feature is not stably supported; its behavior can change in the future

warning: `htmf` (lib) generated 3 warnings
warning: `htmf-bots` (lib) generated 3 warnings (3 duplicates)
warning: `htmf-wasm` (lib) generated 3 warnings (3 duplicates)
    Finished release [optimized] target(s) in 0.05s
[INFO]: ⬇️  Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: Optional fields missing from Cargo.toml: 'description', 'repository', and 'license'. These are not necessary, but recommended
[INFO]: ✨   Done in 0.52s
[INFO]: 📦   Your wasm pkg is ready to publish at /Users/jhemphill/oss/htmf/wasm/pkg.

> htmf@0.5.0 dev /Users/jhemphill/oss/htmf/wasm/www
> vite


  VITE v5.0.12  ready in 123 ms

  ➜  Local:   http://localhost:5173/
  ➜  Network: use --host to expose
  ➜  press h + enter to show help

However, navigating there will give you a relatively blank page, with this error in the JS console:

Uncaught CompileError: WebAssembly.instantiate(): expected magic word 00 61 73 6d, found 3c 21 64 6f @+0

This is because the request for the .wasm file didn't go through. As a result, we got 404 HTML back, which does not in fact begin with the correct sequence of bytes to be interpreted as webassembly!

Navigating to the network tab, you can see that we're requesting the .wasm file from http://localhost:5173/node_modules/.vite/deps/htmf_wasm_bg.wasm, but the dependency optimizer didn't move the .wasm file there.

To see what this project is like when it works:

1. https://jthemphill.github.io/htmf/ contains a working production version
2. pnpm run preview builds the project correctly, without dependency optimization
3. The main branch, which still uses npm, will work if you run npm run dev.

[patak-dev]

Would you create an issue so we can properly track this? This discussion was about build time optimization and you are talking about optimization during dev.

[jthemphill]

Yes, sorry about that.

[patak-dev]

Deps optimization at build time was removed in vite@5.1. Check out the rationale at:

• refactor: remove build time pre-bundling #15184

With Rollup 4 switching its parser to native, and Rolldown being worked on, both the performance and the dev-vs-build inconsistency story for this feature are no longer valid. If we want to focus on improving dev/build consistency, then using Rolldown for prebundling during dev and Rolldown as-is during build is a better bet moving forward. Rolldown may also implement caching in a way that is a lot more efficient during build than deps prebundling.

[KevinGhadyani-Okta]

Many of my projects only uses Vitest, not Vite.

Because Vitest uses Vite under-the-hood, I'd love to remove dependency optimization altogether with an option like this.

Dependency optimization doesn't help with CI as far as I understand, and it's causing strange issues in the CI where tests fail the first run only while it's creating the node_modules/.vite/deps/ directory, but after it's created, the tests run fine from there. Could be an internal Vite issue.

Also, running vitest run is the issue, not vanilla vitest which I think also has something to do with how deps are optimized on the first run.

Lastly, if I use the Vitest VSCode plugin, it doesn't create deps/; therefore, those same tests pass no problem even on the first run.

I would recommend having some way to configure around having dependency optimization.

[KevinGhadyani-Okta]

Looking through other comments and the docs, it says optimizing deps before bundling was removed in Vite 5.1.x. The Vitest version I have pulls in Vite 6.2.0. I'm wondering if I'm not understanding this feature or the option to disable optimized deps altogether.

[KevinGhadyani-Okta]

Vite handles workspaces differently. optimizeDeps needs to be placed on each workspace. Then you can configure it from there. That's not covered in the TS rules though.

And it's building both workspaces dependencies at the same time. If they conflict, that's where problems start.

Sorry, this is all my particular issue. I'll leave it here in case someone finds it useful in the future.