🗣️ Webpack improvements

[rigor789]

Note: this document is still a draft and will change

• Goal: Improve usage and maintenance overhead of @nativescript/webpack
• Leads: @farfromrefug, @rigor789

Webpack configs are currently handled by having separate config templates for different project flavors. These templates are copied in the project's root folder in an npm postinstall hook (unless a config already exists). The correct config is determined based on installed dependencies.

Making changes to the official webpack configs is difficult as the changes usually have to be repeated for each template. After an update, users need to manually update their configs, or alternatively delete their existing ones and install @nativescript/webpack again in order to re-generate the config. This is generally not an issue, but many times projects need to customize the configs, making the update process even more difficult.

Project Overview

Create a new package in the workspace packages/webpack5 - it would still be published as @nativescript/webpack. The reason is to be able to start from scratch and rebuild it from the ground up with maintenance and better usage in mind - and allow to maintain current configs without interference.

Simplify maintenance by standardizing custom loader implementations, and their location in a loaders folder (or where it makes sense).

Design Goals

• build an extensible baseConfig
• different flavor-specific configs to extend from baseConfig (chainable)
• user configs to extend from flavor-specific configs

The baseConfig would generally be a regular webpack config that exports a function

module.exports = env => {
  // regular webpack rules that are shared across all flavors
  // workers, entries etc.
  return {
    // config
  }
}

A flavor specific config would extend the baseConfig:

const baseConfig = require('./baseConfig')

module.exports = env => {
  const config = baseConfig(env)
  
  config.module.rules.push({
    // a flavor specific module rule for example
  });

  return config;
}

Users would have a webpack.config.js that extends from the flavor config:

const { baseVueConfig } = require('@nativescript/webpack')

module.exports = env => {
  // allows making changes to env prior to calling the base config
  const config = baseVueConfig(env)
  // allows making changes/additions to the base config here
  return config;
}

[jamescodesthings]

+1 for this, one of the painful parts of the build for me is that any time the template is updated I have to rewrite portions of the new template.

Currently:

• The angular template is not fully ready for worker use, so I have to update the rules as per the nativescript-worker-loader guides.
• Aliases: We have a few that have to be pumped in.
• Env & Define: We use a few additional --env options to turn on/off features at build time. That are then pushed as vars via DefinePlugin.

If the config was extendable so that it easier to update that would be awesome.

[farfromrefug]

I personally was thinking of something a lot more strait forward in the sense of requiring a lot let work from us.
Basically the user calls tns which calls webpack (CLI or Api?) Using it s defined webpack config (default generated or custom one defined in config).
That webpack config,as said @rigor789 , calls a flavor one
That flavor one calls the basic default N one.
Now here is the magic part I am thinking about. I would like the flavor and the basic one to call N hooks on the config they generate! That way plugin can anything the webpack way!
Now is it doable? @rigor789 how did you manage to inject the "object" which handles the nativescript config in webpack ? You showed me the api which was able to read from the ts config file. Could we do it that way?

[rigor789]

@farfromrefug an example is in the PR: NativeScript/nativescript-cli#5381

We could probably call into hooks with that.

I would go for explicit inclusion: plugins can export a function users need to call from their webpack configs:

const { baseVueConfig } = require('@nativescript/webpack')
const setupSomePlugin = require('someplugin/setup') // naming can be a convention we decide

module.exports = env => {
  // allows making changes to env prior to calling the base config
  const config = baseVueConfig(env)

  // apply rules from the plugin by calling it's exported setup function
  // it's free to do any modifications to the config
  // optionally - it could be called prior to calling the base config, since these would be plain functions
  setupSomePlugin(env, config);

  // allows making changes/additions to the base config here
  return config;
}

Magically including webpack configs could lead to hard to debug/track down build issues in my opinion.

[farfromrefug]

I dont want to do it magically. I am thinking more in the lines of doing it in the right order:
Base config
Bade config hooks
Flavor config
Flavor hooks
User config
And it would be a call explicit but not at the user level. And we could imagine an env option ofr something to disiable those so that the user still gets the opportunity to disable them.

[farfromrefug]

@NathanaelA I agree with you on most things
I just don't see any need for your process includes stuff nor any logic our our side. Everything can be simply done within the normal webpack config files.
This is what we did for svelte for quite some time and it work well.
My point is the less special handling we add the less we have to maintain. And users can refer to webpack doc. We then don't need to write any.
@rigor789 looked at webpack 5 and I don't see any blocking thing there. And damn I want it for tree shaking !

[rigor789]

I strongly believe the cli & webpack should NOT be dependent upon each other beyond the cli calling a bundler then calling the native build tools. It should be divorced as much as possible from both...

@NathanaelA That is one of my goals as well - however the CLI is already fairly coupled to webpack, and decoupling it is out of scope for this project. I would love to revisit it at a later point though, especially now that the bundler space is very much improving and alive (vite, esbuild, parcel etc.).

So I hope you aren't also advocating rolling back that automagical support too as it would make plugins a lot more painful to use for users.

That's just twisting my words. That's not what I suggested...

The flavors include.webpack file ships with the flavor as part of the flavor and is maintained with and by the flavor author, and the main webpack file would just include it just like it would any other plugin

I feel like managing the flavor specific configuration in the @nativescript/webpack plugin is easier to maintain - no need to over-complicate things, that's what leads to unmaintainable mess.

As for the example usage you are suggesting - it's beyond what webpack api's provide, introducing a new "DSL" and complexity is not what we should be focusing on, but we can definitely iterate on the idea!

Auto-discovery of webpack configs can be much simpler, and easy to opt-out. Plugins could ship a configuration file in a standard location, with the standard webpack syntax:

// the file-name is debatable, since non-ns plugins could potentially include a webpack config if not packaged correctly
// node_modules/nativescript-plugin-example/webpack.config.js

// the only deviation from a standard webpack config would be the second parameter
module.exports = (env, config) => {
  // the plugin is free to make changes to the config
  config.plugins.push(new CopyWebpackPlugin({ /* ... */ }))
}

The project configuration would then explicitly call a applyPluginConfigs (the name is a placeholder - i'm sure it can be improved):

const { baseVueConfig, applyPluginConfigs } = require('@nativescript/webpack')

module.exports = env => {
  // allows making changes to env prior to calling the base config
  const config = baseVueConfig(env)

  // apply rules from all plugins that have customizations
  applyPluginConfigs(env, config)

  // optionally, the user can be specific by passing the plugin names
  applyPluginConfigs(env, config, {
     // this syntax would even allow passing parameters to plugins
    'nativescript-plugin-example': {
      someOption: true,
      maxSomething: 200
    }
  })
  
  // allows making changes/additions to the base config here

  return config;
}

The syntax and API is up for discussion, there are missing scenarios that we can solve by providing different methods:

• apply all plugins, but allow specifying options for specific plugins.

This approach would get us closer to best-of-both-worlds.

[rigor789]

The details aren't clear yet - as we are still moving things around and figuring stuff out, but the progress on the PR looks like it will be able to solve all the issues you mentioned, as we opted to use webpack-chain which gives us the flexibility to modify anything in the config.

A very early example of it is something like: tap into the existing ts-loader options, and just add appendTsSuffixTo: [/\.vue$/],. (we probably want to merge in this case - since there may already be values in appendTsSuffixTo)

// set up ts support in vue files
config.module
  .rule('ts')
  .use('ts-loader')
  .loader('ts-loader')
  .tap((options) => {
    return {
      ...options,
      appendTsSuffixTo: [/\.vue$/],
    };
  });

How the webpack-chain configs can be exposed to the user - we are still testing different approaches to make it as straight forward as possible, for both users and plugin authors.

[rigor789]

@NathanaelA Yes, it is in the PR, but far from something useful - we only just started:
https://github.com/NativeScript/NativeScript/blob/feat/webpack5/packages/webpack5/src/configuration/vue.ts

We set up jest for testing, here's what the vue stuff looks so far (this isn't the actual config, rather a human-readable version from webpack-chain using toString() - pretty cool format for debugging, it adds comments to each block, how you may call it from webpack-chain):
https://github.com/NativeScript/NativeScript/blob/feat/webpack5/packages/webpack5/__tests__/configuration/__snapshots__/vue.spec.ts.snap

It builds on top of base (which needs lots more work to be something we can use). No loaders or plugins have been ported into the new package yet, as the goal is to only bring things we actually need - there's a lot of legacy code in the current webpack package.

[shirakaba]

Very curious about webpack-chain. Looks nifty; might do the trick.

I’ve felt very limited by straight up extension of Webpack configs (how we’re doing it right now). Hacking around with plugins.find() and plugins.splice() has been very messy. And defining extra WebpackDefinePlugin definitions wasn’t fun, either.

Worst of all has been trying to roll back CopyWebpackPlugin because it is invoked with a new (and mutually exclusive) API in the base config, yet there’s no way to inherit it and swap it out without instantiating an instance of CopyWebpackPlugin (which inevitably causes a crash if you’ve deliberately installed the older version of the plugin, due to the incompatible construction args).

I couldn’t think of an elegant answer to the frictions I encountered (though a few user- and dev-unfriendly options came to mind), so I’m very interested to see what options webpack-chain opens up through its builder pattern. Good find, @rigor789!

[edusperoni]

I suggest we take a look at https://github.com/GoogleChromeLabs/worker-plugin. The current worker-loader implementation is based on https://github.com/webpack-contrib/worker-loader

Mainly we don't need stuff like import Worker from 'worker-loader!./Worker.js'; and it already has support for reusing plugins (https://github.com/GoogleChromeLabs/worker-plugin#plugins-array) which was something I had to add to ensure compatibility with Angular (NativeScript/worker-loader#39)

[rigor789]

@edusperoni seems like Webpack 5 supports workers out of the box, with a slightly different syntax:

const worker = new Worker(new URL('./worker.js', import.meta.url));

I've tested the worker-plugin and it doesn't work with webpack5, and the built-in way is mentioned in their docs:

Compatibility Note: Webpack 5 now includes worker bundling. It uses a slightly different syntax:
new Worker(new URL("./my_worker.js", import.meta.url))

This still needs testing - or perhaps if the worker-plugin releases a new version that just converts the syntax to the webpack syntax, that would be pretty nice!

[rigor789]

Closing in favor of the pending RFC: #2

Please continue discussion under the RFC!