questions about creating a preset

[faassen]

I discovered presetter yesterday and today we played with it. I'm excited about a tool that can turn boilerplate into a reusable package! I have a few questions and comments; my apologies if issues are not the right channel for it.

I looked at the sample preset. Unfortunately it contains quite a bit of code, which makes it somewhat difficult to comprehend what is essential for a preset and what is incidental. I think I understand from the documentation that the essential contract is quite minimal, but it's hard to see. I think a very minimal preset that just contains one config file, a single development dependency and a single script would be very helpful.

The example preset offers a particular system for managing configuration files as yaml files and then converting them into a json format, with the option for overrides. I think it would be interesting to consider extracting this and create a reusable package that presets can build on. That would be another way to make the example preset less intimidating as well.

My first reaction is to wonder why the files are maintained in the yaml format while they are generated as JSON. It makes it harder for me to recognize familiar configuration files. Is this because with yaml you can address overrides in a convenient manner? It won't work for all possible configuration files as not everything is expressed as JSON; I see for npmignore there's a special system for a list-based config.

I spent a bit of time looking for a way to interpolate template variables, but I didn't find any. While most config files don't need a variables, in some cases it would be very useful to be able to refer to the repository or package name within a particular config file. Perhaps the system could even automatically extract this from package.json and make it available.

In the example preset I don't see any examples of configuration that is stored in a directory. husky is an example of such a configuration system. Does presetter support this? Another use case is to store github actions configuration in .github in a preset.

Finally I have a question about the way scripts are integrated. We noticed that when running a script it would modify the package.json and then after run replace it again with the original version. I saw that you mention in the FAQ what to do if that fails and leaves the package.json changed by accident. I was trying to understand why modifying the scripts section is necessary in the first place. Why can't presetter run work without this modification? Is this because it needs those scripts to exist to allow executing them exactly as if they were really in the source code? It'd be nice if there were a way to issue commands programmatically in the same context as scripts.

Thanks again for producing this very interesting tool! I'm going to experiment with it some more.

[alvis]

Hi @faassen. I really appreciate your comments. It took me a few days to digest them and think about how to improve presetter.

In fact, when I'm using presetter myself, I faced a similar issue when I was creating other presets for different project types, for example, a project for my backend with Pulumi, a project for a Gatsby site, or a project for a React component library.

Initially, presetter was designed to let each preset decide how to generate the configuration files. They can dynamically decide which content to generate based on the supplied config from .presetterrc and whether the content is from a yaml file, or toml or even just all in the typescript. Hence you see a bit of code there in the example preset.

However, the more preset I create the more I realise there is a common pattern on how a preset is created and I don't want to copy and paste the same code to create a preset too!

So to simplify the preset creation process, I'm writing V3. And here is an example of how a future preset can be created: Just one single simple typescript file as a resource manifests together with a number of templates (like this), no more other code is need. Have a look:

import { resolve } from 'path';

// paths to the template directory
const TEMPLATES = resolve(__dirname, '..', 'templates');

/** config for this preset */
export interface PresetConfig {
  /** configuration to be merged with .babelrc */
  babel?: Record<string, unknown>;
  /** configuration to be merged with .eslintrc */
  eslint?: Record<string, unknown>;
  /** configuration to be merged with .jestrc */
  jest?: Record<string, unknown>;
  /** configuration to be merged with .lintstagedrc */
  lintstaged?: Record<string, unknown>;
  /** patterns to be added to .npmignore */
  npmignore?: string[];
  /** configuration to be merged with .presetterrc */
  prettier?: Record<string, unknown>;
  /** configuration to be merged with tsconfig.json */
  tsconfig?: Record<string, unknown>;
}

/** List of configurable variables */
export interface Variable {
  /** the directory containing the whole repository (default: .) */
  root?: string;
  /** the directory containing all source code (default: source) */
  source?: string;
  /** the directory containing all extra typing files (default: types) */
  types?: string;
  /** the directory containing all the compiled files (default: lib) */
  output?: string;
  /** the directory containing all test files (default: spec) */
  test?: string;
}

/** detail of linked/created configuration files and script templates  */
export interface PresetAsset {
  /** mapping of files generated from configuration template files (key: file path relative to the target project's root, value: template path) */
  create?: Record<string, string>;
  /** mapping of symlinks to generated configuration (key: file path relative to the target project's root, value: template path) */
  symlink?: Record<string, string>;
  /** path to the scripts template */
  scripts?: string;
  /** default variables */
  variable?: Variable;
}

export const DEFAULT_VARIABLE: Variable = {
  root: '.',
  source: 'source',
  types: 'types',
  output: 'lib',
  test: 'spec',
};

/**
 * get the list of templates provided by this preset
 * @returns list of preset templates
 */
export default async function (): Promise<PresetAsset> {
  return {
    create: {
      '.gitignore': resolve(TEMPLATES, 'gitignore'),
      '.github/workflows/test.yaml': resolve(TEMPLATES, 'github_workflows', 'test.yaml'),
    },
    symlink: {
      '.babelrc.json': resolve(TEMPLATES, 'babelrc.yaml'),
      '.eslintrc.json': resolve(TEMPLATES, 'eslintrc.yaml'),
      '.jestrc.json': resolve(TEMPLATES, 'jestrc.yaml'),
      '.lintstagedrc.json': resolve(TEMPLATES, 'lintstagedrc.yaml'),
      '.npmignore': resolve(TEMPLATES, 'npmignore'),
      '.prettierrc.json': resolve(TEMPLATES, 'prettierrc.yaml'),
      'tsconfig.json': resolve(TEMPLATES, 'tsconfig.yaml'),
      'tsconfig.build.json': resolve(TEMPLATES, 'tsconfig.build.yaml'),
    },
    scripts: resolve(TEMPLATES, 'scripts.yaml'),
    variable: DEFAULT_VARIABLE,
  };
}

Regarding your question, with V3, you can provide a configuration template in json or yaml or text.

For a text template, for example, .npmignore or .gitignore, the merging strategy is a bit different. Any customization will be appended to these files. So you can grow your list to include files you want.

For json and yaml, it's just your personal preference, but I prefer yaml because I can add some comments to my template files. For example, those comments in my eslint config. Any customization will be simply deep merged with the template. So let's say you have an eslint.yaml template like this

extends:
  - eslint:recommended

and you want to further extend it with prettier, you can setup a .presetterrc with the following content:

{
  "preset": "<your preset package>",
  "config": {
    "eslint": {
      "extends": ["prettier"]
    }
  }
}

Then your generated .eslintrc.json will look like this

{
  "extends": ["eslint:recommended", "prettier"]
}

The output format will be completely depending on the extension given by the preset. For example, .babelrc.json in symlink will be in json format, while the .github/workflows/test.yaml will be in yaml format.

For variables, now I think you can see how they are set from the preset. Also, you can configure them in .presetterrc. e.g.

{
  "preset": "<your preset package>",
  "variable": {
    "output": "dist"
  }
}

Let me know what do you think?

[faassen]

PS. Happy to do voice or video chat if that makes this conversation easier.

[faassen]

(or text chat. I'm looking for ways to contribute and collaborate)

[alvis]

I see the interfaces Variable and PresetAsset. But aren't these generic among all presets? Right now you don't have a preset types library that I can import those interfaces from, but if we did, the preset code would become that much simpler as I don't see a reason to keep defining them again.

Yes, PresetAsset is generic among all presets. You can import them from presetter and you can see an example here.

For Variable, its interface is simple Record<string, string>.
I made a Variable interface just to make clear which variables are expected by the preset.
Also, it helps to define a sensible DEFAULT_VARIABLE to be exported together with the preset.

[alvis]

You use __dirname to resolve files in the preset. I don't know too much about ESM modules yet, but I do know that __dirname isn't compatible with it. It might be worthwhile to consider letting the presetter framework itself resolve to a config file given a path, though that might require messing around with import.meta. Then again, this is probably not a big deal as the ESM equivalent could be written.

Yes, __dirname (and __filename) is no longer available in ESM :(
The reason for using __dirname is simply a way to resolve the directory containing those template files.

But as a dev tool, it is fine because all the goods from ESM, e.g. tree shake, don't apply. Plus, node doesn't run ESM code natively by default and possibly never for compatibility reasons. Having said that, if one day really node doesn't support CJS anymore and have to use ESM everywhere, I'm not sure though.

[alvis]

PS. Happy to do voice or video chat if that makes this conversation easier.

Of course, I'm happy to chat. You can drop me an email at alvis at hilbert dot space & let's arrange. ;)

[alvis]

Regarding how scripts are integrated with your existing package.json, here is the workflow:

1. Compute additional scripts from the templates
2. Back up the existing package.json to ~package.json
3. Integrate the additional scripts with the existing one to package.json
4. Invoke npm run <task name> as usual
5. After the task is done, restore the backup package.json

The reason for this flow is to avoid reinventing the npm run mechanism.
Also, it makes sure other toolsets (e.g. npm-run-all) work too.

[faassen]

I was just hoping that there is some npm API that would allow us to replicate npm run without actually having to add scripts. But I don't know whether such a thing exists.

[alvis]

I tried to dig into that, but unfortunately, there's no public API for that.

I had a search before. npm run is implemented in https://github.com/npm/cli and it has 2 problems:

1. It doesn't export the function for public use, which is less troublesome because private code doesn't exist for JS 😅 , but
2. Every time npm run is invoked, it reads package.json so package.json must have the merged scripts. Therefore if your scripts provided by a preset have chained scripts (e.g. "build": "npm run clean && npm run rollup") then it won't work...

[alvis]

@faassen presetter 3.0 is out! There are some example presets. Check them out
https://github.com/alvis/presetter#demo-presets