[RFC] Define a design token structure starting with colors

[MichaelArestad]

Summary

I propose we take a pass at coming up with a defined token structure that will be used as a basis for eventual design token implementations. This will help organize our design tokens, make them predictable, reduce the need for one-off implementations, and afford flexibility for user-defined tokens and themes.

In the near-term, I want to agree on a structure to start implementing updates to color schemes for Storybook 10. Long term, this will help us plan for, and eventually implement, design tokens.

Problem Statement

There are a range of problems with our current theme and color implementation:

• The current color objects are a mix of semantic and primitive definitions. They are used in a way that is difficult to predict and often require overrides based on the theme.
• The current color objects are unpredictable as there isn't a defined pattern implemented for naming and grouping of properties.
• The current theme object is a mix of primitive, semantic, and component (ish) tokens that make it difficult to predict and rather inflexible.

Non-goals

• Implementation of this
• Switching to vanilla CSS

Implementation

This is a WIP and I will do my best to keep the latest proposal updated here.

Here is roughly what I am picturing along with a few questions:

// primitives
{
    "color": {
        "black": {...},
        "white": {...},
        // Should we use `neutral` over `monochrome`?
        "neutral": {
            "0": {
                $value: '{color.white}',
                $type: 'color'
            },
            "1": {...},
            "2": {...},
            // ...
        },
        "blue": {...},
        "green": {...},
        "yellow": {...},
        // ...
    },
}

// semantic
// overall structure: `property.modifierA.modifierB.state`
// example: `fgColor.status.success.hover`
{
    fgColor: {
        // Should `default`, `muted`, `inverted`, etc., be grouped like `status` colors?
        "default": {
            $value: '{color.neutral.7}',
            $type: 'color'
        },
        "muted": {...},
        "inverted": {...},
        "disabled": {...},
        "accent": {...},
        "status": {
            "success": {...},
            "warning": {...},
            "critical": {...},
        }
    }
}

// component
{
    "button": {
        "default": {
            "fgColor": {
                "rest": {
                    $value: '{fgColor.default}',
                    $type: 'color'
                },
                "hover": {...},
                "active": {...},
                "selective": {...},
                "disabled": {...},
            },
            "bgColor": {...},
            "borderColor": {...},
            "iconColor": {...},
        },
        "primary": {...},
        "outline": {...},
        "ghost": {...},
        "danger": {...},
    },
}

Prior Art

There is a ton of prior research and opinions on this stuff. Here are a few resources worth peeking at, many of which were recommended to me by @Sidnioulz :

• Intuit's taxonomy - good primer on differences between primitive, semantic, component tokens
• Codifying design systems - solid video via Adobe
• Github Primer Primitives - very complex, but nifty example
• Mise en mode - fancy way of thinking and organizing in modes
• designtokens.fyi - nice reference for various Design Systems terms
• Scaling Spectrum (Adobe)
• Spectrum tokens
• Color tools for design systems

I will add more to this list as this is a WIP.

Deliverables

We will document a defined token structure to eventually use. Likely, there will be spikes for implementation, but this RFC is primarily to get us aligned.

Risks

• We might disagree on proposed solutions. Can be mitigated with positive attitude and/or swag.

Unresolved Questions

• Should we use both Semantic and Component tokens?
• What structure would you prefer? Example property.modifier.state? Something else?
• How will this impact theming?
• Could all of our color tokens derive from a single hue via relative color transforms?
• Should we start with color tokens?

Alternatives considered / Abandoned Ideas

No response

[Sidnioulz]

To reiterate what I shared when we last talked, for people reading this RFC:

• My usual plan with tokens is to figure out the long-term vision, define a token taxonomy and test it thoroughly, and only then plan a migration which can take as long as needed
• trying to save time in defining the taxonomy, or partially rolling out tokens with a different taxonomy than the long-term vision, results, in my experience, in having to migrate the code multiple times rather than once
• though Storybook is very familiar with codemodding, and we could automate small adjustments

Hard agree on the problem statement:

• It's currently hard to design new UI and components both internally and for addon devs
• It's hard to theme because theme keys get used all over the place in all sorts of combinations with their neighbours

Desired token properties

Now, to develop the right taxonomy, let's first align on desirable properties:

1. We want support for light and dark modes
2. We want to allow themes/modes to apply by region, e.g. sidebar, toolbar, docs, preview, panels (doesn't necessarily affect taxo)
3. Two options for degree of themeability:
   • Light changes only: we let users redefine accent colours (and maybe in the same vein, a border radius base and component spacing base?) and we compute a theme for them
   • Fully customisable: users can implement most brand identities by customising the theme further and adjusting edge cases

1. Support for light and dark modes

This is a must to match current features of the theming API v1. Adopting modes implies no side effects on tooling or code architecture. Modes can also easily be extracted from tokens to inject in the core stories and Chromatic, so they would be fully viewable and testable with the current tools.

2. Applying modes by region

@shilman curious to see if you think there is user demand here, just for process's sake.

Re: feasibility, injecting modes/themes anywhere in the app is trivial if we use CSS properties to deliver tokens. It's typically done by adding a CSS class to the element where you want to inject properties. As DOM is not modifiable by end users, we could allow @apply rules in *-head.html and preprocess them to apply modes to e.g. .sidebar, .toolbar, .docs, or we could provide a JS API to set the names of the modes to apply to each region.

3. Degree of themeability

I have no stake or comment on which theming option is right. I think this is a product/business question. I'm just providing implementation comments, and I must warn that switching between both will be prohibitively expensive, and so the choice is likely definitive.

Light theming

The computing of a theme is easily done through Style Dictionary / Terrazzo, though this involves a pre-build step. It is also possible to build a CSS system that fully automates computations from a base value, especially with colour because we have CSS color functions. For spacing/typography sizes, we'd likely need a lot of calc and that would make the output less easy to debug. More on that later.

Fully customisable

This can't be carried by semantic tokens alone, as there is no universal taxonomy for every brand ID out there. Try to implement Merck's brand identity on a Material app for instance. From experience, every taxonomy needs constraints to ensure different roles can be placed next to each other and be recognisable. Most orgs build a taxonomy that constrains around their product/UI needs and their brand identity.

To do full white-label, we need escape hatches, aka component tokens. They allow reworking a component that won't behave as expected. The direct implications are more tokens to maintain, and more well thought-out theming APIs for each component.

Component tokens also provide more API stability. If the underlying semantic tokens change, components are less likely to need a rework. This makes sense for e.g. Adobe because they have many implementations but less for us with a single codebase.

Size mode automation

Adobe does something interesting with its size tokens though: they have very few base values and then get transformed based on components' size prop. Combined with mise en mode, we could end up with something simple. Reusing the corner radius example from Garth's video:

Input code:

<Button size="small">Accept this RFC</Button>

.button {
  border-radius: var(--border-radius-default);
}

Generated code:

:root {
  /* For tiny components e.g. checkbox */
  --border-radius-small: 4px;
  /* For 95% of components */
  --border-radius-default: 6px;
}

.size-small {
  --border-radius-small: 3px;
  --border-radius-default: 4px;
}

.size-large {
  --border-radius-small: 5px;
  --border-radius-default: 8px;
}

// etc

Output:

// 4px radius
<button class="button size-small">Accept this RFC</button>

We can automate/simplify even further by computing the small/large values of a token from a base modifier to the extent where these size modifiers just scale a component up and down. We mostly use sizes to make the UI more dense, so let's apply this just to spacings, as scaling down typography would ruin our UI.

(I believe this is what they're doing in Spectrum 2 based on https://spectrum.adobe.com/page/platform-scale/, I can ask around if you want confirmation)

Input code:

<Button density="dense">Accept this RFC</Button>

.button {
  border-radius: var(--border-radius-default);
}

Generated code:

:root {
  --spacing-base: 1;
  --border-radius-default: calc(--spacing-base * 6px, 6px);
}

.density-dense {
  --spacing-base: .8;
}

.density-airy {
  --spacing-base: 1.15;
}

// etc

Output:

// 4.8px radius; paddings also scaled down by 80% but not font
<button class="button density-dense">Accept this RFC</button>

Color mode automation

Astro Starlight successfully exposes hues for major colours like success, warning, note. They don't even do semantic tokens but we could do that just for more clarity for end users. We could add the main brand and an accent colour and it would cover the needs for light theming, and it would allow a first layer of easy customisation if we go the full theming route.

Taxonomy structure

I'd love to do a quick audit of product/UX concepts we need, and of the interaction patterns used to convey something is clickable, focusable, currently selected, etc. So we know what vocabulary we want for semantic tokens.

I'd definitely keep the structure identical within a type of token though, to allow for adding values without name conflicts.

It's also useful to separate types of information in separate bits of a token's name:

• type of token
• categorical information (e.g. error colors, but also product concepts like onboarding or testing if you wanted to give them a visual style)
• hierarchical information (prominent, normal, muted colours, heading, copy, label typography)
• anything related to a type of mode e.g. themes/color modes, size modes, component states, usually at the extremity of a token name so you can easily hide it behind an automation in code

[shilman]

@shilman curious to see if you think there is user demand here, just for process's sake.

Not AFAIK but y'all might have more insight into this than me!

[MichaelArestad]

It's low-demand, but useful. I think it's worth considering, but not a requirement.

[Sidnioulz]

To add to the resources list, more cool Adobe stuff:

• https://www.knapsack.cloud/blog/garth-braithwaite-on-design-tokens-governance-and-scaling-spectrum-at-adobe
• https://opensource.adobe.com/spectrum-tokens/visualizer/
• https://leonardocolor.io/#

[MichaelArestad]

Updated the list. These are great!

[JReinhold]

The proposed structure is JSON/JS object, that looks like format from the Design Tokens Community Group (@Sidnioulz can you help me out here, what is the actual name of the format? I can't find it anywhere)

1. How do you propose we author this? Do we write this by hand, or do you see a conversion tool that outputs this in the end?
2. How do you propose that we consume this? I believe there's consensus on consuming the Storybook theme via CSS Custom Properties (and by extension, allowing users to override these CSS Custom Properties), but how do we go from this JS-based format to CSS Custom Properties? Does a tool exists that will do this for us, or do we need to write a parser+writer that will convert this to CSS at build time?
3. If I'm understanding it correctly, I'm seeing both Tier 1, 2 and 3 colors in your proposal. I think there is sense in splitting these up by files somehow/somewhere, so that every component's tokens (Tier 3) doesn't live alongside the base Tier 1 tokens, as that would get big very quickly.

It looks like Brad+Ian Frost's Design Tokens Course answers a lot of my questions above, so it might make sense to invest time for one of us to actually go through the course and see what they recommend.
It would also just be great to get @bradfrost's input on this whole RFC.

I explored the idea of the format instead just being hand-authored CSS Custom Properties, but I'm not sure if that is too error prone on a larger scale, and I'm not sure how to do that when considering Tier 2 and 3 also:

:root {
  --sb-color-white-0: hsl(bla, bla bla);
  --sb-color-white-1: hsl(bla, bla bla);
  --sb-color-white-2: hsl(bla, bla bla);
  --sb-color-white-3: hsl(bla, bla bla);

  --sb-color-neutral-0: hsl(bla, bla bla);
  --sb-color-neutral-1: hsl(bla, bla bla);
  --sb-color-neutral-2: hsl(bla, bla bla);
  --sb-color-neutral-3: hsl(bla, bla bla);
}

[Sidnioulz]

By the way, I just want to acknowledge that there are two conversations that must happen:

• how should the tokens be structured (this RFC)
• how they should be stored / edited / synchronised

Even with pure CSS variables, no DTCG files and no transforms, we'd need to have a sound taxonomy that addresses our goals.

[MichaelArestad]

FWIW, I would prefer the code be the source of truth for color, typography, and size tokens. I have seen projects like Primer that output files that Figma can consume.

[Sidnioulz]

@MichaelArestad by that, do you mean that

1. You want the SSOT outside of Figma
2. You want the SSOT to be the values consumed in the production code

The latter may sound beneficial (I love to say prod is the actual source of truth :D) but it isn't necessarily simpler to parse a JS or CSS data structure and extract tokens than to parse the now-standard DTCG format. There'll likely be a broader offering of tools to handle DTCG in the long run.

[MichaelArestad]

@Sidnioulz I prefer the source of truth to be outside of Figma. There are no safeguards in Figma to prevent someone from tweaking a variable as far as I know. Our design team is small, but it's still a risk.

[Sidnioulz]

Makes sense! Though, I think there's a way you can protect the collection: you can handle it in a file where you set more restrictive permissions, and then publish the file and reference it in your designs. I think that's how large orgs do it but I haven't managed a Figma org in a while.

[bradfrost]

Hey all! Great conversation, and apologies for being late to the party. I'm happy our course has helped address some of the questions, and happy to weigh in in more detail. Here are a few thoughts:

• Light and dark mode are fantastic, and I also think you all have a tremendous opportunity to help people customize the UI to meet their brand's needs. I've certainly been in many conversations where that's been the center of the conversation around whether or not to adopt Storybook.
• Tokens can be defined in an implementation-agnostic format like JSON, allowing you to convert things into whatever format you need. It's not a good idea to have the source of truth defined in Figma, especially given the nature of tool and core competency.
• Implementing the tokens as CSS Custom Properties would be amazing, as those variables are live and can be manipulated in the browser. You can do all sorts of fun stuff with that!
• The Tier 2 semantic tokens should do the overwhelming amount of the heavy lifting. But we've found that a handful of targeted Tier 3 component-specific tokens for prominent-yet-variable components like buttons and for other targeted use cases like focus ring color.

If there's other things I can be helpful with, just let me know!