Modularize RTK App

[sam-mfb]

This is re-post (at Mark's suggestion) of this SO question, with some updates

The Question

Is it possible to separate out the feature of an RTK-based application that depend on different slices of a the redux store into separate node packages? Assuming so, what is the best way to do that?

Background

We have a large, and growing, app that is based around Redux Toolkit. Where possible we try to separate parts of the application into their own node packages. We find there are a lot of benefits to doing this, including:

• Maintainability of codebase
• Fine-grained control over intra-application dependencies
• Testability

It's easy enough to do this for cross-cutting things, like logging, http requests, routing, etc. But we would like to go further and modularize the "features" of our app. For example, have the "address book" feature of our application live in a different module than, say, the "messages" feature, with them all composed together via an "app" package.

The benefits we see here are ones we have found in other codebases and have been discussed in other places. (E.g., here for iOS). But, in brief: (1) you can see and control intra-app dependencies. For example, you can easily see if the "messages" feature depends on the "address book" feature and make explicit decisions about how you will expose the one feature to the other via what you export; (2) you can build fully testable sub-parts of the app by simply having a "preview" package that only composes in the things you want to test, e.g., you could have a "contact app" package that only depends on the "contact" feature for building and testing just that; (3) you can speed up CI/CD times by not needing to compile (TS/babel), pack/minify, and unit test every part; (4) you can utilize various analytics tools to get more fine-grained pictures of how each feature is developing.

There may well be other ways to achieve these things, and some may disagree with the premise that this is a good way to do it. That's not the focus of the question, but I'm open to the possibility it may be the best answer (e.g., some one with significant Redux experience may explain why this is a bad idea).

The Problem

We've struggled to come up with a good way to do this with Redux Toolkit. The problem seems to boil down to -- is there a good way to modularize (via separate node packages) the various "slices" used in RTK? (This may apply to other Redux implementations but we are heavily invested in RTK).

It's easy enough to have a package that exports the various items that will be used by the redux store, i.e., the slice state, action creators, async thunks, and selectors. And RTK will then compose those very nicely in the higher-level app. In other words, you can easily have an "app" package that holds the store, and then a "contacts" package that exports the "contacts" slice, with its attendant actions, thunks, selectors, etc.

The problem comes if you also want the components and hooks that use that portion of slice to live in the same package as the slice, e.g., in the "contacts" package. Those components/hooks will need access to the global dispatch and the global useSelector hook to really work, but that only exists in the "app" component, i.e., the feature that composes together the various feature packages.

Possibilities Considered

We could export the global dispatch and useSelector from the "higher" level "app" package, but then our sub-components now depend on the higher level packages. That means we can no longer build alternate higher level packages that compose different arrangements of sub packages.

We could use separate stores. This has been discussed in the past regarding Redux and has been discouraged, although there is some suggestion it might be OK if you are trying to achieve modularization. These discussions are also somewhat old.

Current thinking

In-line with Mark's suggestion on the SO question, we are thinking the best way to approach it may be to export an initialize function from each of the subprojects that the the app project calls, that basically injects the global dispatch and the useSelector functions from the global store. This would allow a kind of dependency injection, so there does not need to be a tight coupling of the parent app to sub projects. There may be some trickiness around typing but conceptually I feel like this should work. We are going to work on a proof of concept and see how it goes.

Any thoughts or suggestions welcome.

[sam-mfb]

For completeness, here's Mark's post on SO:

This question has come up in other contexts, most notably how to write selector functions that need to know where a given slice's state is attached to the root state object. Randy Coulman had an excellent and insightful series of blog posts on that topic back in 2016 and a follow-up post in 2018 that cover several related aspects - see Solving Circular Dependencies in Modular Redux for that post and links to the prior ones.

My general thought here is that you'd need to have these modules provide some method that allows injecting the root dispatch or asking the module for its provided pieces, and then wires those together at the app level. I haven't had to deal with any of this myself, but I agree it's probably one of the weaker aspects of using Redux due to the architectural aspects.

For some related prior art, you might want to look at these libraries:

https://github.com/ioof-holdings/redux-dynostore (deprecated / unmaintained, but relevant)
https://github.com/microsoft/redux-dynamic-modules (also may be unmaintained at this point - still seems to rely on React-Redux v5)
https://github.com/fostyfost/redux-eggs (brand new - the author just posted this on the RTK "Discussions" section recently)

Might also be worth filing this same question over in the RTK "Discussions" area as well so we can talk about it further.

[markerikson]

For reference I also linked the SO question on Twitter and got a couple responses, although they were more along the lines of "we code-split some reducers" rather than "publishing an entire feature as a package by itself":

https://twitter.com/acemarke/status/1472240035458170888

[sam-mfb]

Right. I should clarify for others looking at this thread that this is different (though perhaps related) to code-splitting or runtime injection. We are looking to be able to develop/test/build separately. Currently we are Ok with the app itself running as a monolith (or at least, we treat that as a separate question).

[sam-mfb]

OK, here is a proof of concept for the dependency injection model. https://github.com/sam-mfb/rtk-split

It works, but there are a couple of places where I'm hiding typescript issues. I think these issues are not fundamental, however, but just related to an issue typescript has with pnpm symlinks. More info on that at the end.

This basically just takes the default redux-typescript template from create-react-app and breaks out the "counter" feature as a separate package. The key thing that lets it work is the following bits in counterSlice.ts:

const localStore = configureStore({
  reducer: { counter: counterSlice.reducer },
});

type RootState = { counter: CounterState } & Record<string, any>;
type AppThunk<ReturnType = void> = ThunkAction<
  ReturnType,
  RootState,
  unknown,
  Action<string>
>;
type SliceDispatch = typeof localStore.dispatch;

export let useSliceSelector: TypedUseSelectorHook<
  { counter: CounterState } & Record<string, any>
> = useSelector;

export let useSliceDispatch = () =>
  useDispatch<SliceDispatch & ThunkDispatch<any, any, any>>();

export const initializeSlicePackage = (
  useAppDispatch: typeof useSliceDispatch,
  useAppSelector: typeof useSliceSelector
) => {
  useSliceDispatch = useAppDispatch;
  useSliceSelector = useAppSelector;
};

Basically I'm typing RootState (and the derivative types) as any composed state that has at least the state of the slice i'm using in the package (and same for the dispatch). Those are then used throughout this package (i.e. in Counter.tsx), but... there's also the initializeSlicePackage() that's exported that allows these "local" objects to be overridden by the global useAppDispatch and useAppSelector from the parent app. That's done by a simple call in hooks.ts in the parent app.

This all works and is basically what I was hoping to achieve. I think the typing could be improved. Currently to get the type for the local dispatch I'm actually creating it in Javascript. I feel like there should be a way to infer it entirely in Typescript without any runtime, but I haven't spent enough time there.

(The TS problem in the current version is masked by typing counterSlice as any. If you remove that assertion, TS complains but I don't think this is a fundamental problem with the inference. It's complaining about being unable to name the inferred type because of a symlinked package, which I think is a pnpm/typescript interaction issue. See, e.g., microsoft/TypeScript#42873 That's also why I have a @ts-ignore in hooks.js. I obviously need to fix this issue for this to be a viable solution but I don't think its a problem with the overall approach.)

[sam-mfb]

I fixed the TS problem (it just required an explicit import of immer so it could use WriteableDraft in the declaration).

That's updated in the repo now.

[sam-mfb]

I cleaned up the typing and added some documentation. Here's the full solution, which is also on the repo (suggestions and improvements welcome):

// RootStateInterface is defined as including at least this slice and any other slices that
// might be added by a calling package
type RootStateInterface = { counter: CounterState } & Record<string, any>;

// A version of AppThunk that uses the RootStateInterface just defined
type AppThunkInterface<ReturnType = void> = ThunkAction<
  ReturnType,
  RootStateInterface,
  unknown,
  Action<string>
>;

// A version of use selector that includes the RootStateInterface we just defined
export let useSliceSelector: TypedUseSelectorHook<RootStateInterface> =
  useSelector;

// This function would configure a "local" store if called, but currently it is
// not called, and is just used for type inference.
const configureLocalStore = () =>
  configureStore({
    reducer: { counter: counterSlice.reducer },
  });

// Infer the type of the dispatch that would be needed for a store that consisted of
// just this slice
type SliceDispatch = ReturnType<typeof configureLocalStore>["dispatch"];

// AppDispatchInterface is defined as including at least this slices "local" dispatch and
// the dispatch of any slices that might be added by the calling package.
type AppDispatchInterface = SliceDispatch & ThunkDispatch<any, any, any>;

export let useSliceDispatch = () => useDispatch<AppDispatchInterface>();

// Allows initializing of this package by a calling package with the "global"
// dispatch and selector hooks of that package, provided they satisfy this packages
// state and dispatch interfaces--which they will if the imported this package and
// used it to compose their store.
export const initializeSlicePackage = (
  useAppDispatch: typeof useSliceDispatch,
  useAppSelector: typeof useSliceSelector
) => {
  useSliceDispatch = useAppDispatch;
  useSliceSelector = useAppSelector;
};

[omdxp]

@markerikson it's how @sam-mfb explained. I just want to see how this works when you have something like this:

const initialState = documentsAdapter.getInitialState<LearnScreenState>({
  status: "idle",
  error: null,
});

where the state of the slice will contains the following properties: status, error, entities and ids.

In this example, to select the documents we can use:

export const { selectAll: selectDocuments } = documentsAdapter.getSelectors(
  (state: RootState) => state.learnScreen
);

[sam-mfb]

So if you are looking at the example above, what you have should work fine, except you would type your selector's state with RootStateInterface rather than RootState, i.e., tell it that the root state will at least include your slice state, rather than giving it the full RootState that will be composed at the parent app (and that your child package doesn't need to know about).

[omdxp]

@sam-mfb I would like to include documents state within the useSliceSelector which can be selected with selectDocuments selector. if I use useSliceSelector as mentioned above I wouldn't get all the documents but instead I'd get the entities and ids which is not ergonomic to map through the ids and return the document for each one

[sam-mfb]

Would you be able to create a fork off of this repo that demonstrates the problem you are having? I have a fair bit of experience with doing this sort of modularization with RTK, but not much experience with EntityAdapter, so I'm just having a little trouble following the problem in the abstract.

[omdxp]

Ok, will do it later

[FaberVitale]

Are you aware of the react-boilerplate approach used in react-bolierplate-cra-template?

It injects dynamically reducers and sagas.
It is based on this package: https://www.npmjs.com/package/redux-injectors

It probably deserves a closer look to see if there good ideas we could take inspiration from.

[markerikson]

I think the big difference there is that react-boilerplate is "just" separating things into different folders in one app, while the OP wants to actually maintain each feature as an entirely separate package.

[asherccohen]

Looking into his topic too.

We're modularizing our apps in a large monorepo and need to extract features in separate libraries.

We've currently tried all the packages and all have their trade offs.

In the near future we'll also explore module-federation, so being able to import/inject slices and APIs even at runtime is fundamental.

I'll keep updating this thread with our findings.

[sam-mfb]

BTW, in the year since I made this post, we have fully modularized our application's features using this strategy. Each feature it its own npm package and has its own redux slice that is entirely internal to it, and all the features get composed in a parent app using the dependency injection method explained here. This does not, however, happen at runtime--it's all part of the build process.

[umashankar-pardhi-amla]

@sam-mfb : Can you please create the Repo and share here this will help us lot

[sam-mfb]

The repo is the same one we posted above. Here it is again. My only point in this post was that we have extended this strategy to our working production code (which is proprietary). But the basic approach and strategy is the same.

If there's interest, I can go through it and see if there are any minor tweaks to add based on our production code, but I'm a bit underwater right now and won't have time for a while. In the meantime though, if you have specific questions about trying out this approach feel free to post and I will try to answer. We have really liked having this modularity.

[upardhi]

What I am trying to achieve is I can register my slice/reducer to their only package, and also we can access the counter in multiplier directly through the store.

I have multiple A, B, C, D ..... Z features and some customers are using A feature only, Some Are using B and C only so I want to make it modular so that I can not mix the dependencies at the time of application load

I do not want to register all slices initially .. Thoughts?

[sam-mfb]

This functionality is actually there in the example app, but it is hidden (intentionally) by the typing.

The useAppSelector and useAppDispatch hooks injected into Multiplier actually can access the full store as a matter of javascript. That is, they are just the useSelector and useDispatch hooks generated by react-redux--we haven't altered them in any way. The only reason they appear not to be able to is because of the narrow Typescript typings we put on them inside the multiplier package to prevent devs from accidentally going outside of feature scope. I.e., we explicitly do not want our dev accessing one feature from inside another.

Since you want to be able to go out of feature scope and access the counter store directly, you need to broaden those typings.

Making the broader types is a little tricky as you can't just use these lines like we do in the Main App:

export type AppDispatch = typeof store.dispatch;
export type RootState = ReturnType<typeof store.getState>;

Because your Multiplier app won't "know" about the shape of the Counter state and there isn't any way to "inject" types down from the Main App package into the Multiplier packages. (Since injection is a runtime concept, and typescript is compile time.). Off the top of my head, the way I would handle that would be to make type-only package, let's call it "counter-types" that will be imported as dev-dependency by "Multiplier" and will be used to get the shape of the counter store and selectors for purposes of constructing the store and selector types inside multiplier. The idea being you want to give Multiplier access to Counter store's types without giving it access to its code. You'll then have to compose the AppDispatch and RootState types manually inside your Multiplier package but it shouldn't be overly difficult.

All that said, I would humbly suggest, based on our experience, that you will find the structure of our repo much easier to maintain long term than the one you are proposing. The "Multiplier" package you are proposing will be end up leading to the same kind of maintenance headaches that this modularity approach we are proposing is meant to avoid. I.e., every time you change some selector or reducer interface in Counter you have to go into Multiplier and change code there as well. Multiply that by x number of features and you are back in feature dependency hell :). If you want to be able to provide some users with feature A and some users with features A B, I would just construct two separate Main Apps. The beauty of this solution is that the Main Apps end up being super simple to write since all they are doing is composing together multiple features. In fact, in our repo, we always create little preview Main Apps that only use one feature for easy development and testing. That's just my 2 cents.