RFC - End-to-End type safety

[thetutlage]

This RFC covers the end-to-end type-safety of an AdonisJS application when using Inertia or creating a separate frontend within a monorepo.

End-to-end type safety refers to having a single source of truth for your types across the backend and the frontend.

Whenever your frontend application interacts with the backend, such as sending API requests, consuming responses, and handling errors, it should have type safety around all these operations.

Similarly, if your backend can interact with the frontend directly, like rendering Inertia pages, then this operation should also be type-safe.

Before we start

Let's start by defining the scope of this project.

• The generated types rely on import references and will work when both the frontend and the backend are part of the same codebase.

• We will not parse the entire codebase into an Abstract Syntax Tree, since doing that on every code change is expensive and will worsen the DX with changes propagating slowly. Instead, we will use AST-GREP to scan the codebase based on certain conventions.

• The generated types or runtime values will be written within the .adonisjs directory within your project root. The types used by the frontend will go inside the .adonisjs/client directory, and the types used by the server will go inside the .adonisjs/server directory.

• We will not change how you write your AdonisJS applications, since we deeply care about the abstractions we have created over the years and want to continue using them.

• The framework core will expose the APIs for scanning the codebase and extracting metadata from it, but it won't generate any types or runtime code.

• Tuyau will use these low-level APIs and will bring e2e type-safety to your apps.

• The same set of APIs used by Tuyau can be used by other 3rd party packages to add additional features that rely on a deeper understanding of the codebase.

Rendering Inertia pages

One of the core goals of end-to-end type safety is to ensure that the data passed from the backend to the frontend is fully typed and validated, even when using server-driven rendering with Inertia.

In AdonisJS, the first thing you typically do in an Inertia app is render pages written using a frontend framework like Vue or React. Each page receives a set of props from the backend, and defining the types for those props plays a central role in maintaining type safety.

In the following example, we define the PostsIndex page that accepts an array of Post.

Note

In this example, the Post type is defined manually within the frontend component. This is a common approach today, though we'll explore ways to reduce duplication and better align types between frontend and backend in the sections that follow.

type Post = {
  id: number
  slug: string
  title: string
  excerpt: string
  commentsCount: number
}

export default function PostsIndex(props: Post[]) {
  return <>
    // Render posts here
  </>
}

Just like functions in TypeScript, a component should be free to define the types for its parameters, and it should be the job of the caller to provide the correct data at the time of rendering the component.

In our case, the caller is the inertia.render method.

Therefore, the first thing we have to do is make the inertia.render method call type-safe. This will be done by inspecting all the components within the inertia/pages directory and creating a centralized types file for it.

import type PostsIndex from '../../inertia/pages/posts/index.tsx'

declare module '@adonisjs/interia' {
  export interface InertiaPages {
    'posts/index': PageProps<PostsIndex>
  }
}

Once we have the type information within the InertiaPages interface, we can make the inertia.render method rely on it.

function render<Page extends keyof InertiaPages>(
  page: Page,
  props: InertiaProps<InertiaPages[Page]>
) {
}

Inertia page props

While defining prop types manually works, it introduces duplication between the controller and the frontend component. Ideally, the frontend should not have to redefine the shape of data that already exists on the backend.

To address this, AdonisJS offers the InferPageProps helper, which allows you to infer the props for a page directly from the return type of a controller method. However, this approach comes with some caveats.

import type { InferPageProps } from '@adonisjs/inertia/types'
import type PostsController from '../../../app/controllers/posts_controller'

export function PostsIndex(
  props: InferPageProps<PostsController, 'index'>
) {
  props.posts
}

1. It tightly couples a page with a controller. What if two controllers are rendering the same page, but with different props? Which controller will you use for inferring the props?.
2. It gives the illusion of a page having a relationship with the controller. We assume that the PostsController.index method is rendering the posts/index.tsx page, and therefore its props can be used as an input. But, there is no system enforcing it.
3. Not directly related to InferPageProps, but serializing rich data-types (like models) from the backend never leads to accurate types, hence a lot of manual type-casting is required.
4. How about re-usable micro components like a User or Product card? These components also rely on the backend data. However, inferring their props from a controller return type feels weird 😬.

To solve these issues, we first have to decouple Inertia pages from a controller (or at least remove that illusion), because in reality, a page could be rendered from multiple controllers.

We will resume this discussion after talking about HTTP transformers.

Introducing HTTP transformers

To precisely know what types your backend will return, we need an additional layer of serialization that will convert rich data types like classes to JSON data types with precise type information.

Whenever the data is flowing between two systems, having great control and visibility of the data becomes super important. This is where we want you to use HTTP transformers for all Inertia responses.

Note

I will be using HTTP transformers with Lucid models. However, the transformers are not coupled to models and can be used with plain objects or any other JavaScript data types.

Creating transformers

Let's take an example of rendering the PostsIndex page. In this case, we must create a transformer for the Post model.

We have intentionally kept the API of transformers simple and not used any declarative API like DTOs. The imperative API of transformers allows you to prepare data at runtime with complete freedom of writing conditionals, running loops, injecting dependencies, and so on.

import { Post } from '#models/post'
import { BaseTransformer } from '@adonisjs/core/http'

export class PostTransformer extends BaseTransformer<Post> {
  async toObject() {
    return {
      id: this.resource.id,
      slug: this.resource.slug,
      title: this.resource.title,
      excerpt: this.resource.excerpt,
      commentsCount: this.withCount('comments')
    }
  }
}

Using transformers

Now, within the controller, we can use this transformer to transform a collection of Post model instances.

import Post from '#models/post'
import { HttpContext } from '@adonisjs/core/http'

export default class PostsController {
  async index({ inertia, auth, transform }: HttpContext) {
    const posts = await Post.all()

    return interia.render('posts/index', {
      posts: transform(posts, PostTransformer),
      user: transform(auth.user, UserTransformer)
    })
  }
}

Accessing transformer types within the frontend codebase

We will generate types by scanning for all the transformers within your codebase and make them available as Data objects within the frontend codebase.

You can update the PostsIndex page to use the Post data-type as follows.

Tip

All the non-page components in the frontend codebase can also rely on the Data objects for type information.

import Data from '~/data'

export function PostsIndex(
  props: { posts: Data.Post[] }
) {
  props.posts
}

Completing the full circle

Let's re-inspect the type-safety of rendering Inertia pages with this new approach.

• The frontend codebase can rely on Data objects for its types without typing them manually.
• The pages are decoupled from controllers. Multiple controllers can render the same page (as long as they provide the needed props).
• The interia.render calls are type-safe. Hence, the TypeScript compiler will scream if you provide the wrong data.

Inertia forms

Let's move to forms and see how we can make form submissions type-safe. Here, we would want the list of available routes and the data they accept.

Generating types

This is done by scanning for all the routes that are using a controller for handling requests. Additionally, the controller's validators and return types are collected.

The following is a rough representation of a scanned route.

{
  pattern: '/users',
  name: 'users.store',
  method: 'post',
  validation_schema: InferValidationSchema<typeof import('#validators/users'), 'createUserValidator'>
  response: InferControllerResponse<typeof import('#controllers/users_controller'), 'store'>
}

Since our scanners won't walk through the entire codebase, you will have to define the HTTP boundaries within the controller method itself.

The boundaries here are the validator usage and the controller's return value.

import { HttpContext } from '@adonisjs/core/http'

export class UsersController {
  async store({ request, response }: HttpContext) {
    /**
     * ✅ The validation call has to be within the controller method
     */
    await request.validateUsing(someValidator)

    /**
     * Controller must explicitly return a response
     */
    return response.redirect().back()
  }
}

The useForm helper

Once we have all the routes and their validation schemas, we can use the useForm helper from Tuyau that will rely on the generated types to provide the needed type-safety

import { useForm } from '@tuyau/inertia/react'

const form = useForm('users.store', {
  fullName: '',
  email: '',
  password: '',
})

form.submit()
form.errors
form.processing

Forms response type-safety

Since form submissions in Inertia always lead to server-side redirects, there is no need to manually process the response. Hence, form response type-safety is not even a concern with Inertia applications.

Link component type-safety

The Link component exported by Tuyau is type-safe. You may specify the routes and other attributes as follows.

import { Link } from '@tuyau/inertia/react'

<Link
  route="users.show"
  params={['1']}
  only={['profile']}
/>

Router visits

You may use the useRouter import from Tuyau to make the router.visit method calls type-safe.

import { useRouter } from '@tuyau/inertia/react'

const router = useRouter()

router.visit('users.show', [id], {
  only: [] // type-safe
  except: [] // type-safe
})

Type-safe API client

Along with the Inertia helpers, Tuyau also comes with a type-safe API client for making fetch calls to your backend.

The API client is helpful for applications with a separate frontend. However, it can also be used with Inertia to make one-off API calls for advanced use cases.

For those who already know Tuyau, the philosophy remains the same, but we are making a few breaking changes:

• The biggest one is that we will revamp the chainable API to use route names for chaining, instead of the route URI.

• Currently, Tuyau does not throw errors for 4xx/5xx status codes, unless you chain the $unwrap method. We will be changing this behavior, and all non-success status codes will throw an Error.

• We are removing the $ prefixes for special methods. Instead, we will mark those keywords as reserved, and they cannot be used to name routes on the backend.

Chainable API

The new chainable API will be built around the route names and controllers rather than URLs. Here's a quick difference between the two.

Route definition

router
  .post('posts/:id/comments', [PostCommentsController, 'store'])
  .as('posts.comments.store')

Existing API
The existing API uses the route URI as the properties for chaining, and you will use the $post, $get, or $put methods to make an API call for a given specific HTTP method.

Also, you pass the route params for each segment independently by calling intermediate methods.

import { client } from './tuyau'

await client.posts({ id: 1 }).comments.$post({
  data: {}
})

/**
 * /posts/:id/comments/:comment_id
 * If the route needs two params, this is how you will pass
 * them.
 */
await client.posts({ id: 1 }).comments({ commentId: 1 }).$get()

The new proposed API
The new proposed API is designed around the route names. You do not have to think in URIs or the HTTP methods anymore. Just use route names everywhere.

import { client } from './tuyau'

await client.posts.comments.store({ id: 1 }, {
  data: {}
})

/**
 * /posts/:id/comments/:comment_id
 * Pass all params to the final method call
 */
await client.posts.comments.index({
  id: 1,
  commentId: 1
})

How about unnamed routes?
For unnamed routes, we will create the chainable API using the controller + method combination. For example:

class UsersController {
  async show({ params }) {
  }
}
router.get('/users/:id', [UsersController, 'show'])

const result = await client.users.show({
  id: 1
})

The request method

If you are not a big fan of the chainable API, then you may use the client.request method with the route name to make an API request.

// Pass params and additional options
client.request('users.show', { id: 1 }, {
  query: { include: 'profile' }
})

// No params are accepted
client.request('users.store', {
  body: { fullName: 'John Doe' }
})

// Pass params as an array
client.request('posts.comments.index', [1])

Error handling

Currently, Tuyau does not throw an error for 4xx/5xx status codes unless you chain the .unwrap() method.

However, we noticed that most users always use unwrap; therefore, we have decided to always throw an error for non-success status codes.

Existing behavior

const { data, error } = await client.users.$get()
const data = await client.users.$get().unwrap()

New proposed behavior

const data = await client.users.index()

const { data, error } = await safe(client.users.index())

Making requests using URLs

Finally, we will expose client.get/post/put/delete methods to make fetch calls directly using URIs. While we do not recommend this, we still want to offer an API that is independent of naming routes and following certain conventions.

const result = await client.get('/users/:id', {
  id: 1
}, {
  query: { include: 'profile' }
  headers: { 'X-Custom-Header': 'value' }
})

URL builder

Alongside the API client, you will also have access to a URL builder that you may use to create URIs for known routes.

import { urlFor } from './tuyau'

urlFor('users.store')
// OUTPUT - /users

urlFor.post('users.store')
// OUTPUT - { method: 'POST', url: '/users' }

urlFor.get('posts.comments.index', ['1'])
// OUTPUT - { method: 'GET', url: '/posts/1/comments' }

Request and response types

Currently, to get the types for the request and response returned by an endpoint, you have to use the InferResponseType and InferRequestType helpers.

However, we will be providing direct access to the types of endpoints either via the route names or the route paths.

import type { Routes, Paths } from '~/generated/routes'

type UserStoreRequest = Routes['users.store']['Request']
type UserStoreResponse = Routes['users.show']['Response']
type UserStoreParams = Routes['users.show']['Params']

Paths['/users/:id']['Request']
Paths['/users/:id']['Response']
Paths['/users/:id']['Params']

Checking the current route

Inside the Inertia codebase, it could be handy to know if we are on a specific route or not. A typical use case is to highlight the active link in a navigation menu.

import { client } from '~/tuyau'

client.current('users.posts.show', { id: 1, postId: 2 }) // true
client.current('users.posts.show', { id: 12 }) // false
client.current('users.posts.show', {}) // false

Tanstack Query integration

Tuyau will also provide Tanstack query integration based on the Query Options API.

// lib/tuyau.ts
import { createTuyau } from '@tuyau/client'
import { QueryClient } from '@tanstack/react-query'
import { createTuyauReactQueryClient } from '@tuyau/react-query'

export const queryClient = new QueryClient()
export const client = createTuyau({ baseUrl: `http://localhost:3333` })
export const queryClient = createTuyauReactQueryClient({ client, queryClient })

export const App = () => {
  const { data, error, isLoading } = useQuery(
    queryClient.users.show({ userId: 5 })
  )

  if (!data || isLoading) {
    return 'Loading...'
  }

  if (error) {
    return `An error occured: ${error.message}`
  }

  return <div>{data.firstname}</div>
};

For every nesting level of your route names, you will be able to access pathKey(), which returns a query key that allows you to invalidate all queries under a specific route "namespace".

// Let's say you have the following routes
'users.posts.show' // GET /users/:userId/posts/:postId
'users.posts.index' // GET /users/:userId/posts
'users.comments.index' // GET /users/:userId/comments

// Invalidate all cached data for routes starting with "users"
// This will invalidate users.posts.show, users.posts.index, users.comments.index, etc.
queryClient.invalidateQueries({ queryKey: client.users.pathKey() })

// Invalidate all cached data for routes starting with "users.posts"
// This will only invalidate users.posts.show and users.posts.index
queryClient.invalidateQueries({ queryKey: client.users.posts.pathKey() })

Same, you will be able to access every queryKey for a route name using the queryKey() method.

client.users.posts.show({ userId: 5, postId: 10 }).queryKey()
// OUTPUT - ['users', 'posts', 'show', { path: { userId: 5, postId: 10 } }]

You will also be able to make calls using URLs.

await client.queryOptions('get', '/users/:id', { id: 1 }, {
  query: { include: 'profile' }
})

A few notes about the API client and the URL builder

• The API client and the URL builder only consider routes using an explicit name or a controller.
• The client.request/get/post/put/delete methods are reserved keywords. Therefore, you cannot name your routes to start with request, get, post, put, or delete.
• The route params are accepted as camelCase. For example, /posts/:post_id param will be accepted as postId.

[Julien-R44]

Quick update regarding request and response types:

import type { Routes, Paths } from '~/generated/routes'

Routes['users.store']['Request']
Routes['users.show']['Response']
Routes['users.show']['Params']

Paths['/users/:id']['Request']
Paths['/users/:id']['Response']
Paths['/users/:id']['Params']

The issue with using Paths like this is that /users/:id can match multiple endpoints (depending on the HTTP method). So we’d need something more specific like:

Paths['/users/:id']['GET']['Params']

But that starts to feel a bit clunky, too many index levels (subjectively). Thats why we are suggesting to go with a namespace + generics approach instead:

type UserShowRequest = Path.Request<'GET', '/users/:id'>

Feels a bit cleaner. Of course, subjective, so open to feedback.
And for consistency, we will follow the same pattern for Routes:

type UserStoreRequest = Route.Request<'users.store'>

So the final proposed API would look like this:

// Named routes
type UserStoreRequest = Route.Request<'users.store'>
type UserStoreResponse = Route.Response<'users.store'>
type UserStoreParams = Route.Params<'users.store'>
type UserStoreErrors = Route.Errors<'users.store'>

// Raw URLs
type UserShowRequest = Path.Request<'GET', '/users/:id'>
type UserShowResponse = Path.Response<'GET', '/users/:id'>
type UserShowParams = Path.Params<'GET', '/users/:id'>
type UserShowErrors = Path.Errors<'GET', '/users/:id'>

[arnauddoub]

Will ~/generated/routes be generated by Adonis directly, or do we need to use Tuyau for that?

Also, not sure if this is the right place to bring it up, but currently the preload in Lucid isn't typed.
Do you know if there are any plans to improve that?

Thanks for the update, it looks good!

[Julien-R44]

Take a quick look at the "Before we start" section in the RFC. TLDR: AdonisJS core will provide some built-in helpers to scan the codebase, and Tuyau will rely on those to generate the Route and Path typings.
Also Tuyau will be included by default in AdonisJS apps starting from v7.

Regarding preload and Lucid, yeah I think thats a separate topic. Adding some extra type safety to Lucid would probably deserve its own dedicated RFC 😅

[DavideCarvalho]

I don't know if I didn't understand, so feel free to correct me

From the RFC, we can understand that multiple controllers can render the same page, and that's correct

But when I see

return interia.render('posts/index', {
      posts: transform(posts, PostTransformer),
      user: transform(auth.user, UserTransformer)
    })

and my Page looks like this:

import Data from '~/data'

export function PostsIndex(
  props: { posts: Data.Post[] }
) {
  props.posts
}

Well, we're sending more data than the page needs

But if we had something like this

return interia.render('posts/index', {
      user: transform(auth.user, UserTransformer)
    })

This should indeed have a type error since we're not sending posts.

My train of thought here is the frontend is the one that knows what the page needs to correctly render, so if the backend doesn't send all of the data, it should error out

Is this what you are thinking as well?

[Julien-R44]

Yup, inertia.render will be fully type-safe. So if you forget required props or pass unneeded ones, Typescript will throw an error

[DavideCarvalho]

Got it ty!

[diego-sepulveda-lavin]

This sound really nice specially for Inertia and Tanstack query users, and will definitely improve DX ❤️.

My only question is more related to Tuyau itself since is tightly coupled to ky.
How compatible is it with NextJS? (many of us are using the fetch wrapper included in next since it provides some caching/revalidating features).

Moving to tuyau would be so convenient but losing control on the fetcher itself can be a hard decision to make. Maybe just using the API route definitions and the response/request types could be a intermediate solution for us.

[Julien-R44]

I am not super familiar with Next, but there shouldnt be any issues : we already have users running the current version of Tuyau with Next.js without any problems (even have a dedicated guide ) Also, since ky uses fetch under the hood, I dont expect any conflicts.

But yeah feel free to share if you foresee any potential issues we could have with Next.js. happy to address them!

[diego-sepulveda-lavin]

Next.js implements a wrapper around the native fetch for server-side (here you can check the docs), which has all the core features of fetch, but also adds some tools for caching, for example:

1. fetch(https://my-api.com/users, { next: { revalidate: 60 } }) // this enables caching users for one minute, so any request inside that time frame will be deduplicated to avoid redundant requests

2. fetch(https://my-api.com/users, { cache: 'force-cache' }) // will enforce caching on that endpoint until you decide to revalidate

So as you said, there are probably no conflicts using Tuyau with Next.js, but the trade-off would be losing these specific fetch features. I imagine that a middle-ground solution for Next.js devs is to only use Tuyau to generate the backend routes and the input/output types, while keeping the use of Next.js fetch (instead of the built-in Ky)

[Julien-R44]

Thanks! According to this issue, we shouldnt run into any problems: sindresorhus/ky#531

Tuyau accepts any options that Ky accepts, and Ky also forwards those options to plain fetch. We might double check that, but we shouldnt have any issues using Ky. I mean you should be able to pass Next specific options

[diego-sepulveda-lavin]

Oh I wasn't aware of that, thanks for pointing it out! I'm pretty excited now! 🤩