Transforomer use

[Vishwas-verma]

Hye geeks does somebody know how to use transformer in adonisJS.I have used bumblebee transformer for adonisJS but they somehow are not fully compatible with adonisJS.

Any alternative idea

[aayush123]

The maintainer of adonis-bumblebee did not have any immediate plans to port the package for V5 when he last commented on an issue raised by me.
So I ended up creating an implementation of my own in my project.
I guess the best route right now is to create a custom implementation using serializers provided in the framework (that's what I did) 🤷‍♂️

[Vishwas-verma]

Hey @aayush123 can uh share the approach you have used for transformers.

[aayush123]

As compared to the bumblebee package, which loads the transform method into the HttpContext, I opted for a singleton approach instead.

In my project, I created a base Transformer abstract class which is inherited by individual transformers pertaining to models.

The base class defines public methods such as TransformItem, TransformCollection (and whatever you may need for your use-case) etc.
The inherited Transformers, the ones that implement this base transformer (eg. UserTransformer) just need to define, at minimum, a protected transform method, which basically defines how to convert data fetched from DB (or in my case, serialized data) into whichever shape you may want.
This transform method is internally called by the public methods TransformItem & TransformCollection to transform data retrieved from DB.

The reason I used the singleton approach is that I knew I would need transformed data outside of controllers too (eg. saving snapshots of current user data while creating an order). So I thought it best to reuse the same logic by using singletons.

However, I'm sure if required, it would be pretty straightforward to extend the same thing with some modifications and load a transform property into the HttpContext.

I plan to convert this into a proper package (including loading the transform property into HttpContext) once there is more documentation on how to create packages for Adonis V5, and more importantly, once I have more time on my hands.

Meanwhile, here's the code from the base Transformer.ts class:

import { LucidRow, ModelPaginatorContract } from '@ioc:Adonis/Lucid/Model';
import { RequestContract } from '@ioc:Adonis/Core/Request';
import { DateTime } from 'luxon';
import { pascalCase } from 'change-case';
import { ModelPaginator } from '@ioc:Adonis/Lucid/Orm';

export type IncludeMethodOptions = {
  nestedIncludes: string;
};

export default abstract class Transformer<
  Model extends LucidRow = LucidRow,
  TransformedEntity extends CustomTypes.GenericObject = any
> {
  /**
   * Transform serializedData into object
   * @param  serializedData serializedData to be transformed
   * @param  item           LucidRow instance being transformed
   * @return                Promise that resolves to JSON with transformed data
   */
  protected abstract transform(serializedData: any, item?: LucidRow): Promise<object>;
  protected defaultIncludes: string[] = [];
  protected availableIncludes: string[] = [];

  /**
   * Method to transform a single Model Object
   * @param  item                    Object to be transformed
   * @param  options                 Object of options. Can include transformMethod, include, exclude & request keys.
   * @param  options.transformMethod Method to be used to transform item. Must be defined in model specific transformer. If not given, 'transform' will be used
   * @param  options.include         Comma-separated string of relationships to be included.
   * @param  options.exlcude         Comma-separated string of relationships to be excluded.
   * @param  options.request         Request context. If provided, include & exclude inputs will be parsed to dynamically include & exclude relationships
   * @return                         Promise that resolves to transformed JSON.
   */
  public async transformItem(
    item: LucidRow | null,
    options?: TransformItemOptions<true>
  ): Promise<TransformedEntity>;
  public async transformItem(
    item: LucidRow | null,
    options: TransformItemOptions<false>
  ): Promise<TransformedEntity | null>;
  public async transformItem(
    item: LucidRow | null,
    options: TransformItemOptions<boolean> = {}
  ): Promise<TransformedEntity | null> {
    const {
      transformMethod = 'transform',
      include = '',
      exclude = '',
      request = undefined,
    } = options;
    if (!item) {
      if (options.throwError === false) {
        return null;
      }
      throw new Error('Model not passed to transformer');
    }
    const serializedData = item.serialize();
    const inclusionList = this.parseInclusions(include, exclude, request);
    const inclusions = {};
    await Promise.all(
      inclusionList.map(async (eachInclusion) => {
        const nestedIncludeArray = eachInclusion.split('.');
        const topInclusion = nestedIncludeArray[0];
        const methodName = `include${pascalCase(topInclusion)}`;
        nestedIncludeArray.splice(0, 1);
        const inclusionData = await this[methodName](item, {
          nestedIncludes: nestedIncludeArray.length > 0 ? nestedIncludeArray.join('.') : undefined,
        });
        inclusions[topInclusion] = inclusionData;
      })
    );
    const transformedItem = await this[transformMethod](serializedData, item);
    return {
      ...transformedItem,
      ...inclusions,
    };
  }

  /**
   * Method to transform collection.
   * Method checks if collection is a paginated contract or not and calls appropriate method.
   * Will call transformItem on every Model.
   * @param  collection              Array of models
   * @param  options                 Object of options. Can include transformMethod, include, exclude & request keys.
   * @param  options.transformMethod Method to be used to transform item. Must be defined in model specific transformer. If not given, 'transform' will be used
   * @param  options.include         Comma-separated string of relationships to be included.
   * @param  options.exlcude         Comma-separated string of relationships to be excluded.
   * @param  options.request         Request context. If provided, include & exclude inputs will be parsed to dynamically include & exclude relationships
   * @return                         Promise that resolves to transformed JSON.
   */
  public async transformCollection(
    collection: Array<Model> | undefined,
    options?: TransformCollectionOptions
  ): Promise<any[]>;
  public async transformCollection(
    collection: ModelPaginatorContract<Model> | undefined,
    options?: TransformCollectionOptions
  ): Promise<CustomTypes.TransformedDataWithMetadata<CustomTypes.PaginationMetadata>>;
  public async transformCollection(
    collection: Array<Model> | ModelPaginatorContract<Model> | undefined,
    options: TransformCollectionOptions = {}
  ) {
    if (!collection) {
      throw new Error('Collection not passed to transformer');
    }
    if (collection instanceof ModelPaginator) {
      return this.transformCollectionWithPagination(collection, options);
    }
    return this.transformCollectionWithoutPagination(collection, options);
  }

  /**
   * Method to transform collection. Will call transformItem on every Model.
   * @param  collection              Array of models
   * @param  options                 Object of options. Can include transformMethod, include, exclude & request keys.
   * @param  options.transformMethod Method to be used to transform item. Must be defined in model specific transformer. If not given, 'transform' will be used
   * @param  options.include         Comma-separated string of relationships to be included.
   * @param  options.exlcude         Comma-separated string of relationships to be excluded.
   * @param  options.request         Request context. If provided, include & exclude inputs will be parsed to dynamically include & exclude relationships
   * @return                         Promise that resolves to transformed JSON.
   */
  protected async transformCollectionWithoutPagination(
    collection: Array<LucidRow> | undefined,
    options: TransformCollectionOptions = {}
  ): Promise<any[]> {
    if (!collection) {
      throw new Error('Collection not passed to transformer');
    }
    const transformedItems = await Promise.all(
      collection.map(async (eachItem) => await this.transformItem(eachItem, options))
    );
    return transformedItems;
  }

  /**
   * Method to transform collection with pagination. Will call transformItem on every Model.
   * @param  paginator               Instance of paginator
   * @param  options                 Object of options. Can include transformMethod, include, exclude & request keys.
   * @param  options.transformMethod Method to be used to transform item. Must be defined in model specific transformer. If not given, 'transform' will be used
   * @param  options.include         Comma-separated string of relationships to be included.
   * @param  options.exlcude         Comma-separated string of relationships to be excluded.
   * @param  options.request         Request context. If provided, include & exclude inputs will be parsed to dynamically include & exclude relationships
   * @return                         Promise that resolves to transformed JSON.
   */
  protected async transformCollectionWithPagination(
    paginator: ModelPaginatorContract<any> | undefined,
    options: TransformCollectionOptions = {}
  ): Promise<CustomTypes.TransformedDataWithMetadata<CustomTypes.PaginationMetadata>> {
    if (!paginator) {
      throw new Error('Paginator not passed to transformer');
    }
    const transformedItems = await Promise.all(
      paginator.all().map(async (eachItem) => await this.transformItem(eachItem, options))
    );
    return {
      data: transformedItems,
      meta: paginator.getMeta(),
    };
  }

  /**
   * Method to get transformed time stamps from model.
   * @param   {object}  serializedData  Object containing the timestamps retrieved from serialized data
   * @return  {object}                  Object containing transformed timestamps
   */
  protected getTransformedTimeStamps(serializedData: {
    created_at: string;
    updated_at: string;
    deleted_at?: string;
  }): object {
    return {
      created_at: DateTime.fromISO(serializedData.created_at)
        .startOf('second')
        .toISO({ suppressMilliseconds: true }),
      updated_at: DateTime.fromISO(serializedData.updated_at)
        .startOf('second')
        .toISO({ suppressMilliseconds: true }),
      ...(serializedData.deleted_at
        ? {
            deleted_at: DateTime.fromISO(serializedData.deleted_at).startOf('second').toISO({
              suppressMilliseconds: true,
            }),
          }
        : {}),
    };
  }

  /**
   * Method to parse includes & excludes.
   * @param  include    comma-separated string of relationships to be included
   * @param  exclude    comma-separated string of relationships to be excluded
   * @param  requestCtx Request context. If provided, will be used to dynamically parse and include / exclude relationships passed in request inputs.
   * @return            Array of relationships to finally be included
   */
  protected parseInclusions(
    include: string = '',
    exclude: string = '',
    requestCtx?: RequestContract
  ): Array<string> {
    const dynamicExcludes: string = requestCtx?.only(['exclude']).exclude || '';
    const dynamicIncludes: string = requestCtx?.only(['include']).include || '';
    const exclusionSet = new Set([...exclude.split(','), ...dynamicExcludes.split(',')]);
    const parsedInclusions = new Set([
      ...(include ? include.split(',') : []),
      ...dynamicIncludes.split(','),
      ...this.defaultIncludes,
    ]);
    for (let elem of exclusionSet) {
      parsedInclusions.delete(elem);
    }
    const possibleIncludes = new Set([...this.availableIncludes, ...this.defaultIncludes]);
    for (let elem of parsedInclusions) {
      if (!possibleIncludes.has(elem.split('.')[0])) {
        parsedInclusions.delete(elem);
      }
    }
    return [...parsedInclusions];
  }
}

type TransformCollectionOptions = {
  transformMethod?: string;
  include?: string;
  exclude?: string;
  request?: RequestContract;
};

type TransformItemOptions<ThrowError extends boolean> = {
  transformMethod?: string;
  include?: string;
  exclude?: string;
  request?: RequestContract;
  throwError?: ThrowError;
};

Here is a sample CategoryTransformer.ts that extends this class:

import Category from 'App/Models/Category';
import Categories from 'App/Repositories/CategoryRepository';
import Transformer from 'App/Transformers/Transformer';
import ProductTransfomer from 'App/Transformers/ProductTransformer';

class CategoryTransformer extends Transformer {
  constructor() {
    super();
  }

  protected defaultIncludes = [];
  protected availableIncludes = ['products'];

  protected async transform(serializedData: any) {
    return {
      id: serializedData.id,
      name: serializedData.name,
      slug: serializedData.slug,
      ...this.getTransformedTimeStamps(serializedData),
    };
  }

  protected async includeProducts(item: Category) {
    const products = await Categories.fetchRelated(item, 'products');
    return await ProductTransfomer.transformCollection(products);
  }
}

export default new CategoryTransformer();

Notice the export of a singleton from this class. This can be imported anywhere in your codebase and be used to transform data in a similar manner across your project.
It also has an include method for transforming relationships.

Here is the transformer being used in a controller:

...
  response.success(await CategoryTransformer.transformItem(category, { request }));
...

The request context is passed in the second argument to cater to dynamic include and exclude requests. Passing include=products in the query string or request body will fetch a category, along with products associated with the category.
Other options in the second argument are

• transformMethod: uses a method other than transform from CategoryTransformer, the method must also be defined
• include: to include relationships programmatically.
• exclude: to exclude relationships programmatically.

Hope this helps :)

[Vishwas-verma]

Hey @aayush123 your approach seems to be satisfied and working but there are few updates in adonis library as for now there is no ModelPaginator in '@IOC:Adonis/Lucid/Orm' also the ModelPaginatorContract don't seems to exist to in '@IOC:Adonis/Lucid/Model'.

Instead there is SimplePaginator and SimplePaginatorContract.

[aayush123]

Iirc, I had to change from simple paginator to model paginator once I upgraded to the stable release from preview.
Are you on preview or the latest release?

[Vishwas-verma]

Not in the team but I am following adonisJS since long.
What code structure do you use repository based or MVC - service based.If repository based structure, is your any work on github following repository architecture in AdonisJS.

[aayush123]

I use a mix of both, unfortunately, nothing that I can upload on GitHub publically.

[Vishwas-verma]

@aayush123 Basically my purpose of asking that why you are using repositorey architecture it only adds a extra level of abstraction?If using ORM you dont need it.
It's basically used when you have more than one data souce like if its storing in frontend and api.
Also its suitable for basically .net projects.

[aayush123]

It's basically used when you have more than one data souce like if its storing in frontend and api.
Also its suitable for basically .net projects.

As far as I know, Repository based architecture is a design pattern. Not something related to, or suitable for a particular technology.

Basically my purpose of asking that why you are using repositorey architecture it only adds a extra level of abstraction?If using ORM you dont need it.

True, that it only adds an extra level of abstraction, but that's exactly what I'm trying to achieve.
I use repositories as a seam in my codebase, a separation between my services and the ORM.

Say you have services that regularly use aggregates such as count or max.
It will still be fine if you use ORM methods directly in your services, but that is code that you don't have control over.
Say with the latest update, the ORM API changes and you have to use a slightly different signature for getting count than what was previously being used (this is not that uncommon either).

Now you have to find all the services which use the count method & update them accordingly.
If you use a repository and had a max function defined in it, which was, in turn, being used by all the services, you just have to make a change in the Base Repository and services are left untouched.

This way, you create a seam in between your data accessing logic and the data consuming logic and protect the data consumers from API changes in data accessors as you are in control of the code that goes in repositories.

Repositories also allow me to create a Base abstract repository, like the transformer I posted above and then derive repositories for models using that.
This way, I can ensure all repositories have some common methods and are forced to enforce some methods, for eg. a search method.

I believe, it eventually depends on the scale of your project and how decoupled do you want your code to be from its dependencies.

PS: There is also another huge benefit, albeit one that you hope you never have to reap, that in case you change your DB, or ORM, you just have to update your Repository methods, not all the code that is consuming it.
All you need to do is ensure that you are taking the same inputs and giving the same output from repositories and your effort in migration gets reduced significantly.

[Vishwas-verma]

You have a legitmate
point I agree what I was trying to say that this type of design pattern is mainly used in .net as there the data source changes frequently.
Althought Repository have its own benefit but only in extreme rare cases,as hardly uh have heard that Database or ORM has been changed in project.
Also the function signature too will not change if something out of the box doesn't come up.
Just for being in safer side for the cases that are not really going to come in sdlc , extra abstraction layer doesnt worth it , it impacts on server response time on large scale applications.
My sole purpose of this talk with you was that 99 % of projects/software dont need repository pattern but if you are handling 1% project than its of great use.

[aayush123]

Like I said, it's a design pattern, a way of writing code.
At the end of the day, it's the developer's choice whether to use it or not.