500 Number of resources limit and 1000000 Template size limit

[MarlonJD]

How did you install the Amplify CLI?

nem

If applicable, what version of Node.js are you using?

v18.20.2

Amplify CLI Version

1.0.1

What operating system are you using?

MacOS

Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.

No

Describe the bug

I have 60 models. I'm trying to migrate to Gen 2.npx ampz sandbox command tried to push, then got this warning:

Number of resources: 421 is approaching allowed maximum of 500
Template size is approaching limit: 824168/1000000. Split resources into multiple stacks or set suppressTemplateIndentation to reduce template size. [ack: @aws-cdk/core:Stack.templateSize]

then I tried to test reach limit and got error like this:

Caused By: Error: Number of resources in stack 'amplify-amplifygen2demo-marlonjd-sandbox-608dc1dabf/data/amplifyData/ConnectionStack': 505 is greater than allowed maximum of 500: AWS::AppSync::FunctionConfiguration (336), AWS::AppSync::Resolver (168), AWS::CDK::Metadata (1)

So, I was able to fix this limit issue by creating custom stacks for some resolvers like in documents, Place AppSync Resolvers in Custom-named Stacks.
And I could fix template size limit issue by doing amplify push --minify

I couldn't do these in Gen 2. So how can we fix this?

Just info:
I saw this example to splitting stack with cdk, it may help this article
I saw people could solve this issue when they're using aws-cdk and serverless (framework) by using split-stacks.

Expected behavior

Should update stack

Reproduction steps

Creating big schema will give this error. I added some test models like this. If models have much relations, it will started to increase resource counts quickly.

Test0: a
    .model({
      name: a.string(),
      test1Id: a.id(),
      test1: a.belongsTo("Test1", "test1Id"),
      test2id: a.id(),
      test2: a.belongsTo("Test2", "test2id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test1: a
    .model({
      name: a.string(),
      test0s: a.hasMany("Test0", "test1Id"),
      test2id: a.id(),
      test2: a.belongsTo("Test2", "test2id"),
      test7s: a.hasMany("Test7", "test1id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test2: a
    .model({
      name: a.string(),
      test1s: a.hasMany("Test1", "test2id"),
      test3id: a.id(),
      test3: a.belongsTo("Test3", "test3id"),
      test7s: a.hasMany("Test7", "test2id"),
      test0s: a.hasMany("Test0", "test2id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test3: a
    .model({
      name: a.string(),
      test2s: a.hasMany("Test2", "test3id"),
      test4id: a.id(),
      test4: a.belongsTo("Test4", "test4id"),
      test5id: a.id(),
      test5: a.belongsTo("Test5", "test5id"),
      test7s: a.hasMany("Test7", "test3id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test4: a
    .model({
      name: a.string(),
      test3s: a.hasMany("Test3", "test4id"),
      test6id: a.id(),
      test6: a.belongsTo("Test6", "test6id"),
      test7s: a.hasMany("Test7", "test4id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test5: a
    .model({
      name: a.string(),
      test3s: a.hasMany("Test3", "test5id"),
      test7s: a.hasMany("Test7", "test5id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test6: a
    .model({
      name: a.string(),
      test4s: a.hasMany("Test4", "test6id"),
      test7id: a.id(),
      test7: a.belongsTo("Test7", "test7id"),
      test8id: a.id(),
      test8: a.belongsTo("Test8", "test8id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test7: a
    .model({
      name: a.string(),
      test6s: a.hasMany("Test6", "test7id"),
      test1id: a.id(),
      test1: a.belongsTo("Test1", "test1id"),
      test2id: a.id(),
      test2: a.belongsTo("Test2", "test2id"),
      test3id: a.id(),
      test3: a.belongsTo("Test3", "test3id"),
      test4id: a.id(),
      test4: a.belongsTo("Test4", "test4id"),
      test5id: a.id(),
      test5: a.belongsTo("Test5", "test5id"),
    })
    .authorization((allow) => [allow.authenticated()]),

  Test8: a
    .model({
      name: a.string(),
      test6s: a.hasMany("Test6", "test8id"),
    })
    .authorization((allow) => [allow.authenticated()]),

Project Identifier

No response

Log output

# Put your logs below this line

Additional information

No response

Before submitting, please confirm:

• I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
• I have removed any sensitive information from my code snippets and submission.

[MarlonJD]

I think I fixed this, I can fix this by adding these codes to backend.ts:

// Get the first resolver
const amplifyData = amplifyDataStack[0];

// Get resolvers from the first resolver
const connectionStack = amplifyData.node.children.filter((child) =>
  child.node.id.includes("ConnectionStack")
);

// If the connectionStack has more than 500 resources
if (connectionStack[0].node.children.length > 500) {
  console.log("All Resources Count:", connectionStack[0].node.children.length);

  // Crete new nestedStacks every nestedStacks can hold 200 resources, each.
  // connectionStack[0].node.children.length / 200 = number of nestedStacks
  const nestedStacksCount = Math.ceil(
    connectionStack[0].node.children.length / 200
  );

  console.log("Splitting, Nested Stacks Count:", nestedStacksCount);

  for (let i = 0; i < nestedStacksCount; i++) {
    const nestedStack = new NestedStack(
      connectionStack[0],
      `ConnectionStackSplit${i}`
    );

    // Get items with start i * 200 to (i + 1) * 200 resources
    const startIndex = i * 200;
    const endIndex = (i + 1) * 200;

    // Add the resources to the nestedStack
    connectionStack[0].node.children
      .slice(startIndex, endIndex)
      .forEach((child) => {
        nestedStack.node.addDependency(child.node);
      });

    amplifyData.node.addDependency(nestedStack);
  }

  // Remove stack from the data stack
  connectionStack[0].node.children.forEach((child) => {
    connectionStack[0].node.tryRemoveChild(child.node.id);
  });
}

I'm still doing some tests, I'll let you know about news

[AnilMaktala]

Hey @MarlonJD, Thank you for raising this issue and sharing the alternative solution. We will incorporate the workaround instructions into the troubleshooting section of the documentation. Additionally, we are marking this as a bug for further evaluation by the team.

[MarlonJD]

Hello @AnilMaktala. Thanks for you reply. I'm glad to hear that, I'm glad there's a solution, even if it's temporary for now.

[MarlonJD]

Hello @AnilMaktala

Not works old or new version, if it's creating from scratch. I created splitted NestedStack but I cannot move old resources to new nestedStacks. It's really important issue. If anyone can help with cdk and CloudFormation stacks resource moving it should work.

It's only working if any new 500 or less stacks available for update. So I cannot create first time deploy. I can deploy like 3-4 part scheme. It's awful solution.

I cannot migrate to Gen 2 for this. It's huge blocker for big project. I hope it's will fix soon.

[AnilMaktala]

I can replicate this issue using schema with 70+ models.

[LukaASoban]

I am about to hit this limit. Is there any work around for this for now?

Template size is approaching limit: 903572/1000000. Split resources into multiple stacks or set suppressTemplateIndentation to reduce template size.

[MarlonJD]

You can fix this error with custom stack mapping but it's not merged yet on gen 2, you have to modify the source and push manually for now, I hope it can fix soon

[LukaASoban]

@MarlonJD would you be able to kindly share your work-around for this? So you are saying that using this work-around makes it so that I can no longer "push" in 1 go and have to split it up into multiple deployments?

[MarlonJD]

Hey @LukaASoban,Which one are you using, Amplify Gen 1 or Gen 2?