Lambda Layer Resource List?

[rob-straughan]

Hi,

Is there anywhere a full index of all the packages available for the Lambda layers published via S3?

E.g. https://gorules-public-eu-west-1.s3.eu-west-1.amazonaws.com/lambda/layers/python/python-zen_engine-v0.4.1-x86_64-linux-gnu.zip

The above is the layer I am using but I think it is out of date, since expressionNode appears to cause an exception.

I can't find a resource list or any documentation in the release notes indicating the available packages.

Regards,
Rob.

[ivanmiletic]

Hi @rob-straughan we didn't update layer for a long time as it was just example. We are working on a new features and will not have time to update it over next couple of weeks.

We would suggest creating your own layer with the latest zen-engine.

Best regards,
Ivan

[ivanmiletic]

Layers are quite simple:

You need to install @gorules/zen-engine but must be for lambda architecture independently from your local laptop architecture.

[raymondKelly]

Thanks @ivanmiletic I've worked with layers before and have done similar things with the likes of Prisma, like OP has. There are however some differences here.

For example, if I put the packages in a layer there is no way for my Lambda to know where to import zen-engine from. Layers do not store their code alongside the lambda itself. A lambda typically stores the index.js file at /var/task/ whereas a layer stores its files at /opt. That likely means the lambda has no way to find the node_modules that the layer prepackaged.

Another issue is that esbuild which is used by most frameworks to bundle a lambda or layer needs a way to get those node modules into the layer, I suspect maybe a code asset instead of bundling would suffice, this does however have a bit of a drawback in that the code size will be larger and will cause an increase in cold start time.

[raymondKelly]

After further reading it seems

When you add a Lambda layer with Node.js modules, AWS Lambda will merge the node_modules from the layer with any node_modules in your Lambda function's deployment package.

I was not aware of this, this may solve the unknown I had

[raymondKelly]

I decided to not put it into its own Layer for now and instead package in the Lambda as there is only going to be one Lambda using the engine.

For anyone else that finds this discussion. An example of the CDK Bundling code I used was

    externalModules: ['@gorules/zen-engine', '@gorules/zen-engine-linux-x64-gnu'],
    commandHooks: {
      beforeInstall: (i, o) => [
      ],
      beforeBundling: () => [],
      afterBundling: (i, o) => [
        `mkdir -p ${o}/node_modules/@gorules`,
        `cp -R ${i}/node_modules/@gorules/zen-engine ${o}/node_modules/@gorules/zen-engine`,
        `cp -R ${i}/node_modules/@gorules/zen-engine-linux-x64-gnu ${o}/node_modules/@gorules/zen-engine-linux-x64-gnu`,
      ]
    },

[rob-straughan]

Sorry for the delay, haven't been monitoring this

We did successfully move the nodejs packages for this into a Layer, there were a couple of quirks to it:

• When you build the layer, you need to put a dev dependency to the underlying specific implementation package for the engine, the top level one from npm is just a switch that expects to find the specific arch package for the execution environment, so when you build the layer, unless your build environment happens to match the same arch as lambda, you need to explicitly get the correct underlying required package.
• The Lambda layer is actually quite finicky about package location, there's a top level folder path that is supposed to be compatible with all versions of nodejs, however we found we couldn't get the layer to work unless we dropped the files into the correct specific folder for the nodejs version we were using.

We've been using the layer successfully, although we haven't seen a significant performance benefit from it, so I suspect doing this is only helpful to ease re-use of the packages or reduce number of files deployed in the Lambda deployment.

Regards,
Rob.