The other day, I was building something with AWS Lambda that requires an additional package. Usually, AWS has provided some scientific packages such as scikit-learn and pandas as a Layer that is ready to use. Unfortunately, one of the packages that I need (seaborn) isn’t available yet, so in order to use it in my function, I need to create a Layer manually.
Usually, this is pretty straightforward, all you need to do is to install the package inside a folder and zip it. The command to run it:
pip install -r requirements.txt -t python/
This command will install the package listed in the requirements.txt file into a folder python.
Then, you could zip this folder and upload it as an AWS Layer.
However, I happen to be using a MacBook Air with M1 Processor. In some cases, when installing something from a source, the AWS Lambda failed to use it due to incompatibility of the processor type.
To handle this, I create a docker container with x86 architecture that will do the installing and zipping for me, and then I upload it to the final file to AWS.
So far the result is good, all I need to do is change the requirements.txt to what package I want to install, run the docker, copy the zip to my local machine, and upload it.
The Dockerfile configuration I ended up is:
FROM amd64/python:3.7.17-bookworm
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN apt update
RUN apt install -y zip
RUN mkdir -p ./python
RUN pip install -r requirements.txt -t python/
RUN zip -r python.zip python
ENTRYPOINT ["tail", "-f", "/dev/null"]
You can find the whole code and the bash script to run it can be found in my git.