Lambda Configuration
This configuration covers point 6 and 7 of the diagram. In particular, it describes the configuration for setting up the lambda function running XSpec tests. This entails:
Retrieving the GitHub code stored in a zip file in S3.
Unzipping the file and storing it in a local directory accessible by the lambda function.
Setting up environment variables required by XSpec (i.e. paths to Saxon and HTML report).
Running the XSpec test suite for all the XSpec tests stored in a given directory.
Storing the HTML report in S3.
Notifying the developer in case a test failed and providing a link for accessing the HTML report in S3.
A step-by-step guide to replicate this configuration and the code for the lambda function is available in my GitHub account [markupuk2019]. It is worth highlighting the following points:
Custom Runtime: AWS Lambda natively supports lambda functions written in Java, Go, PowerShell, Node.js, C#, Python, and Ruby. AWS Lambda also provides a Runtime API allowing to implement the lambda function in any programming language [custom_runtime]. I used the latter approach and implemented the lambda function in bash as calling the
xspec.shshell script is the simplest way to run the XSpec test suite. However, the lambda function could be re-written using virtually any programming language.Layers: the lambda function makes use of layers [layers]. A layer is a zip file containing additional libraries and dependencies. In particular, I used layers for the XSpec runtime, the Saxon HE jar file, and bash [bash_layer]. It is also possible to add a layer for Apache Ant and run the XSpec test suite via Ant which can speed up the test suite execution.
Memory and Time Settings: these are configured in the settings for the lambda function. As running large XSpec test suites can be memory and time intensive, I recommend to adapt the values of memory and timeout accordingly. In my experience 512 MB memory and 2 minute timeout is a minimum threshold for running few XSpec tests.
IAM configuration: the lambda function is granted permissions to interact with other AWS services like S3 and SNS via a IAM role [iam_role]. It is a good security practice to allow access only to the relevant S3 buckets.
Notification: I configured the lambda function to send email notifications via SNS (Figure 3, “Email notification” shows an example of email notification with a failed test). It is also possible to configure SNS to send notifications to a chat messaging service like Slack and this may be more appropriate for large development teams.
Figure 3. Email notification
CI Workflow: the lambda function is configured to stop as soon as a test fails: this is done in order to reduce the execution time and to lower costs. However, it is possible to adapt the lambda function to run the full test suite and report all the failing tests in the notification.
Troubleshooting: debugging a lambda function can be challenging as the environment upon which it runs is stateless. Using CloudWatch to monitor the execution of the function and outputting the results of commands to the CloudWatch logs is extremely useful for troubleshooting.