Automate complex python scripts with AWS Lambda serverless compute and Valispace!

 

In a complex system design process, reliable automation can save valuable engineering time. Complementing the single-source-of-truth availed by storing the design data in Valispace, this integration with AWS Lambda enables a new level of automation to empower your digital systems engineering process.

img1.png

Automate your iterative solvers, optimization routines, planning sequences, and Monte-Carlo simulations, among others to automatically re-compute and provide event-driven design updates. 

 

Example Use-cases: 

  1. Re-solve the low thrust trajectory transfer automatically after every update in the system mass and update the latest propulsion system parameters. 

  2. Re-run multi-objective system optimization for the entire drone system after completion of a design milestone.

  3. Update cost estimates for an electric car after top-level requirements are verified.

 

Getting Started

Valispace's capabilities are extended beyond the web platform with our REST API and the RESThooks. REST API allows request-based information exchange between Valispace database and your external application.  RESThooks allows subscriptions that post event-based information to target URLs, triggered by specific events/actions. 

 

  • When an event occurs on Valispace, specific python scripts can be triggered in a 

Lambda function.

  • The Lambda function is triggered through a RESTHooks listener in the 

API Gateway, in response to an event hook sent from Valispace.

  • During runtime, the python script in the Lambda function can fetch and update relevant system parameters stored in Valispace through pre-defined REST commands.

 

Here's a step-by-step process to set-up this integration.

 

Step 1: Create a Lambda function

If you're new to AWS Lambda, check out Getting Started and serverless 'Hello World' exercises to get up to speed. 

  1.  Sign in to the AWS Console and select the AWS Lambda service.

  2.  Click on Create Function to configure a new Lambda function.

  3.  Choose Author from scratch.

  4.  Select a suitable runtime for your automation program/script. (We use Python 3.7 as the baseline here. You are free to adapt integration to your choice).

  5.  Click Create Function at the bottom.

mceclip0.png

 

Step 2: Configure the dependencies for the script.

To be able to use python/pip packages in your lambda function code, they must be added as packaged layers.

  1.  Get the basic layer with Valispace python API and find out more about how to add other required python packages on our Github. Follow the steps there to get a packaged ZIP for the Lambda layer.

  2.  On AWS, go to the Layers page using the left navigation pane and click on Create Layer.

  3. Enter the name and description of the packages. Upload the packaged ZIP. You might have to use an S3 bucket to store a packaged file larger than 10 MB, and use the link from S3.

  4. Create the layer and go back to the Lambda function you created.

  5. On the Lambda function page, click on the Layers area in the Designer. This reveals the Layers table below the Designer.

  6. Select Add Layer and choose the Custom Layer option. From the drop-down, select the layer created earlier.

mceclip2.pngmceclip1.png

Your python packages are now ready to use in the code!

 

Step 3: Add Trigger via API Gateway Proxy

  1. Click on Add Trigger button in the Designer panel of the lambda function.

  2. Choose API Gateway and create a new REST API. This will act as the listener for RESThooks posted by Valispace.

  3. Choose Open API security. This is because hook payloads are fixed and cannot yet be configured with API keys or other authentication information.

  4. Click Add. The API should be created and linked with the Lambda function.

  5. Click on the API Gateway Trigger in the Designer and follow the API link in the Triggers table. This will take you to the API resources page. Notice the integration schematic similar to the one below.The API will proxy the request body over to the Lambda function as soon as it is received.

  6. Deploy API from the Actions button in the top-right (second navigation pane)

mceclip5.pngmceclip3.png

 

Step 5: Add your custom python code 

The default lambda_handler(event, context) function is the entry point for all triggers. In our case, the request body of the hooks sent by Valispace will be forwarded to the lambda function through the event variable.

Use the Valispace Python API to fetch and update relevant data during execution.

Below is a basic code example layout: 

import json
import valispace

def login():
    return valispace.API(url='https://staging.valispace.com/', username = 'kuldeep', password='******')

def fetch_input_valis(vs_instance):
    values= {}
    for input in input_valis:
        vali = vs_instance.request('GET', f'vali/{input_valis[input]}')
        values[input] = vali["value"]

    return values

def push_output_values(vs_instance, results):
    for output in output_valis:
    vs_instance.update_vali(id=output_valis[output], formula= results[output])

def lambda_handler(event, context):
    vali = login()
    inputs = fetch_input_valis(vali)
    results = solve_elecar(t_interval=200)
    push_output_values(vali,results)

    return {
        "statusCode": 200,
        "headers": {},
        "body": "",
        "isBase64Encoded": False
    }

CODE

Deploy the Lambda function using the Deploy button above the Editor

Use Test buttons to Test the lambda function and the connection with API Gateway using dummy requests on the respective pages.

 

Step 4: Connect everything to a Valispace event! 

Subscribe to a RESThook that notifies the API Gateway (and therefore the Lambda function) to run the script.

  1. Find the API endpoint URL in API Gateway> API > Settings > Default endpoint.
    The URL should look like this:

    https://{api id}.execute-api.{region}.amazonaws.com /{api deployment stage}/{lambda function name}

  2. Configure a hook subscription  using the Valispace admin panel or the REST API depending on the access privilege.

  3. Select the desired event to subscribe to and test the end-to-end integration.

 

Your automation script should now run and update the design every time the specific event occurs!