In this tutorial, we are going to learn how to use AWS Step Functions. But before moving ahead let's have a quick overview of AWS Step Functions.
AWS Step Functions Overview
- AWS Step Functions is a serverless orchestration service that lets us combine Lambda functions and other AWS services to build business-critical applications.
- AWS Step Functions has two workflow types i.e. Standard workflows and Express workflows. Note that Standard workflows have exactly-once workflow execution and can run for up to one year whereas Express workflows have at-least-once workflow execution and can run for up to five minutes.
- We can view our application’s workflow as a series of event-driven steps through AWS Step Functions' graphical console.
- The features of AWS Step Functions include built-in error handling, execution event history, automatic scaling, high availability, pay per use, and administrative security.
- AWS Step Functions is based on state machines and tasks. Note that a state machine is a workflow, a task is a state in a workflow that represents a single unit of work that another AWS service performs and each step in a workflow is a state.
- We can examine the state of each step in our workflow to make sure that our application runs in order and as expected.
- Through AWS Step Functions, we can also create workflows that process and publish machine learning models.
- Through AWS Step Functions, we can control AWS services, such as AWS Glue, to create ETL workflows.
- For applications that require human interaction, we can create long-running, automated workflows through AWS Step Functions.
- The typical uses cases for working with AWS Step Functions include Function orchestration, Branching, Error handling, Human in the loop, Parallel processing, and Dynamic parallelism.
States in State machines
Following is some useful information regarding the states used in state machines.
- States are elements in our state machine. A state is referred to by its name, which can be any string, but which must be unique within the scope of the entire state machine.
- A Task state does some work in a state machine.
- A Choice state makes a choice between branches of execution.
- A Fail or Succeed state stops execution with a failure or success.
- A Pass state simply passes its input to its output or injects some fixed data.
- A Wait state provides a delay for a certain amount of time or until a specified time/date.
- A Parallel state begins parallel branches of execution.
- A Map state dynamically iterates steps.
- A Type field indicates the type of state.
- An optional Comment field can be used in a state to hold a human-readable description of the state.
- Each state (except a Succeed or Fail state) requires a Next field or, alternatively, can become a terminal state by specifying an End field.
Now let's head straight to the implementation part!
Create Hello World State Machine
First, we will log in to our AWS account then type Step Functions under the Services tab. Then we will click on Get Started. We will be running a Hello World workflow in just a few clicks. Let's see the workflow of the Hello World example.
- The workflow starts from a state named Pass which is a type of Pass state.
- Then the next state to transition to is the state name Hello World example?. It is a type of Choice state. Now when we will enter input values for this execution in JSON format the value of IsHelloWorldExample will determine the next state. If IsHelloWorldExample value is true the next state will be Yes. If IsHelloWorldExample value is false the next state will be No. The default state is Yes. Note that in Default, the name of the state to transition to if none of the transitions in Choices is taken is specified.
- The state named Yes is a type of Pass state. The next state, in this case, is Wait 3 sec.
- The state named No is a type of Fail state. Note that it provides a custom failure string that can be used for operational or diagnostic purposes.
- Wait 3 sec is a type of Wait state. Here the time specified is 3 seconds before transitioning to the next state. The next state, in this case, is Parallel State.
- The state named Parallel state is a type of Parallel state. In this example, the Hello and World branches are executed in parallel. The next state, in this case, is Hello World.
- The state named Hello World is a type of Pass state. This is a terminal state as it specifies an End field.
Now we will click on Next.
Here we will specify details like state machine name, execution role, log level, enable/disable X-Ray tracing, and tags. We will leave everything to default for now and click on Create state machine.
Now let's start execution. Here we will define the value of IsHelloWorldExample which will determine the state to transition to in Choice state. Here we have defined it to true. Then we will click on Start execution.
With this, we have just executed a sample workflow. Now let's have a look at the Graph inspector. Here the transitions are in accordance with what we defined in our state machine.
Note the state transitions in Graph inspector when IsHelloWorldExample is set to false.
Create State Machine
Now we will create our own state machine that uses the Lambda function. Our workflow will invoke a Lambda function and check whether the userName returned by the Lambda function is golinuxcloud or not. Depending on the input our workflow will decide pass or fail state.
Create Lambda Function
First, we will create a lambda function. In myDemoLambdaFunction, we will use a Runtime of Node.js 14.x. The lambda function does nothing fancy and only returns the userName passed in the event object.
Now we will get into the Step Functions homepage. Now from the left pane, we will click on State machines. Then we will click on Create state machine.
Here we have three options available i.e. design your workflow visually, write your workflow in code and run a sample project. Here we will select Write your workflow in code. Now we can author our workflow using Amazon States Language. We can generate code snippets to easily build out our workflow. There are two workflow types from which we can select Standard or Express. We will leave it to default for now.
Workflow
Now comes the main part i.e. defining workflow. We can generate code snippets from the options available in the drop-down as shown below.
Following will be our workflow for the state machine.
- The workflow will start from a state named Invoke Lambda function which is a type of Task state.
- Our lambda function as stated earlier returns userName.
- Then the next state to transition to is the state named Choice State.
- Now when we will enter input values for this execution in JSON format the value of userName will determine the next state. If userName value is equal to golinuxcloud the next state will be Is Admin. If userName is any other value except for golinuxcloud, the next state will be Not Admin.
- The state named Is Admin is a type of Pass state. This is a terminal state as it specifies an End field.
- The state named Not Admin is a type of Fail state. Note that it provides a custom failure string that can be used for operational or diagnostic purposes.
{"Comment": "A Demo example of the Amazon States Language using different states",
"StartAt": "Invoke Lambda function",
"States": {
"Invoke Lambda function": {
"Type": "Task",
"Resource": "arn:aws:lambda:us-east-2:407566125470:function:myDemoLambdaFunction",
"Next": "Choice State"
},
"Choice State": {
"Type": "Choice",
"Choices": [
{
"Variable": "$",
"StringEquals": "golinuxcloud",
"Next": "Is Admin"
}
],
"Default": "Not Admin"
},
"Is Admin": {
"Type": "Pass",
"Result": "Welcome Admin!",
"End": true
},
"Not Admin": {
"Type": "Fail",
"Cause": "The output of the lambda function was not golinuxcloud"
}
}
}
Here we can specify details as discussed earlier but we will leave everything to default for now and click on Create state machine.
Execution
Here we will enter input values for the above workflow in JSON format.
We can view workflow execution when golinuxcloud was passed as the userName.
Let's have a look at the Execution event history in the above case. The types of events that occurred include ExecutionStarted,TaskStateEntered, LambdaFunctionScheduled, LambdaFunctionStarted, LambdaFunctionSucceeded, TaskStateExited, ChoiceStateEntered, ChoiceStateExited, PassStateEntered, PassStateExited and ExecutionSucceeded.
Now let's enter a userName that does not belong to admin.
Here we can see workflow execution when any other value for userName was entered except for golinuxcloud.
Following is the execution event history in the above case. Note that the execution fails as defined in the workflow.
Conclusion
With this, we have come to the end of our tutorial. In this tutorial, we learned about AWS Step Functions. We also had a quick overview related to states in state machines. After that, we walked through a Hello World State Machine. Then we created a state machine that invoked the lambda function. And in the end, we tested our implementation using different inputs.
Stay tuned for some more informative tutorials coming ahead. Feel free to leave any feedback in the comments section.
Happy learning!
References