The Amazon Elastic Inference (EI) setup tool is a Python script that enables you to quickly get started with EI.
Elastic Inference allows you to attach low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost of running deep learning inference by up to 75 percent. If you are using EI for the first time, there are a number of dependencies that must be set up: Amazon Web Services (AWS) PrivateLink VPC Endpoints, IAM policies, and security group rules. To accelerate this, The EI setup script makes it easy for you to get started by creating the necessary resources to help you launch EI accelerators in minutes. In this blog post I describe how to use the script, functionalities of the script, and what to expect when you run it.
At a high level, the script does the following:
- Creates an IAM role for the instance with an IAM policy that lets you connect to the AWS Elastic Inference service.
- Creates a security group with the necessary ingress and egress rules to allow the instance to communicate with the accelerator.
- Creates an AWS PrivateLink VPC Endpoint within your desired subnet.
- Launches the desired EC2 instance with an EI accelerator using the latest AWS Deep Learning AMI (DLAMI) for the chosen operating system
To set up EI, run the script linked below. It depends on the following entities:
- Python 3 installed on your local machine where you expect to run the tool.
- The AWS SDK for Python (Boto3).
- An Amazon VPC in the Region where you are launching the instance (could be your default VPC).
- Subnet where you’d like to launch the instance.
- EC2 Key Pair.
- AWS credentials.
With these in place, download the amazonei_setup.py script from GitHub to your local machine and run it from your terminal using following command:
What the tool creates on your behalf
The script creates following AWS resources:
What to expect when you run the tool
The example here illustrates what to expect when you run the script.
- Launch the script. The script can be launched from the command prompt as:
- $ python amazonei_setup.py –region us-west-2 –instance-type m5.xlargeAWS credentials are required to create or modify AWS resources. It uses Boto3, AWS SDK for Python. In order to be able to configure and manage AWS resources, the script needs user credentials. If the script is run without appropriate credentials, it reports the error below:
The solution is to configure AWS credentials using one of the methods described in the Amazon Boto3 documentation. After the credentials are in place, the script is able to proceed.
- Choose Operating System. The script prints informative message and prompts for choosing the OS. It also informs that entering ‘q’ causes the script to exit. Choose ‘1’ for the next step.
- Choose Accelerator size. The script discovered latest DL AMI for Ubuntu, it also discovered one key pair. If it discovers multiple key pairs, it lists those and ask the user to choose desired key pair by typing its index. In general if there are multiple eligible inputs, the script shows them as indexed list and let the user choose an item by typing its index. Thus, script lists supported accelerator sizes and lets user choose.
- Choose VPC. As illustrated, user chose option ‘1’ for Accelerator size and the script confirmed the Accelerator size chosen and proceeded to discover IAM role. Subsequently, it presents list of available VPCs.
- Launch an instance. Once user chooses the VPC ID, the script found a security group with matching inbound rules associated with chosen VPC, it also found one subnet associated with the chosen VPC ID. Additionally it found VPC endpoint for Amazon EI service. As the script has all the details to launch an EC2 instance, the script summarizes all the parameters it uses to launch the instance.
- Launch and wait for the instance to reach running state. As the user typed ‘y’, the script proceeded to launch the instance. The script also printed probable SSH command. The script infers the SSH command based on the OS type, key pair chosen, and the public DNS name. Actual command differs based on location of pem file. The script also warns that the instance may not be immediately accessible via SSH, even though it is in running state. The instance needs to be initialized fully, specifically the SSH daemon needs to be started before it can accept SSH connections. If the pem file is correctly located the user should be able to access the instance and proceed with using Amazon Elastic Inference.