Launch EI accelerators in minutes with the Amazon Elastic Inference setup tool for EC2

Launch EI accelerators in minutes with the Amazon Elastic Inference setup tool for EC2

The Amazon Elastic Inference (EI) setup tool is a Python script that enables you to quickly get started with EI.

Elastic Inference allows you to attach low-cost GPU-powered acceleration to Amazon EC2 and Amazon SageMaker instances to reduce the cost of running deep learning inference by up to 75 percent. If you are using EI for the first time, there are a number of dependencies that must be set up: Amazon Web Services (AWS) PrivateLink VPC Endpoints, IAM policies, and security group rules. To accelerate this, The EI setup script makes it easy for you to get started by creating the necessary resources to help you launch EI accelerators in minutes. In this blog post I describe how to use the script, functionalities of the script, and what to expect when you run it.

At a high level, the script does the following:

    1. Creates an IAM role for the instance with an IAM policy that lets you connect to the AWS Elastic Inference service.
    2. Creates a security group with the necessary ingress and egress rules to allow the instance to communicate with the accelerator.
    3. Creates an AWS PrivateLink VPC Endpoint within your desired subnet.
    4. Launches the desired EC2 instance with an EI accelerator using the latest AWS Deep Learning AMI (DLAMI) for the chosen operating system


To set up EI, run the script linked below. It depends on the following entities:

    1. Python 3 installed on your local machine where you expect to run the tool.
    2. The AWS SDK for Python (Boto3).
    3. An Amazon VPC in the Region where you are launching the instance (could be your default VPC).
    4. Subnet where you’d like to launch the instance.
    5. EC2 Key Pair.
    6. AWS credentials.

With these in place, download the script from GitHub to your local machine and run it from your terminal using following command:

$ python

What the tool creates on your behalf

The script creates following AWS resources:

What to expect when you run the tool

The example here illustrates what to expect when you run the script.

    • Launch the script. The script can be launched from the command prompt as:
    • $ python –region us-west-2 –instance-type m5.xlargeAWS credentials are required to create or modify AWS resources. It uses Boto3, AWS SDK for Python. In order to be able to configure and manage AWS resources, the script needs user credentials. If the script is run without appropriate credentials, it reports the error below:
      $ python --region us-west-2 --instance-type m5.xlarge
      Error setting up Amazon EI configuration - 
       Failed to retrieve VPC endpoints for us-west-2 : An error occurred (RequestExpired)
       when calling the DescribeVpcEndpointServices operation: Request has expired.

      The solution is to configure AWS credentials using one of the methods described in the Amazon Boto3 documentation. After the credentials are in place, the script is able to proceed.

    • Choose Operating System. The script prints informative message and prompts for choosing the OS. It also informs that entering ‘q’ causes the script to exit. Choose ‘1’ for the next step.
      $ python --region us-west-2 --instance-type m5.xlarge
      This script launches Amazon EC2 instances with Amazon Elastic Inference accelerators.
      Performs the following functions:
       1. It uses the Deep Learning AMIs preconfigured with EI-enabled deep learning 
       frameworks to launch the instances.
       2. It creates security groups for the instance and VPC endpoint.
       3. It creates the VPC endpoint needed for your instances to communicate with EI 
       4. It creates an IAM Instance Role and Policy with the permissions needed to 
       connect to accelerators.
       To begin, please choose the Operating System for your instance by typing its index :
       0: Amazon Linux
       1: Ubuntu
      Type 'q' to quit.
    • Choose Accelerator size. The script discovered latest DL AMI for Ubuntu, it also discovered one key pair. If it discovers multiple key pairs, it lists those and ask the user to choose desired key pair by typing its index.  In general if there are multiple eligible inputs, the script shows them as indexed list and let the user choose an item by typing its index. Thus, script lists supported accelerator sizes and lets user choose.
       Using Image ID: ami-0027dfad6168539c7,Image Name: Deep Learning AMI (Ubuntu) Version 21.2
       Using instance type: m5.xlarge
       Using Key Pair: Efti-Default-KeyPair
      Please type index of the accelerator type to use:
       0: eia1.medium (1 GB of accelerator memory)
       1: eia1.large (2 GB of accelerator memory)
       2: eia1.xlarge (4 GB of accelerator memory)
      Type 'q' to quit.
    • Choose VPC. As illustrated, user chose option ‘1’ for Accelerator size and the script confirmed the Accelerator size chosen and proceeded to discover IAM role. Subsequently, it presents list of available VPCs.
      amazonei-wizard> 1 
       Using Amazon EI accelerator type: eia1.large
       Found an IAM role configured for connecting to Amazon EI service. Name - Amazon-Elastic-Inference-Connect-Role, ARN - arn:aws:iam::326228132093:role/Amazon-Elastic-Inference-Connect-Role
      Please select the VPC to use by typing the desired VPC index. Type 0 for default VPC.
       0: VPC Id 'vpc-d7d218af'
       1: VPC Id 'vpc-0c2496c51925ff1be'
      Type 'q' to quit.
    • Launch an instance. Once user chooses the VPC ID, the script found a security group with matching inbound rules associated with chosen VPC, it also found one subnet associated with the chosen VPC ID. Additionally it found VPC endpoint for Amazon EI service. As the script has all the details to launch an EC2 instance, the script summarizes all the parameters it uses to launch the instance.
       Using VPC ID: vpc-0c2496c51925ff1be
       Using Security Group: sg-00aec97685affb306
       Using Subnet: subnet-04881d24764d6e73f
       Discovered VPC endpoint for Amazon EI service, ID: vpce-0d2942a8147305240
       The script will now launch new instance with following configuration. Type 'y' to continue. 
       Accelerator Type: eia1.large
       Region: us-west-2
       Image-ID: ami-0027dfad6168539c7 - (Deep Learning AMI (Ubuntu) Version 21.2)
       Instance Type: m5.xlarge
       Key Pair: Efti-Default-KeyPair
       Security Group ID: sg-00aec97685affb306
       Subnet ID: subnet-04881d24764d6e73f
       Instance Profile: Amazon-Elastic-Inference-Instance-Profile
      Type 'y' to continue. Type 'q' to quit.
    • Launch and wait for the instance to reach running state. As the user typed ‘y’, the script proceeded to launch the instance. The script also printed probable SSH command. The script infers the SSH command based on the OS type, key pair chosen, and the public DNS name. Actual command differs based on location of pem file. The script also warns that the instance may not be immediately accessible via SSH, even though it is in running state. The instance needs to be initialized fully, specifically  the SSH daemon needs to be started before it can accept SSH connections. If the pem file is correctly located the user should be able to access the instance and proceed with using Amazon Elastic Inference.
       Launching Instance ..
       Launched instance successfully. The instance ID is 'i-0969820364c038cca'.
       Waiting for instance to reach running state ...
       You can use the following sample SSH command to connect to your instance: ssh -i "Efti-Default-KeyPair.pem"
       Note: Please wait until instance is fully initialized and ready to accept SSH connections. You may check instance status at EC2 console.
       Also please locate your private key file 'Efti-Default-KeyPair.pem'.
      amazon-elastic-inference-tools $