[INTERNAL] Provisioning an EC2 instance from our Amazon Machine Image

Home

[INTERNAL] Provisioning an EC2 instance from our Amazon Machine Image

This topic explains how to provision an EC2 instance with Unravel Server preinstalled on it, using our Amazon Machine Image (AMI).

Requirements checklist

Platform compatibility

EC2 instance type

Security group / IAM role

AWS permissions

AWS EMR 5.17, 5.18, 5.19

Minimum: R4.2xlarge (61 GiB RAM)

Maximum: R4.8xlarge (244 GiB RAM)

Recommended: R4.4xlarge (122 GiB RAM)

Allowed ports for inbound access to Unravel EC2 node:

Port 3000

Port 4043

Unravel EC2 node can access EMR cluster all ports

Unravel EC2 node has Read permission on the S3 bucket by EMR clusters.

AmazonEC2FullAccess

IAMFullAccess

Note

You need to create an IAM role that has S3 read, write, and list bucket permission for the specific S3 bucket that the EMR cluster will use for logging.

AmazonS3FullAccess

AmazonVPCFullAccess

AmazonSNSReadOnlyAccess

AWSMarketplaceManageSubscriptions

AWSMarketplaceListBuilds

AWSMarketplaceStartBuild

AWSCloudFormation: Permissions set as follows:

{
   "Version": "2012-10-17",
   "Statement": [
       {
           "Effect": "Allow",
           "Action": [
               "cloudformation:*"
           ],
           "Resource": "*"
       }
   ]
}

EC2 instance settings

Name	Default Value	Description
Stack name		The deployed application together with its virtual resources is called a CloudFormation "stack". We recommend you name this stack `UnravelEC2number` or `UnravelEC2date`.
InstanceType	r4.4xlarge	Instance type for the Unravel EC2 instance. Supported values: r4.2xlarge,r4.4xlarge,r4.8xlarge
KeyName	mysshkey	EC2 key name for SSH access to the Unravel EC2 instance. This name can include uppercase letters, lowercase letters, numbers, dashes, and underscores only and must be 1-64 characters long.
TrustedSshIPBlock	10.10.0.0/16	Trusted IP block for SSH, port 3000, and port 4043 access to the Unravel EC2 instance, in CIDR format (x.x.x.x/x).
UnravelInstanceCount	1	Leave this as 1.
UnravelVPCBlock	10.10.0.0/16	Internal subnet (VPC) IP block for the Unravel EC2 instance, in CIDR format (x.x.x.x/x).
Zone	us-east-1a	Availability zone for the Unravel EC2 instance. The zone must match any existing EMR clusters which this Unravel EC2 instance will be monitoring.

Create an EC2 instance from our AMI

Navigate to the EC2 console (https://console.aws.amazon.com/ec2/).
From the menu on the left, select IMAGES | AMIs.
In search box pull-down menu, select Public images.
Search for ami-08d8b2a645bdc7482.
Select this AMI and click Launch. This launches the AMI configuration wizard.
On the Choose an Instance Type page, select R4.2xlarge, and click Next: Configure Instance Details.
(Optional) Modify the configuration of the EC2 instance:
1. On the Configure Instance Details page, modify only the following settings, if necessary:
2. Network: Your selected VPC
3. Subnet: Your selected subnet
4. IAM role: The name of the IAM user you created for S3 access
Click Next: Add Storage.
(Optional) Increase the storage capacity of the EC2 instance to a maximum of 500GiB, depending on the number of clusters, the number of jobs running on those clusters, and whether you plan to enable debug logging.
Click Next: Add Tags.
(Optional) Add tags.
Click Next: Configure Security Group.

On the Configure Security Group page, either create a new security group or select an existing security group in such way that your EMR cluster can access ports 3000 and 4043 of the Unravel EC2 node and your Unravel EC2 node can access port 8082 of the EMR master node. See the sample below for reference. Assume there is no other Network ACL and Unravel is created in the same subnet and VPC of the EMR cluster.

Rule type	Protocol	Port(s)	Source
Inbound: SSH	TCP	22	Your trusted CIDR for SSH access
Inbound: Custom TCP	TCP	3000	0.0.0.0/0 or EMR security group ID or EMR subnet IP block
Inbound: Custom TCP	TCP	4043	EMR security group ID or EMR subnet IP block
Outbound: All traffic	All	All	0.0.0.0/0
Security policy required for all nodes in the EMR cluster (master, core, and task nodes)	All	All	Unravel EC2 node security group ID or Unravel EC2 node private IP address