[INTERNAL] Provisioning an EC2 instance from our CloudFormation template
This topic explains how to provision an EC2 instance with Unravel Server preinstalled on it, using our CloudFormation template.
Requirements checklist
Platform compatibility | EC2 instance type | Security group / IAM role | AWS permissions | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|
AWS EMR 5.17, 5.18, 5.19 | Minimum: R4.2xlarge (61 GiB RAM) Maximum: R4.8xlarge (244 GiB RAM) Recommended: R4.4xlarge (122 GiB RAM) | Allowed ports for inbound access to Unravel EC2 node: Port 3000 Port 4043 Unravel EC2 node can access EMR cluster all ports Unravel EC2 node has Read permission on the S3 bucket by EMR clusters. Allowed port for inbound access to EMR master node: Port 8020 (Name node access for spark event log) Allowed port for inbound access to EMR core node: Port 50010 (Data node access for spark event log) Port 50020 |
NoteYou need to create an IAM role that has S3 read, write, and list bucket permission for the specific S3 bucket that the EMR cluster will use for logging.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": [ "cloudformation:*" ], "Resource": "*" } ] } |
EC2 instance settings
Name | Default value | Description |
---|---|---|
Stack name | The deployed application together with its virtual resources is called a CloudFormation "stack". We recommend you name this stack | |
InstanceType | r4.4xlarge | Instance type for the Unravel EC2 instance. Supported values: r4.2xlarge,r4.4xlarge,r4.8xlarge |
KeyName | mysshkey | EC2 key name for SSH access to the Unravel EC2 instance. This name can include uppercase letters, lowercase letters, numbers, dashes, and underscores only and must be 1-64 characters long. |
TrustedSshIPBlock | 10.10.0.0/16 | Trusted IP block for SSH, port 3000, and port 4043 access to the Unravel EC2 instance, in CIDR format (x.x.x.x/x). |
UnravelInstanceCount | 1 | Leave this as 1. |
UnravelVPCBlock | 10.10.0.0/16 | Internal subnet (VPC) IP block for the Unravel EC2 instance, in CIDR format (x.x.x.x/x). |
Zone | us-east-1a | Availability zone for the Unravel EC2 instance. The zone must match any existing EMR clusters which this Unravel EC2 instance will be monitoring. |
Create an EC2 instance from our CloudFormation template
Download our CloudFormation template from https://s3.amazonaws.com/unraveldatarepo/unravel_aws_marketspace_01.json to your local
/tmp
folder.Use AWS CLI or the AWS Marketplace console to create the Unravel EC2 instance:
From AWS CLI, run the aws cloudformation command with these parameters.
Syntax:
aws cloudformation create-stack --stack-name UnravelEC2 \ --template-body file:///tmp/unravel_aws_marketspace_01.json \ --parameters ParameterKey=Zone,ParameterValue=
AWS_ZONE_NAME
\ ParameterKey=InstanceType,ParameterValue=AWS_INSTANCE_TYPE
\ ParameterKey=KeyName,ParameterValue=my_ssh_key
\ ParameterKey=UnravelVPCBlock,ParameterValue=AWS_VPC_BLOCK
\ ParameterKey=TrustedSshIPBlock, ParameterValue=AWS_SSH_BLOCK
\ --capabilities CAPABILITY_IAM | tee UnravelEC2-stack.jsonExample:
aws cloudformation create-stack --stack-name UnravelEC2number \ --template-body file:///tmp/unravel_aws_marketspace_01.json \ --parameters ParameterKey=Zone,ParameterValue=us-east-1d \ ParameterKey=InstanceType,ParameterValue=r4.2xlarge \ ParameterKey=KeyName,ParameterValue=topcat \ ParameterKey=UnravelVPCBlock,ParameterValue=10.12.0.0/16 \ ParameterKey=TrustedSshIPBlock, ParameterValue=0.0.0.0/0 \ --capabilities CAPABILITY_IAM | tee UnravelEC2-stack.json
From AWS Marketplace (
https://aws.amazon.com/marketplace
):Type
unravel
in the search box:In the search results, select Unravel APM for EMR and click Continue to Subscribe.
Click Continue to Configuration.
On the Configure this software page, select the following:
Fulfillment Option: Unravel EC2 instance
Software Version: 4.5.0.5
Region: Select the region that matches your existing Amazon EMR.
Click Continue to Launch.
On the Launch this software page, click Usage Instructions to download the documentation for the Unravel EC2 instance.
In the Choose Action pull-down menu, select Launch CloudFormation.
Click Launch.
You are now on the CloudFormation portal (https://console.aws.amazon.com/cloudformation/home)
In the Choose a template section, select Specify an Amazon S3 template URL, leave the default URL as is, and click Next.
You are now on the Specify Details page on the CloudFormation portal.
On the Specify Details page, specify your settings for this EC2 instance, and click Next.
On the Options page, leave all settings as default and click Next.
On the Review page, accept the notice about IAM resources and click Create.
The deployed application together with its virtual resources is called a CloudFormation "stack" and is named
UnravelEC2number
.Tip
If you don't see your new stack in the list, wait 30-60 seconds and refresh the page.
When the status of the stack changes to CREATE_COMPLETE, select it and expand its Outputs section: From the AWS CloudFormation portal, click CloudFormation ?
UnravelEC2number
? Outputs. Confirm that you see four keys listed in this section:UnravelBackupS3: The S3 bucket used by the Unravel EC2 node for backups
UnravelAutoScalingGroupId: The Unravel EC2 node's instance ID
VPC: The virtual public cloud (VPC) created for this Unravel EC2 node
SecurityGroup: The security group created for this Unravel EC2 node
For example:
Get the public IP address and private IP address of this EC2 instance.
You can do this through aws CLI or through the EC2 console, (https://console.aws.amazon.com/ec2/).
To use aws CLI, run the
ec2 describe-instances
command using the value of the UnravelAutoScalingGroupId key shown in the Outputs tab/section above.AUTOSCALING_GROUP=UnravelAutoScalingGroupId_value_from_Outputs_tab aws ec2 describe-instances --filters "Name=tag:aws:autoscaling:groupName,Values=$AUTOSCALING_GROUP" |grep PublicIpAddress |head -1 aws ec2 describe-instances --filters "Name=tag:aws:autoscaling:groupName,Values=$AUTOSCALING_GROUP" |grep PrivateIpAddress |head -1
To use the EC2 console, highlight this EC2 instance and look at its IP addresses in its Description tab.
For example:
Log into Unravel UI
Using a web browser, navigate to http://
and log in with username unravel-ec2-ip
:3000admin
with password unraveldata
.
Note
For the free trial version, use the Chrome browser.
Congratulations! Unravel Server is up and running. Proceed to connect to your existing or new EMR cluster .