Skip to main content

Home

Migrations

Overview

The Migration tab helps you to discover your current cluster and its detailed usage such that you can plan and make an effective and informed migration to a cloud platform in a faster and more cost-efficient way.

  • Cluster DiscoveryCluster discovery analyzes your on-prem cluster usage, workload patterns, and your host's hardware configuration.

  • Cloud Mapping per HostCloud mapping per host

    • Lift and Shift is a one-to-one mapping of each existing host's capacity (based on CPU, memory, and disk) to the closest fit on the cloud, which meets or exceeds the host's hardware specifications. This method estimates what your current on-prem cluster configuration can cost on the cloud. Although the Lift and Shift method does not consider the actual workload or resource usage, it minimizes the risks of migrating to the cloud.

    • Cost Reduction is a one-to-one mapping of each existing host’s actual usage (based on CPU, memory, and disk) to the closest fit on the cloud that meets or exceeds the host's usage requirements. This method optimizes cost but not necessarily peak usage. While still a one-to-one mapping, this method is usually more cost-effective than lift and shift as it minimizes over-provisioning (under-utilizing your host's resources).

    The reports are generated by analyzing your cluster workload over 30 days of history.

  • Workload Fit maps your workload onto the cloud and is not a one-tone mapping of your hosts like Cloud Mapping per host. Because it maps workload instead of hosts, it should result in lower costs.

  • Services and versions compatibility compares what is on your cluster and the component versions offered by the target you selected for the report.

  • Analyze on-prem HDP or CDH clusters.

  • Examine the costs of running your clusters on:

    • Azure Databricks

    • AWS Databricks

    • Google Compute Engine (IaaS)

    • Google DataProc

    • Amazon EC2 (Iaas)

    • Amazon EMR

    • Azure (Iaas)

    • Azure HDInsight

You can download all the reports under the Migration tab in a PDF or JSON format.

Limitations

  • Your cluster must have at least seven days of metrics for Unravel to generate useful reports.

  • Clusters running MapR Control System are not supported.

Important

You must set the cluster manager properties to use Migration Planning Reports.

Configuring the Migration reports

Notice

When configuring the Migration report on a CDH or CDP cluster, ensure you have one of the following roles.

  • Full Administrator

  • Cluster Administrator

  • Operator

  • Configurator

These roles are required only in the case of the Cloudera Manager.

Note

Before configuring migration, run ./manager config ondemand show to confirm the on-demand cluster setup.

Then proceed to enable migration.

  • If it displays no cluster, execute ./manager config ondemand select-cluster to set the desired cluster.

  • Confirm the configuration with "Selected cluster 'ClusterFromTemplate' for on-demand."

To configure the Migration reports do the following:

  1. Stop Unravel.

    <Unravel installation directory>/unravel/manager stop

  2. Enable the Migration reports as an admin user. Run the following command:

    <Unravel installation directory>/unravel/manager config ondemand cloud-migration enable

  3. Optional: Specify the cloud providers that must display for the Migration report.

    1. Check the existing cloud providers shown for the Migration report. 

      manager config ondemand cloud-migration provider show

    2. Specify the cloud providers that you want to show for the Migration reports. You can provide a space-separated list of cloud providers when you specify this property.

      <Unravel installation directory>/unravel/manager config ondemand cloud-migration provider set <cloudprovider1> <cloudprovider2>

##For example:
<Unravel installation directory>/unravel/manager config ondemand cloud-migration provider set EMR EC2 DATABRICKS_AWS

    The following options are available for <cloud_provider>:

    • EC2

    • EMR

    • Azure

    • HDI

    • GCE

    • DATAPROC

    • DATABRICKS_AWS

    • DATABRICKS_AZURE

    Note

    This property is applicable only from Unravel version 4.7.1.0

  4. Optional: You can specify the VM instances to be blacklisted for the Cloud Mapping Per Host and Workload Fit reports. Later provide the path to the CSV or TXT file that you can use to blacklist the specific VM instances for all cloud providers.

    1. Create a CSV or TXT file that contains a comma-separated list of VM instances that must not display for the migration reports. This is a blacklist of the VM instances.

    2. Set the property to define the path to the CSV or TXT file.

      <Unravel installation directory>/unravel/manager config properties set com.unraveldata.migrationplanning.blacklist </path/to/CSV or TXT/file>

For example:
/opt/unravel/manager config properties set com.unraveldata.migrationplanning.blacklist /opt/unravel/data/migration/blacklist.txt

      You can also specify the property, such as blacklisting specific VM instances for particular cloud providers. 

      <Unravel installation directory>/unravel/manager config properties set com.unraveldata.migrationplanning.blacklist.<cloud_provider1>, </path/to/CSV or TXT/file>

For example:
/opt/unravel/manager config properties set com.unraveldata.migrationplanning.blacklist.EMR /opt/unravel/data/migration/blacklist.txt
/opt/unravel/manager config properties set com.unraveldata.migrationplanning.blacklist.EC2 /opt/unravel/data/migration/blacklist.txt

      Note

      This property is applicable only from Unravel version 4.7.1.0.

  5. Optional: Specify custom prices for various instance types in Migration reports in a CSV file and later provide the CSV file path.

    1. Create a CSV file with the following column names and corresponding data that contains the custom prices of instances for cloud providers.

      • Region, Instance, Ondemand_Price

      Following is a sample format of the CSV file.

      Region, Instance, Ondemand_Price
US East (Ohio), r5.24xlarge, 5.0
US East (Ohio), r6g.16xlarge, 4.5

    2. Set the property to define the cloud provider and the path to the CSV file.

      <Unravel installation directory>/unravel/manager config properties set com.unraveldata.migrationplanning.vmprices.<cloud_provider1> </path/to/CSV>

For example:
/opt/unravel/manager config properties set com.unraveldata.migrationplanning.vmprices.EMR /opt/unravel/data/migration/customprices.csv

  6. Apply the changes.

    <Unravel installation directory>/unravel/manager config apply

  7. Start Unravel.

    <Unravel installation directory>/unravel/manager start
In this section: