Kerberos KRB5 config mgmt change to false, updated README and added image of RM Variables

Author: smithzc

README.md

@@ -11,47 +11,33 @@ Future development will include support for EDH v5 clusters.  In the meantime, u
 
 Host types can be customized in this template.   Also included with this template is an easy method to customize block volume quantity and size as pertains to HDFS capacity.   See [variables.tf](https://github.com/oracle/oci-quickstart-cloudera/blob/master/terraform/variables.tf#L48-L62)  for more information in-line.
 
-## Prerequisites
-First off you'll need to do some pre deploy setup.  That's all detailed [here](https://github.com/oracle/oci-quickstart-prerequisites).
-
-### Clone the Module
-Now, you'll want a local copy of this repo.  You can make that with the commands:
-
-    git clone https://github.com/oracle/oci-quickstart-cloudera.git
-    cd oci-quickstart-cloudera
+## Resource Manager Deployment
+Using [OCI Resource Manager](https://docs.cloud.oracle.com/iaas/Content/ResourceManager/Concepts/resourcemanager.htm) makes deployment quite easy.  Simply [download the .zip](https://github.com/oracle/oci-quickstart-cloudear/zipball/resource-manager) and follow the [Resource Manager instructions](https://docs.cloud.oracle.com/iaas/Content/ResourceManager/Tasks/usingconsole.htm) for how to build a stack.  Prior to building the Stack, you may want to modify some parts of the deployment detailed in the sections below.
 
 ## Python Deployment using cm_client
 The deployment script "deploy_on_oci.py" uses cm_client against Cloudera Manger API v31.  As such it does require some customization before execution.  Reference the header section in the script, it is highly encouraged you modify the following variables before deployment:
 
 	admin_user_name
 	admin_password
-	cluster_name
-
-Also if you modify the compute.tf in any way to change hostname parameters, you will need to update these variables for pattern matching, otherwise cluster deployment will fail:
-
-	worker_hosts_prefix = 'cdh-worker'
-	namenode_host = 'cdh-master-1'
-	secondary_namenode_host = 'cdh-master-2'
-	cloudera_manager_host = 'cdh-utility-1'
 
-In addition, further customization of the cluster deployment can be done by modification of the following functions:
+In addition, advanced customization of the cluster deployment can be done by modification of the following functions:
 
 	setup_mgmt_rcg
 	update_cluster_rcg_configuration
 
-This does require some knowledge of Python and Cloudera - modify at your own risk.  These functions contain Cloudera specific tuning parameters as well as host mapping for roles.
+This does require some knowledge of Python and Cloudera configuration - modify at your own risk.  These functions contain Cloudera specific tuning parameters as well as host mapping for roles.
 
 ## Kerberos Secure Cluster option
 
 This automation supports using a local KDC deployed on the Cloudera Manager instance for secure cluster operation.  Please read the scripts [README](https://github.com/oracle/oci-quickstart-cloudera/blob/master/scripts/README.md) for information regarding how to set these parameters prior to deployment.
 
 Also - for cluster management, you will need to manually create at a minimum the HDFS Superuser Principal as [detailed here](https://www.cloudera.com/documentation/enterprise/latest/topics/cm_sg_using_cm_sec_config.html#create-hdfs-superuser) after deployment.
 
-Enabling Kerberos is managed using a terraform metadata tag "deployment_type" which is set in [variables.tf](https://github.com/oracle/oci-quickstart-cloudera/blob/master/terraform/variables.tf#L32).   Setting this value to "secure" will enable cluster security as part of the setup process.  Changing this to "simple" will deploy an unsecured cluster.
+Enabling Kerberos is managed using a terraform metadata tag "deployment_type".   Setting this value to "secure" will enable cluster security as part of the setup process.  Changing this to "simple" will deploy an unsecured cluster.  By default this value is set to "simple" for speed of deployment and ease of use for those not familiar with secure cluster operation.
 
 ## High Availability
 
-High Availability is also offered as part of the deployment process.  When secure cluster operation is chosen this is enabled by default.  It can be disabled by either changing the deployment_type to "simple", or modifying the [deploy_on_oci.py](https://github.com/oracle/oci-quickstart-cloudera/blob/master/scripts/deploy_on_oci.py#L60) script and changing the value for "hdfs_ha" to False.
+High Availability is also offered as part of the deployment process.  When secure cluster operation is chosen this is enabled by default.  It can be disabled by either changing the deployment_type to "simple", or modifying the [deploy_on_oci.py](https://github.com/oracle/oci-quickstart-cloudera/blob/master/scripts/deploy_on_oci.py#L60) script and changing the value for "hdfs_ha".
 
 ## Metadata and MySQL
 
@@ -67,20 +53,46 @@ As of the 2.1.0 release, included with this template is a means to deploy cluste
 
 The first should be set to 'True', then replace 'None" with each of the required values.   This configuration will then be pushed as part of the cluster deployment.
 
-## Deployment Syntax
-Deployment of the module is straight forward using the following Terraform commands
+## Resource Manager Variables
+Step 2 for setting up a stack is Configure Variables.   By default all variables are filled in, with the exception of the SSH Public and Private keypair used for host access.   If you don't have a keypair for use with this deployment, generating one on Linux/Mac is simply:
+
+	ssh-keygen -t rsa
+
+Follow the prompts to generate the key, do not associate a password with it.    Copy the contents of each file and paste into the appropriate variable fields as shown here:
+
+![Resource Manager Variables](https://github.com/oracle/oci-quickstart-cloudera/blob/resource-manager/images/RM_variables.png)
 
-	terraform init
-	terraform plan
-	terraform apply
+This list also can be modified to suit your specific deployment requirements.   You should review the settings for the following and ensure you have the capacity in your Tenancy prior to deployment:
+
+	worker_instance_shape
+	worker_node_count
+	block_volumes_per_worker
+	utility_instance_shape
+	master_instance_shape
+	bastion_instance_shape
+
+Note that it is not suggested to modify the data_blocksize_in_gbs to lower than the default value of 700GB.   This is because 700GB is the minimum value to achieve maximum throughput per block volume.  Lowering this has a negative impact on HDFS performance.  If you need more HDFS capacity, best practice is to increase the block_volumes_per_worker which adds more DFS volumes for capacity and aggregate throughput.  For even higher density, the data_blocksize_in_gbs can be increased in tandem.   
+
+When using DenseIO shapes, it's also possible to set the block_volumes_per_worker to "0" to leverage only local NVME disk for HDFS.   In the case that you have both local NVME and block, data tiering will automatically be enabled as part of the deployment process.
+
+## Resource Manager Stack Steps
+After building the stack, it only takes 2 actions to deploy:
+
+	Terraform Actions -> Plan
+	Terraform Actions -> Apply
 
 This will create all the required elements in a compartment in the target OCI tenancy.  This includes VCN and Security List parameters.  Security audit of these in the [network module](https://github.com/oracle/oci-quickstart-cloudera/blob/master/terraform/modules/network/main.tf) is suggested.
 
+The output of the Apply command will contain a URL to access Cloudera Manager.   This is the public IP of the Utility Host, which runs the deployment.   
+
+## Monitoring Cluster Build
+Because all tasks are done in CloudInit, there are two ways to monitor the deployment.   Firstly you can login go the Cloudera Manager URL once it is up and running a few minutes after the Apply command finishes.  Alternatively you can SSH into the Utility node, and monitor the log file "/var/log/cloudera-OCI-initialize.log" which contains detailed output from the deployment.
+
 ## Destroy the Deployment
 
-When you no longer need the deployment, you can run this command to destroy it:
+When you no longer need the deployment, you can destroy it:
 
-	terraform destroy
+	Terraform Actions -> Destroy
 
 ## Deployment Architecture
 

images/RM_variables.png

@@ -558,9 +558,9 @@ def update_parcel_repo(remote_parcel_url, parcel_distribution_rate):
 
     new_parcel_repo_urls = old_parcel_repo_urls + ", " + remote_parcel_url
     repo_cm_config = cm_client.ApiConfig(name='REMOTE_PARCEL_REPO_URLS', value=new_parcel_repo_urls)
-    distribute_cm_config = cm_client.ApiConfig(name="PARCEL_DISTRIBUTE_RATE_LIMIT_KBS_PER_SECOND",
+    distribute_cm_config = cm_client.ApiConfig(name='PARCEL_DISTRIBUTE_RATE_LIMIT_KBS_PER_SECOND',
                                                value=parcel_distribution_rate)
-    phone_home = cm_client.ApiConfig(name='PHONE_HOME', value="false")
+    phone_home = cm_client.ApiConfig(name='PHONE_HOME', value='false')
     new_cm_configs = cm_client.ApiConfigList([repo_cm_config, distribute_cm_config, phone_home])
     updated_cm_configs = cloudera_manager_api.update_config(body=new_cm_configs)
     if debug == 'True':
@@ -1992,7 +1992,7 @@ def config_mgmt_for_kerberos():
     KDC_HOST = cm_client.ApiConfig(name='KDC_HOST', value=cloudera_manager_host)
     MAX_RENEW_LIFE = cm_client.ApiConfig(name='MAX_RENEW_LIFE', value='604800')
     KRB_DNS_LOOKUP_KDC = cm_client.ApiConfig(name='KRB_DNS_LOOKUP_KDC', value='true')
-    KRB_MANAGE_KRB5_CONF = cm_client.ApiConfig(name='KRB_MANAGE_KRB5_CONF', value='true')
+    KRB_MANAGE_KRB5_CONF = cm_client.ApiConfig(name='KRB_MANAGE_KRB5_CONF', value='false')
     kerberos_cm_configs = cm_client.ApiConfigList([KDC_ADMIN_HOST, KDC_ADMIN_PASSWORD, KDC_ADMIN_USER, KDC_HOST,
                                                    MAX_RENEW_LIFE, KRB_DNS_LOOKUP_KDC, KRB_MANAGE_KRB5_CONF])
     updated_cm_configs = cloudera_manager_api.update_config(body=kerberos_cm_configs)