Cloudera Navigator Upgrade and Config Changes

article

Cloudera Navigator: - We need Cloudera Navigator as a fully integrated data management tool and a security system which is installed on Hadoop Platform. Cloudera provides a software platform for the Machine Learning & Big Data analytics Solutions and Services that keeps running in the cloud. Cloudera Navigator is the main complete data administration solution for Hadoop, offering basic abilities, such as data disclosure, metadata management, audit, constant optimization, and policy enforcement.

Scenario: - While doing Cloudera Manager upgrade we have to upgrade Cloudera Navigator too. Below steps will guide us how to do configuration changes and Cloudera Navigator upgrade.

If you are running a version of Cloudera Manager 5.4.x or below and are planning to upgrade to Cloudera Manager 5.5.x or higher, use the following instructions to determine

Applying Configurations for Navigator Metadata Server Upgrade

From Cloudera Manager:

  1. In Cloudera Manager, go to Cloudera Management Services > Configuration > Navigator Metadata Server > Resource Management > Java Heap Size of Navigator Metadata Server in Bytes

    The recommended amount of heap to allocate for the Navigator Metadata Server is ~30 GB.

    article
  2. Alter the value to the newly-calculated heap size for Upgrade Operation
  3. Save the change
  4. Change the Navigator Upgrade Batch size from its default of 100K to a temporary value of 10K.
    To set this advanced configuration property and value:
    1. Cloudera Management Services > Configuration > Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for Cloudera-navigator.properties
    2. In the text field enter:
      nav.upgrade.batch.size=10000
  5. Save the change
  6. Restart Navigator Metadata Server for these configuration changes to take effect
  7. After these values are set and all other preparations made, perform the Cloudera Manager upgrade
  8. After Cloudera Manager is upgraded, load the Navigator Metadata Server UI and log in one time to invoke the schema upgrade for this role.
  9. Monitor progress of Navigator Metadata Server schema upgrade and make any adjustments as needed
    1. a. Using the NMS UI: The upgrade may take several hours depending on the size of the data set. Progress will be noted in the UI. It is not required to keep a browser session open once the schema upgrade is invoked.
      article
    2. Using Cloudera Manager Charts: Track the Navigator Metadata Server heap usage during the upgrade by referencing the live chart for JVM Heap Memory Usage on the Clusters > Cloudera Management Service > Navigator Metadata Server role page
      article
    3. In the rare case any OutOfMemory error is seen during the schema upgrade, allocate additional heap to the Navigator Metadata Server for the duration of schema upgrade based on growth seen from the JVM Heap Memory Usage chart.
  10. Once the Navigator Metadata Server upgrade is complete and Navigator Metadata Server UI is operational and can be used. The total heap requirement will go back down and should be set to the Normal Operation Post-Upgrade value.
  11. Normal Operation Post-Upgrade

    Applying the Right-Sized Heap Configuration for Normal Operation

    From Cloudera Manager

  12. Navigate to Cloudera Management Services > Configuration > Navigator Metadata Server > Resource Management > Java Heap Size of Navigator Metadata Server in Bytes

    The recommended amount of heap to allocate for the Navigator Metadata Server is ~20 GB.

    article
  13. Alter the value to the newly-calculated heap size
  14. Save the change
  15. Revert the prior change to the Navigator Upgrade Batch size from its upgrade setting of 10K: To set this advanced configuration property and value:
    1. Cloudera Management Services > Configuration > Navigator Metadata Server Advanced Configuration Snippet (Safety Valve) for Cloudera-navigator.properties
    2. In the text field find and remove: nav.upgrade.batch.size=10000
  16. Save the change
  17. Restart Navigator Metadata Server for these configuration changes to take effect
  18. Continue to monitor the Navigator Metadata Server JVM Heap Memory Usage periodically and adjust as necessary. Memory usage could increase over time as new elements and relations are established within cluster data stores.

Cloudera has a similar approach to deal with the next generation data analytics management and big data solutions for hybrid deployments. Cloudera, like Hortonworks, believes that data can drive high-speed business model changes and has improved in a manner that advantage the market by creating new revenue possibilities.

If you have followed all the above instructions and the Navigator Metadata Server is not successfully upgrading its schema, is still presenting Out of Memory exceptions or is not starting after an upgrade, see this related knowledge article for further instruction and additional options.

Read More :