10,000 or more regions with 300 or more Region Servers: 12 GB.10,000 or more regions with 200 or more Region Servers: 8 GB.Guide for more information about using non-local storage. Refer to the Cloudera Enterprise Storage Device Acceptance Criteria The majority of the Hadoop platform are optimized to provide high performance by distributing work across a cluster that can utilize data locality and fast local I/O. Warning: Running CDH on storage platforms other than direct-attached physical disks can provide suboptimal performance. Cloudera does not support drives larger than 8 TB.
You could useġ2 x 8 TB spindles or 24 x 4TB spindles. Cloudera does not support exceeding 100 TB per data node. That said, having ultra-dense DNs will affect recovery times in the event of machine or rack failure. The DN’s scalability limits are mostly a function of the number of replicas per DN, not the overall The maximum acceptable size will vary depending upon how large average block size is. Add more cores for highly active clusters. Set this value using the Java Heap Size of DataNode in Bytes HDFS configuration property. For example, 5 million replicas require 5 GB of memory. Replicas above 4 million on the DataNodes. When increasing the memory, Cloudera recommends an additional 1 GB of memory for every 1 million Increase the memory for higher replica counts or a higher number of blocks per DataNode.
Add an additional 1 GB for each additional 1,000,000 blocks.Minimum: 1 GB (for proof-of-concept deployments).Set this value using the Java Heap Size of JournalNode in Bytes HDFS configuration property. That you run the purge function to reclaim disk space and keep data growth (and the corresponding memory requirement) in check. If you have not set up the purge function to run at a scheduled interval, Cloudera recommends Navigator logs includeĮstimates based on the number of objects it is tracking.Ĭonfigure this value using the Java Heap Size of Navigator Metadata Server in Bytes configuration property.ĭata stored by the Metadata server grows indefinitely unless you run the purge function. See Storage Space Planning for Cloudera Navigator.Īdd 20 GB for operating system buffer cache, however memory requirements can be much higher on a busy cluster and could require provisioning a dedicated host. Ideally, the database should not be shared with other services because the audit insertion rate can overwhelm the database server making other services using same database less The database used by the Navigator Audit Server must be able to accommodate hundreds of gigabytes (or tens of millions of rows perĭay). Configure this value using the Java Heap Size of Auditing Server in Bytes configuration property.
0 kommentar(er)
