You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Analytics/Cluster/Hardware: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>Ottomata
imported>Quiddity
m (fix rdr)
 
(3 intermediate revisions by 3 users not shown)
Line 1: Line 1:
Information about the Analytics Cluster hardware and infrastructure.
#REDIRECT [[Analytics/Systems/Cluster#Infrastructure]]
 
 
== Docs ==
 
* [[Analytics/Cluster/Puppet|Puppetization]]
* [[Analytics/Cluster/Ports|Ports]]
 
== Hardware Specs ==
 
=== By Host ===
 
{|class='sortable wikitable' style='text-align: center; width:100%;'
! Type !! # !! Hosts !! Processors !! Memory !! Disk
|-
|Dell PowerEdge R420 ||3|| analytics1001, analytics1002, analytics1003 || 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz || 64G RAM || 4 * 2T
|-
| Dell PowerEdge R720 || 12 || aqs100{1,2,3},kafka10{12,13,14,18,20,22}|| 12 core EW-2620 @ 2.00GHz || 48G RAM || 24T = 12 * 2T
|-
| Dell PowerEdge R720 || 29 || analytics1028 - analytics1057 || 12 core EW-2620 @ 2.00GHz || 64G RAM || 48T = 12 * 4T, + 2 SSDs
|-
| Dell PowerEdge R310 || 5 || analytics1026 - analytics1027 || 4 core X3430 @ 2.40GHz || 8G RAM || 2T = 2 * 1T
|}
 
=== By Machine Type ===
 
{|class='wikitable' style='text-align: center; width:100%;'
! Type !! Dell PowerEdge R420 !!  Dell PowerEdge R720 !! Dell PowerEdge R720 !! Dell PowerEdge R310
|-
| '''Hosts''' || analytics1001, analytics1002, analytics1003 || aqs100{1,2,3},kafka10{12,13,14,18,20,22} || analytics1028-analytics1057 || analytics1026 - analytics1027
|-
| '''Processors''' || 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz || 12 core EW-2620 @ 2.00GHz || 12 core EW-2620 @ 2.00GHz || 4 core X3430 @ 2.40GHz
|-
| '''Memory''' || 64G RAM ||  48G RAM || 64G RAM || 8G RAM
|-
| '''Disk''' || 4 * 2T || 24T = 12 * 2T || 48T = 12 * 4T +  2 SSDs || 2T = 2 * 1T
|}
 
== Roles ==
 
{| class="sortable wikitable" style="width:100%;"
! Hosts !! # !! Type !! Role
|-
| analytics1001 || 2 || R420 || Primary NameNode, Hadoop master node.
|-
| analytics1002 || 2 || R420 || Standby NameNode
|-
|analytics1003
|1
|R420
|Hive, Oozie, MySQL
|-
|analytics1028-analytics1057 || 30 || R720 || Hadoop Worker Nodes (DataNodes)
|-
| kafka1012,kafka1013,kafka1014,kafka1018,kafka1020,kafka1022 || 6 || R720 || Kafka Brokers
|-
| analytics1026 || 1 || R310 || ''spare''
|-
| analytics1027 || 1 || R310 || Frontend Web Service Host (Hive, Hue, Oozie, etc.)
|-
| aqs1001-aqs1003 || 3 || R720 || Analytics Query Service (RESTBase + Cassandra)
|-
|}
 
''Notes:''
 
''- As of June 2015, analytics102[345] which were Zookeeper servers in the Analytics Cluster have been moved out into the general purpose production network and renamed as conf100[123]. These are still Zookeeper servers, but will also host etcd.''
 
- ''In August 2015, analytics10xx kafka brokers have been renamed to kafka10xx with the numbers (e.g. analytics1012 -> kafka1012).''
 
- ''In October 2015, analytics1011,1016,1019 [https://phabricator.wikimedia.org/T116656 were repurposed as aqs1001,1002,1003]. ''

Latest revision as of 20:17, 25 October 2019