You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org
Analytics/Cluster/Hardware
Jump to navigation
Jump to search
Information about the Analytics Cluster hardware and infrastructure.
Docs
Hardware Specs
By Host
| Type | # | Hosts | Processors | Memory | Disk |
|---|---|---|---|---|---|
| Dell PowerEdge R420 | 3 | analytics1001, analytics1002, analytics1003 | 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz | 64G RAM | 4 * 2T |
| Dell PowerEdge R720 | 12 | aqs100{1,2,3},kafka10{12,13,14,18,20,22} | 12 core EW-2620 @ 2.00GHz | 48G RAM | 24T = 12 * 2T |
| Dell PowerEdge R720 | 29 | analytics1028 - analytics1057 | 12 core EW-2620 @ 2.00GHz | 64G RAM | 48T = 12 * 4T, + 2 SSDs |
By Machine Type
| Type | Dell PowerEdge R420 | Dell PowerEdge R720 | Dell PowerEdge R720 |
|---|---|---|---|
| Hosts | analytics1001, analytics1002, analytics1003 | aqs100{1,2,3},kafka10{12,13,14,18,20,22} | analytics1028-analytics1057 |
| Processors | 16 core Intel(R) Xeon(R) CPU E5-2450 v2 @ 2.50GHz | 12 core EW-2620 @ 2.00GHz | 12 core EW-2620 @ 2.00GHz |
| Memory | 64G RAM | 48G RAM | 64G RAM |
| Disk | 4 * 2T | 24T = 12 * 2T | 48T = 12 * 4T + 2 SSDs |
Roles
| Hosts | # | Type | Role |
|---|---|---|---|
| analytics1001 | 2 | R420 | Primary NameNode, Hadoop master node. |
| analytics1002 | 2 | R420 | Standby NameNode |
| analytics1003 | 1 | R420 | Hive, Oozie, MySQL, Camus crons, other crons, HDFS balancer |
| analytics1028-analytics1057 | 30 | R720 | Hadoop Worker Nodes (DataNodes) |
| kafka1012,kafka1013,kafka1014,kafka1018,kafka1020,kafka1022 | 6 | R720 | Kafka Brokers |
| thorium | 1 | Analytics webservice box. Hue. | |
| aqs1001-aqs1003 | 3 | R720 | Analytics Query Service (RESTBase + Cassandra) |
Notes:
- As of June 2015, analytics102[345] which were Zookeeper servers in the Analytics Cluster have been moved out into the general purpose production network and renamed as conf100[123]. These are still Zookeeper servers, but will also host etcd.
- In August 2015, analytics10xx kafka brokers have been renamed to kafka10xx with the numbers (e.g. analytics1012 -> kafka1012).
- In October 2015, analytics1011,1016,1019 were repurposed as aqs1001,1002,1003.