You are browsing a read-only backup copy of Wikitech. The live site can be found at wikitech.wikimedia.org

Nova Resource:Math/Flink

From Wikitech-static
< Nova Resource:Math
Revision as of 02:48, 30 July 2017 by imported>BryanDavis (BryanDavis moved page Flink to Nova Resource:Math/Flink without leaving a redirect)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page document setup of Flink on the math labs cluster.

Install Apache Flink and run job via YARN

  1. A working Hadoop cluster needs to be installed first (follow these instructions).
  2. Sign in to master node (e.g. hadoop000)
  3. Set up Apache Flink
  4. Run Flink job on YARN
    • WordCount.jar prints output to console flink/bin/flink run -m yarn-cluster -yn 1 -yjm 1024 -ytm 4096 flink/examples/batch/WordCount.jar
    • See official Flink YARN documentation for more details.