You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Flink

From Wikitech-static
Revision as of 12:36, 17 December 2016 by imported>Mschwarzer (Added guide how to run an Apache Flink job on YARN)
Jump to navigation Jump to search

This page document setup of Flink on the math labs cluster.

Install Apache Flink and run job via YARN

  1. A working Hadoop cluster needs to be installed first (follow these instructions).
  2. Set up Apache Flink
  3. Run Flink job on YARN
    • WordCount.jar prints output to console flink/bin/flink run -m yarn-cluster -yn 1 -yjm 1024 -ytm 4096 flink/examples/batch/WordCount.jar
    • See official Flink YARN documentation for more details.