You are browsing a read-only backup copy of Wikitech. The primary site can be found at wikitech.wikimedia.org

Analytics/Cluster/Refinery: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
imported>QChris
m (QChris moved page Analytics/Refinery to Analytics/Cluster/Refinery: Refinery is too tied to the cluster to live directly underneath Analytics)
 
imported>Milimetric
 
(11 intermediate revisions by 8 users not shown)
Line 1: Line 1:
Refinery is the software infrastructure that is used on the [[Analytics/Cluster|Analytics Cluster]]. The source code is in the [http://git.wikimedia.org/summary/analytics%2Frefinery.git analytics/source repository].
#REDIRECT [[Analytics/Systems/Cluster/Deploy/Refinery]]
 
== How to deploy ==
 
# Ssh into [[Tin]]
# Run:
#:
#:  <code>cd /srv/deployment/analytics/refinery</code>
#:  <code>git deploy start</code>
#:  <code>git checkout master</code>
#:  <code>git pull</code>
#:  <code>git deploy sync</code>
#:<br />
#:(<code>git deploy sync</code> will complain that only “2/3 minions completed fetch”. You can say “y”es to that)
#:<br/>
#:This part brings the refinery code from gerrit to <code>stat1002</code>.
# Ssh into [[stat1002]]
# Run <code>sudo -u hdfs /srv/deployment/analytics/refinery/bin/refinery-deploy-to-hdfs --verbose --no-dry-run</code>
#:<br>
#:This part brings the refinery code to the HDFS (but it does not resubmit Oozie jobs).
 
=== How to deploy Oozie jobs ===
 
Please see the [[Analytics/Cluster/Oozie#How_to_deploy_Oozie_production_jobs|Deployment section in the Oozie docs]].

Latest revision as of 13:15, 10 July 2017