You are browsing a read-only backup copy of Wikitech. The live site can be found at

Analytics/Cluster/Refinery-source: Difference between revisions

From Wikitech-static
Jump to navigation Jump to search
No edit summary
m (Bot: Fixing double redirect to Analytics/Systems/Cluster/Deploy/Refinery-source)
(8 intermediate revisions by 6 users not shown)
Line 1: Line 1:
Refinery-source is the JVM software on the Analytics Cluster using spark or hive UDFs. The source code is in the [ analytics/refinery/source].
#REDIRECT [[Analytics/Systems/Cluster/Deploy/Refinery-source]]
= How to deploy =
{{Warning | You need archiva passwords in order to follow that procedure. Ask [[User:Ottomata|Andrew]] if needed. }}
* Update the file at the root of the repository whith changes that are going to be deployed.
*  Prepare deployment (change pom.xml files, push to git):
mvn release:prepare
* Check everything looks Ok:
* Actually deploy (jar generation and uploads to archiva)
mvn release:perform
* Download the new version of the jars from archiva ({{Warning | git fat uses IDs to manage files, so it is important to use the correct jars. }}) For convenience, the commonly updated jars are:
* NOTE: to download, click on the new version you just created, then Artifacts, then copy the link to the jar and update with and download that
* Copy the updated jars to the correct refinery path (somewhere like <refinery>/artifacts/org/wikimedia/analytics/refinery/)
* Update the symlinks in refinery/artifacts to the jars you just copied
* Make sure git fat is installed and configured according to the instructions in the Refinery README
* git add . && git commit (git fat will do some magic and replace the jar with a one-line id)
* push for review
Please see the [[Analytics/Cluster/Refinery|refinery page]] to deploy the jars and oozie code.

Latest revision as of 18:58, 13 July 2017