You are browsing a read-only backup copy of Wikitech. The live site can be found at


From Wikitech-static
< Analytics‎ | Cluster
Revision as of 22:29, 7 July 2016 by imported>Madhuvishy (Deployment instructions with Jenkins)
Jump to navigation Jump to search

Refinery-source is the JVM software on the Analytics Cluster using spark or hive UDFs. The source code is in the analytics/refinery/source.

How to deploy with Jenkins

  1. Update the file at the root of the repository with changes that are going to be deployed - commit and merge this change.
  2. Make sure you are logged in to Jenkins to see the pages linked in the steps below.
  3. Next step is to release a new version of the refinery-source jars to Maven. This can be done through our Jenkins setup, by visiting, everything required on the form should be prefilled - the only thing to edit is to check the Specify custom SCM tag checkbox, and change the version to vx.y.z(e.g v0.0.29) format instead of refinery-x.y.z which is the default populated tag on the form.
  4. Hit Schedule Maven Release Build. This will build all the jars and deploy them to Archiva, and once done send a job success email to Analytics-alerts.
  5. Now we need to update the symlinks to the latest refinery-source jars in the refinery repository. This can also be done via Jenkins, by going to, and supplying the version number of the latest jars released (e.g 0.0.29), and hitting Build. This will make a commit to refinery and send an email to Analytics-alerts on success.

How to deploy

  • Update the file at the root of the repository whith changes that are going to be deployed.
  • Prepare deployment (change pom.xml files, push to git):
mvn release:prepare
  • Check everything looks Ok:
  • Actually deploy (jar generation and uploads to archiva)
mvn release:perform

Please see the refinery page to deploy the jars and oozie code.