SBT

Show sbt Version

This is NOT the right way to check sbt's version, it shows the project version:

$ sbt version

This is how to check sbt's version

$ sbt sbtVersion

To print out more info, try

$ sbt about

Use Local Maven Repository

Add Resolver.mavenLocal to resolvers so that all the jars in ~/.m2 folder can be loaded:

resolvers += Resolver.mavenLocal

%% vs %

• %%: auto-append scala version
• %: no scala version appended

libraryDependencies += "org.apache.spark" %% "spark-mllib" % "1.4.1"

is equivalent to

libraryDependencies += "org.apache.spark" % "spark-mllib_2.10" % "1.4.1"

Specify Classifier

E.g. by default pig is built against Hadoop 1.0, to get the jar for Hadoop 2.0, we need to add h2 as classifier

In sbt, add classifier "h2" to the end

libraryDependencies += "org.apache.pig" % "pig" % "0.14.0.2.2.4.0-2633" classifier "h2"

or specify the full url:

libraryDependencies += "org.apache.pig" % "pig" % "0.14.0.2.2.4.0-2633" from "http://repo.hortonworks.com/service/local/repositories/releases/content/org/apache/pig/pig/0.14.0.2.2.4.0-2633/pig-0.14.0.2.2.4.0-2633-h2.jar"

Assembly

Build the fat jar:

Add this to projects/assembly.sbt:

addSbtPlugin("com.eed3si9n" % "sbt-assembly" % "0.14.3")

And execute:

$ sbt assembly

To skip tests, add this to build.sbt:

test in assembly := {}