别老扯什么Hadoop了，你的数据根本不够大 - 极客头条 - CSDN.NET
http://geek.csdn.net/news/detail/2780
我推荐使用Scalding，而不是Hive或者Pig，因为你可以用Scala语言来写级联Hadoop任务，隐藏了MapReduce底层细节。
twitter/scalding: A Scala API for Cascading
https://github.com/twitter/scalding
Scalding is a Scala library that makes it easy to specify Hadoop MapReduce jobs. Scalding is built on top of Cascading, a Java library that abstracts away low-level Hadoop details. Scalding is comparable to Pig, but offers tight integration with Scala, bringing advantages of Scala to your MapReduce jobs.
package com.twitter.scalding.examples
import com.twitter.scalding._
import com.twitter.scalding.source.TypedText
class WordCountJob(args: Args) extends Job(args) {
TypedPipe.from(TextLine(args("input")))
.flatMap { line => tokenize(line) }
.groupBy { word => word } // use each word for a key
.size // in each group, get the size
.write(TypedText.tsv(String, Long))
// Split a piece of text into individual words.
def tokenize(text: String): Array[String] = {
// Lowercase each word and remove punctuation.
text.toLowerCase.replaceAll("[^a-zA-Z0-9\s]", "").split("\s+")
}
}
