sparkcore进行wordcount词频统计

准备一份数据/home/admin/data/helloworld.txt

hello   world   hello
hello   world   welcome

scala> val wc = sc.textFile("file:///home/admin/data/helloworld.txt")
scala> wc.flatMap(x=>x.split("\t")).map(x=>(x,1)).reduceByKey(_+_).collect()
res19: Array[(String, Int)] = Array((hello,3), (welcome,1), (world,2))

按照词频排序

scala> wc.flatMap(x=>x.split("\t")).map(x=>(x,1)).reduceByKey(_+_).sortBy(_._2,true).collect().foreach(println(_))
(welcome,1)
(world,2)
(hello,3)