本文讲述的是如何在IDEA里编写一个计算PI的Spark程序。

主要内容：

• 1.安装Scala
• 2.编写ScalaPI

相关文章：
1.Spark之PI本地
2.Spark之WordCount集群
3.SparkStreaming之读取Kafka数据
4.SparkStreaming之使用redis保存Kafka的Offset
5.SparkStreaming之优雅停止
6.SparkStreaming之写数据到Kafka
7.Spark计算《西虹市首富》短评词云

1.安装Scala

1.安装Scala

下载地址：传送门 （2.11.12）

下载完成解压到任意目录，设置环境变量，直接在Path后加入解压目录

打开控制台输入scala

说明安装成功

2.IDEA安装Scala插件

打开IDEA，File->settings->Plugins->Browse Repositoies->搜索Scala->Install然后重启IDEA即可

2.编写ScalaPI

2.1.导入依赖

<dependencies>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.11</artifactId>
            <version>2.3.0</version>
            <!--<scope>provided</scope>-->
        </dependency>
        <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-sql_2.11 -->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_2.11</artifactId>
            <version>2.3.0</version>
            <!--<scope>provided</scope>-->
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <version>2.15.2</version>
                <executions>
                    <execution>
                        <id>scala-compile-first</id>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                        <configuration>
                            <includes>
                                <include>**/*.scala</include>
                            </includes>
                        </configuration>
                    </execution>
                    <execution>
                        <id>scala-test-compile</id>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

2.2.编写代码

object ScalaPi {
  def main(args: Array[String]) {
    //创建一个Config
    val conf =
      new SparkConf()
        .setAppName("ScalaPI")
        .setMaster("local")

    //核心创建SparkContext对象
    val sc = new SparkContext(conf)

    //计算PI
    val count = sc.parallelize(1 to 10000).filter { _ =>
      val x = math.random
      val y = math.random
      x * x + y * y < 1
    }.count()
    println(s"Pi is roughly ${4.0 * count / 10000}")

    //停止SparkContext对象
    sc.stop()
  }
}

运行结果如下：