Apache Spark does not support native CSV output on disk.

You have 4 available solutions though:

1.  You can convert your Dataframe into an RDD :

方式一：

def convertToReadableString(r : Row) = ???

df.rdd.map{ convertToReadableString }.saveAsTextFile(filepath)

This will create a folder filepath. Under the file path, you'll find partitions files (e.g part-000*)

What I usually do if I want to append all the partitions into a big CSV is

cat filePath/part* > mycsvfile.csv

Some will use coalesce(1,false) to create one partition from the RDD. It's usually a bad practice, since it may overwhelm the driver by pulling all the data you are collecting to it.

Note that df.rdd will return an RDD[Row].

2.With Spark <2, you can use databricks spark-csv library:

Spark 1.4+:

方式二：

df.write.format("com.databricks.spark.csv").save(filepath)

Spark 1.3:

方式三：

df.save(filepath,"com.databricks.spark.csv")

With Spark 2.x the spark-csv package is not needed as it's included in Spark.

方式四：

df.write.format("csv").save(filepath)

You can convert to local Pandas data frame and use to_csv method (PySpark only).

Note: Solutions 1, 2 and 3 will result in CSV format files (part-*) generated by the underlying Hadoop API that Spark calls when you invoke save. You will have one part- file per partition.

另存为txt文件

方式一：

bank.rdd.repartition(1).saveAsTextFile("/tmp/df2.txt")

note: bank is a DataFrame

原文地址：

https://stackoverflow.com/questions/33174443/how-to-save-a-spark-dataframe-as-csv-on-disk

https://community.hortonworks.com/questions/42838/storage-dataframe-as-textfile-in-hdfs.html