文件转成带schema的Parquert
case class Books(
id: Int,
isbn: String,
category: String,
publish_date: String,
publisher: String,
price: Float)
spark.read
.textFile("/user/test/data_1g/books/books")
.map(row =>row.split("\\|"))
.map(r => Books(r(0).toInt, r(1), r(2), r(3), r(4),r(5).toFloat))
.write
.parquet("/user/test/books_parquet_1g")
这可是最基础的东西了
注意：如果文本行是用“|”分隔的，那split()里面应该用\\|
