官方文档地址http://spark.apache.org/docs/latest/sql-getting-started.html
官方提供的测试数据文件http://mirrors.tuna.tsinghua.edu.cn/apache/spark/spark-2.4.3/spark-2.4.3-bin-hadoop2.7.tgz
解压当前文件夹examples/src/main/resources

SparkSession

通过SparkSession.builder()创建

  1. import org.apache.spark.sql.SparkSession
  3. val spark = SparkSession
  4.   .builder()
  5.   .appName("Spark SQL basic example")
  6.   .config("spark.some.config.option", "some-value")
  7.   .getOrCreate()
  9. // For implicit conversions like converting RDDs to DataFrames
  10. import spark.implicits._

读取json文件

  1. package cn.bx.spark
  3. import org.apache.spark.sql.{DataFrame, SparkSession}
  5. object SparkSessionApp {
  6.   def main(args: Array[String]): Unit = {
  7.     val spark: SparkSession = SparkSession.builder().appName("SparkSessionApp").master("local[*]").getOrCreate()
  8.     val people: DataFrame = spark.read.json("resources/people.json")
  9.     people.show()
  10.     spark.stop()
  11.   }
  12. }

打印结果

  1. +----+-------+
  2. | age|   name|
  3. +----+-------+
  4. |null|Michael|
  5. |  30|   Andy|
  6. |  19| Justin|
  7. +----+-------+