准备json数据

users.json

  1. [{"name":"张三" ,"age":18} ,{"name":"李四" ,"age":15}]

注意,必须得是一行,不能是换行的.

Maven依赖

  1.  <dependencies>
  2.         <dependency>
  3.             <groupId>org.apache.spark</groupId>
  4.             <artifactId>spark-sql_2.11</artifactId>
  5.             <version>2.1.1</version>
  6.         </dependency>
  8.     </dependencies>
  9.     <build>
  10.         <plugins>
  11.             <!-- 打包插件, 否则 scala 类不会编译并打包进去 -->
  12.             <plugin>
  13.                 <groupId>net.alchim31.maven</groupId>
  14.                 <artifactId>scala-maven-plugin</artifactId>
  15.                 <version>3.4.6</version>
  16.                 <executions>
  17.                     <execution>
  18.                         <goals>
  19.                             <goal>compile</goal>
  20.                             <goal>testCompile</goal>
  21.                         </goals>
  22.                     </execution>
  23.                 </executions>
  24.             </plugin>
  25.         </plugins>
  26.     </build>

代码

  2. import org.apache.spark.sql.SparkSession
  3. object CreateDataFrame {
  4.   def main(args: Array[String]): Unit = {
  6.     val sparkSession = SparkSession.builder()
  7.       .appName("CreateDataFrame")
  8.       .master("local[2]")
  9.       .getOrCreate()
  11.     val dataFrame = sparkSession.read.json("E:\\ZJJ_SparkSQL\\demo01\\src\\main\\resources\\users.json")
  12.     dataFrame.createOrReplaceTempView("user")
  13.     dataFrame.cache()
  15.     sparkSession.sql("select * from user").show()
  17.     sparkSession.stop()
  20.   }
  22. }

结果

  1. +---+----+
  2. |age|name|
  3. +---+----+
  4. | 18|  张三|
  5. | 15|  李四|
  6. +---+----+

码云地址

https://gitee.com/crow1/ZJJ_SparkSQL/blob/master/demo01/src/main/java/com/CreateDataFrame.scala