资源规划

组件 bigdata-node1 bigdata-node2 bigdata-node3
OS centos7.6 centos7.6 centos7.6
JDK jvm jvm jvm
Zeppelin ZeppelinServer N.A N.A

安装介质

  1. 版本:zeppelin-0.9.0-preview1-bin-all.tgz<br /> 下载:[https://mirror.bit.edu.cn/apache/zeppelin/zeppelin-0.9.0-preview1/zeppelin-0.9.0-preview1-bin-all.tgz](https://mirror.bit.edu.cn/apache/zeppelin/zeppelin-0.9.0-preview1/zeppelin-0.9.0-preview1-bin-all.tgz)

安装Zeppelin

解压缩

  1. cd /share
  2. wget https://mirror.bit.edu.cn/apache/zeppelin/zeppelin-0.9.0-preview1/zeppelin-0.9.0-preview1-bin-all.tgz
  3. tar -zxvf zeppelin-0.9.0-preview1-bin-all.tgz -C ~/modules/
  4. rm -rf zeppelin-0.9.0-preview1-bin-all.tgz

零配置启动

Zeppelin在不用做任何配置修改的情况下即可正常启动。进入bin目录下,执行启动命令。

  1. cd ~/modules/zeppelin-0.9.0-preview1-bin-all/bin
  2. ./zeppelin-daemon.sh start

如果显示如下结果,则表示启动正常:

  1. Zeppelin start [ OK ]

此时,只能在安装本机进行浏览器访问。

  1. curl http://127.0.0.1:8080

Web UI:http://127.0.0.1:8080

自定义配置

访问端口

  1. cd ~/modules/zeppelin-0.9.0-preview1-bin-all/conf
  2. cp zeppelin-site.xml.template zeppelin-site.xml
  3. vi zeppelin-site.xml

修改如下配置:

  1. <property>
  2. <name>zeppelin.server.addr</name>
  3. <value>0.0.0.0</value>
  4. <description>Server address.</description>
  5. </property>
  6. <property>
  7. <name>zeppelin.server.port</name>
  8. <value>9527</value>
  9. <description>Server port.</description>
  10. </property>

用户认证

Zeppelin默认运行匿名用户访问,即没有用户权限要求,如要实现用户权限限制,则需修改zeppelin-site.xml和shiro配置文件。

  1. 修改zeppelin-site.xml配置文件,将以下配置项中的“true”改成“false”。

    1. cd ~/modules/zeppelin-0.9.0-preview1-bin-all/conf
    2. vi zeppelin-site.xml

    配置如下:

    1. <property>
    2. <name>zeppelin.anonymous.allowed</name>
    3. <value>false</value>
    4. <description>Anonymous user allowed by default</description>
    5. </property>
  2. 新增shiro权限配置。

    1. cd ~/modules/zeppelin-0.9.0-preview1-bin-all/conf
    2. cp shiro.ini.template shiro.ini
    3. vi shiro.ini

    配置如下:

    1. [users]
    2. admin = password1, admin

    注意:逗号前面是用户名,逗号后面是登录密码。

  3. 重新启动Zeppelin并验证。

    1. cd ~/modules/zeppelin-0.9.0-preview1-bin-all/bin
    2. ./zeppelin-daemon.sh restart

    重启后,刷新Web页面,发现已经看不到之前的应用Notebook,点击右上角的Login按钮,出现登录框,输入用户和密码登录。
    Web UI:http://bigdata-node1:9527
    账户口令:admin/password1
    注意:当进入页面右上角显示为“**anonymous”(匿名)状态时,请点击“anonymous”,之后单击Interpreter”或者“Notebook Repos”或者“Configuration”均可进入登录页面。**

    Interpreter

    1. Flink Interpreter

  • 配置(Web UI)

    1. FLINK_HOME /home/vagrant/modules/flink-1.10.0
    2. HADOOP_CONF_DIR /home/vagrant/modules/hadoop-2.7.2/etc/hadoop
    3. HIVE_CONF_DIR/home /home/vagrant/modules/apache-hive-2.3.4-bin/conf
    4. flink.execution.mode remote
    5. flink.execution.remote.host 192.168.0.101
    6. flink.execution.remote.port 8381
    7. zeppelin.flink.enableHive true
    8. zeppelin.flink.hive.version 2.3.4
  • 验证 ```scala %flink

// 示例(Flink批处理) val data = benv.fromElements(“hello world”, “hello flink”, “hello hadoop”) data.flatMap(line => line.split(“\s”)) .map(w => (w, 1)) .groupBy(0) .sum(1) .print()

  1. <a name="AToGX"></a>
  2. ### 2. Spark Interpreter
  3. - **配置**
  4. 1. 配置zeppelin-env.sh。
  5. ```bash
  6. export JAVA_HOME=/home/vagrant/modules/jdk1.8.0_221
  7. export MASTER=spark://bigdata-node1:7077
  8. export SPARK_HOME=/home/vagrant/modules/spark-2.0.0
  9. export HADOOP_CONF_DIR=/home/vagrant/modules/hadoop-2.7.2/etc/hadoop
  1. 配置Spark Interpreter(Web UI)。
    1. master spark://bigdata-node1:7077
  • master配置
    • local[*] in local mode
    • spark://master:7077 in standalone cluster
    • yarn-client in Yarn client mode
    • yarn-cluster in Yarn cluster mode
    • mesos://host:5050 in Mesos cluster
  1. 配置file(HDFS,Web UI)
    1. hdfs.url hdfs://bigdata-node1:9000
    2. hdfs.user vagrant
  • 数据准备 ```scala import org.apache.commons.io.IOUtils import java.net.URL import java.nio.charset.Charset

val bankText = sc.parallelize( IOUtils.toString( new URL(“https://s3.amazonaws.com/apache-zeppelin/tutorial/bank/bank.csv“), Charset.forName(“utf8”)).split(“\n”))

case class Bank(age: Integer, job: String, marital: String, education: String, balance: Integer)

val bank = bankText.map(s => s.split(“;”)).filter(s => s(0) != “\”age\””).map( s => Bank(s(0).toInt, s(1).replaceAll(“\””, “”), s(2).replaceAll(“\””, “”), s(3).replaceAll(“\””, “”), s(5).replaceAll(“\””, “”).toInt ) ).toDF() bank.registerTempTable(“bank”)

  1. - **验证**
  2. ```scala
  3. %sql
  4. select age, count(1) value
  5. from bank
  6. where age < 30
  7. group by age
  8. order by age

3. JDBC Interpreter

■ MySQL

  • 依赖

    1. cp /home/vagrant/modules/apache-hive-2.3.4-bin/lib/mysql-connector-java-*.jar /home/vagrant/modules/zeppelin-0.9.0-preview1-bin-all/interpreter/jdbc/
  • 配置(Web UI)

    1. default.url jdbc:mysql://bigdata-node3:3306
    2. default.user root
    3. default.password 123456
    4. default.driver com.mysql.jdbc.Driver
  • 验证 ```sql %mysql

— show databases; — use mysql; show tables;

  1. <a name="kx858"></a>
  2. #### ■ Hive
  3. - **依赖**
  4. ```bash
  5. cp /home/vagrant/modules/apache-hive-2.3.4-bin/jdbc/hive-jdbc-2.3.4-standalone.jar /home/vagrant/modules/zeppelin-0.9.0-preview1-bin-all/interpreter/jdbc/
  • 配置

    1. default.url jdbc:hive2://bigdata-node1:10000
    2. default.driver org.apache.hive.jdbc.HiveDriver
    3. default.user hive2
  • 验证 ```sql %hive

show databases; — use default; — show tables; ```

参考

CSDN:Apache Zeppelin主要界面和基本操作
https://blog.csdn.net/majianxiong_lzu/article/details/89318434
简书:Spark & Zeppelin
https://www.jianshu.com/p/297c3893d7e7
简书:如何在Apache Zeppelin中玩转Spark (1)
https://www.jianshu.com/p/fd049b2887c1