一 ambari 环境规划

1.1 ambari 版本的介绍

目前为止最新的版本是2.7.5 ，当然按照惯例，我们一般都会选择稳定版本或者说次新版本。其实2.7.5确实存在很多明显的bug。所以我选择了2.7.4的这个版本。
最后还是根据自己的需要时选择对应的版本：https://supportmatrix.hortonworks.com/

1.2 机器环境介绍

1.3 离线安装包下载地址

Ambari在线安装特别慢，所以使用离线安装，建议使用迅雷下载

注意：因为网路的原因，即使使用迅雷，也不一定很快。这里我把需要的文件和包都上传到百度云盘了，这样可以加速大家的下载。
链接: https://pan.baidu.com/s/1yxHRz_0lJsSgp9DIDctR5w 密码: fgcv

1.4 集群节点规划

这里我要提醒下，我这个是用docker 安装在一台机器上，因此所谓的16G 内存 100G硬盘都是共享的。

二 环境准备

我前面有提到 我们需要提前下载离线包（用迅雷），然后我们依据下好的离线包构建自己的本地源，即web服务。

2.1 下载离线包

地址上面我贴出来了，用迅雷下载即可。

2.2 docker run httpd

我们制作本地源只需要三步：

1. 拷贝我们前面下载的文件,到指定目录 (略)
2. 编写启动脚本
3. 启动
4. 校验是否成功

   2.2.1 总体操作预览

   目录结构预览
   
   conf 存放配置文件目录，后面存放配置文件 httpd.conf
   logs 存放日志目录，
   htdocs 存放离线包文件目录
   docker-compose.yml 启动脚本

看一下 htdocs 的里面内容吧：

四个包对应着我们前面下载的离线包，注意命名可能不能通，需要解压后重命名即可，
test.html 测试文件可有可无。

2.2.2 制作docker-compose.yml 文件

vi docker-compose.yml

version: '3'
services:
  httpd:
    image: httpd
    restart: always
    container_name: httpd
    privileged: true
    volumes:
      - /Users/hezhaoming/Documents/docker/httpd/htdocs/:/usr/local/apache2/htdocs/
      - /Users/hezhaoming/Documents/docker/httpd/conf/httpd.conf:/usr/local/apache2/conf/httpd.conf
      - /Users/hezhaoming/Documents/docker/httpd/logs/:/usr/local/apache2/logs/ 
    ports:
      - 80:80

环境 httpd.conf 配置文件

#
# This is the main Apache HTTP server configuration file.  It contains the
# configuration directives that give the server its instructions.
# See <URL:http://httpd.apache.org/docs/2.4/> for detailed information.
# In particular, see 
# <URL:http://httpd.apache.org/docs/2.4/mod/directives.html>
# for a discussion of each configuration directive.
#
# Do NOT simply read the instructions in here without understanding
# what they do.  They're here only as hints or reminders.  If you are unsure
# consult the online docs. You have been warned.  
#
# Configuration and logfile names: If the filenames you specify for many
# of the server's control files begin with "/" (or "drive:/" for Win32), the
# server will use that explicit path.  If the filenames do *not* begin
# with "/", the value of ServerRoot is prepended -- so "logs/access_log"
# with ServerRoot set to "/usr/local/apache2" will be interpreted by the
# server as "/usr/local/apache2/logs/access_log", whereas "/logs/access_log" 
# will be interpreted as '/logs/access_log'.

#
# ServerRoot: The top of the directory tree under which the server's
# configuration, error, and log files are kept.
#
# Do not add a slash at the end of the directory path.  If you point
# ServerRoot at a non-local disk, be sure to specify a local disk on the
# Mutex directive, if file-based mutexes are used.  If you wish to share the
# same ServerRoot for multiple httpd daemons, you will need to change at
# least PidFile.
#
ServerRoot "/usr/local/apache2"

#
# Mutex: Allows you to set the mutex mechanism and mutex file directory
# for individual mutexes, or change the global defaults
#
# Uncomment and change the directory if mutexes are file-based and the default
# mutex file directory is not on a local disk or is not appropriate for some
# other reason.
#
# Mutex default:logs

#
# Listen: Allows you to bind Apache to specific IP addresses and/or
# ports, instead of the default. See also the <VirtualHost>
# directive.
#
# Change this to Listen on specific IP addresses as shown below to 
# prevent Apache from glomming onto all bound IP addresses.
#
#Listen 12.34.56.78:80
Listen 80

#
# Dynamic Shared Object (DSO) Support
#
# To be able to use the functionality of a module which was built as a DSO you
# have to place corresponding `LoadModule' lines at this location so the
# directives contained in it are actually available _before_ they are used.
# Statically compiled modules (those listed by `httpd -l') do not need
# to be loaded here.
#
# Example:
# LoadModule foo_module modules/mod_foo.so
#
LoadModule mpm_event_module modules/mod_mpm_event.so
#LoadModule mpm_prefork_module modules/mod_mpm_prefork.so
#LoadModule mpm_worker_module modules/mod_mpm_worker.so
LoadModule authn_file_module modules/mod_authn_file.so
#LoadModule authn_dbm_module modules/mod_authn_dbm.so
#LoadModule authn_anon_module modules/mod_authn_anon.so
#LoadModule authn_dbd_module modules/mod_authn_dbd.so
#LoadModule authn_socache_module modules/mod_authn_socache.so
LoadModule authn_core_module modules/mod_authn_core.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_user_module modules/mod_authz_user.so
#LoadModule authz_dbm_module modules/mod_authz_dbm.so
#LoadModule authz_owner_module modules/mod_authz_owner.so
#LoadModule authz_dbd_module modules/mod_authz_dbd.so
LoadModule authz_core_module modules/mod_authz_core.so
#LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
#LoadModule authnz_fcgi_module modules/mod_authnz_fcgi.so
LoadModule access_compat_module modules/mod_access_compat.so
LoadModule auth_basic_module modules/mod_auth_basic.so
#LoadModule auth_form_module modules/mod_auth_form.so
#LoadModule auth_digest_module modules/mod_auth_digest.so
#LoadModule allowmethods_module modules/mod_allowmethods.so
#LoadModule isapi_module modules/mod_isapi.so
#LoadModule file_cache_module modules/mod_file_cache.so
#LoadModule cache_module modules/mod_cache.so
#LoadModule cache_disk_module modules/mod_cache_disk.so
#LoadModule cache_socache_module modules/mod_cache_socache.so
#LoadModule socache_shmcb_module modules/mod_socache_shmcb.so
#LoadModule socache_dbm_module modules/mod_socache_dbm.so
#LoadModule socache_memcache_module modules/mod_socache_memcache.so
#LoadModule socache_redis_module modules/mod_socache_redis.so
#LoadModule watchdog_module modules/mod_watchdog.so
#LoadModule macro_module modules/mod_macro.so
#LoadModule dbd_module modules/mod_dbd.so
#LoadModule bucketeer_module modules/mod_bucketeer.so
#LoadModule dumpio_module modules/mod_dumpio.so
#LoadModule echo_module modules/mod_echo.so
#LoadModule example_hooks_module modules/mod_example_hooks.so
#LoadModule case_filter_module modules/mod_case_filter.so
#LoadModule case_filter_in_module modules/mod_case_filter_in.so
#LoadModule example_ipc_module modules/mod_example_ipc.so
#LoadModule buffer_module modules/mod_buffer.so
#LoadModule data_module modules/mod_data.so
#LoadModule ratelimit_module modules/mod_ratelimit.so
LoadModule reqtimeout_module modules/mod_reqtimeout.so
#LoadModule ext_filter_module modules/mod_ext_filter.so
#LoadModule request_module modules/mod_request.so
#LoadModule include_module modules/mod_include.so
LoadModule filter_module modules/mod_filter.so
#LoadModule reflector_module modules/mod_reflector.so
#LoadModule substitute_module modules/mod_substitute.so
#LoadModule sed_module modules/mod_sed.so
#LoadModule charset_lite_module modules/mod_charset_lite.so
#LoadModule deflate_module modules/mod_deflate.so
#LoadModule xml2enc_module modules/mod_xml2enc.so
#LoadModule proxy_html_module modules/mod_proxy_html.so
#LoadModule brotli_module modules/mod_brotli.so
LoadModule mime_module modules/mod_mime.so
#LoadModule ldap_module modules/mod_ldap.so
LoadModule log_config_module modules/mod_log_config.so
#LoadModule log_debug_module modules/mod_log_debug.so
#LoadModule log_forensic_module modules/mod_log_forensic.so
#LoadModule logio_module modules/mod_logio.so
#LoadModule lua_module modules/mod_lua.so
LoadModule env_module modules/mod_env.so
#LoadModule mime_magic_module modules/mod_mime_magic.so
#LoadModule cern_meta_module modules/mod_cern_meta.so
#LoadModule expires_module modules/mod_expires.so
LoadModule headers_module modules/mod_headers.so
#LoadModule ident_module modules/mod_ident.so
#LoadModule usertrack_module modules/mod_usertrack.so
#LoadModule unique_id_module modules/mod_unique_id.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule version_module modules/mod_version.so
#LoadModule remoteip_module modules/mod_remoteip.so
#LoadModule proxy_module modules/mod_proxy.so
#LoadModule proxy_connect_module modules/mod_proxy_connect.so
#LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
#LoadModule proxy_http_module modules/mod_proxy_http.so
#LoadModule proxy_fcgi_module modules/mod_proxy_fcgi.so
#LoadModule proxy_scgi_module modules/mod_proxy_scgi.so
#LoadModule proxy_uwsgi_module modules/mod_proxy_uwsgi.so
#LoadModule proxy_fdpass_module modules/mod_proxy_fdpass.so
#LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so
#LoadModule proxy_ajp_module modules/mod_proxy_ajp.so
#LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
#LoadModule proxy_express_module modules/mod_proxy_express.so
#LoadModule proxy_hcheck_module modules/mod_proxy_hcheck.so
#LoadModule session_module modules/mod_session.so
#LoadModule session_cookie_module modules/mod_session_cookie.so
#LoadModule session_crypto_module modules/mod_session_crypto.so
#LoadModule session_dbd_module modules/mod_session_dbd.so
#LoadModule slotmem_shm_module modules/mod_slotmem_shm.so
#LoadModule slotmem_plain_module modules/mod_slotmem_plain.so
#LoadModule ssl_module modules/mod_ssl.so
#LoadModule optional_hook_export_module modules/mod_optional_hook_export.so
#LoadModule optional_hook_import_module modules/mod_optional_hook_import.so
#LoadModule optional_fn_import_module modules/mod_optional_fn_import.so
#LoadModule optional_fn_export_module modules/mod_optional_fn_export.so
#LoadModule dialup_module modules/mod_dialup.so
#LoadModule http2_module modules/mod_http2.so
#LoadModule proxy_http2_module modules/mod_proxy_http2.so
#LoadModule md_module modules/mod_md.so
#LoadModule lbmethod_byrequests_module modules/mod_lbmethod_byrequests.so
#LoadModule lbmethod_bytraffic_module modules/mod_lbmethod_bytraffic.so
#LoadModule lbmethod_bybusyness_module modules/mod_lbmethod_bybusyness.so
#LoadModule lbmethod_heartbeat_module modules/mod_lbmethod_heartbeat.so
LoadModule unixd_module modules/mod_unixd.so
#LoadModule heartbeat_module modules/mod_heartbeat.so
#LoadModule heartmonitor_module modules/mod_heartmonitor.so
#LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
#LoadModule asis_module modules/mod_asis.so
#LoadModule info_module modules/mod_info.so
#LoadModule suexec_module modules/mod_suexec.so
<IfModule !mpm_prefork_module>
    #LoadModule cgid_module modules/mod_cgid.so
</IfModule>
<IfModule mpm_prefork_module>
    #LoadModule cgi_module modules/mod_cgi.so
</IfModule>
#LoadModule dav_fs_module modules/mod_dav_fs.so
#LoadModule dav_lock_module modules/mod_dav_lock.so
#LoadModule vhost_alias_module modules/mod_vhost_alias.so
#LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
#LoadModule imagemap_module modules/mod_imagemap.so
#LoadModule actions_module modules/mod_actions.so
#LoadModule speling_module modules/mod_speling.so
#LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
#LoadModule rewrite_module modules/mod_rewrite.so

<IfModule unixd_module>
#
# If you wish httpd to run as a different user or group, you must run
# httpd as root initially and it will switch.  
#
# User/Group: The name (or #number) of the user/group to run httpd as.
# It is usually good practice to create a dedicated user and group for
# running httpd, as with most system services.
#
User daemon
Group daemon

</IfModule>

# 'Main' server configuration
#
# The directives in this section set up the values used by the 'main'
# server, which responds to any requests that aren't handled by a
# <VirtualHost> definition.  These values also provide defaults for
# any <VirtualHost> containers you may define later in the file.
#
# All of these directives may appear inside <VirtualHost> containers,
# in which case these default settings will be overridden for the
# virtual host being defined.
#

#
# ServerAdmin: Your address, where problems with the server should be
# e-mailed.  This address appears on some server-generated pages, such
# as error documents.  e.g. admin@your-domain.com
#
ServerAdmin you@example.com

#
# ServerName gives the name and port that the server uses to identify itself.
# This can often be determined automatically, but we recommend you specify
# it explicitly to prevent problems during startup.
#
# If your host doesn't have a registered DNS name, enter its IP address here.
#
#ServerName www.example.com:80

#
# Deny access to the entirety of your server's filesystem. You must
# explicitly permit access to web content directories in other 
# <Directory> blocks below.
#
<Directory />
    AllowOverride none
    Require all denied
</Directory>

#
# Note that from this point forward you must specifically allow
# particular features to be enabled - so if something's not working as
# you might expect, make sure that you have specifically enabled it
# below.
#

#
# DocumentRoot: The directory out of which you will serve your
# documents. By default, all requests are taken from this directory, but
# symbolic links and aliases may be used to point to other locations.
#
DocumentRoot "/usr/local/apache2/htdocs"
<Directory "/usr/local/apache2/htdocs">
    #
    # Possible values for the Options directive are "None", "All",
    # or any combination of:
    #   Indexes Includes FollowSymLinks SymLinksifOwnerMatch ExecCGI MultiViews
    #
    # Note that "MultiViews" must be named *explicitly* --- "Options All"
    # doesn't give it to you.
    #
    # The Options directive is both complicated and important.  Please see
    # http://httpd.apache.org/docs/2.4/mod/core.html#options
    # for more information.
    #
    Options Indexes FollowSymLinks

    #
    # AllowOverride controls what directives may be placed in .htaccess files.
    # It can be "All", "None", or any combination of the keywords:
    #   AllowOverride FileInfo AuthConfig Limit
    #
    AllowOverride None

    #
    # Controls who can get stuff from this server.
    #
    Require all granted
</Directory>

#
# DirectoryIndex: sets the file that Apache will serve if a directory
# is requested.
#
<IfModule dir_module>
    DirectoryIndex index.html
</IfModule>

#
# The following lines prevent .htaccess and .htpasswd files from being 
# viewed by Web clients. 
#
<Files ".ht*">
    Require all denied
</Files>

#
# ErrorLog: The location of the error log file.
# If you do not specify an ErrorLog directive within a <VirtualHost>
# container, error messages relating to that virtual host will be
# logged here.  If you *do* define an error logfile for a <VirtualHost>
# container, that host's errors will be logged there and not here.
#
ErrorLog /proc/self/fd/2

#
# LogLevel: Control the number of messages logged to the error_log.
# Possible values include: debug, info, notice, warn, error, crit,
# alert, emerg.
#
LogLevel warn

<IfModule log_config_module>
    #
    # The following directives define some format nicknames for use with
    # a CustomLog directive (see below).
    #
    LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
    LogFormat "%h %l %u %t \"%r\" %>s %b" common

    <IfModule logio_module>
      # You need to enable mod_logio.c to use %I and %O
      LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\" %I %O" combinedio
    </IfModule>

    #
    # The location and format of the access logfile (Common Logfile Format).
    # If you do not define any access logfiles within a <VirtualHost>
    # container, they will be logged here.  Contrariwise, if you *do*
    # define per-<VirtualHost> access logfiles, transactions will be
    # logged therein and *not* in this file.
    #
    CustomLog /proc/self/fd/1 common

    #
    # If you prefer a logfile with access, agent, and referer information
    # (Combined Logfile Format) you can use the following directive.
    #
    #CustomLog "logs/access_log" combined
</IfModule>

<IfModule alias_module>
    #
    # Redirect: Allows you to tell clients about documents that used to 
    # exist in your server's namespace, but do not anymore. The client 
    # will make a new request for the document at its new location.
    # Example:
    # Redirect permanent /foo http://www.example.com/bar

    #
    # Alias: Maps web paths into filesystem paths and is used to
    # access content that does not live under the DocumentRoot.
    # Example:
    # Alias /webpath /full/filesystem/path
    #
    # If you include a trailing / on /webpath then the server will
    # require it to be present in the URL.  You will also likely
    # need to provide a <Directory> section to allow access to
    # the filesystem path.

    #
    # ScriptAlias: This controls which directories contain server scripts. 
    # ScriptAliases are essentially the same as Aliases, except that
    # documents in the target directory are treated as applications and
    # run by the server when requested rather than as documents sent to the
    # client.  The same rules about trailing "/" apply to ScriptAlias
    # directives as to Alias.
    #
    ScriptAlias /cgi-bin/ "/usr/local/apache2/cgi-bin/"

</IfModule>

<IfModule cgid_module>
    #
    # ScriptSock: On threaded servers, designate the path to the UNIX
    # socket used to communicate with the CGI daemon of mod_cgid.
    #
    #Scriptsock cgisock
</IfModule>

#
# "/usr/local/apache2/cgi-bin" should be changed to whatever your ScriptAliased
# CGI directory exists, if you have that configured.
#
<Directory "/usr/local/apache2/cgi-bin">
    AllowOverride None
    Options None
    Require all granted
</Directory>

<IfModule headers_module>
    #
    # Avoid passing HTTP_PROXY environment to CGI's on this or any proxied
    # backend servers which have lingering "httpoxy" defects.
    # 'Proxy' request header is undefined by the IETF, not listed by IANA
    #
    RequestHeader unset Proxy early
</IfModule>

<IfModule mime_module>
    #
    # TypesConfig points to the file containing the list of mappings from
    # filename extension to MIME-type.
    #
    TypesConfig conf/mime.types

    #
    # AddType allows you to add to or override the MIME configuration
    # file specified in TypesConfig for specific file types.
    #
    #AddType application/x-gzip .tgz
    #
    # AddEncoding allows you to have certain browsers uncompress
    # information on the fly. Note: Not all browsers support this.
    #
    #AddEncoding x-compress .Z
    #AddEncoding x-gzip .gz .tgz
    #
    # If the AddEncoding directives above are commented-out, then you
    # probably should define those extensions to indicate media types:
    #
    AddType application/x-compress .Z
    AddType application/x-gzip .gz .tgz

    #
    # AddHandler allows you to map certain file extensions to "handlers":
    # actions unrelated to filetype. These can be either built into the server
    # or added with the Action directive (see below)
    #
    # To use CGI scripts outside of ScriptAliased directories:
    # (You will also need to add "ExecCGI" to the "Options" directive.)
    #
    #AddHandler cgi-script .cgi

    # For type maps (negotiated resources):
    #AddHandler type-map var

    #
    # Filters allow you to process content before it is sent to the client.
    #
    # To parse .shtml files for server-side includes (SSI):
    # (You will also need to add "Includes" to the "Options" directive.)
    #
    #AddType text/html .shtml
    #AddOutputFilter INCLUDES .shtml
</IfModule>

#
# The mod_mime_magic module allows the server to use various hints from the
# contents of the file itself to determine its type.  The MIMEMagicFile
# directive tells the module where the hint definitions are located.
#
#MIMEMagicFile conf/magic

#
# Customizable error responses come in three flavors:
# 1) plain text 2) local redirects 3) external redirects
#
# Some examples:
#ErrorDocument 500 "The server made a boo boo."
#ErrorDocument 404 /missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html
#

#
# MaxRanges: Maximum number of Ranges in a request before
# returning the entire resource, or one of the special
# values 'default', 'none' or 'unlimited'.
# Default setting is to accept 200 Ranges.
#MaxRanges unlimited

#
# EnableMMAP and EnableSendfile: On systems that support it, 
# memory-mapping or the sendfile syscall may be used to deliver
# files.  This usually improves server performance, but must
# be turned off when serving from networked-mounted 
# filesystems or if support for these functions is otherwise
# broken on your system.
# Defaults: EnableMMAP On, EnableSendfile Off
#
#EnableMMAP off
#EnableSendfile on

# Supplemental configuration
#
# The configuration files in the conf/extra/ directory can be 
# included to add extra features or to modify the default configuration of 
# the server, or you may simply copy their contents here and change as 
# necessary.

# Server-pool management (MPM specific)
#Include conf/extra/httpd-mpm.conf

# Multi-language error messages
#Include conf/extra/httpd-multilang-errordoc.conf

# Fancy directory listings
#Include conf/extra/httpd-autoindex.conf

# Language settings
#Include conf/extra/httpd-languages.conf

# User home directories
#Include conf/extra/httpd-userdir.conf

# Real-time info on requests and configuration
#Include conf/extra/httpd-info.conf

# Virtual hosts
#Include conf/extra/httpd-vhosts.conf

# Local access to the Apache HTTP Server Manual
#Include conf/extra/httpd-manual.conf

# Distributed authoring and versioning (WebDAV)
#Include conf/extra/httpd-dav.conf

# Various default settings
#Include conf/extra/httpd-default.conf

# Configure mod_proxy_html to understand HTML4/XHTML1
<IfModule proxy_html_module>
Include conf/extra/proxy-html.conf
</IfModule>

# Secure (SSL/TLS) connections
#Include conf/extra/httpd-ssl.conf
#
# Note: The following must must be present to support
#       starting without SSL on platforms with no /dev/random equivalent
#       but a statically compiled-in mod_ssl.
#
<IfModule ssl_module>
SSLRandomSeed startup builtin
SSLRandomSeed connect builtin
</IfModule>

2.2.3 启动服务

docker-compose  up -d

2.2.4 验证服务

验证成功，注意不要使用127.0.0.1地址，需要用真实的本机ip地址

2.3 准备 repo文件

我们前面把本地源服务搭建起来了，接下来我们编写源文件

2.3.1 总体操作预览

目录结构预览：

总体步骤为两步：

1. 新增文件夹，我都防止repo里面，以备后面使用

2. 编写三个 repo

   2.3.2 编写 repo

   2.3.2.1 编写 ambari.repo

   vi ambari.repo

   #VERSION_NUMBER=2.7.4.0-118
   [ambari-2.7.4.0]
   #json.url = http://public-repo-1.hortonworks.com/HDP/hdp_urlinfo.json
   name=ambari Version - ambari-2.7.4.0
   baseurl=http://10.111.105.50/ambari/centos7/2.7.4.0-118/
   gpgcheck=1
   gpgkey=http://10.111.105.50/ambari/centos7/2.7.4.0-118/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
   enabled=1
   priority=1
   

   2.3.2.2 编写 HDP.repo

   vi HDP.repo

   #VERSION_NUMBER=3.1.4.0-315
   [HDP-3.1.4.0]
   name=HDP Version - HDP-3.1.4.0
   baseurl=http://10.111.105.50/HDP/centos7/3.1.4.0-315/
   gpgcheck=1
   gpgkey=http://10.111.105.50/HDP/centos7/3.1.4.0-315/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
   enabled=1
   priority=1
   

   2.3.2.3 编写 HDP-UTILS.repo

   vi HDP-UTILS.repo

   [HDP-UTILS-1.1.0.22]
   name=HDP-UTILS Version - HDP-UTILS-1.1.0.22
   baseurl=http://10.111.105.50/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/
   gpgcheck=1
   gpgkey=http://10.111.105.50/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
   enabled=1
   priority=1
   

   三 构建ambari镜像

   3.1 构建sephora/ambari_os:7 底层镜像

   3.1.1 总体操作预览

   目录结构预览：
   
   总体而言也是四步：

3. 新增一个文件夹 ambari_os （略）

4. 拷贝前面的离线文件到目录下 jdk （略）
5. 编写 dockerfile
6. build dockerfile

   3.1.2 编写 dockerfile

   ambari_os是基于centos:7然后他将来是做为 ambari-server 、ambari-agent的底层镜像。它主要包括了一些常用的工具组件，为了测试方便，我把很多工具都装上了，你们可以自行取舍。然后这个要配置个ssh免密登录。
   注意：构建镜像的命令、每一个我都写好注释了，可以根据自身需求，或增或减
   vi docker-compose.yml ```dockerfile FROM centos:7

   作者信息 烟雨楼:18915152022@163.com

   MAINTAINER yanyulou:18915152022@163.com

   将 yum 源更换一下，速度更快，本地包可以不用配置

   RUN yum -y install wget \ && mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup\ && curl -o /etc/yum.repos.d/CentOS-Base.repo http://mirrors.aliyun.com/repo/Centos-7.repo\ && yum makecache\ && yum -y update

install 小工具。ssh时必须的，其他都可以去除，但是为了测试调试，建议加上这些常用的工具。

RUN /bin/cp -f /usr/share/zoneinfo/Asia/Shanghai /etc/localtime \ && groupadd -r admin && useradd -g admin admin \ && yum install -y git \ && yum install -y zsh \ && yum install -y which \ && yum install -y openssh-server \ && yum install -y man \ && yum install -y dstat \ && yum install -y unzip \ && yum install -y nc \ && yum install -y tar \ && yum install -y perl \ && yum install -y file \ && yum install -y crontabs

install java

RUN mkdir /usr/local/jdk ADD jdk-8u241-linux-x64.tar.gz /usr/local/jdk ENV JAVA_HOME /usr/local/jdk/jdk1.8.0_241 ENV JRE_HOME /usr/local/jdk/jdk1.8.0_241/jre ENV PATH $JAVA_HOME/bin:$PATH

kerberos client initscripts (service wrapper for servicectl) is need othewise the Ambari is unable to start mysql

RUN yum install -y krb5-workstation && yum install -y initscripts && yum -y install ntp

设置 ssh

RUN mkdir /var/run/sshd RUN echo ‘root:1234’ |chpasswd RUN sed -ri ‘s/^PermitRootLogin\s+./PermitRootLogin yes/‘ /etc/ssh/sshd_config RUN sed -i ‘s/.StrictHostKeyChecking ask/StrictHostKeyChecking no/‘ /etc/ssh/ssh_config

直接在此处设置免密登录,便于后续的操作。

RUN ssh-keygen -t rsa -P ‘’ -f ~/.ssh/id_rsa && \ cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys && \ chmod 700 ~/.ssh && \ chmod 600 ~/.ssh/id_rsa && \ chmod 644 ~/.ssh/authorized_keys RUN ssh-keygen -t rsa -f /etc/ssh/ssh_host_rsa_key \ && ssh-keygen -t dsa -f /etc/ssh/ssh_host_dsa_key\ && ssh-keygen -t ed25519 -f /etc/ssh/ssh_host_ecdsa_key

setting up systemd 50行-63行官网提供的方案，解决systemd无效的情况

ENV container docker RUN (cd /lib/systemd/system/sysinit.target.wants/; for i in ; do [ $i == systemd-tmpfiles-setup.service ] || rm -f $i; done); \ rm -f /lib/systemd/system/multi-user.target.wants/;\ rm -f /etc/systemd/system/.wants/;\ rm -f /lib/systemd/system/local-fs.target.wants/; \ rm -f /lib/systemd/system/sockets.target.wants/udev; \ rm -f /lib/systemd/system/sockets.target.wants/initctl; \ rm -f /lib/systemd/system/basic.target.wants/;\ rm -f /lib/systemd/system/anaconda.target.wants/; RUN yum install -y systemd EXPOSE 22 VOLUME [ “/sys/fs/cgroup” ] ENTRYPOINT [“/usr/sbin/init”]

注意生产环境建议使用openjdk8<br />![image.png](https://cdn.nlark.com/yuque/0/2020/png/1502688/1597721586586-573da86f-a13c-44bb-99d2-d464e7b45781.png#align=left&display=inline&height=204&margin=%5Bobject%20Object%5D&name=image.png&originHeight=408&originWidth=1638&size=98517&status=done&style=none&width=819)
<a name="v8qub"></a>
### 3.1.3  build sephora/ ambari_os:7
构建os7镜像，执行构建镜像命令。注意在dockerfile 同级目录下执行命令
```dockerfile
docker build -t sephora/ambari_os:7 .

验证结果

注意：构建成功后，我们可以run 起来，验证环境 jdk 、ssh好不好使。然后再操作下一步。

3.2 构建 sephora/ambari_agent:2.7.4 镜像

3.2.1总体操作概览

目录结构预览：

总体而言分成四步：

1. 新建一个ambari_agent 文件夹 （略）
2. 拷贝三个源文件 （略）
3. 编写dockerfile

4. 启动脚本

   3.2.2 编写 dockerfile

   vi docker-compose.yml

   FROM sephora/ambari_os:7
   # 作者信息 烟雨楼:18915152022@163.com
   MAINTAINER yanyulou:18915152022@163.com
   # 拷贝本地源 repo
   COPY ambari.repo /etc/yum.repos.d/
   COPY HDP.repo /etc/yum.repos.d/
   COPY HDP-UTILS.repo /etc/yum.repos.d/
   # install ambari-agent
   RUN yum install -y ambari-agent && systemctl enable sshd && systemctl enable ntpd
   # Hdfs ports
   EXPOSE 50010 50020 50070 50075 50090 8020 9000
   # Mapred ports
   EXPOSE 19888
   # Yarn ports
   EXPOSE 8030 8031 8032 8033 8040 8042 8088
   # Other ports
   EXPOSE 49707 2122
   

   我存放的位置:
   

   3.2.3 build sephora/ ambari_agent:2.7.4

   构建ambari_agent镜像，在dockerfile同级目录下执行

   docker build -t sephora/ambari_agent:2.7.4 .
   

   验证结果
   

   3.3 构建 sephora/ambari_server:2.7.4 镜像

   3.3.1 总体操作预览

   目录结构预览：
   
   一共9个文件，3个源文件，两个离线包文件，还有四个脚本。
   总体而言分7步完成：

5. 新增一个文件夹 ambari_server (略)

6. 拷贝三个源文件 + 两个离线文件 (略)
7. 编写flag.conf 存储一些环境变量
8. 编写ambari-setup.sh 脚本，初始化脚本
9. 编写 init.sh 初始化启动脚本
10. 编写dockerfile
11. 启动

    3.3.2 编写自动化脚本

    3.3.2.1 自动设置脚本 setup.sh

    当我们进入容器(server服务器)，执行ambari setup 后需要执行的操作（这个是网上旧版本的设置操作，与新版略有不同）帮组我们理解setup.sh脚本。
    
    上面的手工操作变成下面的脚本。
    vi ambari-setup.sh

    #!/usr/bin/expect
    # excute interactive command
    spawn echo "*******************start ambari-server setup*****************"
    set timeout 300
    set javahome /usr/local/jdk/jdk1.8.0_241
    set mysqlport 3306
    set databasename ambari
    set username root
    set dbpass 1234     
    spawn ambari-server setup
    #expect {
    expect "daemon*"
    send "n\r"
    expect "*?"
    send "y\r"
    expect "choice (*"
    send "2\r"
    expect "JAVA_HOME:"
    send "$javahome\r"
    expect "LZO packages*"
    send "y\r"
    expect "configuration*"
    send "y\r"
    expect "choice (*"
    send "3\r"
    expect "Hostname*"
    send "\r"
    expect "Port (*"
    send "$mysqlport\r"
    expect "Database name (*"
    send "$databasename\r"
    expect "Username (*"
    send "$username\r"
    expect "Database Password (*"
    send "$dbpass\r"
    expect "Re-enter *"
    send "$dbpass\r"
    expect "properties*"
    send "y\r"
    expect eof
    

    3.3.2.2 环境变量脚本 flag.conf

    值得注意的事情，是flag.conf只是一个配置文件控制初始化数据只执行一次，其实它也只能执行一次，我加个条件判断防止误操作。
    vi flag.conf

    #初始化数据参数1：表示需要初始化，0表示不需要初始化
    requireInit=1
    

    3.3.2.3 初始化脚本 init.sh

    这个脚本主要干两个事情，一个是初始化mysql数据库，另一个是启动server服务。值得注意的是这个设置初始化的步骤只需要再第一次启动的时候配置。
    vi init.sh

    #!/bin/bash
    requireInit=`sed '/^requireInit=/!d;s/.*=//' /opt/flag.conf`;
    echo print-requireInit: $requireInit ....
    if [ $requireInit -eq 1 ];
    then
    echo start init....;
    sed -i 's/^requireInit=.*/requireInit=0/' /opt/flag.conf;
    expect /usr/bin/ambari-setup.sh;
    mysql -u root <<EOF
    set password for 'root'@'localhost' = password('1234');
    grant all privileges on *.* to 'root'@'%' identified by '1234';
    grant all privileges on *.* to 'root'@'localhost' identified by '1234';
    grant all privileges on *.* to 'root'@'master' identified by '1234';
    UPDATE mysql.user  t1 SET t1.Grant_priv = 'Y',t1.Super_priv = 'Y' WHERE t1.USER = 'root';
    CREATE DATABASE ambari;
    use ambari;
    CREATE USER 'ambari'@'%' IDENTIFIED BY '1234';
    GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
    CREATE USER 'ambari'@'localhost' IDENTIFIED BY '1234';
    GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost';
    CREATE USER 'ambari'@'master' IDENTIFIED BY '1234';
    GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'master';
    FLUSH PRIVILEGES;
    source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
    show tables;
    use mysql;
    CREATE DATABASE hive;
    use hive;
    CREATE USER 'hive'@'%' IDENTIFIED BY 'hive';
    GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
    CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive';
    GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost';
    CREATE USER 'hive'@'master' IDENTIFIED BY 'hive';
    GRANT ALL PRIVILEGES ON *.* TO 'hive'@'master';
    FLUSH PRIVILEGES;
    CREATE DATABASE oozie;
    use oozie;
    CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie';
    GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
    CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie';
    GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost';
    CREATE USER 'oozie'@'master' IDENTIFIED BY 'oozie';
    GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'master';
    FLUSH PRIVILEGES;
    EOF
    echo init sql success...
    fi
    echo 启动ambari-server....
    ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
    ambari-server start;
    systemctl enable ambari-server;
    

3.3.3 编写 dockerfile

这里面有许多坑，要反复测试验证最好才能得到的配置。
vi docker-compose.yml

FROM sephora/ambari_os:7
# 作者信息 烟雨楼:18915152022@163.com
MAINTAINER yanyulou:18915152022@163.com
# add ambari repo
COPY ambari.repo /etc/yum.repos.d/
COPY HDP.repo /etc/yum.repos.d/
ADD mysql-community-release-el7-5.noarch.rpm /opt/
RUN rpm -ivh /opt/mysql-community-release-el7-5.noarch.rpm
RUN yum install -y  mysql-community-server && yum install -y ambari-server && yum install -y expect && systemctl enable sshd && systemctl enable ntpd
COPY mysql-connector-java-5.1.40.jar /opt/
COPY init.sh /opt/
COPY flag.conf /opt/
RUN mkdir /usr/share/java/
RUN cp /opt/mysql-connector-java-5.1.40.jar /usr/share/java/mysql-connector-java.jar
RUN cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/
RUN echo "" >> /etc/ambari-server/conf/ambari.properties
RUN echo "server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar" >> /etc/ambari-server/conf/ambari.properties
COPY ambari-setup.sh /usr/bin/ambari-setup.sh
RUN chmod 777 /usr/bin/ambari-setup.sh
RUN chmod 777 /opt/init.sh
RUN chmod 777 /opt/flag.conf
RUN chmod +x  /opt/init.sh && chmod +x /etc/rc.d/rc.local
RUN echo /opt/init.sh >> /etc/rc.d/rc.local 
EXPOSE 8080

3.3.4 build sephora/ambari_server:2.7.4

构建 ambari_server 镜像，在dockerfile同级目录执行命令

docker build -t sephora/ambari_server:2.7.4 .

查看验证结果

四 docker run nginx

我们利用Nginx 访问内部docker 内部的ip

4.1 总体操作预览

总体分5步：

1. 新增文件夹nginx （略）
2. 新增logs 日志文件夹 、docs 文件夹 、conf 文件夹（略）
3. 编写docker-compose.yml 脚本
4. 编写 配置文件脚本 nginx.conf
5. 启动服务

4.2 编写docker-compose.yml

注意的是这里的网卡一定要和我自定义的网卡一致（因此 我们自定义的ambar集群先启动，他会创建自定义网卡），我们可以通过docker network ls 查看

version: '3'
services:
  httpd:
    image: nginx
    restart: always
    container_name: nginx-web
    privileged: true
    networks: 
      ambari_ambari-net:
        ipv4_address: 172.33.0.5
    volumes:
      - /Users/hezhaoming/Documents/docker/nginx/conf/nginx.conf:/etc/nginx/nginx.conf
      - /Users/hezhaoming/Documents/docker/nginx/logs:/var/log/nginx
    ports:
      - 18080:80
networks:
   ambari_ambari-net:
    external: true

4.3 编写nginx.conf 配置文件

这个我们目前需要一个ip代理，即访问127.0.0.1:8080(也就是访问172.33.0.5:80,这个docker端口映射帮我们做的)，需要访问目标地址172.33.0.2：8080，因此我们需要一个代理server。请看30行-39行。

    user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    server{
       listen 80;
       charset utf-8;
       server_name 172.33.0.5;

       location / {
          proxy_pass http://172.33.0.2:8080;
          proxy_redirect default;
       }
    }

    include /etc/nginx/conf.d/*.conf;
}

4.4 启动服务

注意：在dockerfile 同级目录执行

docker-compose up -d

验证结果。当然这个组件我已经装好了。至于如何安装配置后面有介绍。

五 Ambari 服务编排

目录结构预览：

就一个编排文件

5.1 编写 docker-compose.yml

version: '3' 
services:
  amb_master:
    image: "sephora/ambari_server:2.7.4" 
    container_name: amb_master 
    hostname: master 
    privileged: true
    ulimits:
      nproc: 65535
      nofile:
        soft: 20000
        hard: 40000
    extra_hosts:
      - "slave1:172.33.0.3"
      - "slave2:172.33.0.4"
    networks: 
      ambari-net:
        ipv4_address: 172.33.0.2 
  amb_agent1:
    image: "sephora/ambari_agent:2.7.4" 
    container_name: amb_slave1 
    hostname: slave1 
    privileged: true
    ulimits:
      nproc: 65535
      nofile:
        soft: 20000
        hard: 40000
    extra_hosts:
      - "master:172.33.0.2"
      - "slave2:172.33.0.4"
    networks: 
      ambari-net:
        ipv4_address: 172.33.0.3 
  amb_agent2:
    image: "sephora/ambari_agent:2.7.4" 
    container_name: amb_slave2 
    hostname: slave2 
    privileged: true
    ulimits:
      nproc: 65535
      nofile:
        soft: 20000
        hard: 40000
    extra_hosts:
      - "master:172.33.0.2"
      - "slave1:172.33.0.3"
    networks: 
      ambari-net:
        ipv4_address: 172.33.0.4 
networks:
   ambari-net:
    driver: bridge
    ipam:
      driver: default
      config:
        - subnet: 172.33.0.1/24

5.2 启动服务

docker-compose up -d

5.3 查看服务运行情况

启动成功，再启动nginx ，然后稍等1分钟。访问127.0.0.1:18080
用户秘密是 admin/admin

六 大数据组件安装

6.1 step0 start

创建一个集群

新增集群名称

6.2 step1 选择版本

使用本地源

http://10.111.105.50/HDP/centos7/3.1.4.0-315/

http://127.0.0.1/HDP-GPL/centos7/3.1.4.0-315/

http://10.111.105.50/HDP-UTILS/HDP-UTILS/centos7/1.1.0.22/

centsos7 选择 redhat7 ,其他的remove

6.3 step2 设置集群参数

设置ssh

设置主机地址 三个主机 分别是 master 、 slave1、 slave2

6.4 step3 确认主机

all pass

6.5 step4 选择服务

6.6 step5 Assign Masters

默认即可

6.7 step6 Assign Slaves and Clients

6.8 step7 Customize Services

6.9 step8 review

6.10 step9 install test

6.11 end

常见服务都ok，一个红色的警告no care

值得注意的是，我刚开始all run 是不行的，后面我查看项目结构的源码，得知他们的依赖关系，一个一个run是OK的。(估计是资源的问题，导致服务启动失败吧)