从零搭建SpringCloud Demo及其拓展

使用 SpringCloud Alibaba + Zuul 搭建一个基本的 SpringCloud 应用.

目标:

  • 1、跑起来
  • 2、拓展 Zuul 的各种规则
  • 3、魔改 Nacos 减少 Http 调用量.

版本信息:

  • 1、SpringBoot 2.0.4.RELEASE
  • 2、SpringCloud Finchley.RELEASE
  • 3、SpringCloud Alibaba 2.1.0.RELEASE

前提:

  • 1、搭建 Nacos 集群环境
  • 2、配置 Nacos VIP 环境

搭建demo

首先使用 Idea 搭建一个基本的 Maven 多模块应用. 可以是 SpringBoot , Spring+Tomcat , Spring 的版本也没有限制.如下所示

  1.     <modules>
  2.         <module>gateway-spring3</module>
  3.         <module>gateway-spring4</module>
  4.         <module>gateway-spring5</module>
  5.         <module>gateway-springboot-1.x</module>
  6.         <module>gateway-springboot-2.x</module>
  7.         <module>gateway-zuul</module>
  8.         <module>gateway-spring-dubbo</module>
  9.         <module>gateway-springboot-dubbo</module>
  10.         <module>gateway-dubbo-common</module>
  11.     </modules>

注册Spring + Tomcat,以 Spring3 为例子，其他版本同理. 加入Spring3的依赖，加入单独的 Nacos 依赖，注册bean. Nacos 配置如下

  1.     <dependency>
  2.         <groupId>com.alibaba.nacos</groupId>
  3.         <artifactId>nacos-client</artifactId>
  4.         <version>1.0.1</version>
  5.     </dependency>
  1.     <nacos:global-properties endpoint="${nacos.endpoint:127.0.0.1}"/>

后配置Conponent,分别添加 @PostConstruct 进行服务注册，@PreDestroy 进行服务注销，同理可使用 Spring 提供的接口进行注册和注销

注册 SpringBoot 2.x , 加入 SpringCloud Alibaba discovery 依赖,配置 bootstrap.properties

  1.     <dependency>
  2.         <groupId>com.alibaba.cloud</groupId>
  3.         <artifactId>spring-cloud-starter-alibaba-nacos-discovery</artifactId>
  4.     </dependency>
  1.     server.port=8088
  2.     spring.application.name=gateway.springboot.2.x
  3.     spring.cloud.nacos.discovery.endpoint=127.0.0.1
  4.     spring.cloud.nacos.discovery.metadata.path=gateway.springboot.2.x

因为starter会自动注册服务，所以无须手动调用 Nacos 进行注册

注册 Zuul ,加入 SpringCloud Alibaba nacos-config 和 nacos-discovery, spring-cloud-starter-netflix-zuul 依赖，增加 SpringCloud和Zuul 启动注解

  1.     @SpringBootApplication
  2.     @EnableDiscoveryClient
  3.     @EnableZuulProxy
  4.     public class ZuulGatewayApplication {
  5.            public static void main(String[] args) {
  6.            SpringApplication.run(ZuulGatewayApplication.class);
  7.           }
  8.     }

为了获取注册到 Nacos 中的服务(在 zuul 中被称之为路由(route), 需要定时从 Nacos 集群中进行全量 service list 查询. 然后遍历每一个服务获取到 服务实例 返回给 zuul 使用.

讲述 zuul 会称之为 路由route , 讲述其他部分会称之为 服务service.

参考文章1，通过自定义 NewZuulRouteLocator 实现 RefreshableRouteLocator 达到动态刷新 route 的效果. 加载完 Nacos 中的实例数据之后，再将其转换成 ZuulProperties.ZuulRoute.

  1.      // 通过http 请求获取全部的ServiceInstance，如果服务多，那么产生的http请求会更多
  2.     private List<ZuulProperties.ZuulRoute> listenerNacos(List<ServiceInstance> serviceInstanceList) {
  3.         List<ZuulProperties.ZuulRoute> entities = new ArrayList<>();
  4.         for (ServiceInstance serviceInstance : serviceInstanceList) {
  5.             ZuulProperties.ZuulRoute zuulRoute = new ZuulProperties.ZuulRoute();
  6.             zuulRoute.setId(serviceInstance.getServiceId());
  7.             zuulRoute.setServiceId(serviceInstance.getServiceId());
  8.             zuulRoute.setPath("/" + serviceInstance.getServiceId() + "/**");
  9.             ConfigurationManager.getConfigInstance().setProperty("hystrix.command." + zuulRoute.getServiceId() + ".execution.isolation.thread.timeoutInMilliseconds", serviceInstance.getMetadata().getOrDefault("timeout", "3000"));
  10.         }
  11.         return entities;
  12.     }

分别启动 Nacos 集群. Spring+Tomcat 应用, SpringBoot 应用 , 执行以下 curl 确定服务是否已经注册上去.(⚠️注意替换服务名)

  1. curl -X GET 127.0.0.1:8848/nacos/v1/ns/instance/list?serviceName=nacos.test.1

观察到如下结果显示注册成功, 注册失败可查看启动日志是否正常.

  1. ➜ curl -X GET 'http://127.0.0.1:8848/nacos/v1/ns/service/list?pageNo=1&pageSize=100'
  2. {"count":1,"doms":["gateway.spring3"]}%

验证网关功能是否可用
分别执行如下 curl 操作即可验证

  1. ➜  ~ curl 'http://127.0.0.1:9912/zuul/gateway.spring3/gateway/spring3.json'

拓展 zuul 的配置

Zuul 的配置是通过 RibbonClientConfiguration 配置类进行引入的，通过该配置，每一个服务都可以拥有一个 IClientConfig IRule 等等。 这一步通过 @RibbonClients 对这些配置类进行覆盖.

  • 定义配置类
  1.     @Configuration
  2.     @EnableConfigurationProperties
  3.     @ConditionalOnBean({SpringClientFactory.class})
  4.     @ConditionalOnRibbonNacos
  5.     @ConditionalOnNacosDiscoveryEnabled
  6.     @AutoConfigureAfter({RibbonAutoConfiguration.class})
  7.     @RibbonClients(defaultConfiguration = {DemoRibbonClientConfiguration.class})
  8.     public class DemoNacosAutoConfiguration {}

实际配置类

  1.     @Configuration
  2.     public class DemoRibbonClientConfiguration {
  4.     @Bean
  5.     @LoadBalanced
  6.     public RestTemplate getRestTemplate() {
  7.         return new RestTemplate();
  8.     }
  10.     @Bean
  11.     public IRule rule(IClientConfig config) {
  12.         RaycloudRule raycloudRule = new RaycloudRule();
  13.         raycloudRule.initWithNiwsConfig(config);
  14.         return raycloudRule;
  15.     }
  17.     @Bean
  18.     @ConditionalOnMissingBean
  19.     public ServerListUpdater ribbonServerListUpdater(IClientConfig config) {
  20.         return new MyPoolUpdater(config);
  21.     }
  23.     @Bean
  24.     @ConditionalOnMissingBean
  25.     public ILoadBalancer ribbonLoadBalancer(IClientConfig config, ServerList serverList, ServerListFilter serverListFilter, IRule rule, IPing ping, ServerListUpdater serverListUpdater) {
  26.         if (this.propertiesFactory.isSet(ILoadBalancer.class, "client")) {
  27.             return this.propertiesFactory.get(ILoadBalancer.class, config, "client");
  28.         }
  29.         return new MyLoad(config, rule, ping, serverList, serverListFilter, serverListUpdater);
  30.     }
  32.     @Autowired
  33.     private PropertiesFactory propertiesFactory;
  34. }

这里分别覆盖了 IRule ServerListUpdater ILoadBalancer ， 可以根据自己的需求分别进行处理

  • 引入配置文件 META-INF/spring.factories
  1. org.springframework.boot.autoconfigure.EnableAutoConfiguration=com.netflix.loadbalancer.DemoNacosAutoConfiguration
  • 自定义 Bean 进行覆盖

这里介绍下为什么引入 ServerListUpdater , 先看代码.

  1. public class MyPoolUpdater extends PollingServerListUpdater {
  3.     public MyPoolUpdater(IClientConfig clientConfig) {
  4.         super(init(clientConfig));
  5.     }
  7.     public static IClientConfig config = null;
  9.     @Autowired
  10.     private SpringClientFactory springClientFactory;
  12.     private static IClientConfig init(IClientConfig clientConfig) {
  13.         config = clientConfig;
  14.         return clientConfig;
  15.     }
  17.     private UpdateAction updateAction;
  19.     @Override
  20.     public synchronized void start(final UpdateAction updateAction) {
  21.         super.start(updateAction);
  22.         this.updateAction = updateAction;
  23.     }
  25.     @Override
  26.     public synchronized void stop() {
  27.         super.stop();
  28.         if (springClientFactory != null) {
  29.             try {
  30.                 Field contexts = springClientFactory.getClass().getSuperclass().getDeclaredField("contexts");
  31.                 contexts.setAccessible(true);
  32.                 Map<String, AnnotationConfigApplicationContext> contextMap = (Map<String, AnnotationConfigApplicationContext>) contexts.get(springClientFactory);
  33.                 AnnotationConfigApplicationContext annotationConfigApplicationContext = contextMap.remove(config.getClientName());
  34.                 if (annotationConfigApplicationContext != null) {
  35.                     annotationConfigApplicationContext.close();
  36.                 }
  37.             } catch (Exception e) {
  38.                 e.printStackTrace();
  39.             }
  40.         }
  41.     }
  43.     public UpdateAction getUpdateAction() {
  44.         return updateAction;
  45.     }
  46. }

核心就在 stop 中，在 stop 的时候，将该服务对应 Spring Context 进行销毁.

魔改 Nacos 减少 Http 调用量

在上述代码中，可以看到，为了获取全部的 ServiceInstance 需要对 Nacos 的服务进行 遍历. 如果服务有上千个，这样会生成上千个http请求. 因此我们可以通过 Memcache Redis 进行同步，两端比较 Service 的 checksum , 前提是 Zuul 和 Nacos 共享一个 缓存集群 .

  • Nacos端进行改造，在 Service#onChange 时进行 checksum 比较，如果 checksum 不一致，那么可以可以将 checksum 存入 缓存, 同时将服务的 实例明细 存入 缓存 即可.
  1. try {
  2.     String result;
  3.     try {
  4.         MessageDigest md5 = MessageDigest.getInstance("MD5");
  5.         result = new BigInteger(1, md5.digest((ipsString.toString()).getBytes(Charset.forName("UTF-8")))).toString(16);
  6.     } catch (Exception e) {
  7.         Loggers.SRV_LOG.error("[NACOS-DOM] error while calculating checksum(md5)", e);
  8.         result = RandomStringUtils.randomAscii(32);
  9.     }
  10.     checksum = result;
  11.     //魔改之后的代码
  12.     if (!oldCheckSum.equalsIgnoreCase(result)) {
  13.         MemcachedClient memcachedClient = memcachedClient();
  14.         memcachedClient.set(KeyBuilder.buildServiceMetaKey(getNamespaceId(), getName().split("@@")[1]), 3600, checksum);
  15.         try {
  16.             System.out.println("写入缓冲明细:" + JSONObject.toJSONString(covertTo(ips)));
  17.             memcachedClient.set(KeyBuilder.buildServiceMetaKey(getNamespaceId(), getName().split("@@")[1]) + "_detail", 3600, JSONObject.toJSONString(covertTo(ips)));
  18.         } catch (Throwable e) {
  19.             e.printStackTrace();
  20.         }
  21.     }
  22. } catch (Exception e) {
  23.     Loggers.SRV_LOG.error("[NACOS-DOM] error while calculating checksum(md5)", e);
  24.     checksum = RandomStringUtils.randomAscii(32);
  25. }
  • Zuul 端对 NewZuulRouteLocator 进行改造，启动一个本地定时任务 . 定时做 Nacos 全量同步(可以事件跨度放大一点，例如1分钟)，定时比较 checksum ，发现不一致进行替换. 代码如下:
  1. @Service
  2. @Slf4j
  3. public class NacosTask implements InitializingBean {
  4.     /**
  5.      * namespace <===> 服务 <===> namespace很少
  6.      */
  7.     public static Map<String, Set<String>> NAMESPACE_SERVER_SET = new ConcurrentHashMap<>(8);
  8.     /**
  9.      * namespace <===>服务 <===> checksum
  10.      */
  11.     public static Map<String, Map<String, String>> CHECKSUM_MAP = new ConcurrentHashMap<>(8);
  13.     /**
  14.      * namespace <===> 服务 <===> 实例集合
  15.      */
  16.     public static Map<String, Map<String, Instances>> INSTANCES_MAP = new ConcurrentHashMap<>(8);
  18.     private final static String DEFAULT_NAMESPACE = "public";
  20.     static {
  21.         NAMESPACE_SERVER_SET.put(DEFAULT_NAMESPACE, new CopyOnWriteArraySet<>());
  22.         CHECKSUM_MAP.put(DEFAULT_NAMESPACE, new ConcurrentHashMap<>());
  23.         INSTANCES_MAP.put(DEFAULT_NAMESPACE, new ConcurrentHashMap<>());
  24.     }
  26.     @Resource
  27.     private MemcachedClient memcachedClient;
  28.     @Resource
  29.     private DiscoveryClient compositeDiscoveryClient;
  30.     @Resource
  31.     private NacosDiscoveryProperties nacosProperties;
  33.     private NamingProxy namingProxy;
  35.     public NamingProxy getNamingProxy() {
  36.         if (namingProxy != null) return namingProxy;
  37.         NamingProxy namingProxy;
  38.         try {
  39.             NacosNamingService namingService = (NacosNamingService) nacosProperties.namingServiceInstance();
  40.             Field serverProxy = namingService.getClass().getDeclaredField("serverProxy");
  41.             serverProxy.setAccessible(true);
  42.             namingProxy = (NamingProxy) serverProxy.get(namingService);
  43.         } catch (NoSuchFieldException | IllegalAccessException e) {
  44.             throw new RuntimeException(e);
  45.         }
  46.         this.namingProxy = namingProxy;
  47.         return this.namingProxy;
  48.     }
  50.     @Override
  51.     public void afterPropertiesSet() {
  52.         executorService.scheduleWithFixedDelay(() -> {
  53.             //获取服务集合===>ServiceController(/nacos/v1/ns/service/list)==>ketSet
  54.             List<String> services = compositeDiscoveryClient.getServices();
  55.             log.debug("[start handle sche task][services:{}][services:{}]", services.size(), JSONArray.toJSON(services));
  56.             NAMESPACE_SERVER_SET.get(DEFAULT_NAMESPACE).addAll(services);
  57.             List<String> removeKey = new ArrayList<>();
  58.             //s===>gateway.spring5 coreKey===> com.alibaba.nacos.naming.domains.meta.public##gateway.spring5
  59.             for (String s : NAMESPACE_SERVER_SET.get(DEFAULT_NAMESPACE)) {
  60.                 try {
  61.                     String coreKey = KeyBuilder.buildServiceMetaKey(DEFAULT_NAMESPACE, s);
  62.                     String remoteCheckSum = memcachedClient.get(coreKey);
  63.                     log.debug("[start handle][key:{}][remoteCheckSum:{}]", coreKey, remoteCheckSum);
  64.                     //如果不为空
  65.                     if (!StringUtils.isEmpty(remoteCheckSum)) {
  66.                         log.info("[checksum changed][service:{}]", s);
  67.                         //比较是否一致
  68.                         Map<String, String> checksumMap = CHECKSUM_MAP.getOrDefault(coreKey, new ConcurrentHashMap<>());
  69.                         String localCheckSum = checksumMap.getOrDefault(coreKey, "d");
  70.                         if (!localCheckSum.equalsIgnoreCase(remoteCheckSum)) {
  71.                             checksumMap.put(coreKey, remoteCheckSum);
  72.                             //拉明细,解析明细直接覆盖
  73.                             String detail = memcachedClient.get(coreKey + "_detail");
  74.                             if (detail != null) {
  75.                                 INSTANCES_MAP.getOrDefault(DEFAULT_NAMESPACE, new ConcurrentHashMap<>()).put(coreKey, new Instances(JSONArray.parseArray(detail, Instance.class)));
  76.                             }
  77.                         }
  78.                     } else {
  79.                         removeKey.add(s);
  80.                         removeKey.add(coreKey);
  81.                     }
  82.                 } catch (Throwable e) {
  83.                     e.printStackTrace();
  84.                 }
  85.             }
  86.             //memcache挂掉不会跑到这里来
  87.             for (String s : removeKey) {
  88.                 NAMESPACE_SERVER_SET.getOrDefault(DEFAULT_NAMESPACE, new CopyOnWriteArraySet<>()).remove(s);
  89.                 INSTANCES_MAP.getOrDefault(DEFAULT_NAMESPACE, new ConcurrentHashMap<>()).remove(s);
  90.                 CHECKSUM_MAP.getOrDefault(DEFAULT_NAMESPACE, new ConcurrentHashMap<>()).remove(s);
  91.             }
  92.         }, 0, 5, TimeUnit.SECONDS);
  93.         ALL_SERVICE.scheduleWithFixedDelay(this::handleAllData, 0, 1, TimeUnit.MINUTES);
  94.     }
  96.     private void handleAllData() {
  97.         try {
  98.             String allFromNacos = getAllFromNacos();
  99.             Map<String, Datum> stringDatumMap = PropertiesAssemble.deserializeMap(allFromNacos);
  100.             if (stringDatumMap != null && stringDatumMap.size() > 0) {
  101.                 log.debug("[handle all sync data from nacos][size:{}]", stringDatumMap.size());
  102.                 stringDatumMap.forEach((key, value) -> {
  103.                     String serviceName = KeyBuilder.getServiceName(key);
  104.                     String namespaceId = KeyBuilder.getNamespace(key);
  105.                     Instances instances = value.instances;
  106.                     String coreKey = KeyBuilder.buildServiceMetaKey(namespaceId, serviceName.split("@@")[1]);
  107.                     INSTANCES_MAP.getOrDefault(namespaceId, new ConcurrentHashMap<>()).put(coreKey, instances);
  108.                 });
  109.             } else {
  110.                 log.warn("[sync data from nacos is empty]");
  111.             }
  112.         } catch (Exception e) {
  113.             log.error("[sync nacos all data occur exception][error:{}]", e.getMessage(), e);
  114.         }
  115.     }
  117.     private String getAllFromNacos() throws Exception {
  118.         NamingProxy namingProxy = getNamingProxy();
  119.         Assert.notNull(namingProxy, "namingProxy can't be null");
  120.         Exception e = null;
  121.         List<String> serverListFromEndpoint = namingProxy.getServerListFromEndpoint();
  122.         for (String server : serverListFromEndpoint) {
  123.             try {
  124.                 return getAllData(server);
  125.             } catch (Exception ex) {
  126.                 e = ex;
  127.             }
  128.         }
  129.         throw e == null ? new RuntimeException("获取nacos全量数据失败") : e;
  130.     }
  132.     private static String getAllData(String server) throws Exception {
  133.         HttpClient.HttpResult result = HttpClient.httpGet(String.format("http://%s/nacos/v1/ns/distro/datums", server),
  134.                 new ArrayList<>(), new HashMap<>(8), "utf-8");
  136.         if (HttpURLConnection.HTTP_OK == result.code) {
  137.             return result.content;
  138.         }
  139.         throw new IOException("failed to req API: " + String.format("http://%s/nacos/v1/ns/distro/datums", server) + ". code: " + result.code + " msg: " + result.content);
  140.     }
  142.     private ScheduledExecutorService executorService = Executors.newSingleThreadScheduledExecutor(r -> {
  143.         Thread thread = new Thread(r);
  144.         thread.setName("scan-nacos");
  145.         thread.setDaemon(true);
  146.         return thread;
  147.     });
  150.     private ScheduledExecutorService ALL_SERVICE = Executors.newSingleThreadScheduledExecutor(r -> {
  151.         Thread thread = new Thread(r);
  152.         thread.setName("all_in_scan");
  153.         thread.setDaemon(true);
  154.         return thread;
  155.     });

参考文章:

(1) spring cloud zuul使用记录
(2) 路由接入流程以及并发刷新问题

2019-08-25