Sentinel规则Pull模式持久化

时间:2022-07-22
本文章向大家介绍Sentinel规则Pull模式持久化，主要内容包括其使用实例、应用技巧、基本知识点总结和需要注意事项，具有一定的参考价值，需要的朋友可以参考一下。

阅读文本大概需要3分钟。

前一篇【使用Nacos存储Sentinel的限流规则】讲了基于Nacos的Push模式持久化，这里讲下基于本地文件的Pull模式持久化。在网上看到一篇讲这个讲得不错的： 

https://blog.csdn.net/weixin_42437633/article/details/106443342

从官网的说明 

https://github.com/alibaba/Sentinel/wiki/在生产环境中使用-Sentinel#Pull模式

有如下一张图，总觉得例子欠缺些什么？？

琢磨一下，发现原来无论官方的例子，还是网友的例子都没有结合Sentinel讲解规则的Pull。现在开始操作，按照官方的步骤开始操作： 

https://github.com/alibaba/Sentinel/wiki/动态规则扩展

在微服务的pom.xml文件引入

<dependency>
    <groupId>com.alibaba.csp</groupId>
    <artifactId>sentinel-datasource-extension</artifactId>
    <version>x.y.z</version>
</dependency>

不知道哪里出了问题，到最后也没有搞通？而且还有如下疑问

  • 微服务到底连接哪个Sentinel服务？如果没有配置，可能可以连接默认的Sentinel服务
  • 如果Sentinel服务的ip和port变了呢？如何配置？也没有找到具体说明？

带着这两个疑问开始学习之旅。因为基于springboot和springcloud讲解，所有Spring Cloud Alibaba这么好的技术，为什么不用呢？

0x01：新建项目olive-pull-sentinel-datasource

  • pom.xml文件引入
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>com.sentinel</groupId>
    <artifactId>olive-pull-sentinel-datasource</artifactId>
    <version>0.0.1-SNAPSHOT</version>
    <packaging>jar</packaging>

    <parent>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-parent</artifactId>
        <version>2.1.3.RELEASE</version>
        <relativePath /> <!-- lookup parent from repository -->
    </parent>

    <name>olive-pull-sentinel-datasource</name>
    <url>http://maven.apache.org</url>

    <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
            <java.version>1.8</java.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>

        <dependency>
            <groupId>com.alibaba.cloud</groupId>
            <artifactId>spring-cloud-starter-alibaba-sentinel</artifactId>
            <version>2.2.1.RELEASE</version>
        </dependency>

        <dependency>
            <groupId>com.alibaba.csp</groupId>
            <artifactId>sentinel-parameter-flow-control</artifactId>
            <version>1.7.1</version>
        </dependency>


          <dependency>
            <groupId>com.alibaba</groupId>
            <artifactId>fastjson</artifactId>
            <version> 1.2.68</version>
        </dependency>

    </dependencies>
</project>
  • 编写如下代码

常量类，主要是定义规则文件的目录和名字 

package com.sentinel.olive.file;

import java.util.HashMap;
import java.util.Map;

public class PersistenceRuleConstant {

    /**
     * 存储文件路径
     */
    public static final String storePath = System.getProperty("user.home")+"\sentinel\rules\";

    /**
     * 各种存储sentinel规则映射map
     */
    public static final Map<String,String> rulesMap = new HashMap<String,String>();

    //流控规则文件
    public static final String FLOW_RULE_PATH = "flowRulePath";

    //降级规则文件
    public static final String DEGRAGE_RULE_PATH = "degradeRulePath";

    //授权规则文件
    public static final String AUTH_RULE_PATH = "authRulePath";

    //系统规则文件
    public static final String SYSTEM_RULE_PATH = "systemRulePath";

    //热点参数文件
    public static final String HOT_PARAM_RULE = "hotParamRulePath";

    static {
        rulesMap.put(FLOW_RULE_PATH,storePath+"flowRule.json");
        rulesMap.put(DEGRAGE_RULE_PATH,storePath+"degradeRule.json");
        rulesMap.put(SYSTEM_RULE_PATH,storePath+"systemRule.json");
        rulesMap.put(AUTH_RULE_PATH,storePath+"authRule.json");
        rulesMap.put(HOT_PARAM_RULE,storePath+"hotParamRule.json");
    }
}

文件操作类，如果规则文件不存在就创建对应的目录和对应的规则文件 

package com.sentinel.olive.file;

import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import java.util.Set;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class RuleFileUtils {

     private static final Logger logger = LoggerFactory.getLogger(RuleFileUtils.class);

    /**
     * 方法实现说明:若路径不存在就创建路径
     * @param filePath:文件存储路径
     */
    public static void mkdirIfNotExits(String filePath) throws IOException {
        File file = new File(filePath);
        if(!file.exists()) {
            logger.info("创建Sentinel规则目录:{}",filePath);
            file.mkdirs();
        }
    }


    /**
     * 方法实现说明:若文件不存在就创建路径
     * @param ruleFileMap 规则存储文件
     */
    public static void createFileIfNotExits(Map<String,String> ruleFileMap) throws IOException {

        Set<String> ruleFilePathSet = ruleFileMap.keySet();

        Iterator<String> ruleFilePathIter = ruleFilePathSet.iterator();

        while (ruleFilePathIter.hasNext()) {
            String ruleFilePathKey = ruleFilePathIter.next();
            String ruleFilePath  = PersistenceRuleConstant.rulesMap.get(ruleFilePathKey).toString();
            File ruleFile = new File(ruleFilePath);
            if(ruleFile.exists()) {
                logger.info("创建Sentinel 规则文件:{}",ruleFile);
                ruleFile.createNewFile();
            }
        }
    }
}

规则的编码和解码操作类 

package com.sentinel.olive.file;

import java.util.List;

import com.alibaba.csp.sentinel.datasource.Converter;
import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRule;
import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRule;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRule;
import com.alibaba.csp.sentinel.slots.system.SystemRule;
import com.alibaba.fastjson.JSON;
import com.alibaba.fastjson.TypeReference;

public class RuleListParserUtils {

    /**
     * 流控列表解析器
     */
    public static final Converter<String, List<FlowRule>> flowRuleListParser = new Converter<String, List<FlowRule>>() {
        @Override
        public List<FlowRule> convert(String source) {
            return JSON.parseObject(source, new TypeReference<List<FlowRule>>() {
            });
        }
    };

    /**
     * 流控列表 编码器
     */
    public static final Converter<List<FlowRule>, String> flowRuleEnCoding = new Converter<List<FlowRule>, String>() {
        @Override
        public String convert(List<FlowRule> source) {
            return JSON.toJSONString(source);
        }
    };

    public static final Converter<String, List<DegradeRule>> degradeRuleListParser = source -> JSON.parseObject(source,
            new TypeReference<List<DegradeRule>>() {
            });

    public static final Converter<List<DegradeRule>, String> degradeRuleEnCoding = new Converter<List<DegradeRule>, String>() {
        @Override
        public String convert(List<DegradeRule> source) {
            return JSON.toJSONString(source);
        }
    };

    public static final Converter<String, List<SystemRule>> systemRuleListParser = source -> JSON.parseObject(source,
            new TypeReference<List<SystemRule>>() {
            });

    public static final Converter<List<SystemRule>, String> systemRuleEnCoding = new Converter<List<SystemRule>, String>() {
        @Override
        public String convert(List<SystemRule> source) {
            return JSON.toJSONString(source);
        }
    };

    public static final Converter<String, List<AuthorityRule>> authorityRuleListParser = source -> JSON
            .parseObject(source, new TypeReference<List<AuthorityRule>>() {
            });


    public static final Converter<List<AuthorityRule>, String> authorityRuleEnCoding = new Converter<List<AuthorityRule>, String>() {
        @Override
        public String convert(List<AuthorityRule> source) {
            return JSON.toJSONString(source);
        }
    };

    public static final Converter<String, List<ParamFlowRule>> paramFlowRuleListParser = source -> JSON
            .parseObject(source, new TypeReference<List<ParamFlowRule>>() {
            });


//    public static final Converter<List<ParamFlowRule>, String> paramFlowRuleEnCoding = new Converter<List<ParamFlowRule>, String>() {
//        @Override
//        public String convert(List<ParamFlowRule> source) {
//            return JSON.toJSONString(source);
//        }
//    };

    public static final Converter<List<ParamFlowRule>, String> paramFlowRuleEnCoding = source -> encodeJson(source);

    private static <T> String encodeJson(T t) {
        return JSON.toJSONString(t);
    }

}

具体pull模式操作类 

package com.sentinel.olive.file;

import java.io.FileNotFoundException;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.List;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import com.alibaba.csp.sentinel.command.handler.ModifyParamFlowRulesCommandHandler;
import com.alibaba.csp.sentinel.datasource.FileRefreshableDataSource;
import com.alibaba.csp.sentinel.datasource.FileWritableDataSource;
import com.alibaba.csp.sentinel.datasource.ReadableDataSource;
import com.alibaba.csp.sentinel.datasource.WritableDataSource;
import com.alibaba.csp.sentinel.init.InitFunc;
import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRule;
import com.alibaba.csp.sentinel.slots.block.authority.AuthorityRuleManager;
import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRule;
import com.alibaba.csp.sentinel.slots.block.degrade.DegradeRuleManager;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.param.ParamFlowRuleManager;
import com.alibaba.csp.sentinel.slots.system.SystemRule;
import com.alibaba.csp.sentinel.slots.system.SystemRuleManager;
import com.alibaba.csp.sentinel.transport.util.WritableDataSourceRegistry;

public class PullModeLocalFileDataSource implements InitFunc {

    private static final Logger logger = LoggerFactory.getLogger(PullModeLocalFileDataSource.class);

    private static final SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");

    @Override
    public void init() throws Exception {
        logger.info("time:{}读取配置", sdf.format(new Date()));
        try {
            // 创建文件存储目录(若路径不存在就创建路径)
            RuleFileUtils.mkdirIfNotExits(PersistenceRuleConstant.storePath);
            // 创建规则文件()
            RuleFileUtils.createFileIfNotExits(PersistenceRuleConstant.rulesMap);
            // 处理流控规则逻辑
            dealFlowRules();
            // 处理降级规则
            dealDegradeRules();
            // 处理系统规则
            dealSystemRules();
            // 热点参数规则
            dealParamFlowRules();
            // 授权规则
            dealAuthRules();
        } catch (Exception e) {
            logger.error("错误原因:{}", e);
        }

    }

    /**
     * 方法实现说明:处理流控规则逻辑
     */
    private void dealFlowRules() throws FileNotFoundException {
        String ruleFilePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.FLOW_RULE_PATH).toString();
        // 创建流控规则的可读数据源
        ReadableDataSource<String, List<FlowRule>> flowRuleRDS = new FileRefreshableDataSource<>(ruleFilePath,
                RuleListParserUtils.flowRuleListParser);
        // 将可读数据源注册至FlowRuleManager 这样当规则文件发生变化时，就会更新规则到内存
        FlowRuleManager.register2Property(flowRuleRDS.getProperty());
        WritableDataSource<List<FlowRule>> flowRuleWDS = new FileWritableDataSource<List<FlowRule>>(ruleFilePath,
                RuleListParserUtils.flowRuleEnCoding);

        // 将可写数据源注册至 transport 模块的 WritableDataSourceRegistry 中.
        // 这样收到控制台推送的规则时，Sentinel 会先更新到内存，然后将规则写入到文件中.
        WritableDataSourceRegistry.registerFlowDataSource(flowRuleWDS);
    }

    // 处理降级规则
    private void dealDegradeRules() throws FileNotFoundException {
        String degradeRulePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.DEGRAGE_RULE_PATH).toString();
        // 降级规则
        ReadableDataSource<String, List<DegradeRule>> degradeRuleRDS = new FileRefreshableDataSource<>(degradeRulePath,
                RuleListParserUtils.degradeRuleListParser);
        DegradeRuleManager.register2Property(degradeRuleRDS.getProperty());
        WritableDataSource<List<DegradeRule>> degradeRuleWDS = new FileWritableDataSource<>(degradeRulePath,
                RuleListParserUtils.degradeRuleEnCoding);
        WritableDataSourceRegistry.registerDegradeDataSource(degradeRuleWDS);

    }

    // 处理系统规则
    private void dealSystemRules() throws FileNotFoundException {
        String systemRulePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.SYSTEM_RULE_PATH).toString();
        // 系统规则
        ReadableDataSource<String, List<SystemRule>> systemRuleRDS = new FileRefreshableDataSource<>(systemRulePath,
                RuleListParserUtils.systemRuleListParser);
        SystemRuleManager.register2Property(systemRuleRDS.getProperty());
        WritableDataSource<List<SystemRule>> systemRuleWDS = new FileWritableDataSource<>(systemRulePath,
                RuleListParserUtils.systemRuleEnCoding);
        WritableDataSourceRegistry.registerSystemDataSource(systemRuleWDS);
    }

    // 热点参数规则
    private void dealParamFlowRules() throws FileNotFoundException {
        String paramFlowRulePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.HOT_PARAM_RULE).toString();
        // 热点参数规则
        ReadableDataSource<String, List<ParamFlowRule>> paramFlowRuleRDS = new FileRefreshableDataSource<>(
                paramFlowRulePath, RuleListParserUtils.paramFlowRuleListParser);
        ParamFlowRuleManager.register2Property(paramFlowRuleRDS.getProperty());
        WritableDataSource<List<ParamFlowRule>> paramFlowRuleWDS = new FileWritableDataSource<>(paramFlowRulePath,
                RuleListParserUtils.paramFlowRuleEnCoding);
        ModifyParamFlowRulesCommandHandler.setWritableDataSource(paramFlowRuleWDS);
    }

    private void dealAuthRules() throws FileNotFoundException {
        String authFlowRulePath = PersistenceRuleConstant.rulesMap.get(PersistenceRuleConstant.AUTH_RULE_PATH).toString();
        // 授权规则
        ReadableDataSource<String, List<AuthorityRule>> authorityRuleRDS = new FileRefreshableDataSource<>(authFlowRulePath,
                RuleListParserUtils.authorityRuleListParser);
        AuthorityRuleManager.register2Property(authorityRuleRDS.getProperty());
        WritableDataSource<List<AuthorityRule>> authorityRuleWDS = new FileWritableDataSource<>(authFlowRulePath,
                RuleListParserUtils.authorityRuleEnCoding);
        WritableDataSourceRegistry.registerAuthorityDataSource(authorityRuleWDS);
    }
}

编写了以上代码就可以，通过SPI扩展机制进行扩展，在微服务工程 olive-pull-sentinel-datasource的resources目录下创建META-INF/services目录，并新建文件名为com.alibaba.csp.sentinel.init.InitFunc文件。内容是PullModeLocalFileDataSource类全路径类名

0x02：配置文件application.yml

看到这张图总有解决了以上的疑问了，提供了配置项，配置Sentinel服务对应的ip和端口

0x03：创建测试控制器和springboot启动类

测试控制器 

package com.sentinel.olive.controller;

import java.util.HashMap;
import java.util.Map;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class UserController {

    @GetMapping("/getUser")
    public Map<String, Object> getUser() {
        Map<String, Object> result = new HashMap<>();
        result.put("code", "000000");
        result.put("message", "ok");
        return result;
    }
}

springboot启动类 

package com.sentinel.olive;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;

@SpringBootApplication
public class Application {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

}

0x04：启动并测试验证

启动服务

  • sentinel-dashboard服务
  • olive-pull-sentinel-datasource微服务

验证

  • sentinel-dashboard创建规则，olive-pull-sentinel-datasource微服务检测到sentinel-dashboard变化
  • 修改json规则文件的规则，sentinel-dashboard检测到json规则编码

访问微服务的http://localhost:8866/getUser接口

访问sentinel-dashboard服务：http://127.0.0.1:8080/

创建流控规则

olive-pull-sentinel-datasource检测到流控规则的变化并产生flowRule.json文件

修改流控规则文件json

sentinel-dashboard查看规则的变化

随机文章
本站知识点必读
最近更新