前言
本系列已写了四篇文章, 读本篇之前, 可以先读前面几篇.
思考大纲:.Net 架构篇: 思考如何设计一款实用的分布式监控系统?
实践篇一:.NetCore 实践篇: 分布式监控客户端 ZipkinTracer 从入门到放弃之路
实践篇二:.NetCore 实践篇: 分布式监控系统 zipkin 踩坑之路(二)
实践篇三:.NetCore 实践篇: 成功解决分布式监控 ZipKin 聚合依赖问题(三)
简要回顾
zipkin
Zipkin 是一种分布式跟踪系统. 它有助于收集解决微服务架构中的延迟问题所需的时序数据
zipkin 官网 https://zipkin.io/
zipkin4Net
zipkin4net 是. NET 客户端库.
- https://GitHub.com/openzipkin/zipkin4net
- zipkin-dependencies
这是一个 Spark 作业, 它将从您的数据存储区收集跨度, 分析服务之间的链接, 并存储它们以供以后在 web UI 中呈现.
使用方法
如果内存不足时, java 后跟上 - Xmx1024m -Xms1024m 参数, JAVA_OPTS 的一些参数可参考 Oracle 官方说明配置默认 JVM 和 Java 参数
- # ex to run the job to process yesterday's traces on OS/X
- $ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -uv-1d +%F`
- # or on Linux
- $ STORAGE_TYPE=cassandra3 java -jar zipkin-dependencies.jar `date -u -d '1 day ago' +%F`
MySQL 存储
* `MYSQL_DB`: 使用的数据库, 默认是 "zipkin".
* `MYSQL_USER` and `MYSQL_PASS`: MySQL 授权, 默认是空.
* `MYSQL_HOST`: 默认主机 (域名 / ip) 是 localhost
* `MYSQL_TCP_PORT`: 默认端口是 3306
* `MYSQL_USE_SSL`: 验证 `javax.NET.ssl.trustStore` 和 `javax.NET.ssl.trustStorePassword`, 默认不验证.
示例
$ STORAGE_TYPE=MySQL MYSQL_USER=root java -jar zipkin-dependencies.jar
详情参考:
https://GitHub.com/openzipkin/zipkin-dependencies
实践
创建使用数据库
- MySQL> create database mytestdb;
- Query OK, 1 row affected (0.01 sec)
- MySQL> show databases;
- +--------------------+
- | Database |
- +--------------------+
- | information_schema |
- | MySQL |
- | mytestdb |
- | performance_schema |
- | sys |
- | ttt |
- +--------------------+
- 7 rows in set (0.00 sec)
- MySQL> use mytestdb
- Database changed
使用 sql 语句创建 zipkin 表
- CREATETABLE IF NOT EXISTS zipkin_spans (
- `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64bit',
- `trace_id` BIGINT NOT NULL,
- `id` BIGINT NOT NULL,
- `name` VARCHAR(255) NOT NULL,
- `parent_id` BIGINT,
- `debug` BIT(1),
- `start_ts` BIGINT COMMENT 'Span.timestamp():epoch micros used for endTs query and to implement TTL',
- `duration` BIGINT COMMENT 'Span.duration():micros used for minDuration and maxDuration query'
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_spans ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `id`) COMMENT'ignore insert on duplicate';
- ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`, `id`) COMMENT 'forjoining with zipkin_annotations';
- ALTERTABLE zipkin_spans ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'forgetTracesByIds';
- ALTERTABLE zipkin_spans ADD INDEX(`name`) COMMENT 'for getTraces and getSpanNames';
- ALTERTABLE zipkin_spans ADD INDEX(`start_ts`) COMMENT 'for getTraces ordering andrange';
- CREATETABLE IF NOT EXISTS zipkin_annotations (
- `trace_id_high` BIGINT NOT NULL DEFAULT 0COMMENT 'If non zero, this means the trace uses 128 bit traceIds instead of 64bit',
- `trace_id` BIGINT NOT NULL COMMENT 'coincideswith zipkin_spans.trace_id',
- `span_id` BIGINT NOT NULL COMMENT 'coincideswith zipkin_spans.id',
- `a_key` VARCHAR(255) NOT NULL COMMENT'BinaryAnnotation.key or Annotation.value if type == -1',
- `a_value` BLOB COMMENT'BinaryAnnotation.value(), which must be smaller than 64KB',
- `a_type` INT NOT NULL COMMENT'BinaryAnnotation.type() or -1 if Annotation',
- `a_timestamp` BIGINT COMMENT 'Used toimplement TTL; Annotation.timestamp or zipkin_spans.timestamp',
- `endpoint_ipv4` INT COMMENT 'Null whenBinary/Annotation.endpoint is null',
- `endpoint_ipv6` BINARY(16) COMMENT 'Null whenBinary/Annotation.endpoint is null, or no IPv6 address',
- `endpoint_port` SMALLINT COMMENT 'Null whenBinary/Annotation.endpoint is null',
- `endpoint_service_name` VARCHAR(255) COMMENT'Null when Binary/Annotation.endpoint is null'
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_annotations ADD UNIQUE KEY(`trace_id_high`, `trace_id`, `span_id`,`a_key`, `a_timestamp`) COMMENT 'Ignore insert on duplicate';
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`, `span_id`)COMMENT 'for joining with zipkin_spans';
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id_high`, `trace_id`) COMMENT 'forgetTraces/ByIds';
- ALTERTABLE zipkin_annotations ADD INDEX(`endpoint_service_name`) COMMENT 'forgetTraces and getServiceNames';
- ALTERTABLE zipkin_annotations ADD INDEX(`a_type`) COMMENT 'for getTraces';
- ALTERTABLE zipkin_annotations ADD INDEX(`a_key`) COMMENT 'for getTraces';
- ALTERTABLE zipkin_annotations ADD INDEX(`trace_id`, `span_id`, `a_key`) COMMENT 'fordependencies job';
- CREATETABLE IF NOT EXISTS zipkin_dependencies (
- `day` DATE NOT NULL,
- `parent` VARCHAR(255) NOT NULL,
- `child` VARCHAR(255) NOT NULL,
- `call_count` BIGINT
- )ENGINE=InnoDB ROW_FORMAT=COMPRESSED CHARACTER SET=utf8 COLLATE utf8_general_ci;
- ALTERTABLE zipkin_dependencies ADD UNIQUE KEY(`day`, `parent`, `child`);
创建成功后, 查询结果.
- MySQL> show tables;
- +---------------------+
- | Tables_in_mytestdb |
- +---------------------+
- | zipkin_annotations |
- | zipkin_dependencies |
- | zipkin_spans |
- +---------------------+
- 3 rows in set (0.00 sec)
启动 zipkin-dependencies
最开始我的密码是[四个字母一个感叹号一个数字] , 再执行启动命令时, 密码那块给我报错自动换成[四个字母 rm -f] , 我修改成[四个字母一个 #号一个数字] 就能执行了
执行成功后, 依然提示 Access denied for user 'root'@'localhost' (using password: NO), 但我在 Linux 的命令中直接用 MySQL -u root -p 相同密码是可以登录成功的. 所以问题出现在哪呢?
- [root@izwz9fwifc2eniq3lbdzmgz cusD]# STORAGE_TYPE=MySQL MYSQL_HOST=localhost MYSQL_TCP_PORT=3306 MYSQL_DB=mytestdb MYSQL_USER=XXXXX MYSQL_PASS=XXXXX java -Xmx1024m -Xms1024m -jar zipkin-dependencies.jar
- Exception in thread "main" java.lang.RuntimeException: java.sql.SQLInvalidAuthorizationSpecException: Access denied for user 'root'@'localhost' (using password: NO)
- at zipkin2.dependencies.MySQL.MySQLDependenciesJob.hasTraceIdHigh(MySQLDependenciesJob.java:233)
- at zipkin2.dependencies.MySQL.MySQLDependenciesJob.run(MySQLDependenciesJob.java:184)
- at zipkin2.dependencies.ZipkinDependenciesJob.main(ZipkinDependenciesJob.java:65)
- Caused by: java.sql.SQLInvalidAuthorizationSpecException: Access denied for user 'root'@'localhost' (using password: NO)
- at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.get(ExceptionMapper.java:173)
- at org.mariadb.jdbc.internal.util.exceptions.ExceptionMapper.getException(ExceptionMapper.java:110)
- at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1115)
- at org.mariadb.jdbc.internal.util.Utils.retrieveProxy(Utils.java:502)
- at org.mariadb.jdbc.MariaDbConnection.newConnection(MariaDbConnection.java:154)
- at org.mariadb.jdbc.Driver.connect(Driver.java:86)
- at java.sql.DriverManager.getConnection(DriverManager.java:664)
- at java.sql.DriverManager.getConnection(DriverManager.java:247)
- at zipkin2.dependencies.MySQL.MySQLDependenciesJob.hasTraceIdHigh(MySQLDependenciesJob.java:229)
- ... 2 more
- Caused by: java.sql.SQLException: Access denied for user 'root'@'localhost' (using password: NO)
- Current charset is UTF-8. If password has been set using other charset, consider using option 'passwordCharacterEncoding'
- at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.authentication(AbstractConnectProtocol.java:862)
- at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.handleConnectionPhases(AbstractConnectProtocol.java:785)
- at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connect(AbstractConnectProtocol.java:456)
- at org.mariadb.jdbc.internal.protocol.AbstractConnectProtocol.connectWithoutProxy(AbstractConnectProtocol.java:1111)
- ... 8 more
追溯源码
逼不得已, 走上查看源码之路, Idea 打开 zipkin-dependencies/MySQL 源码, 查看相关部分代码.
- public static final class Builder {
- Map < String,
- String > sparkProperties = ImmutableMap.of("spark.ui.enabled", "false");
- String db = getEnv("MYSQL_DB", "zipkin");
- String host = getEnv("MYSQL_HOST", "localhost");
- int port = Integer.parseInt(getEnv("MYSQL_TCP_PORT", "3306"));
- String user = getEnv("MYSQL_USER", "");
- String password = getEnv("MYSQL_PASS", "");
- int maxConnections = Integer.parseInt(getEnv("MYSQL_MAX_CONNECTIONS", "10"));
- boolean useSsl = Boolean.parseBoolean(getEnv("MYSQL_USE_SSL", "false"));
- // local[*] master lets us run & test the job locally without setting a Spark cluster
- String sparkMaster = getEnv("SPARK_MASTER", "local[*]");
- // By default the job only works on traces whose first timestamp is today
- long day = midnightUTC(System.currentTimeMillis());
- /**
- * 为减少篇幅, 中间设置属性部分代码省略.
- */
- public MySQLDependenciesJob build() {
- return new MySQLDependenciesJob(this);
- }
- }
- /**
- * 为减少篇幅, 中间部分代码省略.
- */
- MySQLDependenciesJob(Builder builder) {
- this.db = builder.db;
- this.day = builder.day;
- SimpleDateFormat df = new SimpleDateFormat("yyyy-MM-dd");
- df.setTimeZone(TimeZone.getTimeZone("UTC"));
- this.dateStamp = df.format(new Date(builder.day));
- this.url = new StringBuilder("jdbc:MySQL://").append(builder.host).append(":").append(builder.port).append("/").append(builder.db).append("?autoReconnect=true").append("&useSSL=").append(builder.useSsl).toString();
- this.user = builder.user;
- this.password = builder.password;
- this.conf = new SparkConf(true).setMaster(builder.sparkMaster).setAppName(getClass().getName());
- if (builder.jars != null) conf.setJars(builder.jars);
- for (Map.Entry < String, String > entry: builder.sparkProperties.entrySet()) {
- conf.set(entry.getKey(), entry.getValue());
- }
- this.logInitializer = builder.logInitializer;
- }
- void saveToMySQL(List < DependencyLink > links) {
- try (Connection con = DriverManager.getConnection(url, user, password)) {
- PreparedStatement replace = con.prepareStatement("REPLACE INTO zipkin_dependencies (day, parent, child, call_count, error_count) VALUES (?,?,?,?,?)");
- for (DependencyLink link: links) {
- replace.setDate(1, new java.sql.Date(day));
- replace.setString(2, link.parent());
- replace.setString(3, link.child());
- replace.setLong(4, link.callCount());
- replace.setLong(5, link.errorCount());
- replace.executeUpdate();
- }
- } catch(SQLException e) {
- throw new RuntimeException("Could not save links" + links, e);
- }
- }
然并卵, 看完之后, 没看出明显问题. 难道还是我自己的 MySQL 配置问题? 还是启动部分的参数问题? 代码部分也是有些疑惑, password 和 root 为什么没放进 url 里, 难道是为了安全考虑么?
- this.url = new StringBuilder("jdbc:MySQL://")
- .append(builder.host).append(":").append(builder.port)
- .append("/").append(builder.db)
- .append("?autoReconnect=true")
- .append("&useSSL=").append(builder.useSsl).toString();
- this.user = builder.user;
- this.password = builder.password;
文中还提到 Current charset is UTF-8. If password has been set using other charset, consider using option 'passwordCharacterEncoding', 编码格式是否有不同呢?
查看 MySQL 数据库及表编码格式
- MySQL> show variables like 'character_set_database';
- +------------------------+---------+
- | Variable_name | Value |
- +------------------------+---------+
- | character_set_database | utf8mb4 |
- +------------------------+---------+
- 1 row in set (0.01 sec)
参考链接
https://jira.mariadb.org/browse/CONJ-480
将请求参数作为 UTF-8 编码的字符串传递[重复]
zipkin
zipkin 集成到 node,C#
微服务之分布式跟踪系统(springboot+zipkin+MySQL)
第二十九章 springboot + zipkin + MySQL
Linux 下的 MySQL 用命令执行 sql 文件
修改 MySQL 管理员密码
总结
由于启动 zipkin-dependencies 链接 MySQL 报 Access denied for user 'root'@'localhost' (using password: NO)错误, 本次持久化之路最终失败. 但由于我直接使用[MySQL -u 用户 -p] 是能登录成功的, 所以我猜测了以下原因:
客户端自己的 bug, 和我服务器 MySQL 版本不兼容?
编码问题, 编码两者不符?
用户名和密码没有共享全局, 只对一个数据库有效?
别人的博文是面向教学成功编程, 我的是面向失败编程, 也别有一番趣味. 留下疑问, 待日后解决调. 虽然失败了, 但我又收集了一堆链接, 增添了 MySQL 一些故障解决的认识.
今天换了一款个人很喜欢的皮肤, 会根据 h1,h2 自动生成目录, 之前的博文我也都检查了下, 有很大失位的我都调整了过来, 不标准的暂时不改了, 我以后的博文都按照要求的格式写, 排版美观度提升了很多, 感谢作者 bndong, 如果有打算使用这个皮肤的, 一定要开启控件显示公告.
来源: https://www.cnblogs.com/fancunwei/p/9677952.html