系列导航
概述
针对Istio Mixer Adapter的开发，后面会有文章专门讲述源码分析及开发调试的过程，这里仅使用前两篇文章讲述的K8S+Istio环境部署我已经开发好的Istio Mixer Grpc Adapter并配置K8S使其生效
gatewaymetric简介
gatewaymetric是笔者借鉴Istio官网指导开发的Istio Mixer Grpc Adapter，他的功能主要是处理Mixer上报的http访问日志，收集南北向流量（指通过istio-ingressgateway从集群外进入到集群）的访问日志，通过调用promethues官方go程序库，内存生成metrics并暴露exporter端口给promethues采集数据，它的功能目前还很不完善，但是作为学习说明已经足够
本文主要参考官网Wiki进行开发，里面也讲到了部分原理
下面是代码结构
➜ gatewaymetric git:(v1.0.5) ✗ tree.├── cmd│ └── main.go├── config│ ├── adapter.gatewaymetric.config.pb.html│ ├── config.pb.go│ ├── config.proto│ ├── config.proto_descriptor│ └── gatewaymetric.yaml├── gatewaymetric.go├── sample_operator_cfg.yaml└── testdata ├── attributes.yaml ├── gatewaymetric.yaml ├── sample_operator_cfg.yaml └── template.yaml复制代码
上述文件中cmd/main.go为程序入口，其暴露两个端口
1）http:9145为promethues的数据抓取端口 2）grpc:41543为GrpcAdapter的通信端口，也就是与Mixer建立连接的端口，这个后面会用到，要把它显示注册到Mixer当中，使Mixer感知它构建自定义Mixer Grpc Adapter（gatewaymetric）
我们第一步需要打包这个Adapter并制作成Docker镜像发布到Docker Hub中
➜ cmd git:(v1.0.5) ✗ cd /home/ubuntu/go/src/istio.io/istio/mixer/adapter/gatewaymetric/cmd➜ cmd git:(master) CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o gatewaymetric➜ cmd git:(v1.0.5) ✗ CGO_ENABLE➜ cmd git:(v1.0.5) ✗ ./gatewaymetric adapter listening on "[::]:41543"promethues listening on ":9145"D=0 GOOS=linux GOARCH=amd64 go build -a -installsuffix cgo -o gatewaymetric复制代码
通过上述命令，我们基于源码构建了一个我们定制的GrpcAdapter并且成功启动了它，如果读者无法正常执行结束go build命令，则可能是go环境配置的问题，可以绕开这里直接使用源码cmd下面的gatewaymetric文件即可，笔者已经构建好了二进制文件，接下来我们编写Dockerfile
➜ cmd git:(v1.0.5) ✗ touch Dockerfile➜ cmd git:(v1.0.5) ✗ vim DockerfileFROM scratchADD gatewaymetric /ENTRYPOINT ["/gatewaymetric"]复制代码
该Dockerfile使用scratch基础镜像并将gatewaymetirc（我们自定义的Mixer Grpc Adapter后面都会以此称呼）加入到镜像当中，启动入口为"/gatewaymetric"
下面开始构建➜ cmd git:(v1.0.5) ✗ sudo docker build -f Dockerfile .Sending build context to Docker daemon 17.87MBStep 1/3 : FROM scratch ---> Step 2/3 : ADD gatewaymetric / ---> 82ca9cdf54abStep 3/3 : ENTRYPOINT ["/gatewaymetric"] ---> Running in 2630beb8da5dRemoving intermediate container 2630beb8da5d ---> 3747dfc14d90Successfully built 3747dfc14d90➜ cmd git:(v1.0.5) ✗ sudo docker run -p 9145:9145 -p 41543:41543 3747dfc14d90adapter listening on "[::]:41543"promethues listening on ":9145"复制代码
通过模拟程序测试通过（这里使用的是前面提到的当中描述的本地模拟测试的办法），说明打的Docker镜像没有问题，若读者还没有了解这块的知识，忽略这里即可，后面文章会详细描述如何在本地开发及调试Adapter
接下来，我们将该镜像提交到Docker Hub（这里讲述了开发构建并推送镜像的过程，读者可以跳过此步骤直接使用笔者已经构建好并发布到docker hub上的镜像即可）
➜ cmd git:(v1.0.5) ✗ sudo docker tag 3747dfc14d90 zrbcool/gatewaymetric:v1.0 ➜ cmd git:(v1.0.5) ✗ sudo docker push zrbcool/gatewaymetric:v1.0 The push refers to repository [docker.io/zrbcool/gatewaymetric]115a048ee88f: Pushed v1.0: digest: sha256:616fc9e1d91ad963a5ace4b5210caecbde21aca8ea5f2bfe8f98ac20e33a6cfa size: 528复制代码
部署Adapter到K8S + Istio集群
到K8S + Istio集群的VM中，部署已经制作好的镜像
➜ gatewaymetric cat deployment.yaml apiVersion: extensions/v1beta1kind: Deploymentmetadata: name: gatewaymetric namespace: istio-system labels: app: gatewaymetricspec: replicas: 1 selector: matchLabels: app: gatewaymetric template: metadata: labels: app: gatewaymetric spec: containers: - name: gatewaymetric image: "zrbcool/gatewaymetric:v1.0" imagePullPolicy: "IfNotPresent" ports: - containerPort: 9145 - containerPort: 41543复制代码
然后创建Service
➜ gatewaymetric cat service.yaml apiVersion: v1kind: Servicemetadata: labels: app: gatewaymetric name: gatewaymetric namespace: istio-systemspec: selector: app: gatewaymetric ports: type: ClusterIP ports: - name: promethues port: 9145 targetPort: 9145 - name: grpc port: 41543 targetPort: 41543复制代码
简单验证，端口已通，promethues的metrics也可以获取
➜ gatewaymetric kubectl get svc -n istio-systemNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) gatewaymetric ClusterIP 10.104.182.1409145/TCP,41543/TCP ➜ gatewaymetric telnet 10.104.182.140 41543Trying 10.104.182.140...Connected to 10.104.182.140.Escape character is '^]'.➜ gatewaymetric curl 10.104.182.140:9145/metrics# HELP go_gc_duration_seconds A summary of the GC invocation durations.# TYPE go_gc_duration_seconds summarygo_gc_duration_seconds{quantile="0"} 0go_gc_duration_seconds{quantile="0.25"} 0...复制代码
至此，我们的Mixer Grpc Adapter已经部署到了我们的K8S集群当中，接下来我们需要配置CRD，使mixer能够感知到我们的Grpc Adapter
部署下面文件的内容，其包含了自定义Adapter的运行时实例handler，template->logentity的运行时实例instance，以及mixer的匹配规则rule指定handler处理instance的Report数据，如果上面这句话看的有点蒙，不要紧这个都是Mixer的设计者抽象出来的概念，目的就是让Mixer适配器的开发者跟运维者分离并使Mixer Adapter的开发更加灵活，后面会有专门章节详细阐述其各个模型的意义，如果读者想提前了解，请参考下面链接：# handler for adapter gatewaymetricapiVersion: "config.istio.io/v1alpha2"kind: handlermetadata: name: h1 namespace: istio-systemspec: adapter: gatewaymetric connection: address: "gatewaymetric.istio-system.svc.cluster.local:41543" #replaces at runtime by the test params: file_path: "out.txt"---# instance for template metricapiVersion: "config.istio.io/v1alpha2"kind: instancemetadata: name: i1metric namespace: istio-systemspec: template: logentry params: severity: '"warning"' timestamp: request.time variables: host: request.host method: request.method | "" url: request.path | "" responseCode: response.code | 0 responseBodySize: response.size | 0 responseTotalSize: response.total_size | 0 latency: response.duration | "0ms" monitored_resource_type: '"UNSPECIFIED"'---# rule to dispatch to handler h1apiVersion: "config.istio.io/v1alpha2"kind: rulemetadata: name: r1 namespace: istio-systemspec: actions: - handler: h1.istio-system instances: - i1metric---复制代码
在Kubernetes中部署它们
➜ gatewaymetric kubectl apply -f attributes.yaml attributemanifest.config.istio.io/istio-proxy createdattributemanifest.config.istio.io/kubernetes configured➜ gatewaymetric kubectl apply -f template.yaml template.config.istio.io/logentry created➜ gatewaymetric kubectl apply -f gatewaymetric.yaml adapter.config.istio.io/gatewaymetric created➜ gatewaymetric kubectl apply -f operator_cfg.yaml handler.config.istio.io/h1 createdinstance.config.istio.io/i1metric createdrule.config.istio.io/r1 created复制代码
使用浏览器访问http://192.168.101.6:31380/productpage，并查看Mixer的日志，看到
{ "level":"info","time":"2019-02-21T03:05:40.292749Z","instance":"accesslog.logentry.istio-system","apiClaims":"","apiKey":"","clientTraceId":"","connection_security_policy":"unknown","destinationApp":"","destinationIp":"10.32.0.34","destinationName":"","destinationNamespace":"","destinationOwner":"","destinationPrincipal":"","destinationServiceHost":"productpage.default.svc.cluster.local","destinationWorkload":"","httpAuthority":"192.168.101.6:31380","latency":"25.101758ms","method":"GET","protocol":"http","receivedBytes":488,"referer":"","reporter":"source","requestId":"405ed7ca-c988-4cfa-8e6a-e808e2cbeab8","requestSize":0,"requestedServerName":"","responseCode":200,"responseSize":5719,"responseTimestamp":"2019-02-21T03:05:40.317790Z","sentBytes":5858,"sourceApp":"","sourceIp":"0.0.0.0","sourceName":"","sourceNamespace":"istio-system","sourceOwner":"","sourcePrincipal":"","sourceWorkload":"","url":"/productpage","userAgent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.81 Safari/537.36","xForwardedFor":"10.32.0.1"}复制代码 查看我们自定义的Grpc Adapter日志，看到
2019-02-21T03:05:41.002442Z info received request {[&InstanceMsg{Variables:map[string]*istio_policy_v1beta11.Value{host: &Value{Value:&Value_StringValue{StringValue:192.168.101.6:31380,},},latency: &Value{Value:&Value_DurationValue{DurationValue:&Duration{Value:34.547407ms,},},},method: &Value{Value:&Value_StringValue{StringValue:GET,},},responseBodySize: &Value{Value:&Value_Int64Value{Int64Value:5719,},},responseCode: &Value{Value:&Value_Int64Value{Int64Value:200,},},responseTotalSize: &Value{Value:&Value_Int64Value{Int64Value:5858,},},url: &Value{Value:&Value_StringValue{StringValue:/productpage,},},},Timestamp:&istio_policy_v1beta11.TimeStamp{Value:2019-02-21T03:05:39.955797905Z,},Severity:warning,MonitoredResourceType:UNSPECIFIED,MonitoredResourceDimensions:map[string]*istio_policy_v1beta11.Value{},Name:i1metric.instance.istio-system,} &InstanceMsg{Variables:map[string]*istio_policy_v1beta11.Value{host: &Value{Value:&Value_StringValue{StringValue:192.168.101.6:31380,},},latency: &Value{Value:&Value_DurationValue{DurationValue:&Duration{Value:39.579628ms,},},},method: &Value{Value:&Value_StringValue{StringValue:GET,},},responseBodySize: &Value{Value:&Value_Int64Value{Int64Value:5723,},},responseCode: &Value{Value:&Value_Int64Value{Int64Value:200,},},responseTotalSize: &Value{Value:&Value_Int64Value{Int64Value:5862,},},url: &Value{Value:&Value_StringValue{StringValue:/productpage,},},},Timestamp:&istio_policy_v1beta11.TimeStamp{Value:2019-02-21T03:05:40.126271976Z,},Severity:warning,MonitoredResourceType:UNSPECIFIED,MonitoredResourceDimensions:map[string]*istio_policy_v1beta11.Value{},Name:i1metric.instance.istio-system,} &InstanceMsg{Variables:map[string]*istio_policy_v1beta11.Value{host: &Value{Value:&Value_StringValue{StringValue:192.168.101.6:31380,},},latency: &Value{Value:&Value_DurationValue{DurationValue:&Duration{Value:24.084598ms,},},},method: &Value{Value:&Value_StringValue{StringValue:GET,},},responseBodySize: &Value{Value:&Value_Int64Value{Int64Value:5719,},},responseCode: &Value{Value:&Value_Int64Value{Int64Value:200,},},responseTotalSize: &Value{Value:&Value_Int64Value{Int64Value:5858,},},url: &Value{Value:&Value_StringValue{StringValue:/productpage,},},},Timestamp:&istio_policy_v1beta11.TimeStamp{Value:2019-02-21T03:05:40.293294914Z,},Severity:warning,MonitoredResourceType:UNSPECIFIED,MonitoredResourceDimensions:map[string]*istio_policy_v1beta11.Value{},Name:i1metric.instance.istio-system,} &InstanceMsg{Variables:map[string]*istio_policy_v1beta11.Value{host: &Value{Value:&Value_StringValue{StringValue:192.168.101.6:31380,},},latency: &Value{Value:&Value_DurationValue{DurationValue:&Duration{Value:30.813597ms,},},},method: &Value{Value:&Value_StringValue{StringValue:GET,},},responseBodySize: &Value{Value:&Value_Int64Value{Int64Value:4415,},},responseCode: &Value{Value:&Value_Int64Value{Int64Value:200,},},responseTotalSize: &Value{Value:&Value_Int64Value{Int64Value:4554,},},url: &Value{Value:&Value_StringValue{StringValue:/productpage,},},},Timestamp:&istio_policy_v1beta11.TimeStamp{Value:2019-02-21T03:05:40.55426223Z,},Severity:warning,MonitoredResourceType:UNSPECIFIED,MonitoredResourceDimensions:map[string]*istio_policy_v1beta11.Value{},Name:i1metric.instance.istio-system,}] &Any{TypeUrl:type.googleapis.com/adapter.gatewaymetric.config.Params,Value:[10 7 111 117 116 46 116 120 116],} 7051031778836507932}2019-02-21T03:05:41.002468Z info host: %!(EXTRA string=192.168.101.6:31380)2019-02-21T03:05:41.002473Z info url: %!(EXTRA string=/productpage)2019-02-21T03:05:41.002475Z info method: %!(EXTRA string=GET)2019-02-21T03:05:41.002477Z info responseCode: %!(EXTRA string=200)2019-02-21T03:05:41.002479Z info responseBodySize: %!(EXTRA int64=5719)2019-02-21T03:05:41.002483Z info responseTotalSize: %!(EXTRA int64=5858)2019-02-21T03:05:41.002487Z info latency: %!(EXTRA float64=0.034547407)2019-02-21T03:05:41.002500Z info host: %!(EXTRA string=192.168.101.6:31380)2019-02-21T03:05:41.002506Z info url: %!(EXTRA string=/productpage)2019-02-21T03:05:41.002508Z info method: %!(EXTRA string=GET)2019-02-21T03:05:41.002510Z info responseCode: %!(EXTRA string=200)2019-02-21T03:05:41.002512Z info responseBodySize: %!(EXTRA int64=5723)2019-02-21T03:05:41.002514Z info responseTotalSize: %!(EXTRA int64=5862)2019-02-21T03:05:41.002516Z info latency: %!(EXTRA float64=0.039579628)2019-02-21T03:05:41.002520Z info host: %!(EXTRA string=192.168.101.6:31380)2019-02-21T03:05:41.002522Z info url: %!(EXTRA string=/productpage)2019-02-21T03:05:41.002524Z info method: %!(EXTRA string=GET)2019-02-21T03:05:41.002526Z info responseCode: %!(EXTRA string=200)2019-02-21T03:05:41.002528Z info responseBodySize: %!(EXTRA int64=5719)2019-02-21T03:05:41.002530Z info responseTotalSize: %!(EXTRA int64=5858)2019-02-21T03:05:41.002532Z info latency: %!(EXTRA float64=0.024084598)2019-02-21T03:05:41.002535Z info host: %!(EXTRA string=192.168.101.6:31380)2019-02-21T03:05:41.002537Z info url: %!(EXTRA string=/productpage)2019-02-21T03:05:41.002538Z info method: %!(EXTRA string=GET)2019-02-21T03:05:41.002540Z info responseCode: %!(EXTRA string=200)2019-02-21T03:05:41.002542Z info responseBodySize: %!(EXTRA int64=4415)2019-02-21T03:05:41.002544Z info responseTotalSize: %!(EXTRA int64=4554)2019-02-21T03:05:41.002547Z info latency: %!(EXTRA float64=0.030813597)复制代码 再看下promethues暴露的9145端口是否有数据呢
➜ gatewaymetric curl 10.104.182.140:9145/metrics...http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.00099"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.00089"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0007899999999999999"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0006899999999999999"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0005899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0004899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0003899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0002899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-0.0001899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="-8.999999999999979e-05"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="1.0000000000000216e-05"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.00011000000000000022"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.00021000000000000023"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0003100000000000002"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0004100000000000002"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0005100000000000003"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0006100000000000003"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0007100000000000003"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0008100000000000004"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="0.0009100000000000004"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200",le="+Inf"} 34http_request_seconds_histogram_sum{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200"} 1.1806039740000003http_request_seconds_histogram_count{endpoint="/test",fullurl="/productpage",host="192.168.101.6:31380",method="GET",status="200"} 34http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.00099"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.00089"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0007899999999999999"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0006899999999999999"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0005899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0004899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0003899999999999998"} 0http_request_seconds_histogram_bucket{endpoint="/test",fullurl="/ratings/0",host="ratings:9080",method="GET",status="200",le="-0.0002899999999999998"} 0...复制代码 说明一切已经生效，have fun！