kubernetes上部署Fluentd+Elasticsearch+kibana日志收集系统
1 工具介绍
Fluentd：用于收集、处理、传输日志数据。Elasticsearch：用于实时查询和解析数据。Kibana：用于数据可视化。
2 工作流程
Fluentd收集日志，将日志过滤处理后输出到Elasticsearch中，Elasticsearch的watcher插件将异常信息通过邮件发送给相关负责人。Kibana配置上ES服务接口，通过Kibana服务访问即可。3 安装
三个工具全部运行在kubernettes上。
fluentd.yaml配置文件
apiVersion: v1kind: Podmetadata: name: fluentdspec: containers: - name: fluentd-es image: index.tenxcloud.com/google_containers/fluentd-elasticsearch:1.9 imagePullPolicy: Never env: - name: FLUENTD_ARGS value: -qq volumeMounts: - name: containers mountPath: /var/lib/docker/containers - name: varlog mountPath: /varlog - name: podlog mountPath: /var/lib/kubelet/pods volumes: - name: containers hostPath: path: /var/lib/docker/containers - name: varlog hostPath: path: /var/log - name: podlog hostPath: path: /var/lib/kubelet/pods
Elasticsearch-rc.yaml配置文件
apiVersion: v1kind: ReplicationControllermetadata: name: elasticsearch labels: name: elasticsearch-logging kubernetes.io/cluster-service: "true"spec: replicas: 1 selector: name: elasticsearch-logging template: metadata: name: es labels: name: elasticsearch-logging kubernetes.io/cluster-service: "true" spec: containers: - image: index.tenxcloud.com/google_containers/elasticsearch:1.4 imagePullPolicy: Never name: elasticsearch-logging ports: - name: es-port containerPort: 9200 - name: es-tran-port containerPort: 9300 volumeMounts: - name: es-persistent-storage mountPath: /data volumes: - name: es-persistent-storage emptyDir: {}
Elasticsearch-service.yaml配置文件
apiVersion: v1
kind: Servicemetadata: name: elasticsearch-logging labels: name: elasticsearch-logging kubernetes.io/cluster-service: "true"spec: # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. type: NodePort ports: # the port that this service should serve on - port: 9200 targetPort: 9200 nodePort: 30009# - containerPort: 9200 selector: name: elasticsearch-logging
kibana-rc.yaml文件
apiVersion: v1kind: ReplicationControllermetadata: name: kibana-logging labels: name: kibana-logging kubernetes.io/cluster-service: "true"spec: replicas: 1 selector: name: kibana-logging template: metadata: name: kibana labels: name: kibana-logging kubernetes.io/cluster-service: "true" spec: containers: - image: gcr.io/google_containers/kibana:1.3 name: kibana-logging imagePullPolicy: Never env: - name: "ES_SCHEME" value: "http" - name: "ES_HOST" value: "<your-ip>/api/v1/proxy/namespaces/default/services/elasticsearch-logging/" ports: - name: kibana-port containerPort: 80
kibana-service.yaml配置文件
apiVersion: v1kind: Servicemetadata: name: kibana-logging labels: name: kibana-logging kubernetes.io/cluster-service: "true"spec: # if your cluster supports it, uncomment the following to automatically create # an external load-balanced IP for the frontend service. # type: LoadBalancer ports: # the port that this service should serve on - port: 5601 targetPort: 80# - containerPort: 9200 selector: name: kibana-logging
http://blog.csdn.net/ptmozhu/article/details/52301174