你头疼的ELK难题，本文几乎都解决了

一、ELK实用知识点总结

1、编码转换问题

这个问题，主要就是中文乱码。

input中的codec=>plain转码：

codec => plain { 
         charset => "GB2312" 
} 

将GB2312的文本编码，转为UTF-8的编码。

也可以在filebeat中实现编码的转换(推荐)：

filebeat.prospectors: 
- input_type: log 
  paths: 
    - c:UsersAdministratorDesktopperformanceTrace.txt 
  encoding: GB2312 

2、删除多余日志中的多余行

if ([message] =~ "^20.*- task request,.*,start time.*") {   #用正则需删除的多余行 
            drop {} 
    }  

日志示例：

2018-03-20 10:44:01,523 [33]DEBUG Debug - task request,task Id:1cbb72f1-a5ea-4e73-957c-6d20e9e12a7a,start time:2018-03-20 10:43:59   #需删除的行 
-- Request String :  
{"UserName":"15046699923","Pwd":"ZYjyh727","DeviceType":2,"DeviceId":"PC-20170525SADY","EquipmentNo":null,"SSID":"pc","RegisterPhones":null,"AppKey":"ab09d78e3b2c40b789ddfc81674bc24deac","Version":"2.0.5.3"} -- End 
-- Response String :  
{"ErrorCode":0,"Success":true,"ErrorMsg":null,"Result":null,"WaitInterval":30} -- End 

3、grok处理多种日志不同的行

日志示例：

2018-03-20 10:44:01,523 [33]DEBUG Debug - task request,task Id:1cbb72f1-a5ea-4e73-957c-6d20e9e12a7a,start time:2018-03-20 10:43:59 
-- Request String :  
{"UserName":"15046699923","Pwd":"ZYjyh727","DeviceType":2,"DeviceId":"PC-20170525SADY","EquipmentNo":null,"SSID":"pc","RegisterPhones":null,"AppKey":"ab09d78e3b2c40b789ddfc81674bc24deac","Version":"2.0.5.3"} -- End 
-- Response String :  
{"ErrorCode":0,"Success":true,"ErrorMsg":null,"Result":null,"WaitInterval":30} -- End 

在logstash filter中grok分别处理3行：

match => { 
    "message" => "^20.*- task request,.*,start time:%{TIMESTAMP_ISO8601:RequestTime}" 
match => { 
    "message" => "^-- Request String : {"UserName":"%{NUMBER:UserName:int}","Pwd":"(?<Pwd>.*)","DeviceType":%{NUMBER:DeviceType:int},"DeviceId":"(?<DeviceId>.*)","EquipmentNo":(?<EquipmentNo>.*),"SSID":(?<SSID>.*),"RegisterPhones":(?<RegisterPhones>.*),"AppKey":"(?<AppKey>.*)","Version":"(?<Version>.*)"} -- End.*"     
} 
match => { 
    "message" => "^-- Response String : {"ErrorCode":%{NUMBER:ErrorCode:int},"Success":(?<Success>[a-z]*),"ErrorMsg":(?<ErrorMsg>.*),"Result":(?<Result>.*),"WaitInterval":%{NUMBER:WaitInterval:int}} -- End.*" 
} 
... 等多行 

4、日志多行合并处理—multiline插件(重点)

示例：

①日志

2018-03-20 10:44:01,523 [33]DEBUG Debug - task request,task Id:1cbb72f1-a5ea-4e73-957c-6d20e9e12a7a,start time:2018-03-20 10:43:59 
-- Request String :  
{"UserName":"15046699923","Pwd":"ZYjyh727","DeviceType":2,"DeviceId":"PC-20170525SADY","EquipmentNo":null,"SSID":"pc","RegisterPhones":null,"AppKey":"ab09d78e3b2c40b789ddfc81674bc24deac","Version":"2.0.5.3"} -- End 
-- Response String :  
{"ErrorCode":0,"Success":true,"ErrorMsg":null,"Result":null,"WaitInterval":30} -- End 

（编辑：源码网）

【声明】本站内容均来自网络，其相关言论仅代表作者个人观点，不代表本站立场。若无意侵犯到您的权利，请及时与联系站长删除相关内容!