[Filebeat] 설정

다중 라인 로그를 단일 데이터로 수집하기

톰캣 로그 예시

[2019-03-02 13:01:58:922][INFO ][http-nio-8080-exec-1][org.apache.juli.logging.DirectJDKLog.log:173] Initializing Spring DispatcherServlet 'dispatcherServlet'
[2019-03-02 13:01:59:017][DEBUG][http-nio-8080-exec-1][com.example.springbootexample.HomeController.main:12] message : zzs
[2019-03-02 13:02:07:129][ERROR][http-nio-8080-exec-2][org.apache.juli.logging.DirectJDKLog.log:175] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: aaa] with root cause
java.lang.IllegalStateException: aaa
at com.example.springbootexample.HomeController.exception(HomeController.java:18)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

filebeat.yml 수정

• “[2019-03-02 13:01:58:922]” 날짜 형식으로 시작하는 부분부터 그 다음 날짜형식이 나오는 부분까지 다중라인을 하나의 메시지로 묶어 전송하기 위한 설정

filebeat.inputs:
  - type: log
    enabled: true
    paths:
      - /logs/test/input.log
    fields:
      app: test-app
    multiline.pattern: '\[[0-9]{4}-[0-9]{2}-[0-9]{2}\s[0-9]{2}:[0-9]{2}:[0-9]{2}:[0-9]{3}\]'
    multiline.negate: true
    multiline.match: after

데이터

{
    "@timestamp": "2020-09-06T14:15:27.355Z",
    "@metadata": {
        "beat": "filebeat",
        "type": "_doc",
        "version": "7.6.1"
    },
    "log": {
        "flags": [
            "multiline"
        ],
        "offset": 1270,
        "file": {
            "path": "/logs/test/input.log"
        }
    },
    "input": {
        "type": "log"
    },
    "fields": {
        "app": "test-app"
    },
    "ecs": {
        "version": "1.4.0"
    },
    "host": {
        "name": "b46feb80c34e"
    },
    "agent": {
        "type": "filebeat",
        "ephemeral_id": "5c1a7b0d-ab07-430b-8c9b-f913c4d012b6",
        "hostname": "b46feb80c34e",
        "id": "2f4e917d-0c1b-4cda-889f-311eb64d7c6d",
        "version": "7.6.1"
    },
    "message": "[2019-03-02 13:02:07:129][ERROR][http-nio-8080-exec-2][org.apache.juli.logging.DirectJDKLog.log:175] Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is java.lang.IllegalStateException: aaa] with root cause\njava.lang.IllegalStateException: aaa\nat com.example.springbootexample.HomeController.exception(HomeController.java:18)\nat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\nat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\nat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)"
}