[Elasticsearch] 파이프라인

들어가기 전

• Elasticsearch 서버 주소 : 192.168.56.101:9200

파이프라인

• 처리의 흐름을 정의한 것
• 데이터 등록 시에 호출하여 전처리하는 동작을 수행

파이프라인 설정

curl -X PUT http://192.168.56.101:9200/_ingest/pipeline/test-pipeline  -H "Content-Type: application/json" -d '
{
    "description": "parse number and clientip using grok",
    "processors": [
        {
            "grok": {
                "field": "clientInfo",
                "patterns": [
                    "%{NUMBER:duration} %{IP:ip}"
                ]
            },
            "remove": {
                "field": "clientInfo"
            }
        }
    ]
}
'

파이프라인 동작 테스트

curl -X POST http://192.168.56.101:9200/_ingest/pipeline/test-pipeline/_simulate  -H "Content-Type: application/json" -d '
{
    "docs": [
        {
            "_source": {
                "clientInfo": "3.44 192.168.56.101"
            }
        }
    ]
}
'

데이터 추가

curl -X PUT http://192.168.56.101:9200/test-index/test-type/1?pipeline=test-pipeline -H "Content-Type: application/json" -d '
{
    "clientInfo": "3.44 192.168.56.101"
}
'

데이터 확인

curl -X GET http://192.168.56.101:9200/test-index/_search -H "Content-Type: application/json" -d '
{
    "query": {
        "match_all": {}
    }
}
'