« Connector known issues EQL syntax reference »
Elastic Docs ›Elasticsearch Guide

EQL 搜索

edit

EQL 搜索

edit

事件查询语言（EQL）是一种用于基于事件的时间序列数据的查询语言，例如日志、指标和跟踪。

EQL的优势

edit
  • EQL 允许您表达事件之间的关系。
    许多查询语言允许您匹配单个事件。EQL 允许您在不同的事件类别和时间跨度中匹配一系列事件。
  • EQL 学习曲线低。
    EQL 语法 看起来像其他常见的查询语言，如 SQL。EQL 允许您直观地编写和阅读查询，从而实现快速、迭代的搜索。
  • EQL 专为安全用例设计。
    虽然您可以将其用于任何基于事件的数据，但我们创建 EQL 是为了进行威胁狩猎。EQL 不仅支持威胁指标（IOC）搜索，还可以描述超出 IOC 的活动。

必填字段

edit

除了示例查询外，EQL 搜索要求搜索的数据流或索引包含一个 timestamp 字段。默认情况下，EQL 使用来自 Elastic Common Schema (ECS) 的 @timestamp 字段。

EQL 搜索还需要一个 事件类别 字段，除非你使用 any 关键字 来搜索没有事件类别字段的文档。默认情况下，EQL 使用 ECS 的 event.category 字段。

要使用不同的时间戳或事件类别字段，请参阅 指定时间戳或事件类别字段。

虽然使用 EQL 不需要模式，但我们建议使用 ECS。EQL 搜索默认设计为与核心 ECS 字段配合使用。

运行EQL搜索

edit

使用EQL search API来运行一个基本的EQL查询。

GET /my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

默认情况下，基本的EQL查询会返回hits.events属性中10个最新的匹配事件。这些命中结果按时间戳排序，转换为自Unix纪元以来的毫秒数，按升序排列。

{
  "is_partial": false,
  "is_running": false,
  "took": 60,
  "timed_out": false,
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "event": {
            "category": "process",
            "id": "aR3NWVOs",
            "sequence": 4
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      },
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "xLkCaj4EujzdNSxfYLbO",
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "event": {
            "category": "process",
            "id": "GTSmSqgz0U",
            "sequence": 6,
            "type": "termination"
          },
          "process": {
            "pid": 2012,
            "name": "regsvr32.exe",
            "executable": "C:\\Windows\\System32\\regsvr32.exe"
          }
        }
      }
    ]
  }
}

使用 size 参数来获取更小或更大的命中集：

GET /my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "size": 50
}

搜索一系列事件

edit

使用EQL的序列语法来搜索一系列有序的事件。按时间顺序升序列出事件项，最新的事件列在最后：

GET /my-data-stream/_eql/search
{
  "query": """
    sequence
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

响应的 hits.sequences 属性包含最近的10个匹配序列。

{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "OQmfCaduce8zoHT93o4H",
            "_source": {
              "@timestamp": "2099-12-07T11:07:09.000Z",
              "event": {
                "category": "process",
                "id": "aR3NWVOs",
                "sequence": 4
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "command_line": "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              }
            }
          },
          {
            "_index": ".ds-my-data-stream-2099.12.07-000001",
            "_id": "yDwnGIJouOYGBzP0ZE9n",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

使用 with maxspan 来限制匹配序列的时间跨度：

GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

使用 ! 来匹配 缺失事件：在给定时间跨度内未满足条件的序列中的事件：

GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1d
      [ process where process.name == "cmd.exe" ]
      ![ process where stringContains(process.command_line, "ocx") ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

缺失的事件在响应中表示为 missing": true：

{
  ...
  "hits": {
    "total": ...,
    "sequences": [
      {
        "events": [
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "AnpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:06:07.000Z",
              "event": {
                "category": "process",
                "id": "cMyt5SZ2",
                "sequence": 3
              },
              "process": {
                "pid": 2012,
                "name": "cmd.exe",
                "executable": "C:\\Windows\\System32\\cmd.exe"
              }
            }
          },
          {
            "_index": "",
            "_id": "",
            "_source": {},
            "missing": true
          },
          {
            "_index": ".ds-my-data-stream-2023.07.04-000001",
            "_id": "BHpTIYkBrVQ2QEgsWg94",
            "_source": {
              "@timestamp": "2099-12-07T11:07:10.000Z",
              "event": {
                "category": "file",
                "id": "tZ1NWVOs",
                "sequence": 5
              },
              "process": {
                "pid": 2012,
                "name": "regsvr32.exe",
                "executable": "C:\\Windows\\System32\\regsvr32.exe"
              },
              "file": {
                "path": "C:\\Windows\\System32\\scrobj.dll",
                "name": "scrobj.dll"
              }
            }
          }
        ]
      }
    ]
  }
}

使用 by 关键字 来匹配共享相同字段值的事件：

GET /my-data-stream/_eql/search
{
  "query": """
    sequence with maxspan=1h
      [ process where process.name == "regsvr32.exe" ] by process.pid
      [ file where stringContains(file.name, "scrobj.dll") ] by process.pid
  """
}

如果一个字段的值应该在所有事件中共享，请使用sequence by关键字。以下查询与前一个查询等效。

GET /my-data-stream/_eql/search
{
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
  """
}

属性 hits.sequences.join_keys 包含共享的字段值。

{
  ...
  "hits": ...,
    "sequences": [
      {
        "join_keys": [
          2012
        ],
        "events": ...
      }
    ]
  }
}

使用until关键字来指定序列的过期事件。匹配的序列必须在此事件之前结束。

GET /my-data-stream/_eql/search
{
  "query": """
    sequence by process.pid with maxspan=1h
      [ process where process.name == "regsvr32.exe" ]
      [ file where stringContains(file.name, "scrobj.dll") ]
    until [ process where event.type == "termination" ]
  """
}

时间顺序未排序的事件示例

edit

使用 EQL 的 示例语法 来搜索匹配一个或多个连接键和一组过滤器的事件。示例类似于序列，但不按时间顺序返回事件。事实上，示例查询可以在没有时间戳的数据上运行。示例查询可用于查找不总是按相同顺序发生或发生在长时间跨度内的事件之间的相关性。

点击显示以下示例中使用的样本数据 
PUT /my-index-000001
{
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "version": {
                "type": "version"
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "@timestamp": {
              "type": "date"
            },
            "type_test": {
                "type": "keyword"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "dd-MM-yyyy"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000002
{
    "mappings": {
        "properties": {
            "ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "@timestamp_pretty": {
              "type": "date",
              "format": "yyyy-MM-dd"
            },
            "type_test": {
                "type": "keyword"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "host": {
              "type": "keyword"
            },
            "op_sys": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

PUT /my-index-000003
{
    "mappings": {
        "properties": {
            "host_ip": {
                "type":"ip"
            },
            "@timestamp": {
              "type": "date"
            },
            "date": {
              "type": "date"
            },
            "event_type": {
              "type": "keyword"
            },
            "event": {
              "properties": {
                "category": {
                  "type": "alias",
                  "path": "event_type"
                }
              }
            },
            "missing_keyword": {
                "type": "keyword"
            },
            "host": {
              "type": "keyword"
            },
            "os": {
              "type": "keyword"
            },
            "bool": {
              "type": "boolean"
            },
            "uptime" : {
              "type" : "long"
            },
            "port" : {
              "type" : "long"
            }
        }
    }
}

POST /my-index-000001/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567891","@timestamp_pretty":"12-12-2022","missing_keyword":"test","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"os":"win10","version":"1.0.0","id":11}
{"index":{"_id":2}}
{"@timestamp":"1234567892","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"CS","uptime":5,"port":1,"os":"win10","version":"1.2.0","id":12}
{"index":{"_id":3}}
{"@timestamp":"1234567893","@timestamp_pretty":"12-12-2022","event_type":"alert","type_test":"abc","host":"farcry","uptime":1,"port":1234,"bool":false,"os":"win10","version":"2.0.0","id":13}
{"index":{"_id":4}}
{"@timestamp":"1234567894","@timestamp_pretty":"13-12-2022","event_type":"alert","type_test":"abc","host":"GTA","uptime":3,"port":12,"os":"slack","version":"10.0.0","id":14}
{"index":{"_id":5}}
{"@timestamp":"1234567895","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"os":"fedora","version":"20.1.0","id":15}
{"index":{"_id":6}}
{"@timestamp":"1234568896","@timestamp_pretty":"17-12-2022","event_type":"alert","host":"doom","port":65123,"bool":true,"os":"redhat","version":"20.10.0","id":16}
{"index":{"_id":7}}
{"@timestamp":"1234567897","@timestamp_pretty":"17-12-2022","missing_keyword":"yyy","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"os":"redhat","version":"20.2.0","id":17}
{"index":{"_id":8}}
{"@timestamp":"1234567898","@timestamp_pretty":"12-12-2022","missing_keyword":"test","event_type":"success","host":"doom","uptime":16,"port":512,"os":"win10","version":"1.2.3","id":18}
{"index":{"_id":9}}
{"@timestamp":"1234567899","@timestamp_pretty":"15-12-2022","missing_keyword":"test","event_type":"success","host":"GTA","port":12,"bool":true,"os":"win10","version":"1.2.3","id":19}
{"index":{"_id":10}}
{"@timestamp":"1234567893","missing_keyword":null,"ip":"10.0.0.5","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":true,"os":"win10","version":"1.2.3","id":110}

POST /my-index-000002/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1234567991","type_test":"abc","ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":1234,"op_sys":"win10","id":21}
{"index":{"_id":2}}
{"@timestamp":"1234567992","type_test":"abc","event_type":"alert","host":"CS","uptime":5,"port":1,"op_sys":"win10","id":22}
{"index":{"_id":3}}
{"@timestamp":"1234567993","type_test":"abc","@timestamp_pretty":"2022-12-17","event_type":"alert","host":"farcry","uptime":1,"port":1234,"bool":false,"op_sys":"win10","id":23}
{"index":{"_id":4}}
{"@timestamp":"1234567994","event_type":"alert","host":"GTA","uptime":3,"port":12,"op_sys":"slack","id":24}
{"index":{"_id":5}}
{"@timestamp":"1234567995","event_type":"alert","host":"sniper 3d","uptime":6,"port":1234,"op_sys":"fedora","id":25}
{"index":{"_id":6}}
{"@timestamp":"1234568996","@timestamp_pretty":"2022-12-17","ip":"10.0.0.5","event_type":"alert","host":"doom","port":65123,"bool":true,"op_sys":"redhat","id":26}
{"index":{"_id":7}}
{"@timestamp":"1234567997","@timestamp_pretty":"2022-12-17","event_type":"failure","host":"doom","uptime":15,"port":1234,"bool":true,"op_sys":"redhat","id":27}
{"index":{"_id":8}}
{"@timestamp":"1234567998","ip":"10.0.0.1","event_type":"success","host":"doom","uptime":16,"port":512,"op_sys":"win10","id":28}
{"index":{"_id":9}}
{"@timestamp":"1234567999","ip":"10.0.0.1","event_type":"success","host":"GTA","port":12,"bool":false,"op_sys":"win10","id":29}

POST /my-index-000003/_bulk?refresh
{"index":{"_id":1}}
{"@timestamp":"1334567891","host_ip":"10.0.0.1","event_type":"alert","host":"doom","uptime":0,"port":12,"os":"win10","id":31}
{"index":{"_id":2}}
{"@timestamp":"1334567892","event_type":"alert","host":"CS","os":"win10","id":32}
{"index":{"_id":3}}
{"@timestamp":"1334567893","event_type":"alert","host":"farcry","bool":true,"os":"win10","id":33}
{"index":{"_id":4}}
{"@timestamp":"1334567894","event_type":"alert","host":"GTA","os":"slack","bool":true,"id":34}
{"index":{"_id":5}}
{"@timestamp":"1234567895","event_type":"alert","host":"sniper 3d","os":"fedora","id":35}
{"index":{"_id":6}}
{"@timestamp":"1234578896","host_ip":"10.0.0.1","event_type":"alert","host":"doom","bool":true,"os":"redhat","id":36}
{"index":{"_id":7}}
{"@timestamp":"1234567897","event_type":"failure","missing_keyword":"test","host":"doom","bool":true,"os":"redhat","id":37}
{"index":{"_id":8}}
{"@timestamp":"1234577898","event_type":"success","host":"doom","os":"win10","id":38,"date":"1671235200000"}
{"index":{"_id":9}}
{"@timestamp":"1234577899","host_ip":"10.0.0.5","event_type":"success","host":"GTA","bool":true,"os":"win10","id":39}

一个示例查询至少指定一个连接键，使用by关键字，并最多可包含五个过滤器：

GET /my-index*/_eql/search
{
  "query": """
    sample by host
      [any where uptime > 0]
      [any where port > 100]
      [any where bool == true]
  """
}

默认情况下，响应的 hits.sequences 属性包含最多 10 个样本。 每个样本都有一组 join_keys 和一个数组，其中包含每个过滤器的匹配事件。事件按它们匹配的过滤器顺序返回：

{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom"                                      
        ],
        "events": [
          {                                           
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {                                           
            "_index": "my-index-000001",
            "_id": "1",
            "_source": {
              "@timestamp": "1234567891",
              "@timestamp_pretty": "12-12-2022",
              "missing_keyword": "test",
              "type_test": "abc",
              "ip": "10.0.0.1",
              "event_type": "alert",
              "host": "doom",
              "uptime": 0,
              "port": 1234,
              "os": "win10",
              "version": "1.0.0",
              "id": 11
            }
          },
          {                                           
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry"                                    
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          },
          {
            "_index": "my-index-000003",
            "_id": "3",
            "_source": {
              "@timestamp": "1334567893",
              "event_type": "alert",
              "host": "farcry",
              "bool": true,
              "os": "win10",
              "id": 33
            }
          }
        ]
      }
    ]
  }
}

第一个样本中的事件在host字段上的值为doom。

此事件匹配第一个过滤器。

此事件匹配第二个过滤器。

此事件匹配第三个过滤器。

第二个样本中的事件在host字段上的值为farcry。

您可以指定多个连接键：

GET /my-index*/_eql/search
{
  "query": """
    sample by host
      [any where uptime > 0]   by os
      [any where port > 100]   by op_sys
      [any where bool == true] by os
  """
}

此查询将返回样本，其中每个事件的os或op_sys以及host的值相同。例如：

{
  ...
  "hits": {
    "total": {
      "value": 2,
      "relation": "eq"
    },
    "sequences": [
      {
        "join_keys": [
          "doom",                                      
          "redhat"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "7",
            "_source": {
              "@timestamp": "1234567897",
              "@timestamp_pretty": "17-12-2022",
              "missing_keyword": "yyy",
              "event_type": "failure",
              "host": "doom",
              "uptime": 15,
              "port": 1234,
              "bool": true,
              "os": "redhat",
              "version": "20.2.0",
              "id": 17
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568996",
              "@timestamp_pretty": "2022-12-17",
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "op_sys": "redhat",
              "id": 26
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "6",
            "_source": {
              "@timestamp": "1234568896",
              "@timestamp_pretty": "17-12-2022",
              "event_type": "alert",
              "host": "doom",
              "port": 65123,
              "bool": true,
              "os": "redhat",
              "version": "20.10.0",
              "id": 16
            }
          }
        ]
      },
      {
        "join_keys": [
          "farcry",
          "win10"
        ],
        "events": [
          {
            "_index": "my-index-000001",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567893",
              "@timestamp_pretty": "12-12-2022",
              "event_type": "alert",
              "type_test": "abc",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "os": "win10",
              "version": "2.0.0",
              "id": 13
            }
          },
          {
            "_index": "my-index-000002",
            "_id": "3",
            "_source": {
              "@timestamp": "1234567993",
              "type_test": "abc",
              "@timestamp_pretty": "2022-12-17",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": false,
              "op_sys": "win10",
              "id": 23
            }
          },
          {
            "_index": "my-index-000001",
            "_id": "10",
            "_source": {
              "@timestamp": "1234567893",
              "missing_keyword": null,
              "ip": "10.0.0.5",
              "event_type": "alert",
              "host": "farcry",
              "uptime": 1,
              "port": 1234,
              "bool": true,
              "os": "win10",
              "version": "1.2.3",
              "id": 110
            }
          }
        ]
      }
    ]
  }
}

此示例中的事件在host上具有doom的值，并且在os或op_sys上具有redhat的值。

默认情况下，样本查询的响应包含最多10个样本，每个唯一的连接键集对应一个样本。使用size参数可以获取更小或更大的样本集。要为每个连接键集检索多个样本，请使用max_samples_per_key参数。样本查询不支持管道。

GET /my-index*/_eql/search
{
  "max_samples_per_key": 2,     
  "size": 20,                   
  "query": """
    sample
      [any where uptime > 0]   by host,os
      [any where port > 100]   by host,op_sys
      [any where bool == true] by host,os
  """
}

每组连接键最多检索2个样本。

总共检索最多20个样本。

检索选定字段

edit

默认情况下，搜索响应中的每个命中项都包含文档 _source， 即索引文档时提供的整个 JSON 对象。

您可以使用filter_path查询参数来过滤API响应。例如，以下搜索仅返回每个匹配事件的_source中的时间戳和PID。

GET /my-data-stream/_eql/search?filter_path=hits.events._source.@timestamp,hits.events._source.process.pid
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

API返回以下响应。

{
  "hits": {
    "events": [
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:09.000Z",
          "process": {
            "pid": 2012
          }
        }
      },
      {
        "_source": {
          "@timestamp": "2099-12-07T11:07:10.000Z",
          "process": {
            "pid": 2012
          }
        }
      }
    ]
  }
}

您还可以使用fields参数来检索和格式化响应中的特定字段。此字段与搜索API的fields参数相同。

因为它会参考索引映射，fields 参数提供了几个相对于直接引用 _source 的优势。具体来说，fields 参数：

以下搜索请求使用fields参数来检索event.type字段的值，所有以process.开头的字段，以及@timestamp字段的值。该请求还使用filter_path查询参数来排除每个命中的_source。

GET /my-data-stream/_eql/search?filter_path=-hits.events._source
{
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "event.type",
    "process.*",                
    {
      "field": "@timestamp",
      "format": "epoch_millis"  
    }
  ]
}

接受完整的字段名称和通配符模式。

使用 format 参数为字段的值应用自定义格式。

响应包括每个命中项的fields部分中的扁平列表值。

{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "process.name": [
            "regsvr32.exe"
          ],
          "process.name.keyword": [
            "regsvr32.exe"
          ],
          "@timestamp": [
            "4100324829000"
          ],
          "process.command_line": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.command_line.keyword": [
            "regsvr32.exe  /s /u /i:https://...RegSvr32.sct scrobj.dll"
          ],
          "process.executable.keyword": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ],
          "process.pid": [
            2012
          ],
          "process.executable": [
            "C:\\Windows\\System32\\regsvr32.exe"
          ]
        }
      },
      ....
    ]
  }
}

使用运行时字段

edit

使用 runtime_mappings 参数在搜索期间提取并创建 运行时字段。使用 fields 参数在响应中包含运行时字段。

以下搜索从@timestamp创建一个day_of_week运行时字段，并在响应中返回它。

GET /my-data-stream/_eql/search?filter_path=-hits.events._source
{
  "runtime_mappings": {
    "day_of_week": {
      "type": "keyword",
      "script": "emit(doc['@timestamp'].value.dayOfWeekEnum.toString())"
    }
  },
  "query": """
    process where process.name == "regsvr32.exe"
  """,
  "fields": [
    "@timestamp",
    "day_of_week"
  ]
}

API返回：

{
  ...
  "hits": {
    "total": ...,
    "events": [
      {
        "_index": ".ds-my-data-stream-2099.12.07-000001",
        "_id": "OQmfCaduce8zoHT93o4H",
        "fields": {
          "@timestamp": [
            "2099-12-07T11:07:09.000Z"
          ],
          "day_of_week": [
            "MONDAY"
          ]
        }
      },
      ....
    ]
  }
}

指定时间戳或事件类别字段

edit

EQL 搜索 API 默认使用来自 ECS 的 @timestamp 和 event.category 字段。要指定不同的字段，请使用 timestamp_field 和 event_category_field 参数：

GET /my-data-stream/_eql/search
{
  "timestamp_field": "file.accessed",
  "event_category_field": "file.type",
  "query": """
    file where (file.size > 1 and file.type == "file")
  """
}

事件类别字段必须映射为keyword类型族字段。时间戳字段应映射为date字段类型。date_nanos时间戳字段不受支持。您不能使用nested字段或nested字段的子字段作为时间戳或事件类别字段。

指定排序的决胜属性

edit

默认情况下，EQL搜索API按时间戳返回匹配的命中结果。如果两个或更多事件共享相同的时间戳，Elasticsearch使用一个tiebreaker字段值来按升序对事件进行排序。Elasticsearch将没有tiebreaker值的事件排在有值的事件之后。

如果你没有指定一个决胜字段，或者事件也共享相同的决胜值，Elasticsearch 认为这些事件是并发的，并且可能不会以一致的排序顺序返回它们。

要指定一个平局决胜字段，请使用tiebreaker_field参数。如果你使用ECS，我们建议使用event.sequence作为平局决胜字段。

GET /my-data-stream/_eql/search
{
  "tiebreaker_field": "event.sequence",
  "query": """
    process where process.name == "cmd.exe" and stringContains(process.executable, "System32")
  """
}

使用查询DSL进行过滤

edit

参数 filter 使用 Query DSL 来限制 EQL 查询运行的文档。

GET /my-data-stream/_eql/search
{
  "filter": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "query": """
    file where (file.type == "file" and file.name == "cmd.exe")
  """
}

运行异步EQL搜索

edit

默认情况下，EQL搜索请求是同步的，并在返回响应之前等待完整结果。然而，对于跨大数据集或冻结数据的搜索，完整结果可能需要更长时间。

为了避免长时间等待，请运行一个异步EQL搜索。将wait_for_completion_timeout设置为你希望等待同步结果的时间长度。

GET /my-data-stream/_eql/search
{
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

如果请求在超时时间内未完成，搜索将变为异步，并返回一个包含以下内容的响应：

  • 一个搜索ID
  • 一个is_partial值为true，表示搜索结果不完整
  • 一个is_running值为true，表示搜索正在进行中

异步搜索继续在后台运行，不会阻塞其他请求。

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": true,
  "is_running": true,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

要检查异步搜索的进度，请使用带有搜索ID的获取异步EQL搜索API。在wait_for_completion_timeout参数中指定您希望等待完整结果的时间。

GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=2s

如果响应的 is_running 值为 false，则异步搜索已完成。 如果 is_partial 值为 false，则返回的搜索结果是完整的。

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_partial": false,
  "is_running": false,
  "took": 2000,
  "timed_out": false,
  "hits": ...
}

另一种更轻量级的方式来检查异步搜索的进度是使用带有搜索ID的获取异步EQL状态API。

GET /_eql/search/status/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=
{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running": false,
  "is_partial": false,
  "expiration_time_in_millis": 1611690295000,
  "completion_status": 200
}

更改搜索保留期限

edit

默认情况下，EQL 搜索 API 将异步搜索存储五天。在此期间之后，任何搜索及其结果都会被删除。使用 keep_alive 参数来更改此保留期：

GET /my-data-stream/_eql/search
{
  "keep_alive": "2d",
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

您可以使用获取异步EQL搜索API的 keep_alive参数来稍后更改保留期。新的保留期在获取请求运行后开始。

GET /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?keep_alive=5d

使用删除异步EQL搜索API在keep_alive周期结束之前手动删除异步EQL搜索。如果搜索仍在进行中，Elasticsearch将取消搜索请求。

DELETE /_eql/search/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=

存储同步EQL搜索

edit

默认情况下，EQL 搜索 API 仅存储异步搜索。要保存同步搜索，请将 keep_on_completion 设置为 true：

GET /my-data-stream/_eql/search
{
  "keep_on_completion": true,
  "wait_for_completion_timeout": "2s",
  "query": """
    process where process.name == "cmd.exe"
  """
}

响应包括一个搜索ID。is_partial 和 is_running 是 false， 表示EQL搜索是同步的并返回了完整的结果。

{
  "id": "FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=",
  "is_partial": false,
  "is_running": false,
  "took": 52,
  "timed_out": false,
  "hits": ...
}

使用 get async EQL search API 来稍后获取相同的结果：

GET /_eql/search/FjlmbndxNmJjU0RPdExBTGg0elNOOEEaQk9xSjJBQzBRMldZa1VVQ2pPa01YUToxMDY=

保存的同步搜索仍然受keep_alive参数的保留期限制。当此期限结束时，搜索及其结果将被删除。

您还可以通过使用获取异步EQL状态API来仅检查已保存的同步搜索的状态，而不获取结果。

您也可以使用删除异步EQL搜索API手动删除保存的同步搜索。

在集群中运行EQL搜索

edit

此功能处于技术预览阶段，可能会在未来的版本中进行更改或移除。Elastic 将努力修复任何问题，但技术预览版中的功能不受官方 GA 功能支持 SLA 的约束。

EQL 搜索 API 支持 跨集群搜索。 然而，如果本地和 远程集群 的版本早于 7.17.7（包括）或早于 8.5.1（包括），则它们必须使用相同的 Elasticsearch 版本。

以下集群更新设置请求添加了两个远程集群：cluster_one 和 cluster_two。

PUT /_cluster/settings
{
  "persistent": {
    "cluster": {
      "remote": {
        "cluster_one": {
          "seeds": [
            "127.0.0.1:9300"
          ]
        },
        "cluster_two": {
          "seeds": [
            "127.0.0.1:9301"
          ]
        }
      }
    }
  }
}

要定位远程集群上的数据流或索引，请使用:语法。

GET /cluster_one:my-data-stream,cluster_two:my-data-stream/_eql/search
{
  "query": """
    process where process.name == "regsvr32.exe"
  """
}

EQL 断路器设置

edit

相关的断路器设置可以在断路器页面中找到。

« Connector known issues EQL syntax reference »