Multiple search results generate nested objects

[spacepatcher]

I use your plugin to enrich events with data from external reputation database. My goal is to process the enriched data from my external database with JDBC support using visualization in Kibana.

There is a key snippet from my logstash.conf:

statement => "select feed_name, first_seen, last_added from feeds_aggregated where ip = :lookupIP"
parameters => { "lookupIP" => "ip" }
target => "[enrich][fia_nested]"

After Logstash processing I get enriched data with such structure in Elasticsearch:

"enrich": {
      "fia_nested": [
        {
          "feed_name": "blocklist_de_apache",
          "first_seen": "2018-07-19T14:16:03.714Z",
          "last_added": "2018-07-19T14:16:03.714Z"
        },
        {
          "feed_name": "urandomusto_http",
          "first_seen": "2018-07-19T14:18:03.829Z",
          "last_added": "2018-07-19T14:18:03.829Z"
        }
      ]

The restriction of Kibana does not allow properly processing nested objects. (https://www.elastic.co/guide/en/kibana/current/nested-objects.html).

How can I manage the result data structure using you plugin?

[spacepatcher]

I was able to cope with the situation using this ruby code in my logstash.conf:

ruby {
    code => "
      event.get('[enrich][fia_nested]').each_index { |i|
        event.set('[enrich][fia]['+i.to_s+'][feed_name]', event.get('[enrich][fia_nested]['+i.to_s+'][feed_name]'))
        event.set('[enrich][fia]['+i.to_s+'][first_seen]', event.get('[enrich][fia_nested]['+i.to_s+'][first_seen]'))
        event.set('[enrich][fia]['+i.to_s+'][last_added]', event.get('[enrich][fia_nested]['+i.to_s+'][last_added]'))
      }
    "
  }