How to parse json array in Chronicle SIEM

[meet_crest]

Hi All,

I have already created a custom parser that works well for JSON object coming through logs. However, I now need to update the parser to support JSON arrays. but I'm unsure how to loop through a JSON array.

Sample log:  [{"test": "testing","hostname":"hostname1"},{"test":"testing","hostname":"hostname2"}]

Any guidance or assistance on how to resolve this issue would be greatly appreciated.

Thank you!

[mikewilusz]

Details on using "for" loops for looping through a JSON array are available here: https://cloud.google.com/chronicle/docs/reference/parser-syntax#for_loop

-mike

[meet_crest]

@mikewilusz Thanks for your answer, but that's not what I'm looking for. I know how to loop through a JSON array if it exists inside a JSON object. The issue here is that the ingested log is not a single JSON object; instead, it is an array of objects or you can assume this as batch of logs into single log. So, now I need to loop through the array of objects and create multiple events from a single ingested log.

[rajukg]

One of the most important things to do when looping through a "for" loop is to initialize the variables you use within the loop to null at the beginning of the loop.  

for index,record in records {
    mutate {
      replace => {
        "type_label" => ""
        "Id_label" => ""
      }
    }

    mutate {
      convert => {
        "index" => "string"
      }
      on_error => "index_conversion_failed"
    }

    mutate {
      replace => {
        "type_label.key" => "type_%{index}"
        "type_label.value" => "%{record.attributes.type}"
      }
      on_error => "type_not_found"
    }
   

In this example we are initializing the type_label to null.  If you reuse the variable without initializing it to null then you will get unpredictable results - every time you run the parser you may see different output.  Be VERY CAREFUL with this.  

[meet_crest]

@rajukgThanks for the answer. but, that's not something I am looking for. I have updated sample log in post description. Please have a look. that' might be helpful for you to understand my problem. 

[sudeep_singh]

Can you share a sample log?

[meet_crest]

@sudeep_singh I have updated post. You will find sample log in description. 

[dlove40]

You have to put the entire parser in your looping logic so it creates a UDM event for each event in your array. 

[rajukg]

You have to wrap the log with a dummy variable and then parse it with json in this case.  I changed the log as this: {"x":[{"test": "testing","hostname":"hostname1"},{"test":"testing","hostname":"hostname2"}] } and use the x in the for loop.  The following code is not changing the log itself, and for that you have to add mutate gsub.

filter {

  json {
     source => "message"
     array_function => "split_columns"
     on_error => "not_a_json"
  }
  statedump{}

  if ![not_a_json] {

    for msg in x {
      mutate {
        replace => {
          "event" => ""
        }
      }

      mutate {
        replace => {
            "event.idm.read_only_udm.metadata.event_type" => "GENERIC_EVENT"
            "event.idm.read_only_udm.principal.hostname" => "%{msg.hostname}"
        }
      }

      mutate {
          merge => {
            "@output" => "event"
          }
          on_error => "event_generation_failure"
      }
    }
  }
}