Understanding Interflow

Interflow is how Stellar Cyber represents security alerts, network traffic, application, and log data inside the data processor. Interflow is JSON, so it is both human-readable and structured data, allowing us to ingest, store, and analyze massive amounts of data.

Sensors and agents transform raw data into Interflow records as data is ingested and processed. The data reduction from PCAP to Interflow averages 100:1, reducing metadata transmission over wide area links and saving on bandwidth costs.

This provides significant benefits:

  • Evidence that the alert is legitimate, without the size of a full packet capture
  • Indexable, readable, and searchable with the click of a button
  • Pervasive data collection at the core
  • Contextual flow records
  • Captures:
    • Information from L2 – L7
      • Network data
      • Server data
      • Application data
      • User data
      • Syslog logs
    • L4-L7 performance metrics
    • UDLs, URIs, and domain names
    • Code signing certificates within files
    • Authentication failures
    • MD5 hashes of files downloaded
  • Interflow is a record of the captured data represented as metadata
    • Sensors convert various types of data to metadata
    • Network packets, server logs, server process data, file data, and threat intelligence data are all converted to JSON-formatted documents that are transmitted to the DP for deduplication, enrichment, indexing, and storage
  • Allows for the enrichment of data; for example converting an IP address to a geographical location or reputation

Interflow Record

The primary use of Interflow records are to represent events. Typically these originate in sensors in response to some real-world activity but they can also be generated by the Stellar Cyber machine-learning processes. For a description of this process see the Architecture overview page.

Further, Interflow records can be changed during their lifecycle as part of the data enhancement process. For example, an IP address stored in Interflow records will later cause reputation and geographical information to be added. For security events, a calculated score of severity or fidelity will be added.

Each record stores data and metadata as a series of key-value pairs where the values can themselves be nested objects. Because of this flexible and extensible nature, Interflow records are often expressed as JSON (JavaScript Object Notation) objects.

Common Keys

Interflow embraces potentially many hundreds of keys. More are added as new capabilities and features are added to Stellar Cyber. Specifically connectors, which serve as an interface between an outside data source and Stellar Cyber add new keys that originate in the interface specification of the outside source.

The following table shows some of the more common keys. The meaning of other keys can be inferred from context and name.

The definition of an Interflow key is not necessarily intuitive and may be dependent on contextual factors. In cases of ambiguity the Stellar Cyber user interface and logic should be relied on for correct interpretation.

 

Key Definition Example Value
action The category of action that was taken. "opened-file"
appid_family The category/family of the application. "Web"
appid_stdport The value "yes" indicates that the "standard" IP port number was used. "yes"
command The command that was executed. "/usr/sbin/acpid"
dst_reputation Reputation of the destination host.  
dstip The destination IP address. "168.63.129.16"
dstip_geo Geographic information associated with the destination IP address.  
dstip_type The type of destination IP address: public or private. "public"
dstport The port number used at the destination. 32526,
engid_device_class The class of the device. "Linux"
engid_device_desc" The detected Operating System name "ubuntu-16.04"
engid_gateway The IP of the gateway used by the device being observed. "52.231.166.217"
event_category The category of event. "killchain"
event_description A full text description of why the Security Event was created. "A file action was taken on a file such as move, copy, delete or a change in attributes that has never been seen before or at an anomalous rate."
event_name The name of the event "anomalous_file_action"
event_source The source of the event. The value "ml" indicates that the record was created from Machine Learning "ml"
event_type The type of event. "cnc"
fidelity The Fidelity score as determined by the Machine Learning engine. 98.6005455871342
metadata The metadata associated with the HTTP transaction that occurred.  
parent_proc_name The name of the parent process that initiated the action. "systemd"
path The path name determined in the event. "/bin/ghampzejqjbayb"
primary The user identity that executed the process. "root"
process_name The name of the process that was executed. "zngeqm"
program_name The name of the program that was executed. "/bin/zngeqm"
proto The number of the protocol found in the IP header. 6
record_type The type of record. "syscall"
severity The Severity score of the event as determined by ML. 88
src_reputation Reputation of the source host.  
srcip The source IP address "10.0.10.4"
srcip_geo Geographic information regarding the source IP address.  
srcip_reputation_source The source of reputation information used in determining the reputation of the source. "StellarCyber"
srcip_type The type of IP address of the source: public or private. "public"
targetfile_create The file that was created in the system.  
tenant_name The name of the Tenant associated with the current record. "Korea"
tenantid The unique ID of the Tenant. This corresponds to the tenant_name field. "62804025"
timestamp The time the event occurred. 1556466300671
totalbytes The number of bytes transferred. 4891

Example

The following image shows an example of key-value pairs in an Interflow record.