Understanding Interflow
Interflow is how Stellar Cyber represents security alerts, network traffic, application, and log data inside the data processor. Interflow is JSON, so it is both human-readable and structured data, allowing us to ingest, store, and analyze massive amounts of data.
Sensors and agents transform raw data into Interflow records as data is ingested and processed. The data reduction from PCAP to Interflow averages 100:1, reducing metadata transmission over wide area links and saving on bandwidth costs.
This provides significant benefits:
- Evidence that the alert is legitimate, without the size of a full packet capture
- Indexable, readable, and searchable with the click of a button
- Pervasive data collection at the core
- Contextual flow records
- Captures:
- Information from L2 – L7
- Network data
- Server data
- Application data
- User data
- Syslog logs
- L4-L7 performance metrics
- UDLs, URIs, and domain names
- Code signing certificates within files
- Authentication failures
- MD5 hashes of files downloaded
- Information from L2 – L7
- Interflow is a record of the captured data represented as metadata
- Sensors convert various types of data to metadata
- Network packets, server logs, server process data, file data, and threat intelligence data are all converted to JSON-formatted documents that are transmitted to the DP for de-duplication, enrichment, indexing, and storage
- Allows for the enrichment of data; for example converting an IP address to a geographical location or reputation
Interflow Record
The primary use of Interflow records are to represent events. Typically these originate in sensors in response to some real-world activity but they can also be generated by the Stellar Cyber machine-learning processes. For a description of this process see the Architecture overview page.
Further, Interflow records can be changed during their lifecycle as part of the data enhancement process. For example, an IP address stored in Interflow records will later cause reputation and geographical information to be added. For security events, a calculated score of severity or fidelity will be added.
Each record stores data and metadata as a series of key-value pairs where the values can themselves be nested objects. Because of this flexible and extensible nature, Interflow records are often expressed as JSON (JavaScript Object Notation) objects.
Common Keys
Interflow embraces potentially many hundreds of keys. More are added as new capabilities and features are added to Stellar Cyber. Specifically connectors, which serve as an interface between an outside data source and Stellar Cyber add new keys that originate in the interface specification of the outside source.
The following table shows some of the more common keys. The meaning of other keys can be inferred from context and name.
The definition of an Interflow key is not necessarily intuitive and may be dependent on contextual factors. In cases of ambiguity the Stellar Cyber user interface and logic should be relied on for correct interpretation.
Key | Definition | Example Value |
---|---|---|
action | The category of action that was taken. | "opened-file" |
appid_family | The category/family of the application. | "Web" |
appid_stdport | The value "yes" indicates that the "standard" IP port number was used. | "yes" |
command | The command that was executed. | "/usr/sbin/acpid" |
dst_reputation | Reputation of the destination host. | |
dstip | The destination IP address. | "168.63.129.16" |
dstip_geo | Geographic information associated with the destination IP address. | |
dstip_type | The type of destination IP address: public or private. | "public" |
dstport | The port number used at the destination. | 32526, |
engid_device_class | The class of the device. | "Linux" |
engid_device_desc" | The detected Operating System name | "ubuntu-16.04" |
engid_gateway | The IP of the gateway used by the device being observed. | "52.231.166.217" |
event_category | The category of event. | "killchain" |
event_description | A full text description of why the Security Event was created. | "A file action was taken on a file such as move, copy, delete or a change in attributes that has never been seen before or at an anomalous rate." |
event_name | The name of the event | "anomalous_file_action" |
event_source | The source of the event. The value "ml" indicates that the record was created from Machine Learning | "ml" |
event_type | The type of event. | "cnc" |
fidelity | The Fidelity score as determined by the Machine Learning engine. | 98.6005455871342 |
metadata | The metadata associated with the HTTP transaction that occurred. | |
parent_proc_name | The name of the parent process that initiated the action. | "systemd" |
path | The path name determined in the event. | "/bin/ghampzejqjbayb" |
primary | The user identity that executed the process. | "root" |
process_name | The name of the process that was executed. | "zngeqm" |
program_name | The name of the program that was executed. | "/bin/zngeqm" |
proto | The number of the protocol found in the IP header. | 6 |
record_type | The type of record. | "syscall" |
severity | The Severity score of the event as determined by ML. | 88 |
src_reputation | Reputation of the source host. | |
srcip | The source IP address | "10.0.10.4" |
srcip_geo | Geographic information regarding the source IP address. | |
srcip_reputation_source | The source of reputation information used in determining the reputation of the source. | "StellarCyber" |
srcip_type | The type of IP address of the source: public or private. | "public" |
targetfile_create | The file that was created in the system. | |
tenant_name | The name of the Tenant associated with the current record. | "Korea" |
tenantid | The unique ID of the Tenant. This corresponds to the tenant_name field. | "62804025" |
timestamp | The time the event occurred. | 1556466300671 |
totalbytes | The number of bytes transferred. | 4891 |
Example
The following image shows an example of key-value pairs in an Interflow record.