Managing the Data Lake

You must have Root scope to use this feature.

The Data Lake is the central component that stores data within the Stellar Cyber system. You must have at least one Data Lake. You can arrange more than one in a cluster. If you have multiple Data Lakes, one instance must be designated as the "master" node, and all others in the cluster designated as "worker" nodes. Having multiple nodes increases capacity and provides a measure of fault tolerance. Navigate to System | Data Processor | Data Lake to view these configuration options.

Be very careful when changing the Data Lake configurations. Misconfigurations negatively affect the functioning of Stellar Cyber.

Data Lake Table

The screen displays a table of the existing Data Lakes, as in the example in the following image.

The fields of the table include:

  • Name – The name of the Data Lake instance when it was installed.

  • Status – The status of the Data Lake. A green dot indicates operational status.

  • IP Address – The IP address used by the Data Lake.

  • File Sync Interval – The time (in seconds) between file sync operations.

  • Node List – The number of nodes in each Data Lake. Click the button in this column to see a dialog box listing the associated nodes, as below.

  • Last Seen – The time of the last status update.
  • Software Version – The version of software that is running in the Data Lake.

  • Running Coordinating Node – Indicates whether the Data Lake includes DL Coordinating Nodes. You can see details on Coordinating Node status in the Node List, as illustrated below. Refer to Scaling Up the Data Lake with Coordinating Nodes for details on working with Coordinating Nodes.

Actions that can be taken include:

  • Click the   Check box at the start of one or more rows, then click the Download Debug Logs button (details below).

  • Click the button to display the Scale Up Data Lake dialog box (see below).

  • Click the button to delete the Data Lake. This only applies if you have a cluster.

  • Click the File Sync button to initiate a new file sync operation outside of the regular schedule.

Data Lakes are created during the Stellar Cyber installation process. There is no add button on this screen to add a Data Lake.

Scale Up Data Lake

The Scale Up Data Lake dialog box lets you modify the Data Lake configuration.

  • Data Lake Name – The name of the Data Lake when created. You cannot change this.

  • Cluster Name – The name of the Cluster when created. You cannot change this.

  • Enable File Sync – When you enable this you can configure the File Sync Interval.

  • Cluster Nodes – The nodes in the cluster.

  • Note – a free-form field for notes. This field does not support multibyte characters.