Using the Data Sink Restore Tab

You must have Root scope to use this feature.

The System | Data Processor | Data Management | Data Sink Restore tab lets you restore data from storage in a Data Sink. You can filter the import on a combination of Tenants, Indices, and a date range.

Restores give you a way to retrieve data in disaster recovery situations. The only differences between a Data Sink Restore and a Data Sink Import are in retention and disk cleanup:

  • Data restored from a Data Sink is immediately subject to removal by either Retention Group settings or regular ElasticSearch disk cleanups.

  • Data imported from a Data Sink remains available for analysis in the DP until you delete the import task from the Data Sink Import tab.

Snapshots or Data Sinks?

Stellar Cyber can restore data from data sinks or snapshots, depending on how you have configured the system. This topic describes how to restore data from Data Sinks configured in the System | Data Processor | Data Sinks page.

Keep in mind that if you have both snapshot backups and a data sink configured, restoring from a snapshot takes less time.

Restoring Data from a Data Sink

To restore data from a Data Sink:

  1. Click System | Data Processor | Data Management and navigate to the Data Sink Restore tab.

  2. Click Create. The Restore From Data Sink screen appears.

  3. Enter a Name.

  4. Select the Data Sink from which you want to import. The dropdown includes all data sinks configured in the System | Data Processor | Data Sinks page.

  5. Select Tenants.

  6. Select the Indices to import.

  7. Choose a date and time range for the import.

  8. Choose the indices to import.

  9. Click Submit. Stellar Cyber adds the restore task to the list, as illustrated below:

    The task enters the list with a State of CREATED. After a few minutes, the Status indicator illuminates green, indicating the restore is in progress. You can click the Refresh button, as well as the Show Progress link in the Progress column (once it appears) to keep tabs on the status of the restore.

    Note that the possible values in the State column are CREATED , DELETING, and DELETED. The column only indicates whether a task is actively restoring or deleting data. To keep tabs on progress, use the Show Progress link in the Progress column.

One Restore Per Data Sink

Only one restore per Data Sink is allowed at any one time. The Data Sink Restore page will not let you create a second restore from the same data sink.

Resuming a Failed Restore

If for some reason a restore fails and is listed with a red LED in the Status column, you can click the Error Message button in the Message column to see details on the failure. Once the failure is resolved, you can resume the failed task using the following procedure:

  1. Click the Edit button for the task.

  2. You can make changes or not. The key is to open the Edit workflow so you can click the Submit button again.

  3. Click the Submit button.

    Stellar Cyber attempts to resume the failed task.

About "Checkpoint Deleted" Error Messages

Keep in mind that data restored from a Data Sink is immediately subject to removal by either Retention Group settings or regular ElasticSearch disk cleanups. This is true even while the restore task is still in progress – the data restored by an ongoing restore task can be removed by Retention Group settings before the restore has completed. You can tell this is happening when you see either of the following:

  • The restore task is listed with an Error Message reading checkpoint <checkpoint> is being deleted from task <task>. This error message indicates which indices referenced by the restore are being deleted by Retention Group settings or regular disk cleanups.

  • The dates of the data available in the DP do not correspond to those specified in the restore task. You can see the date range of existing data by clicking the Progress link and checking the start_time and end_time reported in the JSON.

Data Sink Import/Restore Priority

  • You can run imports and restores from Data Sinks simultaneously. However, the import task is given higher priority.

  • You cannot run snapshot imports/restores simultaneously with data sink imports/restores.