Configuring AWS CloudTrail Connectors

This connector allows Stellar Cyber to collect data from external sources and add the records to the data lake. The incoming information is included in Interflow records in the AWS Events index.

Integration with AWS and S3 enables organizations to securely store and manage their data while providing real-time threat detection and response capabilities through Stellar Cyber's advanced security analytics and machine learning algorithms.

You can have any number of AWS connectors active in the system. You will typically have one or two connectors for each account being used by anyone in the network.

However, if you have AWS Control Tower set up, you can use an asterisk as a wildcard character instead of a specific Account ID in the Prefix field.

Stellar Cyber connectors with the Collect function (collectors) may skip collecting some data when the ingestion volume is large, which potentially can lead to data loss. This can happen when the processing capacity of the collector is exceeded.

Connector Overview: AWS CloudTrail

Adding an AWS CloudTrail Connector

To add an AWS CloudTrail connector:

1. Create an AWS CloudTrail trail
2. Add a user with the appropriate permissions
3. Create an access key
4. Get the required information
5. (Optional) Set up AWS Control Tower
6. Add the connector in Stellar Cyber
7. Test the connector
8. Verify ingestion

Creating the AWS CloudTrail Trail

To create an AWS CloudTrail trail:

Use our example as a guideline, as you might be using a different software version.

1. Log in to your AWS Management Console at https://aws.amazon.com .

   

2. In Find Services, start entering cloudtrail. As you type, suggestions appear. Click on CloudTrail.

   

   The CloudTrail Dashboard appears.

   

3. Click Create trail. The Choose trail attributes page appears.

   

4. Enter a Trail name.
5. Leave the default configurations for the various event information settings.
6. Under Storage location, choose Yes for Create a new S3 bucket.

7. Enter the name.

   Make note of this, as you need the bucket name when configuring the connector in Stellar Cyber.

8. Enter the AWS KMS alias.
9. Leave the remaining options at the default settings.

10. Click Next. The Choose log events page appears.

    

11. Choose the events you want to send to Stellar Cyber. We recommend choosing all of them.
12. Click Next. The Review and create page appears.
13. Click Create trail. The trail is added, and displayed on the Trails page.

Adding a User with the Appropriate Permissions

To add a user with the appropriate permissions:

1. Log in to your AWS Management Console at https://aws.amazon.com/console. View the services in the Console Home or choose View all services.

   

2. Choose IAM. The IAM Dashboard appears.

3. Choose Policies and then choose Create Policy.

   

4. In the Create policy pane, choose the JSON tab.

   

5. Using the example below as a guide, edit the JSON policy document.

   The following is just an EXAMPLE. You must modify this JSON to match the resources in your own environment.

   Copy

   CloudTrail policy

   1
   2
   3
   4
   5
   6
   7
   8
   9
   10
   11
   12
   13
   14
   15
   16
   17
   18
   19
   20
   21
   22
   23
   24
   25
   26
   27
   28
   29
   30
   31
   32
   33
   34
   35
   36
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject",
                   "s3:ListBucket"
               ],
               "Resource": [
                    "arn:aws:s3:::<bucket_name>",
                    "arn:aws:s3:::<bucket_name>/*",
           }
       ]
   } 
   
   For example:
   
   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject",
                   "s3:ListBucket"
               ],
               "Resource": [
                   "arn:aws:s3:::stellarcybercloudtrail",
                   "arn:aws:s3:::stellarcybercloudtrail/*"
               ]
           }
       ]
   }

6. Choose Next.

7. Give your policy a name to associate it with Stellar Cyber, then choose Create policy.

   

   The policy can now be attached to a user.

8. From the IAM navigation pane, choose Users and then choose Create user.

   

9. On the Specify user details page for User details, enter a User name for the new user.

10. Choose Provide user access to the – AWS Management Console optional to produce login credentials for the new user, such as a password.

11. Choose how you want to create the Console password and then choose Next.

12. On the Set permissions page, choose Attach policies directly. Then search for the policy you created above.

    

13. Select the checkbox to the left of the policy name, then choose Next.

14. On the Review and create page, verify the information and then choose Create user.

Creating an Access Key

1. Log in to the IAM Console with your AWS account ID, your IAM user name, and password. You must have the permissions required to create access keys for a user.

2. Choose Users, then click the user name, and click Security credentials.

   

3. In the Access keys section, choose Create access key. Access keys have two parts: an access key ID and a secret access key. Click Create access key.

   

4. On the Access key best practices & alternatives page, choose Other and then choose Next.

   

5. (Optional) On the Set description tag page, enter a description.

6. Click Create access key.

7. On the Retrieve access keys page, choose Show to reveal the value of the user's secret access key. Save the access key ID and secret access key in a secure location. You will need them when configuring the connector in Stellar Cyber.

   

8. Click Done.

Getting the Required Information

To add the connector in Stellar Cyber you need the following:

• Bucket (you copied this earlier)
• Prefix
• Access key ID (you copied this earlier)
• Secret access key (you copied this earlier)

To get the prefix:

1. In the CloudTrail Dashboard, click Trails. The existing trails appear.

2. For the trail you are using, the Bucket is the name in the S3 bucket column.

   

3. Click the bucket name. The bucket info appears.

4. Click the CloudTrail object.

   

   The regional objects appear.

5. Click your region. The Prefix is the path at the top of the screen, starting with AWSLogs (the default) and ending with the region.

   

   In our example the path is AWSLogs/578591608273/CloudTrail/us-west-1. Copy your path. This is what you paste into the Prefix field in Stellar Cyber.

   Do not copy the trailing "/" symbol.

   If you're configuring an AWS WAF connector and you have a custom prefix , use your custom prefix. If you are using a date in your custom prefix, you must replace static numbers with variables, converting the prefix to a pattern that Stellar Cyber can recognize. Following are the variables.

   Variable	Definition
   //YYYY//	4 digit year
   //m//	1 or 2 digit month
   //mm//	2 digit month
   //d//	1 or 2 digit day
   //dd//	2 digit day
   //H//	1 or 2 digit hour
   //HH//	2 digit hour
   //M//	1 or 2 digit minute
   //MM//	2 digit minute

   For example, CustomPrefix/year=/2020/month=/10/day=/27/ becomes CustomPrefix/year=///YYYY///month=///mm///day=///dd//.

For the CloudTrail connector, the supported prefix syntax may also include Organization ID, as shown here:

• Without Organization ID: optional_prefix_name/AWSLogs/Account ID/CloudTrail/region

• With Organization ID: optional_prefix_name/AWSLogs/OU-ID/Account ID/CloudTrail/region

(Optional) Setting up AWS Control Tower

There are two types of AWS Control Tower: SSE-S3 encryption (default) and SSE-KMS encryption.

To set up the default AWS Control Tower (SSE-S3 encryption), refer to the AWS Control Tower documentation.

To set up the AWS Control Tower with SSE-KMS encryption, follow these procedures:

• Create symmetric encryption KMS keys

• Set up the AWS Control Tower on your management account

• Create an IAM User

Creating Symmetric Encryption KMS Keys

To create symmetric encryption KMS keys:

1. Log in to your management account (YOUR-MANAGEMENT-ACCOUNT).

2. Follow the guidance in the AWS documentation to create your KMS key (YOUR-KMS-KEY-ID).

3. Using the example below as a guide, edit the JSON policy document.

   The following is just an EXAMPLE. You must modify this JSON to match the resources in your own environment.

   Copy

   Policy

   {
       "Version": "2012-10-17",
       "Id": "CustomKMSPolicy",
       "Statement": [
           {
           ... YOUR-EXISTING-POLICIES ...
           },
           {
               "Sid": "Allow Config to use KMS for encryption",
               "Effect": "Allow",
               "Principal": {
                   "Service": "config.amazonaws.com"
               },
               "Action": [
                   "kms:Decrypt",
                   "kms:GenerateDataKey"
               ],
               "Resource": "arn:aws:kms:YOUR-HOME-REGION:YOUR-MANAGEMENT-ACCOUNT-ID:key/YOUR-KMS-KEY-ID"
           },
           {
               "Sid": "Allow CloudTrail to use KMS for encryption",
               "Effect": "Allow",
               "Principal": {
                   "Service": "cloudtrail.amazonaws.com"
               },
               "Action": [
                   "kms:GenerateDataKey*",
                   "kms:Decrypt"
                 ],
               "Resource": "arn:aws:kms:YOUR-HOME-REGION:YOUR-MANAGEMENT-ACCOUNT-ID:key/YOUR-KMS-KEY-ID",
               "Condition": {
                   "StringEquals": {
                       "aws:SourceArn": "arn:aws:cloudtrail:YOUR-HOME-REGION:YOUR-MANAGEMENT-ACCOUNT-ID:trail/aws-controltower-BaselineCloudTrail"
                   },
                   "StringLike": {
                       "kms:EncryptionContext:aws:cloudtrail:arn": "arn:aws:cloudtrail:*:YOUR-MANAGEMENT-ACCOUNT-ID:trail/*"
                   }
               }
           },
           {
               "Sid": "Allow user to use KMS for encryption",
               "Effect": "Allow",
               "Principal": {
                   "AWS": "<user ARN>"
               },
               "Action": "kms:Decrypt",
               "Resource": "arn:aws:kms:YOUR-HOME-REGION:YOUR-MANAGEMENT-ACCOUNT-ID:key/YOUR-KMS-KEY-ID"
           }
       ]
   }

4. In various places in the JSON policy document, provide the following: 

   • YOUR-HOME-REGION

   • YOUR-MANAGEMENT-ACCOUNT-ID

   • YOUR-KMS-KEY-ID

   • <user ARN> (the user that will be used for the connector)

Setting up the AWS Control Tower on your Management Account

You can now set up the AWS Control Tower on your management account with the KMS key you created earlier.

1. Log in to AWS CloudTrail with your AWS account (YOUR-MANAGEMENT-ACCOUNT).

2. Follow the guidance in the following documentation to deploy a landing zone with AWS Control Tower.

3. In the following graphic, YOUR-MANAGEMENT-ACCOUNT ends in 4429. The CloudTrail S3 bucket is in the 9809 AWS account (the Log-archive account).

   

Creating an IAM User

Next, create an IAM user. You can refer to Adding a User with the Appropriate Permissions.

1. Log into the Log-archive account (9809).

2. Using the example below as a guide, edit the JSON policy document.

   The following is just an EXAMPLE. You must modify this JSON to match the resources in your own environment.

   Copy

   Policy for the user

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "VisualEditor0",
               "Effect": "Allow",
               "Action": [
                   "s3:GetObject",
                   "s3:ListBucket",
               ],
               "Resource": [
                   "<S3 bucket ARN>",
                   "<S3 bucket ARN>/*"
               ]
           },
           {
               "Sid": "KMSDecryptionPermission",
               "Effect": "Allow",
               "Action": [
                   "kms:Decrypt"
               ],
               "Resource": [
                   "arn:aws:kms:YOUR-HOME-REGION:YOUR-MANAGEMENT-ACCOUNT-ID:key/YOUR-KMS-KEY-ID"
               ]
           }
       ]
   }

3. In the JSON policy document, provide the following: 

   • YOUR-HOME-REGION

   • YOUR-MANAGEMENT-ACCOUNT-ID

   • YOUR-KMS-KEY-ID

   • <S3 bucket ARN>

4. Generate the access key and security key for this user. You can refer to Creating an Access Key.

Adding the Connector in Stellar Cyber

With the trail set up and the bucket name and path handy, you can add the AWS CloudTrail connector in Stellar Cyber:

1. Log in to Stellar Cyber.

2. Click System | Connectors (under Integrations). The Connector Overview appears.

   

3. Click Create. The General tab of the Add Connector screen appears. The information on this tab cannot be changed after you add the connector.

   

   The asterisk (*) indicates a required field.

4. Choose PaaS from the Category drop-down.

5. Choose AWS CloudTrail from the Type drop-down.

6. For this connector, the supported Function is Collect, which is enabled already.

7. Enter a Name.

   Notes:

   • This field does not accept multibyte characters.
   • It is recommended that you follow a naming convention such as tenantname-connectortype.

8. Choose a Tenant Name. The Interflow records created by this connector include this tenant name.

9. Choose the device on which to run the connector.

   • Certain connectors can be run on either a Sensor or a Data Processor. The available devices are displayed in the Run On menu. If you want to associate your collector with a sensor, you must have configured that sensor prior to configuring the connector or you will not be able to select it during initial configuration. If you select Data Processor, you will need to associate the connector with a Data Analyzer profile as a separate step. That step is not required for a sensor, which is configured with only one possible profile.

   • If the device you're connecting to is on premises, we recommend you run on the local sensor. If you're connecting to a cloud service, we recommend you run on the DP.

10. (Optional) When the Function is Collect, you can apply Log Filters. For information, see Managing Log Filters.

11. Click Next. The Configuration tab appears.

    

    The asterisk (*) indicates a required field.

12. Choose a Sub Type.

    • cloudtrail—feeds account activity information, including actions taken through the AWS Management Console and various AWS SDKs and command line tools. See the AWS CloudTrail documentation for details.

    • vpcflow—provides network-level information. See the AWS VPC documentation for details.

    • waf—provides Web Application Firewall information. See the AWS WAF documentation for details.

13. Enter the Proxy URI, if you have one.

14. Enter the Bucket you noted earlier.

15. Enter the Prefix you noted earlier. Ensure the Prefix does not include a trailing "/" symbol.

    The supported prefix syntax is as follows:

    • Without Organization ID: optional_prefix_name/AWSLogs/Account ID/CloudTrail/region

    • With Organization ID: optional_prefix_name/AWSLogs/OU-ID/Account ID/CloudTrail/region

• With AWS Control Tower: optional_prefix_name/OU-ID/AWSLogs/OU-ID/*/CloudTrail/region

  The Sub Type must be cloudtrail or vpcflow.

16. Enter the Access Key ID from IAM.

17. Enter the Secret Access Key from IAM.

18. Enter the File Type. The cloudtrail and vpcflow sub types should always be gzip. The waf sub type can be gzip or text.

19. (Optional) Choose a Region from the available regions in the drop-down.

20. Click Next. The final confirmation tab appears.

    

21. Click Submit.

    To pull data, a connector must be added to a Data Analyzer profile if it is running on the Data Processor.

22. If you are adding rather than editing a connector with the Collect function enabled and you specified for it to run on a Data Processor, a dialog box now prompts you to add the connector to the default Data Analyzer profile. Click Cancel to leave it out of the default profile or click OK to add it to the default profile.

    • This prompt only occurs during the initial create connector process when Collect is enabled.

    • Certain connectors can be run on either a Sensor or a Data Processor, and some are best run on one versus the other. In any case, when the connector is run on a Data Processor, that connector must be included in a Data Analyzer profile. If you leave it out of the default profile, you must add it to another profile. You need the Administrator Root scope to add the connector to the Data Analyzer profile. If you do not have privileges to configure Data Analyzer profiles, a dialog displays recommending you ask your administrator to add it for you.

    • The first time you add a Collect connector to a profile, it pulls data immediately and then not again until the scheduled interval has elapsed. If the connector configuration dialog did not offer an option to set a specific interval, it is run every five minutes. Exceptions to this default interval are the Proofpoint on Demand (pulls data every 1 hour) and Azure Event Hub (continuously pulls data) connectors. The intervals for each connector are listed in the Connector Types & Functions topic.

    The Connector Overview appears.

The new connector is immediately active and starts collecting logs.

Testing the Connector

If you have AWS Control Tower, the Test button tests all account IDs.

When you add (or edit) a connector, we recommend that you run a test to validate the connectivity parameters you entered. (The test validates authentication and connectivity).

1. Click System | Connectors (under Integrations). The Connector Overview appears.

   

2. Locate the connector by name that you added, or modified, or that you want to test.

3. Click Test at the right side of that row. The test runs immediately.

   Note that you may run only one test at a time.

Verifying Ingestion

To verify ingestion:

1. Click Investigate | Threat Hunting. The Interflow Search tab appears.
2. Change the Indices to include either or both of AWS Events or Traffic. The table immediately updates to show ingested Interflow records.