Build a security monitoring solution with AWS AppFabric and Amazon Security Lake

Monitoring audit log data from software-as-a-service (SaaS) applications helps security analysts and IT administrators quickly identify and respond to possible corporate security threats. While studies indicate that organizations license over 100 SaaS applications for their organization, each SaaS application has different data formats making it challenging to determine how to get the security insights you require from your data. To address this challenge, some teams build a security monitoring solution on AWS to improve their organization’s security posture with low operational overhead.

In this blog, we show you how to use AWS AppFabric to connect your SaaS applications, normalize and transport your audit logs to Amazon Security Lake, and analyze your SaaS logs using Amazon QuickSight. AppFabric is a fully managed service that aggregates and normalizes security data across SaaS applications into the Open Cybersecurity Schema Format (OCSF). OCSF is an open-source, vendor agnostic security schema that improves observability and helps reduce operational effort and cost for cybersecurity teams. Amazon Security Lake automatically centralizes security data from AWS environments, SaaS providers, on premises, and cloud sources into a purpose-built data lake stored in your account. With QuickSight, all users can meet varying analytic needs from the same source of truth through modern interactive dashboards, paginated reports, embedded analytics, and natural language queries.

Prerequisites

Before you get started, you will need to ensure you have the right access in your AWS account and across your SaaS applications. Create a user with administrative permissions for AppFabric and enable Security Lake in a region AppFabric supports. You can find the supported regions for AppFabric in the Accessing AppFabric section of the AWS AppFabric Administration guide. You can find the supported regions for Security Lake in the Amazon Security Lake endpoints section of the AWS General Reference guide.

Solution overview

With this solution, you quickly connect your SaaS applications to AppFabric and send your data to Security Lake. AppFabric fully-manages each integration with the SaaS applications you select so you don’t have to configure any integrations or spend resources managing them over time. Once your data is in Security Lake, you can use Amazon QuickSight to query, analyze, and monitor your SaaS data for any threats or anomalous behavior. In this solution, your data flows from AppFabric to QuickSight through Amazon Kinesis Data Firehose, Amazon Simple Storage Service (Amazon S3), AWS Glue, and Amazon Athena. The full architecture diagram is available in Figure 1 showing how each AWS service is used to enhance your SaaS application observability.

Figure 1: A security data lake solution with SaaS app log sources and AWS native log sources

Figure 1 is described as follows:

1. AppFabric connects your SaaS applications, transforms SaaS audit logs into the OCSF, and sends those audit logs through Kinesis Data Firehose.
2. The Kinesis Data Firehose partitions and sends the audit logs to the Security Lake managed Amazon S3 bucket.
3. Security Lake collects logs from other AWS services, converts them into OCSF, and sends them to the Amazon S3 bucket Security Lake manages.
4. Security Lake creates a crawler and data catalog in AWS Glue for your SaaS application log sources and your AWS log sources.
5. Athena queries the data in the Amazon S3 bucket using the data catalog Security Lake manages.
6. QuickSight analyzes the data using Athena as its data source.

Deploy the solution

There are multiple steps to follow to deploy this solution including connecting at least one SaaS application to AppFabric, creating a custom source in Security Lake, deploying the infrastructure required to connect AppFabric and Security Lake, connecting AppFabric to Security Lake with Kinesis, signing up for QuickSight, and importing your data into QuickSight. In the next section, we go into the details of each step.

Connect a SaaS application to AppFabric

Choose at least one SaaS application your organization uses from the list of AppFabric supported applications. From the AWS Console, connect AppFabric to your SaaS application using the AppFabric getting started guide. Create your AppFabric app bundle in a region both AppFabric and Security Lake support. Choosing the same region for AppFabric and Security Lake minimizes the complexity of the solution and allows you to get started quickly.

Create a custom source in Security Lake for AppFabric

Configure AppFabric as a custom source in Security Lake using the Create a custom source in Security Lake section of the AppFabric administration guide. Select only one of the AppFabric supported OCSF event classes to receive AppFabric data for all supported OCSF event classes.

Deploy the infrastructure required to connect AppFabric and Security Lake

Connecting AppFabric to Security Lake requires an AWS Glue database to define the AppFabric schema, Amazon S3 bucket to support AWS Glue, and Kinesis Data Firehose to partition, compress, and send AppFabric data to Security Lake. You can deploy the resources using the AWS Cloud Development Kit (AWS CDK), the Amazon CloudFormation template, or setup manually. For best performance, it’s recommended to deploy these resources into the same region in which you deployed AppFabric and Security Lake.

Option 1 – Deploy the infrastructure using an AWS CDK app

The AWS CDK is a framework to define your cloud application resources using familiar programming languages. You simplify your AWS onboarding by using constructs that preconfigure cloud resources with proven defaults settings. With AWS CDK, you can customize, share, and reuse constructs within your organization or community, just like any other software library.

Choose the AWS CDK if you want to manage the resources required to connect AppFabric with Security Lake as a part of your software development lifecycle. You can find the latest AppFabric CDK sample app in Python for this solution in the Build a security monitoring solution with AWS AppFabric and Amazon Security Lake GitHub repository. If you haven’t used the AWS CDK before, use the instructions on the Get Started with AWS CDK page to install the CDK to your workstation and bootstrap your AWS account.

Download the GitHub repository to your local workstation and navigate to the root directory of the project. Open the appfabric_security_lake/appfabric_security_lake_stack.py file in a text editor and configure the information at the top of the file with the appropriate values for your environment. Next, run the cdk synth command from the in the root directory of the CDK app on your command line to synthesize an AWS CloudFormation template. Finally, run the cdk deploy command to deploy the infrastructure into your AWS account.

Option 2 – Deploy the infrastructure using CloudFormation.

A CloudFormation template describes your desired resources and their dependencies so you can launch and configure them together as a stack. You can use a template to create, update, and delete an entire stack as a single unit, as often as you need to. This prevents you from managing resources individually. If you prefer to work directly with CloudFormation, download our sample CloudFormation template to your workstation and deploy the stack with the instructions on the Using the AWS CloudFormation console section or the Using the AWS Command Line Interface (AWS CLI) section of the AWS CloudFormation User Guide.

Option 3 – Manual setup

Finally, if you want to manage all of the resources on your own, deploy the resources manually into your AWS account using the Amazon Security Lake guide in the AppFabric documentation.

Connect AppFabric to Security Lake with Kinesis Data Firehose

Once you’ve chosen an option to connect AppFabric to Security Lake, add the new Kinesis Data Firehose as an output destination in AppFabric using the instructions in the Create an output location section of the AWS AppFabric Administration Guide. For this example, you must select OCSF-JSON as the output format in AppFabric.

Sign up for QuickSight

Sign up for the QuickSight Enterprise edition using the Signing up for an Amazon QuickSight subscription documentation in the Amazon QuickSight User Guide.

To create analysis using your Security Lake data you must grant your QuickSight users, or groups, access to the AWS Glue database for Security Lake with Lake Formation. Use the AWS CLI to obtain the Amazon Resource Name (ARN) of your QuickSight users or groups who require access to your AWS Glue database using the ListUsers command or the ListGroups command. Use the instructions in the Granting and revoking permissions on Data Catalog resources section of the AWS Lake Formation developer guide to grant your QuickSight users and groups permissions to your AWS Glue database. You may choose either the named resource method or the Lake Formation tag-based access control method (LF-TBAC). Choose SAML users and groups as the principals in Lake Formation, enter the ARN for your QuickSight user or group, grant Describe database permissions on your AWS Glue database for Security Lake, and grant Select and Describe table permissions for your AppFabric custom source table.

Import your Security Lake data

Before you create a security analysis in QuickSight, you must import the audit logs data from Security Lake. Sign in to QuickSight using the Getting started with Amazon QuickSight data analysis documentation in the Amazon QuickSight User Guide. Create an Athena dataset in QuickSight. Choose the AWS Glue data catalog containing your Security Lake Glue database, your Security Lake Glue database, the table containing the data from your AppFabric data source, and then click Select.

Figure 2: Select your Athena database in QuickSight

Start an analysis in QuickSight

QuickSight offers many different methods for you to analyze and create visualizations with your data. For example, a security analyst, Sara, needs to monitor user behavior across her SaaS applications. She uses AppFabric to send data to QuickSight. With QuickSight, Sara views a map of activity by geographic area to quickly identify any activity originating in unexpected locations.

Figure 3: SaaS application usage by city

She then views the audit logs from her SaaS applications connected to AppFabric to investigate the activity from the unexpected location. Here, she finds the email address of the user, which SaaS applications the user accessed, and the types of activities the user performed to determine if the actions show signs of malicious activity. She has the option to filter the view to focus on the information that matters most to her investigation.

Figure 4: User audit logs table

By using QuickSight with AppFabric Sara saves times and can focus on monitoring her data and configuring security alerts to prevent a crisis. See the Amazon QuickSight User Guide, Starting an analysis in Amazon QuickSight section, to begin analyzing your data.

Cleanup

To avoid incurring charges, delete your AppFabric resources, delete the AppFabric custom source in Security Lake, and then delete the AppFabric data in the Security Lake managed Amazon S3 bucket. Navigate to the Amazon S3 console, click on the Security Lake bucket associated with the region you chose while working through this blog, and delete all of your data in the ext/<custom source name> prefix, where <custom source name> is the name of your AppFabric custom source. Revoke the Lake Formation permissions you granted for your QuickSight users or groups, delete your QuickSight subscription and close your account.

You must also delete the Amazon S3 bucket, AWS Glue database, and Kinesis Data Firehose connecting AppFabric to Security Lake. If you deployed the resources with AWS CDK, navigate to your AWS CDK project on the command line and execute the cdk destroy command. If you deployed the resources using the CloudFormation template, delete the stack using the instructions in the AWS CloudFormation User Guide on the Deleting a stack on the AWS CloudFormation console section, if you use the AWS console, or the Deleting a stack section if you use the AWS CLI.

If you created the resources manually delete the Amazon S3 bucket associated with the Kinesis Data Firehose you use with AppFabric. Delete the AWS Glue database associated with the Kinesis Data Firehose you use with AppFabric. Navigate to the AWS Glue page in the AWS console. Click on Data Catalog > Databases in the navigation panel on the left, click the check box next to your Glue database, click the Delete button on the console page, and click the Delete button on the dialog that appears.

Figure 5: Deleting an AWS Glue database

You must also delete the AppFabric Kinesis Data Firehose you use with AppFabric. Navigate to the Amazon Kinesis page in the AWS console. Click on Data Firehose in the navigation panel. Click on the radio button next to the Kinesis Data Firehose you use with AppFabric, click on the Delete button, and type the stream name and click Delete on the dialog that appears.

Figure 6: Deleting an Amazon Kinesis Data Firehose

Conclusion

Quickly implement a SaaS application security monitoring solution composed of AWS services to enhance your security observability. Using AWS makes it easy to handle the varying data formats produced by SaaS applications so you can focus on monitoring and analyzing your SaaS data. Try this solution today by visiting AWS AppFabric, Amazon Security Lake, and Amazon QuickSight from the AWS console.