How to Scour Office 365 Audit Logs for Suspicious Activity

Jeff Christman

Jeff Christman

Read more posts by this author.

Microsoft Azure provides several tools to monitor and investigate security incidents within Office 365 including Cloud App Security, Microsoft Defender Advanced Threat Protection and Security Center. However, these tools often require additional costs and licenses. Also, the Office 365 audit log has a ton of info but it can be hard to find what you’re looking for.

Introducing the free Hawk PowerShell module.

The Hawk PowerShell module scans the Office 365 audit log, gathers all the information and puts it in a single location on the local drive. The main goal of Hawk is to quickly retrieve data that is needed to review and analyze various logs.

In this article, you will learn how to install Hawk, connect to the Office 365 tenant, run scans against the tenant and users and start the investigation of a security breach.

Prerequisites/Requirements

This article will be a tutorial on how to use the Hawk PowerShell module. If you’d like to follow along, please be sure you have the following prerequisites in place before starting.

  • Windows PowerShell v5 or higher – The Hawk module is not compatible with PowerShell 6+.
  • Internet connection.
  • Windows 7 with Service Pack 1 or higher
  • Office 365 Tenant Subscription

Microsoft offers a free 25 user Office 365 Developer lab which includes 25 user licenses to use. Be sure to install the User and Mail Sample packs. The sample packs will add 16 fictitious users with licenses and mailboxes, including names, metadata, and photos for each user and add Outlook email conversations and calendar events for each of the 16 sample users.

Installing The Hawk Module

Before you can begin sleuthing in the Office 365 Audit logs with the Hawk module, you’ll need to get it installed first. To do so, launch Windows PowerShell as administrator and enter the following:

Install-Module -Name Hawk 

This will take a few minutes as Hawk has several dependencies that are installed automatically when you install it. The additional modules are listed below:

Tip: If you want to see which modules are installed, add the verbose flag to the command. Example:  Install-Module -Name Hawk-Verbose

Understanding The Hawk PowerShell Module

The Hawk module cmdlets are split into two main categories, tenant-based cmdlets, and user-based cmdlets. Tenant-based cmdlets gather auditing and reporting data such as user forwarding rules, global mailbox permission, and simple mailbox permissions for all your users. User-based cmdlets focus on individual user account data such as individual forwarding rules, mailbox changes, and message tracking.

Both the tenant-based and the user-based cmdlets are executed by running the main Start-HawkTenantInvestigation or Start-HawkUserInvestigation command, depending on the area you wish to investigate. Executing the commands will call the appropriate cmdlets and begin to gather the data. Linked below are the cmdlets used in the Hawk Module.

Let’s walk through an Office 365 audit and what it will look like.

The Office 365 Audit Log and Hawk

Auditing your Office 365 tenant begins with running the Hawk Tenant Investigation command as shown below:

PS51> Start-HawkTenantInvestigation

After checking for updates and initializing the modules, Hawk will log into MSOnline as seen below. Log in with your Tenant Global Administrator account to continue.

Sign in MSOnline
Sign in MSOnline

Enter Y to agree to the disclaimer as shown below.

MSOnline Disclaimer
MSOnline Disclaimer

Next, provide a directory where all of the logs and data will be stored. Hawk creates several audit logs and it is a good idea to create a separate directory to store them. In this example, all data will go to the C:\Hawk directory.

Logs and Data Directory
Logs and Data Directory

Hawk will then ask for a search window of which days you want to audit. Hitting Enter here will set the default start date of the search window to the current day minus 90 days. For the last day of the search window, hitting Enter will set the default of the current date.

Search Window of Audit Days
Search Window of Audit Days

To reduce the amount of data you have to review, try to limit the search window to as few days as possible.

Once you provide a search window to audit, Hawk begins the scan the exchange online tenant to gather the tenant-based logs. Log in to Exchange online with your tenant global administrator account as you did previously with MSOnline.

Log into Exchange Online
Log into Exchange Online

Once you’re authenticated to Microsoft Online (MSOnline) and Exchange Online (EXO), Hawk begins to gather auditing data and write several logs and data files to your local drive. Hawk creates folders labeled with the date and time of the scan under the C:\Hawk folder in this example.

Each time you run Hawk, it will create a folder with the current time and date and store all the logs and data in the folder as shown below.

Hawk Tenant Log Files
Hawk Tenant Log Files

If running multiple scans or running scans over a series of days, Hawk will use a time/date stamp to as folder names to make it easier to locate the data quickly. As you can see below, the first four numbers are the year, followed by a two-digit month, two-digit day and them the time expressed in military format.

Date/Month/Day/Time Format
Date/Month/Day/Time Format

Narrowing the Scope

With the tenant scan completed, now start reviewing the auditing data that the Hawk module produced. A good first file is to review _investigate.txt log file. The _invesitgate.txt file is a log report all the email forwarding rules for the tenant. Shown below is a sample _investigate.txt file.

Email Forwarding Rules Logs
Email Forwarding Rules Logs

Forwarding email is a security risk because users may be forwarding sensitive information to vulnerable accounts. It could also potentially be used by spammers as a mail relay and cause damage to your companies reputation.

The next log to review is the Hawk.log file. This log details the changes in the different areas of the tenant the last time it was run. Look for any changes that seem suspicious. Some common items are:

  • assigning permission changes to user accounts
  • inbox rule changes
  • user role changes (i.e. a standard user role changed to administration role)

In the sample audit log below, the highlighted section shows that Hawk found eighteen changes in mailbox permissions. This deserves a closer look.

Sample Audit Log
Sample Audit Log

The next line in the log shows the location of the CSV file to review to get more details. In this case, the log file is located at c:\hawk\20191212_1443\Tenant\Simple_Mailbox_Permissions.csv.

Auditing Office 365 Users

Like tenants, Hawk can also audit user and Office 365 admin activity for potential security breaches. With this information, you can narrow your focus to a few suspicious accounts.

Individual accounts are audited using the following command.

Start-HawkUserInvestigation -UserPrincipalName [email protected]_name.com

The Start-HawkUserInvestigation command will call all the Get-HawkUser cmdlets and perform several actions against the user including auditing of:

  • User configuration
  • Mailbox rules
  • Forwarding rules
  • Folder Statistics
  • Mailbox statistics

As you can see below, Start-HawkUserInvestigation creates a folder labeled with the username and stores all the user-focused log files in that folder.

User log files
User log files

Many of the files created by the Hawk PowerShell Module are in CSV format. For information on how to use a PowerShell script to parse CSV files, be sure to read Managing CSV Files in PowerShell.

Summary

Investigating a security breach requires gathering as much data as you can and it can be a difficult task with lots of logs to review and data to collect. Hawk makes it easier by gathering the necessary logs into a single location. This article showed how you can use Hawk to automate site log collection and aid the search investigation.

Further Reading

Subscribe to Adam the Automator

Get the latest posts delivered right to your inbox

Looks like you're offline!