Monitoring Windows Endpoints with Winlogbeat

2020-01-20 about 1733 words 9 min

Contents

Introduction

This post will introduce the steps needed in order to setup a Windows endpoint to forward logs to Logstash. The following diagram shows a layout of how all the components will interact with each other.

On the Endpoint we have logs that we want to ship to Elasticsearch, these logs include various system logs generated by Windows and Sysmon logs. The Sysmon logs are generated by another program installed on the system which will monitor events related to processes being created and registry values being changed.

The logs stored on the system are read by another program we install called Winlogbeat, after it has read the log it will send the logs to Logstash. This is the main mechanism we use to get logs out of the endpoint onto the ELK Server.

After the logs are on the ELK Server Logstash is capable of performing actions on the logs. It can be as simple as take them and put them into Elasticsearch, or it can get more complicated and alter the format and fields of the logs. For example, if a log file includes IP addresses Logstash can attach a field related to which country and city the IP address is located in.

Configuring Logging Locally

Now that Elasticsearch, Logstash, and Kibana are configured and running the endpoint must be setup to forward logs. The endpoint system is a Windows Server 2019 computer that will be using Winlogbeat to forward logs to Logstash. Furthermore, Sysmon will be installed which will give a deeper insight into what is happening on the system.

The following links to software used:

Sysmon – Download
Winlogbeat – Download
SwiftOnSecurity Sysmon Configuration – Download

Installing Sysmon

Sysmon is a a service that can be installed on Windows which will provide detailed information regarding processes, registry, and network connections. These logs are stored locally on the system and can be forwarded to a central logging server or SIEM.

In order to install and configure Sysmon make sure you have downloaded the official ZIP file and the SwiftOnSecurity Sysmon configuration. It is not mandatory to use this configuration file, or any configuration file for that matter as there are default settings. However, Sysmon can be configured in many different ways and the SwitftOnSecurity configuration is a decent starting point.

Make sure to extract the Sysmon.exe binary that will be used to install the service, also don’t forget to extract the SwiftOnSecurity Sysmon configuration file into the same location. Run the following command in order to install the service.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


PS C:\Users\Administrator\Downloads> .\Sysmon64.exe -accepteula -i .\sysmonconfig-export.xml

System Monitor v10.41 - System activity monitor
Copyright (C) 2014-2019 Mark Russinovich and Thomas Garnier
Sysinternals - www.sysinternals.com

Loading configuration file with schema version 4.00
Sysmon schema version: 4.22
Configuration file validated.
Sysmon64 installed.
SysmonDrv installed.
Starting SysmonDrv.
SysmonDrv started.
Starting Sysmon64..
Sysmon64 started.

Now that Sysmon is installed you can check that the service is active and running.

1
2
3
4
5


PS C:\> Get-Service -Name Sysmon64

Status   Name               DisplayName
------   ----               -----------
Running  Sysmon64           Sysmon64

As said before the Sysmon logs are stored locally, this means that you can view the logs generated right away in Event Viewer. Below we can see a PowerShell command being used to view the logs.

The logs are generated in Microsoft/Windows/Sysmon. As can be seen the following shows a process create event for the powershell.exe binary. This provides us with a lot of information, we can see that the process that launched PowerShell was cmd.exe and that it was done by the Administrator. You can also see that it gives a hash of the binary that has been run, this can be taken to VirusTotal and scanned.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29


PS C:\> Get-WinEvent -FilterHashtable @{ LogName = "Microsoft-Windows-Sysmon/Operational" } | Where-Object  -FilterScript {$_.Id -eq 1} | Select-Object -First 1 | Format-List


TimeCreated  : 11/22/2019 3:51:12 PM
ProviderName : Microsoft-Windows-Sysmon
Id           : 1
Message      : Process Create:
               RuleName:
               UtcTime: 2019-11-22 23:51:12.995
               ProcessGuid: {3ca46963-7470-5dd8-0000-0010f9d43700}
               ProcessId: 2244
               Image: C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
               FileVersion: 10.0.17763.1 (WinBuild.160101.0800)
               Description: Windows PowerShell
               Product: Microsoft® Windows® Operating System
               Company: Microsoft Corporation
               OriginalFileName: PowerShell.EXE
               CommandLine: powershell.exe
               CurrentDirectory: C:\Users\Administrator\
               User: DC\Administrator
               LogonGuid: {3ca46963-e019-5dd8-0000-002066df0200}
               LogonId: 0x2DF66
               TerminalSessionId: 1
               IntegrityLevel: High
               Hashes: MD5=7353F60B1739074EB17C5F4DDDEFE239,SHA256=DE96A6E69944335375DC1AC238336066889D9FFC7D73628EF4FE1B1B160AB32C
               ParentProcessGuid: {3ca46963-746c-5dd8-0000-0010f3cc3700}
               ParentProcessId: 1916
               ParentImage: C:\Windows\System32\cmd.exe
               ParentCommandLine: "C:\Windows\system32\cmd.exe"

Enhanced PowerShell Logging

PowerShell logging has the ability to log what actions have been taken using PowerShell, these including logging commands run, saving the output of these commands, and also show the de-obfuscated versions of commands run.

One of the most important logs related to this is Even ID 4103, we can see an example below in the code snippet. From this we can figure out that the Administrator user has run Get-Process in PowerShell.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


PS C:\> Get-WinEvent -FilterHashtable @{ LogName = "Microsoft-Windows-PowerShell/Operational"} | Where-Object  -FilterScript {$_.Id -eq 4103} |Select-Object -First 1 | Format-List

TimeCreated  : 11/22/2019 5:42:13 PM
ProviderName : Microsoft-Windows-PowerShell
Id           : 4103
Message      : CommandInvocation(Get-Process): "Get-Process"


               Context:
                       Severity = Informational
                       Host Name = ConsoleHost
                       Host Version = 5.1.17763.316
                       Host ID = 1df2f188-5a35-4c23-a094-70b67cdeb9f0
                       Host Application = C:\Windows\system32\WindowsPowerShell\v1.0\PowerShell.exe
                       Engine Version = 5.1.17763.316
                       Runspace ID = da3546f0-d7d1-472f-9721-0daea98de7bd
                       Pipeline ID = 11
                       Command Name = Get-Process
                       Command Type = Cmdlet
                       Script Name =
                       Command Path =
                       Sequence Number = 26
                       User = DC\Administrator
                       Connected User =
                       Shell ID = Microsoft.PowerShell

In order to enable PowerShell logging we must turn on a few options in group polices. The options are located in Administrative Templates -> Windows Components -> Windows PowerShell. A screenshot below shows that we should have enabled three options, module logging, Script Block Logging, and Transcription. This image was found on a FireEye blog post related to this topic, I recommend checking it out for more information.

The following shows a brief list of actions that are needed to be taken in order to enable each option:

Turn on Module Logging

Enable this option. Within the options click “Show” and enter a ‘*’ into the new Window. This will allow all PowerShell modules to be logged.

Turn on PowerShell Script Block Logging

All that needs to be done is to click Enable on this option.

Turn on PowerShell Transcription

All that needs to be done is to click Enable on this option, unless a specific central location is desired for PowerShell transcripts.

Setting Up Logstash

Logstash is where all the hidden power lies, from here it is possible to take logs and format them to your liking. This is a time consuming process, there are two interesting projects that have configuration files you can either copy or use for inspiration. These two projects are HELK and SOF ELK .

The following Logstash configuration will take input from the Windows endpoint and put all the logs in a single index without filtering them. This may be enough for simple setup used for testing, but more advanced environments may need to change the format of logs to meet specific requirements.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18


input {
  beats {
    port => 5044
    add_field => {"[@metadata][source]" => "winlogbeat"}
    ssl => false
  }
}
output {
	elasticsearch {
		hosts => ["http://10.0.0.10:9200"]
		index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
		document_type => "%{[@metadata][type]}"
		ssl => false
		ssl_certificate_verification => false
		sniffing => true
		manage_template => false
	}
}

After this configuration file has been added restart Logstash and configure an endpoint to send logs in the next section.

1

[user@ELKNode]$ sudo systemctl restart logstash

Alternative: Copying HELK

As an alternative it is also possible to copy HELK’s configurations and use them in this instance. Running the following commands will achieve this:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16


[user@ELKNode ~]$ systemctl stop logstash
[user@ELKNode ~]$ rm /etc/logstash/conf.d/*

[user@ELKNode ~]$ cd ~
[user@ELKNode ~]$ git clone https://github.com/Cyb3rWard0g/HELK
[user@ELKNode conf.d]$ cd /etc/logstash/conf.d
[user@ELKNode conf.d]$ cp -rv ~/HELK/docker/helk-logstash/pipeline/* .

[user@ELKNode conf.d]$ mkdir -p /usr/share/logstash/pipeline/ruby
[user@ELKNode conf.d]$ cp -v ruby/ip_clean_and_public.rb /usr/share/logstash/pipeline/ruby/
[user@ELKNode conf.d]$ cp -v ruby/community-id.rb /usr/share/logstash/pipeline/ruby/

[user@ELKNode conf.d]$ sed -i 's/helk-elasticsearch/10.0.0.10/g' *.conf
[user@ELKNode conf.d]$ rm 0002-kafka-input.conf

[user@ELKNode conf.d]$ systemctl start logstash

Shipping Logs with WinlogBeat

Winlogbeat is the Beat that will be used to ship logs from a Windows endpoint to Logstash. Make sure to downlaod the Winlogbeat zip file from the offical website, extract it to a location on the disk. I usually put it into C:\Program Files, however, you may choose to use a different directory.

Next the Winlogbeat configuration file must be setup properly. The location of this configuration file is C:\Program Files\winlogbeat-7.3.2-windows-x86_64\winlogbeat.yml. The following shows a base configuration that can be used which will ship logs from Application, System, Security, and Microsoft-Windows-Sysmon/Operational. The destination location is Logstash located at 10.0.0.10:5044.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22


winlogbeat.event_logs:
  - name: Application
    ignore_older: 30m
  - name: Security
    ignore_older: 30m
  - name: System
    ignore_older: 30m
  - name: Microsoft-windows-sysmon/operational
    ignore_older: 30m
  - name: Microsoft-windows-PowerShell/Operational
    ignore_older: 30m
    event_id: 4103, 4104
  - name: Windows PowerShell
    event_id: 400,600
    ignore_older: 30m
setup.template.settings:
  index.number_of_shards: 1
output.logstash:
  hosts: ["10.0.0.10:5044"]
processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~

After the configuration file is altered the service can be installed by running the following Powershell script, it is located in the same directory as the configuration file.

1
2
3
4
5
6
7
8
9


C:\Program Files\winlogbeat-7.4.2-windows-x86_64>powershell -ExecutionPolicy ByPass
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Program Files\winlogbeat-7.4.2-windows-x86_64> .\install-service-winlogbeat.ps1

Status   Name               DisplayName
------   ----               -----------
Stopped  winlogbeat         winlogbeat

After the service is installed make sure to start the service, as can be seen below.

1
2
3
4
5
6


PS C:\> Start-Service -Name winlogbeat
PS C:\> Get-Service -Name winlogbeat

Status   Name               DisplayName
------   ----               -----------
Running  winlogbeat         winlogbeat

After these steps have been completed you will see that Elasticsearch will have a new index with the winlogbeat suffix. You will have to add a new index pattern in Kibana in order to query this index.