AD ID Logging Dashboards and New Wec Config XML File (#388)

* New wec config xml for ad id logging and new dashboards * updated zip to contain new GPOs * Divine's dashboards * Fixed bug in lme_wec_config, missing asterisk in queries * Fixed casing on Query ID, must be Id * Fixed typo empty path string * Fixed scheduler typo * Renamed dashboards to fit convention * Dashboard descriptions * Fixed spacing * Fixed spacing 2 * Adjusted dashboard names * Typo fix * Upgrade instructions for release 1.4.0 for ID Logging * removing text * Adds code to upgrade to 1.4.0 * Check the latest and version in the upgrade section * Adds a better check for upgrading from 1.3.x to 1.4.x * Modify the version check for upgrades * Updates the dashboards to be all exported by the exporter script * Adds the updated menu to all of the dashboards * Update the upgrade directions for those who downloaded the version --------- Co-authored-by: Brown <rgbrow> Co-authored-by: unknown <[email protected]> Co-authored-by: Clint Baxley <[email protected]> Co-authored-by: cbaxley <[email protected]>
cisagov · Aug 16, 2024 · d4e64b4 · d4e64b4
1 parent 41214aa
commit d4e64b4
Show file tree

Hide file tree

Showing 20 changed files with 880 additions and 142 deletions.
diff --git a/Chapter 1 Files/lme_gpo_for_windows.zip b/Chapter 1 Files/lme_gpo_for_windows.zip
diff --git a/Chapter 1 Files/lme_wec_config.xml b/Chapter 1 Files/lme_wec_config.xml
diff --git a/Chapter 3 Files/dashboard_update.sh b/Chapter 3 Files/dashboard_update.sh
@@ -9,7 +9,7 @@ if [ -r /opt/lme/lme.conf ]; then
   #reference this file as a source
   . /opt/lme/lme.conf
   #check if the version number is equal to the one we want
-  if [ "$version" == "1.3.0" ] || [ "$FRESH_INSTALL" = "true" ]; then
+  if [ "$version" == "1.3.0" ] || [ "$version" == "1.4.0" ] || [ "$FRESH_INSTALL" = "true" ]; then
     echo -e "\e[32m[X]\e[0m Updating from git repo"
     git -C /opt/lme/ pull
     #make sure the hostname variable is present
@@ -19,8 +19,11 @@ if [ -r /opt/lme/lme.conf ]; then
       echo -e "\e[32m[X]\e[0m Uploading the new dashboards to Kibana"
       for db in ${Dashboards};
       do
-        echo -e "\e[32m[X]\e[0m Uploading ${db%%*.} dashboard\n"
-        curl -X POST -k --user dashboard_update:dashboardupdatepassword -H 'kbn-xsrf: true' --form file="@${dashbaord_dir}/${db}" "https://127.0.0.1/api/saved_objects/_import?overwrite=true"
+        filename=${db##*/}
+        filename_no_ext=${filename%.*}
+        echo -e "\e[32m[X]\e[0m Uploading ${filename_no_ext} dashboard\n"
+        curl -X POST -k --user dashboard_update:dashboardupdatepassword -H 'kbn-xsrf: true' --form file="@${db}" "https://127.0.0.1/api/saved_objects/_import?overwrite=true"
+
         echo
       done
 

diff --git a/Chapter 3 Files/deploy.sh b/Chapter 3 Files/deploy.sh
@@ -1087,11 +1087,31 @@ function upgrade() {
 
       info "Updating dashbaords"
       sudo /opt/lme/dashboard_update.sh
+    elif [ "$(printf '%s\n' "$version" "1.3.0" | sort -V | head -n1)" = "1.3.0" ] && \
+       [ "$(printf '%s\n' "$version" "1.3.9" | sort -V | head -n1)" = "$version" ]; then
+      info "Copying lme.conf -> lme.conf.bku"
+      sudo cp -rapf /opt/lme/lme.conf /opt/lme/lme.conf.bku
+
+      info "Copying dashboard_update.sh -> dashboard_update.sh.bku"
+      sudo cp -rapf /opt/lme/dashboard_update.sh /opt/lme/dashboard_update.sh.bku
+
+      info "Setting up new dashboard_update.sh"
+      sudo cp -rapf /opt/lme/Chapter\ 3\ Files/dashboard_update.sh /opt/lme/dashboard_update.sh
+      old_password=$(grep -P -o "(?<=dashboard_update:)[0-9a-zA-Z]+ " /opt/lme/dashboard_update.sh.bku)
+      sudo sed -i "s/dashboardupdatepassword/$old_password/g" /opt/lme/dashboard_update.sh
 
+      #update VERSION NUMBER
+      info "Updating Version to $latest"
+      sudo cp -rapf /opt/lme/lme.conf /opt/lme/lme.conf.bku
+      sudo sed -i -E "s/version=[0-9]+\.[0-9]+\.[0-9]+/version=$latest/g" /opt/lme/lme.conf
+      chmod u+rwx /opt/lme/dashboard_update.sh
+
+      info "Updating dashbaords"
+      sudo /opt/lme/dashboard_update.sh
     elif [ "$version" == $latest ]; then
       info "You're on the latest version!"
-    elif [ "$version" > "1.3.0" ]; then
-      info "There are no upgrades in this version. $latest"
+    elif [ "$(printf '%s\n' "$version" "1.4.0" | sort -V | tail -n1)" == "$version" ]; then
+      info "There are no upgrades in this version. Version: $version Latest: $latest"
     else
       error "Updating directly to LME 1.0 from versions prior to 0.5.1 is not supported. Update to 0.5.1 first."
     fi

diff --git a/Chapter 4 Files/dashboards/alerting_dashboard.ndjson b/Chapter 4 Files/dashboards/alerting_dashboard.ndjson
diff --git a/Chapter 4 Files/dashboards/computer_software_overview.ndjson b/Chapter 4 Files/dashboards/computer_software_overview.ndjson
diff --git a/Chapter 4 Files/dashboards/credential_access_logs_dashboard.ndjson b/Chapter 4 Files/dashboards/credential_access_logs_dashboard.ndjson
diff --git a/...ards/Healthcheckoverview_dashboard.ndjson → ...rds/healthcheck_dashboard_overview.ndjson b/...ards/Healthcheckoverview_dashboard.ndjson → ...rds/healthcheck_dashboard_overview.ndjson
diff --git a/Chapter 4 Files/dashboards/identity_access_management.ndjson b/Chapter 4 Files/dashboards/identity_access_management.ndjson
diff --git a/Chapter 4 Files/dashboards/policy_changes_and_system_activity.ndjson b/Chapter 4 Files/dashboards/policy_changes_and_system_activity.ndjson
diff --git a/Chapter 4 Files/dashboards/privileged_activity_log_dashboards.ndjson b/Chapter 4 Files/dashboards/privileged_activity_log_dashboards.ndjson
diff --git a/Chapter 4 Files/dashboards/process_explorer.ndjson b/Chapter 4 Files/dashboards/process_explorer.ndjson
diff --git a/Chapter 4 Files/dashboards/security_dashboard_security_log.ndjson b/Chapter 4 Files/dashboards/security_dashboard_security_log.ndjson
diff --git a/Chapter 4 Files/dashboards/sysmon_summary.ndjson b/Chapter 4 Files/dashboards/sysmon_summary.ndjson
diff --git a/Chapter 4 Files/dashboards/user_hr.ndjson b/Chapter 4 Files/dashboards/user_hr.ndjson
diff --git a/Chapter 4 Files/dashboards/user_security.ndjson b/Chapter 4 Files/dashboards/user_security.ndjson
diff --git a/Chapter 4 Files/export_dashboards.py b/Chapter 4 Files/export_dashboards.py
@@ -29,14 +29,20 @@ def get_basic_auth(username, password):
         return base64.b64encode(f"{username}:{password}".encode()).decode()
 
     def get_ids(self):
-        url = f'{self.root_url}/api/kibana/management/saved_objects/_find?perPage=500&page=1&fields=id&type=dashboard&sortField=updated_at&sortOrder=desc'
+        url = f'{self.root_url}/api/kibana/management/saved_objects/_find?perPage=500&page=1&type=dashboard&sortField=updated_at&sortOrder=desc'
 
         try:
             response = requests.get(url, headers={'Authorization': f'Basic {self.basic_auth}'}, verify=False)
 
             if response.status_code == 200:
                 data = response.json()
-                ids = {item['id']: item['meta']['title'] for item in data.get('saved_objects', [])}
+                #ids = {item['id']: item['meta']['title'] for item in data.get('saved_objects', [])}
+                #return ids
+                ids = {
+                    item['id']: item['meta']['title']
+                    for item in data.get('saved_objects', [])
+                    if '[' not in item['meta']['title'] and ']' not in item['meta']['title']
+                }
                 return ids
             else:
                 print(f"HTTP request failed with status code: {response.status_code}")

diff --git a/docs/markdown/maintenance/upgrading.md b/docs/markdown/maintenance/upgrading.md
@@ -132,18 +132,43 @@ sudo ./deploy.sh install
 ```
 
 ## 7. Upgrade to latest version 
+#### If you have checked out the repo to /opt/lme
 To fetch the latest changes, on the Linux server, run the following commands as root:
 ```
+sudo su
+cd /opt/lme/
 git pull
 git checkout main
 cd /opt/lme/Chapter\ 3\ Files/
+sudo ./deploy.sh upgrade
+```
+
+#### If you have NOT checked out the repo to /opt/lme
+```
+sudo su
+cd /opt/lme/Chapter\ 3\ Files/
 sudo ./deploy.sh uninstall
+# Follow directions to remove existing volumes
+cd /opt/
+rm -rf lme 
+git clone https://github.com/cisagov/LME.git
+mv LME lme
+cd /opt/lme/Chapter\ 3\ Files/
 sudo ./deploy.sh install
 ```
 
-The deploy.sh script should have now created new files on the Linux server at location /opt/lme/files_for_windows.zip . This file needs to be copied across and used on the Windows Event Collector server like it was explained in Chapter 3 sections [3.2.4 & 3.3 ](/docs/markdown/chapter3/chapter3.md#324-download-files-for-windows-event-collector). 
+The deploy.sh script should have now created new files on the Linux server at location /opt/lme/files_for_windows.zip . This file needs to be copied across and used on the Windows Event Collector server like it was explained in Chapter 3 sections [3.2.4 & 3.3 ](/docs/markdown/chapter3/chapter3.md#324-download-files-for-windows-event-collector).
+
+## 8. Upgrade to v1.4.0 additional steps
+To upgrade, you will need to update the LME Group Policy Objects and the Windows Event Collector.
 
-## 8. v1.4.0 - Upgrading Group Policy Objects
+### 8.1 Updating Group Policy Objects
 1. On the domain controller, open Group Policy Management, and delete "LME-WEC-Client" and "LME-WEC-Server". 
-2. Follow Chapter 1 instructions 1.3 and 1.4 to re-download LME, create the GPOs, import the v1.4.0 GPOs, and link them to your OU. Note: You will not need to create a new OU. You may reuse the OU used in prior installation. (https://github.com/cisagov/LME/blob/main/docs/markdown/chapter1/chapter1.md#13-download-lme)
-3. Right click on the OU the v1.4.0 GPOs are linked to and click "Group Policy Update"
+2. Follow Chapter 1 instructions 1.3 and 1.4 to re-download LME, create the GPOs, import the v1.4.0 GPOs, and link them to both your Clients OU and your Domain Controllers OU. Note: You will not need to create a new OU. You may reuse the OU from your prior installation. (https://github.com/cisagov/LME/blob/main/docs/markdown/chapter1/chapter1.md#13-download-lme)
+3. Right click on the OUs the v1.4.0 GPOs are linked to and click "Group Policy Update".
+
+### 8.2 Updating Windows Event Collector
+1. On the domain controller, open Event Viewer, go to Subscriptions, and delete "LME".
+2. Re-download LME.
+3. Follow steps 4-6 in 1.5 in Chapter 1 instructions to create a new subscription using the new lme_wec_config.xml file. 
+4. Open Group Policy Management and right click on the OU the v1.4.0 GPOs are linked to and click "Group Policy Update".
diff --git a/docs/markdown/reference/dashboard-descriptions.md b/docs/markdown/reference/dashboard-descriptions.md
@@ -35,6 +35,22 @@ The Alert Dashboard enables users to define rules that detect complex conditions
 
 The HealthCheck Dashboard gives users the ability to view different processes such as unexpected shutdowns, events by each machine, total hosts and total number of logged in admins with data that is based on a selected date range.  Users can verify the health of their system by observing events such as if there are more admin users than expected or if an unexpected shutdown occurs. 
 
+## Policy Changes and System Activity
+
+The Policy Changes and System Activity dashboard enables users to monitor policy changes and important system activity. Users will be able to monitor the status of their firewall, including when it is turned on, off, its settings are changed, or exception rules are added or modified. This dashboard will also show when firewall, audit, or Kerberos policies are changed on their domain. Users will also be able to monitor when their PCs are turned on, off, and when RPC (Remote Procedure Call) connections are attempted on their domain. 
+
+## Identity Access Management
+
+The Identity Access Management dashboard provides users with a collection of important security events involving identity and critical object access. This includes when registry objects, task scheduler jobs, and when password hashes are accessed. Users will also be able to monitor when passwords are reset, changed, and when users are locked out of their accounts. This dashboard also tracks when the default domain policy is changed which involves the domain password policy. 
+
+## Privileged Activity Log
+
+The Privileged Activity Log dashboard enables users to carry on audits related to non-sensitive and sensitive events by showcasing the number of privileged service attempts, sensitive privilege attempts and non-sensitive privilege attempts made per host name. It also shows the number of processes created and terminated per host name. Such as process creation count, process termination counts as well as assigned token creation count per host.
+
+## Credential Access Log
+
+The Credential Access Log dashboard, focuses on account logon and account logoff audit events. In this dashboard, users will be able to monitor, audit logon attempts per hosts, logon using explicit credential attempts, account lockout attempts per host, special logon attempts per hosts, disconnection attempts, and credential validation attempts per host. Dashboard panels will also showcase Kerberos authentication services per host.
+
 
 
 For more information or to seek additional help, [Click Here](https://github.com/cisagov/LME) 
diff --git a/testing/development/upgrade_lme.sh b/testing/development/upgrade_lme.sh
@@ -20,7 +20,7 @@ export current_branch=$(git rev-parse --abbrev-ref HEAD)
 # Get the version that we are going to upgrade to
 . ./merging_version.sh
 
-# Checkout the version we are on
+# Checkout the version in /opt/lme that we are on in ~/LME
 sudo  echo "Current branch: $current_branch"
 sudo  echo "Forcing version: $FORCE_LATEST_VERSION"
 sudo  sh -c "cd '/opt/lme/' && git checkout 'Chapter\ 3\ Files/deploy.sh' && git checkout -t origin/$current_branch && git pull"