Feature/aw/p 10 6 1 (#763)

* Remove suppress documentation

* Check for reference ID's

---------

Co-authored-by: Anthony Watherston <[email protected]>

Docs/operational-scripts.md

@@ -15,18 +15,18 @@ This script executes all remediation tasks in a Policy as Code environment speci
 
 #### Links
 
-- [Remediate non-compliant resources with Azure Policy](https://learn.microsoft.com/en-us/azure/governance/policy/how-to/remediate-resources?tabs=azure-portal)
-- [Start-AzPolicyRemediation](https://learn.microsoft.com/en-us/powershell/module/az.policyinsights/start-azpolicyremediation?view=azps-10.1.0)
+* [Remediate non-compliant resources with Azure Policy](https://learn.microsoft.com/en-us/azure/governance/policy/how-to/remediate-resources?tabs=azure-portal)
+* [Start-AzPolicyRemediation](https://learn.microsoft.com/en-us/powershell/module/az.policyinsights/start-azpolicyremediation?view=azps-10.1.0)
 
 ## Documenting Policy
 
 `Build-PolicyDocumentation` builds documentation from instructions in the `policyDocumentations` folder reading the deployed Policy Resources from the EPAC environment. It is also used to generate parameter/effect CSV files for Policy Assignment files. See usage documentation in [Documenting Policy](operational-scripts-documenting-policy.md).
 
 ## Policy Resources Exports
 
-- `Export-AzPolicyResources` exports Azure Policy resources in EPAC. It also generates documentation for the exported resources (can be suppressed with `-SuppressDocumentation`). See usage documentation in [Extract existing Policy Resources](start-extracting-policy-resources.md).
-- `Get-AzExemptions` retrieves Policy Exemptions from an EPAC environment and saves them to files.
-- `Get-AzPolicyAliasOutputCSV` exports Policy Aliases to CSV format.
+* `Export-AzPolicyResources` exports Azure Policy resources in EPAC. See usage documentation in [Extract existing Policy Resources](start-extracting-policy-resources.md).
+* `Get-AzExemptions` retrieves Policy Exemptions from an EPAC environment and saves them to files.
+* `Get-AzPolicyAliasOutputCSV` exports Policy Aliases to CSV format.
 
 ## Hydration Kit
 
@@ -36,36 +36,36 @@ The Hydration Kit is a set of scripts that can be used to deploy an EPAC environ
 
 The scripts `New-AzureDevOpsBug` and `New-GitHubIssue` create a Bug or Issue when there are one or multiple failed Remediation Tasks.
 
-
 ## Export from AzAdvertizer
 
-The script `Export-AzAdvertizer.ps1` creates for you the policyAssignments, policyDefinitions, and policySetDefinitions based on the provided URL in an Output folder under 'ALZ-Export'.
+The script `Export-AzAdvertizerPolicy.ps1` creates for you the policyAssignments, policyDefinitions, and policySetDefinitions based on the provided URL in an Output folder under 'ALZ-Export'.
 
 Parameters:
-- **AzAdvertizerUrl**: Mandatory url of the policy or policy set from AzAdvertizer.
 
-- **OutputFolder**: Output Folder. Defaults to the path 'Output'.
-    
-- **AutoCreateParameters**: Automatically create parameters for Azure Policy Sets and Assginment Files.
+* **AzAdvertizerUrl**: Mandatory url of the policy or policy set from AzAdvertizer.
+
+* **OutputFolder**: Output Folder. Defaults to the path 'Output'.
 
-- **UseBuiltIn**: Default to using builtin policies rather than local versions.
+* **AutoCreateParameters**: Automatically create parameters for Azure Policy Sets and Assginment Files.
 
-- **Scope**: Used to set scope value on each assignment file.
+* **UseBuiltIn**: Default to using builtin policies rather than local versions.
 
-- **PacSelector**: Used to set PacEnvironment for each assignment file.
+* **Scope**: Used to set scope value on each assignment file.
 
-- **OverwriteOutput**: Used to Overwrite the contents of the output folder with each run. Helpful when running consecutively.
+* **PacSelector**: Used to set PacEnvironment for each assignment file.
+
+* **OverwriteOutput**: Used to Overwrite the contents of the output folder with each run. Helpful when running consecutively.
 
 ## Non-compliance Reports
 
 `Export-NonComplianceReports` exports non-compliance reports for EPAC environments . It outputs the reports in the `$OutputFolders/non-compliance-reports` folder.
 
-- `summary-by-policy.csv` contains the summary of the non-compliant resources by Policy definition. The columns contain the resource counts.
-- `summary-by-resource.csv` contains the summary of the non-compliant resources. The columns contain the number of Policies causing the non-compliance.
-- `details-by-policy.csv` contains the details of the non-compliant resources by Policy definition including the non-compliant resource ids. Assignments are combined by Policy definition.
-- `details-by-resource.csv` contains the details of the non-compliant resources sorted by Resource id. Assignments are combined by Resource id.
-- `full-details-by-assignment.csv` contains the details of the non-compliant resources sorted by Policy Assignment id.
-- `full-details-by-resource.csv` contains the details of the non-compliant resources sorted by Resource id including the Policy Assignment details.
+* `summary-by-policy.csv` contains the summary of the non-compliant resources by Policy definition. The columns contain the resource counts.
+* `summary-by-resource.csv` contains the summary of the non-compliant resources. The columns contain the number of Policies causing the non-compliance.
+* `details-by-policy.csv` contains the details of the non-compliant resources by Policy definition including the non-compliant resource ids. Assignments are combined by Policy definition.
+* `details-by-resource.csv` contains the details of the non-compliant resources sorted by Resource id. Assignments are combined by Resource id.
+* `full-details-by-assignment.csv` contains the details of the non-compliant resources sorted by Policy Assignment id.
+* `full-details-by-resource.csv` contains the details of the non-compliant resources sorted by Resource id including the Policy Assignment details.
 
 ### Sample `summary-by-policy.csv`
 
@@ -80,4 +80,3 @@ Parameters:
 | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
 | /subscriptions/******************************** | ******************************** | PAC-DEV-001 |  | subscriptions |  |  | 25 | 481 | 0 | 0 | 0 | 0 |
 | /subscriptions/********************************/providers/microsoft.authorization/roledefinitions/0b00bc79-2207-410c-b9d5-d5d182ad514f | ******************************** | PAC-DEV-001 |  | microsoft.authorization/roledefinitions | 0b00bc79-2207-410c-b9d5-d5d182ad514f |  | 0 | 0 | 0 | 0 | 0 | 0 |
-

Scripts/Helpers/Build-AssignmentDefinitionAtLeaf.ps1

@@ -196,7 +196,14 @@ function Build-AssignmentDefinitionAtLeaf {
         }
         foreach ($nonComplianceMessageRaw in $nonComplianceMessagesList) {
             if ($null -eq $nonComplianceMessageRaw.message -or $nonComplianceMessageRaw.message -eq "") {
-                Write-Error "    Leaf Node $($nodeName): each nonComplianceMessage must conatin a message string: $($nonComplianceMessageRaw | ConvertTo-Json -Depth 100 -Compress)"
+                Write-Error "    Leaf Node $($nodeName): each nonComplianceMessage must contain a message string: $($nonComplianceMessageRaw | ConvertTo-Json -Depth 100 -Compress)"
+                $hasErrors = $true
+            }
+        }
+
+        foreach ($referenceId in $nonComplianceMessagesList.policyDefinitionReferenceId) {
+            if ($referenceId -notin $CombinedPolicyDetails.policySets[$definitionEntry.policyDefinitionId].policyDefinitions.policyDefinitionReferenceId) {
+                Write-Error "   Reference Id $referenceId in nonComplianceMessages is not found in the policy definition: $($definitionEntry.policyDefinitionId)"
                 $hasErrors = $true
             }
         }

Scripts/Helpers/Out-DocumentationForPolicyAssignments.ps1

@@ -287,15 +287,15 @@ function Out-DocumentationForPolicyAssignments {
         $null = $allLines.Add("`n$leadingHeadingHashtag# Policy Effects by Policy`n")
         $null = $allLines.Add("| Category | Policy | Group Names |$addedTableHeader")
         $null = $allLines.Add("| :------- | :----- | :---------- |$addedTableDivider")
-        $null = $assignmentsByCategoryHeader.Add("`n$leadingHeadingHashtag# Security Controls by Policy`n")
+        $null = $assignmentsByCategoryHeader.Add("`n$leadingHeadingHashtag# Policy Effects by Policy`n")
         $null = $assignmentsByCategoryHeader.Add("| Category | Policy | Group Names |$addedTableHeader")
         $null = $assignmentsByCategoryHeader.Add("| :------- | :----- | :---------- |$addedTableDivider")
     }
     else {
         $null = $allLines.Add("`n$leadingHeadingHashtag# Policy Effects by Policy`n")
         $null = $allLines.Add("| Category | Policy |$addedTableHeader")
         $null = $allLines.Add("| :------- | :----- |$addedTableDivider")
-        $null = $assignmentsByCategoryHeader.Add("`n$leadingHeadingHashtag# Security Controls by Policy`n")
+        $null = $assignmentsByCategoryHeader.Add("`n$leadingHeadingHashtag# Policy Effects by Policy`n")
         $null = $assignmentsByCategoryHeader.Add("| Category | Policy |$addedTableHeader")
         $null = $assignmentsByCategoryHeader.Add("| :------- | :----- |$addedTableDivider")
     }
@@ -448,10 +448,21 @@ function Out-DocumentationForPolicyAssignments {
                     # Add to main markdown
                     $null = $allLines.Add("| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)")
                     # Add to sub-page markdown
-                    if ($assignmentsByCategory.ContainsKey($_.category)) {   
-                        $assignmentsByCategory[$_.category].subLines += "| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)"
+                    if ($assignmentsByCategory.ContainsKey($_.category)) {
+                        if ($assignmentsByCategory[$_.category].subLines -match "Policy Parameters by Policy") {
+                            $assignmentsByCategory[$_.category].subLines += "| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)"
+                        }
+                        else {
+                            $null = $assignmentsByCategory[$_.category].subLines += "`n$leadingHeadingHashtag# Policy Parameters by Policy`n"
+                            $null = $assignmentsByCategory[$_.category].subLines += "| Category | Policy |$addedTableHeader"
+                            $null = $assignmentsByCategory[$_.category].subLines += "| :------- | :----- |$addedTableDividerParameters"
+                            $assignmentsByCategory[$_.category].subLines += "| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)"
+                        }
                     }
                     else {
+                        $null = $assignmentsByCategory[$_.category].subLines += "`n$leadingHeadingHashtag# Policy Parameters by Policy`n"
+                        $null = $assignmentsByCategory[$_.category].subLines += "| Category | Policy |$addedTableHeader"
+                        $null = $assignmentsByCategory[$_.category].subLines += "| :------- | :----- |$addedTableDividerParameters"
                         $assignmentsByCategory[$_.category] = @{}
                         $assignmentsByCategory[$_.category].subLines = $assignmentsByCategoryHeader
                         $assignmentsByCategory[$_.category].subLines += "| $($_.category) | **$($_.displayName)**$($inTableAfterDisplayNameBreak)$($_.description) |$($addedParametersColumns)"

Scripts/Operations/Export-AzAdvertizerPolicy.ps1

@@ -44,7 +44,7 @@ param (
     [string] $OutputFolder,
 
     [Parameter(Mandatory = $false, HelpMessage = "Automatically create parameters for Azure Policy Sets and Assginment Files")]
-    [bool] $AutoCreateParameters = $false,
+    [bool] $AutoCreateParameters = $true,
 
     [Parameter(Mandatory = $false, HelpMessage = "Default to using builtin policies rather than local versions")]
     [bool] $UseBuiltIn = $true,
@@ -61,33 +61,40 @@ param (
 
 #region Policy
 
+# Validate session with Azure exists
+if (-not (Get-AzContext)) {
+    $null = Connect-AzAccount
+}
+
+# Pull HTML
+try {
+    $restResponse = Invoke-RestMethod -Uri "$AzAdvertizerUrl" -Method Get
+}
+catch {
+    if ($_.Exception.Response.StatusCode -lt 200 -or $_.Exception.Response.StatusCode -ge 300) {
+        Write-Error "Error gathering Policy Information, please validate AzAdvertizerUrl is valid." -ErrorAction Stop
+    }
+}
+
 # Determine if PolicySet or Policy Assignment
 if ($AzAdvertizerUrl -match "azpolicyadvertizer") {
     $policyType = "policyDefinitions"
+    # Pull Built-In Policies and Policy Sets
+    $builtInPolicies = Get-AzPolicyDefinition -Builtin
+    $builtInPolicyNames = $builtInPolicies.name
 }
 elseif ($AzAdvertizerUrl -match "azpolicyinitiativesadvertizer") {
     $policyType = "policySetDefinitions"
+    # Pull Built-In Policies and Policy Sets
+    $builtInPolicies = Get-AzPolicyDefinition -Builtin
+    $builtInPolicyNames = $builtInPolicies.name
+    $builtInPolicySets = Get-AzPolicySetDefinition -Builtin
+    $builtInPolicySetNames = $builtInPolicySets.name
 }
 else {
     Write-Error "AzAdvertizerUrl is not referencing Policy or Policy Initiative"
 }
 
-# Pull Built-In Policies and Policy Sets
-$builtInPolicies = Get-AzPolicyDefinition -Builtin
-$builtInPolicyNames = $builtInPolicies.name
-$builtInPolicySets = Get-AzPolicySetDefinition -Builtin
-$builtInPolicySetNames = $builtInPolicySets.name
-
-# Pull HTML
-try {
-    $restResponse = Invoke-RestMethod -Uri "$AzAdvertizerUrl" -Method Get
-}
-catch {
-    if ($_.Exception.Response.StatusCode -eq 404) {
-        Write-Error "Error 404: Not Found." -ErrorAction Stop
-    }
-}
-
 # Parse HTML response to find copyEPACDef
 Write-Information "" -InformationAction Continue
 $response = $restResponse.split("function copyEPACDef() ")[1]
@@ -133,16 +140,11 @@ if ($OutputFolder -eq "") {
     $OutputFolder = "Output"
 }
 if ($OverwriteOutput) {
-    if (Test-Path -Path "$OutputFolder/ALZ-Export/policyDefinitions") {
-        Get-ChildItem -Path "$OutputFolder/ALZ-Export/policyDefinitions" -File | Remove-Item
-    }
-    if (Test-Path -Path "$OutputFolder/ALZ-Export/policyAssignments") {
-        Get-ChildItem -Path "$OutputFolder/ALZ-Export/policyAssignments" -File | Remove-Item
-    }
-    if (Test-Path -Path "$OutputFolder/ALZ-Export/policySetDefinitions") {
-        Get-ChildItem -Path "$OutputFolder/ALZ-Export/policySetDefinitions" -File | Remove-Item
+    if (Test-Path -Path "$OutputFolder/ALZ-Export") {
+        Remove-Item -Path "$OutputFolder/ALZ-Export" -Recurse -Force
     }
 }
+
 # Export File
 if ($policyType -eq "policyDefinitions") {
     if ($UseBuiltIn -and $builtInPolicyNames -contains $policyName) {
@@ -317,7 +319,7 @@ if ($AutoCreateParameters) {
     if ($UseBuiltIn) {
         if ($policyType -eq "policySetDefinitions" -and $builtInPolicySetNames -contains $policyName) {
             $policyObject = $builtInPolicySets | Where-Object { $_.Name -eq $policyName }
-            $policySetParameters = $policyObject.properties.parameters
+            $policySetParameters = $policyObject.parameter
             $parameterNames = $policySetParameters | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty Name
             foreach ($parameter in $parameterNames) {
                 $defaultEffect = ""
@@ -332,15 +334,15 @@ if ($AutoCreateParameters) {
         }
         elseif ($policyType -eq "policyDefinitions" -and $builtInPolicyNames -contains $policyName) {
             $policyObject = $builtInPolicies | Where-Object { $_.Name -eq $policyName }
-            $policyParameters = $policyObject.properties.parameters
+            $policyParameters = $policyObject.parameter
             $parameterNames = $policyParameters | Get-Member -MemberType NoteProperty | Select-Object -ExpandProperty Name
             foreach ($parameter in $parameterNames) {
                 $defaultEffect = ""
-                if ($null -eq $($policySetParameters).$($parameter).defaultValue) {
+                if ($null -eq $($policyParameters).$($parameter).defaultValue) {
                     Write-Warning "Default Value not found for '$parameter' in PolicySet '$policyName'" -InformationAction Continue
                 }
                 else {
-                    $defaultEffect = $($policySetParameters).$($parameter).defaultValue
+                    $defaultEffect = $($policyParameters).$($parameter).defaultValue
                 }
                 $assignmentObject.children.parameters | Add-Member -MemberType NoteProperty -Name "$parameter" -Value $defaultEffect
             }

Scripts/Operations/Export-AzPolicyResources.ps1

@@ -3,9 +3,7 @@
 Exports Azure Policy resources in EPAC format or raw format.
 
 .DESCRIPTION
-Exports Azure Policy resources in EPAC format or raw format. It has 4 operating modes - see -Mode parameter for details.
-It also generates documentation for the exported resources (can be suppressed with -SuppressDocumentation).
-To just generate EPAC formatted Definitions without generating documentation files, use -supressEpacOutput.
+Exports Azure Policy resources in EPAC format or raw format. It has 5 operating modes - see -Mode parameter for details.
 
 .PARAMETER DefinitionsRootFolder
 Definitions folder path. Defaults to environment variable $env:PAC_DEFINITIONS_FOLDER or './Definitions'.
@@ -40,9 +38,6 @@ Operating mode:
 Limits the collection to one EPAC environment, useful for non-interactive use in a multi-tenant scenario, especially with -Mode 'collectRawFile'.
 The default is '*' which will execute all EPAC-Environments.
 
-.PARAMETER SuppressDocumentation
-Suppress documentation generation.
-
 .PARAMETER SuppressEpacOutput
 Suppress output generation in EPAC format.
 
@@ -103,8 +98,8 @@ param (
     ")]
     [string] $InputPacSelector = '*',
 
-    [Parameter(Mandatory = $false, HelpMessage = "Suppress documentation generation")]
-    [switch] $SuppressDocumentation,
+    # [Parameter(Mandatory = $false, HelpMessage = "Suppress documentation generation")]
+    # [switch] $SuppressDocumentation,
 
     [Parameter(Mandatory = $false, HelpMessage = "Suppress output generation in EPAC format")]
     [switch] $SuppressEpacOutput,