feat: (recommend) Implement recommend functionality for Docker Client #461

tesla59 · 2024-09-17T10:34:11Z

Signed-off-by: tesla59 <[email protected]>

use a common interface to support other clients as well also create common Object{} to support different k8s obejcts such as Deployment, Daemonset etc Signed-off-by: tesla59 <[email protected]>

Signed-off-by: tesla59 <[email protected]>

…rs as well Signed-off-by: tesla59 <[email protected]>

this removes dependency of recommend command on kubearmor Signed-off-by: tesla59 <[email protected]>

tesla59 · 2024-10-02T11:57:23Z

@daemon1024 please review

daemon1024 · 2024-10-03T06:32:06Z

Can you include screenshots of how it's working?

tesla59 · 2024-10-03T06:54:00Z

Case 1: systemd mode

karmor recommend fails due to --k8s=true by default and no cluster is running. K8s client is used
karmor recommend --k8s=false uses docker client but policy is not generated due to no containers running
karmor recommend --k8s=false generates policy after nginx container is run and policy is generated

Case 2: k8s mode

Works as before

There is an error ERRO[0010] Not a valid tar file file=/tmp/karmor867115595/blobs/sha256/0162fa012a5d588eb52b8edaef90c4aecf89021924a20a2ca62f8dad7b766bf7 but it is not related to this PR. Will push fix for that in another pr

Signed-off-by: tesla59 <[email protected]>

rootxrishabh

Initial review - Changes are working as expected.

rootxrishabh · 2024-11-11T15:54:55Z

Output directory is -server-1 which gives an error if I try to enter it -
rootxrishabh@fedora:/tmp/out$ cd -server-1 bash: cd: -s: invalid option cd: usage: cd [-L|[-P [-e]] [-@]] [dir]

Is this intended?

tesla59 · 2024-11-13T10:32:35Z

cd -server-1

shouldn't this be cd -- -server-1

ill check what is the issue with output directory name

Signed-off-by: tesla59 <[email protected]>

tesla59 · 2024-11-19T05:37:18Z

@rootxrishabh policy directory is fixed. it was due to img.namespace being null in VM mode.

tesla59 · 2024-11-19T05:44:00Z

@rootxrishabh also added commit to trim new line character in final report generation. it fixes the weird number of blank lines after reports table

Old

New

rootxrishabh

karmor crashes when using --k8s and -i together. @tesla59 PTAL

rootxrishabh · 2025-01-02T11:16:46Z

karmor exists with an error. With -k8s=false and an nginx container running on docker. Is this intended?

accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > docker ps
CONTAINER ID   IMAGE     COMMAND                  CREATED          STATUS          PORTS     NAMES
45eb1cf51480   nginx     "/docker-entrypoint.…"   20 minutes ago   Up 20 minutes   80/tcp    modest_pascal

accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend -k=false
ERRO[0000] no Object found to secure, hence nothing to recommend!  namespace=

…recommend

Signed-off-by: tesla59 <[email protected]>

tesla59 · 2025-01-11T11:37:40Z

hey @rootxrishabh
i am unable to reproduce any of the above issues

When running karmor recommend --k8s=false -i nginx

When running karmor recommend -k=false with nginx running as docker container

I think the issue could be due to different envs, although unlikely.
Another thing that i noticed was i missed one error check when listing objects in Recommend() function, which is where the 2nd screenshot is not working as expected. i have pushed the fix for error check can u please check again? i believe there is some error while listing the object which is why its throwing array index out of range. ill push another debug branch which has bunch of log.Info(), it will give more insights on where the panic is actually happening (since in the original screenshot, it is not possible to judge anything)

link to debug branch

rootxrishabh

Tested karmor recommend --k8s=false -i nginx seems to work now. I am not able to reproduce the previous error.

accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend --k8s=false -i nginx
INFO[0000] Found outdated version of policy-templates    Current Version=v0.2.3
INFO[0000] Downloading latest version [v0.2.6]
INFO[0001] policy-templates updated                      Updated Version=v0.2.6
INFO[0001] pulling image                                 image=nginx
latest: Pulling from library/nginx
7ce705000c39: Pull complete
b3e9225c8fca: Pull complete
2b39a3d0829e: Pull complete
6d24e34787c7: Pull complete
066d623ff8e6: Pull complete
49486a4a61a6: Pull complete
34d83bb3522a: Pull complete
Digest: sha256:0a399eb16751829e1af26fea27b20c3ec28d7ab1fb72182879dcae1cca21206a
Status: Downloaded newer image for nginx:latest
INFO[0063] dumped image to tar                           tar=/var/folders/xp/hyg6_b6j7hgc23b7bd7d35l00000gn/T/karmor2062359421/pakTcFqD.tar
Distribution debian
created policy out/nginx-latest/write-under-dev-dir.yaml ...
created policy out/nginx-latest/k8s-client-tool-exec.yaml ...
created policy out/nginx-latest/file-system-mounts.yaml ...
created policy out/nginx-latest/crypto-miners.yaml ...
created policy out/nginx-latest/cis-commandline-warning-banner.yaml ...
created policy out/nginx-latest/file-integrity-monitoring.yaml ...
created policy out/nginx-latest/cronjob-cfg.yaml ...
created policy out/nginx-latest/pkg-mngr-exec.yaml ...
created policy out/nginx-latest/remote-file-copy.yaml ...
created policy out/nginx-latest/write-etc-dir.yaml ...
created policy out/nginx-latest/impair-defense.yaml ...
created policy out/nginx-latest/user-grp-mod.yaml ...
created policy out/nginx-latest/maint-tools-access.yaml ...
created policy out/nginx-latest/write-in-shm-dir.yaml ...
created policy out/nginx-latest/access-ctrl-permission-mod.yaml ...
created policy out/nginx-latest/remote-services.yaml ...
created policy out/nginx-latest/trusted-cert-mod.yaml ...
created policy out/nginx-latest/system-owner-discovery.yaml ...
created policy out/nginx-latest/system-network-env-mod.yaml ...
created policy out/nginx-latest/network-service-scanning.yaml ...
output report in out/report.txt ...
  Container               | nginx:latest
  OS                      | linux
  Arch                    | arm64
  Distro                  | debian
  Output Directory        | out/nginx-latest
  policy-template version | v0.2.3
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
|               POLICY                |           SHORT DESC           | SEVERITY | ACTION |                       TAGS                        |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-under-dev-dir.yaml            | Audit device directory for     | 5        | Audit  | NIST NIST_800-53_AU-2                             |
|                                     | enhanced security              |          |        | NIST_800-53_SI-4 MITRE                            |
|                                     |                                |          |        | MITRE_T1036_masquerading                          |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| k8s-client-tool-exec.yaml           | Prevent execution of container | 5        | Block  | MITRE_T1609_container_administration_command      |
|                                     | administration tools within a  |          |        | MITRE_TA0002_execution                            |
|                                     | container                      |          |        | MITRE_T1610_deploy_container                      |
|                                     |                                |          |        | MITRE NIST_800-53 NIST_800-53_AU-2                |
|                                     |                                |          |        | NIST_800-53_SI-4 NIST                             |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| file-system-mounts.yaml             | Ensure successful file system  | 5        | Audit  | CIS CIS_Linux                                     |
|                                     | mounts are collected           |          |        | CIS_4_Logging_and_Aduditing                       |
|                                     |                                |          |        | CIS_4.1.1_Data_Retention                          |
|                                     |                                |          |        | CIS_4.1.14_file_system_mount                      |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| crypto-miners.yaml                  | Cryptojacking, Crypto mining,  | 10       | Block  | cryptominer                                       |
|                                     | Malware protection             |          |        | MITRE_T1496_resource_hijacking                    |
|                                     |                                |          |        | MITRE                                             |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| cis-commandline-warning-banner.yaml | Command Line Warning Banners   | 5        | Block  | CIS CIS_Linux CIS_1.7_Warning_Banners             |
|                                     |                                |          |        | CIS_1.7.1_Command_Line_Warning_Banners            |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| file-integrity-monitoring.yaml      | File Integrity                 | 1        | Block  | NIST NIST_800-53_AU-2                             |
|                                     | Monitoring/Protection          |          |        | NIST_800-53_SI-4 MITRE                            |
|                                     |                                |          |        | MITRE_T1036_masquerading                          |
|                                     |                                |          |        | MITRE_T1565_data_manipulation                     |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| cronjob-cfg.yaml                    | Audit access to cronjob files  | 5        | Audit  | NIST SI-4                                         |
|                                     | as a part of system monitoring |          |        | NIST_800-53_SI-4                                  |
|                                     | for better integrity           |          |        | CIS CIS_Linux                                     |
|                                     |                                |          |        | CIS_5.1_Configure_Cron                            |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| pkg-mngr-exec.yaml                  | Prohibit package manager       | 5        | Block  | NIST                                              |
|                                     | process execution in           |          |        | NIST_800-53_CM-7(4)                               |
|                                     | containers to maintain system  |          |        | SI-4 process                                      |
|                                     | integrity and limit authorized |          |        | NIST_800-53_SI-4                                  |
|                                     | software versions and sources. |          |        |                                                   |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| remote-file-copy.yaml               | Prevent data exfiltration      | 5        | Block  | MITRE                                             |
|                                     | attempts using utility tooling |          |        | MITRE_TA0008_lateral_movement                     |
|                                     |                                |          |        | MITRE_TA0010_exfiltration                         |
|                                     |                                |          |        | MITRE_TA0006_credential_access                    |
|                                     |                                |          |        | MITRE_T1552_unsecured_credentials                 |
|                                     |                                |          |        | NIST_800-53_SI-4(18) NIST                         |
|                                     |                                |          |        | NIST_800-53 NIST_800-53_SC-4                      |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-etc-dir.yaml                  | Prevent concealment of         | 5        | Block  | NIST_800-53_SI-7 NIST                             |
|                                     | adversarial processes          |          |        | NIST_800-53_SI-4 NIST_800-53                      |
|                                     |                                |          |        | MITRE_T1562.001_disable_or_modify_tools           |
|                                     |                                |          |        | MITRE_T1036.005_match_legitimate_name_or_location |
|                                     |                                |          |        | MITRE_TA0003_persistence                          |
|                                     |                                |          |        | MITRE MITRE_T1036_masquerading                    |
|                                     |                                |          |        | MITRE_TA0005_defense_evasion                      |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| impair-defense.yaml                 | Audit defense control points   | 6        | Audit  | MITRE FGT1562 FIGHT 5G                            |
|                                     | to detect defense impairments  |          |        | MITRE_T1562_Impair _Defenses                      |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| user-grp-mod.yaml                   | Audit access to useradd and    | 1        | Block  | MySQL                                             |
|                                     | groupadd command!              |          |        | CIS                                               |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| maint-tools-access.yaml             | Restrict or limit maintenance  | 1        | Audit  | PCI_DSS MITRE                                     |
|                                     | tool usage                     |          |        | MITRE_T1553_Subvert_Trust_Controls                |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| write-in-shm-dir.yaml               | Restrict adversaries from      | 5        | Block  | MITRE_TA0002_Execution                            |
|                                     | writing malicious code under   |          |        | MITRE                                             |
|                                     | the shm folder                 |          |        |                                                   |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| access-ctrl-permission-mod.yaml     | Ensure discretionary           | 5        | Block  | CIS CIS_Linux CIS_4_Logging_and_Aduditing         |
|                                     | access control permission      |          |        | CIS_4.1.1_Data_Retention                          |
|                                     | modification events are        |          |        | CIS_4.1.11_system_access_control_permission       |
|                                     | collected                      |          |        |                                                   |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| remote-services.yaml                | Audit remote access services   | 3        | Audit  | MITRE FIGHT FGT1021 5G                            |
|                                     |                                |          |        | MITRE_T1021_Remote_Services                       |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| trusted-cert-mod.yaml               | Prevent certificate bundle     | 1        | Block  | MITRE                                             |
|                                     | tampering                      |          |        | MITRE_T1552_unsecured_credentials                 |
|                                     |                                |          |        | FGT1555 FIGHT                                     |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-owner-discovery.yaml         | Limit adversaries from         | 3        | Block  | MITRE                                             |
|                                     | gathering system information   |          |        | MITRE_T1082_system_information_discovery          |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-network-env-mod.yaml         | Ensure events that modify the  | 5        | Block  | CIS CIS_Linux                                     |
|                                     | system's network environment   |          |        | CIS_4_Logging_and_Aduditing                       |
|                                     | are collected                  |          |        | CIS_4.1.1_Data_Retention                          |
|                                     |                                |          |        | CIS_4.1.7_system_network_environment              |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| network-service-scanning.yaml       | Audit execution of network     | 5        | Audit  | MITRE FGT1046 FIGHT 5G                            |
|                                     | service scanning tools         |          |        | MITRE_T1046_Network_Service_Discovery             |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+

rootxrishabh

But the docker client is throwing an error for being newer than the daemon. Please think of a way to handle this as most users could face this issue. I see we are using github.com/docker/docker v25.0.5+incompatible. @daemon1024 any ideas?

accuknox/kubearmor-client - (tesla/non-k8s/karmor-recommend) > ./karmor recommend --k8s=false
Error: Error response from daemon: client version 1.44 is too new. Maximum supported API version is 1.43

…recommend

tesla59 · 2025-01-19T06:36:59Z

hey @rootxrishabh can u share the output of docker version
as per the docker sdk documentation, API versions before v24 are deprecated

github.com/docker/[email protected]+incompatible gives while trying to compile

../../../../go/pkg/mod/oras.land/[email protected]/pkg/auth/docker/login_tls.go:44:20: undefined: types.AuthConfig
../../../../go/pkg/mod/oras.land/[email protected]/pkg/auth/docker/login_tls.go:62:131: undefined: types.AuthConfig
../../../../go/pkg/mod/oras.land/[email protected]/pkg/auth/docker/login_tls.go:122:32: undefined: types.AuthConfig
../../../../go/pkg/mod/oras.land/[email protected]/pkg/auth/docker/login.go:54:16: undefined: types.AuthConfig

github.com/docker/[email protected]+incompatible gives while trying to compile

recommend/registry/registry.go:174:16: undefined: dockerTypes.PullOptions

github.com/docker/[email protected]+incompatible seem to work just fine

I think that begs the question how many Daemon version do we intend to support and which version should we use

tesla59 · 2025-01-19T06:38:51Z

although the v27 docker client was already present in the go.mod in kubearmor-client
https://github.com/kubearmor/kubearmor-client/blob/325762c67aa8e01e0627c5413100c90f2148fe0b/go.mod#L14C2-L14C47

rootxrishabh

My docker version is 24.0.2. So it should be supported?

tesla59 added 3 commits September 15, 2024 08:12

cmd: rename k8s client to k8sclient

88334c6

Signed-off-by: tesla59 <[email protected]>

cmd: wrap k8sClient in interface to use in recommend package

c739a95

Signed-off-by: tesla59 <[email protected]>

k8s: fix ListDeployment method

0feed1a

Signed-off-by: tesla59 <[email protected]>

tesla59 marked this pull request as draft September 17, 2024 10:34

tesla59 added 8 commits September 22, 2024 07:23

recommend: refactor k8s policy generation

ddb361a

use a common interface to support other clients as well also create common Object{} to support different k8s obejcts such as Deployment, Daemonset etc Signed-off-by: tesla59 <[email protected]>

recommend: use image name as Object's name while generating policy

2819a3f

Signed-off-by: tesla59 <[email protected]>

recommend: remove DeploymentName Field from Object{}

5474f33

Signed-off-by: tesla59 <[email protected]>

recommend: use ListOptions to select deployment by labels

a95b469

Signed-off-by: tesla59 <[email protected]>

k8s: generate recommended policies for CronJob, DaemonSet and Jobs

82d6666

Signed-off-by: tesla59 <[email protected]>

k8s: generate recommend policy for statefulset and unowned replicaset

993274e

Signed-off-by: tesla59 <[email protected]>

k8s: remove K8sClientWrapper abstraction

2209139

Signed-off-by: tesla59 <[email protected]>

recommend: initialize dockerClient and generate policies for containe…

c759cd7

…rs as well Signed-off-by: tesla59 <[email protected]>

tesla59 marked this pull request as ready for review September 29, 2024 10:00

recommend: add new flag k8s to specify which client to use

dd2788a

this removes dependency of recommend command on kubearmor Signed-off-by: tesla59 <[email protected]>

tesla59 added 2 commits October 14, 2024 10:30

recommend: fallback to docker client if k8s client is not present

d52989f

Signed-off-by: tesla59 <[email protected]>

recommend: only log the client if images is not specified

f7e473f

Signed-off-by: tesla59 <[email protected]>

rootxrishabh reviewed Nov 11, 2024

View reviewed changes

tesla59 added 2 commits November 19, 2024 05:28

recommend: generate policyDir based on image namespace set to null

e98124d

Signed-off-by: tesla59 <[email protected]>

recommend: trim \n in final report generation

e5d621a

Signed-off-by: tesla59 <[email protected]>

rootxrishabh requested changes Jan 2, 2025

View reviewed changes

tesla59 added 2 commits January 8, 2025 08:48

Merge remote-tracking branch 'origin/main' into tesla/non-k8s/karmor-…

b720655

…recommend

recommend: handle err when listing objects

541d5de

Signed-off-by: tesla59 <[email protected]>

rootxrishabh reviewed Jan 15, 2025

View reviewed changes

rootxrishabh requested changes Jan 15, 2025

View reviewed changes

Merge remote-tracking branch 'origin/main' into tesla/non-k8s/karmor-…

25c981c

…recommend

rootxrishabh reviewed Jan 29, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: (recommend) Implement recommend functionality for Docker Client #461

feat: (recommend) Implement recommend functionality for Docker Client #461

tesla59 commented Sep 17, 2024 •

edited

Loading

tesla59 commented Oct 2, 2024

daemon1024 commented Oct 3, 2024

tesla59 commented Oct 3, 2024

rootxrishabh left a comment

rootxrishabh commented Nov 11, 2024 •

edited

Loading

tesla59 commented Nov 13, 2024

tesla59 commented Nov 19, 2024

tesla59 commented Nov 19, 2024

rootxrishabh left a comment

rootxrishabh commented Jan 2, 2025

tesla59 commented Jan 11, 2025

rootxrishabh left a comment •

edited

Loading

rootxrishabh left a comment

tesla59 commented Jan 19, 2025

tesla59 commented Jan 19, 2025

rootxrishabh left a comment

feat: (recommend) Implement recommend functionality for Docker Client #461

Are you sure you want to change the base?

feat: (recommend) Implement recommend functionality for Docker Client #461

Conversation

tesla59 commented Sep 17, 2024 • edited Loading

tesla59 commented Oct 2, 2024

daemon1024 commented Oct 3, 2024

tesla59 commented Oct 3, 2024

rootxrishabh left a comment

Choose a reason for hiding this comment

rootxrishabh commented Nov 11, 2024 • edited Loading

tesla59 commented Nov 13, 2024

tesla59 commented Nov 19, 2024

tesla59 commented Nov 19, 2024

rootxrishabh left a comment

Choose a reason for hiding this comment

rootxrishabh commented Jan 2, 2025

tesla59 commented Jan 11, 2025

rootxrishabh left a comment • edited Loading

Choose a reason for hiding this comment

rootxrishabh left a comment

Choose a reason for hiding this comment

tesla59 commented Jan 19, 2025

tesla59 commented Jan 19, 2025

rootxrishabh left a comment

Choose a reason for hiding this comment

tesla59 commented Sep 17, 2024 •

edited

Loading

rootxrishabh commented Nov 11, 2024 •

edited

Loading

rootxrishabh left a comment •

edited

Loading