SD RAN 5G Security Demo
curl -sfL http://get.kubearmor.io/ | sudo sh -s -- -b /usr/local/bin
karmor install
kubectl apply -f https://raw.githubusercontent.com/accuknox/discovery-engine/dev/deployments/k8s/deployment.yaml
git clone https://github.com/onosproject/sdran-in-a-box && cd sdran-in-a-box
sdran-in-a-box$ make oai
kubectl get pods -A
Output
NAMESPACE NAME READY STATUS RESTARTS AGE
accuknox-agents discovery-engine-6665599668-25dq5 1/1 Running 0 3d4h
kube-system atomix-controller-6989fbdbf-jxcpz 1/1 Running 0 3d4h
kube-system atomix-raft-storage-controller-746fbdb557-vjq9c 1/1 Running 0 3d4h
kube-system calico-kube-controllers-59466bfc9b-p4mxb 1/1 Running 0 6d5h
kube-system calico-node-gf8vq 1/1 Running 0 6d5h
kube-system coredns-bbb7d66cd-sff96 1/1 Running 0 6d5h
kube-system dns-autoscaler-6f895b87bc-ppbzn 1/1 Running 0 6d5h
kube-system kube-apiserver-node1 1/1 Running 0 6d5h
kube-system kube-controller-manager-node1 1/1 Running 0 6d5h
kube-system kube-multus-ds-amd64-g4sw5 1/1 Running 0 6d5h
kube-system kube-proxy-8srkp 1/1 Running 0 6d5h
kube-system kube-scheduler-node1 1/1 Running 0 6d5h
kube-system kubearmor-annotation-manager-5c9469c4b9-cms8n 2/2 Running 0 6d3h
kube-system kubearmor-host-policy-manager-f44dbc8b9-wlqkn 2/2 Running 0 6d3h
kube-system kubearmor-policy-manager-fdb77c666-ckts7 2/2 Running 0 6d3h
kube-system kubearmor-relay-645667c695-b7pm6 1/1 Running 0 6d3h
kube-system kubearmor-rj4g2 1/1 Running 0 6d3h
kube-system nodelocaldns-vz72k 1/1 Running 0 6d5h
kube-system onos-operator-app-78f8f6b998-wwsmt 1/1 Running 0 3d4h
kube-system onos-operator-topo-68c49f7d9-j5bnz 1/1 Running 0 3d4h
riab oai-enb-cu-0 1/1 Running 0 3d4h
riab oai-enb-du-0 1/1 Running 0 3d4h
riab oai-ue-0 1/1 Running 0 3d4h
riab onos-a1t-77954946fc-gjhhc 2/2 Running 0 3d4h
riab onos-cli-777458fb59-7cx7m 1/1 Running 0 3d4h
riab onos-config-5cf5d77449-9wvd9 4/4 Running 0 3d4h
riab onos-consensus-store-0 1/1 Running 0 3d4h
riab onos-e2t-7c99fd6544-lbmfs 3/3 Running 0 3d4h
riab onos-kpimon-59466bf6d-jkv2j 2/2 Running 0 3d4h
riab onos-pci-7fd57b67d9-jrftv 2/2 Running 0 3d4h
riab onos-topo-56df7985d6-bd2sc 3/3 Running 0 3d4h
riab onos-uenib-6c8c644f54-w5jkt 3/3 Running 0 3d4h
riab ran-simulator-7875695894-bfdzd 1/1 Running 0 3d4h
Now we can see that all the pods in riab namespace are discovered and armored up by the KubeArmor and initially no policy has been applied to any of the pod.
karmor probe
Output
Armored Up pods :
+-----------------+-------------------------------------------------+---------------------------+
| NAMESPACE | NAME | POLICY |
+-----------------+-------------------------------------------------+---------------------------+
| accuknox-agents | discovery-engine-6665599668-25dq5 | |
+-----------------+-------------------------------------------------+---------------------------+
| kube-system | atomix-controller-6989fbdbf-jxcpz | |
+ +-------------------------------------------------+---------------------------+
| | atomix-raft-storage-controller-746fbdb557-vjq9c | |
+ +-------------------------------------------------+---------------------------+
| | onos-operator-app-78f8f6b998-wwsmt | |
+ +-------------------------------------------------+---------------------------+
| | onos-operator-topo-68c49f7d9-j5bnz | |
+-----------------+-------------------------------------------------+---------------------------+
| riab | oai-enb-cu-0 | |
+ +-------------------------------------------------+---------------------------+
| | oai-enb-du-0 | |
+ +-------------------------------------------------+---------------------------+
| | oai-ue-0 | |
+ +-------------------------------------------------+---------------------------+
| | onos-a1t-77954946fc-gjhhc | |
+ +-------------------------------------------------+---------------------------+
| | onos-cli-777458fb59-7cx7m | |
+ +-------------------------------------------------+---------------------------+
| | onos-config-5cf5d77449-9wvd9 | |
+ +-------------------------------------------------+---------------------------+
| | onos-consensus-store-0 | |
+ +-------------------------------------------------+---------------------------+
| | onos-e2t-7c99fd6544-lbmfs | |
+ +-------------------------------------------------+---------------------------+
| | onos-kpimon-59466bf6d-jkv2j | |
+ +-------------------------------------------------+---------------------------+
| | onos-pci-7fd57b67d9-jrftv | |
+ +-------------------------------------------------+---------------------------+
| | onos-topo-56df7985d6-bd2sc | |
+ +-------------------------------------------------+---------------------------+
| | onos-uenib-6c8c644f54-w5jkt | |
+ +-------------------------------------------------+---------------------------+
| | ran-simulator-7875695894-bfdzd | |
+-----------------+-------------------------------------------------+---------------------------+
It will show the Observed behavior of the workloads in the riab namespace at the pod-level.
karmor summary -n riab
Sample Output
Pod Name onos-kpimon-59466bf6d-hlxz2
Namespace Name riab
Cluster Name default
Container Name onos-kpimon
Labels app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-kpimon,name=onos-kpimon,resource=onos-kpimon,type=kpimon
Ingress connections
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 524 | Tue Jan 3 06:14:59 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 567 | Tue Jan 3 07:02:17 UTC 2023 |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
Egress connections
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app=onos-topo,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,heritage=Helm | 1 | Tue Jan 3 05:27:35 UTC 2023 |
| TCP | /usr/local/bin/onos-proxy | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-topo,heritage=Helm,release=sd-ran,app=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,app.kubernetes.io/managed-by=Helm | 1 | Tue Jan 3 05:27:36 UTC 2023 |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
Pod Name onos-topo-56df7985d6-mzzz9
Namespace Name riab
Cluster Name default
Container Name onos-topo
Labels app.kubernetes.io/name=onos-topo,name=onos-topo,resource=onos-topo,type=topo,app=onos,app.kubernetes.io/instance=sd-ran
Ingress connections
+----------+--------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+--------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-topo | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 527 | Tue Jan 3 06:15:03 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-topo | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 566 | Tue Jan 3 07:02:13 UTC 2023 |
+----------+--------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
Pod Name oai-ue-0
Namespace Name riab
Cluster Name default
Container Name oai-ue
Labels
File Data
+---------------------------------+----------------------------------+-------+------------------------------+--------+
| SRC PROCESS | DESTINATION FILE PATH | COUNT | LAST UPDATED TIME | STATUS |
+---------------------------------+----------------------------------+-------+------------------------------+--------+
| /opt/oai-ue/bin/lte-uesoftmodem | /dev/net/tun | 10 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /etc/host.conf | 1 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /etc/hosts | 2 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /etc/ld.so.cache | 1 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /etc/nsswitch.conf | 1 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /etc/resolv.conf | 1 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai-ue/bin/lte-uesoftmodem | /opt/oai-ue/share/.ue_emm.nvram0 | 1 | Thu Jan 1 00:00:00 UTC 1970 |
+---------------------------------+----------------------------------+-------+------------------------------+--------+
Pod Name oai-enb-du-0
Namespace Name riab
Cluster Name default
Container Name oai-enb-du
Labels
File Data
+----------------------------+-----------------------------+-------+------------------------------+--------+
| SRC PROCESS | DESTINATION FILE PATH | COUNT | LAST UPDATED TIME | STATUS |
+----------------------------+-----------------------------+-------+------------------------------+--------+
| /opt/oai/bin/lte-softmodem | /etc/ld.so.cache | 1 | Thu Jan 1 00:00:00 UTC 1970 |
| /opt/oai/bin/lte-softmodem | /usr/local/lib/libcoding.so | 1 | Thu Jan 1 00:00:00 UTC 1970 |
+----------------------------+-----------------------------+-------+------------------------------+--------+
Pod Name onos-uenib-6c8c644f54-g28r2
Namespace Name riab
Cluster Name default
Container Name onos-uenib
Labels name=onos-uenib,resource=onos-uenib,type=uenib,app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-uenib
Ingress connections
+----------+---------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+---------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-uenib | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 526 | Tue Jan 3 06:15:05 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-uenib | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 566 | Tue Jan 3 07:02:15 UTC 2023 |
+----------+---------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
Pod Name onos-e2t-7c99fd6544-7l5dv
Namespace Name riab
Cluster Name default
Container Name onos-e2t
Labels name=onos-e2t,resource=onos-e2t,type=e2t,app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-e2t
Ingress connections
+----------+-------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+-------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-e2t | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 498 | Tue Jan 3 06:15:01 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-e2t | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 527 | Tue Jan 3 07:02:11 UTC 2023 |
+----------+-------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
karmor summary -n riab --container onos-kpimon --agg
Output
local port to be used for port forwarding discovery-engine-6665599668-wdzfp: 9089
Pod Name onos-kpimon-59466bf6d-hlxz2
Namespace Name riab
Cluster Name default
Container Name onos-kpimon
Labels app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-kpimon,name=onos-kpimon,resource=onos-kpimon,type=kpimon
Ingress connections
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 524 | Tue Jan 3 06:14:59 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 607 | Tue Jan 3 07:05:37 UTC 2023 |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------+-------+------------------------------+
Egress connections
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app=onos-topo,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,heritage=Helm | 1 | Tue Jan 3 05:27:35 UTC 2023 |
| TCP | /usr/local/bin/onos-proxy | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-topo,heritage=Helm,release=sd-ran,app=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,app.kubernetes.io/managed-by=Helm | 1 | Tue Jan 3 05:27:36 UTC 2023 |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
Check the monitoring results stored by the kpimon xApp.
make test-kpimon
Output
Helm values.yaml file: /home/azureuser/sdran-in-a-box//sdran-in-a-box-values-master-stable.yaml
HEAD is now at 9f79ab8 Fix the default SRIOV resource name for UPF user plane interfaces
HEAD is now at 29ffaaf update MHO chart to run with RC service model (#1134)
*** Get KPIMON result through CLI ***
Node ID Cell Object ID Cell Global ID Time RRC.Conn.Avg RRC.Conn.Max RRC.ConnEstabAtt.Sum RRC.ConnEstabAtt.sum RRC.ConnEstabSucc.Sum RRC.ConnEstabSucc.sum RRC.ConnMax RRC.ConnMean RRC.ConnReEstabAtt.HOFail RRC.ConnReEstabAtt.Other RRC.ConnReEstabAtt.Sum RRC.ConnReEstabAtt.reconfigFail RRC.ConnReEstabAtt.sum
e2:1/5153 13842601454c001 1454c001 07:08:25.0 3 5 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:1/5153 13842601454c002 1454c002 07:08:24.0 5 7 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:1/5153 13842601454c003 1454c003 07:08:24.0 0 2 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:1/5154 138426014550001 14550001 07:08:24.0 0 2 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:1/5154 138426014550002 14550002 07:08:24.0 0 2 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:1/5154 138426014550003 14550003 07:08:24.0 2 4 0 N/A 0 N/A N/A N/A 0 0 0 0 N/A
e2:4/e00/2/64 1 e0000 07:08:24.0 N/A N/A N/A 1 N/A 1 1 1 N/A N/A N/A N/A 0
kpimon xApp onos-kpimon collects KPIs reported by E2 nodes connected to onos-e2t. The kpimon xApplication works as:
-
It makes a subscription with E2 nodes connected to onos-e2t through
onos-toposubsystem -
After successful subscription it sets report interval and granularity period which are the monitoring interval parameters
-
Then each E2 node starts sending indication messages periodically to report KPIs to
onos-kpimon -
kpimon decodes each indication message that has KPI monitoring reports and store them to both KPIMON local store, or
onos-uenib -
A user can check the stored monitoring results through
onos-cli,$ onos kpimon list metrics
Observe the kpimon xApp behavior now and the entire execution flow described above can be seen in summarized observability data.
karmor summary -n riab --container onos-kpimon
Output
local port to be used for port forwarding discovery-engine-6665599668-wdzfp: 9089
Pod Name onos-kpimon-59466bf6d-hlxz2
Namespace Name riab
Cluster Name default
Container Name onos-kpimon
Labels app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-kpimon,name=onos-kpimon,resource=onos-kpimon,type=kpimon
Ingress connections
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 524 | Tue Jan 3 06:14:59 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 648 | Tue Jan 3 07:08:59 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-kpimon | pod/onos-cli-777458fb59-tt49k | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-cli,app=onos,name=onos-cli,resource=onos-cli,type=cli | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCPv6 | /usr/local/bin/onos-kpimon | pod/calico-kube-controllers-59466bfc9b-p4mxb | 5150 | kube-system | k8s-app=calico-kube-controllers | 19 | Tue Jan 3 07:10:37 UTC 2023 |
+----------+----------------------------+----------------------------------------------+------+-------------+---------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
Egress connections
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| PROTOCOL | COMMAND | POD/SVC/IP | PORT | NAMESPACE | LABELS | COUNT | LAST UPDATED TIME |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app=onos-topo,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,heritage=Helm | 1 | Tue Jan 3 05:27:35 UTC 2023 |
| TCP | /usr/local/bin/onos-proxy | svc/onos-topo | 5150 | riab | app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-topo,heritage=Helm,release=sd-ran,app=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,app.kubernetes.io/managed-by=Helm | 1 | Tue Jan 3 05:27:36 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=onos-topo,heritage=Helm | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | release=sd-ran,app=onos-topo,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,heritage=Helm,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5 | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app=onos-topo,app.kubernetes.io/instance=sd-ran,chart=onos-topo-1.3.4,heritage=Helm | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | app=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=onos-topo,helm.sh/chart=onos-topo-1.3.4,heritage=Helm,release=sd-ran | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,release=sd-ran,app=onos-topo,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,heritage=Helm,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | helm.sh/chart=onos-topo-1.3.4,heritage=Helm,release=sd-ran,app.kubernetes.io/managed-by=Helm,chart=onos-topo-1.3.4,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,app=onos-topo,app.kubernetes.io/instance=sd-ran | 1 | Tue Jan 3 07:08:25 UTC 2023 |
| TCP | /usr/local/bin/onos-kpimon | svc/onos-topo | 5150 | riab | heritage=Helm,release=sd-ran,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/managed-by=Helm,app.kubernetes.io/name=onos-topo,app.kubernetes.io/version=v0.9.5,chart=onos-topo-1.3.4,helm.sh/chart=onos-topo-1.3.4,app=onos-topo | 1 | Tue Jan 3 07:08:25 UTC 2023 |
+----------+----------------------------+---------------+------+-----------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+------------------------------+
karmor recommend -n riab
Sample Output
INFO[0000] pulling image image="onosproject/onos-a1t:v0.2.0"
v0.2.0: Pulling from onosproject/onos-a1t
Digest: sha256:f2c5ad803c69264c1d489eb61719c75c5559d2b3a47fd67a6c6ef81237d361d4
Status: Image is up to date for onosproject/onos-a1t:v0.2.0
INFO[0001] dumped image to tar tar=/tmp/karmor1664169442/wBfaXNaD.tar
Distribution alpine
INFO[0001] No runtime policy generated for riab/onos-a1t/onosproject/onos-a1t:v0.2.0
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-maintenance-tool-access.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-cert-access.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-system-owner-discovery.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-system-monitoring-deny-write-under-bin-directory.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-system-monitoring-write-under-dev-directory.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-system-monitoring-detect-access-to-cronjob-files.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-least-functionality-execute-package-management-process-in-container.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-deny-remote-file-copy.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-deny-write-in-shm-folder.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-deny-write-under-etc-directory.yaml ...
created policy out/riab-onos-a1t/onosproject-onos-a1t-v0-2-0-deny-write-under-etc-directory.yaml ...
INFO[0001] pulling image image="onosproject/onos-cli:v0.9.15"
v0.9.15: Pulling from onosproject/onos-cli
Digest: sha256:1d0419c951d8a3a8d487ff72e8b2614828f1abc13cffe203fdc11ac104a9f9fd
Status: Image is up to date for onosproject/onos-cli:v0.9.15
INFO[0002] dumped image to tar tar=/tmp/karmor4135142654/WVjevjDV.tar
Distribution alpine
INFO[0002] No runtime policy generated for riab/onos-cli/onosproject/onos-cli:v0.9.15
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-maintenance-tool-access.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-cert-access.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-system-owner-discovery.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-system-monitoring-deny-write-under-bin-directory.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-system-monitoring-write-under-dev-directory.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-system-monitoring-detect-access-to-cronjob-files.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-least-functionality-execute-package-management-process-in-container.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-deny-remote-file-copy.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-deny-write-in-shm-folder.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-deny-write-under-etc-directory.yaml ...
created policy out/riab-onos-cli/onosproject-onos-cli-v0-9-15-deny-write-under-etc-directory.yaml ...
Deployment | riab/onos-a1t
Container | onosproject/onos-a1t:v0.2.0
OS | linux
Arch | amd64
Distro | alpine
Output Directory | out/riab-onos-a1t
policy-template version | v0.1.6
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| POLICY | SHORT DESC | SEVERITY | ACTION | TAGS |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0- | Restrict access to maintenance | 1 | Block | PCI_DSS |
| maintenance-tool-access.yaml | tools (apk, mii-tool, ...) | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-cert- | Restrict access to trusted | 1 | Block | MITRE |
| access.yaml | certificated bundles in the OS | | | MITRE_T1552_unsecured_credentials |
| | image | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-system- | System Information Discovery | 3 | Block | MITRE |
| owner-discovery.yaml | - block system owner discovery | | | MITRE_T1082_system_information_discovery |
| | commands | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-system- | System and Information | 5 | Block | NIST NIST_800-53_AU-2 |
| monitoring-deny-write-under-bin- | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| directory.yaml | make directory under /bin/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-system- | System and Information | 5 | Audit | NIST NIST_800-53_AU-2 |
| monitoring-write-under-dev- | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| directory.yaml | make files under /dev/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-system- | System and Information | 5 | Audit | NIST SI-4 |
| monitoring-detect-access-to- | Integrity - System Monitoring | | | NIST_800-53_SI-4 |
| cronjob-files.yaml | Detect access to cronjob files | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-least- | System and Information | 5 | Block | NIST |
| functionality-execute-package- | Integrity - Least | | | NIST_800-53_CM-7(4) |
| management-process-in- | Functionality deny execution | | | SI-4 process |
| container.yaml | of package manager process in | | | NIST_800-53_SI-4 |
| | container | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-deny- | The adversary is trying to | 5 | Block | MITRE |
| remote-file-copy.yaml | steal data. | | | MITRE_TA0008_lateral_movement |
| | | | | MITRE_TA0010_exfiltration |
| | | | | MITRE_TA0006_credential_access |
| | | | | MITRE_T1552_unsecured_credentials |
| | | | | NIST_800-53_SI-4(18) NIST |
| | | | | NIST_800-53 NIST_800-53_SC-4 |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-deny- | The adversary is trying to | 5 | Block | MITRE_execution |
| write-in-shm-folder.yaml | write under shm folder | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-deny- | The adversary is trying to | 5 | Block | NIST_800-53_SI-7 NIST |
| write-under-etc-directory.yaml | avoid being detected. | | | NIST_800-53_SI-4 NIST_800-53 |
| | | | | MITRE_T1562.001_disable_or_modify_tools |
| | | | | MITRE_T1036.005_match_legitimate_name_or_location |
| | | | | MITRE_TA0003_persistence |
| | | | | MITRE MITRE_T1036_masquerading |
| | | | | MITRE_TA0005_defense_evasion |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-a1t-v0-2-0-deny- | Adversaries may delete or | 5 | Block | NIST NIST_800-53 NIST_800-53_CM-5 |
| write-under-etc-directory.yaml | modify artifacts generated | | | NIST_800-53_AU-6(8) |
| | within systems to remove | | | MITRE_T1070_indicator_removal_on_host |
| | evidence. | | | MITRE MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
Deployment | riab/onos-cli
Container | onosproject/onos-cli:v0.9.15
OS | linux
Arch | amd64
Distro | alpine
Output Directory | out/riab-onos-cli
policy-template version | v0.1.6
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| POLICY | SHORT DESC | SEVERITY | ACTION | TAGS |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15- | Restrict access to maintenance | 1 | Block | PCI_DSS |
| maintenance-tool-access.yaml | tools (apk, mii-tool, ...) | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-cert- | Restrict access to trusted | 1 | Block | MITRE |
| access.yaml | certificated bundles in the OS | | | MITRE_T1552_unsecured_credentials |
| | image | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15- | System Information Discovery | 3 | Block | MITRE |
| system-owner-discovery.yaml | - block system owner discovery | | | MITRE_T1082_system_information_discovery |
| | commands | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15- | System and Information | 5 | Block | NIST NIST_800-53_AU-2 |
| system-monitoring-deny-write-under- | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| bin-directory.yaml | make directory under /bin/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15- | System and Information | 5 | Audit | NIST NIST_800-53_AU-2 |
| system-monitoring-write-under-dev- | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| directory.yaml | make files under /dev/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15- | System and Information | 5 | Audit | NIST SI-4 |
| system-monitoring-detect-access-to- | Integrity - System Monitoring | | | NIST_800-53_SI-4 |
| cronjob-files.yaml | Detect access to cronjob files | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-least- | System and Information | 5 | Block | NIST |
| functionality-execute-package- | Integrity - Least | | | NIST_800-53_CM-7(4) |
| management-process-in- | Functionality deny execution | | | SI-4 process |
| container.yaml | of package manager process in | | | NIST_800-53_SI-4 |
| | container | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-deny- | The adversary is trying to | 5 | Block | MITRE |
| remote-file-copy.yaml | steal data. | | | MITRE_TA0008_lateral_movement |
| | | | | MITRE_TA0010_exfiltration |
| | | | | MITRE_TA0006_credential_access |
| | | | | MITRE_T1552_unsecured_credentials |
| | | | | NIST_800-53_SI-4(18) NIST |
| | | | | NIST_800-53 NIST_800-53_SC-4 |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-deny- | The adversary is trying to | 5 | Block | MITRE_execution |
| write-in-shm-folder.yaml | write under shm folder | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-deny- | The adversary is trying to | 5 | Block | NIST_800-53_SI-7 NIST |
| write-under-etc-directory.yaml | avoid being detected. | | | NIST_800-53_SI-4 NIST_800-53 |
| | | | | MITRE_T1562.001_disable_or_modify_tools |
| | | | | MITRE_T1036.005_match_legitimate_name_or_location |
| | | | | MITRE_TA0003_persistence |
| | | | | MITRE MITRE_T1036_masquerading |
| | | | | MITRE_TA0005_defense_evasion |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| onosproject-onos-cli-v0-9-15-deny- | Adversaries may delete or | 5 | Block | NIST NIST_800-53 NIST_800-53_CM-5 |
| write-under-etc-directory.yaml | modify artifacts generated | | | NIST_800-53_AU-6(8) |
| | within systems to remove | | | MITRE_T1070_indicator_removal_on_host |
| | evidence. | | | MITRE MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
It will show recommended policies for each workload in the riab namespace.
The recommended policies for a particular container workload can be generated by executing the command:
karmor recommend -n <namespace> --image <image-name>
i.e.
recommended policies for the onos-kpimon v0.4.4 can be generated as:
karmor recommend -n riab --image=onosproject/onos-kpimon:v0.4.4
Output
INFO[0000] pulling image image="onosproject/onos-kpimon:v0.4.4"
v0.4.4: Pulling from onosproject/onos-kpimon
Digest: sha256:f4c124559060b80babb68f381717e35e890b5ef740306660fb09567343111183
Status: Image is up to date for onosproject/onos-kpimon:v0.4.4
INFO[0001] dumped image to tar tar=/tmp/karmor4040791392/daTAsttT.tar
Distribution alpine
INFO[0001] No runtime policy generated for riab//onosproject/onos-kpimon:v0.4.4
created policy out/riab-onosproject-onos-kpimon-v0-4-4/maintenance-tool-access.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/cert-access.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/system-owner-discovery.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/system-monitoring-deny-write-under-bin-directory.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/system-monitoring-write-under-dev-directory.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/system-monitoring-detect-access-to-cronjob-files.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/least-functionality-execute-package-management-process-in-container.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/deny-remote-file-copy.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/deny-write-in-shm-folder.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/deny-write-under-etc-directory.yaml ...
created policy out/riab-onosproject-onos-kpimon-v0-4-4/deny-write-under-etc-directory.yaml ...
output report in out/report.txt ...
Container | onosproject/onos-kpimon:v0.4.4
OS | linux
Arch | amd64
Distro | alpine
Output Directory | out/riab-onosproject-onos-kpimon-v0-4-4
policy-template version | v0.1.6
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| POLICY | SHORT DESC | SEVERITY | ACTION | TAGS |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| maintenance-tool-access.yaml | Restrict access to maintenance | 1 | Block | PCI_DSS |
| | tools (apk, mii-tool, ...) | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| cert-access.yaml | Restrict access to trusted | 1 | Block | MITRE |
| | certificated bundles in the OS | | | MITRE_T1552_unsecured_credentials |
| | image | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-owner-discovery.yaml | System Information Discovery | 3 | Block | MITRE |
| | - block system owner discovery | | | MITRE_T1082_system_information_discovery |
| | commands | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-monitoring-deny-write-under- | System and Information | 5 | Block | NIST NIST_800-53_AU-2 |
| bin-directory.yaml | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| | make directory under /bin/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-monitoring-write-under-dev- | System and Information | 5 | Audit | NIST NIST_800-53_AU-2 |
| directory.yaml | Integrity - System Monitoring | | | NIST_800-53_SI-4 MITRE |
| | make files under /dev/ | | | MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| system-monitoring-detect-access-to- | System and Information | 5 | Audit | NIST SI-4 |
| cronjob-files.yaml | Integrity - System Monitoring | | | NIST_800-53_SI-4 |
| | Detect access to cronjob files | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| least-functionality-execute- | System and Information | 5 | Block | NIST |
| package-management-process-in- | Integrity - Least | | | NIST_800-53_CM-7(4) |
| container.yaml | Functionality deny execution | | | SI-4 process |
| | of package manager process in | | | NIST_800-53_SI-4 |
| | container | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| deny-remote-file-copy.yaml | The adversary is trying to | 5 | Block | MITRE |
| | steal data. | | | MITRE_TA0008_lateral_movement |
| | | | | MITRE_TA0010_exfiltration |
| | | | | MITRE_TA0006_credential_access |
| | | | | MITRE_T1552_unsecured_credentials |
| | | | | NIST_800-53_SI-4(18) NIST |
| | | | | NIST_800-53 NIST_800-53_SC-4 |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| deny-write-in-shm-folder.yaml | The adversary is trying to | 5 | Block | MITRE_execution |
| | write under shm folder | | | MITRE |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| deny-write-under-etc-directory.yaml | The adversary is trying to | 5 | Block | NIST_800-53_SI-7 NIST |
| | avoid being detected. | | | NIST_800-53_SI-4 NIST_800-53 |
| | | | | MITRE_T1562.001_disable_or_modify_tools |
| | | | | MITRE_T1036.005_match_legitimate_name_or_location |
| | | | | MITRE_TA0003_persistence |
| | | | | MITRE MITRE_T1036_masquerading |
| | | | | MITRE_TA0005_defense_evasion |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| deny-write-under-etc-directory.yaml | Adversaries may delete or | 5 | Block | NIST NIST_800-53 NIST_800-53_CM-5 |
| | modify artifacts generated | | | NIST_800-53_AU-6(8) |
| | within systems to remove | | | MITRE_T1070_indicator_removal_on_host |
| | evidence. | | | MITRE MITRE_T1036_masquerading |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
Let's take an example of the recommended policies
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
| least-functionality-execute- | System and Information | 5 | Block | NIST |
| package-management-process-in- | Integrity - Least | | | NIST_800-53_CM-7(4) |
| container.yaml | Functionality deny execution | | | SI-4 process |
| | of package manager process in | | | NIST_800-53_SI-4 |
| | container | | | |
+-------------------------------------+--------------------------------+----------+--------+---------------------------------------------------+
least-functionality-execute-package-management-process-in-container.yaml
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: onosproject-onos-kpimon-v0-4-4-least-functionality-execute-package-management-process-in-container
namespace: riab
spec:
action: Block
message: Alert! Execution of package management process inside container is denied
process:
matchPaths:
- path: /usr/bin/apt
- path: /usr/bin/apt-get
- path: /bin/apt-get
- path: /bin/apt
- path: /usr/bin/dpkg
- path: /bin/dpkg
- path: /usr/bin/gdebi
- path: /bin/gdebi
- path: /usr/bin/make
- path: /bin/make
- path: /usr/bin/yum
- path: /bin/yum
- path: /usr/bin/rpm
- path: /bin/rpm
- path: /usr/bin/dnf
- path: /bin/dnf
- path: /usr/bin/pacman
- path: /usr/sbin/pacman
- path: /bin/pacman
- path: /sbin/pacman
- path: /usr/bin/makepkg
- path: /usr/sbin/makepkg
- path: /bin/makepkg
- path: /sbin/makepkg
- path: /usr/bin/yaourt
- path: /usr/sbin/yaourt
- path: /bin/yaourt
- path: /sbin/yaourt
- path: /usr/bin/zypper
- path: /bin/zypper
selector:
matchLabels:
kubearmor.io/container.name: onosproject/onos-kpimon
severity: 5
tags:
- NIST
- NIST_800-53_CM-7(4)
- SI-4
- process
- NIST_800-53_SI-4This policy recommeds that no package manager should be allowed to run in the production environment, applying this policy will block the execution of the package managers during runtime.
Save auto-discovered KubeArmor security policies, we'll apply these policies later in upcoming section.
karmor discover -n riab -f yaml > ~/riab-discovered-policies.yaml
riab-discovered-policies.yaml
local port to be used for port forwarding discovery-engine-6665599668-wdzfp: 9089
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-2091361636
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /etc/apt/apt.conf.d/
recursive: true
- dir: /etc/apt/auth.conf.d/
recursive: true
- dir: /etc/apt/preferences.d/
recursive: true
- dir: /etc/apt/sources.list.d/
recursive: true
- dir: /opt/oai/share/
fromSource:
- path: /bin/sed
recursive: true
- dir: /tmp/
recursive: true
- dir: /usr/lib/x86_64-linux-gnu/
recursive: true
- dir: /var/lib/apt/lists/
recursive: true
- dir: /var/lib/apt/lists/partial/
recursive: true
- dir: /var/lib/dpkg/updates/
recursive: true
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /dev/null
- path: /dev/urandom
- path: /etc/apt/trusted.gpg.d
- path: /etc/apt/trusted.gpg
- fromSource:
- path: /usr/bin/dpkg
path: /etc/dpkg/dpkg.cfg.d/docker-apt-speedup
- fromSource:
- path: /usr/bin/dpkg
path: /etc/dpkg/dpkg.cfg
- path: /etc/group
- fromSource:
- path: /sbin/ip
path: /etc/iproute2/group
- fromSource:
- path: /sbin/ip
path: /etc/iproute2/rt_scopes
- path: /etc/ld.so.cache
- path: /etc/nsswitch.conf
- path: /etc/passwd
- fromSource:
- path: /bin/cp
path: /opt/oai/conf_files/cu.conf
- fromSource:
- path: /bin/cp
path: /opt/oai/share/cu.conf
- path: /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
- path: /usr/lib/x86_64-linux-gnu/libapt-private.so.0.0
- path: /usr/lib/x86_64-linux-gnu/liblz4.so.1
- path: /usr/lib/x86_64-linux-gnu/libnettle.so.6
- path: /var/cache/apt/archives/partial
- path: /var/lib/apt/extended_states
- fromSource:
- path: /usr/lib/apt/methods/gpgv
path: /var/lib/apt/lists/archive.ubuntu.com_ubuntu_dists_xenial_InRelease
- fromSource:
- path: /usr/lib/apt/methods/gpgv
path: /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_InRelease
- fromSource:
- path: /usr/lib/apt/methods/store
path: /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_main_binary-amd64_Packages.lz4.WnTGbV
- fromSource:
- path: /usr/lib/apt/methods/store
path: /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_universe_binary-amd64_Packages.gz
- fromSource:
- path: /usr/lib/apt/methods/store
path: /var/lib/apt/lists/partial/archive.ubuntu.com_ubuntu_dists_xenial-updates_universe_binary-amd64_Packages.lz4.oa5TTA
- fromSource:
- path: /usr/lib/apt/methods/gpgv
path: /var/lib/apt/lists/partial/security.ubuntu.com_ubuntu_dists_xenial-security_InRelease
- path: /var/lib/dpkg/lock-frontend
- path: /var/lib/dpkg/lock
- path: /var/lib/dpkg/status
process:
matchDirectories:
- dir: /bin/
fromSource:
- path: /bin/dash
recursive: true
- dir: /usr/bin/
fromSource:
- path: /bin/bash
- path: /bin/dash
recursive: true
matchPaths:
- fromSource:
- path: /bin/bash
path: /bin/cp
- fromSource:
- path: /bin/bash
path: /bin/grep
- fromSource:
- path: /bin/bash
path: /bin/sed
- path: /opt/oai/run_enb_cu.sh
- fromSource:
- path: /bin/bash
path: /sbin/ip
- fromSource:
- path: /usr/lib/apt/methods/gpgv
path: /usr/bin/apt-key
- fromSource:
- path: /usr/bin/apt
- path: /usr/bin/apt-config
path: /usr/bin/dpkg
- fromSource:
- path: /usr/bin/apt
path: /usr/lib/apt/methods/gpgv
- fromSource:
- path: /usr/bin/apt
path: /usr/lib/apt/methods/store
- path: /usr/bin/apt
- path: /usr/bin/apt-config
- path: /bin/mktemp
- path: /usr/bin/find
- path: /bin/cat
- path: /bin/chmod
- path: /bin/dash
- path: /bin/readlink
- path: /bin/rm
- path: /opt/oai/bin/lte-softmodem
- path: /usr/bin/awk
- path: /usr/bin/cut
- path: /usr/bin/gpgv
- path: /usr/bin/head
- path: /usr/bin/sort
- path: /usr/bin/touch
- path: /usr/lib/apt/methods/http
selector:
matchLabels:
app: oai-enb-cu
release: oai-enb-cu
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-882839879
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /tmp/
recursive: true
- dir: /var/lib/atomix/data/onos-consensus-store-0/00000000000000000001/
recursive: true
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /etc/group
- path: /etc/passwd
- path: /var/lib/atomix/data/dragonboat.ds
process:
matchPaths:
- path: /bin/stat
- path: /usr/local/bin/atomix-raft-storage-node
selector:
matchLabels:
app: atomix
cluster: onos-consensus-store
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-1955519470
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: ran-simulator
name: ran-simulator
resource: ran-simulator
type: sim
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-252935527
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-config
name: onos-config
resource: onos-config
type: config
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-3592523051
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /dev/net/tun
process:
matchPaths:
- path: /opt/oai/bin/lte-softmodem
selector:
matchLabels:
app: oai-enb-cu
release: oai-enb-cu
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-3064993133
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /etc/hosts
- path: /etc/passwd
- path: /etc/resolv.conf
process:
matchPaths:
- path: /usr/local/bin/onos
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-cli
name: onos-cli
resource: onos-cli
type: cli
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-2804513539
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /etc/
recursive: true
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /dev/net/tun
- path: /opt/oai-ue/share/.ue_emm.nvram0
process:
matchPaths:
- path: /opt/oai-ue/bin/lte-uesoftmodem
selector:
matchLabels:
app: oai-ue
release: oai-ue
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-3851899263
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-kpimon
name: onos-kpimon
resource: onos-kpimon
type: kpimon
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-883893652
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /
recursive: true
- dir: /var/lib/atomix/data/onos-consensus-store-0/00000000000000000001/logdb-1/
recursive: true
- dir: /var/lib/atomix/data/onos-consensus-store-0/00000000000000000001/snapshot-part-1/snapshot-1-1/
recursive: true
- dir: /lib/x86_64-linux-gnu/
recursive: true
process:
matchPaths:
- path: /bin/stat
- path: /bin/busybox
- path: /usr/local/bin/atomix-raft-storage-node
selector:
matchLabels:
app: atomix
cluster: onos-consensus-store
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-1959068045
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /etc/onos/certs/tls.cacrt
- path: /etc/onos/certs/tls.crt
process:
matchPaths:
- path: /usr/local/bin/onos-a1t
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-a1t
name: onos-a1t
resource: onos-a1t
type: a1t
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-2354148317
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-kpimon
name: onos-kpimon
resource: onos-kpimon
type: kpimon
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-2245614591
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
selector:
matchLabels:
app: onos
app.kubernetes.io/instance: sd-ran
app.kubernetes.io/name: onos-a1t
name: onos-a1t
resource: onos-a1t
type: a1t
severity: 1
---
apiVersion: security.kubearmor.com/v1
kind: KubeArmorPolicy
metadata:
name: autopol-system-1761718882
namespace: riab
spec:
action: Allow
file:
matchDirectories:
- dir: /lib/x86_64-linux-gnu/
recursive: true
matchPaths:
- path: /etc/ld.so.cache
- path: /usr/local/lib/libcoding.so
process:
matchPaths:
- path: /opt/oai/bin/lte-softmodem
selector:
matchLabels:
app: oai-enb-du
release: oai-enb-du
severity: 1
---karmor discover -n riab --network -f yaml
karmor discover -n riab -p NetworkPolicy -f yaml
karmor apply -f ~/riab-discovered-policies.yaml
karmor log --namespace riab --operation File --container onos-topo
The Alert generated by KubeArmor, in case of policy violation (next section) will appear here.
-
The auto-discovered policy for the
onos-topowill be similar to:apiVersion: security.kubearmor.com/v1 kind: KubeArmorPolicy metadata: name: autopol-system-1100549701 namespace: riab spec: action: Allow file: matchDirectories: - dir: /lib/x86_64-linux-gnu/ recursive: true matchPaths: - path: /etc/onos/certs/tls.cacrt - path: /etc/onos/certs/tls.crt - path: /etc/onos/certs/tls.key process: matchPaths: - path: /usr/local/bin/onos-topo selector: matchLabels: name: onos-topo severity: 1
-
Let's try to access
/etc/onos/certs/tls.keyby exec into the podonos-topoPOD_NAME=$(kubectl get pods -n riab -l "app.kubernetes.io/name=onos-topo" -o jsonpath='{.items[0].metadata.name}') && kubectl -n riab exec -it $POD_NAME -- sh -c "cat /etc/onos/certs/tls.key" -
As per the applied policy the binary
/bin/catis not allowed at runtime, and as per thedefault postureconfiguration (by dafault it'sAudit) for container workloads in KubeArmor it will be either audited or blocked. -
We'll see that
/etc/onos/certs/tls.keyfile can be accessed by the/bin/catas current default posture isAuditbut KubeArmor will generate anAlertfor this access because as per the security policy this access is not allowed.== Alert / 2023-01-02 04:17:11.373290 == ClusterName: default HostName: sd-ran-vm NamespaceName: riab PodName: onos-topo-56df7985d6-bd2sc Labels: app=onos,app.kubernetes.io/instance=sd-ran,app.kubernetes.io/name=onos-topo,name=onos-topo,resource=onos-topo,type=topo ContainerName: onos-topo ContainerID: e7d15a69329ed212cd223ccc1fb20ae5c042575862b5421a296f46afe475c1cb ContainerImage: docker.io/onosproject/onos-topo:v0.9.5@sha256:a0993017b0e5a8143e9a1a3b047e07c0069bc4a17e783c3d25a0433ab77b814f Type: MatchedPolicy PolicyName: DefaultPosture Source: /bin/cat /etc/onos/certs/tls.key Resource: /etc/onos/certs/tls.key Operation: File Action: Audit Data: syscall=SYS_OPEN flags=O_RDONLY Enforcer: eBPF Monitor Result: Passed HostPID: 3037475 HostPPID: 3037480 PID: 81 PPID: 3037480 ProcessName: /bin/busybox UID: 65534
