Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DNS woes with Debian Bookworm #300

Closed
jsf9k opened this issue Jan 5, 2024 · 9 comments · Fixed by #242, cisagov/cool-assessment-terraform#233 or cisagov/openvpn-server-tf-module#96
Closed

DNS woes with Debian Bookworm #300

jsf9k opened this issue Jan 5, 2024 · 9 comments · Fixed by #242, cisagov/cool-assessment-terraform#233 or cisagov/openvpn-server-tf-module#96
Assignees
@jsf9k
Labels
bug This issue or pull request addresses broken functionality

Comments

@jsf9k
Copy link
Member

jsf9k commented Jan 5, 2024 •
edited
Loading

🐛 Summary

@mcdonnnj has reported DNS woes when using a Debian Bookworm base AMI. He says that DNS resolution requires the FQDN (e.g., kali1.envX.cool.cyber.dhs.gov); simply using the hostname (e.g., kali1) does not work.

The reason for the difference appears to be that Bookworm uses systemd-resolved whereas previous Debian releases used resolvconf.

To reproduce

  1. Build a new COOL AMI from an AMI based on Debian Bookworm.
  2. Deploy the new COOL AMI along with an old, Bullseye-based COOL AMI into a COOL environment.
  3. Observe that, in the instance based on the new COOL AMI, DNS fails to resolve without using the FQDN; but, DNS does resolve in the instance based on the old AMI without the FQDN.

Expected behavior

The DNS behavior should remain the same, regardless of whether systemd-resolved or resolvconf is used.
@jsf9k jsf9k added the bug This issue or pull request addresses broken functionality label Jan 5, 2024
@jsf9k jsf9k self-assigned this Jan 5, 2024
@mcdonnnj
Copy link
Member

mcdonnnj commented Jan 5, 2024

Please note I ran into issues while testing Debian Bookworm with the images used in the CyHy environment. I have not done any testing with Bookworm images used in a COOL assessment environment or similar.

@jsf9k
Copy link
Member Author

jsf9k commented Jan 10, 2024

I have identified the source of the problem. I'd prefer to modify the netplan configuration to ensure that the correct systemd-networkd configuration files are created, but it may be necessary to edit the latter directly after they are created.

@jsf9k
Copy link
Member Author

jsf9k commented Jan 10, 2024

See also canonical/cloud-init#4764.

jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Jan 10, 2024
@jsf9k
Add script to append required DHCP options to the Netplan config
4a5a9c0 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  This script takes care of the
modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Jan 10, 2024
@jsf9k
Add a cloud-init fragment to reapply the netplan config
5a9c907 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  There is a Python script that fixes
the DHCP4 options in the netplan configuration already generated by
cloud-init.  This cloud-init fragment simply reapplies the Netplan
configuration after the modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Jan 10, 2024
@jsf9k
Add cloud-init foo to append required DHCP options to the Netplan config
39a1e95 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
@jsf9k
Copy link
Member Author

jsf9k commented Jan 10, 2024

See this branch, which together with this branch demonstrates and fixes the problem. A Debian Desktop instance created via these branches has been deployed to the COOL staging environment env6 if anyone from @cisagov/team-ois wants to inspect it.

@mcdonnnj - Can you make similar changes in CyHy and verify that you can safely move to Bookworm now? If so then I'd like to do a mini-Kraken that includes #242.

@jsf9k
Copy link
Member Author

jsf9k commented Jan 10, 2024

I suspect this was already an issue for our Kali instances, but we didn't notice because they don't rely on getting search domains from DHCP; however, our Guacamole instance may rely on such functionality. We certainly rely on it in CyHy.

jsf9k added a commit that referenced this issue Jan 12, 2024
@jsf9k
Add a playbook to install prerequisites for fixing a broken Netplan c…
e315ebe 
…onfiguration

See these issues for more information:
- #300
- canonical/cloud-init#4764
@jsf9k jsf9k linked a pull request Jan 12, 2024 that will close this issue
Update base AMI to Debian Bookworm #242
Merged
9 tasks
jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Jan 12, 2024
@jsf9k
Add cloud-init foo to append required DHCP options to the Netplan config
d9475bc 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
@jsf9k jsf9k mentioned this issue Jan 12, 2024
Merged
7 tasks
@jsf9k jsf9k linked a pull request Jan 12, 2024 that will close this issue
Add fix for Debian Bookworm/cloud-init/Netplan/DHCP issue cisagov/cool-assessment-terraform#233
Merged
7 tasks
@jsf9k
Copy link
Member Author

jsf9k commented Jan 12, 2024

I suspect this was already an issue for our Kali instances, but we didn't notice because they don't rely on getting search domains from DHCP; however, our Guacamole instance may rely on such functionality. We certainly rely on it in CyHy.

This is not the case after all. Kali does not use Netplan.

jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Jan 12, 2024
@jsf9k
Add script to append required DHCP options to the Netplan config
9f5c827 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  This script takes care of the
modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Jan 12, 2024
@jsf9k
Add a cloud-init fragment to reapply the netplan config
48d0a8e 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  There is a Python script that fixes
the DHCP4 options in the netplan configuration already generated by
cloud-init.  This cloud-init fragment simply reapplies the Netplan
configuration after the modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Jan 12, 2024
@jsf9k
Add cloud-init foo to append required DHCP options to the Netplan config
1b6ff3a 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
@jsf9k jsf9k mentioned this issue Jan 12, 2024
Merged
7 tasks
@jsf9k jsf9k linked a pull request Jan 12, 2024 that will close this issue
Fix Bookworm DHCP search domain issue cisagov/openvpn-server-tf-module#96
Merged
7 tasks
This was referenced Jan 12, 2024
Merged
Open
Open
Open
@mcdonnnj
Copy link
Member

After copying the changes in #242 I verified functionality in my testing deployment of the CyHy environment. I confirmed that the cyhy-commander was able to push work and retrieve results. I also verified that I was able to access all of the instances as expected:

$ for host in bastion vulnscan1 portscan1 portscan2 portscan3 portscan4; do ssh "$host".mcdonnnj.cyhy "hostname; lsb_release -d"; done
bastion
Description:	Debian GNU/Linux 12 (bookworm)
vulnscan1
Description:	Debian GNU/Linux 12 (bookworm)
portscan1
Description:	Debian GNU/Linux 12 (bookworm)
portscan2
Description:	Debian GNU/Linux 12 (bookworm)
portscan3
Description:	Debian GNU/Linux 12 (bookworm)
portscan4
Description:	Debian GNU/Linux 12 (bookworm)
$ for host in bastion docker; do ssh "$host".mcdonnnj.bod "hostname; lsb_release -d"; done
bastion
Description:	Debian GNU/Linux 12 (bookworm)
docker
Description:	Debian GNU/Linux 12 (bookworm)

Please see cisagov/cyhy_amis#746.

jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Feb 16, 2024
@jsf9k
Add script to append required DHCP options to the Netplan config
447d8bf 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  This script takes care of the
modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Feb 16, 2024
@jsf9k
Add a cloud-init fragment to reapply the netplan config
3f061b4 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  There is a Python script that fixes
the DHCP4 options in the netplan configuration already generated by
cloud-init.  This cloud-init fragment simply reapplies the Netplan
configuration after the modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/openvpn-server-tf-module that referenced this issue Feb 16, 2024
@jsf9k
Add cloud-init foo to append required DHCP options to the Netplan config
33926c9 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
mcdonnnj pushed a commit that referenced this issue Mar 7, 2024
@jsf9k @mcdonnnj
Add a playbook to install prerequisites for fixing a broken Netplan c…
c9f6edf 
…onfiguration

See these issues for more information:
- #300
- canonical/cloud-init#4764
mcdonnnj pushed a commit that referenced this issue Mar 7, 2024
@jsf9k @mcdonnnj
Add a playbook to install prerequisites for fixing a broken Netplan c…
39e3d31 
…onfiguration

See these issues for more information:
- #300
- canonical/cloud-init#4764
@jsf9k jsf9k reopened this Mar 8, 2024
@jsf9k
Copy link
Member Author

jsf9k commented Mar 8, 2024

There are still some PRs around this issue that need to be merged before it can be closed.

jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Mar 12, 2024
@jsf9k
Add script to append required DHCP options to the Netplan config
dce1295 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  This script takes care of the
modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Mar 12, 2024
@jsf9k
Add a cloud-init fragment to reapply the netplan config
15c5320 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.  There is a Python script that fixes
the DHCP4 options in the netplan configuration already generated by
cloud-init.  This cloud-init fragment simply reapplies the Netplan
configuration after the modification.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
jsf9k added a commit to cisagov/cool-assessment-terraform that referenced this issue Mar 12, 2024
@jsf9k
Add cloud-init foo to append required DHCP options to the Netplan config
d8f8dff 
The Netplan config is created by cloud-init, but it needs to be
slightly modified and reapplied.

See these issues for more details:
- cisagov/skeleton-packer#300
- canonical/cloud-init#4764
This was linked to pull requests Mar 18, 2024
⚠️ CONFLICT! Lineage pull request for: skeleton cisagov/terraformer-packer#48
Merged
⚠️ CONFLICT! Lineage pull request for: skeleton cisagov/pca-gophish-composition-packer#107
Merged
This was unlinked from pull requests Mar 18, 2024
⚠️ CONFLICT! Lineage pull request for: skeleton cisagov/terraformer-packer#48
Merged
⚠️ CONFLICT! Lineage pull request for: skeleton cisagov/pca-gophish-composition-packer#107
Merged
@jsf9k
Copy link
Member Author

jsf9k commented Mar 20, 2024

There are still one more PR around this issue that needs to be merged before it can be closed.

@jsf9k jsf9k reopened this Mar 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jsf9k jsf9k

Labels
bug This issue or pull request addresses broken functionality
Projects
None yet
Milestone
No milestone
2 participants
@jsf9k @mcdonnnj