Errors using PREP_SYSTEM_FOR_DIETPI.sh on latest Raspberry Pi OS Lite

[ghost]

Creating a bug report/issue

Required Information

-DietPi version | v6.35.0 (MichaIng/beta)
-Distro version | Raspberry Pi OS Lite (Debian Linux 10.7)
-Kernel version | Linux raspberrypi 5.4.83-v7+ #1379 SMP Mon Dec 14 13:08:57 GMT 2020 armv7l GNU/Linux
-SBC model | Raspberry Pi 3 A Plus Rev. 1.0
-Power supply used | OnePlus PSU 5V 4A
-SDcard used | KingSton Canvas Select Plus 32GB

Additional Information (if applicable)

-Software title | PREP_SYSTEM_FOR_DIETPI.sh
-Was the software title installed freshly or updated/migrated? | Software title installed freshly on Raspberry Pi OS Lite
-Can this issue be replicated on a fresh installation of DietPi? | Issue replicable on a fresh installation of Raspberry Pi OS Lite

Steps to reproduce

1.Flash latest Raspberry Pi OS Lite using Raspberry Pi Imager on the MicroSD
2.Add to /boot an empty ssh file to enable ssh and a wpa_supplicant.conf file completed with your network details to enable WiFi
3.Turn on the Raspberry Pi and connect to it using SSH
4.Follow these DietPi Docs (https://dietpi.com/docs/hardware/#make-your-own-distribution) choosing the beta branch,Debian Bullseye and maintaining the WiFi packages and complete the setup with the required information
5.It will work for a while and it will stop with the first error that you can see on the image

6.The workaround here is to open a subshell and execute the command apt-get autoremove -y,when it ends exit the subshell and re-run the failed command
7.It will work for a while and it will stop with the second and third error that you can see on these images


8.Ignore them and it will complete the process and at the end shutdown the Raspberry Pi
9.Modify on /boot the file dietpi.txt with AUTO_SETUP_NET_WIFI_ENABLED=1 and AUTO_SETUP_NET_WIFI_COUNTRY_CODE=IT and the file dietpi-wifi.txt with aWIFI_SSID[0]='NETWORK_SSID' and aWIFI_KEY[0]='NETWORK_PASSWORD'
10.Turn on the Raspberry Pi and connect it to a monitor
11.It will stop with some errors,ignore them using a keyboard and pushing the return button and it will wait indefinitely generating DropBear keys

Expected behaviour

You can get to the login prompt and access it using SSH or a keyboard and a monitor

Actual behaviour

You can't get to the login prompt and you can't access it in every way

Extra details

I think there is some problem with the permissions that the second and third errors related to and because these permissions aren't set in a properly way they block the login prompt

[Joulinar]

Hi,

so basically you are trying to migrate a Buster image to a Bullseye system. As well you are switching away from 32bit Raspbian repository to plain Debian repository. Not sure if this going to work that way. But I guess @MichaIng knows it better than I'm.

Why are you not using the DietPi images for RPi? They are based on RPi OS. Means, you could skip these step completely.

[MichaIng]

Buster to Bullseye upgrade yes, but it will remain on the Raspbian repository.

I have never tried to do a Bullseye upgrade via DietPi-PREP on RPi. I'll see if I find time and the reason for the purge error. Kinda strange that an autoremove resolves it as auto-marked packages should not cause any dependency resolver issues, but I've seen similar strange things with APT, and now a distro upgrade comes on top.

/etc/securetty is part of the essential login package, better it was. Indeed it has been removed with Bullseye. Good find! I'll add altering it conditionally based on distro version.

[MichaIng]

Fixed with: 93bd0c0
Changelog: 6e82b58

[MichaIng]

So however, the missing /etc/securetty is hence expected and does not effect the system behaviour.

It will stop with some errors,ignore them using a keyboard and pushing the return button and it will wait indefinitely generating DropBear keys

Errors with interactive prompt that reacts to keyboard input is shown at this pre-login boot stage are critical. Please paste them here, if possible. Dropbear keys generation might take a little while, although on RPi 4 not more than a few seconds. When it hangs, what comes to my mind is lack of entropy. But that's just guessing unless we know the error messages.

[ghost]

I'm creating a custom image of DietPi instead of using the prebuilt ones because i want to build personalized images for my SBCs in the future;now i'll redo the entire process and i'll paste the errors that i'll find here.
I've thought,maybe by installing the package haveged it shouldn't hang anymore during the DropBear keys generation?

[Joulinar]

Usually using PREP script will end up with the image you can download

[MichaIng]

I've thought,maybe by installing the package haveged it shouldn't hang anymore during the DropBear keys generation?

On RPi, a hardware random generator is present and hence rng-tools installed, which generates/gathers entropy with less overhead/RAM/CPU compared to haveged. But if that failed for some reason, yes that might be the issue.

@Joulinar
You have an RPi 1 right? Could you try to replace rng-tools with rng-tools5 there and see whether normal operation and boot is working without delay or hangs? It should but I could only verify until RPi2 so far. rng-tools5 is the successor but has not yet been implemented as full replacement since some old processors seem to work support if fully, so we need to verify for all older devices before we can pre-install it.
Maybe with very now boards, rng-tools starts to not work great anymore, so that this becomes even necessary.

[ghost]

I know that using this script you get an image identical to the prebuilt one provided by you but there are some SBCs that don't have prebuilt images:this is a test done on the Raspberry Pi with Raspberry Pi OS Lite so that i can use the same process for other less popular and not supported SBCs using this time a Debian Linux image built by me and this script to get a suitable DietPi image.

[ghost]

I've redone the entire process in the same way as before and the first 3 errors appear again,but the errors at boot don't appear anymore,now it hangs indefinitely and the last message is DropBear generating the 256 bit ED25519 key;this is all i can report for now.

[MichaIng]

When you reach that point, can you try to hit some random keyboard keys? This produces entropy, so can be a test to check whether this is the issue. At least on a VM the first boot with Dropbear generating the host keys work well, so it's not an issue with the current Dropbear version (Raspbian uses the exact same version) and also on my RPi with Debian (not Raspbian) Bullseye it works fine.

The /etc/securetty issue has been fixed with the dev branch of the script. For the other two I'd need to test on RPi, what I should be able to do tomorrow. An x86_64 image with Buster => Bullseye upgrade btw works fine, so it's a Raspbian pre-image issue.

[ghost]

I've done that before for the entropy and a new message appeared in the mean time saying Cleaning of temporary directories,then i've pressed random keyboard keys for some minutes but nothing,tomorrow i'll try using the dev branch of the script and i will update this issue with new information.

[MichaIng]

The dev branch solves the /etc/securetty bug but note that this has zero effect on your boot issue as the result is exactly the same: No change done to /etc/securetty, as it does not exist 😉.

Cleaning of temporary directories

That is the systemd-tmpfiles service:

2021-03-16 20:16:36 root@micha:/mnt/sda# systemctl cat systemd-tmpfiles-clean.service
# /lib/systemd/system/systemd-tmpfiles-clean.service
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)
DefaultDependencies=no
Conflicts=shutdown.target
After=local-fs.target time-set.target
Before=shutdown.target

[Service]
Type=oneshot
ExecStart=systemd-tmpfiles --clean
SuccessExitStatus=DATAERR
IOSchedulingClass=idle

That is not part of the boot process but run by a systemd timer independently:

2021-03-16 20:16:47 root@micha:/mnt/sda# systemctl cat systemd-tmpfiles-clean.timer
# /lib/systemd/system/systemd-tmpfiles-clean.timer
#  SPDX-License-Identifier: LGPL-2.1-or-later
#
#  This file is part of systemd.
#
#  systemd is free software; you can redistribute it and/or modify it
#  under the terms of the GNU Lesser General Public License as published by
#  the Free Software Foundation; either version 2.1 of the License, or
#  (at your option) any later version.

[Unit]
Description=Daily Cleanup of Temporary Directories
Documentation=man:tmpfiles.d(5) man:systemd-tmpfiles(8)

[Timer]
OnBootSec=15min
OnUnitActiveSec=1d

So it runs 15 minutes after boot and daily from then on.

[Joulinar]

@Joulinar
You have an RPi 1 right? Could you try to replace rng-tools with rng-tools5 there and see whether normal operation and boot is working without delay or hangs?

Correct 😄

@MichaIng
anything special to test?

root@DietPi1:~# dpkg -l rng-tools*
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=================================
un  rng-tools      <none>       <none>       (no description available)
ii  rng-tools5     5-4          armhf        Daemon to use a Hardware TRNG
root@DietPi1:~#

Boot process:

root@DietPi1:~# systemd-analyze blame
          8.211s dietpi-boot.service
          7.896s dev-mmcblk0p2.device
          5.966s [email protected]
          3.408s systemd-udev-trigger.service
          3.368s keyboard-setup.service
          2.422s systemd-journald.service
          1.930s systemd-fsck@dev-disk-by\x2dpartuuid-907af7d0\x2d01.service
          1.422s systemd-fsck-root.service
          1.379s networking.service
          1.148s systemd-remount-fs.service
           866ms dietpi-preboot.service
           768ms systemd-sysctl.service
           731ms dev-mqueue.mount
           729ms fake-hwclock.service
           680ms systemd-modules-load.service
           616ms systemd-tmpfiles-setup.service
           560ms sys-kernel-debug.mount
           480ms systemd-sysusers.service
           430ms systemd-random-seed.service
           398ms systemd-journal-flush.service
           382ms systemd-udevd.service
           331ms dietpi-ramlog.service
           322ms dropbear.service
           310ms sys-kernel-config.mount
           259ms kmod-static-nodes.service
           240ms tmp.mount
           227ms systemd-update-utmp.service
           204ms console-setup.service
           202ms systemd-tmpfiles-setup-dev.service
           201ms systemd-update-utmp-runlevel.service
           189ms ifupdown-pre.service
           189ms boot.mount
           183ms var-swap.swap
           167ms var-log.mount
           133ms systemd-user-sessions.service
root@DietPi1:~#

and some nice graph
bootup.zip

[MichaIng]

Looks good. One other check: dmesg | grep random

---

Ah finally, on Bullseye rng-tools has become a transitional dummy package for rng-tools-debian, so at least one package less in that jungle. rng-tools5 again is recommended there, so makes sense to migrate.

[Joulinar]

root@DietPi1:~# dmesg | grep random
[    0.000000] random: get_random_bytes called from start_kernel+0x324/0x564 with crng_init=0
[    2.461510] random: fast init done
[    6.422057] random: systemd: uninitialized urandom read (16 bytes read)
[    6.659128] random: systemd: uninitialized urandom read (16 bytes read)
[    6.662265] random: systemd: uninitialized urandom read (16 bytes read)
[   15.165006] random: crng init done
[   15.165044] random: 7 urandom warning(s) missed due to ratelimiting
root@DietPi1:~#

[ghost]

What should i do then to help solve this issue?I can help using my Raspberry Pi 3 A Plus but as i see it seems pretty useless to redo the same process again using the dev branch,isn't it?I've tried installing the package rng-tools5 before restarting the Raspberry Pi at the end of the script execution but i can't install it for the modification done to the system by the script itself.

[MichaIng]

What do you mean you can't install it?

apt purge rng-tools
apt update
apt install rng-tools5

should work.

I'm currently not at home but will definitely do the outstanding test and fix for the purge attempt in the script at the beginning of next week.

[ghost]

Hi,sorry for my late answer but i couldn't answer in time,i mean that after the script ends and it prompts you to save the image of the system and/or reboot i try to install rng-tools5 using your same commands but apt doesn't work for the modifications done by the script,so i can't install it.

[MichaIng]

but apt doesn't work for the modifications done by the script,so i can't install it.

Any error message? The script does nothing that could break APT, but it clears the cache and removes the list files, hence apt update is required before any install can be done which should also be shown when using "apt install ..." 🤔.

[MichaIng]

I mark this as closed. Feel free to reopen if issue persists.