How to disable `apt-daily.service` on Ubuntu cloud VM image?
Yes, there was something obvious that I was missing.
Systemd is all about concurrent start of services, so the
cloud-init script is
run at the same time the
apt-daily.service is triggered. By the time
cloud-init gets to execute the user-specified payload,
apt-get update is
already running. So the attempts 2. and 3. failed not because of some namespace
magic, but because they altered the system too late for
pick the changes up.
This also means that there is basically no way of preventing
apt.systemd.daily from running -- one can only kill it after it's started.
This "user data" script takes this route::
#!/bin/bash systemctl stop apt-daily.service systemctl kill --kill-who=all apt-daily.service # wait until `apt-get updated` has been killed while ! (systemctl list-units --all apt-daily.service | egrep -q '(dead|failed)') do sleep 1; done # now proceed with own APT tasks apt install -y python
There is still a time window during which SSH logins are possible yet
will not run, but I cannot imagine another solution that can works on the stock
Ubuntu 16.04 cloud image.
Note: Unfortunately part of the solution below doesn't work on Ubuntu 16.04 systems (such as that of the questioner) because the suggested
systemd-run invocation only works on Ubuntu 18.04 and above (see the comments for details). I'll leave the answer here because this question is still a popular hit regardless of which Ubuntu version you are using...
On Ubuntu 18.04 (and up) there may be up to two services involved in boot time apt updating/upgrading. The first
apt-daily.service refreshes the list of packages. However there can be a second
apt-daily-upgrade.service which actually installs security critical packages. An answer to the "Terminate and disable/remove unattended upgrade before command returns" question gives an excellent example of how to wait for both of these to finish (copied here for convenience):
systemd-run --property="After=apt-daily.service apt-daily-upgrade.service" --wait /bin/true
(note this has to be run as root). If you are trying to disable these services on future boots you will need to mask BOTH services:
systemctl mask apt-daily.service apt-daily-upgrade.service
Alternatively you can
systemctl disable both services AND their associated timers (i.e.
Note the masking/disabling techniques in this answer only prevent the update/upgrade on future boots - they won't stop them if they are already running in the current boot.
You can disable this via the "bootcmd" cloud-init module. This runs before network is brought up, which is required before apt update can get a chance to run.
#cloud-config bootcmd: - echo 'APT::Periodic::Enable "0";' > /etc/apt/apt.conf.d/10cloudinit-disable - apt-get -y purge update-notifier-common ubuntu-release-upgrader-core landscape-common unattended-upgrades - echo "Removed APT and Ubuntu 18.04 garbage early" | systemd-cat
Once you ssh into the instance, you should also wait for the final phases of cloud-init to finish, since it moves apt sources / lists around.
# Wait for cloud-init to finish moving apt sources.list around... # a good source of random failures # Note this is NOT a replacement for also disabling apt updates via bootcmd while [ ! -f /var/lib/cloud/instance/boot-finished ]; do echo 'Waiting for cloud-init to finish...' sleep 3 done
This is also helpful to see how early the bootcmd runs:
# Show microseconds in systemd journal journalctl -r -o short-precise
You can verify this worked as follows:
apt-config dump | grep Periodic # Verify nothing was updated until we run apt update ourselves. cd /var/lib/apt/lists sudo du -sh . # small size ls -ltr # old timestamps