How to customize cloud-init autoinstall (user-data) Ubuntu 20.04

In this article I will cover different scenarios which requires modification of cloud-init autoinstall configuration file such as installing GUI (GNOME), changing partition layout, installing additional packages, configuring network etc. I will share multiple examples of user-data configuration file to cover these scenarios.

Normally when we try to bring up a Ubuntu server using cloud-init then we don't get too many prompts (this is by design) as by default cloud-init considers default value unless you mark those parameters are interactive in the configuration file.

So I will try to share some sample cloud-init autoinstall configuration file (user-data) which you can use based or atleast you get a fair idea of what kind of modification is required for your requirement.

1. Generating encrypted password for user-data config file

In the cloud-init autoinstall configuration file (user-data), the password is expected to be encrypted format. For example:

  identity:
    hostname: <hostname>
    password: <encrypted-password>
    username: <username>

Here we have to replace <encrypted-password> after encrypting the password which you intend to use for your user. You can use the following command to generate an encrypted password with SHA512 algorithm. Replace PASSWORD with your password:

# openssl passwd -6 -stdin <<< PASSWORD
$6$vNiKYMV/149WtcPg$kbIXH.fmh9F.Trmvq21EAxUR9/JDm.qPseETsjgZjkasd3AW0WRBp9RT3gbt/fSvvopRhJeZ21ThzbNdZwVLf1

Or alternatively you can also use following command, here replace 'ubuntu' with your password:

python3 -c 'import crypt; print(crypt.crypt("ubuntu", crypt.mksalt(crypt.METHOD_SHA512)))'
$6$DgID3Ts5aNd5ps4E$Zus7WHKidEv0rkBvl6/kbeaOxCRXSBZB1.d30ez9drre1ku7nfJ7uD.ZN7eY7NMKWC8jQo/AGmk8CvxQAVOqV.

Or alternatively (the recommended method) you can use mkpasswd command:

root@ubuntu:~# mkpasswd --method=SHA-512 --rounds=4096
Password:
$6$rounds=4096$c3PYnICGfjNb2$BwA2nUSowf2fc8xSxvbqX0MWAf0gEWs7MTcAU6tmiIEopNXFrOE4QzrGdazpSd3NdcwpTZXmeYsJISmls7C1f.

Now you can copy this password and place it in your user-data file. It would be better if you place it inside quotes to avoid conflicts with special characters present in the encrypted password.

2. Install Ubuntu with GUI (GNOME Desktop)

By default cloud-init will perform the autoinstall with basic Ubuntu packages i.e. only CLI but most of us are used to GNOME desktop for ease of usage. So if you want to deploy Ubuntu with GUI desktop then add packages section as shown below:

  packages: 
    - ubuntu-desktop

This will install the ubuntu desktop for your server.

3. Execute commands during installation

You can add custom commands to be executed at different stage of the installation. These are defined using following parameters:

• early-commands: A list of shell commands to invoke as soon as the installer starts, in particular before probing for block and network devices. The autoinstall config is available at /autoinstall.yaml (irrespective of how it was provided) and the file will be re-read after the early-commands have run to allow them to alter the config if necessary.
• late-commands: Shell commands to run after the install has completed successfully and any updates and packages installed, just before the system reboots.  They are run in the installer environment with the installed system mounted at /target
• error-commands: Shell commands to run after the install has failed. They are run in the installer environment, and the target system (or as much of it as the installer managed to configure) will be mounted at /target. Logs will be available at /var/log/installer in the live session.

Here is a sample user-data file covering all 3 types of commands:

#cloud-config
autoinstall:
  version: 1
  early-commands:
    - ping -c5 10.43.138.8
  identity:
    hostname: ubuntu-server
    password: "$6$ePjW0.SQBGi0cLeL$HlAKikVCRqeIFic/RrHt90nbhUDFikPzu/HPR5nJflV.3nAfthTcLns/N4tqMzKyFp9lA/1HRUpJWLJ.7BUNK/"
    username: ubuntu
  interactive-sections:
    - storage
  late-commands:
    - |
      cat <<EOF | sudo tee /target/tmp/post-install.sh
      #!/bin/bash
      if grep -iq PermitRootLogin /target/etc/ssh/sshd_config; then
         sed -i -e 's/^#\?PermitRootLogin.*/PermitRootLogin yes/g' /target/etc/ssh/sshd_config
      else
         echo "PermitRootLogin yes" >>  /target/etc/ssh/sshd_config
      fi

      if grep -qi PasswordAuthentication /target/etc/ssh/sshd_config; then
         sed -i -e 's/^#\?PasswordAuthentication.*/PasswordAuthentication yes/g' /target/etc/ssh/sshd_config
      else
         echo "PasswordAuthentication yes" >>  /target/etc/ssh/sshd_config
      fi
      EOF
    - curtin in-target --target /target chmod 755 /tmp/post-install.sh
    - sudo sh /tmp/post-install.sh
  error-commands:
    - tar c /var/log/installer | nc 10.43.138.8 5050

Here,

1. early-commands: We are checking the connectivity towards our host server
2. late-commands: We have created a script to update the sshd_config file. Now since at this stage the filesystem is mounted on /target so we have added our config file path prefixed with /target
3. error-commands: In case of failure we will automatically transfer our logs to host server using netcat command.

To be able to use the nc command, you already should have nc command running on 10.43.138.8 server listening on port 5050 to which our client will transfer the files: For example in my case:

root@ubuntu:~# nc -l  -p 5050 >> /tmp/debug.log

4. Use proxy in cloud-init autoinstall configuration file

If your client is running behind some proxy then you may have to specify the proxy sever details to be able to download and install packages. You can define proxy using the following supported directives:

#cloud-config
autoinstall:
  version: 1
  early-commands:
    - ping -c5 10.43.138.8
  identity:
    hostname: ubuntu-server
    password: "$6$ePjW0.SQBGi0cLeL$HlAKikVCRqeIFic/RrHt90nbhUDFikPzu/HPR5nJflV.3nAfthTcLns/N4tqMzKyFp9lA/1HRUpJWLJ.7BUNK/"
    username: ubuntu
  proxy: http://10.58.80.65:8090

The syntax to be used is:

  proxy: http://[[user][:pass]@]host[:port]/

5. Customize SSH Configuration

We have a couple of options to choose from for SSHD usecase such as:

• install-server: Whether to install OpenSSH server in the target system.
• authorized-keys: A list of SSH public keys to install in the initial user’s account.
• allow-pw: true if authorized_keys is empty, false otherwise

Let's generate a private public key pair using ssh-keygen

root@ubuntu:~# ssh-keygen -t rsa -P "" -f test
Generating public/private rsa key pair.
Your identification has been saved in test
Your public key has been saved in test.pub
The key fingerprint is:
SHA256:hBMCy/gvSqc33fgPPws17PP/nJPWO6GvvFVa/i7PHPw root@ubuntu
The key's randomart image is:
+---[RSA 3072]----+
|  ... .          |
| o . . o         |
|. o   o .        |
| .     o.        |
|  .     S+      o|
|   .    o .   .=.|
| ...o oo o    o+=|
|..o+ o .+.o ..+B*|
|... . ...+o..=*XE|
+----[SHA256]-----+

Next we add the content of test.pub in authorized-keys section of cloud-init file. Here is a sample user-data config file:

#cloud-config
autoinstall:
  version: 1
  identity:
    hostname: ubuntu-server
    password: "$6$ePjW0.SQBGi0cLeL$HlAKikVCRqeIFic/RrHt90nbhUDFikPzu/HPR5nJflV.3nAfthTcLns/N4tqMzKyFp9lA/1HRUpJWLJ.7BUNK/"
    username: ubuntu
  proxy: http://10.158.100.6:8080
  interactive-sections:
    - storage
  ssh:
    allow-pw: false
    authorized-keys:
    - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDIeASHra5Kj8LMJiZvK0KBXjJUgTXgo9OVb6+zsawpzs3VxhryKCPTFA/4MBiDasflVHz5QqEHN/9RkIUPg7iDP8NlrVvPbfjgoK1Qxbks8gM/fO+EYbmndh7uBRFWg4xKd8k5SJXPfg8Psnw1xg7s+qv6kKV8AZEU9PFZflLlVDtJu8VJUVE0cR2IjdY8HIIpY4DAhCQPw4LpN7z80f0yPHW1Mb20/IxuZip1ZEVOVWLkRqCOR/nGIlH293DzzlN81X/9pRuRsoJL4ks4a5sNtJzpufrgIvw+w9Y85klf870C2OSUvT1QbrHHbFirzsAljXfC+msncSu9NzQNnHAJl7htN0IkZLDfIJaghwY2rSSuXo5+eh4GOo7jN9ECy5DIDc8T1kLGBHcSHcPd3APUp0BlymLAPBvSdbVdycjOhsrFP/zC3TxMuc+6/e50jg7aVNtVllrmmZndCcXF222B8NBA0R7INOmbO2lLe4vGdItCRrU7P4y+qyVgUxiouz8= root@ubuntu
    install-server: true

Now, we trigger the installation and post installation of our target node we can verify if we are able to SSH to the default system user using our private key:

root@ubuntu:~# ssh -i test ubuntu@10.43.138.26
The authenticity of host '10.43.138.26 (10.43.138.26)' can't be established.
ECDSA key fingerprint is SHA256:mu8UBLGRlsahbTO7WJSZihsWj0qu1foq8z3hsx981Mw.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '10.43.138.26' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 20.04.3 LTS (GNU/Linux 5.4.0-94-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Thu 13 Jan 2022 05:52:10 PM UTC

  System load:            0.77
  Usage of /:             5.2% of 195.86GB
  Memory usage:           0%
  Swap usage:             0%
  Temperature:            46.0 C
  Processes:              426
  Users logged in:        1
  IPv4 address for eno49: 10.43.138.26
  IPv6 address for eno49: 2001:1:1:1442:217:a4ff:fe77:32
  IPv4 address for eno50: 10.43.138.20
  IPv6 address for eno50: 2001:1:1:1442:217:a4ff:fe77:34


37 updates can be applied immediately.
To see these additional updates run: apt list --upgradable

Failed to connect to https://changelogs.ubuntu.com/meta-release-lts. Check your Internet connection or proxy settings


Last login: Thu Jan 13 17:51:44 2022
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@ubuntu-server:~$

So, we were able to successfully connect to our target server which we recently brought up using the above cloud-init autoinstall configuration file.

6. Customize network configuration

We also have a couple of options to customize our network configuration. The default is to interpret the config for the install media, which runs DHCPv4 on any interface with a name matching “eth*” or “en*” but then disables any interface that does not receive an address.

6.1 Configure static IPv4 network

To configure a static IPv4 network you can use below template. Here the interface name can be a problem in such case so unless you are sure of the network name mapping or you can just disable consistent network device naming by using biosdevname=0 net.ifnames=0 as the kernel menu while performing PXE. As with this option, the kernel will use the old interface naming convention i.e. ethX

  network:
    ethernets:
      eno49:
        addresses: [10.43.138.20/27]
        gateway4: 10.43.138.30
        nameservers:
          addresses: [8.8.8.8]
          search: [example.com]

6.2 Configure static IPv6 network

To configure a static IPv6 network you can use the following template (Here we have enabled DHCP for IPv4 network) . Here also the same rule applies as discussed in previous section. You may have to disable consistent network device naming to be able to assign static address.

      eno50:
        addresses: ['2001:1:1:1442::405/122']
        critical: true
        dhcp-identifier: mac
        dhcp4: true
        gateway6: 2001:1:1:1442::43f
        nameservers:
          addresses: [127.0.0.1, 8.8.8.8]
          search: [example.com]

6.3 Create bond

We can also create a bond using cloud-init autoinstall configuration file. Here is a sample template where I am creating a bond using eno49 and eno50 interface with a static IPv4 address.

  network:
    bonds:
      bond0:
        addresses:
        - 10.43.138.23/27
        gateway4: 10.43.138.30
        interfaces:
        - eno49
        - eno50
        nameservers:
          addresses:
          - 8.8.8.8
          search:
          - example.com
        parameters:
          mode: active-backup

You can get the list of supported bonding modes, I have used active-backup for my environment.

7. Create EFI Boot Partition

If you are trying to bring up Ubuntu using autoinstall configuration file on UEFI BIOS Environment then you will also need an EFI boot partition or else you will get below error during installation:

no efi system partition was found

I stumbled upon the same problem when I was trying to install Ubuntu using kickstart and luckily this article on stackoverflow has the fix. But those steps will not work with cloud-init autoinstall. Here you can consider the following sample template to create an EFI boot partition:

  storage:
    config:
    - name: ubuntu-vg
      devices:
      - partition-2
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-0
    - name: ubuntu-lv
      volgroup: lvm_volgroup-0
      size: 107374182400B
      wipe: superblock
      preserve: false
      type: lvm_partition
      id: lvm_partition-0
    - fstype: ext4
      volume: lvm_partition-0
      preserve: false
      type: format
      id: format-2
    - path: /
      device: format-2
      type: mount
      id: mount-2
    - path: /boot
      device: format-1
      type: mount
      id: mount-1
    - path: /boot/efi
      device: format-0
      type: mount
      id: mount-0

You can check the complete sample autoinstall configuration file at Prepare cloud-init autoinstall file

8. Perform a complete interactive installation

With cloud-init autoinstall, you can explicitly specify directives which you wish to configure manually rather than automated. In such case, you can specify those directives under interactive-sections as shown below:

  interactive-sections:
  - storage
  - network

So, here we have specified that we wish to configure storage and network manually so don't take the default values. Once we trigger an installation using this, the autoinstall will pause for user input to configure network and storage.

But if you wish to perform complete interactive installation (considering no defaults) then you can use * with interactive as shown below:

  interactive-sections:
  - *

Summary

In this tutorial I covered different options with cloud-init autoinstall (user-data) which you can customize based on your requirement. I will try to update this article as and when I find more interesting which we can customize in user-data configuration file. If you have something which you feel will be a great add-on to this article, drop me a message via the comment section.

Further Reading

Automated Server Installs Config File Reference