5. Leverage VirtualBox OCI export with cloud-init

Leverage VirtualBox OCI export with cloud-init


  • Preface

    Starting from VirtualBox v6.1.20 the cloud-init is supported to pass cloud-init script during export VM to OCI or during cloud instance creation. This feature has still being under testing. So any feedback is appreciated.

    if you are not familiar with VirtualBox OCI export there is a set of short articles about exporting VM from VirtualBox to OCI step by step "Export VM from VirtualBox to OCI".

    If you already have done export VM from VirtualBox into OCI you know that the preparation steps are needed to be done on the local VM before the export procedure. It’s about serial console preparation, SSH preparation, users preparation, network preparation, VirtIO preparation and may be something else like some additional packages installation.

    Now all these steps can be done automatically with properly written cloud-init script. This script is passed among others arguments during OCI export action and processed by OCI on the final step when a new instance is created.

    NB! Many OSs have the package "cloud-init" in the distributive media, but it's not the rule. User should check that the package "cloud-init" has already installed on his guest VM.
    The command "VBoxManage export”

    The command line utility VBoxManage has the action “export” which is used to conduct the following types of export as creating OVF/OVA appliance, cloud export. If user chooses cloud export then among the available parameters there is one named “--cloudinitscriptpath”. This parameter allows to pass a cloud-init script to the cloud side.

    The command "VBoxManage cloud instance launch”

    Another command "VBoxManage cloud instance launch” has a little bit different key "--cloud-init-script-path". This key does the same thing - pass a cloud-init script to the cloud side.

    The cloud-init documentation is well organized at least in the parts which mentioned in this article. So just read the original documentation and use the examples from the documentation.

    Cloud-init features

    Below I show some standard features of cloud-init which are commonly used by people.

    Adding users

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#users-and-groups

    Some systems have a default user. For such systems cloud-init provides the “default” user.

    The quote from cloud-init documentation:

    ”The users config key takes a list of users to configure. The first entry in this list is used as the default user for the system. To preserve the standard default user for the distro, the string default may be used as the first entry of the users list”.

    NB! It’s important, the “default” user must be presented in the user list and must be first.

    See examples below.

    User authentication via password

    This part is more or less self-explained and the most settings are clear. About the setting “passwd”. Here user shouldn’t put the password “as is” but use the hash of password. What does it mean? It means to use some utils like mkpasswd to create the password hash.

    The quote from cloud-init documentation:

    “The hash -- not the password itself -- of the password you want to use for this user. You can generate a safe hash via: mkpasswd --method=SHA-512 –rounds=4096 (the command would create from stdin an SHA-512 password hash with 4096 salt rounds)”.

    Example:

    Here the user “user1” is added. His full name is “Jack Smith”, password is “123456”, user is added to the groups “users” and “adm”, user isn’t locked after unsuccessful login attempts, shell is set to “/bin/bash” and user is added to the sudoers group.

    NB! Specifying a hash of a user’s password with passwd is a security risk if the cloud-config can be intercepted. SSH authentication is preferred.
    User authentication via SSH key

    Using public SSH key is much safe than password authentication. Just put a key under setting “ssh-authorized-keys”. User can add several keys if he wants.

    Example:

    Here the user “user2” is added. His full name is “Stephen King”, user is added to the groups “users” and “adm”, user isn’t locked after unsuccessful login attempts, shell is set to “/bin/bash”, user is added to the sudoers group and the public SSH key is added for SSH authentication.

    Install packages

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#package-update-upgrade-install

    This feature is simple, just add the needed packages under the keyword “packages”.

    Example:

    Sometimes it’s may be essential to update the packages (or repositories) before actual installation. In this case use the keywords “package_update” or “package_upgrade”.

    Adding files

    User can set path, content, owner, permissions, encoding, append. All settings are almost self-explained, the detailed description see in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#write-files.

    File awk-add-kernel-boot-settings

    The following snippet of cloud-init script adds the content of this section into a file named “awk-add-kernel-boot-settings.txt” in the folder /run/user on the launched instance. The usage of this file is explained in the chapter "Change kernel boot settings". The snippet is:

    Setup SSH

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#ssh

    Authorized keys for the “default” user defined in the section “users” can be specified using ssh_authorized_keys. For example, OCI OL instance has the default user “opc” and Ubuntu has “ubuntu”.

    If user specifies the public SSH key via “ssh_authorized_keys” it means that the users “opc” or “ubuntu” can be authenticated by the ssh keys from this section.

    Example:

    ssh-authorized-keys: - ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC.....................fEeFIiSSoz - <other ssh public keys> Set passwords

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#set-passwords

    OCI sets the sshd setting “PasswordAuthentication” to “no” by default. It reduces the risk of the standard brute-force attacks. But user exports own local VM where probably there is at least one user with password authentication because we want to keep capability to login into an instance via serial console using user’s credentials in case of any problems with login via SSH. So it’s needed to keep this setting as “yes”.

    Example:

    It’s allowed sshd will be configured to accept password authentication via the setting “ssh_pwauth” set to “True”. And password will never expire because the variable “expire” is set to “False” in the section “chpasswd”. It’s minimal settings that should be enough for VirtualBox OCI export.

    NB! Set the “ssh_pwauth” to “True” is dangerous, so it’s highly recommended to reset this sshd setting after successful OCI export.
    NB! The users specified must already exist on the system. Users will have been created by the cc_users_groups module at this point.
    Using instance metadata for getting information about instance

    The cloud-init has feature that allows the cloud providers keep some metadata inside a launched instance. There are a set of common parameters which widely used by the cloud providers. The detailed explanation is here https://cloudinit.readthedocs.io/en/latest/topics/instancedata.html.

    The follwing snippet of cloud-init script puts the section content into a file instance-info.html in the folder /usr/share/nginx/html/ on the launched instance. Here is shown of usage of instance metadata. The used instance metadata variables here are “v1.availability_zone”, “v1.instance_id”, “ds.meta_data.name”. If NGINX server was installed on the instance user would use the snippet “as is”. After opening this page in the web browser user should see the information about availability domain, instance ID and machine name.

    Example:

    NB! Remember, it's one time executed action. The instance metadata variables are replaced by the actual values during cloud-init execution. It means, in our case, that the file instance-info.html will be updated only once and then will be copied into the destination folder.
    Adding SSH host key

    In the chapter "Setup-SSH" a public SSH key is added for user SSH authentication. But sometimes user may want to add SSH private key for authenticating a launched instance, i.e. the launched instance may connect to the other instance using the provided SSH key pair (private + public keys). The detailed explanation and examples see in https://cloudinit.readthedocs.io/en/latest/topics/modules.html#host-keys.

    NB! when specifying private host keys in cloud-config, care should be taken to ensure that the communication between the data source and the instance is secure.
    Register a Red Hat system

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#red-hat-subscription

    It’s not an essential part for successful VirtualBox OCI export but it’s needed when some additional packages should be installed on RHEL system.

    This feature is very useful for any RHEL versions because without the subscription user may not be allowed to use some RHEL repositories and can’t install needed packages.

    Example:

    Adding yum repository

    https://cloudinit.readthedocs.io/en/latest/topics/modules.html#yum-add-repo

    Use the following template to add yum repository. Just give a name to a repository, replace the variable baseurl with a proper URI and set the others fields with the actual values.

    Example:

    This block does the simple thing just adds the settings into the file /etc/yum.repos.d/epel.repo. For example, if it’s used RHEL version 7 (placeholder version is replaced by 7) then in the epel.repo the following section will be created:

    [epel-release] name=Extra Packages for Enterprise Linux 7 - $basearch baseurl=http://download.fedoraproject.org/pub/epel/7/$basearch failovermethod=priority enabled=1 gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL-7 Running user commands

    As the final step of cloud-init during an instance’s first boot the block with user’s commands is executed. This is done in the section named “runcmd”. The detailed explanation is here https://cloudinit.readthedocs.io/en/latest/topics/modules.html#runcmd. There are several useful examples below that help to successfully finish OCI export.

    Add DNS server runcmd: - [ sed, -i, -e, '$ a\nameserver 169.254.169.254', /etc/resolv.conf ] Setup firewall rules

    The standard application here is the service firewalld. It’s commonly used.

    NB! On Ubuntu the service UFW is used. But to work in OCI user should apply the firewalld rules. Thus Ubuntu firewall must be turned off and firewalld must be used instead.
    runcmd: - [ firewall-offline-cmd, --add-service=ssh, --zone=public ] - [ firewall-offline-cmd, --add-service=http, --zone=public ] - [ systemctl, enable, firewalld ] - [ systemctl, start, firewalld ] - [ systemctl, status, firewalld ]

    The explanation why the firewall-offline-cmd is used here instead of the standard utility firewall-cmd is done in the chapter "Instance is unreachable because firewall-cmd failed during cloud-init execution".

    Restart sshd service runcmd: - [ service, sshd, restart ] Change kernel boot settings (allow and setup serial console)

    These changes are essential in case when there is a problem with SSH connection after VirtualBox OCI export.

    To setup serial console connection the kernel boot settings should be changed. Briefly, the following GRUB variables in the file /etc/default/grub are extended:

    - GRUB_CMDLINE_LINUX is extended by adding “console=tty0 console=ttyS0,115200n8”.

    - GRUB_TERMINAL is extended by adding “serial console”.

    - GRUB_SERIAL_COMMAND is extended by adding “serial --unit=0 --speed=115200 --word=8 --parity=no –stop=1”

    The file “awk-add-kernel-boot-settings.txt” from the chapter "File awk-add-kernel-boot-settings" does it. The file contains the AWK script which is run during an instance’s first boot. It runs in the section “runcmd”.

    Here the original file /etc/default/grub is copied into the file /run/user/original_grub, next the original_grub is modified in-place by SED, next AWK is used to process the original_grub using the file awk-add-kernel-boot-settings.txt (which contains awk script), the processed file original_grub is copied into the file /run/user/modified_grub, finally modified_grub is copied back into /etc/default/grub and generated new boot settings by the utility grub2-mkconfig.

    Snippet of cloud-init runcmd part:

    runcmd: - [ touch, /run/user/original_grub ] - echo "Created /run/user/original_grub" - [ cp, -f, /etc/default/grub, /run/user/original_grub] - echo "Copied /etc/default/grub to /run/user/original_grub" - [ sed, -i, -e, '/GRUB_CMDLINE_LINUX/s/"/@@"/g', /run/user/original_grub ] - echo "Modified in-place /run/user/original_grub with SED" - [ awk, -f, /run/user/awk-add-kernel-boot-settings.txt, /run/user/original_grub ] - echo "Modified /run/user/original_grub with AWK and copied into /run/user/modified_grub" - [ cp, -f, /run/user/modified_grub, /etc/default/grub ] - echo "Copied /run/user/modified_grub into /etc/default/grub" - [ grub2-mkconfig, -o, /boot/grub2/grub.cfg ] Allow service getty runcmd: - [ systemctl, enable, [email protected] ] - [ systemctl, start, [email protected] ] - [ systemctl, daemon-reload ] Cloud-init logging

    The detailed explanation is here https://cloudinit.readthedocs.io/en/latest/topics/logging.html.

    The example below just shows that there are different ways to adapt the logging for user purposes:

    Known issues Instance is unreachable because firewall-cmd failed during cloud-init execution

    Instance may be unreachable from outside because the firewall rules in the instance haven’t been set and activated. It may happen that firewall-cmd doesn’t work as expected during the instance’s first boot. And some errors like:

    cloud-init: ERROR:dbus.proxies:Introspect error on :1.9:/org/fedoraproject/FirewallD1/config: dbus.exceptions.DbusException: org.freedesktop.DBus.Error.NoReply: Did not receive a reply.

    may appear in the cloud-init log.

    To workaround this error try to use the utility firewall-offline-cmd instead firewall-cmd in the section “runcmd” which is the standard place for such user actions. How to do it see in the chapter "Setup firewall rules"

    The packages aren’t installed First cause

    The error may appear during package installation: "Could not resolve host: yum-iad.oracle.com; Unknown error". It may mean that internal OCI DNS service doesn’t work. It may occur if the file /etc/resolv.conf doesn’t contain the internal DNS address. For the public OCI tenancies the internal DNS address is 169.254.169.254. For the private OCI tenancies the internal DNS address may have different value. User should ask his tenancy's administrator about it. This server should be added into the /etc/resolv.conf. How to do it see in the chapter Add DNS server

    Second cause

    Yum can't update\install packages and displays the error like "Could not resolve host: yum.oracle.com; Unknown error". It means that the file  /etc/yum/vars/ociregion is empty. In the common case the file contains a region where the instance has been launched. For instance, the file may contain the line "-iad". This value is used for Ashburn region. The value "-iad" is added into the path "yum.oracle.com" after "yum". The result is "yum-iad.oracle.com". Oracle places the mirrors of yum repositories in the each region. It allows the instances and other cloud entities use the nearest yum repositories. User should find the region where an instance is placed and add a corresponding name into the file /etc/yum/vars/ociregion.

    Anyway user may manually add the known public yum repositories (which are accessible via internet) and try to use them.

    Cloud-init doesn’t process a cloud-init script

    Because cloud-init version 18.3 and earlier don’t support the script template “## template: jinja”.

    If cloud-init script contains the “## template: jinja” and the target OS has installed utility cloud-init version below 18.4 then the cloud-init script won’t be recognized as the proper cloud-init stuff. In this case leave only “#cloud-config” in the cloud-init script.

    Full template of cloud-init script for VirtualBox OCI export

    This example is done for RHEL 7\8 versions.



    https://blogs.oracle.com/virtualization/leverage-virtualbox-oci-export-with-cloud-init
Log in to reply
 

© Lightnetics 2024