標籤

Subiquity

Notice

ISO installer vs cloud instances

subiquity is the ISO image installer, and ISO images are normally used to install Ubuntu on bare metal.
Cloud instances are deployed using the Ubuntu cloud images, and the deployment process does not involve subiquity: the configuration is done only via cloud-init.

Subiquity writes the netplan configuration at install time. Subiquity's only duty is to install the system, it can't be used to reconfigure the installed system after that.

Automated Server Installs

The server installer for 20.04 supports a new mode of operation: automated installation.
Autoinstallation lets you answer all those configuration questions ahead of time with an autoinstall config and lets the installation process run without any interaction.

By default, if there is any autoinstall config at all, the installer:

  • takes the default for any unanswered question
  • fails if there is no default
You can designate particular sections in the config as “interactive”, which means the installer will still stop and ask about those.

Providing the autoinstall config

The autoinstall config is provided via cloud-init configuration.
The easiest way to use autoinstall config will be to provide user-data via the nocloud data source.
The autoinstall config should be provided under the autoinstall key in the cloud-init config.

Running a truly automatic autoinstall

Even if a fully noninteractive autoinstall config is found, the server installer will ask for confirmation before writing to the disks unless autoinstall is present on the kernel command line.
This is to make it harder to accidentally create a USB stick that will reformat a machine it is plugged into at boot.

Many autoinstalls will be done via netboot, where the kernel command line is controlled by the netboot config – just remember to put autoinstall in there!

subiquity-service

When any system is installed using the server installer, an autoinstall file for repeating the install is created at /var/log/installer/autoinstall-user-data.

Init config example


#cloud-config
autoinstall:
  version: 1
  # use interactive-sections to avoid an automatic reboot
  interactive-sections:
    - identity
The following UI screen showed:
  • Welcome
  • Keyboard configuration
  • network connections
  • Configure proxy
  • Configure Ubuntu archive mirror
    • http://archive.ubuntu.com/ubuntu
  • Guided storage configuration
  • Storage configuration
  • Profile setup
  • An error occured during installation
    • 
installing system
  curtin command install
    preparing for installation
      configuring storage
        running 'curtin block-meta simple'
          curtin command block-meta
            removing previous storage devices
  subiquity/Error/158762522.765956640.install_fail/add_info
    The Sorry dialog: 
    
The error report has been saved to 

install-logs-2020-01-01.0/crash/158762522.765956640.install_fail
The crash log is /var/crash/158762522.765956640.install_fail.crash: 

...
 finish: cmd-install/stage-partitioning/builtin/cmd-block-meta/clear-holders: FAIL: removing previous storage devices
 TIMED BLOCK_META: 12.510
 finish: cmd-install/stage-partitioning/builtin/cmd-block-meta: FAIL: curtin command block-meta
...
 2020-04-01 17:30:18,204 DEBUG urllib3.connectionpool:396 http://localhost:None "GET /v2/find?select=refresh HTTP/1.1" 500 264
 2020-04-01 17:30:18,206 ERROR root:39 finish: subiquity/Refresh/check_for_update: FAIL: 500 Server Error: Internal Server Error for url: http+unix://%2Frun%2Fsnapd.socket/v2/find?select=refresh
...
 2020-04-01 17:35:22,764 DEBUG subiquitycore.utils:83 arun_command ['systemd-cat', '--level-prefix=false', '--identifier=curtin_log.1948', '/snap/subiquity/1772/usr/bin/python3', '-m', 'curtin', '--showtrace', '-c', '/var/log/installer/subiquity-curtin-install.conf', 'install'] exited with code 3
 2020-04-01 17:35:22,765 ERROR root:39 finish: subiquity/InstallProgress/curtin_install: FAIL: Command '['systemd-cat', '--level-prefix=false', '--identifier=curtin_log.1948', '/snap/subiquity/1772/usr/bin/python3', '-m', 'curtin', '--showtrace', '-c', '/var/log/installer/subiquity-curtin-install.conf', 'install']' returned non-zero exit status 3.
...
 2020-04-01 17:30:18,206 ERROR root:39 finish: subiquity/Refresh/check_for_update: FAIL: 500 Server Error: Internal Server Error for url: http+unix://%2Frun%2Fsnapd.socket/v2/find?select=refresh
...

Title: install failed crashed with CalledProcessError
...
 OSError: [Errno 16] Device or resource busy: '/dev/nvme0n1p3'

Delete existed partitions on the target then test it again.

The following UI screen showed:
  • Welcome
  • Keyboard configuration
  • Network connections
  • Configure proxy
  • Configure Ubuntu archive mirror
    • http://archive.ubuntu.com/ubuntu
  • Guided storage configuration
  • Storage configuration
    • Confirm destruction action
  • Profile setup
    • name, server name, username, password
  • An error occured during installation
    • 
installing system
  curtin command install
    preparing for installation
      configuring storage
        running 'curtin block-meta simple'
          curtin command block-meta
            removing previous storage devices
            configuring disk: disk-nvme0n1
          configuring partition: partition-0
          configuring format: format-0
          configuring partition: partition-1
          configuring format: format-1
          configuring mount: mount-1
          configuring mount: mount-0
      configuring network
        running 'curtin net-meta auto'
          curtin command net-meta
      writing install sources to disk
        running 'curtin extract'
          curtin command extract
            acquiring and extracting image from cp://media/filesystem
      configuring installed system
        running '/snap/bin/subiquity.subiquity-configure
        running '/snap/bin/subiquity.subiquity-configure
          curtin command apt-config
          curtin command in-target
  subiquity/Error/158762339.495924234.install_fail/add_info
    The Sorry dialog: 
    
The error report has been saved to 

install-logs-2020-01-01.0/crash/158762339.495924234.install_fail
The crash log: 

...
 2020-04-01 17:32:49,036 DEBUG root:39 finish: subiquity/InstallProgress/curtin_install/cmd-install/stage-curthooks/001-configure-apt/cmd-in-target: SUCCESS: Status.FAIL
 2020-04-01 17:32:49,037 DEBUG root:39 finish: subiquity/InstallProgress/curtin_install/cmd-install/stage-curthooks/001-configure-apt: SUCCESS: Status.FAIL
 2020-04-01 17:32:49,037 DEBUG root:39 finish: subiquity/InstallProgress/curtin_install/cmd-install/stage-curthooks: SUCCESS: Status.FAIL
 2020-04-01 17:32:49,248 DEBUG subiquitycore.utils:83 arun_command ['systemd-cat', '--level-prefix=false', '--identifier=curtin_log.1871', '/snap/subiquity/1772/usr/bin/python3', '-m', 'curtin', '--showtrace', '-c', '/var/log/installer/subiquity-curtin-install.conf', 'install'] exited with code 3
 2020-04-01 17:32:49,249 ERROR root:39 finish: subiquity/InstallProgress/curtin_install: FAIL: Command '['systemd-cat', '--level-prefix=false', '--identifier=curtin_log.1871', '/snap/subiquity/1772/usr/bin/python3', '-m', 'curtin', '--showtrace', '-c', '/var/log/installer/subiquity-curtin-install.conf', 'install']' returned non-zero exit status 3.
 2020-04-01 17:32:49,249 ERROR subiquitycore.controller.installprogress:134 curtin_error
...

Setup the IP manually during the installation.

The following UI screen showed:
  • Welcome
  • Keyboard configuration
  • Network connections
  • Configure proxy
  • Configure Ubuntu archive mirror
    • http://archive.ubuntu.com/ubuntu
  • Guided storage configuration
  • Storage configuration
    • Confirm destruction action
  • Profile setup
    • name, server name, username, password
  • SSH Setup
  • Featured Server Snaps
  • Install complete!
    • 
installing system
  curtin command install
    preparing for installation
    configuring storage
        running 'curtin block-meta simple'
          curtin command block-meta
            removing previous storage devices
            configuring disk: disk-nvme0n1
          configuring partition: partition-0
          configuring format: format-0
          configuring partition: partition-1
          configuring format: format-1
          configuring mount: mount-1
          configuring mount: mount-0
    writing install sources to disk
        running 'curtin extract'
          curtin command extract
            acquiring and extracting image from cp://media/filesystem
    configuring installed system
        running '/snap/bin/subiquity.subiquity-configure-apt /snap/subiquity/2393/usr/bin/python3 true'
          curtin command apt-config
          curtin command in-target  
        running 'curtin curthooks'
          
    finalizing installation
        
    executing late commands
final system configuration
  configuring cloud-init
  restoring apt configuration
downloading and installing security updates
subiquity/Late/run
The successful :
  • user-data
    • 
#cloud-config
autoinstall:
  version: 1
  # use interactive-sections to avoid an automatic reboot
  interactive-sections:
    - identity
  locale: en_US.UTF-8
  • /var/log/installer/
    • autoinstall-user-data
      • 
#cloud-config
autoinstall:
  apt:
    geoip: true
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://tw.archive.ubuntu.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  identity: {hostname: se30, password: $6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0,
    realname: UBUNTU, username: ubuntu}
  keyboard:
    layout: en
    variant: uk  
  locale: C.UTF-8
  network:
    ethernets:
      enp47s0:
        addresses: [192.168.2.49/24]
        critical: true
        dhcp-identifier: mac
        gateway4: 192.168.2.1
        nameservers:
          addresses: [8.8.8.8, 10.15.2.29, 8.8.8.8]
          search: [example.org]
    version: 2
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - {ptable: gpt, serial: KAG12ZNS256G KIOXIA_70VPD0JPQ6Z2, wwn: eui.8ce38e0400f5677f,
      path: /dev/nvme0n1, wipe: superblock, preserve: false, name: '', grub_device: false,
      type: disk, id: disk-nvme0n1}
    - {device: disk-nvme0n1, size: 536870912, wipe: superblock, flag: boot, number: 1,
      preserve: false, grub_device: true, type: partition, id: partition-0}
    - {fstype: fat32, volume: partition-0, preserve: false, type: format, id: format-0}
    - {device: disk-nvme0n1, size: 1073741824, wipe: superblock, flag: '', number: 2,
      preserve: false, grub_device: false, type: partition, id: partition-1}
    - {fstype: ext4, volume: partition-1, preserve: false, type: format, id: format-1}
    - {device: disk-nvme0n1, size: 254447452160, wipe: superblock, flag: '', number: 3,
      preserve: false, grub_device: false, type: partition, id: partition-2}
    - name: ubuntu-vg
      devices: [partition-2]
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-0
    - {name: ubuntu-lv, volgroup: lvm_volgroup-0, size: 127221628928B, wipe: superblock,
      preserve: false, type: lvm_partition, id: lvm_partition-0}
    - {fstype: ext4, volume: lvm_partition-0, preserve: false, type: format, id: format-2}
    - {device: format-2, path: /, type: mount, id: mount-2}
    - {device: format-1, path: /boot, type: mount, id: mount-1}
    - {device: format-0, path: /boot/efi, type: mount, id: mount-0}
  updates: security
  version: 1

Tune the /var/log/installer/autoinstall-user-data generated after the successful installation

Use autoinstall on the kernel command line.
Based on the generated user-data to find the correct settings for the target.
The working config for SE30: 

#cloud-config
autoinstall:
  apt:
    geoip: true
    preserve_sources_list: false
    primary:
    - arches: [amd64, i386]
      uri: http://tw.archive.ubuntu.com/ubuntu
    - arches: [default]
      uri: http://ports.ubuntu.com/ubuntu-ports
  identity: {hostname: se30, password: $6$exDY1mhS4KUYCE/2$zmn9ToZwTKLhCw.b4/b.ZRTIZM30JZ4QrOQ2aOXJ8yk96xpcCof0kxKwuX1kqLG/ygbJ1f8wxED22bTL4F46P0,
    realname: UBUNTU, username: ubuntu}
  keyboard: 
    layout: us
    variant: us
  locale: C.UTF-8
  network:
    network:
      version: 2
      ethernets:
        enp47s0:
          addresses: [192.168.2.49/24]
          dhcp-identifier: mac
          gateway4: 192.168.2.1
          nameservers:
            addresses: [8.8.8.8, 1.1.1.1]
  ssh:
    allow-pw: true
    authorized-keys: []
    install-server: false
  storage:
    config:
    - {ptable: gpt, serial: KAG12ZNS256G KIOXIA_70VPD0JPQ6Z2, wwn: eui.8ce38e0400f5677f,
      path: /dev/nvme0n1, wipe: superblock, preserve: false, name: '', grub_device: false,
      type: disk, id: disk-nvme0n1}
    - {device: disk-nvme0n1, size: 536870912, wipe: superblock, flag: boot, number: 1,
      preserve: false, grub_device: true, type: partition, id: partition-0}
    - {fstype: fat32, volume: partition-0, preserve: false, type: format, id: format-0}
    - {device: disk-nvme0n1, size: 1073741824, wipe: superblock, flag: '', number: 2,
      preserve: false, grub_device: false, type: partition, id: partition-1}
    - {fstype: ext4, volume: partition-1, preserve: false, type: format, id: format-1}
    - {device: disk-nvme0n1, size: 254447452160, wipe: superblock, flag: '', number: 3,
      preserve: false, grub_device: false, type: partition, id: partition-2}
    - name: ubuntu-vg
      devices: [partition-2]
      preserve: false
      type: lvm_volgroup
      id: lvm_volgroup-0
    - {name: ubuntu-lv, volgroup: lvm_volgroup-0, size: 127221628928B, wipe: superblock,
      preserve: false, type: lvm_partition, id: lvm_partition-0}
    - {fstype: ext4, volume: lvm_partition-0, preserve: false, type: format, id: format-2}
    - {device: format-2, path: /, type: mount, id: mount-2}
    - {device: format-1, path: /boot, type: mount, id: mount-1}
    - {device: format-0, path: /boot/efi, type: mount, id: mount-0}
  updates: security
  version: 1

./autoinstall-schema.json


        "apt": {
            "type": "object",
            "properties": {
                "preserve_sources_list": {
                    "type": "boolean"
                },
                "primary": {
                    "type": "array"
                },
                "geoip": {
                    "type": "boolean"
                },
                "sources": {
                    "type": "object"
                }
            }
        },

Automated Server Installs Config File Reference

The autoinstall file is YAML.

keyboard

The mapping’s keys correspond to settings in the /etc/default/keyboard configuration file which can be found in generic Linux system.
  • layout
    • Corresponds to the XKBLAYOUT setting.
  • variant
    • Corresponds to the XKBVARIANT setting.
  • toggle
    • Corresponds to the value of grp: option from the XKBOPTIONS setting. The version of subiquity released with 20.04 GA does not accept null for this field due to a bug.

network

netplan formatted network configuration.
This will be applied during installation as well as in the installed system.
The version of subiquity released with 20.04 GA forces you to write this with an extra “network:” key like: 

network:
  network:
    version: 2
    ethernets:
      enp0s31f6:
        dhcp6: yes

canonical/subiquity

subiquity and console-conf

The repository contains the source for the
  • subiquity("ubiquity for servers")
    • new server installer
  • console-conf
    • the snappy first boot experience

subiquity design notes

UI

  • The UI never blocks
    • If something takes more than about 0.1s, it is done in the background, possibly with some kind of indication in the UI and the ability to cancel if appropriate.
    If indication is shown, it is shown for at least 1s to avoid flickering the UI.
    There is a helper, wait_with_text_dialog for this.
  • urwid
    • subiquity is built using the urwid console user interface library for Python.

Code structure

  • Overall architecture
    • Subiquity has a client / server model:
    • there is one server, which collects the data that will go into the curtin config and runs the install
    • one or more client processes which connect to the server
      • One client runs on tty1 (apart from on s390x) and others run on any configured serial console.
      One can also ssh into the live session as another way of starting a client.
    The model is the config that will be passed to curtin, which is broken apart into classes for the configuration of the network, the filesystem, the language, etc, etc.
    The full model lives in subiquity.models.subiquity and the submodels live in modules like subiquitycore.models.network and subiquity.models.keyboard.
    Each model object gets an asyncio.Event object associated with it that is set when the model is ready to be used as part of the installation.
    ubiquity presents itself as a series of screens -- Welcome, Keyboard, Network, etc etc.
    Each screen is managed by an instance of a controller class.

autoinstalls

This mostly impacts the server, which loads the config and the server controllers have methods that are called to load and apply the autoinstall data for each controller.
If the install is to be totally automated: in this case it does not start the urwid-based UI at all and mostly just "listens" to install progress via journald and the meta.status.GET() API call. The server code proceeds in stages:
  1. It starts up, checks for an autoinstall config and runs any early commands.
  2. Then it waits for all the model objects that feed into the curtin config to be configured.
  3. It waits for confirmation.
  4. It runs "curtin install" and waits for that to finish.
  5. It waits for the model objects that feed into the cloud-init config to be configured.
  6. It creates the cloud-init config for the first boot of the installed system.
  7. If there appears to be a working network connection, it downloads and installs security updates.
  8. It runs any late commands.
  9. It waits for the user to click "reboot".

Acquire subiquity from source


$ git clone https://github.com/canonical/subiquity
$ cd subiquity && make install_deps

ubiquity/server/server.py

Testing Subiquity's text UI

After checking out subiquity you can start it: 

$ make dryrun
All of the features are present in dry-run mode.
  • The installer will emit its backend configuration files to /tmp/subiquity-config-* but it won't attempt to run any installer commands (which would fail without root privileges).
  • subiquity can load other machine profiles in case you want to test out the installer without having access to the machine.
    • A few sample machine profiles are available in the repository at ./examples/ and can be loaded via the MACHINE make variable: 
    
make dryrun MACHINE=examples/simple.json

Generating machine profiles

Machine profiles are generated from the probert tool.
To collect a machine profile: 

PYTHONPATH=probert ./probert/bin/probert --all > mymachine.json

Testing changes in KVM

To try out your changes for real, it is necessary to install them into an ISO.
It's much easier to install your version of subiquity into the daily image.
  1. Prepare packages
    2. 
$ sudo apt install openjdk-8-jre
$ wget http://archive.ubuntu.com/ubuntu/pool/universe/x/xkeyboard-config/xkb-data-i18n_2.23.1-1ubuntu1_all.deb
$ sudo apt install ./xkb-data-i18n_2.23.1-1ubuntu1_all.deb  
$ sudo apt install python-distutils-extra
  2. Build your change into a snap
    3. Modify subiquity/ui/views/help.py to change the "ABOUT_INSTALLER".
    
$ snapcraft snap --output subiquity_test.snap
    This version can't use --destructive-mode to build the snap that will generate the following build error: 
    
The linker version '2.27' used by the base 'core18' is incompatible with files in this snap:
  3. Grab the current version of the installer
    4. 
$ urlbase=http://cdimage.ubuntu.com/ubuntu-server/$(distro-info --lts)/daily-live/current
$ isoname=$(distro-info --lts)-live-server-$(dpkg --print-architecture).iso
$ zsync ${urlbase}/${isoname}.zsync
  4. Run the provided script to make a copy of the downloaded installer then load your version of subiquity
    5. 
$ sudo ./scripts/inject-subiquity-snap.sh ${isoname} subiquity_test.snap custom.iso
  5. Boot the modified iso in KVM to do installation
    6. 
$ qemu-img create -f raw target.img 10G
$ kvm -m 1024 -boot d -cdrom custom.iso -hda target.img -serial stdio
  6. test the installed image
    7. 
$ kvm -m 1024 -hda target.img -serial stdio
電子郵件發布
標籤：

留言

張貼留言

熱門文章