Building a local testing lab with Ubuntu, MAAS and netplan

Overview

I'm presenting here the technical aspects of setting up a small-scale testing lab in my basement, using as little hardware as possible, and keeping costs to a minimum. For one thing, systems needed to be mobile if possible, easy to replace, and as flexible as possible to support various testing scenarios. I may wish to bring part of this network with me on short trips to give a talk, for example.

One of the core aspects of this lab is its use of the network. I have former experience with Cisco hardware, so I picked some relatively cheap devices off eBay: a decent layer 3 switch (Cisco C3750, 24 ports, with PoE support in case I'd want to start using that), a small Cisco ASA 5505 to act as a router. The router's configuration is basic, just enough to make sure this lab can be isolated behind a firewall, and have an IP on all networks. The switch's config is even simpler, and consists in setting up VLANs for each segment of the lab (different networks for different things). It connects infrastructure (the MAAS server, other systems that just need to always be up) via 802.1q trunks; the servers are configured with IPs on each appropriate VLAN. VLAN 1 is my "normal" home network, so that things will work correctly even when not supporting VLANs (which means VLAN 1 is set to the the native VLAN and to be untagged wherever appropriate). VLAN 10 is "staging", for use with my own custom boot server. VLAN 15 is "sandbox" for use with MAAS. The switch is only powered on when necessary, to save on electricity costs and to avoid hearing its whine (since I work in the same room). This means it is usually powered off, as the ASA already provides many ethernet ports. The telco rack in use was salvaged, and so were most brackets, except for the specialized bracket for the ASA which was bought separately. Total costs for this setup is estimated to about 500$, since everything comes from cheap eBay listings or salvaged, reused equipment.

The Cisco hardware was specifically selected because I had prior experience with them, so I could make sure the features I wanted were supported: VLANs, basic routing, and logs I can make sense of. Any hardware could do -- VLANs aren't absolutely required, but given many network ports on a switch, it tends to avoid requiring multiple switches instead.

My main DNS / DHCP / boot server is a raspberry pi 2. It serves both the home network and the staging network. DNS is set up such that the home network can resolve any names on any of the networks: using home.example.com or staging.example.com, or even maas.example.com as a domain name following the name of the system. Name resolution for the maas.example.com domain is forwarded to the MAAS server. More on all of this later.

The MAAS server has been set up on an old Thinkpad X230 (my former work laptop); I've been routinely using it (and reinstalling it) for various tests, but that meant reinstalling often, possibly conflicting with other projects if I tried to test more than one thing at a time. It was repurposed to just run Ubuntu 18.04, with a MAAS region and rack controller installed, along with libvirt (qemu) available over the network to remotely start virtual machines. It is connected to both VLAN 10 and VLAN 15.

Additional testing hardware can be attached to either VLAN 10 or VLAN 15 as appropriate -- the C3750 is configured so "top" ports are in VLAN 10, and "bottom" ports are in VLAN 15, for convenience. The first four ports are configured as trunk ports if necessary. I do use a Dell Vostro V130 and a generic Acer Aspire laptop for testing "on hardware". They are connected to the switch only when needed.

Finally, "clients" for the lab may be connected anywhere (but are likely to be on the "home" network). They are able to reach the MAAS web UI directly, or can use MAAS CLI or any other features to deploy systems from the MAAS servers' libvirt installation.

Setting up the network hardware

I will avoid going into the details of the Cisco hardware too much; configuration is specific to this hardware. The ASA has a restrictive firewall that blocks off most things, and allows SSH and HTTP access. Things that need access the internet go through the MAAS internal proxy.

For simplicity, the ASA is always .1 in any subnet, the switch is .2 when it is required (and was made accessible over serial cable from the MAAS server). The rasberrypi is always .5, and the MAAS server is always .25. DHCP ranges were designed to reserve anything .25 and below for static assignments on the staging and sandbox networks, and since I use a /23 subnet for home, half is for static assignments, and the other half is for DHCP there.

MAAS server hardware setup

Netplan is used to configure the network on Ubuntu systems. The MAAS server's configuration looks like this:

network:
    ethernets:
        enp0s25:
            addresses: []
            dhcp4: true
            optional: true
    bridges:
        maasbr0:
            addresses: [ 10.3.99.25/24 ]
            dhcp4: no
            dhcp6: no
            interfaces: [ vlan15 ]
        staging:
            addresses: [ 10.3.98.25/24 ]
            dhcp4: no
            dhcp6: no
            interfaces: [ vlan10 ]
    vlans:
        vlan15:
            dhcp4: no
            dhcp6: no
            accept-ra: no
            id: 15
            link: enp0s25
        vlan10:
            dhcp4: no
            dhcp6: no
            accept-ra: no
            id: 10
            link: enp0s25
    version: 2

Both VLANs are behind bridges as to allow setting virtual machines on any network. Additional configuration files were added to define these bridges for libvirt (/etc/libvirt/qemu/networks/maasbr0.xml):

<network>
<name>maasbr0</name>
<bridge name="maasbr0">
<forward mode="bridge">
</forward></bridge></network>

Libvirt also needs to be accessible from the network, so that MAAS can drive it using the "pod" feature. Uncomment "listen_tcp = 1", and set authentication as you see fit, in /etc/libvirt/libvirtd.conf. Also set:

libvirtd_opts="-l"

In /etc/default/libvirtd, then restart the libvirtd service.

dnsmasq server

The raspberrypi has similar netplan config, but sets up static addresses on all interfaces (since it is the DHCP server). Here, dnsmasq is used to provide DNS, DHCP, and TFTP. The configuration is in multiple files; but here are some of the important parts:

dhcp-leasefile=/depot/dnsmasq/dnsmasq.leases
dhcp-hostsdir=/depot/dnsmasq/reservations
dhcp-authoritative
dhcp-fqdn
# copied from maas, specify boot files per-arch.
dhcp-boot=tag:x86_64-efi,bootx64.efi
dhcp-boot=tag:i386-pc,pxelinux
dhcp-match=set:i386-pc, option:client-arch, 0 #x86-32
dhcp-match=set:x86_64-efi, option:client-arch, 7 #EFI x86-64
# pass search domains everywhere, it's easier to type short names
dhcp-option=119,home.example.com,staging.example.com,maas.example.com
domain=example.com
no-hosts
addn-hosts=/depot/dnsmasq/dns/
domain-needed
expand-hosts
no-resolv
# home network
domain=home.example.com,10.3.0.0/23
auth-zone=home.example.com,10.3.0.0/23
dhcp-range=set:home,10.3.1.50,10.3.1.250,255.255.254.0,8h
# specify the default gw / next router
dhcp-option=tag:home,3,10.3.0.1
# define the tftp server
dhcp-option=tag:home,66,10.3.0.5
# staging is configured as above, but on 10.3.98.0/24.
# maas.example.com: "isolated" maas network.
# send all DNS requests for X.maas.example.com to 10.3.99.25 (maas server)
server=/maas.example.com/10.3.99.25
# very basic tftp config
enable-tftp
tftp-root=/depot/tftp
tftp-no-fail
# set some "upstream" nameservers for general name resolution.
server=8.8.8.8
server=8.8.4.4

DHCP reservations (to avoid IPs changing across reboots for some systems I know I'll want to reach regularly) are kept in /depot/dnsmasq/reservations (as per the above), and look like this:

de:ad:be:ef:ca:fe,10.3.0.21

I did put one per file, with meaningful filenames. This helps with debugging and making changes when network cards are changed, etc. The names used for the files do not match DNS names, but instead are a short description of the device (such as "thinkpad-x230"), since I may want to rename things later.

Similarly, files in /depot/dnsmasq/dns have names describing the hardware, but then contain entries in hosts file form:

10.3.0.21 izanagi

Again, this is used so any rename of a device only requires changing the content of a single file in /depot/dnsmasq/dns, rather than also requiring renaming other files, or matching MAC addresses to make sure the right change is made.

Installing MAAS

At this point, the configuration for the networking should already be completed, and libvirt should be ready and accessible from the network.

The MAAS installation process is very straightforward. Simply install the maas package, which will pull in maas-rack-controller and maas-region-controller.

Once the configuration is complete, you can log in to the web interface. Use it to make sure, under Subnets, that only the MAAS-driven VLAN has DHCP enabled. To enable or disable DHCP, click the link in the VLAN column, and use the "Take action" menu to provide or disable DHCP.

This is necessary if you do not want MAAS to fully manage all of the network and provide DNS and DHCP for all systems. In my case, I am leaving MAAS in its own isolated network since I would keep the server offline if I do not need it (and the home network needs to keep working if I'm away).

Some extra modifications were made to the stock MAAS configuration to change the behavior of deployed systems. For example; I often test packages in -proposed, so it is convenient to have that enabled by default, with the archive pinned to avoid accidentally installing these packages. Given that I also do netplan development and might try things that would break the network connectivity, I also make sure there is a static password for the 'ubuntu' user, and that I have my own account created (again, with a static, known, and stupidly simple password) so I can connect to the deployed systems on their console. I have added the following to /etc/maas/preseed/curtin_userdata:

late_commands:
[...]
  pinning_00: ["curtin", "in-target", "--", "sh", "-c", "/bin/echo 'Package: *' >> /etc/apt/preferences.d/proposed"]
  pinning_01: ["curtin", "in-target", "--", "sh", "-c", "/bin/echo 'Pin: release a={{release}}-proposed' >> /etc/apt/preferences.d/proposed"]
  pinning_02: ["curtin", "in-target", "--", "sh", "-c", "/bin/echo 'Pin-Priority: -1' >> /etc/apt/preferences.d/proposed"]
apt:
  sources:
    proposed.list:
      source: deb $MIRROR {{release}}-proposed main universe
write_files:
  userconfig:
    path: /etc/cloud/cloud.cfg.d/99-users.cfg
    content: |
      system_info:
        default_user:
          lock_passwd: False
          plain_text_passwd: [REDACTED]
      users:
        - default
        - name: mtrudel
          groups: sudo
          gecos: Matt
          shell: /bin/bash
          lock-passwd: False
          passwd: [REDACTED]

The pinning_ entries are simply added to the end of the "late_commands" section.

For the libvirt instance, you will need to add it to MAAS using the maas CLI tool. For this, you will need to get your MAAS API key from the web UI (click your username, then look under MAAS keys), and run the following commands:

maas login local   http://localhost:5240/MAAS/  [your MAAS API key]
maas local pods create type=virsh power_address="qemu+tcp://127.0.1.1/system"

The pod will be given a name automatically; you'll then be able to use the web interface to "compose" new machines and control them via MAAS. If you want to remotely use the systems' Spice graphical console, you may need to change settings for the VM to allow Spice connections on all interfaces, and power it off and on again.

Setting up the client

Deployed hosts are now reachable normally over SSH by using their fully-qualified name, and specifying to use the ubuntu user (or another user you already configured):

ssh ubuntu@vocal-toad.maas.example.com

There is an inconvenience with using MAAS to control virtual machines like this, they are easy to reinstall, so their host hashes will change frequently if you access them via SSH. There's a way around that, using a specially crafted ssh_config (~/.ssh/config). Here, I'm sharing the relevant parts of the configuration file I use:

CanonicalDomains home.example.com
CanonicalizeHostname yes
CanonicalizeFallbackLocal no
HashKnownHosts no
UseRoaming no
# canonicalize* options seem to break github for some reason
# I haven't spent much time looking into it, so let's make sure it will go through the
# DNS resolution logic in SSH correctly.
Host github.com
  Hostname github.com.
Host *.maas
  Hostname %h.example.com
Host *.staging
  Hostname %h.example.com
Host *.maas.example.com
  User ubuntu
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null

Host *.staging.example.com
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
Host *.lxd
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  ProxyCommand nc $(lxc list -c s4 $(basename %h .lxd) | grep RUNNING | cut -d' ' -f4) %p
Host *.libvirt
  StrictHostKeyChecking no
  UserKnownHostsFile /dev/null
  ProxyCommand nc $(virsh domifaddr $(basename %h .libvirt) | grep ipv4 | sed 's/.* //; s,/.*,,') %p

As a bonus, I have included some code that makes it easy to SSH to local libvirt systems or lxd containers.

The net effect is that I can avoid having the warnings about changed hashes for MAAS-controlled systems and machines in the staging network, but keep getting them for all other systems.

Now, this means that to reach a host on the MAAS network, a client system only needs to use the short name with .maas tacked on:

vocal-toad.maas

And the system will be reachable, and you will not have any warning about known host hashes (but do note that this is specific to a sandbox environment, you definitely want to see such warnings in a production environment, as it can indicate that the system you are connecting to might not be the one you think).

It's not bad, but the goal would be to use just the short names. I am working around this using a tiny script:

#!/bin/sh
ssh $@.maas

And I saved this as "sandbox" in ~/bin and making it executable.

And with this, the lab is ready.

Usage

To connect to a deployed system, one can now do the following:

$ sandbox vocal-toad
Warning: Permanently added 'vocal-toad.maas.example.com,10.3.99.12' (ECDSA) to the list of known hosts.
Welcome to Ubuntu Cosmic Cuttlefish (development branch) (GNU/Linux 4.15.0-21-generic x86_64)
[...]
ubuntu@vocal-toad:~$
ubuntu@vocal-toad:~$ id mtrudel
uid=1000(mtrudel) gid=1000(mtrudel) groups=1000(mtrudel),27(sudo)

Mobility

One important point for me was the mobility of the lab. While some of the network infrastructure must remain in place, I am able to undock the Thinkpad X230 (the MAAS server), and connect it via wireless to an external network. It will continue to "manage" or otherwise control VLAN 15 on the wired interface. In these cases, I bring another small configurable switch: a Cisco Catalyst 2960 (8 ports + 1), which is set up with the VLANs. A client could then be connected directly on VLAN 15 behind the MAAS server, and is free to make use of the MAAS proxy service to reach the internet. This allows me to bring the MAAS server along with all its virtual machines, as well as to be able to deploy new systems by connecting them to the switch. Both systems fit easily in a standard laptop bag along with another laptop (a "client").

All the systems used in the "semi-permanent" form of this lab can easily run on a single home power outlet, so issues are unlikely to arise in mobile form. The smaller switch is rated for 0.5amp, and two laptops do not pull very much power.

Next steps

One of the issues that remains with this setup is that it is limited to either starting MAAS images or starting images that are custom built and hooked up to the raspberry pi, which leads to a high effort to integrate new images:

• Custom (desktop?) images could be loaded into MAAS, to facilitate starting a desktop build.
• Automate customizing installed packages based on tags applied to the machines.
• juju would shine there; it can deploy workloads based on available machines in MAAS with the specified tags.
• Also install a generic system with customized packages, not necessarily single workloads, and/or install extra packages after the initial system deployment.
• This could be done using chef or puppet, but will require setting up the infrastructure for it.
• Integrate automatic installation of snaps.
• Load new images into the raspberry pi automatically for netboot / preseeded installs
• I have scripts for this, but they will take time to adapt
• Space on such a device is at a premium, there must be some culling of old images

Call for testing: netplan.io in 18.04

Since 17.10, netplan has been the default network configuration tool in Ubuntu. Since then, it has grown in features, bug fixes, and even got its package renamed in the archive from "nplan" to netplan.io. We added better routing, improved handling for bridges, support for marking devices as "optional" for boot (so that the system doesn't wait for them to come up at boot time), lots of documentation updates... There's even been work to get it building for other distros.


We have a website for it, too: netplan.io


As we get closer to the release of Ubuntu 18.04, it is past due to involve everyone in testing netplan and making sure it is solid and as featureful as possible for a wide range of use cases.


This is where you get to participate.


Let us know about any feature gaps that remain in what
netplan supports, so that we can add the features when it's possible, or so that these feature gaps can be properly documented if they can't be closed by release time.


Report any bugs you find in netplan on Launchpad.


If you are unsure whether something is a bug, it might well be, so it doesn't hurt to file a bug. At the very least, we do want to know if something feels really difficult to do, so we can look into improving the experience.


If you're unsure how to do something you can look up questions and answers, or add your own, on AskUbuntu here:
https://askubuntu.com/questions/tagged/netplan


Netplan is being actively developed and we can use your help; so if there's one feature you care deeply about, or a bug that bugs you and you want to have a hand in fixing it, you can also jump right in to the code in Github: http://github.com/CanonicalLtd/netplan

Quick and easy network configuration with Netplan

Earlier this week I uploaded netplan 0.21 in artful, with SRUs in progress for the stable releases. There are still lots of features coming up, but it's also already quite useful. You can already use it to describe typical network configurations on desktop and servers, all the way to interesting, complicated setups like bond over a bridge over multiple VLANs...

Getting started

The simplest netplan configuration might look like this:

# Let NetworkManager manage all devices on this system
network:
  version: 2
  renderer: NetworkManager

At boot, netplan will see this configuration (which happens to be installed already on all new systems since 16.10) and generate a single , empty file: /run/NetworkManager/conf.d/10-globally-managed-devices.conf. This tells the system that NetworkManager is the only renderer for network configuration on the system, and will manage all devices by default.

Working from there: a simple server

Let's look at it on a hypothetical web server; such as for my favourite test: www.perdu.com.

network:
  version: 2
  ethernets:
    eth0:
      dhcp4: true

This incredibly simple configuration tells the system that the eth0 device is to be brought up using DHCP4. Netplan also supports DHCPv6, as well as static IPs, setting routes, etc.

Building up to something more complex

Let's say I want a team of two NICs, and use them to reach VLAN 108 on my network:

            network:

              version: 2

              ethernets:

                eth0:

                  dhcp4: n

                eth1:

                  mtu: 1280

                  dhcp4: n

              bonds:

                bond0:

                  interfaces:

                  - eth1

                  - eth0

                  mtu: 9000

              vlans:

                bond0.108:

                  link: bond0

                  id: 108

I think you can see just how simple it is to configure even pretty complex networks, all in one file. The beauty in it is that you don't need to worry about what will actually set this up for you.

A choice of backends

Currently, netplan supports either NetworkManager or systemd-networkd as a backend. The default is to use systemd-networkd, but given that it does not support wireless networks, we still rely on NetworkManager to do just that.

This is why you don't need to care what supports your config in the end: netplan abstracts that for you. It generates the required config based on the "renderer" property, so that you don't need to know how to define the special device properties in each backend.

As I mentioned previously, we are still hard at work adding more features, but the core is there: netplan can set up bonds, bridges, vlans, standalone network interfaces, and do so for both static or DHCP addresses. It also supports many of the most common bridge and bond parameters used to tweak the precise behaviour of bonded or bridged devices.

Coming up...

I will be adding proper support for setting a "cloned" MAC on a device. I'm reviewing the code already to do this, and ironing out the last issues.

There are also plans on better handling administrative states for devices; along with a few bugs that relate to support MaaS, where having a simple configuration style really shines.

I'm really excited for where netplan is going. It seems like it has a lot of potential to address some of the current shortcomings in other tools. I'm also really happy to hear of stories of how it is being used in the wild, so if you use it, don't hesitate to let me know about it!

Contributing

All of the work on netplan happens on Launchpad. Its source code is at https://code.launchpad.net/netplan; we always welcome new contributions.

Preseeding installations

In early February, I completed a move from Canonical's Phonedations team to the Foundations team. Part of this new work means debugging a lot of different failure cases in the installer, grub, and other early boot or low-level sofware, some of which requiring careful reproduction steps and probably quite a few install runs in VMs on on hardware.

Given the number of installations I do I've started to keep around preseed files; the text files used to configure automatic installations. I've made them available at http://people.canonical.com/~mtrudel/preseed/ so that they can be reused as necessary. Most of these preseed files make heavy use of the network to get the installation data and packages from the web, so they will need to be tweaked for use in an isolated network. They are annotated enough that it should be possible for anyone to improve on them to suit their own needs. I will add to these files as I run across things to test and automate. I hope we can use some of them soon in new automated QA tests where appropriate, so that it can help catch regressions.

For those not familiar with preseeding, these files can be used and referred to in the installation command-line when starting from a network PXE boot or a CDROM or pretty much any other installation medium. They are useful to tell the installer how you want the installation to be done without having to answer all of the individual questions one by one in the forms in ubiquity or debian-installer. The installer will read the preseed file and use these answers without showing the prompts. This also means some of the files I make available should not be used lightly, as they will happily wipe disks without asking. You've been warned :)

To use this, you'll want to specify "preseed/file=/path/to/file" (or just file=) for a file directly accessible as a file system or through TFTP, or "preseed/url=http://URI/to/file" (or just url=) if it's available using HTTP. On d-i installs, this means you may also need to add "auto=true priority=critical" to avoid having to fill in language settings and the like (since the preseeds are typically only read after language, country, and network have been configured); and on ubiquity installs (for example, using a CD), you'll want to add 'only-ubiquity automatic-ubiquity' to the kernel command-line, again to keep the automated, minimal look and feel.

I plan on writing another entry soon on how to debug early boot issues in VMs or hardware using serial. Stay tuned.

Call for testing: urfkill / Getting flight mode to rock on Ubuntu and Ubuntu Touch

Last month, I blogged about urfkill, and what it's meant to be used for.

The truth is, flight mode and proper killswitch (read: disabling radios on devices) handling is something that needs to happen on any device that deems itself "mobile". As such, it's one thing we need to look into for Ubuntu Touch.

I spent the last month or so working on improving urfkill. I've implemented better logging, a way to get debugging logs, flight mode (with patches from my friend Tony Espy), persistence, ...

At this point, urfkill seems to be in the proper state to make it, with the latest changes from the upstream git repository, into the distro. There is no formal release yet, but this is likely to happen very soon. So, I uploaded a git snapshot from the urfkill upstream repository into Trusty. It's now time to ask people to try it out, see how well it works on their systems, and just generally get to know how solid it is, and whether it's time to enable it on the desktop.

In time, it would be nice to replace the current implementation we have of killswitch persistence (saving and restoring the state of the "soft" killswitches) currently in two upstart jobs — rfkill-store and rfkill-restore — with urfkill as a first step, for the 14.04 release (and to handle flight mode on Touch, of course). In the end, my goal would be to achieve convergence on this particular aspect of the operating system sooner than later, since it's a relatively small part of the overall communications/networking picture.

So I call on everyone running Trusty to try to install the urfkill package, and file bugs or otherwise get me feedback on the software. :)

urfkill : a daemon to centrally control RF killswitches

Here's another project of the u-daemon variety. The latest addition to upower, udev, etc. Meet urfkill.

urfkill is meant to be a daemon that centralizes killswitch handling, rather than having all kind of different applications and daemon handle Wi-Fi, Bluetooth, WWAN and whatnot separately, and potentially fighting over them, you can have just one system that tracks the states and makes it possible to switch just one type of killswitch on all at one, or turn everything off should you so desire...

One reason I've taken an interest in urfkill in Ubuntu is that as we build a phone, we have to keep thinking about how users of the devices will be mobile. That's even more the case when you think about a phone or tablet than a laptop: on a laptop, you may have to think of WiFi and Bluetooth, but you're just about as likely to have your laptop off or not have it at all; whereas phones and tablets have become ubiquitous in our way of life.

Like anyone, thinking mobile I'd first think of walking around, driving, or other methods of travel. Granted, nobody needs to turn off Wi-Fi when getting in their car, but what about on planes?

This is the first thing everything brings up when talking about killswitches. Planes. Alright, you really do need to turn the device off on take off and landing, but some airlines do now allow wifi to be on and offer in-flight service. They still require you to keep cellular and bluetooth off. Also, while I sometimes do take my laptop out of my bag on long flights, it's just cramped. Space is at a premium on a flight (hey, I fly economy...), you'll likely want to have a drink, people besides you may need to get up, spillage could occur if there is turbulence...

I don't really enjoy using my laptop on a flight, even though it's quite small. It's just so much trouble and not very comfortable.

However, I do love to watch saved movies, listen to music, and play games on a tablet. That tablet will most likely need to have radios turned off. My phone will typically just stay off and stowed far enough, since I don't really change SIM cards until I can do so safely without risking to lose the thing.

But then, one can also think of how you should avoid using transmitting equipment in a hospital. They have similar rules about radios as planes to avoid interfering with cardiac stimulators, MRI equipment, etc.

Having all kind of different applications handle each type of killswitches separately is quite risky and complicated. How are you certain that things have been turned off? How do you check in the UI whether it's the case? Can you see it quickly by scanning the screen?

What about the actual process of switching things off? Do you need to go through three different interfaces to toggle everything? What do you need to do if you don't have a physical switch to use?

What about persistence after a reboot?

urfkill is meant to, in time, address all such questions. At the moment, it still needs a lot of work though.

I've spent the last day fixing bugs I could find while testing urfkill on my laptop, as well as porting it to logind (still in progress). In time, we should be able to use it efficiently in Ubuntu to handle all the killswitches. With some more work, we will also be able to use it to manage the modem and other systems on Touch.

For the time being, urfkill is available in the archive, for those who want to start experimenting with it.

Hacking with a Samsung ARM Chromebook on Trusty

So I decided it was about time to update / reinstall my Samsung Chromebook (the ARM one...) to Trusty, or at least to use Saucy. Turns out it's not that simple.

First, you need to know where to get the right stuff. I installed straight on the device, so chrUbuntu was the obvious choice. It's a pretty nice script that allows you to do just about anything necessary.

1) Bring your Chromebook to developer mode.

I'm not going to give the details. It's findable on the Internet, and unsafe enough that you should only do this if you know what you're doing... That counts double for running Trusty on the Samsung Chromebook.

From there, get into crosh (Ctrl+Alt+T), in shell mode (type shell at the crosh prompt).

2) Download the script:

cd ~/Downloads
wget http://goo.gl/s9ryd

3) Run the script:

sudo bash s9ryd xubuntu-desktop dev

This will do the gory install step, partition your device and format the new partition, download the ubuntu core tarball, and from there install the metapackage you've asked for as a first argument.

Be aware that if you have never repartitioned the device, you'll likely notice the system rebooting during the process -- if that happens, just re-run the same command to pick this back up where they ended. It's clear when the process is done and the system installed -- the script requires you to press Enter to reboot.

This was where things got fun.

Turns out my device booted fine into Trusty, but it would only show a black screen with the mouse cursor. If you moved the mouse, you could see the cursor changing but still nothing else. Switching to another VT (Ctrl-Alt-arrow (F1) or whatnot) would work to get you a text-mode login, but only if you switched early enough while X was getting ready to load... otherwise, you'd just get a pretty garbled display.

I hacked at the whole thing for a good while. I already know xf86-video-armsoc was involved in ChromeOS at some point, so I tried to install that.

Still no love. Tried to copy the libs from ChromeOS to the device, in case it was some libmali or EGL/GLES issue... Still nothing better.

I even touched /etc/X11/xorg.conf with some black magic, looking up the details using w3m in a text console...

Turns out the problem was with xf86-video-armsoc itself. I initially clued in when I looked at the dates for upload of the X packages and xf86-video-armsoc itself -- it didn't seem quite right: X was newer by a bit. I knew there could be some issue with the ABI in some cases; but after more careful investigation, that's probably fine too -- armsoc properly depends on -abi-14.

After much more work and trial and error, I updated xf86-video-armsoc to 0.6.0 from the Linaro git tree and also reverted one commit changing flags and it's not mostly working. X runs, I get lightdm, I can run apps -- "compositing" in Xubuntu works too, to get transparency and gradients... all with some minimal display corruption of the window decorations.

So the end of the line is -- if you want to run Trusty on your Chromebook and run into similar black screen issues, and you feel daring, feel free to try my newly-built xf86-video-armsoc package in my PPA:

https://launchpad.net/~mathieu-tl/+archive/ppa/+sourcepub/3627079/+listing-archive-extra

It's simple; once you're in a text console on the machine (login as user/user):

nmcli dev wifi connect <your wifi network> password <your wifi password>
sudo add-apt-repository ppa:mathieu-tl/ppa
sudo apt-get update
sudo apt-get install xserver-xorg-video-armsoc-exynos
sudo reboot

These updated packages, or at least some kind of permanent fix, should make it into Trusty soon. Stay tuned :)

Implementing MTP on Ubuntu Touch

Some of the fun parts of working at Canonical is how you get to work on so many different things.

I've spent the last few days working on an implementation of the MTP protocol for Ubuntu Touch, based on some amazing initial work from Thomas Voß to port the Android MTP libraries to C++.

I hadn't touched C++ in just about ten years. Last time was for school stuff, and it seems like things have changed a fair amount since, so I first had to get back in touch with the quirks of C++, learning Qt (which I used for my initial work), and learning Boost; in particular Boost::Filesystem, and some of the most fun pieces of Boost like BOOST_FOREACH, range adaptors, etc.

But MTP has been progressing very nicely, with code surely ready to land pretty soon in the Touch images. It was initially only exposing image files in /home/phablet/Pictures, but the latest code (still at lp:~mathieu-tl/mtp/images) now exposes all files in /home/phablet.

Not everything is implemented yet: for instance, you won't really have access to see or change permissions, copy or move files around the exposed filesystem, or delete files, but you can already browse and retrieve files, just like you can use the current MTP code to push files to an Ubuntu Touch device.

I'm thrilled at what's to come on that front, this is going to be a lot of fun.

If you're interested in testing this, feel free to take a copy of my bzr branch and experiment. You can build mtp simply:

cd mtp/

sudo apt-get install bzr bzr-builddeb debhelper build-essential cmake libboost-dev libboost-filesystem-dev

bzr bd

Let's keep working on making Ubuntu and the Touch images rock!

Call for testing: BlueZ A2DP and HSP/HFP profiles

We just landed bluez 4.98-2ubuntu2 in Precise. The key change was to enable the Source and Gateway profiles by default, which will now allow people to use their system as an audio output device. It works great on Android, but any help in testing this on a bunch of different devices (especially on iPhone) would be much appreciated!

The Source profile is what enables the A2DP bluetooth profile. With it, you can use your system as a stereo audio output device: in other words, you can send output audio from an external device such as a phone or a tablet to your computer (though this will require a bit of work on the PulseAudio side to get the audio stream to go through the right devices).

There's the AskUbuntu question "Can I use my computer as an A2DP receiver?" that describes the steps to use this right now. Thanks to Steve Langasek for figuring out the details. I've also written a draft script that implements the suggested steps and makes it simple:

#!/bin/sh

BTSOURCE=`pactl list short sources | grep bluez_source | awk '{print $2;}'`
SINK=`pactl list short sinks | grep -v Monitor | grep alsa_output.pci | awk '{ print $2; }'`

case $1 in
enable)
 pactl load-module module-loopback source=$BTSOURCE sink=$SINK
 ;;
disable)
 pactl unload-module $(pactl list short modules | grep loopback | grep $BTSOURCE | cut -f 1) || true
 ;;
esac 

That script is meant to be called as '"whatever_you_named_it" enable'. Don't forget the "enable" part, or "disable" to turn off the streams, otherwise it won't work.

The Gateway profile enables the HSP/HFP bluetooth profiles. This means we're getting closer to supporting phone calls from a bluetooth-connected phone on a Ubuntu computer. There's already some amounts of support for this via Ofono and the telepathy-ring project, although there is some extra work needed -- hopefully we can fix this by the Precise release. :D

Global Jam in Montréal

So we're doing it again!

It's obviously getting a bit on the late side to announce this, but I only got the confirmation for our location this weekend -- after work or when I had a second, I wrote the Ubuntu-Quebec mailing list invitation, dented about the event, and created the LoCo directory entry for this event already.

So once again, people wanting to help improve Natty Narwhal or who would like to know more about bugs, triaging, fixing bugs, testing the new release or even writing documentation for applications, and living around Montreal, Qc, don't hesitate to come join us at SUPInfo Montréal this weekend. The closest subway station is McGill on the green line, then about 3 minutes walking time ;)

We're holding our sessions on *both* Saturday and Sunday, from 11am to 5pm on each day.

SUPInfo Montréal
752 Sherbrooke West
Montreal, Quebec



Agrandir le plan

How to contribute to NetworkManager (or nm-applet)

Since NetworkManager (and all of the attached parts; VPN plugins, clients, backend...) is such a complex system, we're always looking for people to help out in whichever way they can.

One of the things I'd like to focus on this cycle (even if this announcement should have come long ago), is test cases and generally getting a good grasp of what works and what doesn't with NM, the clients, or plugins. Test cases are useful for us to know that things are performing as they should and even more important where development snapshots are involved, given the "in flux" nature of the code we're providing users. It's not that it's unstable code really (in fact, it's clearly not... 0.8 is now a stable branch that got tons of bug fixes, and seems rock solid if I believe the type and number of reports we get). Still, I'm interested to know about failures. That's partly why I wrote a number of test cases on the Desktop Testing Team tracker. There's a need for more though, and that's where the help of the community comes in.

If you know of some things that are important to you in NetworkManager, nm-applet, or the VPN plugins, write test cases for them! It will help everyone make sure those remain working, iron out the small annoying bugs, and just improve on the overall usefulness of NetworkManager.

Obviously, test cases and testing isn't the only place you can help. Are you annoyed at some little thing that works, but you'd love for it to go one step further in providing an amazing experience? Then we need you.

It's also very much the way I started contributing to Ubuntu, dealing with small annoyances which would make my life easier in using the desktop every day... And it was as simple and providing support for saving group or user password for the VPNC plugin.

I can already think of a few more of these small things, such as hiding the SIM PIN when prompted for it (since it's a password)...

I've started identifying small (but important!) bugs like this in Launchpad. As for other projects, look for the 'bitesize' tag. Here's an example.
They may not all be tagged bitesize, but if there's something that almost works for you and you want to take a look at fixing it, jump in! We'll all be glad you did. The same principle applies to many of the bugs marked as Wishlist, which are good ideas, maybe some need more discussion upstream, but they are all ready to be worked on (and have varying degrees of difficulty).

If you're looking to work on NM, the first thing you may want to look at is the wiki at http://live.gnome.org/NetworkManager. It's generally also a good idea to subscribe to the mailing list (see the previous link), so you can discuss changes with the community at large. Finally, there's truly no place more useful than the DebuggingTips page even for non-technical users. It's where you can poke NetworkManager to tell you exactly what goes on. In. Details.

In the cases where changes can be reasonably integrated into Ubuntu and/or Debian directly, feel free to submit merge requests and patches. This includes helping out fixing issues in the growing indicator patch for the applet :)

Finally if you're just curious and want to help out, you should know that there is a lot of work being done at the moment to simplify NM in various ways; all of which to in time become version 0.9. If you want to know more about the plans for NM 0.9, it's here: http://live.gnome.org/NetworkManager/ApiSimplify

When I'm connected I'm always happy to answer questions and help out in any way possible, both on NM and general desktop-ish stuff. You can reach me on #ubuntu-desktop or #nm on Freenode. I'm cyphermox.

Booting to ISO images from a USB key

I'm pretty sure this has been done before (I recall seeing a blog post about it... if you're the one who posted on Planet Ubuntu with a similar howto, let me know as I want to talk to you, at least to say thanks), but here it is anyway:

A couple of days ago I wrote this quick script to generate a roughly correct grub.cfg from the contents of a directory filled with .iso files. The goal: generate a USB bootable key that runs GRUB and allows you to choose which ISO to boot. It could be Ubuntu desktop, netbook, etc, doesn't matter, as long as you have enough room on the key.

This is done by leveraging the loopback grub command and the isoscan parameter. But first, setting it up...

You'll need (to):

• one free USB key, formatted to vfat (mkfs.vfat -F 32 /dev/sdX1) the more space the better
• create a directory called "iso" on it
• install grub on it:

  grub-install --root-directory=/media/MOUNTPOINT /dev/sdX

• this will create the boot/grub directories and install everything grub needs
• copy iso files to the iso/ directory
• run update-grub.py (available in bzr branch: lp:~mathieu-tl/+junk/bootable-iso-usb) from the key's mountpoint 
• Careful: it's only quickly tested, and overwrites boot/grub/grub.cfg from the current working directory.

Sorry for the unimaginative naming of the script.

In short, this script attempts to guess what kind of distribution is in the ISO file. I've tested desktop and alternate with success, both seem to boot and properly get you to an install (or for desktop, "maybe-ubiquity", which means you'll get a prompt for whether you want to run the live session or just ubiquity to install). All this needed was to make use of the iso naming scheme and more importantly of the contents of the ISO files, as read by isoinfo (from the genisoimage package).

Even if it never ends up being of any use, it was at least a fun little thing to write.

Ubuntu Global Jam in Montreal: help testing and fixing bugs in Maverick

We're doing it again!

Here in Montreal we're just about ready to start jamming this weekend. Rooms have been confirmed, and we already have a fair amount of interest from what I can see in the LoCo directory event listing! We're also happy to have been sharing ideas and plans with txwikinger who's been organizing the Global Jam event in the Waterloo/Kitchener area in Ontario.

Do you want to help make Ubuntu better?
Do you look forward to meeting some of the people who do testing, triaging and bug fixing on Ubuntu in Montreal, and maybe ask them to look at your own little pet-peeve bugs?

Then come join us at the SupInfo Montréal offices this Saturday, starting from 9am! All you need to bring is a computer and willingness to participate. :)

	SupInfo Montréal
	752 Rue Sherbrooke Ouest
	Montreal
	Canada
	http://www.supinfo.com/en/News0eaaf665-d9bf-4563-befb-ac97f8b035c2.aspx

If you choose to pass by, please let us know by registering for the event on the LoCo Directory... just so we know roughly how many people to expect.

You can get all of the information about our event on the QuebecTeam wiki: QuebecTeam/GlobalJam.

As a side note, we should have a local archive mirror, and copies of maverick and lucid ISO images. I've spent the past weeks preparing it on a portable USB hard drive, after getting some help in starting from Dustin Kirkland. Thanks Dustin!

Wubi: installing Ubuntu inside Windows

Do you know about Wubi?

Wubi is a very nice tool if you want to try out Ubuntu for the first time. It allows you to install Ubuntu alongside Windows, without re-partitioning your hard-drive. All this from the CD-ROM if you load it in computer while Windows is running, or downloadable as a standalone app.

The Ubuntu QA team is always looking for help to test installing Ubuntu using Wubi. It's both a great way to try Ubuntu if you really still need to run Windows on your computer and want to help out with testing Ubuntu, or as any kind of first jump into testing and QA. Testing efforts are being tracked on the Ubuntu ISO Tracker. If you want to help, please read the documentation first.

So if you chose to download Wubi's standalone application, double-click on its icon, or load in the Ubuntu CD-ROM you may have just burned (or received from shipit or your LoCo team).

If you're using a CD-ROM, Wubi should start automatically (or in Windows 7, you should see an autorun dialog).










The install application gives you three choices. The first one being to reboot your system and use the CD-ROM directly to demo and install Ubuntu, the third sends you to the Ubuntu website.

For the purpose of this post I will instead walk you through option 2, which is to install Ubuntu alongside Windows, from within Windows.

You will then be presented with all the questions that make sure the installer knows everything it needs to know about the installation of Ubuntu you want to do. Choose your username, password, and where you want to install Ubuntu (which Windows harddrive, C:, D:, or whatever), what language you want to see Ubuntu in, etc.


If you're using the standalone Wubi app, you will need to wait a bit while the Ubuntu CD ISO image is being downloaded. The image is necessary, as the standalone app does not contain the required packages for the install.




Wubi will then proceed to the next part of the installation, which is to copy everything it needs to your hard drive in a ubuntu folder on the hard drive you selected in the preceeding screen.





Once the first part of the installation is over, select what you want to do... my preference is to reboot right away and carry on with the rest of the installation procedure.





Once your computer has booted, you will be presented with a menu. That menu will time out and default to Windows after a few seconds. Choose Ubuntu to continue with the installation.




You are now booting into the real thing: at this point you are in Ubuntu, but the installation isn't quite done yet, so grab a (quick) coffee while the last tidbits get copied and configured.





At the end of the second part of the installation, your computer will reboot again, present you again with the first menu where you will want to choose Ubuntu, and show an additional menu where you can choose to start Ubuntu in recovery mode, should you need to fix a broken installation. From there, you can also go back to Windows.


Voilà, the installation is done. At this point, all that is left to do is click on your username in the lis and enter the password you chose back in Windows during the first phase of the installation to get to the Ubuntu desktop.