Thursday 20 September 2018

Help needed to improve proposed migration

Hi!

Every once in a while, in the Foundations team, we do a coding day. A year ago, Lukasz and I wrote a script, following an idea from Steve Langasek, to provide "hints" and help for the next steps necessary for a package to migrate from -proposed to -release.

"ubuntu-archive-assistant" was born. I just pushed this to lp:ubuntu-dev-tools, after it being on its own in a separate git tree for a long while. I'd love to get help for feedback, as well as more people contributing fixes, etc. ubuntu-archive-assistant is designed to let you look at a specific package in -proposed and try to tell you what to do next to ensure it migrates from -proposed.

This is great work for new contributors wishing to work on something, say, to get upload privileges in Ubuntu.

Here's how it works (it uses subcommands right now).

Without any further options than "ubuntu-archive-assistant proposed", it will list packages in -proposed and let you pick:
$ ./ubuntu-archive-assistant proposed
No source package name was provided. The following packages are blocked in proposed:

(1) gnome-shell-extension-multi-monitors (Age: 338 days)
(2) node-is-glob (Age: 278 days)
(3) node-concat-with-sourcemaps (Age: 264 days)
(4) node-postcss (Age: 231 days)
(5) node-source-map (Age: 229 days)
(6) android-platform-system-core (Age: 226 days)
(7) libdigidocpp (Age: 226 days)
(8) qesteidutil (Age: 225 days)
(9) schleuder (Age: 218 days)
(10) ncbi-blast+ (Age: 216 days)
(11) node-postcss-filter-plugins (Age: 213 days)
(12) node-postcss-load-options (Age: 213 days)
(13) node-postcss-load-plugins (Age: 213 days)
(14) node-postcss-minify-font-values (Age: 213 days)
(15) node-postcss-load-config (Age: 209 days)
(16) live-config (Age: 207 days)
Page -1-. Press any key for next page or Q to select a package.
Which package do you want to look at? 9
Next steps for schleuder 3.2.2-1:
  Fix missing builds: amd64
     https://launchpad.net/ubuntu/+source/schleuder/3.2.2-1

If you specify which package you want to look at, it will give you the specifics for that package (examples here are for what is currently in -proposed):
$ ./ubuntu-archive-assistant proposed -s qesteidutil
Next steps for qesteidutil 0.3.1-0ubuntu4:
  Fix missing builds: amd64, arm64, armhf, i386, ppc64el, s390x
     https://launchpad.net/ubuntu/+source/qesteidutil/0.3.1-0ubuntu4
$ ./ubuntu-archive-assistant proposed -s android-platform-system-core
Next steps for android-platform-system-core 1:7.0.0+r33-2build1:
  Fix missing builds: amd64, arm64, armhf, i386
     https://launchpad.net/ubuntu/+source/android-platform-system-core/1:7.0.0+r33-2build1 

You can even get more information about the next steps for a package, by enabling --debug or --verbose:

$ ./ubuntu-archive-assistant proposed -s live-config                 Next steps for live-config 5.20180224:
  Fix unsatisfiable dependencies in live-config:

$ ./ubuntu-archive-assistant proposed --verbose -s live-config
live-config is not considered ✘
Next steps for live-config 5.20180224:
  Fix unsatisfiable dependencies in live-config:
    sysvinit-core | sysvinit (<< 2.88dsf-44) can not be satisfied on amd64 ✘
      sysvinit-core only exists in Debian ✘
 
$ ./ubuntu-archive-assistant proposed --debug -s live-config
live-config is not considered ✘
Next steps for live-config 5.20180224:
DEBUG: reasons: ['depends'] 
  Fix unsatisfiable dependencies in live-config:
    sysvinit-core | sysvinit (<< 2.88dsf-44) can not be satisfied on amd64 ✘
      sysvinit-core only exists in Debian ✘
         DEBUG: Is this package blacklisted? Should it be synced?

We've covered some of the common reasons for a package to be stuck in proposed, but there are a ton of others. We'll need help to improve the tooling and make it useful for everyone wishing to work on proposed migration. There's a lot more that can be done, including spending time to parse update_output.txt (or better yet, a YAML representation of it) and testing package installation automatically to figure out what packages need no-change rebuilds, etc. A lot of it is integration of other tools that already exist.

That's where you come in.

This is a great way to learn a lot more about what happens to packages after they are uploaded, and what more you can do to ensure your own uploads move quickly to be accessible to all Ubuntu users; and many of the improvements can be as simple as contributing a simple test for one failure case for packages in -proposed.

More to come about ubuntu-archive-assistant. There are other subcommands in it than just "proposed". :)