Scripting around your Launchpad project

Iurii Kovalenko, 123RF.com

Iurii Kovalenko, 123RF.com

Lift Off!

Launchpad is a great way to get bleeding-edge and niche software for your Ubuntu, but it also provides a complete API, which allows you to write scripts to automate your access. We look into the Python 3 bindings for the Launchpad API.

Launchpad (LP) is well known to most people involved with Ubuntu development, because it's the main place for almost all Ubuntu core involvement. It's a very popular web service for driving and maintaining software projects – both open and closed source. It offers bug-tracking functionality, version control through Bazaar (bzr) or Git, translations, Debian package builders, private package archives, and many other features useful when working in a distributed environment. But, it's not just a useful tool for web users; Launchpad also offers a very rich API allowing scripts to access the same functionality available for humans through the web browser: the Launchpad API. In this article, I will concentrate on the most popular and convenient bindings: the Python launchpadlib .

Figure 1: Output and usage of the Launchpad bug script.

Quick Setup

Ubuntu currently ships two separate Python bindings for Launchpad: one for Python 2.x and one for Python 3.x. In all the examples here, I will use the latter, as there is no real reason not to switch to 3.x whenever it's possible. To get all the required bits in place, you need to install the python3-launchpadlib (or python-launchpadlib for 2.x) package along with all its dependencies [1].

sudo apt-get install python3-launchpadlib

No additional steps are needed, as all the required packages are in the main archives. The same can be achieved by using the pip installer.

pip3 install launchpadlib

for non-Ubuntu distributions.

First Application

I'll start with a very simple Python 3 example. Listing 1 shows a small script whose purpose is to write out the main title of a selected bug number. In Launchpad, each bug is assigned an unique bug number that identifies the bug in a global scope.

Listing 1

bug_details.py

01 #!/usr/bin/python3
02
03 import sys
04 from launchpadlib.launchpad import Launchpad
05
06 def main():
07         if len(sys.argv) < 2:
08                 print('Usage: bug_details.py BUG_NUMBER')
09                 exit(1)
10
11         number = sys.argv[1]
12         if not number.isdigit():
13                 print('BUG_NUMBER needs to be an integer.')
14                 exit(1)
15
16         lp = Launchpad.login_anonymously(
17                 consumer_name='bug-details-test', service_root='production',
18                 version='devel')
19
20         number = int(sys.argv[1])
21         try:
22                 print(lp.bugs[number].title)
23         except KeyError:
24                 print('Bug #{} does not exist.'.format(number))
25
26 if __name__ == '__main__':
27         main()

I'll look at the example in detail – you can skip the first argument check bits as those are unrelated to launchpadlib usage. First, obviously, you need to establish a connection to Launchpad itself. This can be done in two ways – anonymously or with selected credentials. The latter is required if your scripts need write privileges to modify something on Launchpad, like adding bugs, requesting package rebuilds, etc. Anonymous users only get read access, which is sufficient for the example case here.

To log in and get a Launchpad object to work, the login_anonymously() method is used. It generally takes three arguments: consumer_name , which can be any string defining our purpose (only used for statistics, can be anything); service_root , defining which LP instance should be used – the production one or staging; and, finally, version , which is the version of the API to be used. Here, I use the devel version, as the last stable version (which was 1.0 at this writing) is heavily outdated and not recommended.

The login method returns a Launchpad object that you can use for communication with the Launchpad API. The number of calls it provides is massive, but for now I only want to use it for fetching Launchpad bug information. The LP object I created has a property called "bugs," which is a collection storing all registered bug reports accessible through their unique numbers. Next, I try to access the "bug" object for the selected bug number from the bugs collection. If such a bug does not exist, Launchpadlib will throw a KeyError. A bug object has multiple interesting properties, one of which is "title," which I print out to the user before exit. Figure 2 shows a sample usage of the first script.

Figure 2: Launchpad prompt for authorization access.

The good part about the Launchpad API is also the documentation, which is always kept up to date [2]. All the properties and calls listed there are available both in the Python bindings as in the raw HTTP API.

Write Access

As mentioned earlier, you can also acquire Launchpad read-write access in your Python scripts. The only difference in the connection setup is how you get the Launchpad connection object.

lp = Launchpad.login_with(
        consumer_name='test',
service_root='production',
        version='devel')

When using login_with() instead of login_anonymously() , the provided arguments are almost exactly the same as before. The difference is that once the script is run by the user in an interactive environment, Launchpad will first try to authorize the connection by using your login credentials. This usually results in a new web browser tab opening up with an authorization request (Figure 2). The user can then choose to allow the computer and its script to access Launchpad for a selected period of time (or indefinitely).

If the script is to be run remotely, the login_with() method must be executed once on an interactive system with the credentials_file keyword argument filled to generate an authorization token to the selected file. That file is then your gateway to Launchpad login, so keep it safe from unauthorized access. Afterward, all operations on this script will be executed with your Launchpad user credentials.

Once launchpadlib is authorized, your scripts have now basically the same powers as you would have when accessing Launchpad through a web browser, allowing you to, for example, write bug comments, modify their contents, copy packages between PPAs, create merge proposals, and more. As an example, I'll slightly modify the bug printing script to a bug commenting application. After adding an additional command-line argument and using login_with() instead of login_anonymously , I just need to modify the part between the try-except statement like this:

bug = lp.bugs[number]
bug.newMessage(content=text)

After running the script with arguments – bug number and message – launchpadlib will create a new comment using the authorized user identity. As mentioned earlier, you're not limited to bug operations only, but this was the easiest thing to show as an example.

More Than Bugs

Almost anything that can be done manually on the Launchpad web site also can be realized through the API. My earlier examples concentrated on bug manipulation, but other tasks can be done as easily. A very convenient feature is the fact that most LP URLs from the web browser – such as bugs, PPAs, builds, users, teams, branches – can be easily loaded and used in your Python launchpadlib script with only a slight modification. The Launchpad object offers a load() method for this purpose.

This method takes a web service API URL and loads it, returning its corresponding Python object. For instance, if you're interested in the details of the unity8 package in the Ubuntu archives, the standard web URL for that is: https://launchpad.net/ubuntu/+source/unity8. To get the API URL for the same object, you prepend the URL with "api" and add the API version string right after the main domain part. You can then load it with the following code:

source = lp.load('https://api.launchpad.net/devel/ubuntu/+source/unity8')

The source object is now available for further usage, allowing you to search all bugs assigned to this source package or manage package bug subscriptions. When trying to open the URL in a web browser, an XML file from the web API will be shown – this is the raw representation that can be used when creating any other language bindings.

Another useful example from the Launchpad API is getting information about published source packages. Suppose you want to see all the unity8 versions published in the Xenial (16.04) series:

distro = lp.distributions['ubuntu']
xenial = distro.getSeries(name_or_version='xenial')
srcs = distro.main_archive.getPublishedSources(\
  source_name='unity8', distro_series=xenial, order_by_date=True, exact_match=True)
for src in srcs:
        print(src.source_package_version)

First, you fetch the Ubuntu distribution object, using it to get a series object for Xenial and then accessing its main_archive member for archive-related queries. The getPublishedSources call lists all source_package_publishing_history objects for source packages that fit the selected query. Here, you ask for all unity8 packages from the Xenial series, ordered by date (descending), and requesting an exact match – without this, LP API will also return all similarly named results. You then iterate over the returned collection and print out the version number for each package published in the archive.

The Launchpad Shell

Sometimes a convenient way to access the Launchpad API without having to write a complete script is needed. Think of it this way: You want to quickly gather data about several packages in a PPA. Doing it manually through the web browser would be tedious, whereas writing a separate script would be pointless if you only plan on doing that once. The good news is that a convenient tool lets you perform LP API calls from a Python interpreter: the lp-shell . Ubuntu users can easily get this tool through the lptools package available in universe.

sudo apt-get install lptools

The lp-shell script is easy to use – just run it from a terminal and a Python interpreter ready for LP API usage is now available for work. Just use the predefined lp variable, which has the Launchpad object initialized (Figure 3).

Figure 3: Interactive Launchpad shell.

By default lp-shell uses login_with() , so whenever you use the LP object, a login request will be initialized in your browser. If you want to use anonymous access instead, just run lp-shell with the -a command-line argument.

The interactive shell also supports some additional parameters. By default, the 1.0 LP API is used, so to get the latest API changes, you need to specify the API version in the command line instead.

lp-shell production devel

The first argument is always the Launchpad service to be used – it can be either production or staging. The second one is the API level to use. In most cases, devel is preferred because, as mentioned previously, the stable APIs are currently outdated, missing out on many of the interesting features and fixes.

Final Words

The LP API is vast and very useful for any advanced Launchpad user with basic Python knowledge. All this was made possible by the great Launchpad team working tirelessly on bug fixes and feature improvements. Big thanks to them for all their awesome work! Because Launchpad now supports both Bazaar and Git for code hosting, why not give it a try for your own project?

Infos

  1. Launchpadlib setup and basic tutorial: https://help.launchpad.net/API/launchpadlib
  2. Launchpad API documentation for the devel API version: https://api.launchpad.net/devel