From The Linux Foundation
Revision as of 20:05, 24 July 2013 by Herrold (Talk | contribs)

Jump to: navigation, search

This is an introductory guide to working with the LSB specification sources. It will walk you through obtaining the software you need, making a copy of the spec you can edit locally, modifying the spec, testing that the spec builds correctly, and merging your changes up into the official version control branches. It's not expected to answer every question, but will take you through the general steps.

Installing software

Distribution requirements for the packages needed to generate specification source and books vary somewhat. As an (old) example, the following packages were the only ones needed for Ubuntu Hardy Heron (8.04 LTS). Depending on the Linux distribution you're running, and which packages you already have installed, you may need to install a somewhat different set. The packages may also have different names in different distributions, so you may have to experiment. The Debian/Ubuntu name is given in parentheses below for each package after its human-readable name. Hey: updating this wiki with instructions for a particular distro, if they differ considerably from the ones here, would be helpful to all!

  • Bazaar, easy to use distributed version control system (bzr)
  • Collection of tools for Bazaar (bzrtools)
  • DocBook, standard SGML representation system for technical documents (docbook)
  • Modular DocBook DSSSL stylesheets for print and HTML (docbook-dsssl)
  • Convert Docbook files to other formats: HTML, RTF, PS, man, PDF (docbook-utils)
  • MySQL database server (mysql-server)
  • OpenJade implementation of the DSSSL language (openjade)

Thus, under Debian GNU/Linux or a Debian-based distrubution such as Ubuntu, the following command line should download and install most (all?) of the software you need to work on the specification, including dependencies:

# apt-get install bzr bzrtools docbook docbook-dsssl docbook-utils mysql-server openjade

Of course, if your distribution isn't based on Debian, you would use its standard installation method, such as yum for Red Hat or Fedora, zypper for OpenSUSE or SLES (or their graphically oriented equivalents).

Checking out branches

The LSB project uses a distributed version control system, where you create branches off the official LSB branches, and then do your work in those local branches. You need to make a directory to contain the spec branches, then enter the directory.

$ cd ~
$ mkdir LSB
$ cd LSB

Next, make a subdirectory within the main LSB directory for any groups of branches you will be editing, then make the local copies of the branches that you will need to build the documentation.

$ mkdir devel
$ cd devel
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/books
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/booksets
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/build_env
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/lsbspec
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/specdb

Important: If you are not working with the development branches, you will need to change the string devel in some of the lines above. If you don't, you may cause problems for everyone else using the LSB version control system if you try to push changes from a particular branche to the master branch for a different version. If you're lucky, such a push will fail, but if unlucky, it might work and creates a situation that's not desired. Just use the version number; for example for the LSB 4.0 branches use these checkouts:

$ mkdir 4.0
$ cd 4.0
$ bzr branch http://bzr.linuxfoundation.org/lsb/4.0/books
$ bzr branch http://bzr.linuxfoundation.org/lsb/4.0/booksets
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/build_env
$ bzr branch http://bzr.linuxfoundation.org/lsb/4.0/lsbspec
$ bzr branch http://bzr.linuxfoundation.org/lsb/devel/specdb

Notice that two of the branches are actually from devel, this is because they've been designed to be relevant to all versions of LSB and there are no version-specific branches of these. If you are going to be working on multiple versions it is suggested to branch them in a devel subdirectory and make symbolic links in the numbered version subdirectories. The specification sources themselves, and the generated specification documents, are of course version-specific and so separate branches are maintained for each version.

Of course there are many ways to approach this, the above is an example. If you're sure you're going to work only on devel, there's really no reason to make a subdirectory, but the examples in the rest of this page may assume you have, for consistency.

Creating the database

For some work, you will need a copy of the LSB specification database. It is possible to work against the official database on the Linux Foundation server, but for performance reasons it's vastly preferable to set up a local server. To create the database, first add the following lines to your profile (.bash_profile or the equivalent):

export LSBDB=lsb
export LSBUSER=root
export LSBDBHOST=localhost
export LSBDBPASSWD="yourpassword"

The last can be blank if you don't choose to set a MySQL password. Source the profile or open a new shell after you add these lines.

First time, you must also create the database, so as superuser, type the following:

# mysqladmin create lsb

Then enter the specdb directory, and type the following command (as yourself, but with the environment settings above):

$ make restore

This will populate your local copy of the database. Now you're ready to work on the spec documents.

Writing and editing documents

If you are merely editing an existing document, skip to the Checking in changes section below. However, if you wish to add a new document to LSB, you will probably take something like the following steps.

  1. Create/edit the file
  2. $ make gensrc
  3. $ make
  4. Test your changes by building the relevant book

Let's go through these steps one at a time. Suppose you are creating a "man page" for the Linux seq command.

Note: What we call man pages are not really meant to be displayed by the man command, but they are similar in scope and subject matter to Linux man pages, so LSB uses that terminology.

Create/edit the file

Starting in your LSB/devel directory if you are working in the development branches, you would enter the lsbspec directory, which is where most of the spec work is done, then the LSB subdirectory, which contains subdirectories for each of the architectures supported by the LSB, such as AMD4 and PPC32. Of these, select generic, which contains documentation applicable to all architectures. The file for the seq command is called seq.sgml and is located in the subdirectory command, which contains documentation for basic Linux commands. Finally, create the file seq.sgml. All together, type the following commands (for Emacs substitute your favorite text editor).

$ cd lsbspec/LSB/generic/command
$ vi seq.sgml

The seq.sgml file is a flat text file that contains DocBook SGML markup. DocBook markup is beyond the scope of this document. We'll just assume you have made your changes and saved the file. However, it should be noted there's a script called mklsbmanpage in the top level directory which can create a template for your manpage, including building the function prototype markup, if applicable, from the database.

Generate source from database

If you're adding to the specification, you need to regenerate a part of the spec source. If you're just editing existing documentation, skip this step (although it won't hurt anything).

$ make gensrc

This assumes that there is already a database entry for the new document (that is, for the interface or command the document will be referring to), this step pulls data from the database and interpolates it into the generated code. Adding to the official database is both a privileged operation and beyond the scope of this document, but it is certainly possible to write some simple sql commands to add a basic entity to the local copy of the LSB database.


Still in the same directory, type:

$ make

If all goes well, make will display a line of text like the following one.

m4 -P -Uindex -Uformat command.m4 >command.sgml

This shows that seq.sgml was incorporated into the file command.sgml, which "rolls up" all of the commands in the command directory into one file.

Test your changes

The lsbspec directory only contains the spec sources. The specifications LSB publish are built in the following two directories:

  • books - individual items of documentation
  • booksets - modules that roll up smaller pieces of documentation (that might be found in books)

In this example, we're working on the Core specification. Core is actually eight specifications, generic plus seven architecture-specific parts, but normally commands apply across all the architectures, so generic is the one affected here:

$ cd ~/LSB/devel/booksets/LSB-Core-generic
$ make

If you have made a syntax error in the file you were editing, the build process will fail and return a series of messages, ending in something like the following:

make: *** [htmlspec] Error 8

Unfortunately, because there's a two-step process in the source generation, the error messages are a bit unhelpful. For the example we've been following, you've edited seq.sgml, but the book is built from the combined (generated) file command.sgml, so you'll get an error message indicating a line number in the latter file and will have to fix it by figuring out what the corresponding line in the former file is. A tool could probably be built to help with this (contributions welcome!) but for now it's a manual step, probably editing each file in a separate window will make it easiest.

If the build process succeeds, you will see a series of messages that end without any error.

Now you can confirm your changes by opening the appropriate HTML file in the current directory with a web browser. From the command line, for example:

$ firefox LSB-Core-generic.html

An additional approach is to look at the text version of the generated spec. This will not show you all of the formatting generated by the markup, but is much easier to scan visually if you are trying to identify the way your changes have affected the overall specification, because any change tends to skew some of the generated tag content with the result that a diff of the html pages will be surprisingly large, and profoundly difficult to view visually. Since you're working in a version-controlled directory, the following command may help in producing a visually-useful diff from the last committed version (again, following this example):

$ bzr diff --diff-options=-dbBw --ifdef=ERRATA LSB-Core-generic.txt > LSB-Core-generic.txt.DIFF

If you poke through the .DIFF file with an editor, searching for the string ERRATA (if you prefer something else, change the command line just above) will show the places where a change has taken place.

When you are satisfied with the changes you've made, check them in, as described in the next section.

Checking in changes

Once you have made the changes you want to the specification, such as creating, updating, or deleting documents, you will probably take something like the following steps (the "add" step assumes you added something, adjust as needed)

$ bzr add pagesource.sgml
$ bzr commit .
$ bzr merge
$ bzr commit
$ bzr push https+urllib://bzr.linuxfoundation.org/lsb/devel/lsbspec

Let's go through these steps one at a time.

Record file changes

If you created a file, you need to tell bazaar about it, as in:

$ bzr add pagesource.sgml

If you're removing a file, you can do so by running:

$ bzr rm pagesource.sgml

It's not strictly necessary to do this, current versions of bazaar will detect a manually deleted file (just plain rm pagesource.sgml) and act on it at the next commit, but using bazaar itself to remove the file may be more clear in some cases.

If you're changing an existing file, there's nothing special that has to be done in this step, the commit step will just pick up the changes.

Commit changes

$ bzr commit file1 file2 file3

Using the bzr commit command followed by the name of every file you changed will create a new revision that contains your changes. If you leave out the list of filenames, bzr will offer to commit all the changes in the branch you're working in, which may or may not be what you want. To see what bzr thinks has changed relative to the last commit, do bzr status. The code example at the beginning of the section uses a version in between the two (bzr commit .</code), meaning commit all the changes in the current directory. Some specification changes affect more than one directory, for example adding an interface not only changes files in the relevant source directory, but also the matching appendix directory, because the appendix contains a list of interfaces. That is, an interface addition that appears in <code>lsbspec/LSB/generic/baselib will also mean a change in lsbspec/LSB/generic/appendix. The steps above about generating spec code haven't really covered this step, we imply that you do: make gensrc; make in lsbspec/LSB/generic/baselib, but it turns out that the complete approach is to do the same thing but one level higher, namely in lsbspec/LSB/generic.

If you're working on a lot of changes you may want to commit as individual parts start to work, you don't need to save up everything for one big commit. This is mostly a matter of working style.

LSB prefers (but does not require) that you cryptographically sign your commits. You set this up by defining a signature (email entry) that matches one found in your gpg signing key (in ~/.bazaar/bazaar.conf), and then enabling signing in the ~/.bazaar/locations.conf file, like this example stanza:

create_signatures = always
check_signatures = require

If the change you're working on is in response to a bugzilla entry, please include the bug number in the commit message. A typical, and acceptable, way to do this would be something like:

bzr commit -m "add frobnoz argument to fooble() (bug 9927)" file1 file2 file3

Merge with upstream version

$ bzr merge

Prior to pushing your changes up to the official branches, you need to make sure you have picked up any other changes that may have been pushed up by other developers while you were working disconnected in your own branches. This process is called merging. It's actually the same thing that happens when you push up your changes, but in the other direction (you'll merge your changes into the official branch instead of merging changes from the official branch into your branch). If there are changes in the official branch and you don't merge them, then your merge up to the official branch will simply fail ("these branches have diverged").

One way to do this is to use the merge command. This will merge any new revisions in the "parent" branch (the offficial branch) into your working branch. There are a few possible outcomes:

  1. no changes to merge
  2. changes were merged
  3. there were merge issues that bzr could not resolve automatically

In the latter case, you will need to work to resolve those issues. That's beyond the scope of this guide, so we'll skip that possibility for now.

Commit to finalize upstream changes

If you merge after committing your local changes, and bzr reports changes, you have a modified working tree (this is true whether or not you to have to resolve conflicts). The action to take in this case is to commit the changes, as long as you've convinced yourself they're reasonable and don't break what you've just done. For this, it's probably wise to first do a:

$ bzr diff

then a

$ bzr commit

Push your changes to the master branch

You need to have commit access to push to official branches. An LSB administrator will explain how this works.

Important: Again, if you are not working with the development branches, change the string devel in the URL above appropriately. If you don't, you may cause problems for everyone else on the project, if you end up modifying a branch other than the one you intended.

$ bzr push https+urllib://bzr.linuxfoundation.org/lsb/devel/lsbspec

The first time you push, bazaar will record the push path used so you don't have to type it again. To check what it's remembering, do bzr info and look for the push branch entry. If it seems to be remembering the wrong thing, you can force it with bzr push --remember path.


You have now successfully made changes in your own repository and merged them with the central repository. Hopefully this tutorial has made a little more clear how the distributed version control system bazaar (bzr) is used in maintaining the LSB specification sources.