The Linux Foundation

 
Book/BinaryCompatibility

From The Linux Foundation

Contents

Ensuring Binary Compatibility

Binary compatibility is a feature assurance given by an LSB certified runtime environment to an application that you can take advantage of by using the LSB as guidance when coding your application. This is part of the promise and value of the Linux Standard Base. This chapter explains the concept of binary compatibility and its importance in developing LSB certified applications. It also describes each step in the process of ensuring binary compatibility and discusses the different source code standards the LSB specification adheres to. Finally, there is a discussion of binary versus source code compatibility.

Illustration of Binary Compatibility

Linux inherently has binary compatibility. Instead of porting or rebuilding source code from one Linux release or distribution to another, Linux has achieved binary compatibility between them all. You only have to build once for each Linux architecture, such as IA-32, PowerPC, or Itanium, and then deploy. The LSB has set some rules and guidelines that make this feature practical for applications. The path to shrink-wrapping an LSB application for Linux includes the following steps:

  • Coding to the portability layer
  • Using the correct ABIs
  • Testing with the LSB application checker
  • Following LSB packaging guidelines
  • Seeking LSB certification

The following sections describe each of these steps in detail. [1]

Coding to the portability layer

Rather than let applications access operating system resources directly, open systems such as UNIX and Linux have a portability layer for applications to be coded to. For UNIX, this coding layer is the set of POSIX application programming interfaces (APIs). However, since we know that "GNU's Not UNIX," the LSB has referenced POSIX for Linux in the form of the LSB specification standard.

Layers.png

Figure 1. LSB Layer

Applications must not use private interfaces of the operating system, or subvert the portability layer by accessing operating system resources directly. Some applications access system resources directly in order to be "platform tuned." Platform tuning an application may increase performance, but it greatly diminishes the possibility that the software will be binary compatible between systems or releases. This kind of software is sometimes platform-branded to work for a particular release of an operating system, which is fine if that is the intent. This style of programming; however, is outside the scope of the LSB.

Applications that are coded to the LSB specification have taken the first step to becoming binary compatible among the set of LSB-branded systems. By coding to the specification, the application is limited to using only the the following system libraries: libc, libdl, libm, libutil, libcrypt, libz, libpthread, libncurses, libX11, libXext, LibXt, libICE, libSM, and libGL.

If an application cannot limit itself to the interfaces of the libraries previously listed, then -- to minimize runtime errors -- the application must either bundle the non-specified library as part of the application, or it must statically link the library to the application. However, the libraries themselves must be LSB compliant by using only the interfaces of the libraries listed in the previous paragraph.

Using the correct ABIs

Restricting your development to the source API specification is not enough for binary compatibility because different releases and different systems have different versions of the libraries. To become binary compatible, you must develop to the ABI. Linux has the ability to version individual ABIs. So, if an imaginary function, myfunction(x,y), currently returns a double, but a more recent version of the ABI returns an integer, then you must use the version that was specified to return the original aggregate data type. To help you do this, the LSB has created stub libraries. You can link to the LSB stub libraries and to the LSB runtime linker. Then the LSB-specified ABI alone can be used. Either the application build will get an unresolved symbol error because it was using something not specified by the LSB, or the LSB stub libraries will ensure the correct ABIs per the binary specification.

To simplify the compilation of an application with the correct ABIs, the LSB provides an lsbcc wrapper script. This script uses the LSB stub libraries with the correct ABIs, the LSB runtime linker (ld-lsb.so.1), and header files corresponding to the LSB specification. You can download the lsb-base and lsb-cc packages, and then integrate them into your normal build procedure.

Testing with the LSB application checker

The LSB application checker, lsbappchk, is a test tool that checks for LSB ABI compliance. The lsbappchk tool compares the ABI symbols used by an application to ABI symbols defined by the LSB written specification. Following is an example "Hello World" application that uses the getpid() API. Listing 1. "Hello World"

Example 1. Helloworld

   #include <stdio.h>
#include <unistd.h>

main()
{
printf("hello world: %d\n", getpid());
}

Once compiled, you can check helloworld.c to determine if it is LSB compliant.

Example 2. Check for LSB Compliance

   $ lsbcc -o hw_good helloworld.c

$ lsbappchk hw_good
lsbappchk for LSB Specification 2.0
Checking binary hw_good

We see from the lsbappchk output that it did not find any compliance anomalies; however, if we were to slightly change the previous example to use the private function _getpid(), we see different results.

Example 3. _getpic Function

   $ lsbcc -o hw_bad helloworld.c
/tmp/cc5CITzio.o: In function 'main':
/tmp/cc5CITzio.o(.text_0xd): undefined reference to '_getpid'
collect2: ld returned 1 exit status

From the previous standard error of lsbcc (stderr) output, we can see that lsbcc would not compile the application with a non-compliant ABI. Using the LSB lsbcc tool helps you avoid creating non-compliant applications. But what if we used the native compiler?

Example 4. Native Compiler

   $ cc -o hw_bad helloworld.c

The native compiler allows the application to compile with the private interface _getpid()of the system, but we know from the LSB specification that this is wrong. In addition, we know that we should never use any interface that is prefixed with an underscore.

Example 5. ?

   $ lsbappchk hw_bad
lsbappchk for LSB Specification 2.0
Checking binary hw_bad
Incorrect program interpreter: /lib/ld-linux.so.2
Symbol _getpid used, but not part of LSB

From the previous example of the lsbappchk stderr output, we can see the application is associated to the wrong runtime loader and is using the non-compliant _getpid() ABI. So, even if you do not use lsbcc to catch non-conformance issues, you can still use lsbappchk later to validate the application. This methodology is not bulletproof, but it is a good indicator.

Following LSB packaging guidelines

Once your application is built using the LSB headers and linked with the LSB stub libraries and runtime loader, you can start packaging your application the LSB way. The LSB suggests that you are to package your application in an RPM v3 formatted file. In addition, you may not use triggers, nor depend on the execution order of pre-install or pre-uninstall scripts. You are also limited to using only commands specified by the LSB in those scripts and in your application, because other commands are not guaranteed to be present or to behave in expected ways. The LSB does not specify the tool to install these RPM-packaged applications. You can use the rpm tool on RPM-based systems, and alien on Debian. For more information on packaging, see the chapter called Packaging Your LSB Application.

In addition to avoiding name space collisions in the filesystem hierarchy, LSB-conforming packages are prefixed with "lsb-". If the name of the package only contains one hyphen, its name must be registered with LANANA. If the package name contains more than one hyphen, then the area between the first set of hyphens must be either an LSB provider name registered with LANANA, or your fully-qualified domain name in lower case. For example, lsb-java may be a name registered with LANANA by Sun Microsystems, but there may be another Java package name, lsb-unregistered.org-java, which is not registered. For more information on LANANA, see the Section called Package Naming in the chapter called Packaging Your LSB Application.

Seeking LSB certification

The last step in developing an LSB application is getting certified. Any certified LSB application can run on any LSB certified distribution. Today, there are dozens of certified distributions, and the LSB has built example conforming open source applications.

During LSB certification, you will complete the following steps:

  • Register yourself on the LSB Certification Web site.
  • Self-test your application using the LSB-provided tests, and then upload the results.
  • Warrant that it passes your own FVT.
  • Complete the Conformance Statement Questionnaire.
  • Sign the LSB Trademark License Agreement.
  • Receive confirmation from the Certification Authority.
  • Pay the certification fees and sign the LSB Certification Agreement.

The LSB trademark and can only be used by LSB Certified applications, no other statement of compliance can be made. For more information, see the Section called Achieving LSB Application Certification in the chapter called LSB Certifying for Software Products.

Notes

[1]

Portions of this chapter were derived with permisson from Five steps to binary-compatible Linux applications, George Kraft IV, IBM DeveloperWorks, October 2002

Source Code Standards

The standardization of a set of Application Programming Interfaces (APIs) for the purpose of recompile compatibility between systems is a source code standard.

POSIX

The Institute of Electrical and Electronics Engineers (IEEE) is the group that has and continues to develop the POSIX family of standards. Beginning in the mid-1980s, this group picked up on the work already done by the UNIX user organization UniForum (the 1984 /usr/group Standard) and began to produce an operating system standard under Project 1003 (POSIX).

The best known POSIX document is IEEE Std 1003.1 (also known as POSIX 1003.1), but the term "POSIX" actually covers a set of related operating system specifications. The various sub-projects instituted under 1003 are staffed by individual volunteers representing a cross-section of skills from hardware manufacture, OS development, software and tools development, academia, governments, and more.

[The name POSIX was suggested by Richard Stallman, one of the participants. It is pronounced pahz-icks, as in positive.]

As stated in the cover material for these standards, there were several principles guiding their development key among which were application source portability, specification of the interface without requiring a particular implementation, and reliance on historical practice.

Another goal is "consensus". There are some features of historical (and modern) UNIX-based operating systems, which are not found in POSIX, because no consensus could be reached at the time work was being done. This is the reason there was no mention of either "tar" or "cpio" in the 1003.2 specification, for example. Where there was no consensus, the feature was left out to produce a specification with a high enough level of approval to pass IEEE voting rules: - at least a 75% return on ballots - no more than 30% of eligible balloters abstaining - at least 75% YES votes.

More Information about the Portable Application Standards Committee (PASC), the guiding body for IEEE POSIX, is available online.

POSIX 1003.1 System Application Interface (C API)

This and 1003.2 (following in the POSIX 1003.2 Shell and Utilities section) are the base upon which the POSIX family of standards has been built. This is the "API and header files" part of the pair, the other being the "shell and utilities". In keeping with its original focus on the UNIX system, it is aimed at interactive timesharing computing environments.

POSIX 1003.1 was first published in 1988. A revised edition was published in 1990. The 1990 edition became an international standard, ISO/IEC 9945-1:1990.

In 1996, a new edition was created, which added Realtime and Threads features as optional extensions sets to the base 1990 standard. This also became an international standard, ISO/IEC 9945-1:1996. From this point other optional feature sets were created, but remained as separate documents (called "amendments" to the base 1990 standard) until 1999.

A major revision to the core base standard was approved in 1998, which would roll up the various amendments, options and interpretations. The proposal also included merging with 1003.2 to create one document encompassing the OS space. The result became IEEE POSIX 1003.1-2001 comprised of four volumes totalling 3760 pages.

POSIX 1003.2 Shell and Utilities

This specification defines a portable shell scripting environment (the so-called "POSIX shell") and a base portable set of utilities. In some places, where there was no consensus on the current tools (and a definite need was shown), a new tool was invented. The "pax" archive tool is an example of this.

The emphasis is again on defining things required to make applications portable. There is a section on the preferred syntax for new utility arguments and options. In some cases "user portability" was considered, such as the inclusion of the 'vi' editor.

Single UNIX Specification

The Single UNIX Specification (or SUS) is a set of specifications required for UNIX trademark compliance. It is developed and managed by The Open Group, an industry consortium and marketing group. The Open Group holds the definition of what a UNIX system is and its associated trademark in trust for the industry. The SUS specifies the programming and user environment that must be offered by an operating system before the system can use the UNIX trademark.

Version 3 of the Single UNIX Specification is the latest in a series of UNIX specifications that began with the X/Open Company Portability Guides (XPGs), which were operating system specifications created prior to, and then concurrently with, the initial POSIX efforts. In late 1998, The Open Group, IEEE, and ISO/IEC SC22/WG15 agreed to work together to form a single common set of specifications with the development group known as the Austin Group (described later in this section).

The Single UNIX Specification has always been aligned with the formal API standards such as POSIX and ISO C, being an upward compatible superset. It contains more APIs and features than POSIX, as well as making mandatory some features that POSIX keeps optional. POSIX Threads, for example, are required for UNIX trademark compliance. The SUS also covers X11, Curses, and many more mandatory utilities than POSIX.

A HTML version of the specification is freely available from The Open Group's Single UNIX Specification Web site.

More information about the SUS and the Open Group can be found online from their respective Web sites.

The Austin Group

"The Austin Group" is the name given to a joint working group created by the IEEE, ISO, and The Open Group in September 1998. These specification bodies agreed that the best thing for OS implementors and application developers was to have one set of documents that contained the text of the POSIX, ISO, and SUS specifications. In this way, conflicts and confusion between the standards could be reduced, and the work needed to maintain the various documents be centralized and possibly reduced as well.

This body consists of individual volunteers, including members of the free/open source communities. The Group is chaired by the ISO Project Editor, and the founding bodies (IEEE, ISO, and Open Group) each have one Organizational Representative. The ORs hold controlling votes, which are only used when issues without a clear consensus require a vote. IEEE and The Open Group hold joint copyright over the resulting work.

The Austin Group is the actual technical body that produced the standard known variously as IEEE POSIX 1003.1-2001, the Single UNIX Specification Version 3, and ISO/IEC 9945:2002.

Information on the Austin Group is available from its Web site.

Binary Standards

In the past there have been many binary standards such as System V Application Binary Interface for the MIPS(R) RISC processor, 88open for the Motorola 88000 RISC processor, and PowerOpen Environment for the PowerPC processor; however, none have been as successful as the Linux Standard Base. The LSB ensures a LSB certified application complied for a particular family of processor can run on any corresponding LSB certified runtime environment.

LSB Binary Compatibility

Binary Compatibility is the expectation that a properly-coded application, which executes sucessfully and correctly on one instantiation of an operating system, will continue to do so on another. The LSB is dedicated to the idea that Linux-based applications can be portable across Linux operating system distributions. By providing a specification that both operating system providers and application developers can agree upon, users are already benefitting from increased compatibility of their applications and tools.

Binary vs Source compatibility

Binary compatibility is not the same as source compatibility. A set of source code that describes the same operation on two instances of an OS can become a binary that will only operate on one of them, even on the same hardware platform. Binary compatibility is an "contract" between the OS and the application; the OS will provide a set of known function in a specified manner, and the application will use that function and avoid other functions for which there is no specification.

The LSB provides this contract. For the OS, it describes both the set of known function and the known manner. It does this via its programming interface (API) and binary interface (ABI) documents for each hardware platform. For applications, the LSB offers testing and certification to assure that only the functions provided for by the LSB are used.

Coding compatible applications

Applications using only LSB documented interfaces will run unmodified on all LSB certified OSes for a given hardware architecture family, as described by that LSB ABI specification architecture. Application software that meets this specification and testing successfully using the LSB Application Certification process can be considered to be portable on all members of that architecture.

This means that the application uses no undocumented interfaces, and as well, has no dependencies on specific types of hardware (for example, specific mass storage devices or specific I/O or networking adapters). This also means that only LSB certified OSes for that given architecture can be relied upon as a base for the application.

Forward Looking

Applications that are LSB certified are binary compatible within an LSB release set. The LSB does not support application compatibility between major releases. However, it may be possible for a Linux distribution to conform to multiple versions of the LSB at the same time.

Value proposition

In the past, the benefits of source code portability between variants of UNIXes and hardware platforms were well established. Today, there is no larger success demonstrating binary compatibility than Linux as specified by the LSB. Software development evolving from source code portability to binary compatibility is dramatically increasing product reliability in this multi-vendor Linux market. It is enabling the redirection of time spent on porting and testing to afford more time on product inovation.

"Shrink-wrapped" applications

Linux software that is LSB certified to be binary compatible can be "shrink-wrapped" and sold in retail stores with a longer shelf life and wider customer base. Dependencies to specific releases of certain Linux systems is obsolete.


[Article] [Discussion] [View source] [History]