Difference between revisions of "Packaging Software"
m (→Creating a Product Hierarchy)
|Line 171:||Line 171:|
This is, perhaps, the most important portion of the packaging process and is the reason for recording
This is, perhaps, the most important portion of the packaging process and is the reason for recording the softwareduring the build stage above. Find the ''package version'' of each dependency using the [http://techpubs.sgi.com/library/tpl/cgi-bin/getdoc.cgi?coll=0650&db=man&fname=/usr/share/catman/a_man/cat1/versions.z versions(1)] command. For example:
(prompt) versions -n neko_foo
(prompt) versions -n neko_foo
Revision as of 12:25, 23 December 2006
- 1 Introduction
- 2 Packages
- 3 Tools Required
- 4 Build Environment
- 5 Creating a Sandbox
- 6 Building the Software
- 7 Creating a Patch
- 8 Installing the Software
- 9 Package Preparation
- 10 Creating a Package
- 11 Tagging
- 12 Fixing Paths
- 13 Quickstarting
- 14 Testing and Building
- 15 Credits
- 16 Further Reading
This article provides a basic introduction to packaging software for distribution to other IRIX systems through the Nekoware software archive. The guidelines presented here are exactly that: guidelines. Please ask in the forums at Nekochan if you encounter any problems following this guide or need help building software from source.
IRIX packages are called tardists and, as the name implies, are essentially tar(1)'ed sets of distribution files. These files describe the location and permissions of all files related to a particular piece of software. These tardists can then be distributed and installed on similarly-configured SGI machines.
GNU/Linux, BSD and some other UNIX variants use the GNU C Compiler suite for building their software. In IRIX, gcc(1) does not work well for building widely distributed software packages. Nekoware tardists nearly always are built using SGI's compiler suite: MIPSPro.
SGI built some rather exotic machines and, as such, requires a more highly optimized compiler for code built to run on these machines. gcc(1) is designed to be a general-purpose C compiler suite that will build on nearly every machine made: past, present and future. This portability comes at a cost, however. gcc(1) tends to produce less well-optimized code for "specialty" hardware, such as those SGI produced. For this reason, MIPSPro should be used to build IRIX software wherever possible.
In order to build software for Nekoware, a version of IRIX greater than 6.5.21 and a R5000 MIPS processor, or better, is required. The primary branch of Nekoware is optimized for the MIPS4 architecture and compatible only with the environment provided by IRIX versions greater than 6.5.22.
Other tools may be required to build a particular piece of software that are not present in the IRIX environment. Such tools as GNU make and autoconf are commonly used to build Open Source software, but are not distributed with IRIX. If your software requires any of these tools, be sure to install the latest version from Nekoware and document the dependency on these tools during the packaging process.
Be sure to set some global variables for development on IRIX. The following is a set of sensible defaults:
export CC=c99 export CFLAGS='-O3 -mips4 -I/usr/nekoware/include -TARG:platform=IP27:proc=r10000' export CXXFLAGS=$CFLAGS export CPPFLAGS='-I/usr/nekoware/include -I/usr/include' export CXX=cc export F77=f77 export LDFLAGS='-L/usr/nekoware/lib -Wl,-rpath -Wl,/usr/nekoware/lib' export PKG_CONFIG=/usr/nekoware/bin/pkg-config export PKG_CONFIG_PATH='/usr/nekoware/lib/pkgconfig' export PKG_CONFIG_LIBDIR='/usr/nekoware/lib' export LD_LIBRARY_PATH='/usr/nekoware/lib' export LD_LIBRARYN32_PATH='/usr/nekoware/lib' export LD_LIBRARY64_PATH='/usr/nekoware/lib64' export GNOME2_DIR='/usr/nekoware' export GNUMAKE='/usr/nekoware/bin/make'
Furthermore, there are some MIPSPro options that should be set:
Also present are additional optimizations being done for the R10000 platform (more specifically the Origin200 IP27), which is a common processor among the O2, Octane, and Origin platforms. This build will work on R5000, but will see additional optimization on R10000 or better. If the software is optimization-friedly, e.g. mplayer or blender - it is possible to provide different builds, optimized for specific processors.
Alternatively, the -TARG option could be disgarded entirely to build without any platform-specific optimizations whatsoever. Many Nekoware packages use only -03 -mips4.
Creating a Sandbox
Typically, a location under your home directory would be used for holding source tarballs, build directories and other working files. Under this directory, creating two copies of the build directory will aid the patch creation process later on. Suppose we were building fooware, version 1.2.3, within this working directory:
(prompt) gunzip -c fooware-1.2.3.tar.gz | tar xf - (prompt) cp -r fooware-1.2.3 fooware-1.2.3-PATCHED
We've now created two identical versions of the source directory. By maintaining the original source and working within the "-PATCHED" directory, creating a patch for the package will be much more simple.
Building the Software
Typically, after a sandbox has been created, the software should be configured from within the "-PATCHED" directory (following the directions above). Remember, all Nekoware packages are installed relative to /usr/nekoware, so the --prefix option should be passed to configure, like so:
(prompt) ./configure --prefix=/usr/nekoware
Be sure to specify any other configure arguments necessary to build the software. It may be necessary to write down any dependancies configure checks to create accurate rules later.
After the software has been successfully configured, compile the source as usual. As an example, something like this might be used to compile the software on a dual-processor machine:
(prompt) gmake -j3
If the software successfully compiled, it may be possible to simply run the software from within the working directory, without installing anything. If so, try that now and verify that all desired functionality is present. If not, the software should now be installed using something similar to the following:
(prompt) su (prompt) gmake install (prompt) exit
After installation, test the software as above to verify the successful build.
Creating a Patch
Often, another developer or packager will want to re-create a package, perhaps from a newer release of the software. If drastic changes have been made to the source code, this could lead to duplication of work and frustration for the future packager. To help them along, the original source code and a patch to generate the buildable source code should be included in the package. To generate this patch, change directories to the parent of the sandbox and the clean source code and issue the following (using the example above:
(prompt) diff -urnp fooware-1.2.3 fooware-1.2.3.-PATCHED | \ grep -v "^Only" > neko_fooware-1.2.3_irix.patch
Having the clean source code makes this process much easier. However, the above command ignores any new files that may have been created in the "-PATCHED" directory. To include these files in the patch, review the manual page for diff(1).
Installing the Software
To easily find what files and directories are created when the software is installed, it is best to create a temporary directory, typically under /tmp, to install to. This directory will contain only the new file structure created by the installation routine. An IRIX package requires a list of all files created by the software and this temporary directory will easily provide that. Assuming write access to /tmp, issue the following from the working directory ("-PATCHED" above):
(prompt) mkdir /tmp/build (prompt) gmake DESTDIR=/tmp/build install
The full application tree is now created under /tmp/build, populated only by the files created by the software's installation routine.
Nekoware packages use a consistent directory structure for patches, sources, release notes and distribution files. Create this structure under the /tmp/build directory (created above), now:
(prompt) mkdir /tmp/build/usr/nekoware/patches (prompt) mkdir /tmp/build/usr/nekoware/src (prompt) mkdir /tmp/build/usr/nekoware/relnotes (prompt) mkdir /tmp/build/usr/nekoware/dist
Copy the original source distribution and the patch file to their respective directory:
(prompt) cp neko_fooware-1.2.3_irix.patch /tmp/build/usr/nekoware/patches (prompt) cp fooware-1.2.3.tar.gz /tmp/build/usr/nekoware/sources
Using another Nekoware package as a template, copy a Release Notes file and edit it to suit the software. Common information to include is:
- The packager's name and contact address
- The software's name, version and description.
- Information about the build machine
- The version of the MIPSPro suite and the IRIX build environment
- The environment variables you set - see before
- Extra build notes: if building the software required more than the patch, what else needs to be done?
- Remarks about known bugs or missing features
- Known dependencies
Once this file is finished, copy it to the appropriate directory:
(prompt) cp neko_fooware.txt /tmp/build/usr/nekoware/relnotes
Creating a Package
Now that the temporary directory contains the binary software, the source code, the patch and a release notes file, create a distribution directory and run Software Packager:
(prompt) mkdir /tmp/dist (prompt) swpkg
The Software Packager interface is designed to be intuitive, however we'll walk through the interface and provide more explanation where necessary. For more information about Software Packager, read the manual provided in the External Links section of this article.
Creating a Product Hierarchy
On the first worksheet, a tree is displayed showing a default arrangement of subsystems within a package. This may, or may not, suit the software's layout, but it is provided as a starting point for creating a new package.
The first step is to give the package a name on the root node. Something like neko_fooware is recommended. Add a description to this node on the right using other packages in the Software Manager as a guide. Be sure to assign any changes or they will be lost.
Create the other nodes according to the typical Nekoware hierarchy, as follows:
neko_fooware.sw.eoe (binaries and files for execution only) neko_fooware.sw.hdr (headers) neko_fooware.sw.lib (shared libraries) neko_fooware.man.manpages (manual pages) neko_fooware.opt.src (the original source tarball) neko_fooware.opt.patches (patches against the original source) neko_fooware.opt.renotes (release notes for this package) neko_fooware.opt.dist (distribution files, generated by swpkg)
Not all of the tree may be required by the software. Use common sense and other Nekoware package as a guide.
Every package will have two version numbers that should never be confused: the package version and the software version.
The package version is assigned to all children of the root node on the first worksheet of the Software Packager. This number is incremented for each build of this package independantly of the software version contained within the package.
For example, the software version of this application is 1.2.3, however the initial package version is 1. If, in the future, another package is built for this software, the package version will be incremented to 2. In this future package, the software version may be 2.0, 1.2.4 or remain at 1.2.3.
Remember: a package's version is independant of the version of the software it contains.
Also, all package version numbers must be equal within a package. If, for example, the execution-only subsystem has changed in a new package, all other subsystem's must have the version numbers incremented, as well. This holds true even if no other subsystems have changed between package versions.
This is, perhaps, the most important portion of the packaging process and is the reason for recording the software's dependencies during the build stage above. Find the package version of each dependency using the versions(1) command. For example:
(prompt) versions -n neko_foo (prompt) versions -n neko_bar (prompt) versions -n neko_baz
Each versions(1) command will output several lines, one for each subsystem in the package. Write down the version numbers for each subsystem the software depends on. Be sure to be specific. Which subsystem does the software depend on? Just the shared libraries, or the execution environment? Both?
Now, in Software Packager, specify the depedencies for each subsystem in the Rules section. The rules will look somewhat like the following:
replaces self prereq ( neko_fooware.sw.lib 1 maxint neko_foo.sw.lib 2 maxint neko_bar.sw.eoe 7 maxint neko_baz.sw.hdr 1 maxint )
There is also an "install by default" checkbox that should be checked for any subsystems that require it. Typically, the sw.eoe, sw.lib, sw.hdr and man.manpages will have this option checked.
Before continuing, be sure to save the .spec and .idb files that Software Packager creates to /tmp/build/usr/nekoware/dist. This will make them available to the next step.
Every file that the software creates and installs must be tagged with a particular subsystem within the package. Above, we created a clean application tree in /tmp/build for this very purpose. Had we not, each file that the software installed would need to be found in /usr/nekoware!
Import the entire /tmp/build directory and tag each file. For example, files installed in lib/ would be tagged with sw.lib, those in bin/ and share/ should be in sw.eoe and include/ is tagged with sw.hdr.
All directory entries should be removed from the distribution entirely. Leaving them is considered sloppy and clutters the package.
Set all files as owned by the root user and sys group and save the .spec and .idb files once more and exit Software Packager.
The current package now contains all files installed by the software with the correct permissions and tagged with the correct subsystem. However, due to using the sandbox, /tmp/build, all of the paths are incorrect. Thankfully, correcting them is an easy proposition:
(prompt) cat /tmp/build/usr/nekoware/dist/neko_fooware.idb | \ sed 's/\/tmp\/build//g' > tmp.idb (prompt) mv tmp.idb /tmp/build/usr/nekoware/dist/neko_fooware.idb
Alternatively, the .idb file can be opened using a text editor and the necessary replacements done using the editor's find-and-replace functionality.
Re-launch the Software Packager and open the .spec and .idb files. Verify that the paths are correct.
During most package installations using inst(1M) or swmgr(1M), certain files need to be "Re-quickstarted" using rqsall(1). When creating a new package, certain files must be marked as not needing this process. In the software's fileset, any files that are not ELF binaries should be marked with the norqs option.
Testing and Building
Save the .spec and .idb files once more and move to the next worksheet. This worksheet provides the final steps in Software Packager. Click the Test button to test the package build process. It is likely that the process will complain about files missing from /usr/nekoware. Move to a shell and copy these files from /tmp/build into their respective directories in /usr/nekoware and run the test again.
Once the test routine passes without complaint, click the Build button and exit Software Packager. The distribution files for the package should now be in /tmp/dist. Change to that directory and archive these files:
(prompt) cd /tmp/dist (prompt) tar -cf neko_fooware-1.2.3.tardist *
The next, and final, step is crucial. Change to the working directory, "-PATCHED" above, and uninstall the software:
(prompt) su (prompt) gmake uninstall (prompt) rm /usr/nekoware/dist/neko_fooware.* (prompt) rm /usr/nekoware/src/fooware-1.2.3.tar.gz (prompt) rm /usr/nekoware/relnotes/neko_fooware.txt (prompt) rm /usr/nekoware/patches/neko_fooware-1.2.3_irix.patch
Now, install the package as though it were a normal Nekoware tardist:
(prompt) inst -f /tmp/build/neko_fooware-1.2.3.tardist
Verify that the software installs correctly and runs as before with no errors. If all goes well, the package may now be uploaded to the Nekochan FTP site for testing by other users and, hopefully, inclusion in the archive. Congratulations!
This content originally appeared here and has been reproduced with the permission of the original author.