370 lines
18 KiB
Text
370 lines
18 KiB
Text
$Id$
|
|
|
|
1. INTRODUCTION::
|
|
|
|
To automate package builds from the BLFS book instructions is a huge
|
|
task. Some of the issues are: the BLFS book isn't linear; some package
|
|
pages use a custom layout; there are circular dependencies; several
|
|
packages can be installed on a non-default prefix; build commands can
|
|
change based on what dependencies will be used, etc.
|
|
|
|
That being said, the goal of the blfs-tool is to help you solve package
|
|
dependencies, create build scripts and a Makefile. Not all the auto-generated
|
|
build scripts and Makefile will work "as is", thus, as a general rule,
|
|
you will need to review and edit the scripts while reading the book.
|
|
|
|
Since version 3.0 of jhalfs, the blfs tools allow also to update packages
|
|
from the LFS book. LFS packages which may be updated appear in the menu
|
|
interface. When selected, their scriptlet is generated in the same manner
|
|
as for BLFS packages.
|
|
|
|
2. PREREQUISITES::
|
|
|
|
In addition to a full LFS system, the following packages and their
|
|
dependencies are needed by this tool:
|
|
- required: libxml2, libxslt, DocBook XML DTD
|
|
- recommended: wget (to download the package tarballs) and sudo (to build
|
|
as a user)
|
|
- optional: lynx (allows to read the generated linearized book), GPM (to
|
|
cut and paste commands from the book), subversion (to update the book
|
|
sources), openssl (used by wget for all https:// sites)
|
|
Note that the optional dependencies are recommended for ease of use of the
|
|
tool.
|
|
|
|
You should also have the following personal skills:
|
|
- Ability to write and debug shell scripts: as said in the introduction,
|
|
not all the generated scripts can be used directly. They need to be
|
|
edited to produce an error free build.
|
|
- Ability to debug build failures, like missing dependencies or
|
|
installation directories not known to the system (when you install in
|
|
/opt for example).
|
|
- Ability to choose the tools you need to configure and administrate
|
|
your system: in the BLFS book, nothing is mandatory, nothing is
|
|
useless. You are on your own in choosing what to build, but wrong
|
|
decisions may lead to a non functional system...
|
|
|
|
3. INSTALL::
|
|
|
|
There are two ways to install the BLFS tools on an LFS system, described
|
|
in paragraphs 3.1 and 3.2, respectively:
|
|
|
|
3.1 INSTALLATION ON A RUNNING SYSTEM
|
|
|
|
Select "Use Book --> Beyond Linux From Scratch" in the jhalfs menu:
|
|
The tools are installed in $HOME$BLFS_ROOT (the default for $BLFS_ROOT
|
|
is /blfs_root). The BLFS book is downloaded or copied to its directory.
|
|
The tracking directory (see below) is initialized but not created: before
|
|
the installation, you should ensure the tracking directory (default location
|
|
/var/lib/jhalfs/BLFS) exists and is writable by the user. After the
|
|
intallation, you should perform the following additional steps:
|
|
|
|
- Configure sudo, adding the needed privileges for the user. For
|
|
newer sudo version, do not forget to add a line `Defaults secure_path='
|
|
containing /sbin and /usr/sbin (in /etc/sudoers), otherwise some
|
|
executables are not found.
|
|
|
|
- Although it is not strictly necessary, it is recommended to install
|
|
the bash shell startup files (as per `3.After LFS Configuration
|
|
Issues' of the BLFS book), as some instructions in BLFS rely on
|
|
their being present.
|
|
|
|
- In this case, the tool has no way to know which version of LFS packages
|
|
is installed, so that the menu interface will show all the LFS packages,
|
|
as if they were not installed. If you have a released version of LFS, or
|
|
the date of your SVN version of LFS is known, you should run the
|
|
update-lfs.sh script. If you have updated some
|
|
LFS packages since first installation, or have been using a custom
|
|
working copy of the LFS book, the only (tedious) way is to create
|
|
empty files with names <package>-<installed-version> in the tracking
|
|
directory, and run the tool.
|
|
|
|
- If you have also installed some BLFS packages, they are not in the
|
|
tracking file. The only way is to create empty files with names
|
|
<package>-<installed-version> in the tracking directory, and run the tool.
|
|
|
|
3.2 INSTALLATION ON A JUST BUILT xLFS SYSTEM
|
|
|
|
For books that support it (only LFS for jhalfs version 2.4),
|
|
there is an option to install the BLFS tools right after building
|
|
the xLFS system: just tick `BOOK Settings/Add blfs-tool support' in
|
|
jhalfs configuration menu. The tools are installed in $BLFS_ROOT
|
|
(default /blfs_root) on the xLFS system, and the dependencies are built
|
|
at the end of the jhalfs run, before the custom tools.
|
|
|
|
(TODO: blfs-tools have not been tested with current (version 3.0) of CLFS,
|
|
and certainly need some adaptation to run)
|
|
|
|
After booting the new xLFS system some steps are needed to finish
|
|
the installation of the automated tools:
|
|
|
|
- A user account must be created. You must be logged on that user
|
|
account to use blfs-tool. This is not strictly necessary,
|
|
since the packages can be built as root, too, but it is
|
|
never a good idea to build packages as root.
|
|
|
|
- Move /blfs-root to that user's home and change ownership of the
|
|
directory and files to the user.
|
|
|
|
- Give the user read and write privileges over the $TRACKING_DIR
|
|
directory and the files that it contains.
|
|
|
|
- Configure sudo and add the bash shell startup files, as described
|
|
above
|
|
|
|
- Note that the versions of LFS packages are automatically known to
|
|
the tool in this case, and there is no need to run the update-lfs.sh
|
|
script.
|
|
|
|
We assume that the BLFS tools will be used on a booted xLFS system.
|
|
Using them to build BLFS packages in a chroot jail is also possible,
|
|
but not supported.
|
|
|
|
3.3 DIRECTORY LAYOUT IN THE $BLFS_ROOT DIRECTORY
|
|
|
|
blfs-xml/* SVN tree of the selected BLFS book version
|
|
lfs-xml/* SVN tree of the selected LFS book version
|
|
lib/constants.inc functions libraries
|
|
/func_dependencies for building the dependency tree
|
|
menu/* lxdialog and menuconfig source code
|
|
xsl/gen_pkg_list.xsl XSL stylesheet to generate the package database
|
|
/gen_config.xsl XSL stylesheet to generate the Config.in file
|
|
for use in the menuconfig system
|
|
/dependencies.xsl XSL stylesheet to generate the dependency list
|
|
of a package
|
|
/make_book.xsl XSL stylesheet to generate the linear book.xml
|
|
/lfs_make_book.xsl XSL stylesheet to incoporate LFS pages into the
|
|
linear book.xml
|
|
/scripts.xsl XSL stylesheet to generate the scriptlets from
|
|
book.xml
|
|
/bump.xsl XSL stylesheet to update the tracking file
|
|
README.BLFS this file
|
|
TODO developers notes (well, not updated often)
|
|
gen_pkg_book.sh resolves dependencies and generates linear BLFS
|
|
books and build scripts
|
|
gen-makefile.sh generates the target Makefile
|
|
progress_bar.sh the target Makefile progress bar
|
|
gen-special.sh Helper script for generating the package database
|
|
Makefile Used by make to update the package database from
|
|
the SVN tree, then launch the menuconfig interface,
|
|
and run gen_pkg_book.sh based on configuration
|
|
settings
|
|
packdesc.dtd a simple DTD describing the format of the package
|
|
database and the tracking file.
|
|
envars.conf envars needed when running the target build scripts
|
|
|
|
Working files: several files are generated when first running the tool
|
|
|
|
packages.xml auto-generated packages database
|
|
Config.in input file for the menu driven choices
|
|
configuration file generated by the menuconfig process
|
|
dependencies/* files recording the dependency tree
|
|
book.xml the linearized book
|
|
book-html/* the linearized book rendered in html
|
|
scripts/* the scriptlets
|
|
|
|
3.4 INSTALLED PACKAGES TRACKING SYSTEM:
|
|
|
|
This tool includes a very simple tracking system to log which packages
|
|
have been installed using the tool. It is used to skip installed packages
|
|
from target selection menu and to test if an installed package has been
|
|
updated in the BLFS book. Do not rely on this feature as a package
|
|
management tool.
|
|
|
|
The tracking system itself is an XML file: instpkg.xml. It is
|
|
initialized when <make> is first run in blfs_root. It resides in a
|
|
directory, which is created when needed during the process of building
|
|
custom tools or blfs dependencies, right after xLFS. You can specify
|
|
that directory location in the blfs-tools sub-menu of jhalfs. You may
|
|
need to update permissions and/or ownership of this directory before
|
|
using the blfs tool (see README in jhalfs).
|
|
|
|
The default location of the tracking directory is /var/lib/jhalfs/BLFS.
|
|
NB : after the initial build, that directory is only used to contain
|
|
instpkg.xml, unless custom tools have been built. In the latter case,
|
|
it also contains empty files whose name are "$PKG-$VERSION" for each
|
|
versionned package built. The information about those packages is
|
|
included into instpkg.xml the next time the tool is run.
|
|
|
|
4. USAGE::
|
|
|
|
From now on, all the work must be done from inside the installation
|
|
root directory.
|
|
|
|
Due to the complexity of the BLFS book, the scripts and Makefile
|
|
generation is done in several steps:
|
|
|
|
4.1 UPDATING BOOK SOURCES::
|
|
|
|
If you are using the development book version and you want to update
|
|
installed packages to the latest version found in that book, you need to
|
|
update the XML sources and packages database. This is not necessary if
|
|
you just built xLFS, and you can skip to step 3.4.
|
|
|
|
To do that, run "make update". It may happen that the subversion
|
|
version of your building host is older than the version you just
|
|
built. This may generate weird errors like "'.' omitted". The easiest
|
|
thing to do in that case, is to completely remove the blfs-xml directory
|
|
and run "make update". With recent versions of subversion, you can also
|
|
run "svn upgrade" from inside the blfs-xml directory.
|
|
|
|
On the next configuration run, packages already installed but listed
|
|
with a new version in the book will be available for target selection
|
|
and used to solve dependencies.
|
|
|
|
4.2 CONFIGURING AND PARSING THE BOOK::
|
|
|
|
The next step is to create a book and build scripts in dependency
|
|
build order for one or several packages.
|
|
|
|
Run <make> to launch the configuration interface. The main menu contains
|
|
two blocks: individual package selection, and build options.
|
|
|
|
In the build options section, the dependencies level and default packages
|
|
used to solve alternatives are set (currently, only for the MTA). You can
|
|
also select whether the build will be made as a normal user or as root.
|
|
Those settings are saved to be reused in future configuration runs.
|
|
|
|
Note that you may select as many targets as you want, not just one
|
|
as in the previous version of this tool. But we suggest to not select
|
|
too many at a time to be able to sort issues!
|
|
|
|
When you are done with the menu, a few checks occur, and the dependency
|
|
chain is generated. Each dependency appears with its priority (required,
|
|
recommended, optional, or external), and it's level. There is a root level
|
|
1. The selected packages have level 2. The dependencies of selected packages
|
|
have level 3, the dependencies of the dependencies have level 4, and so on.
|
|
When circular dependencies are found, they appear with a priority of
|
|
"circular". This means that two (or more) dependency chains arrive at the
|
|
same package. The algorithm chooses the chain with the highest priority and
|
|
reorders dependencies to remove the other chain(s). This is not always the
|
|
solution an user would prefer, but we have found no way to do it better.
|
|
|
|
You end up with a book.xml file which contains the linearized book,
|
|
and a rendered HTML, in the directory book-html, which you can browse with
|
|
"lynx book-html/index.html" (or with any other browser).
|
|
|
|
Furthermore, there is a directory "scripts", which contains the generated
|
|
scriptlets.
|
|
|
|
There is also another directory, "dependencies" that contains files
|
|
generated while resolving dependencies.
|
|
|
|
4.3 EDITING BUILD SCRIPTS::
|
|
|
|
Now it is time to review the generated book and scripts, making any
|
|
changes to the scripts necessary to fix generation bugs or to suit your
|
|
needs.
|
|
|
|
Scripts for additional packages (i.e., for non-BLFS packages) can be
|
|
easily inserted. For example, if you want to install the external dependency
|
|
"bar" before "foo" package and the "foo" script is named "064-z-foo", you
|
|
just need to create a "064-y-bar" build script.
|
|
|
|
Remember, the package tracking system isn't a package management tool
|
|
and knows nothing about packages not in the BLFS book.
|
|
|
|
IMPORTANT: Review and edit envars.conf, at least after installing the
|
|
tool. This file is used to set global envars needed by the build scripts.
|
|
If you use package management, the variable JH_PACK_INSTALL should point to
|
|
the directory where the packInstall.sh script resides.
|
|
|
|
4.4 CREATING THE MAKEFILE::
|
|
|
|
When the build scripts are ready to be run, the Makefile can be
|
|
created. Create an empty directory (for example "mkdir work") and cd
|
|
to that directory. Then run ../gen-makefile.sh
|
|
|
|
Review the Makefile, and, if all looks sane, start the build by running
|
|
"make".
|
|
|
|
5. GENERATED BUILD SCRIPTS ISSUES::
|
|
|
|
In this section, known issues with the generated build scripts are
|
|
discussed. They are due to build procedures and/or BLFS layout
|
|
particularities that we can't handle. In several cases, editing the
|
|
build scripts is mandatory.
|
|
You may also need to insert some build scripts created by you to resolve
|
|
unhandled dependencies and/or to remove some script installing an unneeded
|
|
package (unneeded packages may be pulled in the dependency chain, if
|
|
they occur as an "or" with another package).
|
|
When there are circular dependencies (only one known in BLFS 8.0 for
|
|
recommended dependencies), you may need to move around scripts so that they
|
|
run in the order script-A script-B script-A. This involves copying script-A
|
|
to another name (using the xxx-a- fields), and possibly renaming the xxx-a-
|
|
fields of each involved script.
|
|
|
|
5.1 BLFS BOOTSCRIPTS::
|
|
|
|
Normally, bootscript installation should work. On the other hand, the
|
|
book does not give instruction for running them, so you might have to
|
|
manually insert "/etc/init.d/rc.d/<initscript> start" at some place during
|
|
the build.
|
|
|
|
5.2 PACKAGE CONFIGURATION::
|
|
|
|
For those packages that have a "Configuration" section, you should
|
|
edit the build script to fit the needs of your system. Sometimes, the
|
|
bash startup files are modified. The shipped 'envars.conf' contains a
|
|
line 'source /etc/profile', which ensures that the proper environment
|
|
variables are used.
|
|
|
|
5.3 PAGES WITH TWO OR MORE PACKAGES::
|
|
|
|
For example: sane, poppler, audacious, freetts, which, etc.
|
|
|
|
On the pages for those packages, the BLFS book actually has instructions
|
|
to download and install two or more packages. You must edit the scripts to
|
|
fix this. A common pitfall is that the variable PACKAGE may be used for
|
|
several tarballs. Be sure to save the PACKAGE variable to some other
|
|
name (for example PKG1, PKG2, etc) after each download. The unpacking
|
|
instructions may need to be repeated for each tarball in turn.
|
|
|
|
5.4 XORG7
|
|
|
|
The book has special page layouts for the Xorg7 packages. The tool
|
|
breaks those pages into individual pages for each packages in the linear
|
|
book. Also, the menu gives the choice to select each package individually.
|
|
|
|
To build the whole Xorg7 chapter, select twm. The (recommended)
|
|
dependency chain brings in the whole set of Xorg packages.
|
|
|
|
5.5 PATCHES
|
|
|
|
Please, make sure that all scripts have the commands to download/apply
|
|
the required patches. Due to book layout issues, some patches may be
|
|
missing (as of BLFS 8.0, all the patches seem to be downloaded).
|
|
|
|
5.6 ROOT COMMANDS
|
|
|
|
If building as a normal user (the default setting), be sure that all
|
|
commands that require root privileges are run using sudo. Also make sure
|
|
necessary root privilege commands are visible in your PATH. Or use
|
|
the `Defaults secure_path=' in /etc/sudoers.
|
|
|
|
For commands necessitating root privileges, the generated scripts wrap
|
|
them with the construct:
|
|
sudo -E sh << ROOT_EOF
|
|
<commands to be executed as root with `$', ``', and `\' escaped>
|
|
ROOT_EOF
|
|
The -E switch ensures the whole environment is passed to the
|
|
commands to be run with root privileges. It is effective only if the
|
|
/etc/sudoers file contains `Defaults setenv', or SETENV in the user
|
|
attributes. If you think it is a security issue, you may forbid this
|
|
flag in /etc/sudoers, but then, you have to un-escape `$' for variables
|
|
coming from the environment in the instructions.
|
|
Although this construct is rather strong, it can fail in some corner
|
|
cases, so carefully review those instructions.
|
|
|
|
WARNING: One variable from the environment is not passed through the
|
|
-E switch, namely PATH. This is because "sudo" always reset the PATH to
|
|
the default "secure_path". If you need to have the same PATH as the user
|
|
"root" would have, you may want to add "source /etc/profile" at the
|
|
beginning of the commands to be executed as root.
|
|
|
|
Due to book layout issues, some sudo commands may be missing.
|
|
|
|
5.7 OTHERS
|
|
|
|
There may be other issues that we are not aware of. If you find
|
|
any, please report it to <alfs-discuss@linuxfromscratch.org>.
|
|
|