376 lines
18 KiB
Plaintext
376 lines
18 KiB
Plaintext
1. INTRODUCTION::
|
|
|
|
To automate package builds from the BLFS book instructions is a huge
|
|
task. Some of the issues are: the BLFS book isn't linear; some package
|
|
pages use a custom layout; there are circular dependencies; several
|
|
packages can be installed on a non-default prefix; build commands can
|
|
change based on what dependencies will be used, etc.
|
|
|
|
That being said, the goal of the blfs-tool is to help you solve package
|
|
dependencies, create build scripts and a Makefile. Not all the auto-generated
|
|
build scripts and Makefile will work "as is", thus, as a general rule,
|
|
you will need to review and edit the scripts while reading the book.
|
|
|
|
The blfs tools allow also to update packages from the LFS book. LFS
|
|
packages which may be updated appear in the menu interface. When selected,
|
|
their scriptlet is generated in the same manner as for BLFS packages.
|
|
(TODO: presently, when an LFS package needs a patch, you'll have to
|
|
donwload it manually to your $SRC_ARCHIVE directory (usually /sources)).
|
|
|
|
2. PREREQUISITES::
|
|
|
|
In addition to a full LFS system, the following packages and their
|
|
dependencies are needed by this tool:
|
|
- required: libxml2, libxslt, DocBook XML DTD
|
|
- recommended: wget (to download the package tarballs) and sudo (to build
|
|
as a user)
|
|
- optional: lynx (allows to read the generated linearized book), GPM (to
|
|
cut and paste commands from the book), git (to update the book
|
|
sources)
|
|
Note that the optional dependencies are recommended for ease of use of the
|
|
tool.
|
|
|
|
You should also have the following personal skills:
|
|
- Ability to write and debug shell scripts: as said in the introduction,
|
|
not all the generated scripts can be used directly. They need to be
|
|
edited to produce an error free build.
|
|
- Ability to debug build failures, like missing dependencies or
|
|
installation directories not known to the system (when you install in
|
|
/opt for example).
|
|
- Ability to choose the tools you need to configure and administrate
|
|
your system: in the BLFS book, nothing is mandatory, nothing is
|
|
useless. You are on your own in choosing what to build, but wrong
|
|
decisions may lead to a non functional system...
|
|
|
|
3. INSTALL::
|
|
|
|
There are two ways to install the BLFS tools on an LFS system, described
|
|
in paragraphs 3.1 and 3.2, respectively:
|
|
|
|
3.1 INSTALLATION ON A RUNNING SYSTEM
|
|
|
|
Select "Use Book --> Beyond Linux From Scratch" in the jhalfs menu:
|
|
The tools are installed in $HOME$BLFS_ROOT (the default for $BLFS_ROOT
|
|
is /blfs_root). The BLFS book is downloaded or copied to its directory.
|
|
The tracking directory (see below) is created (if it does not already
|
|
exist) and initialized. Before running "make", you should ensure the
|
|
tracking directory (default location /var/lib/jhalfs/BLFS) can be:
|
|
- either created by the user running "make", if it does not exist
|
|
- or that it is writable by the user running "make", if it exists.
|
|
|
|
After the intallation, you should perform the following additional steps:
|
|
- Configure sudo, adding the needed privileges for the user.
|
|
- Although it is not strictly necessary, it is recommended to install
|
|
the bash shell startup files (as per `3. After LFS Configuration
|
|
Issues' of the BLFS book), as some instructions in BLFS rely on
|
|
their being present.
|
|
- At this point, the tool has no way to know which versions of LFS packages
|
|
are installed, so that the menu interface will show all the LFS packages,
|
|
as if they were not installed. If you have a released version of LFS, or
|
|
the date of your GIT version of LFS is known, you should run the
|
|
update-lfs.sh script. If you have updated some
|
|
LFS packages since first installation, or have been using a custom
|
|
working copy of the LFS book, the only (tedious) way is to create
|
|
empty files with names <package>-<installed-version> in the tracking
|
|
directory, and run the tool.
|
|
|
|
- If you have also installed some BLFS packages, they are not in the
|
|
tracking file. The only way is to create empty files with names
|
|
<package>-<installed-version> in the tracking directory, and run the tool.
|
|
|
|
3.2 INSTALLATION ON A JUST BUILT LFS SYSTEM
|
|
|
|
For books that support it (only LFS),
|
|
there is an option to install the BLFS tools right after building
|
|
the LFS system: just tick `BOOK Settings/Add blfs-tool support' in
|
|
jhalfs configuration menu. The tools are installed in $BLFS_ROOT
|
|
(default /blfs_root) on the LFS system, and the dependencies are built
|
|
at the end of the jhalfs run, before the custom tools.
|
|
|
|
After booting the new LFS system some steps are needed to finish
|
|
the installation of the automated tools:
|
|
|
|
- A user account must be created. You must be logged on that user
|
|
account to use blfs-tool. This is not strictly necessary,
|
|
since the packages can be built as root, too, but it is
|
|
never a good idea to build packages as root.
|
|
|
|
- Move /blfs-root to that user's home and change ownership of the
|
|
directory and files to the user.
|
|
|
|
- Give the user read and write privileges over the $TRACKING_DIR
|
|
directory and the files that it contains.
|
|
|
|
- Configure sudo and add the bash shell startup files, as described
|
|
above
|
|
|
|
- Note that the versions of LFS packages are automatically known to
|
|
the tool in this case, and there is no need to run the update-lfs.sh
|
|
script.
|
|
|
|
We assume that the BLFS tools will be used on a booted LFS system.
|
|
Using them to build BLFS packages in a chroot jail is also possible,
|
|
but not supported.
|
|
|
|
3.3 DIRECTORY LAYOUT IN THE $BLFS_ROOT DIRECTORY
|
|
|
|
blfs-xml/* GIT tree of the selected BLFS book version
|
|
lfs-xml/* GIT tree of the selected LFS book version
|
|
lib/constants.inc functions libraries
|
|
/func_dependencies for building the dependency tree
|
|
menu/* menuconfig source code
|
|
xsl/gen_pkg_list.xsl XSL stylesheet to generate the package database
|
|
/gen_config.xsl XSL stylesheet to generate the Config.in file
|
|
for use in the menuconfig system
|
|
/dependencies.xsl XSL stylesheet to generate the dependency list
|
|
of a package
|
|
/make_book.xsl XSL stylesheet to generate the linear book.xml
|
|
/lfs_make_book.xsl XSL stylesheet to incoporate LFS pages into the
|
|
linear book.xml
|
|
/scripts.xsl XSL stylesheet to generate the scriptlets from
|
|
book.xml
|
|
/bump.xsl XSL stylesheet to update the tracking file
|
|
/process-install.xsl XSL stylesheet included by scripts.xsl, for
|
|
outputting cleanly install instructions
|
|
/process-replaceable.xsl XSL stylesheet included by scripts.xsl, for
|
|
generating correct instructions when a
|
|
<replaceable> tag is encountered.
|
|
README.BLFS this file
|
|
TODO developers notes (well, not updated often)
|
|
gen_pkg_book.sh resolves dependencies, generates a linear BLFS
|
|
book, and finally generates build scripts
|
|
gen-makefile.sh generates the target Makefile
|
|
progress_bar.sh the target Makefile progress bar
|
|
gen-special.sh Helper script for generating the package
|
|
database
|
|
Makefile Used by make to update the package database
|
|
from the GIT tree, then launch the menuconfig
|
|
interface, and run gen_pkg_book.sh based on
|
|
configuration settings
|
|
packdesc.dtd a simple DTD describing the format of the
|
|
package database and the tracking file.
|
|
|
|
Working files: several files are generated when first running the tool:
|
|
|
|
packages.xml auto-generated package database
|
|
Config.in input file for the menu driven choices
|
|
configuration file generated by the menuconfig process
|
|
dependencies/* files recording the dependency tree
|
|
book.xml the linearized book
|
|
book-html/* the linearized book rendered in html
|
|
scripts/* the scriptlets
|
|
|
|
3.4 INSTALLED PACKAGES TRACKING SYSTEM:
|
|
|
|
This tool includes a very simple tracking system to log which packages
|
|
have been installed using the tool. It is used to skip installed packages
|
|
from target selection menu and to test if an installed package has been
|
|
updated in the BLFS book. Do not rely on this feature as a package
|
|
management tool.
|
|
|
|
The tracking system itself is an XML file: instpkg.xml. It is
|
|
initialized when <make> is first run in blfs_root. It resides in a
|
|
directory, which is created when needed during the process of building
|
|
custom tools or blfs dependencies, right after LFS. You can specify
|
|
that directory location in the blfs-tools sub-menu of jhalfs. You may
|
|
need to update permissions and/or ownership of this directory before
|
|
using the blfs tool (see README in jhalfs).
|
|
|
|
The default location of the tracking directory is /var/lib/jhalfs/BLFS.
|
|
NB : after the initial build, that directory is only used to contain
|
|
instpkg.xml, unless custom tools have been built. In the latter case,
|
|
it also contains empty files whose name are "$PKG-$VERSION" for each
|
|
versionned package built. The information about those packages is
|
|
included into instpkg.xml the next time the tool is run.
|
|
|
|
4. USAGE::
|
|
|
|
From now on, all the work must be done from inside the installation
|
|
root directory.
|
|
|
|
Due to the complexity of the BLFS book, the scripts and Makefile
|
|
generation is done in several steps:
|
|
|
|
4.1 UPDATING BOOK SOURCES::
|
|
|
|
If you are using the development book version and you want to update
|
|
installed packages to the latest version found in that book, you need to
|
|
update the XML sources and packages database. To do that, run
|
|
"make update". This is not necessary if you just built LFS, and you
|
|
can skip to step 4.2.
|
|
|
|
On the next configuration run, packages already installed but listed
|
|
with a new version in the book will be available for target selection
|
|
and used to solve dependencies.
|
|
|
|
4.2 CONFIGURING AND PARSING THE BOOK::
|
|
|
|
The next step is to create a book and build scripts in dependency
|
|
build order for one or several packages.
|
|
|
|
Run <make> to launch the configuration interface. The main menu contains
|
|
four blocks: individual package selection, Build settings, Build layout,
|
|
and Optimization.
|
|
|
|
In the package selection block, menus and submenus are organized
|
|
as the book's parts, chapters and sections. You can navigate those menus
|
|
and select as many targets as you want. But we suggest to not select
|
|
too many at a time to be able to sort issues!
|
|
|
|
In the "Build settings" submenu, the dependency level and default
|
|
packages used to solve alternatives are set (currently, only for the MTA).
|
|
You can also select whether the build will be made as a normal user or as
|
|
root, whether to use "porg style" package management, whether to remove
|
|
".la" files, and wheter statistics for the package are generated (build
|
|
time, memory footprint and "DESTDIR" install). If you use package
|
|
management, you have to enter the path to the packInstall.sh script too.
|
|
|
|
In the "Build layout" submenu, you can select where the source tarballs
|
|
reside and are downloaded, where the packages are built, and whether to
|
|
keep the build tree after installation.
|
|
|
|
In the "Optimization" submenu, you can select the number of parallel
|
|
jobs, and set the usual CFLAGS, CXXFLAGS, and LDFLAGS. the special
|
|
keyword "EMPTY" can be used for those flags to ensure they are unset.
|
|
|
|
Note that there are help strings associated to those menus. Please
|
|
read them for details!
|
|
|
|
Those settings are saved to be reused in future configuration runs.
|
|
|
|
When you are done with the menu, a few checks occur, and the dependency
|
|
chain is generated. Each dependency appears with its priority (required,
|
|
recommended, optional, or external), and it's level. There is a root level
|
|
1. The selected packages have level 2. The dependencies of selected packages
|
|
have level 3, the dependencies of the dependencies have level 4, and so on.
|
|
When circular dependencies are found, they appear with a priority of
|
|
"circular". This means that two (or more) dependency chains arrive at the
|
|
same package. The algorithm chooses the chain with the highest priority and
|
|
reorders dependencies to remove the other chain(s). This is not always the
|
|
solution an user would prefer, but we have found no way to do it better.
|
|
|
|
You end up with a book.xml file which contains the linearized book,
|
|
and a rendered HTML, in the directory book-html, which you can browse with
|
|
"lynx book-html/index.html" (or with any other browser).
|
|
|
|
Furthermore, there is a directory "scripts", which contains the generated
|
|
scriptlets.
|
|
|
|
There is yet another directory, "dependencies" that contains files
|
|
generated while resolving dependencies.
|
|
|
|
4.3 EDITING BUILD SCRIPTS::
|
|
|
|
Now it is time to review the generated book and scripts, making any
|
|
changes to the scripts necessary to fix generation bugs or to suit your
|
|
needs.
|
|
|
|
Scripts for additional packages (i.e., for non-BLFS packages) can be
|
|
easily inserted. For example, if you want to install the external dependency
|
|
"bar" before "foo" package and the "foo" script is named "064-z-foo", you
|
|
just need to create a "064-y-bar" build script.
|
|
|
|
Remember, the package tracking system isn't a package management tool
|
|
and knows nothing about packages not in the BLFS book.
|
|
|
|
4.4 CREATING THE MAKEFILE::
|
|
|
|
When the build scripts are ready to be run, the Makefile can be
|
|
created. Create an empty subdirectory (for example "mkdir work") and cd
|
|
to that directory. Then run ../gen-makefile.sh. Note that the directory
|
|
is completely emptied before generating the Makefile, so to prevent
|
|
erasing useful data, the script ensures that the name of the current
|
|
working directory starts with "work".
|
|
|
|
Review the Makefile, and, if all looks sane, start the build by running
|
|
"make".
|
|
|
|
5. GENERATED BUILD SCRIPTS ISSUES::
|
|
|
|
In this section, known issues with the generated build scripts are
|
|
discussed. They are due to build procedures and/or BLFS layout
|
|
particularities that we can't handle. In several cases, editing the
|
|
build scripts is mandatory.
|
|
You may also need to insert some build scripts created by you to resolve
|
|
unhandled dependencies and/or to remove some script installing an unneeded
|
|
package (unneeded packages may be pulled in the dependency chain, if
|
|
they occur as an "or" with another package).
|
|
When there are circular dependencies (only one known in BLFS 8.0 for
|
|
recommended dependencies), you may need to move around scripts so that they
|
|
run in the order script-A script-B script-A. This involves copying script-A
|
|
to another name (using the xxx-a- fields), and possibly renaming the xxx-a-
|
|
fields of each involved script.
|
|
|
|
5.1 BLFS BOOTSCRIPTS::
|
|
|
|
Normally, bootscript installation should work. On the other hand, the
|
|
book does not give instruction for running them, so you might have to
|
|
manually insert "/etc/init.d/rc.d/<initscript> start" at some place during
|
|
the build.
|
|
|
|
5.2 PACKAGE CONFIGURATION::
|
|
|
|
For those packages that have a "Configuration" section, you should
|
|
edit the build script to fit the needs of your system. Sometimes, the
|
|
bash startup files are modified. The generated scripts contain a
|
|
line 'source /etc/profile', which ensures that the proper environment
|
|
variables are used.
|
|
|
|
5.3 PAGES WITH TWO OR MORE PACKAGES::
|
|
|
|
For example: sane, poppler, audacious, freetts, which, etc.
|
|
|
|
On the pages for those packages, the BLFS book actually has instructions
|
|
to download and install two or more packages. You must edit the scripts to
|
|
fix this. A common pitfall is that the variable PACKAGE may be used for
|
|
several tarballs. Be sure to save the PACKAGE variable to some other
|
|
name (for example PKG1, PKG2, etc) after each download. The unpacking
|
|
instructions may need to be repeated for each tarball in turn.
|
|
|
|
5.4 XORG7
|
|
|
|
The book has special page layouts for the Xorg7 packages. The tool
|
|
breaks those pages into individual pages for each packages in the linear
|
|
book. Also, the menu gives the choice to select each package individually.
|
|
|
|
To build the whole Xorg7 chapter, select xinit. The (recommended)
|
|
dependency chain brings in the whole set of Xorg packages.
|
|
|
|
5.5 PATCHES
|
|
|
|
Please, make sure that all scripts have the commands to download/apply
|
|
the required patches. Due to book layout issues, some patches may be
|
|
missing (as of BLFS 8.0, all the patches seem to be downloaded).
|
|
|
|
5.6 ROOT COMMANDS
|
|
|
|
If building as a normal user (the default setting), be sure that all
|
|
commands that require root privileges are run using sudo. Also make sure
|
|
necessary root privilege commands are visible in your PATH. The scripts
|
|
ensure that /usr/sbin is appended to the user's PATH when running
|
|
privileged commands.
|
|
|
|
For commands necessitating root privileges, the generated scripts wrap
|
|
them with the construct:
|
|
sudo -E sh -e << ROOT_EOF
|
|
<commands to be executed as root with `$', ``', and `\' escaped>
|
|
ROOT_EOF
|
|
The "-e" switch to sh ensures the command block exits with error if an
|
|
error occurs. The "-E" switch to sudo ensures the whole environment is
|
|
passed to the commands to be run with root privileges. It is effective
|
|
only if the /etc/sudoers file contains `Defaults setenv', or SETENV in
|
|
the user attributes (this is implicit if the command the user is allowed
|
|
to run is `ALL'). If you think it is a security issue, you may forbid
|
|
this flag in /etc/sudoers, but then, you have to un-escape `$' for
|
|
variables coming from the environment in the instructions. Although this
|
|
construct is rather strong, it can fail in some corner cases, so
|
|
carefully review those instructions.
|
|
|
|
Due to book layout issues, some sudo commands may be missing.
|
|
|
|
5.7 OTHERS
|
|
|
|
There may be other issues that we are not aware of. If you find
|
|
any, please report it to <alfs-discuss@linuxfromscratch.org>.
|
|
|