If all cores are used, it should be set to 0, which will use nproc
after the cpuset has been defined, and not nproc (which is the number
of cores/threads on the machine).
Now that MAKEFLAGS is defined at the beginning of the scriptlet,
there is no need to pass MAKEFLAGS as an envar to the scriptlet.
But still NINJAJOBS is not defined. In most cases, when we want
to use all cores, this is not a problem. But if N_PARALLEL is
defined, it needs to be passed. Note that we also pass
MAKEFLAGS in this case, because we use the old "wrt_xxx" function,
but it is not used.
Previously, the "update" target was just added to the end of
the "all" target prerequisites, so that it was run at the end.
But now, we enter make with some -j value in MAKEFLAGS, so that
the "update" target may well be run before the other targets, which
is wrong: for example if it is run before the "xxx-z-libxslt" target,
xsltproc is not found and the update target fails.
To fix, define "update" as the default target, and have it depend
on "all".
Now that we have MAKEFLAGS in the environment, "make" in chroot
is run with MAKEFLAGS set. This cause it to pass the -j value
into MAKEFLAGS to inferiors (scriptlets), but it is passed with
a --jobserver-auth value added. If we do not precede the command
running the scriptlet with a +, it passes --jobserver-auth=-2,-2
meaning that the jobserver is disabled, and "make" in the scriptlet
understands that it should only run one job at a time (defeating
the value of the -j flag). But even if we do precede the command
running the scriptlet with a +, it passes --jobserver-auth=3,4,
which shows it is running, but this jobserver will authorize only
one job at a time since the parent make contains the target
.NOTPARALLEL!
To work around this problem, tje only solution is to change the
MAKEFLAGS at the beginning of the scriptlet. Fortunately, lfs.xsl
knows what we want as job number (either $(nproc) or a fixed number),
so use it to set MAKEFLAGS.
When using "sh << EOF", what is between this line and EOF runs
with no attached terminal. But we need a terminal for running the
build. So we have to use "sh -c ". But then nesting of double
quotes, $, etc is too complicated. So run as root, and become back
user only when exec'ing. For some reason $@ does not work, so use
a variable set to $@.
Still WIP but some progress...
When moving a shell to a cgroup not associated with a session,
then subsequent calls to pam_systemd or pam_elogind create
a new session, and a new cgroup for that session, so that the
cgroup of the calling process is not used (this is a problem
with both systemd and elogind). For systemd, the problem can be
solved by passing --slice <user slice> to systemd-run. For elogind,
we need to first move the shell to a non session cgroup, then run
sudo so that a new session is created, then pass the cpuset to that
session's cgroup. Hopefully, if neither systemd nor elgind is used,
then the former solution should work (to be tested!!!).
If using a new book, MAKEFLAGS is set to -j$(nproc), so that if
we want to easure timings with 4 processors and our computer has
more than that, we need to restrict the number of processors.
Jhalfs allows to do that using th cpuset cgroup controller, so that
it only restrits the make job and not the whole machine. For that
use a small scripts written by Xi Ruoyao that restarts make this
the cpuset set.
There is no more $(nproc || echo 1). Note that we may replace
it with $(nproc) if using all cores, so don't replace it in
the comment-test template (otherwise we have infinite recursion).
- Put parallelization menu first
- Fix the logic for variable definitions in this menu
- change jhalfs to reflect the separation of optimization and
parallelization
- validate new variables
- remove MAKEFLAGS from the validation of optimize variables
- also remove the black list, which has not been used in years.
Our gen-special.sh transforms a dependency on compound packages
(xorg7-*) on a dependency on the list of packages. This generates
a lot of trials when removing circular dependencies, because if the
dependency on xorg7-* is optional, all packages in the list are tried
in turn, while it is sure that anyway the dependecy must be eliminated.
If the dependency is only on the last package, the circular dep
is eliminated at first try.
If the shell exits with an error, teardown is not run in target
chroot, which may lead to unwanted effects, such as preventing
unmounting the $BUILD_DIR. Tell make to ignore the error.
Now that devices.sh can be run even if the fs are already mounted
all the targets after mounting the virtual kernel fs can depend
on "devices". This way, if for some reason a partial build is
restarted after a reboot, the virtual kernel fs are automatically
re-mounted.
Fixes#1737
Those scripts could only be run once, because mount errors out
if it tries to mount an already mounted fs (or unmount a non
mounted fs). This changes generation of the scripts so that
any mount is preceded with "mountpoint -q || ", and any umount
is preceded with "mountpoint -q && ".
If the kernel-config file is not up to date (missing options or
whatever), the kernel "make" will hang, waiting for input. So
run "make oldconfig" explicitly, with a timeout.
Fixes: #1736
gen-special.sh uses the list of files and md5 from the
cat instructions to generate an xml page for each of
the individual components. Until now, there were no commented
out component, so gen-special.sh was not prepared to handle
comments. Now, it removes lines containing comments.
We use grep -l ^"${otherlink[*]"\$ to find parentNode, since the
parentNode is the only file with a link line that matches. Problem
is that sometimes the priority line may match too. This makes a
parentNode variable containing several lines, and when redirecting
to $parentNode, creates those "ambiguous redirect" messages.
Fix: use a loop, and only grep on the first line of each file.
Add several packages that shouldn't be removed when doing dependency
checking, two of which are actually not in lfs, but needed for
script: porg tzdata sudo wget
Add an XSL stylesheet for listing packages in LFS. Those are
not listed anywhere else, but they are considered dependencies of
everything, so should never be erased.
For outputting DESTDIR=, we need to test whether the text contains
"make". But when it contains "qmake" (or any different word containing
"make" but not exactly "make), we don't want to output anything more
than the text. So strengthen the test.
We output text()[1], literal (literally), text(2). This does
not work if there are several <literal> tags. Change it to
outputing preceding-sibling::text()[1], literal for each
literal tag, then output text()[last()] (which works also
when there are no literal tag, so it removes a choose.
For additional downloads, the normal book layout is a <bridgehead>
containing "Additional Downloads" followed by one or more
<itemizedlist>, one for each file to download. But on some pages,
there are several <bridgehead>, each followed by one or more
<itemizedlist>. So we have to check that only the <itemizedlist>
correponding to the current <bridgehead> are treated. Otherwise,
the download code may be generated more than once. This imposes
that no other bridgehead is inserted between the one containing
"Additional" (or in some case "Recommended/Optional Download") and
the <itemizedlist>.
