summaryrefslogtreecommitdiffstats
path: root/Makefile
Commit message (Collapse)AuthorAgeFilesLines
* Fix errors being ignored in many phases of the build since the bmake ↵bdrewery2015-12-011-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | integration. Say it with me, "I will not chain commands with && in Makefiles" This was originally fixed and explained quite well by bde@ in r36074. The initial bmake integration caused 'set -e' to stop being used which lead to r252419. Later 'set -e' expectations were fixed with bmake in r254980. Because of the && here, errors would be ignored when building in parallel and a dependency failed. Such as bootstrap-tools since it builds everything in parallel. If any tool failed in obj/depend/all, it would just ignore the error and continue to build. This later would result in cascaded errors that only confused the real issue. This could also cause commands after the failed command to still execute, leading to more confusion. This should be fine if the command is in a sub-shell such as: (cmd1 && cmd2) This reverts r252419. MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
* Rename META_MODE option to DIRDEPS_BUILDsjg2015-11-141-5/+5
| | | | | | | | | This allows META_FILES option to be renamed META_MODE. Also add META_COOKIE_TOUCH for use in targets that can benefit from a cookie when in meta mode. Differential Revision: https://reviews.freebsd.org/D4153 Reviewed by: bdrewery
* Move 'make cleandir' from Makefile to Makefile.inc1.bdrewery2015-11-091-33/+1
| | | | | | | | This leads the way for fixing cross-build cleanup, and eventually replacing 'cleandir' with it during the build. Sponsored by: EMC / Isilon Storage Division MFC after: 2 weeks
* Add built-in ccache build support via WITH_CCACHE_BUILD option.bdrewery2015-11-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
* Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.bdrewery2015-10-171-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | Bmake has a documented feature of '-N' to skip executing commands which is specifically intended for debugging top-level builds and not recursing into sub-directories. This matches the older 'make -n' behavior we added which made '-n -n' the recursing target and '-n' a non-recursing target. Removing the '-n -n' feature allows the build to work as documented in the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also not documented anywhere that I could see. Note that the ${_+_} var is still needed as currently bmake incorrectly executes '+' commands when '-N' is specified. The '-n' and '-n -n' features were broken for several reasons prior to this. r251748 made '_+_' never expand with '-n -n' which resulted in many sub-directories not being visited until fixed 2 years later in r288391, and many targets were given .MAKE over the past few years which resulted in non-sub-make commands, such as rm and ln and mtree, to be executed. This should also allow removing some indirection hacks in bsd.subdir.mk and other cases of .USE that have a .MAKE by using '+'. Sponsored by: EMC / Isilon Storage Division Discussed on: arch@ (mostly silence)
* Tweak the guard more to suggest 'all' if SUBDIR_OVERRIDE is specified. In thatbdrewery2015-10-161-1/+1
| | | | | | | | case 'all' does make sense. MFC after: 2 weeks X-MFC-With: r289411 Sponsored by: EMC / Isilon Storage Division
* Tweak the default target to not suggest 'all' since it really doesn't dobdrewery2015-10-161-2/+2
| | | | | | | anything useful for most users. MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
* Consider top-level targets to be .PHONY as bmake won't build thembdrewery2015-10-151-1/+1
| | | | | | | otherwise if a file with the same name is found in the directory. MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
* Add installconfig as a top level targetbapt2015-10-091-1/+1
|
* Fix the .MAKE added in r251750 to properly support the historical -n -n.bdrewery2015-09-291-2/+2
| | | | | | | | | | | The condition used matches the condition in sys.mk for setting _+_ to blank or +. With this -n will continue to not descend into Makefile.inc1, while -n -n will and cause Makefile.inc1's target to run with -n. MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
* Rework r287900 to keep arm64/aarch64 stable in the TARGETS list.bdrewery2015-09-171-14/+15
| | | | | | | This is relevant for makeman using the 'make targets' output in src.conf(5). This makes a _UNIVERSE_TARGETS that removes arm64 if the build requirements are not met.
* Get arm64/aarch64 into 'make targets' output to fix makeman.bdrewery2015-09-171-3/+3
|
* Fix generation of src.conf.5sjg2015-07-031-1/+3
| | | | | Since makeman turns all options on, we need to guard somethings from make(showconfig)
* targets now needs .PHONYsjg2015-06-221-1/+1
|
* Move the universe done dependency on the worlds completing out of theandrew2015-06-161-1/+1
| | | | | | loop, we only need to add it once. Obtained from: ABT Systems Ltd
* Remove unneeded dependencies, these are now handled through the _doneandrew2015-06-161-2/+0
| | | | | | target. Obtained from: ABT Systems Ltd
* Add a new target universe_${target}_done to print the completion message.andrew2015-06-161-0/+6
| | | | | | | Without this we could print this message in the wrong place when building with MAKE_JUST_WORLDS is set. Obtained from: ABT Systems Ltd
* Add a universe_${target}_worlds target to simplify the logic to find whenandrew2015-06-161-6/+4
| | | | | | to start building kernels. Obtained from: ABT Systems Ltd
* A more compatible fix to MK_META_MODE not being defined. Also, encaseimp2015-06-161-1/+6
| | | | | | | bmake specific constructs not needed for make bootstrap so fmake doesn't see them. This works with fmake just well enough for us to build bmake to build the rest of the tree without fatal errors. Tested only with fmake package.
* Add a comment to the end of the world and kernel cases in the universeandrew2015-06-161-2/+3
| | | | | | target to help follow the make magic. Obtained from: ABT Systems Ltd
* If MK_META_MODE is unset, assume a value of no.rodrigc2015-06-141-1/+1
| | | | | | | | | This is needed to build HEAD on FreeBSD 10.1, which has bmake, but does not have /usr/share/mk/src.opts.mk Reviewed by: sjg Reported by: jenkins
* Since sys.mk now handles META_MODE and META_FILES optionssjg2015-06-101-12/+8
| | | | | we need not worry about them here. Checking for .PARSEDIR no longer needed.
* Merge sync of headsjg2015-05-271-3/+14
|\
| * Don't add arm64 to universe builds if the user provided a TARGETS listemaste2015-04-281-4/+4
| | | | | | | | | | | | | | Differential Revision: https://reviews.freebsd.org/D2375 Reported by: andrew Reviewed by: andrew, imp Sponsored by: The FreeBSD Foundation
| * Fix a typo (linnker -> linker)ngie2015-04-271-1/+1
| |
| * Add arm64 to universe if binutils is available.emaste2015-04-161-0/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | arm64 relies on an external binutils port or package right now, because the in-tree linker from binutils 2.17.50 does not support arm64. Add arm64 to universe if the linker is available. If not output a message that arm64 is skipped. buildworld and buildkernel use the external binutils automatically, so it's sufficient to run 'pkg install aarch64-binutils' to build FreeBSD/arm64. Differential Revision: https://reviews.freebsd.org/D2302 Reviewed by: andrew, imp Sponsored by: The FreeBSD Foundation
| * Make TARGET_ARCH=powerpc64 work without TARGET=powerpc.imp2015-03-211-1/+1
| |
| * Start to import support for the AArch64 architecture from ARM. This changeandrew2015-03-191-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | only adds support for kernel-toolchain, however it is expected further changes to add kernel and userland support will be committed as they are reviewed. As our copy of binutils is too old the devel/aarch64-binutils port needs to be installed to pull in a linker. To build either TARGET needs to be set to arm64, or TARGET_ARCH set to aarch64. The latter is set so uname -p will return aarch64 as existing third party software expects this. Differential Revision: https://reviews.freebsd.org/D2005 Relnotes: Yes Sponsored by: The FreeBSD Foundation
* | If we are doing META_MODE use targets/Makefile as top-levelsjg2014-11-301-0/+6
| | | | | | | | since this one isn't suitable.
* | Merge from head@274682sjg2014-11-191-1/+3
|\ \ | |/
| * Revert r271159, Mis-patched the tree.kevlo2014-09-051-36/+0
| | | | | | | | Pointed out by: kib
| * The USB LED driver for the Dream Cheeky WebMail Notifier.kevlo2014-09-051-0/+36
| | | | | | | | Reviewed by: hselasky
| * Create the native-xtools target. This target creates only the crossimp2014-08-181-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | building toolchain for the host computer. This toolchain produces TARGET_ARCH and assumes the rest of the system contains libraries for the target. It is intended to be used in a "qemu-user jail" where all the binaries would otherwise be the target architecture's to build ports. However, emulation of the compilers is too slow, so we build native binaries for that. Rather than use the xdev produced binaries, with all their weird links and paths, these binaries use the native paths. They will not work unless installed into the qemu-user jail. Differential Revision: https://phabric.freebsd.org/D518 Reviewed by: sbruno@
* | Merge head from 7/28sjg2014-08-191-15/+18
|\ \ | |/
| * Add compat shims for XDEV and XDEV_ARCH so we don't break all theimp2014-07-241-0/+7
| | | | | | | | | | users of them. Also, add a note to updating. We'll keep these shims at least until the 12 branch.
| * Explicitly disable the build of tests when building bmake.jmmv2014-07-131-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | During "make buildworld", building bmake is (one of) the very first steps and we should not be building any of its tests. Conceptually, this is the right thing to do 1) for build simplicity reasons and 2) because there is no need to build any tests this early on. In practice, this fixes tinderbox builds of CURRENT from 9.x when MK_TESTS is enabled. This is because bsd.test.mk needs some modern bmake features not present in 9.x (:tW) and tinderbox is forcing the build to use the CURRENT share/mk files from the very beginning (see r266617). By skipping the build of the tests when still using the host make, we omit the problem. Arguably, what tinderbox is doing is wrong and needs to be addressed, but that is a separate issue.
| * Separate out the links creation from the other targets. This wasimp2014-07-101-0/+2
| | | | | | | | | | | | | | supposed to have been done for the original commit, but somebody forgot. Pointy-hat-to: imp@
| * Remove ia64 from the list of known architectures and add an entry tomarcel2014-07-021-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | UPDATING. This is the first step towards the removal of ia64 from head. A buildworld for ia64 will now yield: % make buildworld make[1]: "/usr/src/Makefile.inc1" line 151: Unknown target ia64:ia64. While here, trim the ia64-specific additions from ObsoleteFiles.inc Discussed at: BSDcan
| * grep -L returns non-zero status if none of the files had the patternimp2014-05-101-1/+1
| | | | | | | | | | | | in them. This is often the case, so just ignore the return code. Actual errors that are found will also be detected downstream in the rare cases where the return code is 2 instead of 1.
| * We haven't done anything with _UPGRADING in ~forever (was present, butimp2014-05-101-1/+1
| | | | | | | | | | not needed, in FreeBSD 6.x, and has been absent in newer versions). This was needed to upgrade from 3.x -> 4.x, once upon a time.
| * Remove a few more vestiges of allowing WITHOUT_BMAKE to imply you wantimp2014-05-101-11/+4
| | | | | | | | to buid with fmake.
* | Merge from headsjg2014-05-081-9/+20
|\ \ | |/
| * No need to install man pages for bootstrapping new make. Someimp2014-05-061-1/+1
| | | | | | | | | | | | | | | | | | | | environments (that I can't reproduce locally, but that others have reported) seem to get tripped up by this man page install. There's really no need to do it, so turn off the man pages using the most portable method. We can't just directly set MK_MAN=no here because we're bootstrapping in the host environment and such a setting was forbidden until very recently. NO_MAN= can produce a warning, but for now the warning is benign.
| * Remove support for WITHOUT_BMAKE. bmake is now the only make that canimp2014-05-061-5/+1
| | | | | | | | | | | | | | | | build world, so it is the only make we build or install. fmake is still in the tree, but disconnected, and upgrades from older systems that still have bmake has not been removed, but its state has not been tested (it should work given how minimal the work to upgrade to bmake is).
| * Omit from the universe build all config files tagged withimp2014-04-301-1/+7
| | | | | | | | | | | | | | | | | | | | | | #NO_UNIVERSE. Many of these config files are important examples, but add little to no regresive value to the intended purpose of UNIVERSE. We now build over 120 kernels during universe. There's really little to no value to this over building say 60 or even 30 of them (either is still a way too big number). This is especially true for kernels that are nothing more than including a common base and adding a static DTB file. Start by pruning 1/3 of the arm kernels that add little regresion value.
| * Merge from CheriBSD:brooks2014-04-291-2/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit 1d1b908107255ffdff4d17f015d8f057d73cc6cb Author: Brooks Davis <brooks@one-eyed-alien.net> Date: Fri Mar 28 16:24:45 2014 +0000 Add a long needed seatbelt. Exit with an error when make is called without a target at the top level rather than poluting the source tree and causing use confusion in future builds. commit a9d9aa341b2f4308a227ab460ba85f1f287ad028 Author: Brooks Davis <brooks@one-eyed-alien.net> Date: Tue Apr 29 16:06:12 2014 +0000 Simplify seatbelt added in 1d1b908 based in feedback. Discussed with: imp@FreeBSD.org Reviewed by: imp Sponsored by: DARPA, AFRL
* | If building WITH_META_FILES we do not want STAGINGsjg2014-05-051-1/+2
| |
* | Merge headsjg2014-04-281-19/+9
|\ \ | |/
| * Prevent building tests when bootstrapping make.jmmv2014-04-211-1/+2
| | | | | | | | | | Should fix the breakage reported by tinderbox when WITHOUT_BMAKE is set, which was probably introduced in r263346..
| * Document the xdev* targets.brueffer2014-04-141-0/+4
| | | | | | | | | | | | | | PR: 188519 Submitted by: Idwer Vollering <vidwer at gmail.com> Reviewed by: bapt MFC after: 1 week
OpenPOWER on IntegriCloud