summaryrefslogtreecommitdiffstats
path: root/Makefile
Commit message (Collapse)AuthorAgeFilesLines
* Follow-up r298220: Don't pass down META_MODE which will still enable it.bdrewery2016-04-181-0/+1
| | | | | Reported by: Nikolai Lifanov <lifanov@mail.lifanov.com> Sponsored by: EMC / Isilon Storage Division
* META_MODE: Disable during installworld and similar.bdrewery2016-04-181-0/+8
| | | | | | | | | META_MODE may create cookies during staging of files to WORLDTMP that would also prevent installation of the files to the final DESTDIR, since the cookie already exists. This is not yet the case but will be soon. Prevent other similar issues by disabling META_MODE for any top-level install targets. Sponsored by: EMC / Isilon Storage Division
* MFHgjb2016-04-161-3/+3
|\ | | | | | | Sponsored by: The FreeBSD Foundation
| * Mark some more .PHONY targets.bdrewery2016-04-141-2/+2
| | | | | | | | Sponsored by: EMC / Isilon Storage Division
| * Define the *soft targets properly.bdrewery2016-04-141-1/+1
| | | | | | | | Sponsored by: EMC / Isilon Storage Division
* | MFHgjb2016-04-041-1/+1
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Remove the old depend (mkdep) code and make FAST_DEPEND the one true way.bdrewery2016-03-301-1/+1
| | | | | | | | | | | | | | Reviewed by: emaste, hselasky (partial), brooks (brief) Discussed on: arch@ Sponsored by: EMC / Isilon Storage Division Differential Revision: https://reviews.freebsd.org/D5742
* | MFHgjb2016-03-141-4/+7
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Use the newly minted Makefile.libcompat to implement libsoft librariesimp2016-03-121-1/+2
| | | | | | | | | | | | | | | | for the armv6 ABI switch. This also make WITH_LIBSOFT functional on the arm platform. As a transition thing, this seems to work even without switching the ABI (we basically build the same libraries twice when MK_LIBSOFT=yes until the ABI cut over next month). MK_LIBSOFT remains default no.
| * Avoid MK_TESTS error on stable/10 by just preventing SUBDIR recursion.bdrewery2016-03-111-1/+1
| |
| * Avoid bmake upgrade NO_MAN warning by just setting MAN to empty.bdrewery2016-03-111-1/+1
| | | | | | | | Suggested by: imp
| * Revert r296645 as it breaks stable/10->head builds.bdrewery2016-03-111-2/+1
| |
| * FAST_DEPEND: Use .dinclude to enable full .depend logic in bmake.bdrewery2016-03-111-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The inclusion of .MAKE.DEPENDFILE (.depend) has special logic in make to ignore stale/missing dependencies. bmake 20160220 added a '.dinclude' directive that uses the special logic for .depend when including the file. This fixes a build error when a file is moved or deleted that exists in a .depend.OBJ file. This happened in r292782 when sha512c.c "moved" and an incremental build of lib/libmd would fail with: make: don't know how to make /usr/src/lib/libcrypt/../libmd/sha512c.c. Stop Now this will just be seen as a stale dependency and cause a rebuild: make: /usr/obj/usr/src/lib/libmd/.depend.sha512c.o, 13: ignoring stale .depend for /usr/src/lib/libcrypt/../libmd/sha512c.c --- sha512c.o --- ... This rebuild will only be done once since the .depend.sha512c.o will be updated on the build with the -MF flags. This also removes -MP being passed for the .depend.OBJ generation (which would create fake targets for system headers) since the logic is no longer needed to protect from missing files. Sponsored by: EMC / Isilon Storage Division
| * Fix bmake upgrade NO_MAN warnings.bdrewery2016-03-111-1/+2
| | | | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
| * Fix make -n upgrade_checks.bdrewery2016-03-111-1/+1
| | | | | | | | | | MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
* | MFHgjb2016-03-101-1/+1
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Don't ever create object directories here with MK_AUTO_OBJ.bdrewery2016-03-081-1/+1
| | | | | | | | Sponsored by: EMC / Isilon Storage Division
* | MFHgjb2016-03-021-4/+3
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Add order for installworld/installkernel.bdrewery2016-02-241-0/+1
| | | | | | | | | | MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
| * Support a WANT_MAKE_VERSION.bdrewery2016-02-241-4/+2
| | | | | | | | | | | | | | This will be used soon for .dinclude support in FAST_DEPEND. MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
* | Include a 'package-pkg' target, intended for use forgjb2016-02-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | architectures we do not provide upstream pkg(8) packages. This is not tied to anything as-is, and likely will break your system if used (based on experience with testing with powerpc). There is an overwhelming amount of evil happening here, so until the issues are fixed, it will not be tied into the 'packages' target. Sponsored by: The FreeBSD Foundation
* | MFHgjb2016-02-101-1/+2
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Fix make universe when running with non POSIX/C locales using a locale sensitivebapt2016-02-101-1/+2
| | | | | | | | | | | | | | pattern Reported by: many Submitted by: jilles
* | Add logic to rotate the package repository, keyed on PKG_VERSION,gjb2016-02-091-1/+1
| | | | | | | | | | | | | | and create a 'latest' symlink to the PKG_VERSION repository path. Suggested by: des Sponsored by: The FreeBSD Foundation
* | Rename the signpackages target to sign-packages, and include it in thedes2016-02-081-1/+1
| | | | | | | | packages meta-target so 'make packages' now does everything.
* | Split the packages target into stage-packages and create-packages to makedes2016-02-081-2/+3
| | | | | | | | | | it possible to roll new packages from an existing build without having to restage them.
* | MFHgjb2016-02-081-3/+8
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Simplify running the FreeBSD test suitengie2016-02-071-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Replace `make regress` (legacy test make target) and `make test` (incomplete test make target added with the FreeBSD test suite) with make check as it's consistent with other open source projects. `make check` defaults to running tests from `.OBJDIR`, but can be overridden with the `CHECKDIR` variable. Add `make checkworld` target to simplify running the FreeBSD test suite from `TESTSBASE` (i.e. the top-level tests directory), similar to buildworld. Document `make check` and `make checkworld` in build(7). Other minor changes: - Rename intermediate file (`Kyuafile.auto`) to `Kyuafile` to simplify `make check`. - Remove terse warnings attached to `beforetest`/`aftertest`. - Add kyua binary check to check target in suite.test.mk; error out if it's not found The MFC is [partly] contingent on other build related changes being MFCed. Differential Revision: https://reviews.freebsd.org/D4406 MFC after: 2 months X-MFC to: stable/10 Relnotes: yes Reviewed by: bdrewery, Evan Cramer <eccramer@gmail.com> Sponsored by: EMC / Isilon Storage Division
* | MFHgjb2016-02-021-1/+8
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Add order for installworld/distribution.bdrewery2016-02-021-0/+1
| | | | | | | | Sponsored by: EMC / Isilon Storage Division
| * Add make universe targets "kernels" and "worlds".ian2016-01-311-1/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | "make kernels" is now shorthand for "make universe -DMAKE_JUST_KERNELS" "make worlds" is now shorthand for "make universe -DMAKE_JUST_WORLDS" The kernels target includes modules (unless you add -DNO_MODULES). And of course you can still add all the other universe options, such as "make kernels TARGETS=arm" to build kernels for all arm arches, or TARGET_ARCH=armv6 to build all armv6 kernels, etc. Reviewed by: imp
* | Add 'stagekernel' target, which invokes 'distributekernel'gjb2016-01-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | with -DNO_ROOT to create the METALOG mtree(8) file. Separate the default STAGEDIR for world (WSTAGEDIR) and kernel (KSTAGEDIR). Fix the 'create-kernel-packages' target to work properly. Evaluate if 'kernel' is set when invoking mtree-to-plist.awk, which splits the kernel and kernel.debug into separate plist files. Fix METALOG creation when building/packaging multiple kernels. Sponsored by: The FreeBSD Foundation
* | Add a 'signpackages' target, which creates the pkg repositorygjb2016-01-251-1/+1
| | | | | | | | | | | | | | metadata files and uses the path of PKGSIGNKEY for signing, if set. Sponsored by: The FreeBSD Foundation
* | MFHgjb2016-01-121-1/+1
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Always try to upgrade to bmake if not already using it.bdrewery2016-01-071-1/+1
| | | | | | | | | | | | | | | | | | This is mostly targetting stable/10 which requires bmake to build and has issues upgrading from <10. MFC after: 1 week Sponsored by: EMC / Isilon Storage Division PR: 198062
| * Revert r293286. It was not intended to come in yet.bdrewery2016-01-071-0/+7
| |
| * Move the MAKEOBJDIRPREFIX value guard to sys.mk and expand to MAKEOBJDIR.bdrewery2016-01-071-7/+0
| | | | | | | | | | | | | | | | | | | | | | | | This will ensure that the variable was not set as a make override, in make.conf, src.conf or src-env.conf. It allows setting the value in src-env.conf when using WITH_AUTO_OBJ since that case properly handles changing .OBJDIR (except if MAKEOBJDIRPREFIX does not yet exist which is being discussed to be changed). This change allows setting a default MAKEOBJDIRPREFIX via local.sys.env.mk. Sponsored by: EMC / Isilon Storage Division
* | MFH r289384-r293170gjb2016-01-041-57/+23
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Update advice on obj disk space requirementsemaste2015-12-141-2/+2
| | | | | | | | | | The Makefile is not the best place for this information, but at least it is now less out of date.
| * Start support for the RISC-V 64-bit architecture developed by UC Berkeley.br2015-12-111-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | RISC-V is a new ISA designed to support computer research and education, and is now become a standard open architecture for industry implementations. This is a minimal set of changes required to run 'make kernel-toolchain' using external (GNU) toolchain. The FreeBSD/RISC-V project home: https://wiki.freebsd.org/riscv. Reviewed by: andrew, bdrewery, emaste, imp Sponsored by: DARPA, AFRL Sponsored by: HEIF5 Differential Revision: https://reviews.freebsd.org/D4445
| * Fix errors being ignored in many phases of the build since the bmake ↵bdrewery2015-12-011-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | integration. Say it with me, "I will not chain commands with && in Makefiles" This was originally fixed and explained quite well by bde@ in r36074. The initial bmake integration caused 'set -e' to stop being used which lead to r252419. Later 'set -e' expectations were fixed with bmake in r254980. Because of the && here, errors would be ignored when building in parallel and a dependency failed. Such as bootstrap-tools since it builds everything in parallel. If any tool failed in obj/depend/all, it would just ignore the error and continue to build. This later would result in cascaded errors that only confused the real issue. This could also cause commands after the failed command to still execute, leading to more confusion. This should be fine if the command is in a sub-shell such as: (cmd1 && cmd2) This reverts r252419. MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
| * Rename META_MODE option to DIRDEPS_BUILDsjg2015-11-141-5/+5
| | | | | | | | | | | | | | | | | | This allows META_FILES option to be renamed META_MODE. Also add META_COOKIE_TOUCH for use in targets that can benefit from a cookie when in meta mode. Differential Revision: https://reviews.freebsd.org/D4153 Reviewed by: bdrewery
| * Move 'make cleandir' from Makefile to Makefile.inc1.bdrewery2015-11-091-33/+1
| | | | | | | | | | | | | | | | This leads the way for fixing cross-build cleanup, and eventually replacing 'cleandir' with it during the build. Sponsored by: EMC / Isilon Storage Division MFC after: 2 weeks
| * Add built-in ccache build support via WITH_CCACHE_BUILD option.bdrewery2015-11-081-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | ccache is mostly beneficial for frequent builds where -DNO_CLEAN is not used to achieve a safe pseudo-incremental build. This is explained in more detail upstream [1] [2]. It incurs about a 20%-28% hit to populate the cache, but with a full cache saves 30-50% in build times. When combined with the WITH_FAST_DEPEND feature it saves up to 65% since ccache does cache the resulting dependency file, which it does not do when using mkdep(1)/'CC -E'. Stats are provided at the end of this message. This removes the need to modify /etc/make.conf with the CC:= and CXX:= lines which conflicted with external compiler support [3] (causing the bootstrap compiler to not be built which lead to obscure failures [4]), incorrectly invoked ccache in various stages, required CCACHE_CPP2 to avoid Clang errors with parenthesis, and did not work with META_MODE. The option name was picked to match the existing option in ports. This feature is available for both in-src and out-of-src builds that use /usr/share/mk. Linking, assembly compiles, and pre-processing avoid using ccache since it is only overhead. ccache does nothing special in these modes, although there is no harm in calling it for them. CCACHE_COMPILERCHECK is set to 'content' when using the in-tree bootstrap compiler to hash the content of the compiler binary to determine if it should be a cache miss. For external compilers the 'mtime' option is used as it is more efficient and likely to be correct. Future work may optimize the 'content' check using the same checks as whether a bootstrap compiler is needed to be built. The CCACHE_CPP2 pessimization is currently default in our devel/ccache port due to Clang requiring it. Clang's -Wparentheses-equality, -Wtautological-compare, and -Wself-assign warnings do not mix well with compiling already-pre-processed code that may have expanded macros that trigger the warnings. GCC has so far not had this issue so it is allowed to disable the CCACHE_CPP2 default in our port. Sharing a cache between multiple checkouts, or systems, is explained in the ccache manual. Sharing a cache over NFS would likely not be worth it, but syncing cache directories between systems may be useful for an organization. There is also a memcached backend available [5]. Due to using an object directory outside of the source directory though you will need to ensure that both are in the same prefix and all users use the same layout. A possible working layout is as follows: Source: /some/prefix/src1 Source: /some/prefix/src2 Source: /some/prefix/src3 Objdir: /some/prefix/obj Environment: CCACHE_BASEDIR='${SRCTOP:H}' MAKEOBJDIRPREFIX='${SRCTOP:H}/obj' This will use src*/../obj as the MAKEOBJDIRPREFIX and tells ccache to replace all absolute paths to be relative. Using something like this is required due to -I and -o flags containing both SRC and OBJDIR absolute paths that ccache adds into its hash for the object without CCACHE_BASEDIR. distcc can be hooked into by setting CCACHE_PREFIX=/usr/local/bin/distcc. I have not personally tested this and assume it will not mix well with using the bootstrap compiler. The cache from buildworld can be reused in a subdir by first running 'make buildenv' (from r290424). Note that the cache is currently different depending on whether -j is used or not due to ccache enabling -fdiagnostics-color automatically if stderr is a TTY, which bmake only does if not using -j. The system I used for testing was: WITNESS Build options: -j20 WITH_LLDB=yes WITH_DEBUG_FILES=yes WITH_CCACHE_BUILD=yes DISK: ZFS 3-way mirror with very slow disks using SSD l2arc/log. The arc was fully populated with src tree files and ccache objects. RAM: 76GiB CPU: Intel(R) Xeon(R) CPU L5520 @2.27GHz 2 package(s) x 4 core(s) x 2 SMT threads = hw.ncpu=16 The WITH_FAST_DEPEND feature was used for comparison here as well to show the dramatic time savings with a full cache. buildworld: x buildworld-before + buildworld-ccache-empty * buildworld-ccache-full % buildworld-ccache-full-fastdep # buildworld-fastdep +-------------------------------------------------------------------------------+ |% * # +| |% * # +| |% * # xxx +| | |A | | A| | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 3744.13 3794.31 3752.25 3763.5633 26.935139 + 3 4519 4525.04 4520.73 4521.59 3.1104823 Difference at 95.0% confidence 758.027 +/- 43.4565 20.1412% +/- 1.15466% (Student's t, pooled s = 19.1726) * 3 1823.08 1827.2 1825.62 1825.3 2.0785572 Difference at 95.0% confidence -1938.26 +/- 43.298 -51.5007% +/- 1.15045% (Student's t, pooled s = 19.1026) % 3 1266.96 1279.37 1270.47 1272.2667 6.3971113 Difference at 95.0% confidence -2491.3 +/- 44.3704 -66.1952% +/- 1.17895% (Student's t, pooled s = 19.5758) # 3 3153.34 3155.16 3154.2 3154.2333 0.91045776 Difference at 95.0% confidence -609.33 +/- 43.1943 -16.1902% +/- 1.1477% (Student's t, pooled s = 19.0569) buildkernel: x buildkernel-before + buildkernel-ccache-empty * buildkernel-ccache-empty-fastdep % buildkernel-ccache-full # buildkernel-ccache-full-fastdep @ buildkernel-fastdep +-------------------------------------------------------------------------------+ |# @ % * | |# @ % * x + | |# @ % * xx ++| | MA | | MA| | A | | A | |A | | A | +-------------------------------------------------------------------------------+ N Min Max Median Avg Stddev x 3 571.57 573.94 571.79 572.43333 1.3094401 + 3 727.97 731.91 728.06 729.31333 2.2492295 Difference at 95.0% confidence 156.88 +/- 4.17129 27.4058% +/- 0.728695% (Student's t, pooled s = 1.84034) * 3 527.1 528.29 528.08 527.82333 0.63516402 Difference at 95.0% confidence -44.61 +/- 2.33254 -7.79305% +/- 0.407478% (Student's t, pooled s = 1.02909) % 3 400.4 401.05 400.62 400.69 0.3306055 Difference at 95.0% confidence -171.743 +/- 2.16453 -30.0023% +/- 0.378128% (Student's t, pooled s = 0.954969) # 3 201.94 203.34 202.28 202.52 0.73020545 Difference at 95.0% confidence -369.913 +/- 2.40293 -64.6212% +/- 0.419774% (Student's t, pooled s = 1.06015) @ 3 369.12 370.57 369.3 369.66333 0.79033748 Difference at 95.0% confidence -202.77 +/- 2.45131 -35.4225% +/- 0.428227% (Student's t, pooled s = 1.0815) [1] https://ccache.samba.org/performance.html [2] http://www.mail-archive.com/ccache@lists.samba.org/msg00576.html [3] https://reviews.freebsd.org/D3484 [5] https://github.com/jrosdahl/ccache/pull/30 PR: 182944 [4] MFC after: 3 weeks Sponsored by: EMC / Isilon Storage Division Relnotes: yes
| * Rework the 'make -n -n' feature such that '-n' recurses and '-N' does not.bdrewery2015-10-171-7/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bmake has a documented feature of '-N' to skip executing commands which is specifically intended for debugging top-level builds and not recursing into sub-directories. This matches the older 'make -n' behavior we added which made '-n -n' the recursing target and '-n' a non-recursing target. Removing the '-n -n' feature allows the build to work as documented in the bmake manpage with '-n' and '-N'. The older '-n -n' feature was also not documented anywhere that I could see. Note that the ${_+_} var is still needed as currently bmake incorrectly executes '+' commands when '-N' is specified. The '-n' and '-n -n' features were broken for several reasons prior to this. r251748 made '_+_' never expand with '-n -n' which resulted in many sub-directories not being visited until fixed 2 years later in r288391, and many targets were given .MAKE over the past few years which resulted in non-sub-make commands, such as rm and ln and mtree, to be executed. This should also allow removing some indirection hacks in bsd.subdir.mk and other cases of .USE that have a .MAKE by using '+'. Sponsored by: EMC / Isilon Storage Division Discussed on: arch@ (mostly silence)
| * Tweak the guard more to suggest 'all' if SUBDIR_OVERRIDE is specified. In thatbdrewery2015-10-161-1/+1
| | | | | | | | | | | | | | | | case 'all' does make sense. MFC after: 2 weeks X-MFC-With: r289411 Sponsored by: EMC / Isilon Storage Division
| * Tweak the default target to not suggest 'all' since it really doesn't dobdrewery2015-10-161-2/+2
| | | | | | | | | | | | | | anything useful for most users. MFC after: 2 weeks Sponsored by: EMC / Isilon Storage Division
* | MFH to r289370gjb2015-10-151-1/+1
|\ \ | |/ | | | | Sponsored by: The FreeBSD Foundation
| * Consider top-level targets to be .PHONY as bmake won't build thembdrewery2015-10-151-1/+1
| | | | | | | | | | | | | | otherwise if a file with the same name is found in the directory. MFC after: 1 week Sponsored by: EMC / Isilon Storage Division
* | Merge from headbapt2015-10-091-1/+1
|\ \ | |/
OpenPOWER on IntegriCloud