summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* block: blk-exec.c: Cleaning up local variable address returndRickard Strandqvist2014-06-081-0/+5
| | | | | | | | | Address of local variable assigned to a function parameter This was partly found using a static code analysis program called cppcheck. Signed-off-by: Rickard Strandqvist <rickard_strandqvist@spectrumdigital.se> Signed-off-by: Jens Axboe <axboe@fb.com>
* mtip32xx: minor performance enhancementsSam Bradshaw2014-06-062-10/+13
| | | | | | | | | | | | | | | | | | | | | This patch adds the following: 1) Compiler hinting in the fast path. 2) A prefetch of port->flags to eliminate moderate cpu stalling later in mtip_hw_submit_io(). 3) Eliminate a redundant rq_data_dir(). 4) Reorder members of driver_data to eliminate false cacheline sharing between irq_workers_active and unal_qdepth. With some workload and topology configurations, I'm seeing ~1.5% throughput improvement in small block random read benchmarks as well as improved latency std. dev. Signed-off-by: Sam Bradshaw <sbradshaw@micron.com> Add include of <linux/prefetch.h> Signed-off-by: Jens Axboe <axboe@fb.com>
* blk-mq: ->timeout should be cleared in blk_mq_rq_ctx_init()Jens Axboe2014-06-061-0/+2
| | | | | | | | | | It'll be used in blk_mq_start_request() to set a potential timeout for the request, so clear it to zero at alloc time to ensure that we know if someone has set it or not. Fixes random early timeouts on NVMe testing. Signed-off-by: Jens Axboe <axboe@fb.com>
* blk-mq: don't allow queue entering for a dying queueKeith Busch2014-06-061-2/+4
| | | | | | | | | If the queue is going away, don't let new allocs or queueing happen on it. Go through the normal wait process, and exit with ENODEV in that case. Signed-off-by: Keith Busch <keith.busch@intel.com> Signed-off-by: Jens Axboe <axboe@fb.com>
* blk-mq: bump max tag depth to 10K tagsJens Axboe2014-06-062-2/+13
| | | | | | | | | | | For some scsi-mq cases, the tag map can be huge. So increase the max number of tags we support. Additionally, don't fail with EINVAL if a user requests too many tags. Warn that the tag depth has been adjusted down, and store the new value inside the tag_set passed in. Signed-off-by: Jens Axboe <axboe@fb.com>
* block: add blk_rq_set_block_pc()Jens Axboe2014-06-0617-20/+41
| | | | | | | | | | | | | | | | With the optimizations around not clearing the full request at alloc time, we are leaving some of the needed init for REQ_TYPE_BLOCK_PC up to the user allocating the request. Add a blk_rq_set_block_pc() that sets the command type to REQ_TYPE_BLOCK_PC, and properly initializes the members associated with this type of request. Update callers to use this function instead of manipulating rq->cmd_type directly. Includes fixes from Christoph Hellwig <hch@lst.de> for my half-assed attempt. Signed-off-by: Jens Axboe <axboe@fb.com>
* block: add notion of a chunk size for request mergingJens Axboe2014-06-053-2/+41
| | | | | | | | | | | | Some drivers have different limits on what size a request should optimally be, depending on the offset of the request. Similar to dividing a device into chunks. Add a setting that allows the driver to inform the block layer of such a chunk size. The block layer will then prevent merging across the chunks. This is needed to optimally support NVMe with a non-zero stripe size. Signed-off-by: Jens Axboe <axboe@fb.com>
* Merge branch 'x86-efi-for-linus' of ↵Linus Torvalds2014-06-0511-263/+361
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next Pull x86 EFI updates from Peter Anvin: "A collection of EFI changes. The perhaps most important one is to fully save and restore the FPU state around each invocation of EFI runtime, and to not choke on non-ASCII characters in the boot stub" * 'x86-efi-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: efivars: Add compatibility code for compat tasks efivars: Refactor sanity checking code into separate function efivars: Stop passing a struct argument to efivar_validate() efivars: Check size of user object efivars: Use local variables instead of a pointer dereference x86/efi: Save and restore FPU context around efi_calls (i386) x86/efi: Save and restore FPU context around efi_calls (x86_64) x86/efi: Implement a __efi_call_virt macro x86, fpu: Extend the use of static_cpu_has_safe x86/efi: Delete most of the efi_call* macros efi: x86: Handle arbitrary Unicode characters efi: Add get_dram_base() helper function efi: Add shared printk wrapper for consistent prefixing efi: create memory map iteration helper efi: efi-stub-helper cleanup
| * efivars: Add compatibility code for compat tasksMatt Fleming2014-04-171-26/+116
| | | | | | | | | | | | | | | | | | | | | | | | | | It seems people are using 32-bit efibootmgr on top of 64-bit kernels, which will currently fail horribly when using the efivars interface, which is the traditional efibootmgr backend (the other being efivarfs). Since there is no versioning info in the data structure, figure out when we need to munge the structure data via judicious use of is_compat_task(). Cc: Mike Waychison <mikew@google.com> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efivars: Refactor sanity checking code into separate functionMatt Fleming2014-04-171-20/+32
| | | | | | | | | | | | | | | | Move a large chunk of code that checks the validity of efi_variable into a new function, because we'll also need to use it for the compat code. Cc: Mike Waychison <mikew@google.com> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efivars: Stop passing a struct argument to efivar_validate()Matt Fleming2014-04-173-19/+23
| | | | | | | | | | | | | | | | | | | | In preparation for compat support, we can't assume that user variable object is represented by a 'struct efi_variable'. Convert the validation functions to take the variable name as an argument, which is the only piece of the struct that was ever used anyway. Cc: Mike Waychison <mikew@google.com> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efivars: Check size of user objectMatt Fleming2014-04-171-0/+6
| | | | | | | | | | | | | | | | | | Unbelieavably there are no checks to see whether the data structure passed to 'new_var' and 'del_var' is the size that we expect. Let's add some for better robustness. Cc: Mike Waychison <mikew@google.com> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efivars: Use local variables instead of a pointer dereferenceMatt Fleming2014-04-171-13/+35
| | | | | | | | | | | | | | | | | | In order to support a compat interface we need to stop passing pointers to structures around, since the type of structure is going to depend on whether the current task is a compat task. Cc: Mike Waychison <mikew@google.com> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * x86/efi: Save and restore FPU context around efi_calls (i386)Ricardo Neri2014-04-171-2/+15
| | | | | | | | | | | | | | | | | | | | | | Do a complete FPU context save/restore around the EFI calls. This required as runtime EFI firmware may potentially use the FPU. This change covers only the i386 configuration. Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * x86/efi: Save and restore FPU context around efi_calls (x86_64)Ricardo Neri2014-04-171-0/+3
| | | | | | | | | | | | | | | | | | | | | | Do a complete FPU context save/restore around the EFI calls. This required as runtime EFI firmware may potentially use the FPU. This change covers only the x86_64 configuration. Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * x86/efi: Implement a __efi_call_virt macroRicardo Neri2014-04-172-2/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For i386, all the EFI system runtime services functions return efi_status_t except efi_reset_system_system. Therefore, not all functions can be covered by the same macro in case the macro needs to do more than calling the function (i.e., return a value). The purpose of the __efi_call_virt macro is to be used when no return value is expected. For x86_64, this macro would not be needed as all the runtime services return u64. However, the same code is used for both x86_64 and i386. Thus, the macro __efi_call_virt is also defined to not break compilation. Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * x86, fpu: Extend the use of static_cpu_has_safeMatt Fleming2014-04-171-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | It may be necessary to save and restore the FPU context during EFI runtime system services calls. However, this may happen during boot and before alternatives have run. Thus, we need to use static_cpu_has_safe instead. The rationale behind the use of static_cpu_has_safe is the same as in commit 5f8c4218148822fde6ee ("x86, fpu: Use static_cpu_has_safe before alternatives") by Borislav Petkov. Signed-off-by: Matt Fleming <matt.fleming@intel.com> Signed-off-by: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Cc: Borislav Petkov <bp@suse.de>
| * x86/efi: Delete most of the efi_call* macrosMatt Fleming2014-04-175-172/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | We really only need one phys and one virt function call, and then only one assembly function to make firmware calls. Since we are not using the C type system anyway, we're not really losing much by deleting the macros apart from no longer having a check that we are passing the correct number of parameters. The lack of duplicated code seems like a worthwhile trade-off. Cc: Ricardo Neri <ricardo.neri-calderon@linux.intel.com> Cc: Borislav Petkov <bp@suse.de> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efi: x86: Handle arbitrary Unicode charactersH. Peter Anvin2014-04-172-22/+68
| | | | | | | | | | | | | | | | | | | | | | Instead of truncating UTF-16 assuming all characters is ASCII, properly convert it to UTF-8. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> [ Bug and style fixes. ] Signed-off-by: Roy Franz <roy.franz@linaro.org> Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efi: Add get_dram_base() helper functionRoy Franz2014-04-171-0/+30
| | | | | | | | | | | | | | | | Add the get_dram_base() function, shared by arm/arm64. Signed-off-by: Roy Franz <roy.franz@linaro.org> Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efi: Add shared printk wrapper for consistent prefixingRoy Franz2014-04-141-5/+8
| | | | | | | | | | | | | | | | | | Add a wrapper for printk to standardize the prefix for informational and error messages from the EFI stub. Signed-off-by: Roy Franz <roy.franz@linaro.org> Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efi: create memory map iteration helperMark Salter2014-04-141-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | There are a lot of places in the kernel which iterate through an EFI memory map. Most of these places use essentially the same for-loop code. This patch adds a for_each_efi_memory_desc() helper to clean up all of the existing duplicate code and avoid more in the future. Signed-off-by: Mark Salter <msalter@redhat.com> Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
| * efi: efi-stub-helper cleanupLeif Lindholm2014-04-141-13/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | An #ifdef CONFIG_ARM clause in efi-stub-helper.c got included with some of the generic stub rework by Roy Franz. Drop it here to make subsequent patches less confusing. Also, In handle_cmdline_files(), fh is not initialized, and while the overall logic around this handling appears safe, gcc does not always pick this up. Initialize to NULL to remove the resulting warning. Signed-off-by: Leif Lindholm <leif.lindholm@linaro.org> Signed-off-by: Matt Fleming <matt.fleming@intel.com>
* | Merge branch 'x86/vdso' of ↵Linus Torvalds2014-06-0540-608/+794
|\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next Pull x86 cdso updates from Peter Anvin: "Vdso cleanups and improvements largely from Andy Lutomirski. This makes the vdso a lot less ''special''" * 'x86/vdso' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/vdso, build: Make LE access macros clearer, host-safe x86/vdso, build: Fix cross-compilation from big-endian architectures x86/vdso, build: When vdso2c fails, unlink the output x86, vdso: Fix an OOPS accessing the HPET mapping w/o an HPET x86, mm: Replace arch_vma_name with vm_ops->name for vsyscalls x86, mm: Improve _install_special_mapping and fix x86 vdso naming mm, fs: Add vm_ops->name as an alternative to arch_vma_name x86, vdso: Fix an OOPS accessing the HPET mapping w/o an HPET x86, vdso: Remove vestiges of VDSO_PRELINK and some outdated comments x86, vdso: Move the vvar and hpet mappings next to the 64-bit vDSO x86, vdso: Move the 32-bit vdso special pages after the text x86, vdso: Reimplement vdso.so preparation in build-time C x86, vdso: Move syscall and sysenter setup into kernel/cpu/common.c x86, vdso: Clean up 32-bit vs 64-bit vdso params x86, mm: Ensure correct alignment of the fixmap
| * | x86/vdso, build: Make LE access macros clearer, host-safeH. Peter Anvin2014-05-312-39/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make it a little clearer what the littleendian access macros in vdso2c.[ch] actually do. This way they can probably also be moved to a central location (e.g. tools/include) for the benefit of other host tools. We should avoid implementation namespace symbols when writing code that is compiling for the compiler host, so avoid names starting with double underscore or underscore-capital. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Cc: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/2cf258df123cb24bad63c274c8563c050547d99d.1401464755.git.luto@amacapital.net
| * | x86/vdso, build: Fix cross-compilation from big-endian architecturesAndy Lutomirski2014-05-302-28/+50
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This adds a macro GET(x) to convert x from big-endian to little-endian. Hopefully I put it everywhere it needs to go and got all the cases needed for everyone's linux/elf.h. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/2cf258df123cb24bad63c274c8563c050547d99d.1401464755.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | x86/vdso, build: When vdso2c fails, unlink the outputAndy Lutomirski2014-05-302-16/+14
| | | | | | | | | | | | | | | | | | | | | | | | This avoids bizarre failures if make is run again. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/1764385fe9931e8940b9d001132515448ea89523.1401464755.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | Merge remote-tracking branch 'origin/x86/urgent' into x86/vdsoH. Peter Anvin2014-05-21195-1191/+2274
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | Resolved Conflicts: arch/x86/vdso/vdso32-setup.c Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| | * | x86, vdso: Fix an OOPS accessing the HPET mapping w/o an HPETAndy Lutomirski2014-05-211-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The oops can be triggered in qemu using -no-hpet (but not nohpet) by running a 32-bit program and reading a couple of pages before the vdso. This should send SIGBUS instead of OOPSing. The bug was introduced by: commit 7a59ed415f5b57469e22e41fc4188d5399e0b194 Author: Stefani Seibold <stefani@seibold.net> Date: Mon Mar 17 23:22:09 2014 +0100 x86, vdso: Add 32 bit VDSO time support for 32 bit kernel which is new in 3.15. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/e99025d887d6670b6c4d81e6ccfeeb83770b21e9.1400109621.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | Merge remote-tracking branch 'origin/x86/espfix' into x86/vdsoH. Peter Anvin2014-05-21414-2934/+4935
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Merge x86/espfix into x86/vdso, due to changes in the vdso setup code that otherwise cause conflicts. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, mm: Replace arch_vma_name with vm_ops->name for vsyscallsAndy Lutomirski2014-05-201-8/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This removes the last vestiges of arch_vma_name from x86, replacing it with vm_ops->name. Good riddance. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/e681cb56096eee5b8b8767093a4f6fb82839f0a4.1400538962.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, mm: Improve _install_special_mapping and fix x86 vdso namingAndy Lutomirski2014-05-208-51/+94
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Using arch_vma_name to give special mappings a name is awkward. x86 currently implements it by comparing the start address of the vma to the expected address of the vdso. This requires tracking the start address of special mappings and is probably buggy if a special vma is split or moved. Improve _install_special_mapping to just name the vma directly. Use it to give the x86 vvar area a name, which should make CRIU's life easier. As a side effect, the vvar area will show up in core dumps. This could be considered weird and is fixable. [hpa: I say we accept this as-is but be prepared to deal with knocking out the vvars from core dumps if this becomes a problem.] Cc: Cyrill Gorcunov <gorcunov@openvz.org> Cc: Pavel Emelyanov <xemul@parallels.com> Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/276b39b6b645fb11e345457b503f17b83c2c6fd0.1400538962.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | mm, fs: Add vm_ops->name as an alternative to arch_vma_nameAndy Lutomirski2014-05-203-0/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | arch_vma_name sucks. It's a silly hack, and it's annoying to implement correctly. In fact, AFAICS, even the straightforward x86 implementation is incorrect (I suspect that it breaks if the vdso mapping is split or gets remapped). This adds a new vm_ops->name operation that can replace it. The followup patches will remove all uses of arch_vma_name on x86, fixing a couple of annoyances in the process. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/2eee21791bb36a0a408c5c2bdb382a9e6a41ca4a.1400538962.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Fix an OOPS accessing the HPET mapping w/o an HPETAndy Lutomirski2014-05-201-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The oops can be triggered in qemu using -no-hpet (but not nohpet) by reading a couple of pages past the end of the vdso text. This should send SIGBUS instead of OOPSing. The bug was introduced by: commit 7a59ed415f5b57469e22e41fc4188d5399e0b194 Author: Stefani Seibold <stefani@seibold.net> Date: Mon Mar 17 23:22:09 2014 +0100 x86, vdso: Add 32 bit VDSO time support for 32 bit kernel which is new in 3.15. This will be fixed separately in 3.15, but that patch will not apply to tip/x86/vdso. This is the equivalent fix for tip/x86/vdso and, presumably, 3.16. Cc: Stefani Seibold <stefani@seibold.net> Reported-by: Sasha Levin <sasha.levin@oracle.com> Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/c8b0a9a0b8d011a8b273cbb2de88d37190ed2751.1400538962.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Remove vestiges of VDSO_PRELINK and some outdated commentsAndy Lutomirski2014-05-053-16/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | These definitions had no effect. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/946c104e40c47319f8ab406e54118799cb55bd99.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Move the vvar and hpet mappings next to the 64-bit vDSOAndy Lutomirski2014-05-0511-77/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This makes the 64-bit and x32 vdsos use the same mechanism as the 32-bit vdso. Most of the churn is deleting all the old fixmap code. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/8af87023f57f6bb96ec8d17fce3f88018195b49b.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Move the 32-bit vdso special pages after the textAndy Lutomirski2014-05-058-166/+173
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This unifies the vdso mapping code and teaches it how to map special pages at addresses corresponding to symbols in the vdso image. The new code is used for all vdso variants, but so far only the 32-bit variants use the new vvar page position. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/b6d7858ad7b5ac3fd3c29cab6d6d769bc45d195e.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Reimplement vdso.so preparation in build-time CAndy Lutomirski2014-05-0518-260/+400
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Currently, vdso.so files are prepared and analyzed by a combination of objcopy, nm, some linker script tricks, and some simple ELF parsers in the kernel. Replace all of that with plain C code that runs at build time. All five vdso images now generate .c files that are compiled and linked in to the kernel image. This should cause only one userspace-visible change: the loaded vDSO images are stripped more heavily than they used to be. Everything outside the loadable segment is dropped. In particular, this causes the section table and section name strings to be missing. This should be fine: real dynamic loaders don't load or inspect these tables anyway. The result is roughly equivalent to eu-strip's --strip-sections option. The purpose of this change is to enable the vvar and hpet mappings to be moved to the page following the vDSO load segment. Currently, it is possible for the section table to extend into the page after the load segment, so, if we map it, it risks overlapping the vvar or hpet page. This happens whenever the load segment is just under a multiple of PAGE_SIZE. The only real subtlety here is that the old code had a C file with inline assembler that did 'call VDSO32_vsyscall' and a linker script that defined 'VDSO32_vsyscall = __kernel_vsyscall'. This most likely worked by accident: the linker script entry defines a symbol associated with an address as opposed to an alias for the real dynamic symbol __kernel_vsyscall. That caused ld to relocate the reference at link time instead of leaving an interposable dynamic relocation. Since the VDSO32_vsyscall hack is no longer needed, I now use 'call __kernel_vsyscall', and I added -Bsymbolic to make it work. vdso2c will generate an error and abort the build if the resulting image contains any dynamic relocations, so we won't silently generate bad vdso images. (Dynamic relocations are a problem because nothing will even attempt to relocate the vdso.) Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/2c4fcf45524162a34d87fdda1eb046b2a5cecee7.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Move syscall and sysenter setup into kernel/cpu/common.cAndy Lutomirski2014-05-053-32/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This code is used during CPU setup, and it isn't strictly speaking related to the 32-bit vdso. It's easier to understand how this works when the code is closer to its callers. This also lets syscall32_cpu_init be static, which might save some trivial amount of kernel text. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/4e466987204e232d7b55a53ff6b9739f12237461.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, vdso: Clean up 32-bit vs 64-bit vdso paramsAndy Lutomirski2014-05-055-22/+30
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Rather than using 'vdso_enabled' and an awful #define, just call the parameters vdso32_enabled and vdso64_enabled. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/87913de56bdcbae3d93917938302fc369b05caee.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | x86, mm: Ensure correct alignment of the fixmapAndy Lutomirski2014-05-052-3/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The early_ioremap code requires that its buffers not span a PMD boundary. The logic for ensuring that only works if the fixmap is aligned, so assert that it's aligned correctly. To make this work reliably, reserve_top_address needs to be adjusted. Signed-off-by: Andy Lutomirski <luto@amacapital.net> Link: http://lkml.kernel.org/r/e59a5f4362661f75dd4841fa74e1f2448045e245.1399317206.git.luto@amacapital.net Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* | | | | Merge branch 'x86/espfix' of ↵Linus Torvalds2014-06-0514-41/+387
|\ \ \ \ \ | | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip into next Pull x86-64 espfix changes from Peter Anvin: "This is the espfix64 code, which fixes the IRET information leak as well as the associated functionality problem. With this code applied, 16-bit stack segments finally work as intended even on a 64-bit kernel. Consequently, this patchset also removes the runtime option that we added as an interim measure. To help the people working on Linux kernels for very small systems, this patchset also makes these compile-time configurable features" * 'x86/espfix' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: Revert "x86-64, modify_ldt: Make support for 16-bit segments a runtime option" x86, espfix: Make it possible to disable 16-bit support x86, espfix: Make espfix64 a Kconfig option, fix UML x86, espfix: Fix broken header guard x86, espfix: Move espfix definitions into a separate header file x86-32, espfix: Remove filter for espfix32 due to race x86-64, espfix: Don't leak bits 31:16 of %esp returning to 16-bit stack
| * | | | Merge commit '7ed6fb9b5a5510e4ef78ab27419184741169978a' into x86/espfixH. Peter Anvin2014-05-21542-3607/+5769
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Merge in Linus' tree with: fa81511bb0bb x86-64, modify_ldt: Make support for 16-bit segments a runtime option ... reverted, to avoid a conflict. This commit is no longer necessary with the proper fix in place. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| | * | | | Revert "x86-64, modify_ldt: Make support for 16-bit segments a runtime option"H. Peter Anvin2014-05-212-11/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reverts commit fa81511bb0bbb2b1aace3695ce869da9762624ff in preparation of merging in the proper fix (espfix64). Signed-off-by: H. Peter Anvin <hpa@zytor.com>
| * | | | | x86, espfix: Make it possible to disable 16-bit supportH. Peter Anvin2014-05-044-5/+43
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Embedded systems, which may be very memory-size-sensitive, are extremely unlikely to ever encounter any 16-bit software, so make it a CONFIG_EXPERT option to turn off support for any 16-bit software whatsoever. Signed-off-by: H. Peter Anvin <hpa@zytor.com> Link: http://lkml.kernel.org/r/1398816946-3351-1-git-send-email-hpa@linux.intel.com
| * | | | | x86, espfix: Make espfix64 a Kconfig option, fix UMLH. Peter Anvin2014-05-044-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Make espfix64 a hidden Kconfig option. This fixes the x86-64 UML build which had broken due to the non-existence of init_espfix_bsp() in UML: since UML uses its own Kconfig, this option does not appear in the UML build. This also makes it possible to make support for 16-bit segments a configuration option, for the people who want to minimize the size of the kernel. Reported-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: H. Peter Anvin <hpa@zytor.com> Cc: Richard Weinberger <richard@nod.at> Link: http://lkml.kernel.org/r/1398816946-3351-1-git-send-email-hpa@linux.intel.com
| * | | | | x86, espfix: Fix broken header guardH. Peter Anvin2014-05-021-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Header guard is #ifndef, not #ifdef... Reported-by: Fengguang Wu <fengguang.wu@intel.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | | x86, espfix: Move espfix definitions into a separate header fileH. Peter Anvin2014-05-013-3/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sparse warns that the percpu variables aren't declared before they are defined. Rather than hacking around it, move espfix definitions into a proper header file. Reported-by: Fengguang Wu <fengguang.wu@intel.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
| * | | | | x86-32, espfix: Remove filter for espfix32 due to raceH. Peter Anvin2014-04-301-5/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It is not safe to use LAR to filter when to go down the espfix path, because the LDT is per-process (rather than per-thread) and another thread might change the descriptors behind our back. Fortunately it is always *safe* (if a bit slow) to go down the espfix path, and a 32-bit LDT stack segment is extremely rare. Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Link: http://lkml.kernel.org/r/1398816946-3351-1-git-send-email-hpa@linux.intel.com Cc: <stable@vger.kernel.org> # consider after upstream merge
| * | | | | x86-64, espfix: Don't leak bits 31:16 of %esp returning to 16-bit stackH. Peter Anvin2014-04-3010-26/+329
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The IRET instruction, when returning to a 16-bit segment, only restores the bottom 16 bits of the user space stack pointer. This causes some 16-bit software to break, but it also leaks kernel state to user space. We have a software workaround for that ("espfix") for the 32-bit kernel, but it relies on a nonzero stack segment base which is not available in 64-bit mode. In checkin: b3b42ac2cbae x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels we "solved" this by forbidding 16-bit segments on 64-bit kernels, with the logic that 16-bit support is crippled on 64-bit kernels anyway (no V86 support), but it turns out that people are doing stuff like running old Win16 binaries under Wine and expect it to work. This works around this by creating percpu "ministacks", each of which is mapped 2^16 times 64K apart. When we detect that the return SS is on the LDT, we copy the IRET frame to the ministack and use the relevant alias to return to userspace. The ministacks are mapped readonly, so if IRET faults we promote #GP to #DF which is an IST vector and thus has its own stack; we then do the fixup in the #DF handler. (Making #GP an IST exception would make the msr_safe functions unsafe in NMI/MC context, and quite possibly have other effects.) Special thanks to: - Andy Lutomirski, for the suggestion of using very small stack slots and copy (as opposed to map) the IRET frame there, and for the suggestion to mark them readonly and let the fault promote to #DF. - Konrad Wilk for paravirt fixup and testing. - Borislav Petkov for testing help and useful comments. Reported-by: Brian Gerst <brgerst@gmail.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Link: http://lkml.kernel.org/r/1398816946-3351-1-git-send-email-hpa@linux.intel.com Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> Cc: Borislav Petkov <bp@alien8.de> Cc: Andrew Lutomriski <amluto@gmail.com> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: Dirk Hohndel <dirk@hohndel.org> Cc: Arjan van de Ven <arjan.van.de.ven@intel.com> Cc: comex <comexk@gmail.com> Cc: Alexander van Heukelum <heukelum@fastmail.fm> Cc: Boris Ostrovsky <boris.ostrovsky@oracle.com> Cc: <stable@vger.kernel.org> # consider after upstream merge
OpenPOWER on IntegriCloud