summaryrefslogtreecommitdiffstats
path: root/include/linux/unaligned
Commit message (Collapse)AuthorAgeFilesLines
* include/linux/unaligned: force inlining of byteswap operationsDenys Vlasenko2016-03-171-12/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Sometimes gcc mysteriously doesn't inline very small functions we expect to be inlined. See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=66122 With this .config: http://busybox.net/~vda/kernel_config_OPTIMIZE_INLINING_and_Os, the following functions get deinlined many times. Examples of disassembly: <get_unaligned_be16> (24 copies, 108 calls): 66 8b 07 mov (%rdi),%ax 55 push %rbp 48 89 e5 mov %rsp,%rbp 86 e0 xchg %ah,%al 5d pop %rbp c3 retq <get_unaligned_be32> (25 copies, 181 calls): 8b 07 mov (%rdi),%eax 55 push %rbp 48 89 e5 mov %rsp,%rbp 0f c8 bswap %eax 5d pop %rbp c3 retq <get_unaligned_be64> (23 copies, 94 calls): 48 8b 07 mov (%rdi),%rax 55 push %rbp 48 89 e5 mov %rsp,%rbp 48 0f c8 bswap %rax 5d pop %rbp c3 retq <put_unaligned_be16> (2 copies, 11 calls): 89 f8 mov %edi,%eax 55 push %rbp c1 ef 08 shr $0x8,%edi c1 e0 08 shl $0x8,%eax 09 c7 or %eax,%edi 48 89 e5 mov %rsp,%rbp 66 89 3e mov %di,(%rsi) <put_unaligned_be32> (8 copies, 43 calls): 55 push %rbp 0f cf bswap %edi 89 3e mov %edi,(%rsi) 48 89 e5 mov %rsp,%rbp 5d pop %rbp c3 retq <put_unaligned_be64> (26 copies, 157 calls): 55 push %rbp 48 0f cf bswap %rdi 48 89 3e mov %rdi,(%rsi) 48 89 e5 mov %rsp,%rbp 5d pop %rbp c3 retq This patch fixes this via s/inline/__always_inline/. It only affects arches with efficient unaligned access insns, such as x86. (arched which lack such ops do not include linux/unaligned/access_ok.h) Code size decrease after the patch is ~8.5k: text data bss dec hex filename 92197848 20826112 36417536 149441496 8e84bd8 vmlinux 92189231 20826144 36417536 149432911 8e82a4f vmlinux6_unaligned_be_after Signed-off-by: Denys Vlasenko <dvlasenk@redhat.com> Acked-by: Ingo Molnar <mingo@kernel.org> Cc: Thomas Graf <tgraf@suug.ch> Cc: Peter Zijlstra <peterz@infradead.org> Cc: David Rientjes <rientjes@google.com> Cc: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* include/linux/unaligned/packed_struct.h: use __packedAndrew Morton2011-01-131-3/+3
| | | | | | Cc: Will Newton <will.newton@gmail.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* include/linux/unaligned: pack the whole struct rather than just the fieldWill Newton2010-12-221-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current packed struct implementation of unaligned access adds the packed attribute only to the field within the unaligned struct rather than to the struct as a whole. This is not sufficient to enforce proper behaviour on architectures with a default struct alignment of more than one byte. For example, the current implementation of __get_unaligned_cpu16 when compiled for arm with gcc -O1 -mstructure-size-boundary=32 assumes the struct is on a 4 byte boundary so performs the load of the 16bit packed field as if it were on a 4 byte boundary: __get_unaligned_cpu16: ldrh r0, [r0, #0] bx lr Moving the packed attribute to the struct rather than the field causes the proper unaligned access code to be generated: __get_unaligned_cpu16: ldrb r3, [r0, #0] @ zero_extendqisi2 ldrb r0, [r0, #1] @ zero_extendqisi2 orr r0, r3, r0, asl #8 bx lr Signed-off-by: Will Newton <will.newton@gmail.com> Cc: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* include/linux/unaligned/{l,b}e_byteshift.h: fix usage for compressed kernelsAlbin Tonnerre2009-09-242-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When unaligned accesses are required for uncompressing a kernel (such as for LZO decompression on ARM in a patch that follows), including <linux/kernel.h> causes issues as it brings in a lot of things that are not available in the decompression environment. linux/kernel.h brings at least: extern int console_printk[]; extern const char hex_asc[]; which causes errors at link-time as they are not available when compiling the pre-boot environement. There are also a few others: arch/arm/boot/compressed/misc.o: In function `valid_user_regs': arch/arm/include/asm/ptrace.h:158: undefined reference to `elf_hwcap' arch/arm/boot/compressed/misc.o: In function `console_silent': include/linux/kernel.h:292: undefined reference to `console_printk' arch/arm/boot/compressed/misc.o: In function `console_verbose': include/linux/kernel.h:297: undefined reference to `console_printk' arch/arm/boot/compressed/misc.o: In function `pack_hex_byte': include/linux/kernel.h:360: undefined reference to `hex_asc' arch/arm/boot/compressed/misc.o: In function `hweight_long': include/linux/bitops.h:45: undefined reference to `hweight32' arch/arm/boot/compressed/misc.o: In function `__cmpxchg_local_generic': include/asm-generic/cmpxchg-local.h:21: undefined reference to `wrong_size_cmpxchg' include/asm-generic/cmpxchg-local.h:42: undefined reference to `wrong_size_cmpxchg' arch/arm/boot/compressed/misc.o: In function `__xchg': arch/arm/include/asm/system.h:309: undefined reference to `__bad_xchg' However, those files apparently use nothing from <linux/kernel.h>, all they need is the declaration of types such as u32 or u64, so <linux/types.h> should be enough Signed-off-by: Albin Tonnerre <albin.tonnerre@free-electrons.com> Cc: Sam Ravnborg <sam@ravnborg.org> Cc: Russell King <rmk@arm.linux.org.uk> Cc: Ingo Molnar <mingo@elte.hu> Cc: Thomas Gleixner <tglx@linutronix.de> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: Phillip Lougher <phillip@lougher.demon.co.uk> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* kernel: add common infrastructure for unaligned accessHarvey Harrison2008-04-2910-0/+510
Create a linux/unaligned directory similar in spirit to the linux/byteorder folder to hold generic implementations collected from various arches. Currently there are five implementations: 1) packed_struct.h: C-struct based, from asm-generic/unaligned.h 2) le_byteshift.h: Open coded byte-swapping, heavily based on asm-arm 3) be_byteshift.h: Open coded byte-swapping, heavily based on asm-arm 4) memmove.h: taken from multiple implementations in tree 5) access_ok.h: taken from x86 and others, unaligned access is ok. All of the new implementations checks for sizes not equal to 1,2,4,8 and will fail to link. API additions: get_unaligned_{le16|le32|le64|be16|be32|be64}(p) which is meant to replace code of the form: le16_to_cpu(get_unaligned((__le16 *)p)); put_unaligned_{le16|le32|le64|be16|be32|be64}(val, pointer) which is meant to replace code of the form: put_unaligned(cpu_to_le16(val), (__le16 *)p); The headers that arches should include from their asm/unaligned.h: access_ok.h : Wrappers of the byteswapping functions in asm/byteorder Choose a particular implementation for little-endian access: le_byteshift.h le_memmove.h (arch must be LE) le_struct.h (arch must be LE) Choose a particular implementation for big-endian access: be_byteshift.h be_memmove.h (arch must be BE) be_struct.h (arch must be BE) After including as needed from the above, include unaligned/generic.h and define your arch's get/put_unaligned as (for LE): Signed-off-by: Harvey Harrison <harvey.harrison@gmail.com> Cc: <linux-arch@vger.kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
OpenPOWER on IntegriCloud