summaryrefslogtreecommitdiffstats
path: root/arch/parisc/kernel/cache.c
Commit message (Collapse)AuthorAgeFilesLines
* parisc: Fix cache routines to ignore vma's with an invalid pfnJohn David Anglin2013-07-311-64/+71
| | | | | | | | | | | | | | | | | | | | | | | | | The parisc architecture does not have a pte special bit. As a result, special mappings are handled with the VM_PFNMAP and VM_MIXEDMAP flags. VM_MIXEDMAP mappings may or may not have a "struct page" backing. When pfn_valid() is false, there is no "struct page" backing. Otherwise, they are treated as normal pages. The FireGL driver uses the VM_MIXEDMAP without a backing "struct page". This treatment caused a panic due to a TLB data miss in update_mmu_cache. This appeared to be in the code generated for page_address(). We were in fact using a very circular bit of code to determine the physical address of the PFN in various cache routines. This wasn't valid when there was no "struct page" backing. The needed address can in fact be determined simply from the PFN itself without using the "struct page". The attached patch updates update_mmu_cache(), flush_cache_mm(), flush_cache_range() and flush_cache_page() to check pfn_valid() and to directly compute the PFN physical and virtual addresses. Signed-off-by: John David Anglin <dave.anglin@bell.net> Cc: <stable@vger.kernel.org> # 3.10 Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: Ensure volatile space register %sr1 is not clobberedJohn David Anglin2013-07-091-1/+1
| | | | | | | | | | | | | | | | | | | | I still see the occasional random segv on rp3440. Looking at one of these (a code 15), it appeared the problem must be with the cache handling of anonymous pages. Reviewing this, I noticed that the space register %sr1 might be being clobbered when we flush an anonymous page. Register %sr1 is used for TLB purges in a couple of places. These purges are needed on PA8800 and PA8900 processors to ensure cache consistency of flushed cache lines. The solution here is simply to move the %sr1 load into the TLB lock region needed to ensure that one purge executes at a time on SMP systems. This was already the case for one use. After a few days of operation, I haven't had a random segv on my rp3440. Signed-off-by: John David Anglin <dave.anglin@bell.net> Cc: <stable@vger.kernel.org> # 3.10 Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: remove the second argument of kmap_atomicZhao Hongjiang2013-05-061-3/+3
| | | | | | | kmap_atomic allows only one argument now, just move the second. Signed-off-by: Zhao Hongjiang <zhaohongjiang@huawei.com> Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: use spin_lock_irqsave/spin_unlock_irqrestore for PTE updatesJohn David Anglin2013-04-251-4/+1
| | | | | | | | | | | | | | | | | | | | | User applications running on SMP kernels have long suffered from instability and random segmentation faults. This patch improves the situation although there is more work to be done. One of the problems is the various routines in pgtable.h that update page table entries use different locking mechanisms, or no lock at all (set_pte_at). This change modifies the routines to all use the same lock pa_dbit_lock. This lock is used for dirty bit updates in the interruption code. The patch also purges the TLB entries associated with the PTE to ensure that inconsistent values are not used after the page table entry is updated. The UP and SMP code are now identical. The change also includes a minor update to the purge_tlb_entries function in cache.c to improve its efficiency. Signed-off-by: John David Anglin <dave.anglin@bell.net> Cc: Helge Deller <deller@gmx.de> Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: disable preemption while flushing D- or I-caches through TMPALIAS regionJohn David Anglin2013-02-201-0/+2
| | | | | | | | It is necessary to disable preemption during cache flushes done through the TMPALIAS region to ensure that the TLB setup is not clobbered by another flush. Signed-off-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: fixes and cleanups in page cache flushing (4/4)John David Anglin2013-02-201-0/+64
| | | | | | | | | | | | CONFIG_PARISC_TMPALIAS enables clear_user_highpage and copy_user_highpage. These are essentially alternative implementations of clear_user_page and copy_user_page. They don't have anything to do with x86 high pages, but they build on the infrastructure to save a few instructions. Read the comment in clear_user_highpage as it is very important to the implementation. For this reason, there isn't any gain in using the TMPALIAS/highpage approach. Signed-off-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: fixes and cleanups in page cache flushing (3/4)John David Anglin2013-02-201-7/+81
| | | | | | | | | | | | | | | | | flush_cache_mm, for the non current case also uses flush_dcache_page_asm and flush_icache_page_asm which are TMPALIAS flushes. For the non current case, the algorithm used by get_ptep is derived from the vmalloc_to_page implementation in vmalloc.c. It is essentially a generic page table lookup. The other alternative was to duplicate the lookup in entry.S. The break point for switching to a full cache flush is somewhat arbitrary. The same approach is used in flush_cache_range for non current case. In a GCC build and check, many small programs are executed and this change provided a significant performance enhancement, e.g. GCC build time was cut almost in half on a rp3440 at j4. Previously, we always flushed the entire cache. Signed-off-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Helge Deller <deller@gmx.de>
* parisc: fixes and cleanups in page cache flushing (1/4)John David Anglin2013-02-201-28/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This is the first patch in a series of 4, with which the page cache flushing of parisc will gets fixed and enhanced. This even fixes the nasty "minifail" bug (http://wiki.parisc-linux.org/TestCases?highlight=%28minifail%29) which prevented parisc to stay an official debian port. Basically the flush in copy_user_page together with the TLB patch from commit 7139bc1579901b53db7e898789e916ee2fb52d78 is what fixes the minifail bug. This patch still uses the TMPALIAS approach. The new copy_user_page implementation calls flush_dcache_page_asm to flush the user dcache page (crucial for minifail fix) via a kernel TMPALIAS mapping. After that, it just copies the page using the kernel mapping. It does a final flush if needed. Generally it is hard to avoid doing some cache flushes using the kernel mapping (e.g., copy_to_user_page and copy_from_user_page). This patch depends on a subsequent change to pacache.S implementing clear_page_asm and copy_page_asm. These are optimized routines to clear and copy a page. The calls in clear_user_page and copy_user_page could be replaced by calls to memset and memcpy, respectively. I tested prefetch optimizations in clear_page_asm and copy_page_asm but didn't see any significant performance improvement on rp3440. I'm not sure if these are routines are significantly faster than memset and/or memcpy, but they are there for further performance evaluation. Signed-off-by: John David Anglin <dave.anglin@bell.net> Signed-off-by: Helge Deller <deller@gmx.de>
* [PARISC] Purge existing TLB entries in set_pte_at and ptep_set_wrprotectJohn David Anglin2013-01-191-0/+18
| | | | | | | | | | | | | | This patch goes a long way toward fixing the minifail bug, and it  significantly improves the stability of SMP machines such as the rp3440.  When write  protecting a page for COW, we need to purge the existing translation.  Otherwise, the COW break doesn't occur as expected because the TLB may still have a stale entry which allows writes. [jejb: fix up checkpatch errors] Signed-off-by: John David Anglin <dave.anglin@bell.net> Cc: stable@vger.kernel.org Signed-off-by: James Bottomley <JBottomley@Parallels.com>
* mm: replace vma prio_tree with an interval treeMichel Lespinasse2012-10-091-2/+1
| | | | | | | | | | | | | | | | | | | | | | Implement an interval tree as a replacement for the VMA prio_tree. The algorithms are similar to lib/interval_tree.c; however that code can't be directly reused as the interval endpoints are not explicitly stored in the VMA. So instead, the common algorithm is moved into a template and the details (node type, how to get interval endpoints from the node, etc) are filled in using the C preprocessor. Once the interval tree functions are available, using them as a replacement to the VMA prio tree is a relatively simple, mechanical job. Signed-off-by: Michel Lespinasse <walken@google.com> Cc: Rik van Riel <riel@redhat.com> Cc: Hillf Danton <dhillf@gmail.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Catalin Marinas <catalin.marinas@arm.com> Cc: Andrea Arcangeli <aarcange@redhat.com> Cc: David Woodhouse <dwmw2@infradead.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* Disintegrate asm/system.h for PA-RISCDavid Howells2012-03-281-1/+0
| | | | | | | Disintegrate asm/system.h for PA-RISC. Signed-off-by: David Howells <dhowells@redhat.com> cc: linux-parisc@vger.kernel.org
* [PARISC] prevent speculative re-read on cache flushJames Bottomley2011-04-151-1/+12
| | | | | | | | | | | | | | | According to Appendix F, the TLB is the primary arbiter of speculation. Thus, if a page has a TLB entry, it may be speculatively read into the cache. On linux, this can cause us incoherencies because if we're about to do a disk read, we call get_user_pages() to do the flush/invalidate in user space, but we still potentially have the user TLB entries, and the cache could speculate the lines back into userspace (thus causing stale data to be used). This is fixed by purging the TLB entries before we flush through the tmpalias space. Now, the only way the line could be re-speculated is if the user actually tries to touch it (which is not allowed). Signed-off-by: James Bottomley <James.Bottomley@suse.de>
* parisc: flush pages through tmpalias spaceJames Bottomley2011-01-151-92/+17
| | | | | | | | | | | | | | | | | | | The kernel has an 8M tmpailas space (originally designed for copying and clearing pages but now only used for clearing). The idea is to place zeros into the cache above a physical page rather than into the physical page and flush the cache, because often the zeros end up being replaced quickly anyway. We can also use the tmpalias space for flushing a page. The difference here is that we have to do tmpalias processing in the non access data and instruction traps. The principle is the same: as long as we know the physical address and have a virtual address congruent to the real one, the flush will be effective. In order to use the tmpalias space, the icache miss path has to be enhanced to check for the alias region to make the fic instruction effective. Signed-off-by: James Bottomley <James.Bottomley@suse.de>
* parisc: remove trailing space in messagesFrans Pop2010-03-061-2/+2
| | | | | | | | Signed-off-by: Frans Pop <elendil@planet.nl> Cc: linux-parisc@vger.kernel.org Cc: Kyle McMartin <kyle@mcmartin.ca> Cc: Helge Deller <deller@gmx.de> Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
* MM: Pass a PTE pointer to update_mmu_cache() rather than the PTE itselfRussell King2010-02-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On VIVT ARM, when we have multiple shared mappings of the same file in the same MM, we need to ensure that we have coherency across all copies. We do this via make_coherent() by making the pages uncacheable. This used to work fine, until we allowed highmem with highpte - we now have a page table which is mapped as required, and is not available for modification via update_mmu_cache(). Ralf Beache suggested getting rid of the PTE value passed to update_mmu_cache(): On MIPS update_mmu_cache() calls __update_tlb() which walks pagetables to construct a pointer to the pte again. Passing a pte_t * is much more elegant. Maybe we might even replace the pte argument with the pte_t? Ben Herrenschmidt would also like the pte pointer for PowerPC: Passing the ptep in there is exactly what I want. I want that -instead- of the PTE value, because I have issue on some ppc cases, for I$/D$ coherency, where set_pte_at() may decide to mask out the _PAGE_EXEC. So, pass in the mapped page table pointer into update_mmu_cache(), and remove the PTE value, updating all implementations and call sites to suit. Includes a fix from Stephen Rothwell: sparc: fix fallout from update_mmu_cache API change Signed-off-by: Stephen Rothwell <sfr@canb.auug.org.au> Acked-by: Benjamin Herrenschmidt <benh@kernel.crashing.org> Signed-off-by: Russell King <rmk+kernel@arm.linux.org.uk>
* parisc: ensure broadcast tlb purge runs single threadedHelge Deller2009-07-031-8/+15
| | | | | | | | | | | | The TLB flushing functions on hppa, which causes PxTLB broadcasts on the system bus, needs to be protected by irq-safe spinlocks to avoid irq handlers to deadlock the kernel. The deadlocks only happened during I/O intensive loads and triggered pretty seldom, which is why this bug went so long unnoticed. Signed-off-by: Helge Deller <deller@gmx.de> [edited to use spin_lock_irqsave on UP as well since we'd been locking there all this time anyway, --kyle] Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
* parisc: remove CVS keywordsAlexander Beregalov2009-07-031-2/+1
| | | | | | | Signed-off-by: Alexander Beregalov <a.beregalov@gmail.com> Acked-by: Matthew Wilcox <willy@linux.intel.com> Acked-by: Grant Grundler <grundler@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
* parisc: BUG_ON() cleanupHelge Deller2009-03-131-4/+1
| | | | | | | | - convert a few "if (xx) BUG();" to BUG_ON(xx) - remove a few printk()s, as we get a backtrace with BUG_ON() anyway Signed-off-by: Helge Deller <deller@gmx.de> Signed-off-by: Kyle McMartin <kyle@mcmartin.ca>
* on_each_cpu(): kill unused 'retry' parameterJens Axboe2008-06-261-3/+3
| | | | | | | | | It's not even passed on to smp_call_function() anymore, since that was removed. So kill it. Acked-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Reviewed-by: Paul E. McKenney <paulmck@linux.vnet.ibm.com> Signed-off-by: Jens Axboe <jens.axboe@oracle.com>
* arch/parisc/: Spelling fixesJoe Perches2008-02-031-1/+1
| | | | | Signed-off-by: Joe Perches <joe@perches.com> Signed-off-by: Adrian Bunk <bunk@kernel.org>
* Detach sched.h from mm.hAlexey Dobriyan2007-05-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | First thing mm.h does is including sched.h solely for can_do_mlock() inline function which has "current" dereference inside. By dealing with can_do_mlock() mm.h can be detached from sched.h which is good. See below, why. This patch a) removes unconditional inclusion of sched.h from mm.h b) makes can_do_mlock() normal function in mm/mlock.c c) exports can_do_mlock() to not break compilation d) adds sched.h inclusions back to files that were getting it indirectly. e) adds less bloated headers to some files (asm/signal.h, jiffies.h) that were getting them indirectly Net result is: a) mm.h users would get less code to open, read, preprocess, parse, ... if they don't need sched.h b) sched.h stops being dependency for significant number of files: on x86_64 allmodconfig touching sched.h results in recompile of 4083 files, after patch it's only 3744 (-8.3%). Cross-compile tested on all arm defconfigs, all mips defconfigs, all powerpc defconfigs, alpha alpha-up arm i386 i386-up i386-defconfig i386-allnoconfig ia64 ia64-up m68k mips parisc parisc-up powerpc powerpc-up s390 s390-up sparc sparc-up sparc64 sparc64-up um-x86_64 x86_64 x86_64-up x86_64-defconfig x86_64-allnoconfig as well as my two usual configs. Signed-off-by: Alexey Dobriyan <adobriyan@gmail.com> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
* [PARISC] Fixes /proc/cpuinfo cache output on B160LHelge Deller2007-02-171-1/+1
| | | | | Signed-off-by: Helge Deller <deller@gmx.de> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Clean up the cache and tlb headersRandolph Chung2007-02-171-11/+175
| | | | | | | No changes in functionality. Signed-off-by: Randolph Chung <tausq@debian.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Export clear_user_page to modulesMatthew Wilcox2006-10-041-2/+2
| | | | | Signed-off-by: Matthew Wilcox <willy@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Fix CONFIG_DEBUG_SPINLOCKMatthew Wilcox2006-10-041-3/+0
| | | | | | | | | Joel Soete points out that we refer to pa_tlb_lock but only define it if CONFIG_SMP which breaks a uniprocessor build with CONFIG_DEBUG_SPINLOCK enabled. No module refers to pa_tlb_lock, so we can delete the export. Signed-off-by: Matthew Wilcox <willy@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] parisc specific kmap API implementation for pa8800James Bottomley2006-10-041-1/+44
| | | | | | | | | | | | | | | | This patch fixes the pa8800 at a gross level (there are still other subtle incoherency issues which can still cause crashes and HPMCs). What it does is try to force eject inequivalent aliases before they become visible to the L2 cache (which is where we get the incoherence problems). A new function (parisc_requires_coherency) is introduced in asm/processor.h to identify the pa8x00 processors (8800 and 8900) which have the issue. Signed-off-by: James Bottomley <James.Bottomley@SteelEye.com> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Ensure Space ID hashing is turned offKyle McMartin2006-06-271-1/+9
| | | | | | | | | | | Check PDC_CACHE to see if spaceid hashing is turned on, and fail to boot if that is the case. However, some old machines do not implement the PDC_CACHE_RET_SPID firmware call, so continue to boot if the call fails because of PDC_BAD_OPTION (but fail in all other error returns). Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Match show_cache_info with realityKyle McMartin2006-06-271-8/+10
| | | | | | | | show_cache_info and struct pdc_cache_cf were out of sync with published documentation. Fix the reporting of cache associativity and update the pdc_cache_cf bitfields to match documentation. Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Misc. janitorial workHelge Deller2006-04-211-2/+2
| | | | | | | | Fix a spelling mistake, add a KERN_INFO flag, and fix some whitespace uglies. Signed-off-by: Helge Deller <deller@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Add parisc implementation of flush_kernel_dcache_page()James Bottomley2006-03-301-3/+3
| | | | | | | | We need to do a little renaming of our original syntax because of the difference in arguments. Signed-off-by: James Bottomley <jejb@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Make local cache flushes take a void *Matthew Wilcox2006-01-101-4/+4
| | | | | | | | | | | | | Make flush_data_cache_local, flush_instruction_cache_local and flush_tlb_all_local take a void * so they don't have to be cast when using on_each_cpu(). This becomes a problem when on_each_cpu is a macro (as it is in current -mm). Also move the prototype of flush_tlb_all_local into tlbflush.h and remove its declaration from .c files. Signed-off-by: Matthew Wilcox <willy@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PARISC] Add __read_mostly section for pariscHelge Deller2006-01-101-6/+6
| | | | | | | | Flag a whole bunch of things as __read_mostly on parisc. Also flag a few branches as unlikely() and cleanup a bit of code. Signed-off-by: Helge Deller <deller@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* [PATCH] mm: parisc pte atomicityHugh Dickins2005-10-291-15/+9
| | | | | | | | | | | | There's a worrying function translation_exists in parisc cacheflush.h, unaffected by split ptlock since flush_dcache_page is using it on some other mm, without any relevant lock. Oh well, make it a slightly more robust by factoring the pfn check within it. And it looked liable to confuse a camouflaged swap or file entry with a good pte: fix that too. Signed-off-by: Hugh Dickins <hugh@veritas.com> Signed-off-by: Andrew Morton <akpm@osdl.org> Signed-off-by: Linus Torvalds <torvalds@osdl.org>
* [PARISC] Fix parisc_setup_cache_timing to choose a better flush thresholdStuart Brady2005-10-211-4/+6
| | | | | | | | | | | | | | | | update comment about CAFL_STRIDE Signed-off-by: Kyle McMartin <kyle@parisc-linux.org> Fixed a bug in parisc_setup_cache_timing() which caused it to calculate a poor value for parisc_cache_flush_threshold. Thanks to Joel Soete for spotting the bug. Thanks to James Bottomley for pointing out the clean way to fix this. Signed-off-by: Stuart Brady <sdb@parisc-linux.org> Signed-off-by: Kyle McMartin <kyle@parisc-linux.org>
* Linux-2.6.12-rc2v2.6.12-rc2Linus Torvalds2005-04-161-0/+366
Initial git repository build. I'm not bothering with the full history, even though we have it. We can create a separate "historical" git archive of that later if we want to, and in the meantime it's about 3.2GB when imported into git - space that would just make the early git days unnecessarily complicated, when we don't have a lot of good infrastructure for it. Let it rip!
OpenPOWER on IntegriCloud