summaryrefslogtreecommitdiffstats
path: root/arch/x86/include/asm/uv
Commit message (Collapse)AuthorAgeFilesLines
* x86: UV BAU: Increase maximum CPUs per socket/hubJames Custer2014-11-031-2/+2
| | | | | | | | | | | | We have encountered hardware with 18 cores/socket that gives 36 CPUs/socket with hyperthreading enabled. This exceeds the current MAX_CPUS_PER_SOCKET causing a failure in get_cpu_topology. Increase MAX_CPUS_PER_SOCKET to 64 and MAX_CPUS_PER_UVHUB to 128. Signed-off-by: James Custer <jcuster@sgi.com> Cc: Russ Anderson <rja@sgi.com> Link: http://lkml.kernel.org/r/1414952199-185319-1-git-send-email-jcuster@sgi.com Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
* uv: Replace __get_cpu_varChristoph Lameter2014-08-261-5/+5
| | | | | | | | | | Use __this_cpu_read instead. Cc: Hedi Berriche <hedi@sgi.com> Cc: Mike Travis <travis@sgi.com> Cc: Dimitri Sivanich <sivanich@sgi.com> Signed-off-by: Christoph Lameter <cl@linux.com> Signed-off-by: Tejun Heo <tj@kernel.org>
* x86: Replace __get_cpu_var usesChristoph Lameter2014-08-261-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | __get_cpu_var() is used for multiple purposes in the kernel source. One of them is address calculation via the form &__get_cpu_var(x). This calculates the address for the instance of the percpu variable of the current processor based on an offset. Other use cases are for storing and retrieving data from the current processors percpu area. __get_cpu_var() can be used as an lvalue when writing data or on the right side of an assignment. __get_cpu_var() is defined as : #define __get_cpu_var(var) (*this_cpu_ptr(&(var))) __get_cpu_var() always only does an address determination. However, store and retrieve operations could use a segment prefix (or global register on other platforms) to avoid the address calculation. this_cpu_write() and this_cpu_read() can directly take an offset into a percpu area and use optimized assembly code to read and write per cpu variables. This patch converts __get_cpu_var into either an explicit address calculation using this_cpu_ptr() or into a use of this_cpu operations that use the offset. Thereby address calculations are avoided and less registers are used when code is generated. Transformations done to __get_cpu_var() 1. Determine the address of the percpu instance of the current processor. DEFINE_PER_CPU(int, y); int *x = &__get_cpu_var(y); Converts to int *x = this_cpu_ptr(&y); 2. Same as #1 but this time an array structure is involved. DEFINE_PER_CPU(int, y[20]); int *x = __get_cpu_var(y); Converts to int *x = this_cpu_ptr(y); 3. Retrieve the content of the current processors instance of a per cpu variable. DEFINE_PER_CPU(int, y); int x = __get_cpu_var(y) Converts to int x = __this_cpu_read(y); 4. Retrieve the content of a percpu struct DEFINE_PER_CPU(struct mystruct, y); struct mystruct x = __get_cpu_var(y); Converts to memcpy(&x, this_cpu_ptr(&y), sizeof(x)); 5. Assignment to a per cpu variable DEFINE_PER_CPU(int, y) __get_cpu_var(y) = x; Converts to __this_cpu_write(y, x); 6. Increment/Decrement etc of a per cpu variable DEFINE_PER_CPU(int, y); __get_cpu_var(y)++ Converts to __this_cpu_inc(y) Cc: Thomas Gleixner <tglx@linutronix.de> Cc: x86@kernel.org Acked-by: H. Peter Anvin <hpa@linux.intel.com> Acked-by: Ingo Molnar <mingo@kernel.org> Signed-off-by: Christoph Lameter <cl@linux.com> Signed-off-by: Tejun Heo <tj@kernel.org>
* x86/uv: Update the UV3 TLB shootdown logicCliff Wickman2014-06-051-8/+11
| | | | | | | | | | | | | | | | | | | | | | | Update of TLB shootdown code for UV3. Kernel function native_flush_tlb_others() calls uv_flush_tlb_others() on UV to invalidate tlb page definitions on remote cpus. The UV systems have a hardware 'broadcast assist unit' which can be used to broadcast shootdown messages to all cpu's of selected nodes. The behavior of the BAU has changed only slightly with UV3: - UV3 is recognized with is_uv3_hub(). - UV2 functions and structures (uv2_xxx) are in most cases simply renamed to uv2_3_xxx. - Some UV2 error workarounds are not needed for UV3. (see uv_bau_message_interrupt and enable_timeouts) Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/E1WkgWh-0001yJ-3K@eag09.americas.sgi.com [ Removed a few linebreak uglies. ] Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/UV: Set n_lshift based on GAM_GR_CONFIG MMR for UV3Dimitri Sivanich2014-04-012-12/+42
| | | | | | | | | | | | | | | | | | The value of n_lshift for UV is currently set based on the socket m_val. For UV3, set the n_lshift value based on the GAM_GR_CONFIG MMR. This will allow bios to control the n_lshift value independent of the socket m_val. Then n_lshift can be assigned a fixed value across a multi-partition system, allowing for a fixed common global physical address format that is independent of socket m_val. Cleanup unneeded macros. Signed-off-by: Dimitri Sivanich <sivanich@sgi.com> Link: http://lkml.kernel.org/r/20140331143700.GB29916@sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/uv/nmi: Fix Sparse warningsMike Travis2014-01-251-2/+0
| | | | | | | | | | | | | | | | | | Make uv_register_nmi_notifier() and uv_handle_nmi_ping() static to address sparse warnings. Fix problem where uv_nmi_kexec_failed is unused when CONFIG_KEXEC is not defined. Signed-off-by: Mike Travis <travis@sgi.com> Reviewed-by: Hedi Berriche <hedi@sgi.com> Cc: Russ Anderson <rja@sgi.com> Cc: Jason Wessel <jason.wessel@windriver.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Link: http://lkml.kernel.org/r/20140114162551.480872353@asylum.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* Revert "x86/UV: Add uvtrace support"Ingo Molnar2013-11-111-8/+0
| | | | | | | | | | | | | | | | | | | | | | | This reverts commit 8eba18428ac926f436064ac281e76d36d51bd631. uv_trace() is not used by anything, nor is uv_trace_nmi_func, nor uv_trace_func. That's not how we do instrumentation code in the kernel: we add tracepoints, printk()s, etc. so that everyone not just those with magic kernel modules can debug a system. So remove this unused (and misguied) piece of code. Signed-off-by: Ingo Molnar <mingo@kernel.org> Cc: Mike Travis <travis@sgi.com> Cc: Dimitri Sivanich <sivanich@sgi.com> Cc: Hedi Berriche <hedi@sgi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: Jason Wessel <jason.wessel@windriver.com> Link: http://lkml.kernel.org/n/tip-tumfBffmr4jmnt8Gyxanoblg@git.kernel.org
* x86/UV: Add uvtrace supportMike Travis2013-09-241-0/+8
| | | | | | | | | | | | | | | | | This patch adds support for the uvtrace module by providing a skeleton call to the registered trace function. It also provides another separate 'NMI' tracer that is triggered by the system wide 'power nmi' command. Signed-off-by: Mike Travis <travis@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Reviewed-by: Hedi Berriche <hedi@sgi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: Jason Wessel <jason.wessel@windriver.com> Link: http://lkml.kernel.org/r/20130923212501.185052551@asylum.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/UV: Update UV support for external NMI signalsMike Travis2013-09-242-2/+86
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The current UV NMI handler has not been updated for the changes in the system NMI handler and the perf operations. The UV NMI handler reads an MMR in the UV Hub to check to see if the NMI event was caused by the external 'system NMI' that the operator can initiate on the System Mgmt Controller. The problem arises when the perf tools are running, causing millions of perf events per second on very large CPU count systems. Previously this was okay because the perf NMI handler ran at a higher priority on the NMI call chain and if the NMI was a perf event, it would stop calling other NMI handlers remaining on the NMI call chain. Now the system NMI handler calls all the handlers on the NMI call chain including the UV NMI handler. This causes the UV NMI handler to read the MMRs at the same millions per second rate. This can lead to significant performance loss and possible system failures. It also can cause thousands of 'Dazed and Confused' messages being sent to the system console. This effectively makes perf tools unusable on UV systems. To avoid this excessive overhead when perf tools are running, this code has been optimized to minimize reading of the MMRs as much as possible, by moving to the NMI_UNKNOWN notifier chain. This chain is called only when all the users on the standard NMI_LOCAL call chain have been called and none of them have claimed this NMI. There is an exception where the NMI_LOCAL notifier chain is used. When the perf tools are in use, it's possible that the UV NMI was captured by some other NMI handler and then either ignored or mistakenly processed as a perf event. We set a per_cpu ('ping') flag for those CPUs that ignored the initial NMI, and then send them an IPI NMI signal. The NMI_LOCAL handler on each cpu does not need to read the MMR, but instead checks the in memory flag indicating it was pinged. There are two module variables, 'ping_count' indicating how many requested NMI events occurred, and 'ping_misses' indicating how many stray NMI events. These most likely are perf events so it shows the overhead of the perf NMI interrupts and how many MMR reads were avoided. This patch also minimizes the reads of the MMRs by having the first cpu entering the NMI handler on each node set a per HUB in-memory atomic value. (Having a per HUB value avoids sending lock traffic over NumaLink.) Both types of UV NMIs from the SMI layer are supported. Signed-off-by: Mike Travis <travis@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Reviewed-by: Hedi Berriche <hedi@sgi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: Jason Wessel <jason.wessel@windriver.com> Link: http://lkml.kernel.org/r/20130923212500.353547733@asylum.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/UV: Move NMI supportMike Travis2013-09-241-0/+2
| | | | | | | | | | | | | | | | | | | This patch moves the UV NMI support from the x2apic file to a new separate uv_nmi.c file in preparation for the next sequence of patches. It prevents upcoming bloat of the x2apic file, and has the added benefit of putting the upcoming /sys/module parameters under the name 'uv_nmi' instead of 'x2apic_uv_x', which was obscure. Signed-off-by: Mike Travis <travis@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Reviewed-by: Hedi Berriche <hedi@sgi.com> Cc: Peter Zijlstra <a.p.zijlstra@chello.nl> Cc: Paul Mackerras <paulus@samba.org> Cc: Arnaldo Carvalho de Melo <acme@ghostprotocols.net> Cc: Jason Wessel <jason.wessel@windriver.com> Link: http://lkml.kernel.org/r/20130923212500.183295611@asylum.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86, trace: Add irq vector tracepointsSeiji Aguchi2013-06-201-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [Purpose of this patch] As Vaibhav explained in the thread below, tracepoints for irq vectors are useful. http://www.spinics.net/lists/mm-commits/msg85707.html <snip> The current interrupt traces from irq_handler_entry and irq_handler_exit provide when an interrupt is handled. They provide good data about when the system has switched to kernel space and how it affects the currently running processes. There are some IRQ vectors which trigger the system into kernel space, which are not handled in generic IRQ handlers. Tracing such events gives us the information about IRQ interaction with other system events. The trace also tells where the system is spending its time. We want to know which cores are handling interrupts and how they are affecting other processes in the system. Also, the trace provides information about when the cores are idle and which interrupts are changing that state. <snip> On the other hand, my usecase is tracing just local timer event and getting a value of instruction pointer. I suggested to add an argument local timer event to get instruction pointer before. But there is another way to get it with external module like systemtap. So, I don't need to add any argument to irq vector tracepoints now. [Patch Description] Vaibhav's patch shared a trace point ,irq_vector_entry/irq_vector_exit, in all events. But there is an above use case to trace specific irq_vector rather than tracing all events. In this case, we are concerned about overhead due to unwanted events. So, add following tracepoints instead of introducing irq_vector_entry/exit. so that we can enable them independently. - local_timer_vector - reschedule_vector - call_function_vector - call_function_single_vector - irq_work_entry_vector - error_apic_vector - thermal_apic_vector - threshold_apic_vector - spurious_apic_vector - x86_platform_ipi_vector Also, introduce a logic switching IDT at enabling/disabling time so that a time penalty makes a zero when tracepoints are disabled. Detailed explanations are as follows. - Create trace irq handlers with entering_irq()/exiting_irq(). - Create a new IDT, trace_idt_table, at boot time by adding a logic to _set_gate(). It is just a copy of original idt table. - Register the new handlers for tracpoints to the new IDT by introducing macros to alloc_intr_gate() called at registering time of irq_vector handlers. - Add checking, whether irq vector tracing is on/off, into load_current_idt(). This has to be done below debug checking for these reasons. - Switching to debug IDT may be kicked while tracing is enabled. - On the other hands, switching to trace IDT is kicked only when debugging is disabled. In addition, the new IDT is created only when CONFIG_TRACING is enabled to avoid being used for other purposes. Signed-off-by: Seiji Aguchi <seiji.aguchi@hds.com> Link: http://lkml.kernel.org/r/51C323ED.5050708@hds.com Signed-off-by: H. Peter Anvin <hpa@linux.intel.com> Cc: Steven Rostedt <rostedt@goodmis.org>
* x86, uv, uv3: Trim MMR register definitions after code changes for SGI UV3Mike Travis2013-02-111-2479/+1
| | | | | | | | | | | | | This patch trims the MMR register definitions after the updates for the SGI UV3 system have been applied. Note that because these definitions are automatically generated from the RTL we cannot control the length of the names. Therefore there are lines that exceed 80 characters. Signed-off-by: Mike Travis <travis@sgi.com> Link: http://lkml.kernel.org/r/20130211194509.173026880@gulag1.americas.sgi.com Acked-by: Russ Anderson <rja@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* x86, uv, uv3: Update Hub Info for SGI UV3Mike Travis2013-02-111-8/+36
| | | | | | | | | | | | This patch updates the UV HUB info for UV3. The "is_uv3_hub" and "is_uvx_hub" (UV2 or UV3) functions are added as well as the addresses and sizes of the MMR regions for UV3. Signed-off-by: Mike Travis <travis@sgi.com> Link: http://lkml.kernel.org/r/20130211194508.610723192@gulag1.americas.sgi.com Acked-by: Russ Anderson <rja@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* x86, uv, uv3: Update MMR register definitions for SGI Ultraviolet System 3 (UV3)Mike Travis2013-02-111-108/+3366
| | | | | | | | | | | | | | | | | | | This patch updates the MMR register definitions for the SGI UV3 system. Note that because these definitions are automatically generated from the RTL we cannot control the length of the names. Therefore there are lines that exceed 80 characters. All the new MMR definitions are added in this patch. The patches that follow then update the references. The last patch is a "trim" patch which reduces the size of the MMR definitions file by about a third. This keeps "bi-sectability" in place as the intermediate patches would not compile correctly if the trimmed MMR defines were done first. Signed-off-by: Mike Travis <travis@sgi.com> Link: http://lkml.kernel.org/r/20130211194508.326204556@gulag1.americas.sgi.com Acked-by: Russ Anderson <rja@sgi.com> Reviewed-by: Dimitri Sivanich <sivanich@sgi.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* arch/x86/platform/uv: Fix incorrect tlb flush all issueAlex Shi2013-01-241-1/+1
| | | | | | | | | | | | | | | | | | | | | | The flush tlb optimization code has logical issue on UV platform. It doesn't flush the full range at all, since it simply ignores its 'end' parameter (and hence also the "all" indicator) in uv_flush_tlb_others() function. Cliff's notes: | I tested the patch on a UV. It has the effect of either | clearing 1 or all TLBs in a cpu. I added some debugging to | test for the cases when clearing all TLBs is overkill, and in | practice it happens very seldom. Reported-by: Jan Beulich <jbeulich@suse.com> Signed-off-by: Alex Shi <alex.shi@intel.com> Signed-off-by: Cliff Wickman <cpw@sgi.com> Tested-by: Cliff Wickman <cpw@sgi.com> Cc: "H. Peter Anvin" <hpa@zytor.com> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Ingo Molnar <mingo@kernel.org>
* Merge branch 'x86-mm-for-linus' of ↵Linus Torvalds2012-07-261-2/+3
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip Pull x86/mm changes from Peter Anvin: "The big change here is the patchset by Alex Shi to use INVLPG to flush only the affected pages when we only need to flush a small page range. It also removes the special INVALIDATE_TLB_VECTOR interrupts (32 vectors!) and replace it with an ordinary IPI function call." Fix up trivial conflicts in arch/x86/include/asm/apic.h (added code next to changed line) * 'x86-mm-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86/tlb: Fix build warning and crash when building for !SMP x86/tlb: do flush_tlb_kernel_range by 'invlpg' x86/tlb: replace INVALIDATE_TLB_VECTOR by CALL_FUNCTION_VECTOR x86/tlb: enable tlb flush range support for x86 mm/mmu_gather: enable tlb flush range in generic mmu_gather x86/tlb: add tlb_flushall_shift knob into debugfs x86/tlb: add tlb_flushall_shift for specific CPU x86/tlb: fall back to flush all when meet a THP large page x86/flush_tlb: try flush_tlb_single one by one in flush_tlb_range x86/tlb_info: get last level TLB entry number of CPU x86: Add read_mostly declaration/definition to variables from smp.h x86: Define early read-mostly per-cpu macros
| * x86/flush_tlb: try flush_tlb_single one by one in flush_tlb_rangeAlex Shi2012-06-271-2/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | x86 has no flush_tlb_range support in instruction level. Currently the flush_tlb_range just implemented by flushing all page table. That is not the best solution for all scenarios. In fact, if we just use 'invlpg' to flush few lines from TLB, we can get the performance gain from later remain TLB lines accessing. But the 'invlpg' instruction costs much of time. Its execution time can compete with cr3 rewriting, and even a bit more on SNB CPU. So, on a 512 4KB TLB entries CPU, the balance points is at: (512 - X) * 100ns(assumed TLB refill cost) = X(TLB flush entries) * 100ns(assumed invlpg cost) Here, X is 256, that is 1/2 of 512 entries. But with the mysterious CPU pre-fetcher and page miss handler Unit, the assumed TLB refill cost is far lower then 100ns in sequential access. And 2 HT siblings in one core makes the memory access more faster if they are accessing the same memory. So, in the patch, I just do the change when the target entries is less than 1/16 of whole active tlb entries. Actually, I have no data support for the percentage '1/16', so any suggestions are welcomed. As to hugetlb, guess due to smaller page table, and smaller active TLB entries, I didn't see benefit via my benchmark, so no optimizing now. My micro benchmark show in ideal scenarios, the performance improves 70 percent in reading. And in worst scenario, the reading/writing performance is similar with unpatched 3.4-rc4 kernel. Here is the reading data on my 2P * 4cores *HT NHM EP machine, with THP 'always': multi thread testing, '-t' paramter is thread number: with patch unpatched 3.4-rc4 ./mprotect -t 1 14ns 24ns ./mprotect -t 2 13ns 22ns ./mprotect -t 4 12ns 19ns ./mprotect -t 8 14ns 16ns ./mprotect -t 16 28ns 26ns ./mprotect -t 32 54ns 51ns ./mprotect -t 128 200ns 199ns Single process with sequencial flushing and memory accessing: with patch unpatched 3.4-rc4 ./mprotect 7ns 11ns ./mprotect -p 4096 -l 8 -n 10240 21ns 21ns [ hpa: http://lkml.kernel.org/r/1B4B44D9196EFF41AE41FDA404FC0A100BFF94@SHSMSX101.ccr.corp.intel.com has additional performance numbers. ] Signed-off-by: Alex Shi <alex.shi@intel.com> Link: http://lkml.kernel.org/r/1340845344-27557-3-git-send-email-alex.shi@intel.com Signed-off-by: H. Peter Anvin <hpa@zytor.com>
* | x86/uv: Work around UV2 BAU hangsCliff Wickman2012-06-251-10/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | On SGI's UV2 the BAU (Broadcast Assist Unit) driver can hang under a heavy load. To cure this: - Disable the UV2 extended status mode (see UV2_EXT_SHFT), as this mode changes BAU behavior in more ways then just delivering an extra bit of status. Revert status to just two meaningful bits, like UV1. - Use no IPI-style resets on UV2. Just give up the request for whatever the reason it failed and let it be accomplished with the legacy IPI method. - Use no alternate sending descriptor (the former UV2 workaround bcp->using_desc and handle_uv2_busy() stuff). Just disable the use of the BAU for a period of time in favor of the legacy IPI method when the h/w bug leaves a descriptor busy. -- new tunable: giveup_limit determines the threshold at which a hub is so plugged that it should do all requests with the legacy IPI method for a period of time -- generalize disable_for_congestion() (renamed disable_for_period()) for use whenever a hub should avoid using the BAU for a period of time Also: - Fix find_another_by_swack(), which is part of the UV2 bug workaround - Correct and clarify the statistics (new stats s_overipilimit, s_giveuplimit, s_enters, s_ipifordisabled, s_plugged, s_congested) Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20120622131459.GC31884@sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* | x86/uv: Implement UV BAU runtime enable and disable control via /proc/sgi_uv/Cliff Wickman2012-06-251-0/+2
|/ | | | | | | | | | | | | | | | | | This patch enables the BAU to be turned on or off dynamically. echo "on" > /proc/sgi_uv/ptc_statistics echo "off" > /proc/sgi_uv/ptc_statistics The system may be booted with or without the nobau option. Whether the system currently has the BAU off can be seen in the /proc file -- normally with the baustats script. Each cpu will have a 1 in the bauoff field if the BAU was turned off, so baustats will give a count of cpus that have it off. Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20120622131330.GB31884@sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/uv: Fix UV2 BAU legacy modeCliff Wickman2012-06-081-1/+0
| | | | | | | | | | | | | | | | | The SGI Altix UV2 BAU (Broadcast Assist Unit) as used for tlb-shootdown (selective broadcast mode) always uses UV2 broadcast descriptor format. There is no need to clear the 'legacy' (UV1) mode, because the hardware always uses UV2 mode for selective broadcast. But the BIOS uses general broadcast and legacy mode, and the hardware pays attention to the legacy mode bit for general broadcast. So the kernel must not clear that mode bit. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: <stable@kernel.org> Link: http://lkml.kernel.org/r/E1SccoO-0002Lh-Cb@eag09.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@kernel.org>
* x86/uv: Fix uv_gpa_to_soc_phys_ram() shiftRuss Anderson2012-01-261-2/+2
| | | | | | | | | | uv_gpa_to_soc_phys_ram() was inadvertently ignoring the shift values. This fix takes the shift into account. Signed-off-by: Russ Anderson <rja@sgi.com> Cc: <stable@kernel.org> Link: http://lkml.kernel.org/r/20120119020753.GA7228@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86/UV2: Add accounting for BAU strong nacksCliff Wickman2012-01-171-0/+1
| | | | | | | | | This patch adds separate accounting of UV2 message "strong nack's" in the BAU statistics. Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20120116212238.GF5767@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86/UV2: Work around BAU bugCliff Wickman2012-01-171-1/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | This patch implements a workaround for a UV2 hardware bug. The bug is a non-atomic update of a memory-mapped register. When hardware message delivery and software message acknowledge occur simultaneously the pending message acknowledge for the arriving message may be lost. This causes the sender's message status to stay busy. Part of the workaround is to not acknowledge a completed message until it is verified that no other message is actually using the resource that is mistakenly recorded in the completed message. Part of the workaround is to test for long elapsed time in such a busy condition, then handle it by using a spare sending descriptor. The stay-busy condition is eventually timed out by hardware, and then the original sending descriptor can be re-used. Most of that logic change is in keeping track of the current descriptor and the state of the spares. The occurrences of the workaround are added to the BAU statistics. Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20120116211947.GC5767@sgi.com Cc: <stable@kernel.org> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86/UV2: Fix new UV2 hardware by using native UV2 broadcast modeCliff Wickman2012-01-171-7/+86
| | | | | | | | | | | | | Update the use of the Broadcast Assist Unit on SGI Altix UV2 to the use of native UV2 mode on new hardware (not the legacy mode). UV2 native mode has a different format for a broadcast message. We also need quick differentiaton between UV1 and UV2. Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20120116211750.GA5767@sgi.com Cc: <stable@kernel.org> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Fix UV2 hub part numberJack Steiner2011-12-051-0/+1
| | | | | | | | | | | | There was a mixup when the SGI UV2 hub chip was sent to be fabricated, and it ended up with the wrong part number in the HRP_NODE_ID mmr. Future versions of the chip will (may) have the correct part number. Change the UV infrastructure to recognize both part numbers as valid IDs of a UV2 hub chip. Signed-off-by: Jack Steiner <steiner@sgi.com> Link: http://lkml.kernel.org/r/20111129210058.GA20452@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* Merge branch 'x86-uv-for-linus' of ↵Linus Torvalds2011-10-282-3/+35
|\ | | | | | | | | | | | | git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip * 'x86-uv-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: x86: uv2: Workaround for UV2 Hub bug (system global address format)
| * x86: uv2: Workaround for UV2 Hub bug (system global address format)Jack Steiner2011-09-212-3/+35
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a workaround for a UV2 hub bug that affects the format of system global addresses. The GRU API for UV2 was inadvertently broken by a hardware change. The format of the physical address used for TLB dropins and for addresses used with instructions running in unmapped mode has changed. This change was not documented and became apparent only when diags failed running on system simulators. For UV1, TLB and GRU instruction physical addresses are identical to socket physical addresses (although high NASID bits must be OR'ed into the address). For UV2, socket physical addresses need to be converted. The NODE portion of the physical address needs to be shifted so that the low bit is in bit 39 or bit 40, depending on an MMR value. It is not yet clear if this bug will be fixed in a silicon respin. If it is fixed, the hub revision will be incremented & the workaround disabled. Signed-off-by: Jack Steiner <steiner@sgi.com> Cc: Ingo Molnar <mingo@elte.hu> Cc: "H. Peter Anvin" <hpa@zytor.com> Cc: <stable@kernel.org> Signed-off-by: Andrew Morton <akpm@linux-foundation.org> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
* | x86: Use xadd helper more widelyJeremy Fitzhardinge2011-08-291-5/+1
|/ | | | | | | | This covers the trivial cases from open-coded xadd to the xadd macros. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Link: http://lkml.kernel.org/r/4E5BCC40.3030501@goop.org Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* x86, UV: Correct UV2 BAU destination timeoutcpw@sgi.com2011-06-211-5/+13
| | | | | | | | | | | | | | | | | | | | Correct the UV2 broacast assist unit's destination timeout period. And the activation status register in UV2 should be tested for a destination timeout with a 4, not a 2. The values for Active versus Timeout were reversed. This patch is critical for TLB shootdown on an Altix UV2 system (i.e. the follow-on to the current Altix UV). Destination timeout period: The period is set in 4 bits of memory-mapped register MISC_CONTROL. The left bit toggles base period between 10us and 80us. The other 3 bits are the multiplier. Decimal 15, hex f, gives the maximum: 7 * 80us Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20110621122243.117324443@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Remove cpumask_t from the stackcpw@sgi.com2011-06-211-0/+1
| | | | | | | | | | | | | | Remove the large stack-resident cpumask_t from reset_with_ipi()'s stack by allocating one per uvhub. Due to the limited size of the stack the potentially huge cpumask_t may cause stack overrun. We haven't seen it happen yet, but we need to make it a practice not to push such structures onto the stack. Signed-off-by: Cliff Wickman <cpw@sgi.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Link: http://lkml.kernel.org/r/20110621122242.832589130@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Rename hubmask to pnmaskcpw@sgi.com2011-06-211-6/+6
| | | | | | | | | | | | Rename 'bau_targ_hubmask' to 'pnmask' for clarity. The BAU distribution bit mask is indexed by pnode number, not hub or blade number. This important fact is not clear while the mask is called a 'hubmask'. Signed-off-by: Cliff Wickman <cpw@sgi.com> Link: http://lkml.kernel.org/r/20110621122242.630995969@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Inline header file functionscpw@sgi.com2011-06-211-14/+14
| | | | | | | | | | | | Make all the functions in uv_bau.h inline so that it can be included in the fake prom (used in simulations). If not inlined the unused functions will generate compiler warnings. Signed-off-by: Cliff Wickman <cpw@sgi.com> Reviewed-by: Pekka Enberg <penberg@kernel.org> Link: http://lkml.kernel.org/r/20110621122242.230529678@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Clean up uv_mmrs.hJack Steiner2011-05-301-1281/+1608
| | | | | | | | | | | | | | | | | | | | | | | | No code changes. Reformat definitions to make it more readable. I fixed alignment of comments in the structure definitions. Also aligned comments and most field definitions & values. Also sorted the defines for the SHIFT & MASK values for each MMR. This make the file visually much more acceptable. Some of the symbol names are still quite long. The file is based on post-processing of verilog definitions that are used for the node controller chip design. Although some symbol names are not what I would chose, I would like to maintain compatibility with the names used by the chip designers. We have a number of cross-reference utilities & having common names is important. Signed-off-by: Jack Steiner <steiner@sgi.com> Link: http://lkml.kernel.org/r/20110527145256.GA31224@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu> -- arch/x86/include/asm/uv/uv_mmrs.h | 2873 +++++++++++++++++++++----------------- 1 file changed, 1600 insertions(+), 1273 deletions(-)
* x86, UV: Clean up uv_tlb.cCliff Wickman2011-05-251-203/+351
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | SGI UV's uv_tlb.c driver has become rather hard to read, with overly large functions, non-standard coding style and (way) too long variable, constant and function names and non-obvious code flow sequences. This patch improves the readability and maintainability of the driver significantly, by doing the following strict code cleanups with no side effects: - Split long functions into shorter logical functions. - Shortened some variable and structure member names. - Added special functions for reads and writes of MMR regs with very long names. - Added the 'tunables' table to shortened tunables_write(). - Added the 'stat_description' table to shorten uv_ptc_proc_write(). - Pass fewer 'stat' arguments where it can be derived from the 'bcp' argument. - Function definitions consistent on one line, and inline in few (short) cases. - Moved some small structures and an atomic inline function to the header file. - Moved some local variables to the blocks where they are used. - Updated the copyright date. - Shortened uv_write_global_mmr64() etc. using some aliasing; no line breaks. Renamed many uv_.. functions that are not exported. - Aligned structure fields. [ note that not all structures are aligned the same way though; I'd like to keep the extensive commenting in some of them. ] - Shortened some long structure names. - Standard pass/fail exit from init_per_cpu() - Vertical alignment for mass initializations. - More separation between blocks of code. Tested on a 16-processor Altix UV. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: penberg@kernel.org Link: http://lkml.kernel.org/r/E1QOw12-0004MN-Lp@eag09.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Add support for SGI UV2 hub chipJack Steiner2011-05-253-207/+918
| | | | | | | | | | | | | | | | This patch adds support for a new version of the SGI UV hub chip. The hub chip is the node controller that connects multiple blades into a larger coherent SSI. For the most part, UV2 is compatible with UV1. The majority of the changes are in the addresses of MMRs and in a few cases, the contents of MMRs. These changes are the result in changes in the system topology such as node configuration, processor types, maximum nodes, physical address sizes, etc. Signed-off-by: Jack Steiner <steiner@sgi.com> Link: http://lkml.kernel.org/r/20110511175028.GA18006@sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: Fix UV BAU for non-consecutive nasidsCliff Wickman2011-05-121-4/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This is a fix for the SGI Altix-UV Broadcast Assist Unit code, which is used for TLB flushing. Certain hardware configurations (that customers are ordering) cause nasids (numa address space id's) to be non-consecutive. Specifically, once you have more than 4 blades in a IRU (Individual Rack Unit - or 1/2 rack) but less than the maximum of 16, the nasid numbering becomes non-consecutive. This currently results in a 'catastrophic error' (CATERR) detected by the firmware during OS boot. The BAU is generating an 'INTD' request that is targeting a non-existent nasid value. Such configurations may also occur when a blade is configured off because of hardware errors. (There is one UV hub per blade.) This patch is required to support such configurations. The problem with the tlb_uv.c code is that is using the consecutive hub numbers as indices to the BAU distribution bit map. These are simply the ordinal position of the hub or blade within its partition. It should be using physical node numbers (pnodes), which correspond to the physical nasid values. Use of the hub number only works as long as the nasids in the partition are consecutive and increase with a stride of 1. This patch changes the index to be the pnode number, thus allowing nasids to be non-consecutive. It also provides a table in local memory for each cpu to translate target cpu number to target pnode and nasid. And it improves naming to properly reflect 'node' and 'uvhub' versus 'nasid'. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: <stable@kernel.org> Link: http://lkml.kernel.org/r/E1QJmxX-0002Mz-Fk@eag09.americas.sgi.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Fix NMI handler for UV platformsJack Steiner2011-05-102-1/+17
| | | | | | | | | | | | | | | | | | This fixes problems seen on UV systems handling NMIs from the node controller. I isolated the "dazed..." messages that I saw earlier to a bug in the BMC on our platform. It was sending NMIs w/o properly setting a register that indicated the source of NMI. So rather than _assuming_ any unhandled NMI came from the UV system maintenance console (SMC), add a check to verify that the SMC actually sent the NMI. Signed-off-by: Jack Steiner <steiner@sgi.com> Cc: gorcunov@gmail.com Cc: dzickus@redhat.com Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Initialize the broadcast assist unit base destination node id properlyCliff Wickman2011-03-091-1/+1
| | | | | | | | | | | The BAU's initialization of the broadcast description header is lacking the coherence domain (high bits) in the nasid. This causes a catastrophic system failure when running on a system with multiple coherence domains. Signed-off-by: Cliff Wickman <cpw@sgi.com> LKML-Reference: <E1PxKBB-0005F0-3U@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV, BAU: Extend for more than 16 cpus per socketCliff Wickman2011-01-031-4/+5
| | | | | | | | | | | | | | | | | | | Fix a hard-coded limit of a maximum of 16 cpu's per socket. The UV Broadcast Assist Unit code initializes by scanning the cpu topology of the system and assigning a master cpu for each socket and UV hub. That scan had an assumption of a limit of 16 cpus per socket. With Westmere we are going over that limit. The UV hub hardware will allow up to 32. If the scan finds the system has gone over that limit it returns an error and we print a warning and fall back to doing TLB shootdowns without the BAU. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: <stable@kernel.org> # .37.x LKML-Reference: <E1PZol7-0000mM-77@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86: UV: Address interrupt/IO port operation conflictDimitri Sivanich2010-11-182-1/+22
| | | | | | | | | | | | | | | | | | | | | | | This patch for SGI UV systems addresses a problem whereby interrupt transactions being looped back from a local IOH, through the hub to a local CPU can (erroneously) conflict with IO port operations and other transactions. To workaound this we set a high bit in the APIC IDs used for interrupts. This bit appears to be ignored by the sockets, but it avoids the conflict in the hub. Signed-off-by: Dimitri Sivanich <sivanich@sgi.com> LKML-Reference: <20101116222352.GA8155@sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu> ___ arch/x86/include/asm/uv/uv_hub.h | 4 ++++ arch/x86/include/asm/uv/uv_mmrs.h | 19 ++++++++++++++++++- arch/x86/kernel/apic/x2apic_uv_x.c | 25 +++++++++++++++++++++++-- arch/x86/platform/uv/tlb_uv.c | 2 +- arch/x86/platform/uv/uv_time.c | 4 +++- 5 files changed, 49 insertions(+), 5 deletions(-)
* x86, UV: Update node controller MMRsJack Steiner2010-11-101-93/+96
| | | | | | | | | | | | | | | A new version of the SGI UV hub node controller is being developed. A few of the MMRs (control registers) that exist on the current hub no longer exist on the new hub. Fortunately, there are alternate MMRs that are are functionally equivalent and that exist on both hubs. This patch changes the UV code to use MMRs that exist in BOTH versions of the hub node controller. Signed-off-by: Jack Steiner <steiner@sgi.com> LKML-Reference: <20101106204056.GA27584@sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, uv: Enable Westmere support on SGI UVRuss Anderson2010-10-261-3/+18
| | | | | | | | | | | Enable Westmere support on SGI UV. The UV initialization code is dependent on the APICID bits. Westmere-EX uses different APIC bit mapping than Nehalem-EX. This code reads the apic shift value from a UV MMR to do the proper bit decoding to determint the pnode. Signed-off-by: Russ Anderson <rja@sgi.com> LKML-Reference: <20101026212728.GB15071@sgi.com> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
* x86, UV: Modularize BAU send and waitCliff Wickman2010-06-081-1/+0
| | | | | | | | | | | | Streamline the large uv_flush_send_and_wait() function by use of a couple of helper functions. And remove some excess comments. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004ay-IH@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: BAU broadcast to the local hubCliff Wickman2010-06-081-0/+5
| | | | | | | | | | | | | | | | | | | | | | Make the Broadcast Assist Unit driver use the BAU for TLB shootdowns of cpu's on the local uvhub. It was previously thought that IPI might be faster to the cpu's on the local hub. But the IPI operation would have to follow the completion of the BAU broadcast anyway. So we broadcast to the local uvhub in all cases except when the current cpu was the only local cpu in the mask. This simplifies uv_flush_send_and_wait() in that it returns either all shootdowns complete, or none. Adjust the statistics to account for shootdowns on the local uvhub. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004aq-G7@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Remove BAU check for stay-busyCliff Wickman2010-06-081-1/+0
| | | | | | | | | | | | | | | | | | Remove a faulty assumption that a long running BAU request has encountered a hardware problem and will never finish. Numalink congestion can make a request appear to have encountered such a problem, but it is not safe to cancel the request. If such a cancel is done but a reply is later received we can miss a TLB shootdown. We depend upon the max_bau_concurrent 'throttle' to prevent the stay-busy case from happening. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004ad-BV@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: BAU structure rearrangingCliff Wickman2010-06-081-39/+51
| | | | | | | | | | | Move some structure definitions from the C code to the BAU header file, and change the organization of that header file a little. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004aI-54@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Shorten access to BAU statistics structureCliff Wickman2010-06-081-0/+1
| | | | | | | | | | | Use a pointer from the per-cpu BAU control structure to the per-cpu BAU statistics structure. We nearly always know the first before needing the second. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004aB-2k@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Disable BAU on network congestionCliff Wickman2010-06-081-0/+4
| | | | | | | | | | | | | | | The numalink network can become so congested that TLB shootdown using the Broadcast Assist Unit becomes slower than using IPI's. In that case, disable the use of the BAU for a period of time. The period is tunable. When the period expires the use of the BAU is re-enabled. A count of these actions is added to the statistics file. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNy-0004a4-0a@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: BAU tunables into a debugfs fileCliff Wickman2010-06-081-16/+37
| | | | | | | | | | | | | | | | | Make the Broadcast Assist Unit driver's nine tuning values variable by making them accessible through a read/write debugfs file. The file will normally be mounted as /sys/kernel/debug/sgi_uv/bau_tunables. The tunables are kept in each cpu's per-cpu BAU structure. The patch also does a little name improvement, and corrects the reset of two destination timeout counters. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNx-0004Zx-Uo@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
* x86, UV: Calculate BAU destination timeoutCliff Wickman2010-06-081-0/+12
| | | | | | | | | | | | | | | Calculate the Broadcast Assist Unit's destination timeout period from the values in the relevant MMR's. Store it in each cpu's per-cpu BAU structure so that a destination timeout can be differentiated from a 'plugged' situation in which all software ack resources are already allocated and a timeout is pending. That case returns an immediate destination error. Signed-off-by: Cliff Wickman <cpw@sgi.com> Cc: gregkh@suse.de LKML-Reference: <E1OJvNx-0004Zq-RK@eag09.americas.sgi.com> Signed-off-by: Ingo Molnar <mingo@elte.hu>
OpenPOWER on IntegriCloud