1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
|
.\" -*- nroff -*-
.\"
.\" Copyright (c) 2000 Doug Rabson
.\"
.\" All rights reserved.
.\"
.\" This program is free software.
.\"
.\" Redistribution and use in source and binary forms, with or without
.\" modification, are permitted provided that the following conditions
.\" are met:
.\" 1. Redistributions of source code must retain the above copyright
.\" notice, this list of conditions and the following disclaimer.
.\" 2. Redistributions in binary form must reproduce the above copyright
.\" notice, this list of conditions and the following disclaimer in the
.\" documentation and/or other materials provided with the distribution.
.\"
.\" THIS SOFTWARE IS PROVIDED BY THE DEVELOPERS ``AS IS'' AND ANY EXPRESS OR
.\" IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
.\" OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
.\" IN NO EVENT SHALL THE DEVELOPERS BE LIABLE FOR ANY DIRECT, INDIRECT,
.\" INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
.\" NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
.\" DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
.\" THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
.\" (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
.\" THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
.\"
.\" $FreeBSD$
.\"
.Dd April 26, 2011
.Dt TASKQUEUE 9
.Os
.Sh NAME
.Nm taskqueue
.Nd asynchronous task execution
.Sh SYNOPSIS
.In sys/param.h
.In sys/kernel.h
.In sys/malloc.h
.In sys/queue.h
.In sys/taskqueue.h
.Bd -literal
typedef void (*task_fn_t)(void *context, int pending);
typedef void (*taskqueue_enqueue_fn)(void *context);
struct task {
STAILQ_ENTRY(task) ta_link; /* link for queue */
u_short ta_pending; /* count times queued */
u_short ta_priority; /* priority of task in queue */
task_fn_t ta_func; /* task handler */
void *ta_context; /* argument for handler */
};
struct timeout_task;
.Ed
.Ft struct taskqueue *
.Fn taskqueue_create "const char *name" "int mflags" "taskqueue_enqueue_fn enqueue" "void *context"
.Ft struct taskqueue *
.Fn taskqueue_create_fast "const char *name" "int mflags" "taskqueue_enqueue_fn enqueue" "void *context"
.Ft void
.Fn taskqueue_free "struct taskqueue *queue"
.Ft int
.Fn taskqueue_enqueue "struct taskqueue *queue" "struct task *task"
.Ft int
.Fn taskqueue_enqueue_fast "struct taskqueue *queue" "struct task *task"
.Ft int
.Fn taskqueue_enqueue_timeout "struct taskqueue *queue" "struct timeout_task *timeout_task" "int ticks"
.Ft int
.Fn taskqueue_cancel "struct taskqueue *queue" "struct task *task" "u_int *pendp"
.Ft int
.Fn taskqueue_cancel_timeout "struct taskqueue *queue" "struct timeout_task *timeout_task" "u_int *pendp"
.Ft void
.Fn taskqueue_drain "struct taskqueue *queue" "struct task *task"
.Ft void
.Fn taskqueue_drain_timeout "struct taskqueue *queue" "struct timeout_task *timeout_task"
.Ft int
.Fn taskqueue_member "struct taskqueue *queue" "struct thread *td"
.Ft void
.Fn taskqueue_run "struct taskqueue *queue"
.Fn TASK_INIT "struct task *task" "int priority" "task_fn_t func" "void *context"
.Fn TASKQUEUE_DECLARE "name"
.Fn TASKQUEUE_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" "init"
.Fn TASKQUEUE_FAST_DEFINE "name" "taskqueue_enqueue_fn enqueue" "void *context" "init"
.Fn TASKQUEUE_DEFINE_THREAD "name"
.Fn TASKQUEUE_FAST_DEFINE_THREAD "name"
.Fn TIMEOUT_TASK_INIT "struct taskqueue *queue" "struct timeout_task *timeout_task" "int priority" "task_fn_t func" "void *context"
.Sh DESCRIPTION
These functions provide a simple interface for asynchronous execution
of code.
.Pp
The function
.Fn taskqueue_create
is used to create new queues.
The arguments to
.Fn taskqueue_create
include a name that should be unique,
a set of
.Xr malloc 9
flags that specify whether the call to
.Fn malloc
is allowed to sleep,
a function that is called from
.Fn taskqueue_enqueue
when a task is added to the queue,
and a pointer to the memory location where the identity of the
thread that services the queue is recorded.
.\" XXX The rest of the sentence gets lots in relation to the first part.
The function called from
.Fn taskqueue_enqueue
must arrange for the queue to be processed
(for instance by scheduling a software interrupt or waking a kernel
thread).
The memory location where the thread identity is recorded is used
to signal the service thread(s) to terminate--when this value is set to
zero and the thread is signaled it will terminate.
If the queue is intended for use in fast interrupt handlers
.Fn taskqueue_create_fast
should be used in place of
.Fn taskqueue_create .
.Pp
The function
.Fn taskqueue_free
should be used to free the memory used by the queue.
Any tasks that are on the queue will be executed at this time after
which the thread servicing the queue will be signaled that it should exit.
.Pp
To add a task to the list of tasks queued on a taskqueue, call
.Fn taskqueue_enqueue
with pointers to the queue and task.
If the task's
.Va ta_pending
field is non-zero,
then it is simply incremented to reflect the number of times the task
was enqueued, up to a cap of USHRT_MAX.
Otherwise,
the task is added to the list before the first task which has a lower
.Va ta_priority
value or at the end of the list if no tasks have a lower priority.
Enqueueing a task does not perform any memory allocation which makes
it suitable for calling from an interrupt handler.
This function will return
.Er EPIPE
if the queue is being freed.
.Pp
The function
.Fn taskqueue_enqueue_fast
should be used in place of
.Fn taskqueue_enqueue
when the enqueuing must happen from a fast interrupt handler.
This method uses spin locks to avoid the possibility of sleeping in the fast
interrupt context.
.Pp
When a task is executed,
first it is removed from the queue,
the value of
.Va ta_pending
is recorded and then the field is zeroed.
The function
.Va ta_func
from the task structure is called with the value of the field
.Va ta_context
as its first argument
and the value of
.Va ta_pending
as its second argument.
After the function
.Va ta_func
returns,
.Xr wakeup 9
is called on the task pointer passed to
.Fn taskqueue_enqueue .
.Pp
The
.Fn taskqueue_enqueue_timeout
is used to schedule the enqueue after the specified amount of
.Va ticks .
Only non-fast task queues can be used for
.Va timeout_task
scheduling.
.Pp
The
.Fn taskqueue_cancel
function is used to cancel a task.
The
.Va ta_pending
count is cleared, and the old value returned in the reference
parameter
.Fa pendp ,
if it is non-
.Dv NULL .
If the task is currently running,
.Dv EBUSY
is returned, otherwise 0.
To implement a blocking
.Fn taskqueue_cancel
that waits for a running task to finish, it could look like:
.Bd -literal -offset indent
while (taskqueue_cancel(tq, task, NULL) != 0)
taskqueue_drain(tq, task);
.Ed
.Pp
Note that, as with
.Fn taskqueue_drain ,
the caller is responsible for ensuring that the task is not re-enqueued
after being canceled.
.Pp
Similarly, the
.Fn taskqueue_cancel_timeout
function is used to cancel the scheduled task execution.
.Pp
The
.Fn taskqueue_drain
function is used to wait for the task to finish, and
the
.Fn taskqueue_drain_timeout
function is used to wait for the scheduled task to finish.
There is no guarantee that the task will not be
enqueued after call to
.Fn taskqueue_drain .
.Pp
The
.Fn taskqueue_member
function returns
.No 1
if the given thread
.Fa td
is part of the given taskqueue
.Fa queue
and
.No 0
otherwise.
.Pp
The
.Fn taskqueue_run
function will run all pending tasks in the specified
.Fa queue .
Normally this function is only used internally.
.Pp
A convenience macro,
.Fn TASK_INIT "task" "priority" "func" "context"
is provided to initialise a
.Va task
structure.
A macro
.Fn TIMEOUT_TASK_INIT "queue" "timeout_task" "priority" "func" "context"
initializes the timeout_task structure.
The values of
.Va priority ,
.Va func ,
and
.Va context
are simply copied into the task structure fields and the
.Va ta_pending
field is cleared.
.Pp
Five macros
.Fn TASKQUEUE_DECLARE "name" ,
.Fn TASKQUEUE_DEFINE "name" "enqueue" "context" "init" ,
.Fn TASKQUEUE_FAST_DEFINE "name" "enqueue" "context" "init" ,
and
.Fn TASKQUEUE_DEFINE_THREAD "name"
.Fn TASKQUEUE_FAST_DEFINE_THREAD "name"
are used to declare a reference to a global queue, to define the
implementation of the queue, and declare a queue that uses its own thread.
The
.Fn TASKQUEUE_DEFINE
macro arranges to call
.Fn taskqueue_create
with the values of its
.Va name ,
.Va enqueue
and
.Va context
arguments during system initialisation.
After calling
.Fn taskqueue_create ,
the
.Va init
argument to the macro is executed as a C statement,
allowing any further initialisation to be performed
(such as registering an interrupt handler etc.)
.Pp
The
.Fn TASKQUEUE_DEFINE_THREAD
macro defines a new taskqueue with its own kernel thread to serve tasks.
The variable
.Vt struct taskqueue *taskqueue_name
is used to enqueue tasks onto the queue.
.Pp
.Fn TASKQUEUE_FAST_DEFINE
and
.Fn TASKQUEUE_FAST_DEFINE_THREAD
act just like
.Fn TASKQUEUE_DEFINE
and
.Fn TASKQUEUE_DEFINE_THREAD
respectively but taskqueue is created with
.Fn taskqueue_create_fast .
.Ss Predefined Task Queues
The system provides four global taskqueues,
.Va taskqueue_fast ,
.Va taskqueue_swi ,
.Va taskqueue_swi_giant ,
and
.Va taskqueue_thread .
The
.Va taskqueue_fast
queue is for swi handlers dispatched from fast interrupt handlers,
where sleep mutexes cannot be used.
The swi taskqueues are run via a software interrupt mechanism.
The
.Va taskqueue_swi
queue runs without the protection of the
.Va Giant
kernel lock, and the
.Va taskqueue_swi_giant
queue runs with the protection of the
.Va Giant
kernel lock.
The thread taskqueue
.Va taskqueue_thread
runs in a kernel thread context, and tasks run from this thread do
not run under the
.Va Giant
kernel lock.
If the caller wants to run under
.Va Giant ,
he should explicitly acquire and release
.Va Giant
in his taskqueue handler routine.
.Pp
To use these queues,
call
.Fn taskqueue_enqueue
with the value of the global taskqueue variable for the queue you wish to
use
.Va ( taskqueue_swi ,
.Va taskqueue_swi_giant ,
or
.Va taskqueue_thread ) .
Use
.Fn taskqueue_enqueue_fast
for the global taskqueue variable
.Va taskqueue_fast .
.Pp
The software interrupt queues can be used,
for instance, for implementing interrupt handlers which must perform a
significant amount of processing in the handler.
The hardware interrupt handler would perform minimal processing of the
interrupt and then enqueue a task to finish the work.
This reduces to a minimum
the amount of time spent with interrupts disabled.
.Pp
The thread queue can be used, for instance, by interrupt level routines
that need to call kernel functions that do things that can only be done
from a thread context.
(e.g., call malloc with the M_WAITOK flag.)
.Pp
Note that tasks queued on shared taskqueues such as
.Va taskqueue_swi
may be delayed an indeterminate amount of time before execution.
If queueing delays cannot be tolerated then a private taskqueue should
be created with a dedicated processing thread.
.Sh SEE ALSO
.Xr ithread 9 ,
.Xr kthread 9 ,
.Xr swi 9
.Sh HISTORY
This interface first appeared in
.Fx 5.0 .
There is a similar facility called work_queue in the Linux kernel.
.Sh AUTHORS
This manual page was written by
.An Doug Rabson .
|