From: Thomas Gleixner This is a new, much-cleaned up version of the ktimers subsystem. We reworked the patch thoroughly and we hope to have addressed all points raised on lkml. Special thanks go to Andrew Morton and Arjan van de Ven for detailed code-review. The text below is from Documentation/ktimers.txt, which will hopefully clarify most of the remaining conceptual issues raised on lkml. Comments, reviews, reports welcome! ktimers - subsystem for high-precision kernel timers ---------------------------------------------------- This patch introduces a new subsystem for high-precision kernel timers. Why two timer subsystems? After a lot of back and forth trying to integrate high-precision and high-resolution features into the existing timer framework, and after testing various such high-resolution timer implementations in practice, we came to the conclusion that the timer wheel code is fundamentally not suitable for such an approach. We initially didnt believe this ('there must be a way to solve this'), and we spent a considerable effort trying to integrate things into the timer wheel, but we failed. There are several reasons why such integration is impossible: - the forced handling of low-resolution and high-resolution timers in the same way leads to a lot of compromises, macro magic and #ifdef mess. The timers.c code is very "tightly coded" around jiffies and 32-bitness assumptions, and has been honed and micro-optimized for a narrow use case for many years - and thus even small extensions to it frequently break the wheel concept, leading to even worse compromises. - the unpredictable [O(N)] overhead of cascading leads to delays which necessiate a more complex handling of high resolution timers, which decreases robustness. Such a design still led to rather large timing inaccuracies. Cascading is a fundamental property of the timer wheel concept, it cannot be 'designed out' without unevitabling degrading other portions of the timers.c code in an unacceptable way. - the implementation of the current posix-timer subsystem on top of the timer wheel has already introduced a quite complex handling of the required readjusting of absolute CLOCK_REALTIME timers at settimeofday or NTP time - showing the rigidity of the timer wheel data structure. - the timer wheel code is most optimal for use cases which can be identified as "timeouts". Such timeouts are usually set up to cover error conditions in various I/O paths, such as networking and block I/O. The vast majority of those timers never expire and are rarely recascaded because the expected correct event arrives in time so they can be removed from the timer wheel before any further processing of them becomes necessary. Thus the users of these timeouts can accept the granularity and precision tradeoffs of the timer wheel, and largely expect the timer subsystem to have near-zero overhead. Timing for them is not a core purpose, it's most a necessary evil to guarantee the processing of requests, which should be as cheap and unintrusive as possible. The primary users of precision timers are user-space applications that utilize nanosleep, posix-timers and itimer interfaces. Also, in-kernel users like drivers and subsystems with a requirement for precise timed events can benefit from the availability of a seperate high-precision timer subsystem as well. The ktimer subsystem is easily extended with high-resolution capabilities, and patches for that exist and are maturing quickly. The increasing demand for realtime and multimedia applications along with other potential users for precise timers gives another reason to separate the "timeout" and "precise timer" subsystems. Another potential benefit is that such seperation allows for future optimizations of the existing timer wheel implementation for the low resolution and low precision use cases - once the precision-sensitive APIs are separated from the timer wheel and are migrated over to ktimers. E.g. we could decrease the frequency of the timeout subsystem from 250 Hz to 100 HZ (or even smaller). ktimer subsystem implementation details --------------------------------------- the basic design considerations were: - simplicity - robust, extensible abstractions - data structure not bound to jiffies or any other granularity - simplification of existing, timing related kernel code >From our previous experience with various approaches of high-resolution timers another basic requirement was the immediate enqueueing and ordering of timers at activation time. After looking at several possible solutions such as radix trees and hashes, the red black tree was choosen as the basic data structure. Rbtrees are available as a library in the kernel and are used in various performance-critical areas of e.g. memory management and file systems. The rbtree is solely used for the time sorted ordering, while a seperate list is used to give the expiry code fast access to the queued timers, without having to walk the rbtree. (This seperate list is also useful for high-resolution timers where we need seperate pending and expired queues while keeping the time-order intact.) The time-ordered enqueueing is not purely for the purposes of the high-resolution timers extension though, it also simplifies the handling of absolute timers based on CLOCK_REALTIME. The existing implementation needed to keep an extra list of all armed absolute CLOCK_REALTIME timers along with complex locking. In case of settimeofday and NTP, all the timers (!) had to be dequeued, the time-changing code had to fix them up one by one, and all of them had to be enqueued again. The time-ordered enqueueing and the storage of the expiry time in absolute time units removes all this complex and poorly scaling code from the posix-timer implementation - the clock can simply be set without having to touch the rbtree. This also makes the handling of posix-timers simpler in general. The locking and per-CPU behavior of ktimers was mostly taken from the existing timer wheel code, as it is mature and well suited. Sharing code was not really a win, due to the different data structures. Also, the ktimer functions now have clearer behavior and clearer names - such as ktimer_try_to_cancel() and ktimer_cancel() [which are roughly equivalent to del_timer() and del_timer_sync()] - and there's no direct 1:1 mapping between them on the algorithmical level. The internal representation of time values (ktime_t) is implemented via macros and inline functions, and can be switched between a "hybrid union" type and a plain "scalar" 64bit nanoseconds representation (at compile time). The hybrid union type exists to optimize time conversions on 32bit CPUs. This build-time-selectable ktime_t storage format was implemented to avoid the performance impact of 64-bit multiplications and divisions on 32bit CPUs. Such operations are frequently necessary to convert between the storage formats provided by kernel and userspace interfaces and the internal time format. (See include/linux/ktime.h for further details.) We used the high-resolution timer subsystem ontop of ktimers to verify the ktimer implementation details in praxis, and we also ran the posix timer tests in order to ensure specification compliance. The ktimer patch converts the following kernel functionality to use ktimers: - nanosleep - itimers - posix-timers The conversion of nanosleep and posix-timers enabled the unification of nanosleep and clock_nanosleep. The code was successfully compiled for the following platforms: i386, x86_64, ARM, PPC, PPC64, IA64 The code was run-tested on the following platforms: i386(UP/SMP), x86_64(UP/SMP), ARM, PPC ktimers were also integrated into the -rt tree, along with a ktimers-based high-resolution timer implementation, so the ktimers code got a healthy amount of testing and use in practice. Thomas Gleixner, Ingo Molnar Signed-off-by: Thomas Gleixner Signed-off-by: Ingo Molnar Signed-off-by: Miklos Szeredi Signed-off-by: Andrew Morton --- Documentation/DocBook/kernel-api.tmpl | 5 Documentation/ktimers.txt | 239 +++++++ arch/um/kernel/time.c | 4 fs/exec.c | 7 fs/proc/array.c | 6 include/linux/calc64.h | 49 + include/linux/jiffies.h | 11 include/linux/ktime.h | 367 +++++++++++ include/linux/ktimer.h | 177 +++++ include/linux/posix-timers.h | 122 ++- include/linux/sched.h | 4 include/linux/time.h | 96 +-- include/linux/timer.h | 2 init/main.c | 1 kernel/Makefile | 3 kernel/exit.c | 2 kernel/fork.c | 5 kernel/itimer.c | 108 +-- kernel/ktimers.c | 1052 ++++++++++++++++++++++++++++++++++ kernel/posix-cpu-timers.c | 23 kernel/posix-timers.c | 820 +++++--------------------- kernel/time.c | 97 +++ kernel/timer.c | 57 - 23 files changed, 2365 insertions(+), 892 deletions(-) diff -puN Documentation/DocBook/kernel-api.tmpl~ktimers-kt2 Documentation/DocBook/kernel-api.tmpl --- devel/Documentation/DocBook/kernel-api.tmpl~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/Documentation/DocBook/kernel-api.tmpl 2005-11-12 14:08:45.000000000 -0800 @@ -54,6 +54,11 @@ !Ekernel/sched.c !Ekernel/timer.c + High-precision timers +!Iinclude/linux/ktime.h +!Iinclude/linux/ktimer.h +!Ekernel/ktimers.c + Internal Functions !Ikernel/exit.c !Ikernel/signal.c diff -puN /dev/null Documentation/ktimers.txt --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/Documentation/ktimers.txt 2005-11-12 14:08:45.000000000 -0800 @@ -0,0 +1,239 @@ + +ktimers - subsystem for high-precision kernel timers +---------------------------------------------------- + +This patch introduces a new subsystem for high-precision kernel timers. + +Why two timer subsystems? After a lot of back and forth trying to +integrate high-precision and high-resolution features into the existing +timer framework, and after testing various such high-resolution timer +implementations in practice, we came to the conclusion that the timer +wheel code is fundamentally not suitable for such an approach. We +initially didnt believe this ('there must be a way to solve this'), and +we spent a considerable effort trying to integrate things into the timer +wheel, but we failed. There are several reasons why such integration is +impossible: + +- the forced handling of low-resolution and high-resolution timers in + the same way leads to a lot of compromises, macro magic and #ifdef + mess. The timers.c code is very "tightly coded" around jiffies and + 32-bitness assumptions, and has been honed and micro-optimized for a + narrow use case for many years - and thus even small extensions to it + frequently break the wheel concept, leading to even worse + compromises. + +- the unpredictable [O(N)] overhead of cascading leads to delays which + necessiate a more complex handling of high resolution timers, which + decreases robustness. Such a design still led to rather large timing + inaccuracies. Cascading is a fundamental property of the timer wheel + concept, it cannot be 'designed out' without unevitabling degrading + other portions of the timers.c code in an unacceptable way. + +- the implementation of the current posix-timer subsystem on top of + the timer wheel has already introduced a quite complex handling of + the required readjusting of absolute CLOCK_REALTIME timers at + settimeofday or NTP time - showing the rigidity of the timer wheel + data structure. + +- the timer wheel code is most optimal for use cases which can be + identified as "timeouts". Such timeouts are usually set up to cover + error conditions in various I/O paths, such as networking and block + I/O. The vast majority of those timers never expire and are rarely + recascaded because the expected correct event arrives in time so they + can be removed from the timer wheel before any further processing of + them becomes necessary. Thus the users of these timeouts can accept + the granularity and precision tradeoffs of the timer wheel, and + largely expect the timer subsystem to have near-zero overhead. Timing + for them is not a core purpose, it's most a necessary evil to + guarantee the processing of requests, which should be as cheap and + unintrusive as possible. + +The primary users of precision timers are user-space applications that +utilize nanosleep, posix-timers and itimer interfaces. Also, in-kernel +users like drivers and subsystems with a requirement for precise timed +events can benefit from the availability of a seperate high-precision +timer subsystem as well. + +The ktimer subsystem is easily extended with high-resolution +capabilities, and patches for that exist and are maturing quickly. The +increasing demand for realtime and multimedia applications along with +other potential users for precise timers gives another reason to +separate the "timeout" and "precise timer" subsystems. + +Another potential benefit is that such seperation allows for future +optimizations of the existing timer wheel implementation for the low +resolution and low precision use cases - once the precision-sensitive +APIs are separated from the timer wheel and are migrated over to +ktimers. E.g. we could decrease the frequency of the timeout subsystem +from 250 Hz to 100 HZ (or even smaller). + +ktimer subsystem implementation details +--------------------------------------- + +the basic design considerations were: + +- simplicity +- robust, extensible abstractions +- data structure not bound to jiffies or any other granularity +- simplification of existing, timing related kernel code + +From our previous experience with various approaches of high-resolution +timers another basic requirement was the immediate enqueueing and +ordering of timers at activation time. After looking at several possible +solutions such as radix trees and hashes, the red black tree was choosen +as the basic data structure. Rbtrees are available as a library in the +kernel and are used in various performance-critical areas of e.g. memory +management and file systems. The rbtree is solely used for the time +sorted ordering, while a seperate list is used to give the expiry code +fast access to the queued timers, without having to walk the rbtree. +(This seperate list is also useful for high-resolution timers where we +need seperate pending and expired queues while keeping the time-order +intact.) + +The time-ordered enqueueing is not purely for the purposes of the +high-resolution timers extension though, it also simplifies the handling +of absolute timers based on CLOCK_REALTIME. The existing implementation +needed to keep an extra list of all armed absolute CLOCK_REALTIME timers +along with complex locking. In case of settimeofday and NTP, all the +timers (!) had to be dequeued, the time-changing code had to fix them up +one by one, and all of them had to be enqueued again. The time-ordered +enqueueing and the storage of the expiry time in absolute time units +removes all this complex and poorly scaling code from the posix-timer +implementation - the clock can simply be set without having to touch the +rbtree. This also makes the handling of posix-timers simpler in general. + +The locking and per-CPU behavior of ktimers was mostly taken from the +existing timer wheel code, as it is mature and well suited. Sharing code +was not really a win, due to the different data structures. Also, the +ktimer functions now have clearer behavior and clearer names - such as +ktimer_try_to_cancel() and ktimer_cancel() [which are roughly equivalent +to del_timer() and del_timer_sync()] - and there's no direct 1:1 mapping +between them on the algorithmical level. + +The internal representation of time values (ktime_t) is implemented via +macros and inline functions, and can be switched between a "hybrid +union" type and a plain "scalar" 64bit nanoseconds representation (at +compile time). The hybrid union type exists to optimize time conversions +on 32bit CPUs. This build-time-selectable ktime_t storage format was +implemented to avoid the performance impact of 64-bit multiplications +and divisions on 32bit CPUs. Such operations are frequently necessary to +convert between the storage formats provided by kernel and userspace +interfaces and the internal time format. (See include/linux/ktime.h for +further details.) + +ktimers - rounding of timer values +---------------------------------- + +Why do we need rounding at all ? + +Firstly, the POSIX specification requires rounding to the resolution - +whatever that means. The POSIX specification is quite imprecise on the +details of rounding though, so a practical interpretation had to be +found. + +The first question is which resolution value should be returned to the +user by the clock_getres() interface. + +The simplest case is when the hardware is capable of 1 nsec resolution: +in that case we can fulfill all wishes and there is no rounding :-) + +Another simple case is when the clock hardware has a limited resolution +that the kernel wants to fully offer to user-space: in this case that +limited resolution is returned to userspace. + +The hairy case is when the underlying hardware is capable of finer +grained resolution, but the kernel is not willing to offer that +resolution. Why would the kernel want to do that? Because e.g. the +system could easily be DoS-ed with high-frequency timer interrupts. Or +the kernel might want to cluster high-res timer interrupts into groups +for performance reasons, so that extremely high interrupt rates are +avoided. So the kernel needs some leeway in deciding the 'effective' +resolution that it is willing to expose to userspace. + +In this case, the clock_getres() decision is easy: we want to return the +'effective' resolution, not the 'theoretical' resolution. Thus an +application programmer gets correct information about what granularity +and accuracy to expect from the system. + +What is much less obvious in both the 'hardware is low-res' and 'kernel +wants to offer low-res' cases is the actual behavior of timers, and +where and how to round time values to the 'effective' resolution of the +clock. + +For this we first need to see what types of expiries there exist for +ktimers, and how rounding affects them. Ktimers have the following +variants: + +- relative one-shot timers +- absolute one-shot timers +- relative interval timers +- absolute interval timers + +Interval timers can be led back to one-shot timers: they are a series of +one-shot timers with the same interval. Relative one-shot timers can be +handled identically to absolute one-shot timers after adding the +relative expiry time to the current time of the respective clock. + +We picked to handle two cases of rounding: + +- the rounding of the absolute value of the first expiry time +- the rounding of the timer interval + +An alternative implementation would be to not round the interval and to +implicitly round at every timer event, but it's not clear what the +advantages would be from doing that. There are a couple of +disadvantages: + +- the technique seems to contradict the standard's requirement that + 'time values ... be rounded' (which the interval clearly is). + +- other OSs implement the rounding in the way we implemented it. + +- also, there is an application surprise factor, the 'do not round + intervals' technique can lead to the following sample sequence of + events: + + Interval: 1.7ms + Resolution: 1ms + + Event timeline: + + 2ms - 4ms - 6ms - 7ms - 9ms - 11ms - 12ms - 14ms - 16ms - 17ms ... + + this 2,2,1,2,2,1...msec 'unpredictable and uneven' relative distance + of events could surprise applications. + +(as a sidenote, current POSIX APIs could be extended with a method of +periodic timers to have an 'average' frequency, where there is no +rounding of the interval. No such API exists at the moment.) + +ktimers - testing and verification +---------------------------------- + +We used the high-resolution timer subsystem ontop of ktimers to verify +the ktimer implementation details in praxis, and we also ran the posix +timer tests in order to ensure specification compliance. + +The ktimer patch converts the following kernel functionality to use +ktimers: + + - nanosleep + - itimers + - posix-timers + +The conversion of nanosleep and posix-timers enabled the unification of +nanosleep and clock_nanosleep. + +The code was successfully compiled for the following platforms: + + i386, x86_64, ARM, PPC, PPC64, IA64 + +The code was run-tested on the following platforms: + + i386(UP/SMP), x86_64(UP/SMP), ARM, PPC + +ktimers were also integrated into the -rt tree, along with a +ktimers-based high-resolution timer implementation, so the ktimers code +got a healthy amount of testing and use in practice. + + Thomas Gleixner, Ingo Molnar diff -puN fs/exec.c~ktimers-kt2 fs/exec.c --- devel/fs/exec.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/fs/exec.c 2005-11-12 14:08:45.000000000 -0800 @@ -642,10 +642,11 @@ static inline int de_thread(struct task_ * synchronize with any firing (by calling del_timer_sync) * before we can safely let the old group leader die. */ - sig->real_timer.data = (unsigned long)current; + sig->real_timer.data = current; spin_unlock_irq(lock); - if (del_timer_sync(&sig->real_timer)) - add_timer(&sig->real_timer); + if (ktimer_cancel(&sig->real_timer)) + ktimer_start(&sig->real_timer, NULL, + KTIMER_RESTART|KTIMER_NOCHECK); spin_lock_irq(lock); } while (atomic_read(&sig->count) > count) { diff -puN fs/proc/array.c~ktimers-kt2 fs/proc/array.c --- devel/fs/proc/array.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/fs/proc/array.c 2005-11-12 14:08:45.000000000 -0800 @@ -330,7 +330,7 @@ static int do_task_stat(struct task_stru unsigned long min_flt = 0, maj_flt = 0; cputime_t cutime, cstime, utime, stime; unsigned long rsslim = 0; - unsigned long it_real_value = 0; + DEFINE_KTIME(it_real_value); struct task_struct *t; char tcomm[sizeof(task->comm)]; @@ -386,7 +386,7 @@ static int do_task_stat(struct task_stru utime = cputime_add(utime, task->signal->utime); stime = cputime_add(stime, task->signal->stime); } - it_real_value = task->signal->it_real_value; + it_real_value = task->signal->real_timer.expires; } ppid = pid_alive(task) ? task->group_leader->real_parent->tgid : 0; read_unlock(&tasklist_lock); @@ -435,7 +435,7 @@ static int do_task_stat(struct task_stru priority, nice, num_threads, - jiffies_to_clock_t(it_real_value), + (long) ktime_to_clock_t(it_real_value), start_time, vsize, mm ? get_mm_rss(mm) : 0, diff -puN /dev/null include/linux/calc64.h --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/include/linux/calc64.h 2005-11-12 14:08:45.000000000 -0800 @@ -0,0 +1,49 @@ +#ifndef _linux_CALC64_H +#define _linux_CALC64_H + +#include +#include + +/* + * div_long_long_rem was moved out of jiffies.h as it is + * a general math function useful for other things than + * jiffy code. + * + * This is a generic macro which is used when the architecture + * specific div64.h does not provide a optimized one. + * + * The 64bit dividend is divided by the divisor (data type long), the + * result is returned and the remainder stored in the variable + * referenced by remainder (data type long *). In contrast to the + * do_div macro the dividend is kept intact. + */ +#ifndef div_long_long_rem +#define div_long_long_rem(dividend,divisor,remainder) \ +({ \ + u64 result = dividend; \ + *remainder = do_div(result,divisor); \ + result; \ +}) +#endif + +/* + * Sign aware variation of the above. On some architectures a + * negative dividend leads to an divide overflow exception, which + * is avoided by the sign check. + */ +static inline long div_long_long_rem_signed(long long dividend, + long divisor, + long *remainder) +{ + long res; + + if (unlikely(dividend < 0)) { + res = -div_long_long_rem(-dividend, divisor, remainder); + *remainder = -(*remainder); + } else { + res = div_long_long_rem(dividend, divisor, remainder); + } + return res; +} + +#endif diff -puN include/linux/jiffies.h~ktimers-kt2 include/linux/jiffies.h --- devel/include/linux/jiffies.h~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/include/linux/jiffies.h 2005-11-12 14:08:45.000000000 -0800 @@ -1,21 +1,12 @@ #ifndef _LINUX_JIFFIES_H #define _LINUX_JIFFIES_H +#include #include #include #include #include #include /* for HZ */ -#include - -#ifndef div_long_long_rem -#define div_long_long_rem(dividend,divisor,remainder) \ -({ \ - u64 result = dividend; \ - *remainder = do_div(result,divisor); \ - result; \ -}) -#endif /* * The following defines establish the engineering parameters of the PLL diff -puN /dev/null include/linux/ktime.h --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/include/linux/ktime.h 2005-11-12 14:08:45.000000000 -0800 @@ -0,0 +1,367 @@ +/* + * include/linux/ktime.h + * + * ktime_t - nanosecond-resolution time format. + * + * Copyright(C) 2005, Thomas Gleixner + * Copyright(C) 2005, Red Hat, Inc., Ingo Molnar + * + * data type definitions, declarations, prototypes and macros. + * + * Started by: Thomas Gleixner and Ingo Molnar + * + * For licencing details see kernel-base/COPYING + */ +#ifndef _LINUX_KTIME_H +#define _LINUX_KTIME_H + +#include +#include + +/* + * ktime_t: + * + * On 64-bit CPUs a single 64-bit variable is used to store the ktimers + * internal representation of time values in scalar nanoseconds. The + * design plays out best on 64-bit CPUs, where most conversions are + * NOPs and most arithmetic ktime_t operations are plain arithmetic + * operations. + * + * On 32-bit CPUs an optimized representation of the timespec structure + * is used to avoid expensive conversions from and to timespecs. The + * endian-aware order of the tv struct members is choosen to allow + * mathematical operations on the tv64 member of the union too, which + * for certain operations produces better code. + * + * For architectures with efficient support for 64/32-bit conversions the + * plain scalar nanosecond based representation can be selected by the + * config switch CONFIG_KTIME_SCALAR. + */ + +#define KTIME_ZERO 0 +#define KTIME_MAX (~((u64)1 << 63)) + +/* + * ktime_t definitions when using the 64-bit scalar representation: + */ + +#if (BITS_PER_LONG == 64) || defined(CONFIG_KTIME_SCALAR) + +typedef s64 ktime_t; + +/* Define a ktime_t variable and initialize it to zero: */ +#define DEFINE_KTIME(kt) ktime_t kt = 0 + +/* + * Compare two ktime_t variables. The comparison operand is + * given as a literal in the macro call (e.g. <, >, ==): + * + * ( E.g. "ktime_cmp(t1, <, t2) is still more readable to programmers + * than ktime_before()/ktime_after() would be. ) + */ +#define ktime_cmp(a, op, b) ((a) op (b)) + +/* + * Compare a ktime_t variable and a constant. The comparison operand is + * given as a literal in the macro call (e.g. <, >, ==): + */ +#define ktime_cmp_val(a, op, b) ((a) op (b)) + +/** + * ktime_set - Set a ktime_t variable from a seconds/nanoseconds value + * + * @secs: seconds to set + * @nsecs: nanoseconds to set + * + * Return the ktime_t representation of the value + */ +#define ktime_set(sec, nsec) (((s64)(sec) * NSEC_PER_SEC) + (s64)(nsec)) + +/* + * Set a ktime_t variable to a value in a scalar nanosecond representation + * + * NOTE: use only with KTIME_ZERO or KTIME_MAX to maintain compability + * with the union type version. + */ +#define ktime_set_scalar(kt, s) (kt) = (s) + +/* + * The following 3 macros are used for the nanosleep restart handling + * to store the "low" and "high" part of a 64-bit ktime variable. + * (on 32-bit CPUs the restart block has 32-bit fields, so we have to + * split the 64-bit value up into two pieces) + * + * In the scalar representation we have to split up the 64-bit scalar: + */ + +/* Set the "low" and "high" part of a ktime_t variable: */ +#define ktime_set_low_high(l, h) ((s64)((u64)(l)) | (((s64)(h)) << 32)) + +/* Get the "low" part of a ktime_t variable: */ +#define ktime_get_low(kt) ((kt) & 0xFFFFFFFF) + +/* Get the "high" part of a ktime_t variable: */ +#define ktime_get_high(kt) ((kt) >> 32) + +/* Subtract two ktime_t variables. rem = lhs -rhs: */ +#define ktime_sub(lhs, rhs) ((lhs) - (rhs)) + +/* Add two ktime_t variables. res = lhs + rhs: */ +#define ktime_add(lhs, rhs) ((lhs) + (rhs)) + +/* + * Add a ktime_t variable and a scalar nanosecond value. + * res = kt + nsval: + */ +#define ktime_add_ns(kt, nsval) ((kt) + (nsval)) + +/* convert a timespec to ktime_t format: */ +#define timespec_to_ktime(ts) ktime_set((ts).tv_sec, (ts).tv_nsec) + +/* convert a timeval to ktime_t format: */ +#define timeval_to_ktime(tv) ktime_set((tv).tv_sec, (tv).tv_usec * 1000) + +/* Map the ktime_t to timespec conversion to ns_to_timespec function */ +#define ktime_to_timespec(ts, kt) ns_to_timespec(ts, kt) + +/* Map the ktime_t to timeval conversion to ns_to_timeval function */ +#define ktime_to_timeval(tv, kt) ns_to_timeval(tv, kt) + +/* Map the ktime_t to clock_t conversion to the inline in jiffies.h: */ +#define ktime_to_clock_t(kt) nsec_to_clock_t(kt) + +/* Convert ktime_t to nanoseconds - NOP in the scalar storage format: */ +#define ktime_to_ns(kt) (kt) + +#else + +/* + * Helper macros/inlines to get the ktime_t math right in the timespec + * representation. The macros are sometimes ugly - their actual use is + * pretty okay-ish, given the circumstances. We do all this for + * performance reasons. The pure scalar nsec_t based code was nice and + * simple, but created too many 64-bit / 32-bit conversions and divisions. + * + * Be especially aware that negative values are represented in a way + * that the tv.sec field is negative and the tv.nsec field is greater + * or equal to zero but less than nanoseconds per second. This is the + * same representation which is used by timespecs. + * + * tv.sec < 0 and 0 >= tv.nsec < NSEC_PER_SEC + */ + +typedef union { + s64 tv64; + struct { +#ifdef __BIG_ENDIAN + s32 sec, nsec; +#else + s32 nsec, sec; +#endif + } tv; +} ktime_t; + +/* Define a ktime_t variable and initialize it to zero: */ +#define DEFINE_KTIME(kt) ktime_t kt = { .tv64 = 0 } + +/* + * Compare two ktime_t variables. The comparison operand is + * given as a literal in the macro call (e.g. <, >, ==): + */ +#define ktime_cmp(a, op, b) ((a).tv64 op (b).tv64) + +/* + * Compare a ktime_t variable and a constant. The comparison operand is + * given as a literal in the macro call (e.g. <, >, ==): + */ +#define ktime_cmp_val(a, op, b) ((a).tv64 op (b)) + +/* Set a ktime_t variable to a value in sec/nsec representation: */ +static inline ktime_t ktime_set(long secs, unsigned long nsecs) +{ + return (ktime_t) { .tv = { .sec = secs, .nsec = nsecs } }; +} + +/* + * Set the scalar value of a ktime variable (union type) + * NOTE: use only with KTIME_ZERO or KTIME_MAX! + */ +#define ktime_set_scalar(kt, s) (kt).tv64 = (s) + +/* + * The following 3 macros are used for the nanosleep restart handling + * to store the "low" and "high" part of a 64-bit ktime variable. + * (on 32-bit CPUs the restart block has 32-bit fields, so we have to + * split the 64-bit value up into two pieces) + * + * In the union type representation this is just storing and restoring + * the sec and nsec members of the tv structure: + */ + +/* Set the "low" and "high" part of a ktime_t variable: */ +#define ktime_set_low_high(l, h) ktime_set(h, l) + +/* Get the "low" part of a ktime_t variable: */ +#define ktime_get_low(kt) (kt).tv.nsec + +/* Get the "high" part of a ktime_t variable: */ +#define ktime_get_high(kt) (kt).tv.sec + +/** + * ktime_sub - subtract two ktime_t variables + * + * @lhs: minuend + * @rhs: subtrahend + * + * Returns the remainder of the substraction + */ +static inline ktime_t ktime_sub(ktime_t lhs, ktime_t rhs) +{ + ktime_t res; + + res.tv64 = lhs.tv64 - rhs.tv64; + if (res.tv.nsec < 0) + res.tv.nsec += NSEC_PER_SEC; + + return res; +} + +/** + * ktime_add - add two ktime_t variables + * + * @add1: addend1 + * @add2: addend2 + * + * Returns the sum of addend1 and addend2 + */ +static inline ktime_t ktime_add(ktime_t add1, ktime_t add2) +{ + ktime_t res; + + res.tv64 = add1.tv64 + add2.tv64; + /* + * performance trick: the (u32) -NSEC gives 0x00000000Fxxxxxxx + * so we subtract NSEC_PER_SEC and add 1 to the upper 32 bit. + * + * it's equivalent to: + * tv.nsec -= NSEC_PER_SEC + * tv.sec ++; + */ + if (res.tv.nsec >= NSEC_PER_SEC) + res.tv64 += (u32)-NSEC_PER_SEC; + + return res; +} + +/** + * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable + * + * @kt: addend + * @nsec: the scalar nsec value to add + * + * Returns the sum of kt and nsec in ktime_t format + */ +extern ktime_t ktime_add_ns(ktime_t kt, u64 nsec); + +/** + * timespec_to_ktime - convert a timespec to ktime_t format + * + * @ts: the timespec variable to convert + * + * Returns a ktime_t variable with the converted timespec value + */ +static inline ktime_t timespec_to_ktime(struct timespec ts) +{ + return (ktime_t) { .tv = { .sec = (s32)ts.tv_sec, + .nsec = (s32)ts.tv_nsec } }; +} + +/** + * timeval_to_ktime - convert a timeval to ktime_t format + * + * @tv: the timeval variable to convert + * + * Returns a ktime_t variable with the converted timeval value + */ +static inline ktime_t timeval_to_ktime(struct timeval tv) +{ + return (ktime_t) { .tv = { .sec = (s32)tv.tv_sec, + .nsec = (s32)tv.tv_usec * 1000 } }; +} + +/** + * ktime_to_timespec - convert a ktime_t variable to timespec format + * + * @ts: pointer to timespec variable to store result + * @kt: the ktime_t variable to convert + * + * Stores the timespec representation of the ktime value in + * the timespec variable pointed to by @ts + */ +static inline void ktime_to_timespec(struct timespec *ts, ktime_t kt) +{ + ts->tv_sec = (time_t) kt.tv.sec; + ts->tv_nsec = (long) kt.tv.nsec; +} + +/** + * ktime_to_timeval - convert a ktime_t variable to timeval format + * + * @tv: pointer to timeval variable to store result + * @kt: the ktime_t variable to convert + * + * Stores the timeval representation of the ktime value in + * the timeval variable pointed to by @tv + */ +static inline void ktime_to_timeval(struct timeval *tv, ktime_t kt) +{ + tv->tv_sec = (time_t) kt.tv.sec; + tv->tv_usec = (suseconds_t) (kt.tv.nsec / NSEC_PER_USEC); +} + +/** + * ktime_to_clock_t - convert a ktime_t variable to clock_t format + * @kt: the ktime_t variable to convert + * + * Returns a clock_t variable with the converted value + */ +static inline clock_t ktime_to_clock_t(ktime_t kt) +{ + return nsec_to_clock_t( (u64) kt.tv.sec * NSEC_PER_SEC + kt.tv.nsec); +} + +/** + * ktime_to_ns - convert a ktime_t variable to scalar nanoseconds + * @kt: the ktime_t variable to convert + * + * Returns the scalar nanoseconds representation of kt + */ +static inline u64 ktime_to_ns(ktime_t kt) +{ + return (u64) kt.tv.sec * NSEC_PER_SEC + kt.tv.nsec; +} + +#endif + +/* + * The resolution of the clocks. The resolution value is returned in + * the clock_getres() system call to give application programmers an + * idea of the (in)accuracy of timers. Timer values are rounded up to + * this resolution values. + */ +#define KTIME_REALTIME_RES (NSEC_PER_SEC/HZ) +#define KTIME_MONOTONIC_RES (NSEC_PER_SEC/HZ) + +/* Get the monotonic time in ktime_t format: */ +extern ktime_t ktime_get(void); + +/* Get the real (wall-) time in ktime_t format: */ +extern ktime_t ktime_get_real(void); + +/* Get the monotonic time in timespec format: */ +extern void ktime_get_ts(struct timespec *ts); + +/* Get the real (wall-) time in timespec format: */ +#define ktime_get_real_ts(ts) getnstimeofday(ts) + +#endif diff -puN /dev/null include/linux/ktimer.h --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/include/linux/ktimer.h 2005-11-12 14:08:45.000000000 -0800 @@ -0,0 +1,177 @@ +/* + * include/linux/ktimer.h + * + * ktimers - high-precision kernel timers + * + * Copyright(C) 2005, Thomas Gleixner + * Copyright(C) 2005, Red Hat, Inc., Ingo Molnar + * + * data type definitions, declarations, prototypes + * + * Started by: Thomas Gleixner and Ingo Molnar + * + * For licencing details see kernel-base/COPYING + */ +#ifndef _LINUX_KTIMER_H +#define _LINUX_KTIMER_H + +#include +#include +#include +#include +#include + +/* + * Mode arguments of xxx_ktimer functions: + */ +enum ktimer_rearm { + KTIMER_ABS = 1, /* Time value is absolute */ + KTIMER_REL, /* Time value is relative to now */ + KTIMER_INCR, /* Time value is relative to previous expiry time */ + KTIMER_FORWARD, /* Timer is rearmed with value. Overruns accounted */ + KTIMER_REARM, /* Timer is rearmed with interval. Overruns accounted */ + KTIMER_RESTART, /* Timer is restarted with the stored expiry value */ + + /* + * Expiry must not be checked when the timer is started: + * (can be OR-ed with another above mode flag) + */ + KTIMER_NOCHECK = 0x10000, + /* + * Rounding is required when the time is set up. Thats an + * optimization for relative timers as we read current time + * in the enqueing code so we do not need to read is twice. + */ + KTIMER_ROUND = 0x20000, + + /* (used internally: no rearming) */ + KTIMER_NOREARM = 0 +}; + +/* + * Timer states: + */ +enum ktimer_state { + KTIMER_INACTIVE, /* Timer is inactive */ + KTIMER_PENDING, /* Timer is pending */ +}; + +struct ktimer_base; + +/** + * struct ktimer - the basic ktimer structure + * + * @node: red black tree node for time ordered insertion + * @list: list head for easier access to the time ordered list, + * without walking the red black tree. + * @expires: the absolute expiry time in the ktimers internal + * representation. The time is related to the clock on + * which the timer is based. + * @expired: the absolute time when the timer expired. Used for + * simplifying return path calculations and for debugging + * purposes. + * @interval: the timer interval for automatic rearming + * @overrun: the number of intervals missed when rearming a timer + * @state: state of the timer + * @function: timer expiry callback function + * @data: argument for the callback function + * @base: pointer to the timer base (per cpu and per clock) + * + * The ktimer structure must be initialized by init_ktimer_#CLOCKTYPE() + */ +struct ktimer { + struct rb_node node; + struct list_head list; + ktime_t expires; + ktime_t expired; + ktime_t interval; + int overrun; + enum ktimer_state state; + void (*function)(void *); + void *data; + struct ktimer_base *base; +}; + +/** + * struct ktimer_base - the timer base for a specific clock + * + * @index: clock type index for per_cpu support when moving a timer + * to a base on another cpu. + * @lock: lock protecting the base and associated timers + * @active: red black tree root node for the active timers + * @pending: list of pending timers for simple time ordered access + * @count: the number of active timers + * @resolution: the resolution of the clock, in nanoseconds + * @get_time: function to retrieve the current time of the clock + * @curr_timer: the timer which is executing a callback right now + * @wait: waitqueue to wait for a currently running timer + * @name: string identifier of the clock + */ +struct ktimer_base { + int index; + spinlock_t lock; + struct rb_root active; + struct list_head pending; + int count; + unsigned long resolution; + ktime_t (*get_time)(void); + struct ktimer *curr_timer; + wait_queue_head_t wait; + char *name; +}; + +#define KTIMER_POISON ((void *) 0x00100101) + +/* + * clock_was_set() is a NOP for non- high-resolution systems. The + * time-sorted order guarantees that a timer does not expire early and + * is expired in the next softirq when the clock was advanced. + */ +#define clock_was_set() do { } while (0) + +/* Exported timer functions: */ + +/* Initialize timers: */ +extern void ktimer_init(struct ktimer *timer); +extern void ktimer_init_real(struct ktimer *timer); + +/* Basic timer operations: */ +extern int ktimer_start(struct ktimer *timer, ktime_t *tim, int mode); +extern int ktimer_restart(struct ktimer *timer, ktime_t *tim, int mode); +extern int ktimer_cancel(struct ktimer *timer); +extern int ktimer_try_to_cancel(struct ktimer *timer); + +/* Query timers: */ +extern ktime_t ktimer_get_remtime(struct ktimer *timer); +extern ktime_t ktimer_get_expiry(struct ktimer *timer, ktime_t *now); +extern int ktimer_get_res(clockid_t which_clock, struct timespec *tp); +extern int ktimer_get_res_real(clockid_t which_clock, struct timespec *tp); + +static inline int ktimer_active(struct ktimer *timer) +{ + return timer->state != KTIMER_INACTIVE; +} + +/* Convert with rounding based on resolution of timer's clock: */ +extern ktime_t ktimer_round_timeval(struct ktimer *timer, struct timeval *tv); +extern ktime_t ktimer_round_timespec(struct ktimer *timer, struct timespec *ts); + +/* Precise sleep: */ +extern long ktimer_nanosleep(struct timespec *rqtp, + struct timespec __user *rmtp, int mode); +extern long ktimer_nanosleep_real(struct timespec *rqtp, + struct timespec __user *rmtp, int mode); + +#ifdef CONFIG_SMP +extern void wait_for_ktimer(struct ktimer *timer); +#else +# define wait_for_ktimer(t) do { } while (0) +#endif + +/* Soft interrupt function to run the ktimer queues: */ +extern void ktimer_run_queues(void); + +/* Bootup initialization: */ +extern void __init ktimers_init(void); + +#endif diff -puN include/linux/posix-timers.h~ktimers-kt2 include/linux/posix-timers.h --- devel/include/linux/posix-timers.h~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/include/linux/posix-timers.h 2005-11-12 14:08:45.000000000 -0800 @@ -51,10 +51,9 @@ struct k_itimer { struct sigqueue *sigq; /* signal queue entry. */ union { struct { - struct timer_list timer; - struct list_head abs_timer_entry; /* clock abs_timer_list */ - struct timespec wall_to_prev; /* wall_to_monotonic used when set */ - unsigned long incr; /* interval in jiffies */ + struct ktimer timer; + ktime_t incr; + int overrun; } real; struct cpu_timer_list cpu; struct { @@ -66,10 +65,6 @@ struct k_itimer { } it; }; -struct k_clock_abs { - struct list_head list; - spinlock_t lock; -}; struct k_clock { int res; /* in nano seconds */ int (*clock_getres) (clockid_t which_clock, struct timespec *tp); @@ -77,7 +72,7 @@ struct k_clock { int (*clock_set) (clockid_t which_clock, struct timespec * tp); int (*clock_get) (clockid_t which_clock, struct timespec * tp); int (*timer_create) (struct k_itimer *timer); - int (*nsleep) (clockid_t which_clock, int flags, struct timespec *); + int (*nsleep) (clockid_t which_clock, int flags, struct timespec *, struct timespec __user *); int (*timer_set) (struct k_itimer * timr, int flags, struct itimerspec * new_setting, struct itimerspec * old_setting); @@ -91,37 +86,104 @@ void register_posix_clock(clockid_t cloc /* Error handlers for timer_create, nanosleep and settime */ int do_posix_clock_notimer_create(struct k_itimer *timer); -int do_posix_clock_nonanosleep(clockid_t, int flags, struct timespec *); +int do_posix_clock_nonanosleep(clockid_t, int flags, struct timespec *, struct timespec __user *); int do_posix_clock_nosettime(clockid_t, struct timespec *tp); /* function to call to trigger timer event */ int posix_timer_event(struct k_itimer *timr, int si_private); -struct now_struct { - unsigned long jiffies; -}; - -#define posix_get_now(now) (now)->jiffies = jiffies; -#define posix_time_before(timer, now) \ - time_before((timer)->expires, (now)->jiffies) - -#define posix_bump_timer(timr, now) \ - do { \ - long delta, orun; \ - delta = now.jiffies - (timr)->it.real.timer.expires; \ - if (delta >= 0) { \ - orun = 1 + (delta / (timr)->it.real.incr); \ - (timr)->it.real.timer.expires += \ - orun * (timr)->it.real.incr; \ - (timr)->it_overrun += orun; \ - } \ - }while (0) +#if (BITS_PER_LONG < 64) +static inline ktime_t forward_posix_timer(struct k_itimer *t, ktime_t now) +{ + ktime_t delta = ktime_sub(now, t->it.real.timer.expires); + unsigned long orun = 1; + + if (ktime_cmp_val(delta, <, KTIME_ZERO)) + goto out; + + if (unlikely(ktime_cmp(delta, >, t->it.real.incr))) { + + int sft = 0; + u64 div, dclc, inc, dns; + + dclc = dns = ktime_to_ns(delta); + div = inc = ktime_to_ns(t->it.real.incr); + /* Make sure the divisor is less than 2^32 */ + while(div >> 32) { + sft++; + div >>= 1; + } + dclc >>= sft; + do_div(dclc, (unsigned long) div); + orun = (unsigned long) dclc; + if (likely(!(inc >> 32))) + dclc *= (unsigned long) inc; + else + dclc *= inc; + t->it.real.timer.expires = ktime_add_ns(t->it.real.timer.expires, + dclc); + } else { + t->it.real.timer.expires = ktime_add(t->it.real.timer.expires, + t->it.real.incr); + } + /* + * Here is the correction for exact. Also covers delta == incr + * which is the else clause above. + */ + if (ktime_cmp(t->it.real.timer.expires, <=, now)) { + t->it.real.timer.expires = ktime_add(t->it.real.timer.expires, + t->it.real.incr); + orun++; + } + t->it_overrun += orun; + + out: + return ktime_sub(t->it.real.timer.expires, now); +} +#else +static inline ktime_t forward_posix_timer(struct k_itimer *t, ktime_t now) +{ + ktime_t delta = ktime_sub(now, t->it.real.timer.expires); + unsigned long orun = 1; + + if (ktime_cmp_val(delta, <, KTIME_ZERO)) + goto out; + + if (unlikely(ktime_cmp(delta, >, t->it.real.incr))) { + + u64 dns, inc; + + dns = ktime_to_ns(delta); + inc = ktime_to_ns(t->it.real.incr); + + orun = dns / inc; + t->it.real.timer.expires = ktime_add_ns(t->it.real.timer.expires, + orun * inc); + } else { + t->it.real.timer.expires = ktime_add(t->it.real.timer.expires, + t->it.real.incr); + } + /* + * Here is the correction for exact. Also covers delta == incr + * which is the else clause above. + */ + if (ktime_cmp(t->it.real.timer.expires, <=, now)) { + t->it.real.timer.expires = ktime_add(t->it.real.timer.expires, + t->it.real.incr); + orun++; + } + t->it_overrun += orun; + out: + return ktime_sub(t->it.real.timer.expires, now); +} +#endif int posix_cpu_clock_getres(clockid_t which_clock, struct timespec *); int posix_cpu_clock_get(clockid_t which_clock, struct timespec *); int posix_cpu_clock_set(clockid_t which_clock, const struct timespec *tp); int posix_cpu_timer_create(struct k_itimer *); -int posix_cpu_nsleep(clockid_t, int, struct timespec *); +int posix_cpu_nsleep(clockid_t, int, struct timespec *, + struct timespec __user *); int posix_cpu_timer_set(struct k_itimer *, int, struct itimerspec *, struct itimerspec *); int posix_cpu_timer_del(struct k_itimer *); diff -puN include/linux/sched.h~ktimers-kt2 include/linux/sched.h --- devel/include/linux/sched.h~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/include/linux/sched.h 2005-11-12 14:08:45.000000000 -0800 @@ -105,6 +105,7 @@ extern unsigned long nr_iowait(void); #include #include #include +#include #include @@ -411,8 +412,7 @@ struct signal_struct { struct list_head posix_timers; /* ITIMER_REAL timer for the process */ - struct timer_list real_timer; - unsigned long it_real_value, it_real_incr; + struct ktimer real_timer; /* ITIMER_PROF and ITIMER_VIRTUAL timers for the process */ cputime_t it_prof_expires, it_virt_expires; diff -puN include/linux/time.h~ktimers-kt2 include/linux/time.h --- devel/include/linux/time.h~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/include/linux/time.h 2005-11-12 14:09:39.000000000 -0800 @@ -4,6 +4,7 @@ #include #ifdef __KERNEL__ +#include #include #endif @@ -38,38 +39,15 @@ static __inline__ int timespec_equal(str return (a->tv_sec == b->tv_sec) && (a->tv_nsec == b->tv_nsec); } -/* Converts Gregorian date to seconds since 1970-01-01 00:00:00. - * Assumes input in normal date format, i.e. 1980-12-31 23:59:59 - * => year=1980, mon=12, day=31, hour=23, min=59, sec=59. - * - * [For the Julian calendar (which was used in Russia before 1917, - * Britain & colonies before 1752, anywhere else before 1582, - * and is still in use by some communities) leave out the - * -year/100+year/400 terms, and add 10.] - * - * This algorithm was first published by Gauss (I think). - * - * WARNING: this function will overflow on 2106-02-07 06:28:16 on - * machines were long is 32-bit! (However, as time_t is signed, we - * will already get problems at other places on 2038-01-19 03:14:08) - */ -static inline unsigned long +#define timespec_valid(ts) \ +(((ts)->tv_sec >= 0) && (((unsigned) (ts)->tv_nsec) < NSEC_PER_SEC)) + +typedef s64 nsec_t; + +extern unsigned long mktime (unsigned int year, unsigned int mon, unsigned int day, unsigned int hour, - unsigned int min, unsigned int sec) -{ - if (0 >= (int) (mon -= 2)) { /* 1..12 -> 11,12,1..10 */ - mon += 12; /* Puts Feb last since it has leap day */ - year -= 1; - } - - return ((( - (unsigned long) (year/4 - year/100 + year/400 + 367*mon/12 + day) + - year*365 - 719499 - )*24 + hour /* now have hours */ - )*60 + min /* now have minutes */ - )*60 + sec; /* finally seconds */ -} + unsigned int min, unsigned int sec); extern struct timespec xtime; extern struct timespec wall_to_monotonic; @@ -80,6 +58,8 @@ static inline unsigned long get_seconds( return xtime.tv_sec; } +extern void set_normalized_timespec (struct timespec *ts, time_t sec, long nsec); + struct timespec current_kernel_time(void); #define CURRENT_TIME (current_kernel_time()) @@ -88,8 +68,7 @@ struct timespec current_kernel_time(void extern void do_gettimeofday(struct timeval *tv); extern int do_settimeofday(struct timespec *tv); extern int do_sys_settimeofday(struct timespec *tv, struct timezone *tz); -extern void clock_was_set(void); // call when ever the clock is set -extern int do_posix_clock_monotonic_gettime(struct timespec *tp); +extern void do_posix_clock_monotonic_gettime(struct timespec *ts); extern long do_utimes(char __user * filename, struct timeval * times); struct itimerval; extern int do_setitimer(int which, struct itimerval *value, struct itimerval *ovalue); @@ -98,21 +77,37 @@ extern void getnstimeofday (struct times extern struct timespec timespec_trunc(struct timespec t, unsigned gran); -static inline void -set_normalized_timespec (struct timespec *ts, time_t sec, long nsec) +/** + * timespec_to_ns - Convert timespec to nanoseconds + * @ts: pointer to the timespec variable to be converted + * + * Returns the scalar nanosecond representation of the timespec + * variable + */ +static inline nsec_t timespec_to_ns(struct timespec *ts) { - while (nsec >= NSEC_PER_SEC) { - nsec -= NSEC_PER_SEC; - ++sec; - } - while (nsec < 0) { - nsec += NSEC_PER_SEC; - --sec; - } - ts->tv_sec = sec; - ts->tv_nsec = nsec; + nsec_t res = (nsec_t) ts->tv_sec * NSEC_PER_SEC; + + return res + (nsec_t) ts->tv_nsec; +} + +/** + * timeval_to_ns - Convert timeval to nanoseconds + * @ts: pointer to the timeval variable to be converted + * + * Returns the scalar nanosecond representation of the timeval + * variable + */ +static inline nsec_t timeval_to_ns(struct timeval *tv) +{ + nsec_t res = (nsec_t) tv->tv_sec * NSEC_PER_SEC; + + return res + (nsec_t) tv->tv_usec * NSEC_PER_USEC; } +extern void ns_to_timespec(struct timespec *ts, nsec_t nsec); +extern void ns_to_timeval(struct timeval *tv, nsec_t nsec); + #endif /* __KERNEL__ */ #define NFDBITS __NFDBITS @@ -145,23 +140,18 @@ struct itimerval { /* * The IDs of the various system clocks (for POSIX.1b interval timers). */ -#define CLOCK_REALTIME 0 -#define CLOCK_MONOTONIC 1 +#define CLOCK_REALTIME 0 +#define CLOCK_MONOTONIC 1 #define CLOCK_PROCESS_CPUTIME_ID 2 #define CLOCK_THREAD_CPUTIME_ID 3 -#define CLOCK_REALTIME_HR 4 -#define CLOCK_MONOTONIC_HR 5 /* * The IDs of various hardware clocks */ - - #define CLOCK_SGI_CYCLE 10 #define MAX_CLOCKS 16 -#define CLOCKS_MASK (CLOCK_REALTIME | CLOCK_MONOTONIC | \ - CLOCK_REALTIME_HR | CLOCK_MONOTONIC_HR) -#define CLOCKS_MONO (CLOCK_MONOTONIC & CLOCK_MONOTONIC_HR) +#define CLOCKS_MASK (CLOCK_REALTIME | CLOCK_MONOTONIC) +#define CLOCKS_MONO (CLOCK_MONOTONIC) /* * The various flags for setting POSIX.1b interval timers. diff -puN include/linux/timer.h~ktimers-kt2 include/linux/timer.h --- devel/include/linux/timer.h~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/include/linux/timer.h 2005-11-12 14:08:45.000000000 -0800 @@ -96,6 +96,6 @@ static inline void add_timer(struct time extern void init_timers(void); extern void run_local_timers(void); -extern void it_real_fn(unsigned long); +extern void it_real_fn(void *); #endif diff -puN init/main.c~ktimers-kt2 init/main.c --- devel/init/main.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/init/main.c 2005-11-12 14:08:45.000000000 -0800 @@ -490,6 +490,7 @@ asmlinkage void __init start_kernel(void init_IRQ(); pidhash_init(); init_timers(); + ktimers_init(); softirq_init(); time_init(); diff -puN kernel/exit.c~ktimers-kt2 kernel/exit.c --- devel/kernel/exit.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/exit.c 2005-11-12 14:08:45.000000000 -0800 @@ -841,7 +841,7 @@ fastcall NORET_TYPE void do_exit(long co } group_dead = atomic_dec_and_test(&tsk->signal->live); if (group_dead) { - del_timer_sync(&tsk->signal->real_timer); + ktimer_cancel(&tsk->signal->real_timer); exit_itimers(tsk->signal); acct_process(code); } diff -puN kernel/fork.c~ktimers-kt2 kernel/fork.c --- devel/kernel/fork.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/fork.c 2005-11-12 14:08:45.000000000 -0800 @@ -801,10 +801,9 @@ static inline int copy_signal(unsigned l init_sigpending(&sig->shared_pending); INIT_LIST_HEAD(&sig->posix_timers); - sig->it_real_value = sig->it_real_incr = 0; + ktimer_init(&sig->real_timer); sig->real_timer.function = it_real_fn; - sig->real_timer.data = (unsigned long) tsk; - init_timer(&sig->real_timer); + sig->real_timer.data = tsk; sig->it_virt_expires = cputime_zero; sig->it_virt_incr = cputime_zero; diff -puN kernel/itimer.c~ktimers-kt2 kernel/itimer.c --- devel/kernel/itimer.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/itimer.c 2005-11-12 14:08:45.000000000 -0800 @@ -12,36 +12,49 @@ #include #include #include +#include #include -static unsigned long it_real_value(struct signal_struct *sig) +/** + * itimer_get_remtime - get remaining time for the timer + * + * @timer: the timer to read + * @fake: a pending, but expired timer returns fake (itimers kludge) + * + * Returns the delta between the expiry time and now, which can be + * less than zero or the fake value described above. + */ +static ktime_t itimer_get_remtime(struct ktimer *timer, long fake) { - unsigned long val = 0; - if (timer_pending(&sig->real_timer)) { - val = sig->real_timer.expires - jiffies; - - /* look out for negative/zero itimer.. */ - if ((long) val <= 0) - val = 1; - } - return val; + ktime_t rem = ktimer_get_remtime(timer); + + /* + * Racy but safe: if the itimer expires after the above + * ktimer_get_remtime() call but before this condition + * then we return KTIMER_ZERO - which is correct. + */ + if (ktimer_active(timer)) { + if (ktime_cmp_val(rem, <=, KTIME_ZERO)) + rem = ktime_set(0, fake); + } else + ktime_set_scalar(rem, KTIME_ZERO); + + return rem; } int do_getitimer(int which, struct itimerval *value) { struct task_struct *tsk = current; - unsigned long interval, val; + ktime_t interval, val; cputime_t cinterval, cval; switch (which) { case ITIMER_REAL: - spin_lock_irq(&tsk->sighand->siglock); - interval = tsk->signal->it_real_incr; - val = it_real_value(tsk->signal); - spin_unlock_irq(&tsk->sighand->siglock); - jiffies_to_timeval(val, &value->it_value); - jiffies_to_timeval(interval, &value->it_interval); + interval = tsk->signal->real_timer.interval; + val = itimer_get_remtime(&tsk->signal->real_timer, NSEC_PER_USEC); + ktime_to_timeval(&value->it_value, val); + ktime_to_timeval(&value->it_interval, interval); break; case ITIMER_VIRTUAL: read_lock(&tasklist_lock); @@ -113,59 +126,36 @@ asmlinkage long sys_getitimer(int which, } -void it_real_fn(unsigned long __data) +/* + * The timer is automagically restarted, when interval != 0 + */ +void it_real_fn(void *data) { - struct task_struct * p = (struct task_struct *) __data; - unsigned long inc = p->signal->it_real_incr; - - send_group_sig_info(SIGALRM, SEND_SIG_PRIV, p); - - /* - * Now restart the timer if necessary. We don't need any locking - * here because do_setitimer makes sure we have finished running - * before it touches anything. - * Note, we KNOW we are (or should be) at a jiffie edge here so - * we don't need the +1 stuff. Also, we want to use the prior - * expire value so as to not "slip" a jiffie if we are late. - * Deal with requesting a time prior to "now" here rather than - * in add_timer. - */ - if (!inc) - return; - while (time_before_eq(p->signal->real_timer.expires, jiffies)) - p->signal->real_timer.expires += inc; - add_timer(&p->signal->real_timer); + send_group_sig_info(SIGALRM, SEND_SIG_PRIV, data); } int do_setitimer(int which, struct itimerval *value, struct itimerval *ovalue) { struct task_struct *tsk = current; - unsigned long val, interval, expires; + struct ktimer *timer; + ktime_t expires; cputime_t cval, cinterval, nval, ninterval; switch (which) { case ITIMER_REAL: -again: - spin_lock_irq(&tsk->sighand->siglock); - interval = tsk->signal->it_real_incr; - val = it_real_value(tsk->signal); - /* We are sharing ->siglock with it_real_fn() */ - if (try_to_del_timer_sync(&tsk->signal->real_timer) < 0) { - spin_unlock_irq(&tsk->sighand->siglock); - goto again; - } - tsk->signal->it_real_incr = - timeval_to_jiffies(&value->it_interval); - expires = timeval_to_jiffies(&value->it_value); - if (expires) - mod_timer(&tsk->signal->real_timer, - jiffies + 1 + expires); - spin_unlock_irq(&tsk->sighand->siglock); + timer = &tsk->signal->real_timer; + ktimer_cancel(timer); if (ovalue) { - jiffies_to_timeval(val, &ovalue->it_value); - jiffies_to_timeval(interval, - &ovalue->it_interval); - } + ktime_to_timeval(&ovalue->it_value, + itimer_get_remtime(timer, NSEC_PER_USEC)); + ktime_to_timeval(&ovalue->it_interval, timer->interval); + } + timer->interval = ktimer_round_timeval(timer, + &value->it_interval); + expires = timeval_to_ktime(value->it_value); + if (ktime_cmp_val(expires, != , KTIME_ZERO)) + ktimer_restart(timer, &expires, + KTIMER_REL | KTIMER_NOCHECK | KTIMER_ROUND); break; case ITIMER_VIRTUAL: nval = timeval_to_cputime(&value->it_value); diff -puN /dev/null kernel/ktimers.c --- /dev/null 2003-09-15 06:40:47.000000000 -0700 +++ devel-akpm/kernel/ktimers.c 2005-11-12 14:08:45.000000000 -0800 @@ -0,0 +1,1052 @@ +/* + * linux/kernel/ktimers.c + * + * Copyright(C) 2005, Thomas Gleixner + * Copyright(C) 2005, Red Hat, Inc., Ingo Molnar + * + * High-precision kernel timers + * + * In contrast to the low-resolution timeout API implemented in + * kernel/timer.c, ktimers provide finer resolution and accuracy + * depending on system configuration and capabilities. + * + * These timers are currently used for: + * - itimers + * - POSIX timers + * - nanosleep + * - precise in-kernel timing + * + * Started by: Thomas Gleixner and Ingo Molnar + * + * Credits: + * based on kernel/timer.c + * + * For licencing details see kernel-base/COPYING + */ + +#include +#include +#include +#include +#include +#include +#include + +#include + +/* + * The timer bases: + */ + +#define MAX_KTIMER_BASES 2 + +static DEFINE_PER_CPU(struct ktimer_base, ktimer_bases[MAX_KTIMER_BASES]) = +{ + { + .index = CLOCK_REALTIME, + .name = "Realtime", + .get_time = &ktime_get_real, + .resolution = KTIME_REALTIME_RES, + }, + { + .index = CLOCK_MONOTONIC, + .name = "Monotonic", + .get_time = &ktime_get, + .resolution = KTIME_MONOTONIC_RES, + }, +}; + +/** + * ktime_get - get the monotonic time in ktime_t format + * + * returns the time in ktime_t format + */ +ktime_t ktime_get(void) +{ + struct timespec now; + + ktime_get_ts(&now); + + return timespec_to_ktime(now); +} + +EXPORT_SYMBOL_GPL(ktime_get); + +/** + * ktime_get_real - get the real (wall-) time in ktime_t format + * + * returns the time in ktime_t format + */ +ktime_t ktime_get_real(void) +{ + struct timespec now; + + getnstimeofday(&now); + + return timespec_to_ktime(now); +} + +EXPORT_SYMBOL_GPL(ktime_get_real); + +/** + * ktime_get_ts - get the monotonic clock in timespec format + * + * @ts: pointer to timespec variable + * + * The function calculates the monotonic clock from the realtime + * clock and the wall_to_monotonic offset and stores the result + * in normalized timespec format in the variable pointed to by ts. + */ +void ktime_get_ts(struct timespec *ts) +{ + struct timespec tomono; + unsigned long seq; + + do { + seq = read_seqbegin(&xtime_lock); + getnstimeofday(ts); + tomono = wall_to_monotonic; + + } while (read_seqretry(&xtime_lock, seq)); + + set_normalized_timespec(ts, ts->tv_sec + tomono.tv_sec, + ts->tv_nsec + tomono.tv_nsec); +} + +/* + * Functions and macros which are different for UP/SMP systems are kept in a + * single place + */ +#ifdef CONFIG_SMP + +#define set_curr_timer(b, t) (b)->curr_timer = (t) +#define wake_up_timer_waiters(b) wake_up(&(b)->wait) + +/** + * wait_for_ktimer - Wait for a running ktimer + * + * @timer: timer to wait for + * + * The function waits in case the timers callback function is + * currently executed on the waitqueue of the timer base. The + * waitqueue is woken up after the timer callback function has + * finished execution. + */ +void wait_for_ktimer(struct ktimer *timer) +{ + struct ktimer_base *base = timer->base; + + if (base) + wait_event(base->wait, + base->curr_timer != timer); +} + +/* + * We are using hashed locking: holding per_cpu(ktimer_bases)[n].lock + * means that all timers which are tied to this base via timer->base are + * locked, and the base itself is locked too. + * + * So __run_timers/migrate_timers can safely modify all timers which could + * be found on the lists/queues. + * + * When the timer's base is locked, and the timer removed from list, it is + * possible to set timer->base = NULL and drop the lock: the timer remains + * locked. + */ +static struct ktimer_base *lock_ktimer_base(struct ktimer *timer, + unsigned long *flags) +{ + struct ktimer_base *base; + + for (;;) { + base = timer->base; + if (likely(base != NULL)) { + spin_lock_irqsave(&base->lock, *flags); + if (likely(base == timer->base)) + return base; + /* The timer has migrated to another CPU */ + spin_unlock_irqrestore(&base->lock, *flags); + } + cpu_relax(); + } +} + +/* + * Switch the timer base to the current CPU when possible. + */ +static inline struct ktimer_base * +switch_ktimer_base(struct ktimer *timer, struct ktimer_base *base) +{ + struct ktimer_base *new_base; + + new_base = &__get_cpu_var(ktimer_bases[base->index]); + + if (base != new_base) { + /* + * We are trying to schedule the timer on the local CPU. + * However we can't change timer's base while it is running, + * so we keep it on the same CPU. No hassle vs. reprogramming + * the event source in the high resolution case. The softirq + * code will take care of this when the timer function has + * completed. There is no conflict as we hold the lock until + * the timer is enqueued. + */ + if (unlikely(base->curr_timer == timer)) + return base; + + /* See the comment in lock_timer_base() */ + timer->base = NULL; + spin_unlock(&base->lock); + spin_lock(&new_base->lock); + timer->base = new_base; + } + return new_base; +} + +/* + * Get the timer base unlocked + * + * Take care of timer->base = NULL in switch_ktimer_base ! + */ +static inline struct ktimer_base * +get_ktimer_base_unlocked(struct ktimer *timer) +{ + struct ktimer_base *base; + + while (!(base = timer->base)) + cpu_relax(); + + return base; +} + +#else /* CONFIG_SMP */ + +#define set_curr_timer(b, t) do { } while (0) +#define wake_up_timer_waiters(b) do { } while (0) + +static inline struct ktimer_base * +lock_ktimer_base(struct ktimer *timer, unsigned long *flags) +{ + struct ktimer_base *base = timer->base; + + spin_lock_irqsave(&base->lock, *flags); + + return base; +} + +#define switch_ktimer_base(t, b) (b) +#define get_ktimer_base_unlocked(t) (t)->base + +#endif /* !CONFIG_SMP */ + +/* + * Functions for the union type storage format of ktime_t which are + * too large for inlining: + */ +#if (BITS_PER_LONG < 64) && !defined(CONFIG_KTIME_SCALAR) + +/** + * ktime_add_ns - Add a scalar nanoseconds value to a ktime_t variable + * + * @kt: addend + * @nsec: the scalar nsec value to add + * + * Returns the sum of kt and nsec in ktime_t format + */ +ktime_t ktime_add_ns(ktime_t kt, u64 nsec) +{ + ktime_t tmp; + + if (likely(nsec < NSEC_PER_SEC)) { + tmp.tv64 = nsec; + } else { + unsigned long rem = do_div(nsec, NSEC_PER_SEC); + + tmp = ktime_set((long)nsec, rem); + } + + return ktime_add(kt, tmp); +} +#endif + +/* + * Counterpart to lock_timer_base above. + */ +static inline +void unlock_ktimer_base(struct ktimer *timer, unsigned long *flags) +{ + spin_unlock_irqrestore(&timer->base->lock, *flags); +} + +/** + * ktimer_round_timespec - convert timespec to ktime_t with resolution + * adjustment + * + * @timer: ktimer to retrieve the base + * @ts: pointer to the timespec value to be converted + * + * Returns the resolution adjusted ktime_t representation of the + * timespec. + * + * Note: We can access base without locking here, as ktimers can + * migrate between CPUs but can not be moved from one clock source to + * another. The clock source binding is set at init_ktimer_XXX time. + */ +ktime_t ktimer_round_timespec(struct ktimer *timer, struct timespec *ts) +{ + struct ktimer_base *base = get_ktimer_base_unlocked(timer); + long rem = ts->tv_nsec % base->resolution; + ktime_t t; + + t = ktime_set(ts->tv_sec, ts->tv_nsec); + + /* Check, if the value has to be rounded */ + if (rem) + t = ktime_add_ns(t, base->resolution - rem); + + return t; +} + +/** + * ktimer_round_timeval - convert timeval to ktime_t with resolution + * adjustment + * + * @timer: ktimer to retrieve the base + * @tv: pointer to the timeval value to be converted + * + * Returns the resolution adjusted ktime_t representation of the + * timeval. + */ +ktime_t ktimer_round_timeval(struct ktimer *timer, struct timeval *tv) +{ + struct timespec ts; + + ts.tv_sec = tv->tv_sec; + ts.tv_nsec = tv->tv_usec * NSEC_PER_USEC; + + return ktimer_round_timespec(timer, &ts); +} + +/* + * enqueue_ktimer - internal function to (re)start a timer + * + * The timer is inserted in expiry order. Insertion into the + * red black tree is O(log(n)). Must hold the base lock. + */ +static int enqueue_ktimer(struct ktimer *timer, struct ktimer_base *base, + ktime_t *tim, int mode) +{ + struct rb_node **link = &base->active.rb_node; + struct list_head *prev = &base->pending; + struct rb_node *parent = NULL; + struct ktimer *entry; + ktime_t now; + + /* Get current time */ + now = base->get_time(); + + /* + * Calculate the absolute expiry time based on the + * timer expiry mode: + */ + switch (mode & ~(KTIMER_NOCHECK | KTIMER_ROUND)) { + + case KTIMER_ABS: + timer->expires = *tim; + break; + + case KTIMER_REL: + timer->expires = ktime_add(now, *tim); + break; + + case KTIMER_INCR: + timer->expires = ktime_add(timer->expires, *tim); + break; + + case KTIMER_FORWARD: + while ktime_cmp(timer->expires, <= , now) { + timer->expires = ktime_add(timer->expires, *tim); + timer->overrun++; + } + goto nocheck; + + case KTIMER_REARM: + while ktime_cmp(timer->expires, <= , now) { + timer->expires = ktime_add(timer->expires, + timer->interval); + timer->overrun++; + } + goto nocheck; + + case KTIMER_RESTART: + break; + + default: + /* illegal mode */ + BUG(); + } + + /* + * Rounding is requested for one shot timers and the first + * event of interval timers. It's done here, so we don't + * have to read the current time twice for relative timers. + */ + if (mode & KTIMER_ROUND) { + unsigned long rem; + + rem = ((unsigned long)ktime_get_low(timer->expires)) % + base->resolution; + if (rem) + timer->expires = ktime_add_ns(timer->expires, + base->resolution - rem); + } + + /* Expiry time in the past: */ + if (unlikely(ktime_cmp(timer->expires, <=, now))) { + timer->expired = now; + /* The caller takes care of expiry */ + if (!(mode & KTIMER_NOCHECK)) + return -1; + } + nocheck: + + /* + * Find the right place in the rbtree: + */ + while (*link) { + parent = *link; + entry = rb_entry(parent, struct ktimer, node); + /* + * We dont care about collisions. Nodes with + * the same expiry time stay together. + */ + if (ktime_cmp(timer->expires, <, entry->expires)) + link = &(*link)->rb_left; + else { + link = &(*link)->rb_right; + prev = &entry->list; + } + } + + /* + * Insert the timer to the rbtree and to the sorted list: + */ + rb_link_node(&timer->node, parent, link); + rb_insert_color(&timer->node, &base->active); + list_add(&timer->list, prev); + + timer->state = KTIMER_PENDING; + base->count++; + + return 0; +} + +/* + * __remove_ktimer - internal function to remove a timer + * + * The function also allows automatic rearming for interval timers. + * Must hold the base lock. + */ +static void +__remove_ktimer(struct ktimer *timer, struct ktimer_base *base, + enum ktimer_rearm rearm) +{ + /* + * Remove the timer from the sorted list and from the rbtree: + */ + list_del(&timer->list); + rb_erase(&timer->node, &base->active); + timer->node.rb_parent = KTIMER_POISON; + + timer->state = KTIMER_INACTIVE; + base->count--; + BUG_ON(base->count < 0); + + /* Auto rearm the timer ? */ + if (rearm && ktime_cmp_val(timer->interval, !=, KTIME_ZERO)) + enqueue_ktimer(timer, base, NULL, KTIMER_REARM); +} + +/* + * remove ktimer, called with base lock held + */ +static inline int remove_ktimer(struct ktimer *timer, struct ktimer_base *base) +{ + if (ktimer_active(timer)) { + __remove_ktimer(timer, base, KTIMER_NOREARM); + return 1; + } + return 0; +} + +/* + * Internal function to (re)start a timer. + */ +static int +internal_restart_ktimer(struct ktimer *timer, ktime_t *tim, int mode) +{ + struct ktimer_base *base, *new_base; + unsigned long flags; + int ret; + + BUG_ON(!timer->function); + + base = lock_ktimer_base(timer, &flags); + + /* Remove an active timer from the queue */ + ret = remove_ktimer(timer, base); + + /* Switch the timer base, if necessary */ + new_base = switch_ktimer_base(timer, base); + + /* + * When the new timer setting is already expired, + * let the calling code deal with it. + */ + if (enqueue_ktimer(timer, new_base, tim, mode)) + ret = -1; + + unlock_ktimer_base(timer, &flags); + + return ret; +} + +/** + * ktimer_start - start a timer on the current CPU + * + * @timer: the timer to be added + * @tim: expiry time (optional, if not set in the timer) + * @mode: timer setup mode + * + * Returns: + * 0 on success + * -1 when the new time setting is already expired + */ +int ktimer_start(struct ktimer *timer, ktime_t *tim, int mode) +{ + BUG_ON(ktimer_active(timer)); + + return internal_restart_ktimer(timer, tim, mode); +} + +EXPORT_SYMBOL_GPL(ktimer_start); + +/** + * ktimer_restart - modify a running timer + * + * @timer: the timer to be modified + * @tim: expiry time (required) + * @mode: timer setup mode + * + * Returns: + * 0 when the timer was not active + * 1 when the timer was active + * -1 when the new time setting is already expired + */ +int ktimer_restart(struct ktimer *timer, ktime_t *tim, int mode) +{ + BUG_ON(!tim); + + return internal_restart_ktimer(timer, tim, mode); +} + +EXPORT_SYMBOL_GPL(ktimer_restart); + +/** + * ktimer_try_to_cancel - try to deactivate a timer + * + * @timer: ktimer to stop + * + * Returns: + * 0 when the timer was not active + * 1 when the timer was active + * -1 when the timer is currently excuting the callback function and + * can not be stopped + */ +int ktimer_try_to_cancel(struct ktimer *timer) +{ + struct ktimer_base *base; + unsigned long flags; + int ret = -1; + + base = lock_ktimer_base(timer, &flags); + + if (base->curr_timer != timer) { + ret = remove_ktimer(timer, base); + if (ret) + timer->expired = base->get_time(); + } + + unlock_ktimer_base(timer, &flags); + + return ret; + +} + +EXPORT_SYMBOL_GPL(ktimer_try_to_cancel); + +/** + * ktimer_cancel - cancel a timer and wait for the handler to finish. + * + * @timer: the timer to be cancelled + * + * Returns: + * 0 when the timer was not active + * 1 when the timer was active + */ +int ktimer_cancel(struct ktimer *timer) +{ + for (;;) { + int ret = ktimer_try_to_cancel(timer); + + if (ret >= 0) + return ret; + wait_for_ktimer(timer); + } +} + +EXPORT_SYMBOL_GPL(ktimer_cancel); + +/** + * ktimer_get_remtime - get remaining time for the timer + * + * @timer: the timer to read + * + * Returns the delta between the expiry time and now, which can be + * less than zero. + */ +ktime_t ktimer_get_remtime(struct ktimer *timer) +{ + struct ktimer_base *base; + unsigned long flags; + ktime_t rem; + + base = lock_ktimer_base(timer, &flags); + rem = ktime_sub(timer->expires, base->get_time()); + unlock_ktimer_base(timer, &flags); + + return rem; +} + +/** + * ktimer_get_expiry - get expiry time for the timer + * + * @timer: the timer to read + * @now: if != NULL then store current base->time into it + */ +ktime_t ktimer_get_expiry(struct ktimer *timer, ktime_t *now) +{ + struct ktimer_base *base; + unsigned long flags; + ktime_t expiry; + + base = lock_ktimer_base(timer, &flags); + expiry = timer->expires; + if (now) + *now = base->get_time(); + unlock_ktimer_base(timer, &flags); + + return expiry; +} + +/* + * Functions related to clock sources + */ + +static inline void ktimer_common_init(struct ktimer *timer) +{ + memset(timer, 0, sizeof(struct ktimer)); + timer->node.rb_parent = KTIMER_POISON; +} + +/** + * ktimer_init - initialize a timer to the monotonic clock + * + * @timer: the timer to be initialized + */ +void ktimer_init(struct ktimer *timer) +{ + struct ktimer_base *bases; + + ktimer_common_init(timer); + bases = per_cpu(ktimer_bases, raw_smp_processor_id()); + timer->base = &bases[CLOCK_MONOTONIC]; +} + +EXPORT_SYMBOL_GPL(ktimer_init); + +/** + * ktimer_init_real - initialize a timer to the real (wall-) clock + * + * @timer: the timer to be initialized + */ +void ktimer_init_real(struct ktimer *timer) +{ + struct ktimer_base *bases; + + ktimer_common_init(timer); + bases = per_cpu(ktimer_bases, raw_smp_processor_id()); + timer->base = &bases[CLOCK_REALTIME]; +} + +EXPORT_SYMBOL_GPL(ktimer_init_real); + +/** + * ktimer_get_res - get the monotonic timer resolution + * + * @which_clock: unused parameter for compability with the posix timer code + * @tp: pointer to timespec variable to store the resolution + * + * Store the resolution of clock monotonic in the variable pointed to + * by tp. + */ +int ktimer_get_res(clockid_t which_clock, struct timespec *tp) +{ + struct ktimer_base *bases; + + tp->tv_sec = 0; + bases = per_cpu(ktimer_bases, raw_smp_processor_id()); + tp->tv_nsec = bases[CLOCK_MONOTONIC].resolution; + + return 0; +} + +/** + * ktimer_get_res_real - get the real timer resolution + * + * @which_clock: unused parameter for compability with the posix timer code + * @tp: pointer to timespec variable to store the resolution + * + * Store the resolution of clock realtime in the variable pointed to + * by tp. + */ +int ktimer_get_res_real(clockid_t which_clock, struct timespec *tp) +{ + struct ktimer_base *bases; + + tp->tv_sec = 0; + bases = per_cpu(ktimer_bases, raw_smp_processor_id()); + tp->tv_nsec = bases[CLOCK_REALTIME].resolution; + + return 0; +} + +/* + * Expire the per base ktimer-queue: + */ +static inline void run_ktimer_queue(struct ktimer_base *base) +{ + ktime_t now = base->get_time(); + + spin_lock_irq(&base->lock); + + while (!list_empty(&base->pending)) { + struct ktimer *timer; + void (*fn)(void *); + void *data; + + timer = list_entry(base->pending.next, struct ktimer, list); + if (ktime_cmp(now, <=, timer->expires)) + break; + + timer->expired = now; + fn = timer->function; + data = timer->data; + set_curr_timer(base, timer); + __remove_ktimer(timer, base, KTIMER_REARM); + spin_unlock_irq(&base->lock); + + fn(data); + + spin_lock_irq(&base->lock); + set_curr_timer(base, NULL); + } + spin_unlock_irq(&base->lock); + + wake_up_timer_waiters(base); +} + +/* + * Called from timer softirq every jiffy, expire ktimers: + */ +void ktimer_run_queues(void) +{ + struct ktimer_base *base = __get_cpu_var(ktimer_bases); + int i; + + for (i = 0; i < MAX_KTIMER_BASES; i++) + run_ktimer_queue(&base[i]); +} + +/* + * Sleep related functions: + */ + +/* + * Process-wakeup callback: + */ +static void ktimer_wake_up(void *data) +{ + wake_up_process(data); +} + +/** + * schedule_ktimer - sleep until timeout + * + * @timer: ktimer variable initialized with the correct clock base + * @t: timeout value + * @mode: timeout value is abs/rel + * + * Make the current task sleep until @timeout is + * elapsed. + * + * You can set the task state as follows - + * + * %TASK_UNINTERRUPTIBLE - at least @timeout is guaranteed to + * pass before the routine returns. The routine will return 0 + * + * %TASK_INTERRUPTIBLE - the routine may return early if a signal is + * delivered to the current task. In this case the remaining time + * will be returned + * + * The current task state is guaranteed to be TASK_RUNNING when this + * routine returns. + */ +static ktime_t __sched +schedule_ktimer(struct ktimer *timer, ktime_t *t, int mode) +{ + timer->data = current; + timer->function = ktimer_wake_up; + + if (unlikely(ktimer_start(timer, t, mode) < 0)) { + __set_current_state(TASK_RUNNING); + } else { + if (current->state != TASK_RUNNING) + schedule(); + ktimer_cancel(timer); + } + + /* Store the absolute expiry time */ + *t = timer->expires; + + /* Return the remaining time */ + return ktime_sub(timer->expires, timer->expired); +} + +static ktime_t __sched +schedule_ktimer_interruptible(struct ktimer *timer, ktime_t *t, int mode) +{ + set_current_state(TASK_INTERRUPTIBLE); + + return schedule_ktimer(timer, t, mode); +} + +static long __sched +nanosleep_restart(struct ktimer *timer, struct restart_block *restart) +{ + void *rfn_save = restart->fn; + struct timespec __user *rmtp; + struct timespec tu; + ktime_t t, rem; + + restart->fn = do_no_restart_syscall; + + t = ktime_set_low_high(restart->arg0, restart->arg1); + + rem = schedule_ktimer_interruptible(timer, &t, KTIMER_ABS); + + if (ktime_cmp_val(rem, <=, KTIME_ZERO)) + return 0; + + rmtp = (struct timespec __user *) restart->arg2; + ktime_to_timespec(&tu, rem); + if (rmtp && copy_to_user(rmtp, &tu, sizeof(tu))) + return -EFAULT; + + restart->fn = rfn_save; + + /* The other values in restart are already filled in */ + return -ERESTART_RESTARTBLOCK; +} + +static long __sched nanosleep_restart_mono(struct restart_block *restart) +{ + struct ktimer timer; + + ktimer_init(&timer); + + return nanosleep_restart(&timer, restart); +} + +static long __sched nanosleep_restart_real(struct restart_block *restart) +{ + struct ktimer timer; + + ktimer_init_real(&timer); + + return nanosleep_restart(&timer, restart); +} + +static long __ktimer_nanosleep(struct ktimer *timer, struct timespec *rqtp, + struct timespec __user *rmtp, int mode, + long (*rfn)(struct restart_block *)) +{ + struct timespec tu; + ktime_t rem, t; + struct restart_block *restart; + + t = timespec_to_ktime(*rqtp); + + /* t is updated to absolute expiry time ! */ + rem = schedule_ktimer_interruptible(timer, &t, mode | KTIMER_ROUND); + + if (ktime_cmp_val(rem, <=, KTIME_ZERO)) + return 0; + + ktime_to_timespec(&tu, rem); + + if (rmtp && copy_to_user(rmtp, &tu, sizeof(tu))) + return -EFAULT; + + restart = ¤t_thread_info()->restart_block; + restart->fn = rfn; + restart->arg0 = ktime_get_low(t); + restart->arg1 = ktime_get_high(t); + restart->arg2 = (unsigned long) rmtp; + + return -ERESTART_RESTARTBLOCK; +} + +long ktimer_nanosleep(struct timespec *rqtp, + struct timespec __user *rmtp, int mode) +{ + struct ktimer timer; + + ktimer_init(&timer); + + return __ktimer_nanosleep(&timer, rqtp, rmtp, mode, + nanosleep_restart_mono); +} + +long ktimer_nanosleep_real(struct timespec *rqtp, + struct timespec __user *rmtp, int mode) +{ + struct ktimer timer; + + ktimer_init_real(&timer); + return __ktimer_nanosleep(&timer, rqtp, rmtp, mode, + nanosleep_restart_real); +} + +asmlinkage long +sys_nanosleep(struct timespec __user *rqtp, struct timespec __user *rmtp) +{ + struct timespec tu; + + if (copy_from_user(&tu, rqtp, sizeof(tu))) + return -EFAULT; + + if (!timespec_valid(&tu)) + return -EINVAL; + + return ktimer_nanosleep(&tu, rmtp, KTIMER_REL); +} + +/* + * Functions related to boot-time initialization: + */ +static void __devinit init_ktimers_cpu(int cpu) +{ + struct ktimer_base *base = per_cpu(ktimer_bases, cpu); + int i; + + for (i = 0; i < MAX_KTIMER_BASES; i++) { + spin_lock_init(&base->lock); + INIT_LIST_HEAD(&base->pending); + init_waitqueue_head(&base->wait); + base++; + } +} + +#ifdef CONFIG_HOTPLUG_CPU + +static void migrate_ktimer_list(struct ktimer_base *old_base, + struct ktimer_base *new_base) +{ + struct ktimer *timer; + struct rb_node *node; + + while ((node = rb_first(&old_base->active))) { + timer = rb_entry(node, struct ktimer, node); + remove_ktimer(timer, old_base); + timer->base = new_base; + enqueue_ktimer(timer, new_base, NULL, KTIMER_RESTART); + } +} + +static void migrate_ktimers(int cpu) +{ + struct ktimer_base *old_base, *new_base; + int i; + + BUG_ON(cpu_online(cpu)); + old_base = per_cpu(ktimer_bases, cpu); + new_base = get_cpu_var(ktimer_bases); + + local_irq_disable(); + + for (i = 0; i < MAX_KTIMER_BASES; i++) { + + spin_lock(&new_base->lock); + spin_lock(&old_base->lock); + + BUG_ON(old_base->curr_timer); + + migrate_ktimer_list(old_base, new_base); + + spin_unlock(&old_base->lock); + spin_unlock(&new_base->lock); + old_base++; + new_base++; + } + + local_irq_enable(); + put_cpu_var(ktimer_bases); +} +#endif /* CONFIG_HOTPLUG_CPU */ + +static int __devinit ktimer_cpu_notify(struct notifier_block *self, + unsigned long action, void *hcpu) +{ + long cpu = (long)hcpu; + + switch(action) { + + case CPU_UP_PREPARE: + init_ktimers_cpu(cpu); + break; + +#ifdef CONFIG_HOTPLUG_CPU + case CPU_DEAD: + migrate_ktimers(cpu); + break; +#endif + + default: + break; + } + + return NOTIFY_OK; +} + +static struct notifier_block __devinitdata ktimers_nb = { + .notifier_call = ktimer_cpu_notify, +}; + +void __init ktimers_init(void) +{ + ktimer_cpu_notify(&ktimers_nb, (unsigned long)CPU_UP_PREPARE, + (void *)(long)smp_processor_id()); + register_cpu_notifier(&ktimers_nb); +} + diff -puN kernel/Makefile~ktimers-kt2 kernel/Makefile --- devel/kernel/Makefile~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/Makefile 2005-11-12 14:08:45.000000000 -0800 @@ -7,7 +7,8 @@ obj-y = sched.o fork.o exec_domain.o sysctl.o capability.o ptrace.o timer.o user.o \ signal.o sys.o kmod.o workqueue.o pid.o \ rcupdate.o intermodule.o extable.o params.o posix-timers.o \ - kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o + kthread.o wait.o kfifo.o sys_ni.o posix-cpu-timers.o \ + ktimers.o obj-$(CONFIG_FUTEX) += futex.o obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o diff -puN kernel/posix-cpu-timers.c~ktimers-kt2 kernel/posix-cpu-timers.c --- devel/kernel/posix-cpu-timers.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/posix-cpu-timers.c 2005-11-12 14:08:45.000000000 -0800 @@ -1411,7 +1411,7 @@ void set_process_cpu_timer(struct task_s static long posix_cpu_clock_nanosleep_restart(struct restart_block *); int posix_cpu_nsleep(clockid_t which_clock, int flags, - struct timespec *rqtp) + struct timespec *rqtp, struct timespec __user *rmtp) { struct restart_block *restart_block = ¤t_thread_info()->restart_block; @@ -1436,7 +1436,6 @@ int posix_cpu_nsleep(clockid_t which_clo error = posix_cpu_timer_create(&timer); timer.it_process = current; if (!error) { - struct timespec __user *rmtp; static struct itimerspec zero_it; struct itimerspec it = { .it_value = *rqtp, .it_interval = {} }; @@ -1483,7 +1482,6 @@ int posix_cpu_nsleep(clockid_t which_clo /* * Report back to the user the time still remaining. */ - rmtp = (struct timespec __user *) restart_block->arg1; if (rmtp != NULL && !(flags & TIMER_ABSTIME) && copy_to_user(rmtp, &it.it_value, sizeof *rmtp)) return -EFAULT; @@ -1491,6 +1489,7 @@ int posix_cpu_nsleep(clockid_t which_clo restart_block->fn = posix_cpu_clock_nanosleep_restart; /* Caller already set restart_block->arg1 */ restart_block->arg0 = which_clock; + restart_block->arg1 = (unsigned long) rmtp; restart_block->arg2 = rqtp->tv_sec; restart_block->arg3 = rqtp->tv_nsec; @@ -1504,10 +1503,15 @@ static long posix_cpu_clock_nanosleep_restart(struct restart_block *restart_block) { clockid_t which_clock = restart_block->arg0; - struct timespec t = { .tv_sec = restart_block->arg2, - .tv_nsec = restart_block->arg3 }; + struct timespec __user *rmtp; + struct timespec t; + + rmtp = (struct timespec __user *) restart_block->arg1; + t.tv_sec = restart_block->arg2; + t.tv_nsec = restart_block->arg3; + restart_block->fn = do_no_restart_syscall; - return posix_cpu_nsleep(which_clock, TIMER_ABSTIME, &t); + return posix_cpu_nsleep(which_clock, TIMER_ABSTIME, &t, rmtp); } @@ -1528,9 +1532,10 @@ static int process_cpu_timer_create(stru return posix_cpu_timer_create(timer); } static int process_cpu_nsleep(clockid_t which_clock, int flags, - struct timespec *rqtp) + struct timespec *rqtp, + struct timespec __user *rmtp) { - return posix_cpu_nsleep(PROCESS_CLOCK, flags, rqtp); + return posix_cpu_nsleep(PROCESS_CLOCK, flags, rqtp, rmtp); } static int thread_cpu_clock_getres(clockid_t which_clock, struct timespec *tp) { @@ -1546,7 +1551,7 @@ static int thread_cpu_timer_create(struc return posix_cpu_timer_create(timer); } static int thread_cpu_nsleep(clockid_t which_clock, int flags, - struct timespec *rqtp) + struct timespec *rqtp, struct timespec __user *rmtp) { return -EINVAL; } diff -puN kernel/posix-timers.c~ktimers-kt2 kernel/posix-timers.c --- devel/kernel/posix-timers.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/posix-timers.c 2005-11-12 14:11:13.000000000 -0800 @@ -48,21 +48,6 @@ #include #include -#ifndef div_long_long_rem -#include - -#define div_long_long_rem(dividend,divisor,remainder) ({ \ - u64 result = dividend; \ - *remainder = do_div(result,divisor); \ - result; }) - -#endif -#define CLOCK_REALTIME_RES TICK_NSEC /* In nano seconds. */ - -static inline u64 mpy_l_X_l_ll(unsigned long mpy1,unsigned long mpy2) -{ - return (u64)mpy1 * mpy2; -} /* * Management arrays for POSIX timers. Timers are kept in slab memory * Timer ids are allocated by an external routine that keeps track of the @@ -148,18 +133,18 @@ static DEFINE_SPINLOCK(idr_lock); */ static struct k_clock posix_clocks[MAX_CLOCKS]; + /* - * We only have one real clock that can be set so we need only one abs list, - * even if we should want to have several clocks with differing resolutions. + * These ones are defined below. */ -static struct k_clock_abs abs_list = {.list = LIST_HEAD_INIT(abs_list.list), - .lock = SPIN_LOCK_UNLOCKED}; +static int common_nsleep(clockid_t, int flags, struct timespec *t, + struct timespec __user *rmtp); +static void common_timer_get(struct k_itimer *, struct itimerspec *); +static int common_timer_set(struct k_itimer *, int, + struct itimerspec *, struct itimerspec *); +static int common_timer_del(struct k_itimer *timer); -static void posix_timer_fn(unsigned long); -static u64 do_posix_clock_monotonic_gettime_parts( - struct timespec *tp, struct timespec *mo); -int do_posix_clock_monotonic_gettime(struct timespec *tp); -static int do_posix_clock_monotonic_get(clockid_t, struct timespec *tp); +static void posix_timer_fn(void *data); static struct k_itimer *lock_timer(timer_t timer_id, unsigned long *flags); @@ -205,21 +190,25 @@ static inline int common_clock_set(clock static inline int common_timer_create(struct k_itimer *new_timer) { - INIT_LIST_HEAD(&new_timer->it.real.abs_timer_entry); - init_timer(&new_timer->it.real.timer); - new_timer->it.real.timer.data = (unsigned long) new_timer; + return -EINVAL; +} + +static int timer_create_mono(struct k_itimer *new_timer) +{ + ktimer_init(&new_timer->it.real.timer); + new_timer->it.real.timer.data = new_timer; + new_timer->it.real.timer.function = posix_timer_fn; + return 0; +} + +static int timer_create_real(struct k_itimer *new_timer) +{ + ktimer_init_real(&new_timer->it.real.timer); + new_timer->it.real.timer.data = new_timer; new_timer->it.real.timer.function = posix_timer_fn; return 0; } -/* - * These ones are defined below. - */ -static int common_nsleep(clockid_t, int flags, struct timespec *t); -static void common_timer_get(struct k_itimer *, struct itimerspec *); -static int common_timer_set(struct k_itimer *, int, - struct itimerspec *, struct itimerspec *); -static int common_timer_del(struct k_itimer *timer); /* * Return nonzero iff we know a priori this clockid_t value is bogus. @@ -239,19 +228,44 @@ static inline int invalid_clockid(clocki return 1; } +/* + * Get real time for posix timers + */ +static int posix_ktime_get_real_ts(clockid_t which_clock, struct timespec *tp) +{ + ktime_get_real_ts(tp); + return 0; +} + +/* + * Get monotonic time for posix timers + */ +static int posix_ktime_get_ts(clockid_t which_clock, struct timespec *tp) +{ + ktime_get_ts(tp); + return 0; +} + +void do_posix_clock_monotonic_gettime(struct timespec *ts) +{ + ktime_get_ts(ts); +} /* * Initialize everything, well, just everything in Posix clocks/timers ;) */ static __init int init_posix_timers(void) { - struct k_clock clock_realtime = {.res = CLOCK_REALTIME_RES, - .abs_struct = &abs_list + struct k_clock clock_realtime = { + .clock_getres = ktimer_get_res_real, + .clock_get = posix_ktime_get_real_ts, + .timer_create = timer_create_real, }; - struct k_clock clock_monotonic = {.res = CLOCK_REALTIME_RES, - .abs_struct = NULL, - .clock_get = do_posix_clock_monotonic_get, - .clock_set = do_posix_clock_nosettime + struct k_clock clock_monotonic = { + .clock_getres = ktimer_get_res, + .clock_get = posix_ktime_get_ts, + .clock_set = do_posix_clock_nosettime, + .timer_create = timer_create_mono, }; register_posix_clock(CLOCK_REALTIME, &clock_realtime); @@ -265,117 +279,16 @@ static __init int init_posix_timers(void __initcall(init_posix_timers); -static void tstojiffie(struct timespec *tp, int res, u64 *jiff) -{ - long sec = tp->tv_sec; - long nsec = tp->tv_nsec + res - 1; - - if (nsec >= NSEC_PER_SEC) { - sec++; - nsec -= NSEC_PER_SEC; - } - - /* - * The scaling constants are defined in - * The difference between there and here is that we do the - * res rounding and compute a 64-bit result (well so does that - * but it then throws away the high bits). - */ - *jiff = (mpy_l_X_l_ll(sec, SEC_CONVERSION) + - (mpy_l_X_l_ll(nsec, NSEC_CONVERSION) >> - (NSEC_JIFFIE_SC - SEC_JIFFIE_SC))) >> SEC_JIFFIE_SC; -} - -/* - * This function adjusts the timer as needed as a result of the clock - * being set. It should only be called for absolute timers, and then - * under the abs_list lock. It computes the time difference and sets - * the new jiffies value in the timer. It also updates the timers - * reference wall_to_monotonic value. It is complicated by the fact - * that tstojiffies() only handles positive times and it needs to work - * with both positive and negative times. Also, for negative offsets, - * we need to defeat the res round up. - * - * Return is true if there is a new time, else false. - */ -static long add_clockset_delta(struct k_itimer *timr, - struct timespec *new_wall_to) -{ - struct timespec delta; - int sign = 0; - u64 exp; - - set_normalized_timespec(&delta, - new_wall_to->tv_sec - - timr->it.real.wall_to_prev.tv_sec, - new_wall_to->tv_nsec - - timr->it.real.wall_to_prev.tv_nsec); - if (likely(!(delta.tv_sec | delta.tv_nsec))) - return 0; - if (delta.tv_sec < 0) { - set_normalized_timespec(&delta, - -delta.tv_sec, - 1 - delta.tv_nsec - - posix_clocks[timr->it_clock].res); - sign++; - } - tstojiffie(&delta, posix_clocks[timr->it_clock].res, &exp); - timr->it.real.wall_to_prev = *new_wall_to; - timr->it.real.timer.expires += (sign ? -exp : exp); - return 1; -} - -static void remove_from_abslist(struct k_itimer *timr) -{ - if (!list_empty(&timr->it.real.abs_timer_entry)) { - spin_lock(&abs_list.lock); - list_del_init(&timr->it.real.abs_timer_entry); - spin_unlock(&abs_list.lock); - } -} - static void schedule_next_timer(struct k_itimer *timr) { - struct timespec new_wall_to; - struct now_struct now; - unsigned long seq; - - /* - * Set up the timer for the next interval (if there is one). - * Note: this code uses the abs_timer_lock to protect - * it.real.wall_to_prev and must hold it until exp is set, not exactly - * obvious... - - * This function is used for CLOCK_REALTIME* and - * CLOCK_MONOTONIC* timers. If we ever want to handle other - * CLOCKs, the calling code (do_schedule_next_timer) would need - * to pull the "clock" info from the timer and dispatch the - * "other" CLOCKs "next timer" code (which, I suppose should - * also be added to the k_clock structure). - */ - if (!timr->it.real.incr) + if (ktime_cmp_val(timr->it.real.incr, ==, KTIME_ZERO)) return; - do { - seq = read_seqbegin(&xtime_lock); - new_wall_to = wall_to_monotonic; - posix_get_now(&now); - } while (read_seqretry(&xtime_lock, seq)); - - if (!list_empty(&timr->it.real.abs_timer_entry)) { - spin_lock(&abs_list.lock); - add_clockset_delta(timr, &new_wall_to); - - posix_bump_timer(timr, now); - - spin_unlock(&abs_list.lock); - } else { - posix_bump_timer(timr, now); - } - timr->it_overrun_last = timr->it_overrun; - timr->it_overrun = -1; + timr->it_overrun_last = timr->it.real.overrun; + timr->it.real.overrun = timr->it.real.timer.overrun = -1; ++timr->it_requeue_pending; - add_timer(&timr->it.real.timer); + ktimer_start(&timr->it.real.timer, &timr->it.real.incr, KTIMER_FORWARD); + timr->it.real.overrun = timr->it.real.timer.overrun; } /* @@ -413,14 +326,7 @@ int posix_timer_event(struct k_itimer *t { memset(&timr->sigq->info, 0, sizeof(siginfo_t)); timr->sigq->info.si_sys_private = si_private; - /* - * Send signal to the process that owns this timer. - - * This code assumes that all the possible abs_lists share the - * same lock (there is only one list at this time). If this is - * not the case, the CLOCK info would need to be used to find - * the proper abs list lock. - */ + /* Send signal to the process that owns this timer.*/ timr->sigq->info.si_signo = timr->it_sigev_signo; timr->sigq->info.si_errno = 0; @@ -454,65 +360,28 @@ EXPORT_SYMBOL_GPL(posix_timer_event); * This code is for CLOCK_REALTIME* and CLOCK_MONOTONIC* timers. */ -static void posix_timer_fn(unsigned long __data) +static void posix_timer_fn(void *data) { - struct k_itimer *timr = (struct k_itimer *) __data; + struct k_itimer *timr = data; unsigned long flags; - unsigned long seq; - struct timespec delta, new_wall_to; - u64 exp = 0; - int do_notify = 1; + int si_private = 0; spin_lock_irqsave(&timr->it_lock, flags); - if (!list_empty(&timr->it.real.abs_timer_entry)) { - spin_lock(&abs_list.lock); - do { - seq = read_seqbegin(&xtime_lock); - new_wall_to = wall_to_monotonic; - } while (read_seqretry(&xtime_lock, seq)); - set_normalized_timespec(&delta, - new_wall_to.tv_sec - - timr->it.real.wall_to_prev.tv_sec, - new_wall_to.tv_nsec - - timr->it.real.wall_to_prev.tv_nsec); - if (likely((delta.tv_sec | delta.tv_nsec ) == 0)) { - /* do nothing, timer is on time */ - } else if (delta.tv_sec < 0) { - /* do nothing, timer is already late */ - } else { - /* timer is early due to a clock set */ - tstojiffie(&delta, - posix_clocks[timr->it_clock].res, - &exp); - timr->it.real.wall_to_prev = new_wall_to; - timr->it.real.timer.expires += exp; - add_timer(&timr->it.real.timer); - do_notify = 0; - } - spin_unlock(&abs_list.lock); - } - if (do_notify) { - int si_private=0; + if (ktime_cmp_val(timr->it.real.incr, !=, KTIME_ZERO)) + si_private = ++timr->it_requeue_pending; - if (timr->it.real.incr) - si_private = ++timr->it_requeue_pending; - else { - remove_from_abslist(timr); - } + if (posix_timer_event(timr, si_private)) + /* + * signal was not sent because of sig_ignor + * we will not get a call back to restart it AND + * it should be restarted. + */ + schedule_next_timer(timr); - if (posix_timer_event(timr, si_private)) - /* - * signal was not sent because of sig_ignor - * we will not get a call back to restart it AND - * it should be restarted. - */ - schedule_next_timer(timr); - } unlock_timer(timr, flags); /* hold thru abs lock to keep irq off */ } - static inline struct task_struct * good_sigevent(sigevent_t * event) { struct task_struct *rtn = current->group_leader; @@ -776,39 +645,41 @@ static struct k_itimer * lock_timer(time static void common_timer_get(struct k_itimer *timr, struct itimerspec *cur_setting) { - unsigned long expires; - struct now_struct now; + ktime_t expires, now, remaining; + struct ktimer *timer = &timr->it.real.timer; - do - expires = timr->it.real.timer.expires; - while ((volatile long) (timr->it.real.timer.expires) != expires); - - posix_get_now(&now); - - if (expires && - ((timr->it_sigev_notify & ~SIGEV_THREAD_ID) == SIGEV_NONE) && - !timr->it.real.incr && - posix_time_before(&timr->it.real.timer, &now)) - timr->it.real.timer.expires = expires = 0; - if (expires) { - if (timr->it_requeue_pending & REQUEUE_PENDING || - (timr->it_sigev_notify & ~SIGEV_THREAD_ID) == SIGEV_NONE) { - posix_bump_timer(timr, now); - expires = timr->it.real.timer.expires; - } - else - if (!timer_pending(&timr->it.real.timer)) - expires = 0; - if (expires) - expires -= now.jiffies; - } - jiffies_to_timespec(expires, &cur_setting->it_value); - jiffies_to_timespec(timr->it.real.incr, &cur_setting->it_interval); - - if (cur_setting->it_value.tv_sec < 0) { + memset(cur_setting, 0, sizeof(struct itimerspec)); + expires = ktimer_get_expiry(timer, &now); + remaining = ktime_sub(expires, now); + + /* Time left ? or timer pending */ + if (ktime_cmp_val(remaining, >, KTIME_ZERO) || ktimer_active(timer)) + goto calci; + /* interval timer ? */ + if (ktime_cmp_val(timr->it.real.incr, ==, 0)) + return; + /* + * When a requeue is pending or this is a SIGEV_NONE timer + * move the expiry time forward by intervals, so expiry is > + * now. + * The active (non SIGEV_NONE) rearm should be done + * automatically by the ktimer REARM mode. Thats the next + * iteration. The REQUEUE_PENDING part will go away ! + */ + if (timr->it_requeue_pending & REQUEUE_PENDING || + (timr->it_sigev_notify & ~SIGEV_THREAD_ID) == SIGEV_NONE) { + remaining = forward_posix_timer(timr, now); + } + calci: + /* interval timer ? */ + if (ktime_cmp_val(timr->it.real.incr, !=, KTIME_ZERO)) + ktime_to_timespec(&cur_setting->it_interval, + timr->it.real.incr); + /* Return 0 only, when the timer is expired and not pending */ + if (ktime_cmp_val(remaining, <=, KTIME_ZERO)) cur_setting->it_value.tv_nsec = 1; - cur_setting->it_value.tv_sec = 0; - } + else + ktime_to_timespec(&cur_setting->it_value, remaining); } /* Get the time remaining on a POSIX.1b interval timer. */ @@ -832,6 +703,7 @@ sys_timer_gettime(timer_t timer_id, stru return 0; } + /* * Get the number of overruns of a POSIX.1b interval timer. This is to * be the overrun of the timer last delivered. At the same time we are @@ -858,84 +730,6 @@ sys_timer_getoverrun(timer_t timer_id) return overrun; } -/* - * Adjust for absolute time - * - * If absolute time is given and it is not CLOCK_MONOTONIC, we need to - * adjust for the offset between the timer clock (CLOCK_MONOTONIC) and - * what ever clock he is using. - * - * If it is relative time, we need to add the current (CLOCK_MONOTONIC) - * time to it to get the proper time for the timer. - */ -static int adjust_abs_time(struct k_clock *clock, struct timespec *tp, - int abs, u64 *exp, struct timespec *wall_to) -{ - struct timespec now; - struct timespec oc = *tp; - u64 jiffies_64_f; - int rtn =0; - - if (abs) { - /* - * The mask pick up the 4 basic clocks - */ - if (!((clock - &posix_clocks[0]) & ~CLOCKS_MASK)) { - jiffies_64_f = do_posix_clock_monotonic_gettime_parts( - &now, wall_to); - /* - * If we are doing a MONOTONIC clock - */ - if((clock - &posix_clocks[0]) & CLOCKS_MONO){ - now.tv_sec += wall_to->tv_sec; - now.tv_nsec += wall_to->tv_nsec; - } - } else { - /* - * Not one of the basic clocks - */ - clock->clock_get(clock - posix_clocks, &now); - jiffies_64_f = get_jiffies_64(); - } - /* - * Take away now to get delta and normalize - */ - set_normalized_timespec(&oc, oc.tv_sec - now.tv_sec, - oc.tv_nsec - now.tv_nsec); - }else{ - jiffies_64_f = get_jiffies_64(); - } - /* - * Check if the requested time is prior to now (if so set now) - */ - if (oc.tv_sec < 0) - oc.tv_sec = oc.tv_nsec = 0; - - if (oc.tv_sec | oc.tv_nsec) - set_normalized_timespec(&oc, oc.tv_sec, - oc.tv_nsec + clock->res); - tstojiffie(&oc, clock->res, exp); - - /* - * Check if the requested time is more than the timer code - * can handle (if so we error out but return the value too). - */ - if (*exp > ((u64)MAX_JIFFY_OFFSET)) - /* - * This is a considered response, not exactly in - * line with the standard (in fact it is silent on - * possible overflows). We assume such a large - * value is ALMOST always a programming error and - * try not to compound it by setting a really dumb - * value. - */ - rtn = -EINVAL; - /* - * return the actual jiffies expire time, full 64 bits - */ - *exp += jiffies_64_f; - return rtn; -} /* Set a POSIX.1b interval timer. */ /* timr->it_lock is taken. */ @@ -943,68 +737,52 @@ static inline int common_timer_set(struct k_itimer *timr, int flags, struct itimerspec *new_setting, struct itimerspec *old_setting) { - struct k_clock *clock = &posix_clocks[timr->it_clock]; - u64 expire_64; + ktime_t expires; + int mode; if (old_setting) common_timer_get(timr, old_setting); /* disable the timer */ - timr->it.real.incr = 0; + ktime_set_scalar(timr->it.real.incr, KTIME_ZERO); /* * careful here. If smp we could be in the "fire" routine which will * be spinning as we hold the lock. But this is ONLY an SMP issue. */ - if (try_to_del_timer_sync(&timr->it.real.timer) < 0) { -#ifdef CONFIG_SMP - /* - * It can only be active if on an other cpu. Since - * we have cleared the interval stuff above, it should - * clear once we release the spin lock. Of course once - * we do that anything could happen, including the - * complete melt down of the timer. So return with - * a "retry" exit status. - */ + if (ktimer_try_to_cancel(&timr->it.real.timer) < 0) return TIMER_RETRY; -#endif - } - - remove_from_abslist(timr); timr->it_requeue_pending = (timr->it_requeue_pending + 2) & ~REQUEUE_PENDING; timr->it_overrun_last = 0; timr->it_overrun = -1; - /* - *switch off the timer when it_value is zero - */ - if (!new_setting->it_value.tv_sec && !new_setting->it_value.tv_nsec) { - timr->it.real.timer.expires = 0; + + /* switch off the timer when it_value is zero */ + if (!new_setting->it_value.tv_sec && !new_setting->it_value.tv_nsec) return 0; - } - if (adjust_abs_time(clock, - &new_setting->it_value, flags & TIMER_ABSTIME, - &expire_64, &(timr->it.real.wall_to_prev))) { - return -EINVAL; - } - timr->it.real.timer.expires = (unsigned long)expire_64; - tstojiffie(&new_setting->it_interval, clock->res, &expire_64); - timr->it.real.incr = (unsigned long)expire_64; + mode = flags & TIMER_ABSTIME ? KTIMER_ABS : KTIMER_REL; - /* - * We do not even queue SIGEV_NONE timers! But we do put them - * in the abs list so we can do that right. + /* Posix madness. Only absolute CLOCK_REALTIME timers + * are affected by clock sets. So we must reiniatilize + * the timer. */ + if (timr->it_clock == CLOCK_REALTIME && mode == KTIMER_ABS) + timer_create_real(timr); + else + timer_create_mono(timr); + + expires = timespec_to_ktime(new_setting->it_value); + + /* Convert and round the interval */ + timr->it.real.incr = ktimer_round_timespec(&timr->it.real.timer, + &new_setting->it_interval); + + /* SIGEV_NONE timers are not queued ! See common_timer_get */ if (((timr->it_sigev_notify & ~SIGEV_THREAD_ID) != SIGEV_NONE)) - add_timer(&timr->it.real.timer); + ktimer_start(&timr->it.real.timer, &expires, + mode | KTIMER_NOCHECK | KTIMER_ROUND); - if (flags & TIMER_ABSTIME && clock->abs_struct) { - spin_lock(&clock->abs_struct->lock); - list_add_tail(&(timr->it.real.abs_timer_entry), - &(clock->abs_struct->list)); - spin_unlock(&clock->abs_struct->lock); - } return 0; } @@ -1039,6 +817,7 @@ retry: unlock_timer(timr, flag); if (error == TIMER_RETRY) { + wait_for_ktimer(&timr->it.real.timer); rtn = NULL; // We already got the old time... goto retry; } @@ -1052,24 +831,10 @@ retry: static inline int common_timer_del(struct k_itimer *timer) { - timer->it.real.incr = 0; + ktime_set_scalar(timer->it.real.incr, KTIME_ZERO); - if (try_to_del_timer_sync(&timer->it.real.timer) < 0) { -#ifdef CONFIG_SMP - /* - * It can only be active if on an other cpu. Since - * we have cleared the interval stuff above, it should - * clear once we release the spin lock. Of course once - * we do that anything could happen, including the - * complete melt down of the timer. So return with - * a "retry" exit status. - */ + if (ktimer_try_to_cancel(&timer->it.real.timer) < 0) return TIMER_RETRY; -#endif - } - - remove_from_abslist(timer); - return 0; } @@ -1085,24 +850,17 @@ sys_timer_delete(timer_t timer_id) struct k_itimer *timer; long flags; -#ifdef CONFIG_SMP - int error; retry_delete: -#endif timer = lock_timer(timer_id, &flags); if (!timer) return -EINVAL; -#ifdef CONFIG_SMP - error = timer_delete_hook(timer); - - if (error == TIMER_RETRY) { + if (timer_delete_hook(timer) == TIMER_RETRY) { unlock_timer(timer, flags); + wait_for_ktimer(&timer->it.real.timer); goto retry_delete; } -#else - timer_delete_hook(timer); -#endif + spin_lock(¤t->sighand->siglock); list_del(&timer->list); spin_unlock(¤t->sighand->siglock); @@ -1119,6 +877,7 @@ retry_delete: release_posix_timer(timer, IT_ID_SET); return 0; } + /* * return timer owned by the process, used by exit_itimers */ @@ -1126,22 +885,14 @@ static inline void itimer_delete(struct { unsigned long flags; -#ifdef CONFIG_SMP - int error; retry_delete: -#endif spin_lock_irqsave(&timer->it_lock, flags); -#ifdef CONFIG_SMP - error = timer_delete_hook(timer); - - if (error == TIMER_RETRY) { + if (timer_delete_hook(timer) == TIMER_RETRY) { unlock_timer(timer, flags); + wait_for_ktimer(&timer->it.real.timer); goto retry_delete; } -#else - timer_delete_hook(timer); -#endif list_del(&timer->list); /* * This keeps any tasks waiting on the spin lock from thinking @@ -1170,56 +921,7 @@ void exit_itimers(struct signal_struct * } } -/* - * And now for the "clock" calls - * - * These functions are called both from timer functions (with the timer - * spin_lock_irq() held and from clock calls with no locking. They must - * use the save flags versions of locks. - */ - -/* - * We do ticks here to avoid the irq lock ( they take sooo long). - * The seqlock is great here. Since we a reader, we don't really care - * if we are interrupted since we don't take lock that will stall us or - * any other cpu. Voila, no irq lock is needed. - * - */ - -static u64 do_posix_clock_monotonic_gettime_parts( - struct timespec *tp, struct timespec *mo) -{ - u64 jiff; - unsigned int seq; - - do { - seq = read_seqbegin(&xtime_lock); - getnstimeofday(tp); - *mo = wall_to_monotonic; - jiff = jiffies_64; - - } while(read_seqretry(&xtime_lock, seq)); - - return jiff; -} - -static int do_posix_clock_monotonic_get(clockid_t clock, struct timespec *tp) -{ - struct timespec wall_to_mono; - - do_posix_clock_monotonic_gettime_parts(tp, &wall_to_mono); - - set_normalized_timespec(tp, tp->tv_sec + wall_to_mono.tv_sec, - tp->tv_nsec + wall_to_mono.tv_nsec); - - return 0; -} - -int do_posix_clock_monotonic_gettime(struct timespec *tp) -{ - return do_posix_clock_monotonic_get(CLOCK_MONOTONIC, tp); -} - +/* Not available / possible... functions */ int do_posix_clock_nosettime(clockid_t clockid, struct timespec *tp) { return -EINVAL; @@ -1232,7 +934,8 @@ int do_posix_clock_notimer_create(struct } EXPORT_SYMBOL_GPL(do_posix_clock_notimer_create); -int do_posix_clock_nonanosleep(clockid_t clock, int flags, struct timespec *t) +int do_posix_clock_nonanosleep(clockid_t clock, int flags, struct timespec *t, + struct timespec __user *r) { #ifndef ENOTSUP return -EOPNOTSUPP; /* aka ENOTSUP in userland for POSIX */ @@ -1292,117 +995,33 @@ sys_clock_getres(clockid_t which_clock, } /* - * The standard says that an absolute nanosleep call MUST wake up at - * the requested time in spite of clock settings. Here is what we do: - * For each nanosleep call that needs it (only absolute and not on - * CLOCK_MONOTONIC* (as it can not be set)) we thread a little structure - * into the "nanosleep_abs_list". All we need is the task_struct pointer. - * When ever the clock is set we just wake up all those tasks. The rest - * is done by the while loop in clock_nanosleep(). - * - * On locking, clock_was_set() is called from update_wall_clock which - * holds (or has held for it) a write_lock_irq( xtime_lock) and is - * called from the timer bh code. Thus we need the irq save locks. - * - * Also, on the call from update_wall_clock, that is done as part of a - * softirq thing. We don't want to delay the system that much (possibly - * long list of timers to fix), so we defer that work to keventd. + * nanosleep for monotonic and realtime clocks */ - -static DECLARE_WAIT_QUEUE_HEAD(nanosleep_abs_wqueue); -static DECLARE_WORK(clock_was_set_work, (void(*)(void*))clock_was_set, NULL); - -static DECLARE_MUTEX(clock_was_set_lock); - -void clock_was_set(void) +static int common_nsleep(clockid_t which_clock, int flags, + struct timespec *tsave, struct timespec __user *rmtp) { - struct k_itimer *timr; - struct timespec new_wall_to; - LIST_HEAD(cws_list); - unsigned long seq; + int mode = flags & TIMER_ABSTIME ? KTIMER_ABS : KTIMER_REL; - - if (unlikely(in_interrupt())) { - schedule_work(&clock_was_set_work); - return; + switch (which_clock) { + case CLOCK_REALTIME: + /* Posix madness. Only absolute timers on clock realtime + are affected by clock set. */ + if (mode == KTIMER_ABS) + return ktimer_nanosleep_real(tsave, rmtp, mode); + case CLOCK_MONOTONIC: + return ktimer_nanosleep(tsave, rmtp, mode); + default: + break; } - wake_up_all(&nanosleep_abs_wqueue); - - /* - * Check if there exist TIMER_ABSTIME timers to correct. - * - * Notes on locking: This code is run in task context with irq - * on. We CAN be interrupted! All other usage of the abs list - * lock is under the timer lock which holds the irq lock as - * well. We REALLY don't want to scan the whole list with the - * interrupt system off, AND we would like a sequence lock on - * this code as well. Since we assume that the clock will not - * be set often, it seems ok to take and release the irq lock - * for each timer. In fact add_timer will do this, so this is - * not an issue. So we know when we are done, we will move the - * whole list to a new location. Then as we process each entry, - * we will move it to the actual list again. This way, when our - * copy is empty, we are done. We are not all that concerned - * about preemption so we will use a semaphore lock to protect - * aginst reentry. This way we will not stall another - * processor. It is possible that this may delay some timers - * that should have expired, given the new clock, but even this - * will be minimal as we will always update to the current time, - * even if it was set by a task that is waiting for entry to - * this code. Timers that expire too early will be caught by - * the expire code and restarted. - - * Absolute timers that repeat are left in the abs list while - * waiting for the task to pick up the signal. This means we - * may find timers that are not in the "add_timer" list, but are - * in the abs list. We do the same thing for these, save - * putting them back in the "add_timer" list. (Note, these are - * left in the abs list mainly to indicate that they are - * ABSOLUTE timers, a fact that is used by the re-arm code, and - * for which we have no other flag.) - - */ - - down(&clock_was_set_lock); - spin_lock_irq(&abs_list.lock); - list_splice_init(&abs_list.list, &cws_list); - spin_unlock_irq(&abs_list.lock); - do { - do { - seq = read_seqbegin(&xtime_lock); - new_wall_to = wall_to_monotonic; - } while (read_seqretry(&xtime_lock, seq)); - - spin_lock_irq(&abs_list.lock); - if (list_empty(&cws_list)) { - spin_unlock_irq(&abs_list.lock); - break; - } - timr = list_entry(cws_list.next, struct k_itimer, - it.real.abs_timer_entry); - - list_del_init(&timr->it.real.abs_timer_entry); - if (add_clockset_delta(timr, &new_wall_to) && - del_timer(&timr->it.real.timer)) /* timer run yet? */ - add_timer(&timr->it.real.timer); - list_add(&timr->it.real.abs_timer_entry, &abs_list.list); - spin_unlock_irq(&abs_list.lock); - } while (1); - - up(&clock_was_set_lock); + return -EINVAL; } -long clock_nanosleep_restart(struct restart_block *restart_block); - asmlinkage long sys_clock_nanosleep(clockid_t which_clock, int flags, const struct timespec __user *rqtp, struct timespec __user *rmtp) { struct timespec t; - struct restart_block *restart_block = - &(current_thread_info()->restart_block); - int ret; if (invalid_clockid(which_clock)) return -EINVAL; @@ -1410,125 +1029,8 @@ sys_clock_nanosleep(clockid_t which_cloc if (copy_from_user(&t, rqtp, sizeof (struct timespec))) return -EFAULT; - if ((unsigned) t.tv_nsec >= NSEC_PER_SEC || t.tv_sec < 0) + if (!timespec_valid(&t)) return -EINVAL; - /* - * Do this here as nsleep function does not have the real address. - */ - restart_block->arg1 = (unsigned long)rmtp; - - ret = CLOCK_DISPATCH(which_clock, nsleep, (which_clock, flags, &t)); - - if ((ret == -ERESTART_RESTARTBLOCK) && rmtp && - copy_to_user(rmtp, &t, sizeof (t))) - return -EFAULT; - return ret; -} - - -static int common_nsleep(clockid_t which_clock, - int flags, struct timespec *tsave) -{ - struct timespec t, dum; - DECLARE_WAITQUEUE(abs_wqueue, current); - u64 rq_time = (u64)0; - s64 left; - int abs; - struct restart_block *restart_block = - ¤t_thread_info()->restart_block; - - abs_wqueue.flags = 0; - abs = flags & TIMER_ABSTIME; - - if (restart_block->fn == clock_nanosleep_restart) { - /* - * Interrupted by a non-delivered signal, pick up remaining - * time and continue. Remaining time is in arg2 & 3. - */ - restart_block->fn = do_no_restart_syscall; - - rq_time = restart_block->arg3; - rq_time = (rq_time << 32) + restart_block->arg2; - if (!rq_time) - return -EINTR; - left = rq_time - get_jiffies_64(); - if (left <= (s64)0) - return 0; /* Already passed */ - } - - if (abs && (posix_clocks[which_clock].clock_get != - posix_clocks[CLOCK_MONOTONIC].clock_get)) - add_wait_queue(&nanosleep_abs_wqueue, &abs_wqueue); - - do { - t = *tsave; - if (abs || !rq_time) { - adjust_abs_time(&posix_clocks[which_clock], &t, abs, - &rq_time, &dum); - } - - left = rq_time - get_jiffies_64(); - if (left >= (s64)MAX_JIFFY_OFFSET) - left = (s64)MAX_JIFFY_OFFSET; - if (left < (s64)0) - break; - - schedule_timeout_interruptible(left); - - left = rq_time - get_jiffies_64(); - } while (left > (s64)0 && !test_thread_flag(TIF_SIGPENDING)); - - if (abs_wqueue.task_list.next) - finish_wait(&nanosleep_abs_wqueue, &abs_wqueue); - - if (left > (s64)0) { - - /* - * Always restart abs calls from scratch to pick up any - * clock shifting that happened while we are away. - */ - if (abs) - return -ERESTARTNOHAND; - - left *= TICK_NSEC; - tsave->tv_sec = div_long_long_rem(left, - NSEC_PER_SEC, - &tsave->tv_nsec); - /* - * Restart works by saving the time remaing in - * arg2 & 3 (it is 64-bits of jiffies). The other - * info we need is the clock_id (saved in arg0). - * The sys_call interface needs the users - * timespec return address which _it_ saves in arg1. - * Since we have cast the nanosleep call to a clock_nanosleep - * both can be restarted with the same code. - */ - restart_block->fn = clock_nanosleep_restart; - restart_block->arg0 = which_clock; - /* - * Caller sets arg1 - */ - restart_block->arg2 = rq_time & 0xffffffffLL; - restart_block->arg3 = rq_time >> 32; - - return -ERESTART_RESTARTBLOCK; - } - - return 0; -} -/* - * This will restart clock_nanosleep. - */ -long -clock_nanosleep_restart(struct restart_block *restart_block) -{ - struct timespec t; - int ret = common_nsleep(restart_block->arg0, 0, &t); - - if ((ret == -ERESTART_RESTARTBLOCK) && restart_block->arg1 && - copy_to_user((struct timespec __user *)(restart_block->arg1), &t, - sizeof (t))) - return -EFAULT; - return ret; + return CLOCK_DISPATCH(which_clock, nsleep, (which_clock, flags, &t, rmtp)); } diff -puN kernel/timer.c~ktimers-kt2 kernel/timer.c --- devel/kernel/timer.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/timer.c 2005-11-12 14:08:45.000000000 -0800 @@ -858,6 +858,7 @@ static void run_timer_softirq(struct sof { tvec_base_t *base = &__get_cpu_var(tvec_bases); + ktimer_run_queues(); if (time_after_eq(jiffies, base->timer_jiffies)) __run_timers(base); } @@ -1119,62 +1120,6 @@ asmlinkage long sys_gettid(void) return current->pid; } -static long __sched nanosleep_restart(struct restart_block *restart) -{ - unsigned long expire = restart->arg0, now = jiffies; - struct timespec __user *rmtp = (struct timespec __user *) restart->arg1; - long ret; - - /* Did it expire while we handled signals? */ - if (!time_after(expire, now)) - return 0; - - expire = schedule_timeout_interruptible(expire - now); - - ret = 0; - if (expire) { - struct timespec t; - jiffies_to_timespec(expire, &t); - - ret = -ERESTART_RESTARTBLOCK; - if (rmtp && copy_to_user(rmtp, &t, sizeof(t))) - ret = -EFAULT; - /* The 'restart' block is already filled in */ - } - return ret; -} - -asmlinkage long sys_nanosleep(struct timespec __user *rqtp, struct timespec __user *rmtp) -{ - struct timespec t; - unsigned long expire; - long ret; - - if (copy_from_user(&t, rqtp, sizeof(t))) - return -EFAULT; - - if ((t.tv_nsec >= 1000000000L) || (t.tv_nsec < 0) || (t.tv_sec < 0)) - return -EINVAL; - - expire = timespec_to_jiffies(&t) + (t.tv_sec || t.tv_nsec); - expire = schedule_timeout_interruptible(expire); - - ret = 0; - if (expire) { - struct restart_block *restart; - jiffies_to_timespec(expire, &t); - if (rmtp && copy_to_user(rmtp, &t, sizeof(t))) - return -EFAULT; - - restart = ¤t_thread_info()->restart_block; - restart->fn = nanosleep_restart; - restart->arg0 = jiffies + expire; - restart->arg1 = (unsigned long) rmtp; - ret = -ERESTART_RESTARTBLOCK; - } - return ret; -} - /* * sys_sysinfo - fill in sysinfo struct */ diff -puN arch/um/kernel/time.c~ktimers-kt2 arch/um/kernel/time.c --- devel/arch/um/kernel/time.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/arch/um/kernel/time.c 2005-11-12 14:08:45.000000000 -0800 @@ -114,8 +114,8 @@ void time_init(void) wall_to_monotonic.tv_nsec = -now.tv_nsec; } -/* Declared in linux/time.h, which can't be included here */ -extern void clock_was_set(void); +/* Defined in linux/ktimer.h, which can't be included here */ +#define clock_was_set() do { } while (0) void do_gettimeofday(struct timeval *tv) { diff -puN kernel/time.c~ktimers-kt2 kernel/time.c --- devel/kernel/time.c~ktimers-kt2 2005-11-12 14:08:45.000000000 -0800 +++ devel-akpm/kernel/time.c 2005-11-12 14:08:45.000000000 -0800 @@ -561,6 +561,103 @@ void getnstimeofday(struct timespec *tv) EXPORT_SYMBOL_GPL(getnstimeofday); #endif +/* Converts Gregorian date to seconds since 1970-01-01 00:00:00. + * Assumes input in normal date format, i.e. 1980-12-31 23:59:59 + * => year=1980, mon=12, day=31, hour=23, min=59, sec=59. + * + * [For the Julian calendar (which was used in Russia before 1917, + * Britain & colonies before 1752, anywhere else before 1582, + * and is still in use by some communities) leave out the + * -year/100+year/400 terms, and add 10.] + * + * This algorithm was first published by Gauss (I think). + * + * WARNING: this function will overflow on 2106-02-07 06:28:16 on + * machines were long is 32-bit! (However, as time_t is signed, we + * will already get problems at other places on 2038-01-19 03:14:08) + */ +unsigned long +mktime (unsigned int year, unsigned int mon, + unsigned int day, unsigned int hour, + unsigned int min, unsigned int sec) +{ + if (0 >= (int) (mon -= 2)) { /* 1..12 -> 11,12,1..10 */ + mon += 12; /* Puts Feb last since it has leap day */ + year -= 1; + } + + return ((((unsigned long) + (year/4 - year/100 + year/400 + 367*mon/12 + day) + + year*365 - 719499 + )*24 + hour /* now have hours */ + )*60 + min /* now have minutes */ + )*60 + sec; /* finally seconds */ +} + +/** + * set_normalized_timespec - set timespec sec and nsec parts and normalize + * + * @ts: pointer to timespec variable to be set + * @sec: seconds to set + * @nsec: nanoseconds to set + * + * Set seconds and nanoseconds field of a timespec variable and + * normalize to the timespec storage format + * + * Note: The tv_nsec part is always in the range of + * 0 <= tv_nsec < NSEC_PER_SEC + * For negative values only the tv_sec field is negative ! + */ +void set_normalized_timespec (struct timespec *ts, time_t sec, long nsec) +{ + while (nsec > NSEC_PER_SEC) { + nsec -= NSEC_PER_SEC; + ++sec; + } + while (nsec < 0) { + nsec += NSEC_PER_SEC; + --sec; + } + ts->tv_sec = sec; + ts->tv_nsec = nsec; +} + +/** + * ns_to_timespec - Convert nanoseconds to timespec + * + * @ts: pointer to timespec variable to store result + * @nsec: nanoseconds value to be converted + * + * Stores the timespec representation of the nanoseconds value in + * the timespec variable pointed to by @ts + */ +void ns_to_timespec(struct timespec *ts, nsec_t nsec) +{ + if (nsec) + ts->tv_sec = div_long_long_rem_signed(nsec, NSEC_PER_SEC, + &ts->tv_nsec); + else + ts->tv_sec = ts->tv_nsec = 0; +} + +/** + * ns_to_timeval - Convert nanoseconds to timeval + * + * @tv: pointer to timeval variable to store result + * @nsec: nanoseconds value to be converted + * + * Stores the timeval representation of the nanoseconds value in + * the timeval variable pointed to by @tv + */ +void ns_to_timeval(struct timeval *tv, nsec_t nsec) +{ + struct timespec ts; + + ns_to_timespec(&ts, nsec); + tv->tv_sec = ts.tv_sec; + tv->tv_usec = (suseconds_t) ts.tv_nsec / 1000; +} + #if (BITS_PER_LONG < 64) u64 get_jiffies_64(void) { _