Kevin’s parallel article on the topic may be found here.

Please review the following before posting comments to this blog:
http://www.logicalfallacies.info/relevance/ad-hominem/

It is simply illogical to advance that kernel mode CPU usage is very easy to understand, and therefore suggest that the author’s knowledge is inferior to that of a beginner.

What an author shares in a book does not necessarily communicate the depth of knowledge that an author has about a particular sub-topic in the book. Some of the sections of the book appear as though the author planned to circle back to fill in the limited details provided in some of the paragraphs, while other sections show deep knowledge.

To be absolutely clear, I believe that this book’s author is very knowledgable about Oracle Databases, and demonstrates that repeatedly in the book. He mentions in the book that he worked with Cary Millsap at Oracle Corp (I believe in the same department), and that likely means he has at least 20 years of in-depth knowledge in the functioning of Oracle Databases. The author’s “Forecasting Oracle Performance” book did not appear to have the same pattern of drifting between the internals of modifying the hidden Oracle parameters in one page and then a couple of pages later to simply glossing over problems – essentially stating that “there is nothing the DBA can do when this happens.”

The kernel mode concept is very easy to understand, even, for the beginners too. do you think the author is not aware of it ?

You had enforced to find something wrong. It looks there is a wording game here.

Nice article. Just a couple of quotes from the article:

“Most operations interacting with the system require permissions not available to a user level process, e.g. I/O performed with a device present on the system, or any form of communication with other processes requires the use of system calls… On Unix, Unix-like and other POSIX-compatible operating systems, popular system calls are open, read, write, close, wait, exec, fork, exit, and kill. Many of today’s operating systems have hundreds of system calls. For example, Linux has 319 different calls, and FreeBSD has almost 500. Tools such as strace and truss allow a process to execute from start and report all system calls the process invokes, or can attach to an already running process and intercept any system call made by said process if the operation does not violate the permissions of the user.”

A quote from the same “Oracle Performance Firefighting” book, in this blog article http://hoopercharles.wordpress.com/2010/06/13/db-file-scattered-read-wait-event-what-is-wrong-with-this-quote/ indicates that the book author used strace to demonstrate that Oracle database used readv to read multiple blocks from disk, which the author described as being associated with a db file scattered read wait event. I believe that readv is considered a system call, and I would assume that call would motivate at least some CPU usage in kernel mode. To state that essentially any activity not directly related to a user task is considered kernel CPU time might be considered “missing the boat”.

Page 200 of the Expert Oracle Practices book shows interesting output that is related to this blog article. The output shows output from 64 bit Windows where a 10.2.0.4 database instance was consuming (roughly per second):

User Time: 0.31S    Sys Time: 0.75S
User Time: 0.24S    Sys Time: 0.86S
User Time: 0.22S    Sys Time: 0.84S

That is roughly 80% of the CPU time consumed by the database instance in kernel mode. What happened during that time period? The instance was running a SQL statement that was performing 100% (or very close to it) logical I/O, requiring over an hour at 100% CPU utilization (100% of 1 CPU) to output the first row. The troubling part? In the next chapter this was the same SQL statement that was fixed in the next chapter with the fix results summary information appearing on page 313. This query that started at 1 hour, 6 minutes, and 44 seconds completed in a low of 2.65 seconds just by trying to help the Oracle optimizer understand the intent of the query, and was further dropped to 0.28 seconds by switching from STATISTICS_LEVEL=’ALL’ to STATISTICS_LEVEL=’TYPICAL’

In short, if you just throw up your hands when you see excessive kernel mode CPU usage, thinking that it is just like taxes, completely out of your control, you might be missing an opportunity.

My notes suggest that the following activities that needed to be performed for an Oracle database instance require kernel mode CPU time:
* network I/O
* disk I/O (especially with buffered file systems)
* timer calls (typically gettimeofday calls)
* memory access requests
* memory management
* process creation and termination (think session creation and termination)

It looks like my analysis seems to agree with what you stated above.

Any other discussion?

As far as I know system time is calculated when kernel is doing any system call – http://en.wikipedia.org/wiki/System_call
In that case if software is using any kernel interfaces (disk, network, memory allocation, etc) a system time is calculated and it
is related to program activity. Can it be named pure overhead ? I’m not sure but I’m open to discuss about it.

regards,
Marcin