December | 2009 | Charles Hooper's Oracle Notes

Hard Parses when Using Bind Variables?

31 12 2009

December 31, 2009

Assume that the following tables are created and then statistics are gathered:

CREATE TABLE T3 AS
SELECT
  ROWNUM C1,
  LPAD('A',100,'A') C2
FROM
  DUAL
CONNECT BY
  LEVEL<=10000;

CREATE TABLE T4 AS
SELECT
  ROWNUM C1,
  LPAD('A',100,'A') C2
FROM
  DUAL
CONNECT BY
  LEVEL<=10000;

CREATE INDEX IND_T4 ON T4(C1);

EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T3',CASCADE=>TRUE)
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T4',CASCADE=>TRUE)

If you then set up SQL*Plus with the following commands:

VARIABLE N1 NUMBER
VARIABLE N2 NUMBER
SET AUTOTRACE TRACEONLY STATISTICS
SET ARRAYSIZE 1000

How many hard parses would you see for the session, and how many child cursors for the SQL statement will be in the library cache, if you do the following in SQL*Plus:

EXEC :N1 := 1
EXEC :N2 := 1
SELECT T3.C1, T4.C2 FROM T3, T4 WHERE T3.C1 BETWEEN :N1 AND :N2 AND T3.C1=T4.C1;
SELECT T3.C1, T4.C2 FROM T3, T4 WHERE T3.C1 BETWEEN :N1 AND :N2 AND T3.C1=T4.C1;
SELECT T3.C1, T4.C2 FROM T3, T4 WHERE T3.C1 BETWEEN :N1 AND :N2 AND T3.C1=T4.C1;
SELECT T3.C1, T4.C2 FROM T3, T4 WHERE T3.C1 BETWEEN :N1 AND :N2 AND T3.C1=T4.C1;

Now, repeat the above 9,999 times in the same session, specifying random values for N1 and N2 such that:

N1 >= 1
N2 <= 10,000
N1 <= N2

Does it matter if you test with Oracle 8.0.5 (assuming that you use the ANALYZE command rather than DBMS_STATS), 11.2.0.1, or something in between?

Comments : Leave a Comment »

Categories : Bind Variable, Performance

Excel – The Graphical Master of Oracle Foreign Keys

30 12 2009

December 30, 2009

Foreign keys… those practical rules that help maintain data integrity between parent and child tables, allow for table eliminations in queries beginning with Oracle release 10.2.0.1, and provide the potential for the optimizer to generate additional predicates during query optimization for better cardinality estimates.

Those benefits all sound like they might be helpful to someone, but what good are they for the average person (I mean the average developer)? Foreign keys are great for generating abstract art (note that I am a poor judge of art), with a bunch of odd writing all over the place.

This example demonstrates how to query an Excel spreadsheet as if it were a database. The results of the queries are used to build a graphical model of the relationships between the tables based on the foreign key relationships that are established in the database. Excel 2003 limits text boxes to no more than 255 characters, so use Excel 2007 or later for this example, if possible.

First, we need to build the source Excel spreadsheet that will act as the database that will be queried by the second Excel spreadsheet. The source Excel spreadsheet will have two worksheets (tabs) named “Data Dict Tables” and “Data Dict Foreign Keys”. Use the Microsoft Query Tool, or another approach to bring in the data from the Oracle data dictionary. The queries for each of those worksheets follows:

Data Dict Tables

SELECT
  DT.OWNER,
  DT.TABLE_NAME,
  DTC.COLUMN_NAME,
  DTC.DATA_TYPE,
  DTC.DATA_LENGTH,
  DTC.DATA_PRECISION,
  DTC.DATA_SCALE,
  DTC.NULLABLE,
  DTC.COLUMN_ID,
  DT.TABLESPACE_NAME,
  DTCC.COMMENTS TABLE_COMMENTS,
  SUBSTR(DCC.COMMENTS,1,255) COLUMN_COMMENTS
FROM
  DBA_TABLES DT,
  DBA_TAB_COLUMNS DTC,
  DBA_TAB_COMMENTS DTCC,
  DBA_COL_COMMENTS DCC
WHERE
  DT.OWNER=DTC.OWNER
  AND DT.TABLE_NAME=DTC.TABLE_NAME
  AND DT.OWNER=DTCC.OWNER(+)
  AND DT.TABLE_NAME=DTCC.TABLE_NAME(+)
  AND DTC.OWNER=DCC.OWNER(+)
  AND DTC.TABLE_NAME=DCC.TABLE_NAME(+)
  AND DTC.COLUMN_NAME=DCC.COLUMN_NAME(+)
ORDER BY
  DT.OWNER,
  DT.TABLE_NAME,
  DTC.COLUMN_ID;

—

Data Dict Foreign Keys

SELECT
  DC2.OWNER,
  DC2.CONSTRAINT_NAME PKEY_CONTRAINT,
  DC1.CONSTRAINT_NAME FKEY_CONTRAINT,
  DCC2.TABLE_NAME PKEY_TABLE_NAME,
  DCC2.COLUMN_NAME PKEY_COLUMN_NAME,
  DCC1.TABLE_NAME FKEY_TABLE_NAME,
  DCC1.COLUMN_NAME FKEY_COLUMN_NAME,
  DCC1.POSITION
FROM
  DBA_CONSTRAINTS DC1,
  DBA_CONSTRAINTS DC2,
  DBA_CONS_COLUMNS DCC1,
  DBA_CONS_COLUMNS DCC2
WHERE
  DC1.CONSTRAINT_TYPE='R'
  AND DC1.R_OWNER=DC2.OWNER
  AND DC1.R_CONSTRAINT_NAME=DC2.CONSTRAINT_NAME
  AND DC1.OWNER=DCC1.OWNER
  AND DC1.CONSTRAINT_NAME=DCC1.CONSTRAINT_NAME
  AND DC2.OWNER=DCC2.OWNER
  AND DC2.CONSTRAINT_NAME=DCC2.CONSTRAINT_NAME
  AND DCC1.POSITION=DCC2.POSITION
  AND DC1.OWNER='SYSADM'
ORDER BY
  DC2.OWNER,
  DCC2.TABLE_NAME,
  DCC1.TABLE_NAME,
  DCC2.POSITION;

The source spreadsheet with the data dictionary details should be saved to the root of the C:\ drive with the name C:\Data Dictionary.xls

In a second spreadsheet we are trying to build the following interface:

Next, we need to add a couple ActiveX controls to the second spreadsheet:
B3: Combo Box with a name of cboTableName with a blank Text value
C3: Check Box with a name of chkAliasNames with a Caption of Alias Table Names
C1: Command Button with a name of cmdInitialize with a Caption of Initialize
A1, A3, B1 – type in the text as shown above

Once we add the macro code, clicking the Initialize button will send a query to the other spreadsheet to retrieve a list of the tables:

And with all of the code in place, selecting a table builds the abstract art:

Running down the left side is the list of columns in the selected table. Every foreign key that is defined against the selected table will trigger a recursive lookup for child tables of the child table, with the necessary join conditions listed as well as the columns that are defined in the child table. For instance, there is a foreign key on the CO_PRODUCT table that points back to the primary key on the PART table, and the join between the tables should be P.ID = CP.PART_ID. The CO_PRODUCT table does not have any child tables.

Skipping down to the CUST_ORDER_LINE table, it has a foreign key that references the PART table on P.ID = COL.PART_ID. The RECEIVABLE_LINE table has a foreign key that references the CUST_ORDER_LINE table on COL.CUST_ORDER_ID = RL.CUST_ORDER_ID AND COL.LINE_NO = RL.CUST_ORDER_LINE_NO. The RECV_LINE_BINARY table has a foreign key that references the RECEIVABLE_LINE table on RL.INVOICE_ID = RLB.INVOICE_ID AND RL.LINE_NO = RLB.RECV_LINE_NO. With this tool we just discovered a way to join the PART table to the RECV_LINE_BINARY by analyzing the defined foreign keys.

OK, enough fun, lets enter the code. Right-click the worksheet name (probably Sheet1) rename the sheet to ExcelQueryOfExcel and then right-click the sheet and to select View Code. Let’s start with making the Initialize button work:

'Need to add a reference to Microsoft ActiveX Data Objects 2.8 Library before starting
'Declare a connection object in the general section to hold the connection to the database
Dim dbExcel As New ADODB.Connection

'Declare a set of variables to hold the username and password for the database
Dim strUserName As String
Dim strPassword As String
Dim strDatabase As String

Private Function ConnectDatabase() As Integer
    Dim intResult As Integer

    On Error Resume Next

    If dbExcel.State <> 1 Then
        'Connection to the database if closed
        strDatabase = Sheets("ExcelQueryOfExcel").Cells(1, 2).Value

        strUserName = ""
        strPassword = ""

        'Connect to the database
        dbExcel.ConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & strDatabase & ";Extended Properties=""Excel 8.0;HDR=Yes;IMEX=1"";"

        dbExcel.ConnectionTimeout = 40
        dbExcel.CursorLocation = adUseClient
        dbExcel.Open

        If (dbExcel.State <> 1) Or (Err <> 0) Then
            intResult = MsgBox("Could not connect to the Excel Database." & vbCrLf & Error(Err), 16, "Excel Demo")

            ConnectDatabase = False
        Else
            ConnectDatabase = True
        End If
    Else
        ConnectDatabase = True
    End If
End Function

Private Sub cmdInitialize_Click()
    Dim intResult As Integer
    Dim strSQL As String
    Dim strLastTable As String
    Dim snpData As ADODB.Recordset

    On Error Resume Next

    cboTableName.Clear
    cboTableName = ""
    Sheets("ExcelQueryOfExcel").Range("A6:Z10006").Delete Shift:=xlUp

    intResult = ConnectDatabase
    strLastTable = ""

    If intResult = True Then
        Set snpData = New ADODB.Recordset

        strSQL = "SELECT" & vbCrLf
        strSQL = strSQL & "  *" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  [Data Dict Tables$]" & vbCrLf
        strSQL = strSQL & "ORDER BY" & vbCrLf
        strSQL = strSQL & "  2"     '  Sort the rows by the second column in the data source
        snpData.Open strSQL, dbExcel

        If snpData.State = 1 Then
            Do While Not snpData.EOF
                If strLastTable <> snpData("table name") Then
                    strLastTable = snpData("table name")
                    cboTableName.AddItem snpData("table name")
                End If

                snpData.MoveNext
            Loop
            snpData.Close
        Else
            intResult = MsgBox("Could not send the SQL statement to the Excel Database." & vbCrLf & Error(Err), 16, "Excel Demo")
        End If

        Set snpData = Nothing
    End If
End Sub

The above code should be sufficient for the Initialize button to work – you may need to turn off Design Mode in Excel for the button to work.

Now we need to work on the recursion when a table is selected (and the rest of the code):

Private Function FindRelatedTables(strTableName As String, lngColumn As Long, lngRow As Long, intAliasTableNames As Integer) As Long
    Dim i As Integer
    Dim lngBoxLeft As Long
    Dim lngBoxTop As Long
    Dim lngBoxWidth As Long
    Dim lngBoxHeight As Long
    Dim lngResult As Long
    Dim intTableColumns As Integer
    Dim intForeignColumns As Integer
    Dim intFlag As Integer
    Dim strOut As String
    Dim strLastTable As String
    Dim strSplit() As String  'Use to alias the table names
    Dim strPrimary As String  'Stores the aliased table primary key table name
    Dim strForeign As String  'Stores the aliased table foreign key table name
    Dim strSQL As String
    Dim snpData As ADODB.Recordset
    Dim snpDataForeign As ADODB.Recordset

    On Error Resume Next

    If intAliasTableNames = True Then
        strPrimary = ""
        strSplit = Split(strTableName, "_")

        For i = 0 To UBound(strSplit)
            strPrimary = strPrimary & Left(strSplit(i), 1)
        Next i
    End If

    Set snpDataForeign = New ADODB.Recordset
    Set snpData = New ADODB.Recordset

    strSQL = "SELECT" & vbCrLf
    strSQL = strSQL & "  *" & vbCrLf
    strSQL = strSQL & "FROM" & vbCrLf
    strSQL = strSQL & "  [Data Dict Foreign Keys$]" & vbCrLf
    strSQL = strSQL & "WHERE" & vbCrLf
    strSQL = strSQL & "  [Pkey Table Name] = '" & strTableName & "'" & vbCrLf
    strSQL = strSQL & "ORDER BY" & vbCrLf
    strSQL = strSQL & "  [Fkey Table Name]," & vbCrLf
    strSQL = strSQL & "  [Position]"
    snpDataForeign.Open strSQL, dbExcel

    If snpDataForeign.State = 1 Then
        If Not snpDataForeign.EOF Then
            strOut = strTableName & " " & strPrimary & " :" & vbLf
            intTableColumns = 0
            intForeignColumns = 0
            strLastTable = snpDataForeign("Fkey Table Name")
            Do While Not snpDataForeign.EOF
                If intAliasTableNames = True Then
                    strForeign = ""
                    strSplit = Split(snpDataForeign("Fkey Table Name"), "_")

                    For i = 0 To UBound(strSplit)
                        strForeign = strForeign & Left(strSplit(i), 1)
                    Next i
                    'Verify that the two aliases are not identical
                    If strForeign = strPrimary Then
                        strForeign = strForeign & "F"
                    End If
                    strOut = strOut & strPrimary & "." & snpDataForeign("Pkey Column Name") & " = " & strForeign & "." & snpDataForeign("Fkey Column Name") & vbLf
                Else
                    strOut = strOut & snpDataForeign("Pkey Table Name") & "." & snpDataForeign("Pkey Column Name") & " = " & snpDataForeign("Fkey Table Name") & "." & snpDataForeign("Fkey Column Name") & vbLf
                End If

                intForeignColumns = intForeignColumns + 1

                snpDataForeign.MoveNext

                'See if we need to retrieve the table columns for the previous table
                intFlag = False
                If snpDataForeign.EOF Then
                    intFlag = True
                Else
                    If strLastTable <> snpDataForeign("Fkey Table Name") Then
                        intFlag = True
                    End If
                End If

                If intFlag = True Then
                    strSQL = "SELECT" & vbCrLf
                    strSQL = strSQL & "  *" & vbCrLf
                    strSQL = strSQL & "FROM" & vbCrLf
                    strSQL = strSQL & "  [Data Dict Tables$]" & vbCrLf
                    strSQL = strSQL & "WHERE" & vbCrLf
                    strSQL = strSQL & "  [Table Name] = '" & strLastTable & "'" & vbCrLf
                    strSQL = strSQL & "ORDER BY" & vbCrLf
                    strSQL = strSQL & "  [Column Id]"     '  Sort the rows by the second column in the data source
                    snpData.Open strSQL, dbExcel
                    If snpData.State = 1 Then
                        If Not snpData.EOF Then
                            strOut = strOut & vbLf & strLastTable & " " & strForeign & " :" & vbLf
                            intTableColumns = 0
                            Do While Not snpData.EOF
                                strOut = strOut & snpData("column name") & vbLf
                                intTableColumns = intTableColumns + 1

                                snpData.MoveNext
                            Loop
                        End If
                    End If
                    snpData.Close

                    'Create the text box
                    'Strip off the trailing CrLf
                    strOut = Left(strOut, Len(strOut) - 1)

                    lngBoxLeft = lngColumn * 200
                    lngBoxWidth = 200
                    lngBoxTop = lngRow * 20
                    lngBoxHeight = (intTableColumns + intForeignColumns + 3) * 5.5

                    With Sheets("ExcelQueryOfExcel").Shapes.AddTextbox(msoTextOrientationHorizontal, lngBoxLeft, lngBoxTop, lngBoxWidth, (lngBoxHeight * 2 - 1))
                        With .TextFrame
                            If Application.Version <= 11 Then
                                'There is a 255 character limit on text boxes before Office 2007
                                .Characters.Text = Left(strOut, 255)
                            Else
                                .Characters.Text = strOut
                            End If
                            .Characters.Font.Size = 8
                        End With
                        .Fill.ForeColor.RGB = RGB(255 - lngColumn * 10, 255 - lngColumn * 10, 255)
                    End With

                    'With Sheets("ExcelQueryOfExcel").Shapes.AddTextbox(msoTextOrientationHorizontal, lngBoxLeft, lngBoxTop, lngBoxWidth, (lngBoxHeight * 2 - 1))
                    '    With .TextFrame
                    '        .Characters.Text = strOut
                    '        .Characters.Font.Size = 8
                    '    End With
                    '    .Fill.ForeColor.RGB = RGB(255 - lngColumn * 10, 255 - lngColumn * 10, 255)
                    'End With

                    If lngColumn < 6 Then
                        'Only recursively call to 6 levels
                        'Recursive call into this function
                        lngResult = FindRelatedTables(strLastTable, lngColumn + 1, lngRow, intAliasTableNames)
                    End If

                    'Prepare for the next text box
                    strOut = strTableName & " " & strPrimary & " :" & vbLf
                    intTableColumns = 0
                    intForeignColumns = 0
                    lngRow = lngRow + 3
                End If
                If Not snpDataForeign.EOF Then
                    strLastTable = snpDataForeign("Fkey Table Name")
                End If

                If lngRow > 5000 Then
                    'Protection against a very long wait
                    Exit Do
                End If
            Loop
        End If
        snpDataForeign.Close
    End If

    Set snpDataForeign = Nothing
    Set snpData = Nothing
End Function

Private Sub FindTable()
    Dim intAliasTableNames As Integer
    Dim intResult As Integer
    Dim lngResult As Long
    Dim lngRow As Long
    Dim strSQL As String
    Dim strTableName As String
    Dim snpData As ADODB.Recordset

    On Error Resume Next

    intAliasTableNames = chkAliasNames.Value
    lngRow = 6

    Sheets("ExcelQueryOfExcel").Range("A" & Format(lngRow) & ":Z" & Format(lngRow + 10000)).Delete Shift:=xlUp

    If cboTableName <> "" Then
        Application.ScreenUpdating = False

        strTableName = cboTableName
        Set snpData = New ADODB.Recordset

        strSQL = "SELECT" & vbCrLf
        strSQL = strSQL & "  *" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  [Data Dict Tables$]" & vbCrLf
        strSQL = strSQL & "WHERE" & vbCrLf
        strSQL = strSQL & "  [Table Name] = '" & strTableName & "'" & vbCrLf
        strSQL = strSQL & "ORDER BY" & vbCrLf
        strSQL = strSQL & "  [Column Id]"     '  Sort the rows by the second column in the data source
        snpData.Open strSQL, dbExcel
        If snpData.State = 1 Then
            Sheets("ExcelQueryOfExcel").Cells(lngRow, 1).Value = strTableName
            Sheets("ExcelQueryOfExcel").Cells(lngRow, 1).Font.Bold = True
            lngRow = lngRow + 1

            If Not snpData.EOF Then
                Do While Not snpData.EOF
                    Sheets("ExcelQueryOfExcel").Cells(lngRow, 1).Value = snpData("column name")
                    lngRow = lngRow + 1

                    snpData.MoveNext

                    'Safety net
                    If lngRow > 1000 Then
                        Exit Do
                    End If
                Loop

                lngResult = FindRelatedTables(strTableName, 1, 5, intAliasTableNames)
            End If
            snpData.Close
        Else
            intResult = MsgBox("Could not send the SQL statement to the Excel Database." & vbCrLf & Error(Err), 16, "Excel Demo")
        End If

        Set snpData = Nothing
    End If
    Application.ScreenUpdating = True
End Sub

Private Sub cboTableName_Click()
    FindTable
End Sub

Private Sub cboTableName_GotFocus()
    cboTableName.SelStart = 0
    cboTableName.SelLength = Len(cboTableName)
End Sub

Private Sub cboTableName_KeyUp(ByVal KeyCode As MSForms.ReturnInteger, ByVal Shift As Integer)
    If (KeyCode = 13) Or (KeyCode = 8  ) Then
        FindTable
    End If
End Sub

Private Sub chkAliasNames_Click()
    FindTable
End Sub

I will not try here to explain how all of the above code works. I created this demonstration for a presentation that I gave a couple months ago. It really should be easy to understand if you step through the code starting with the cboTableName_Click Sub.

Comments : Leave a Comment »

Categories : Excel

Which is Most Efficient when Selecting Rows: EXISTS, IN, or a JOIN

29 12 2009

December 29, 2009

This post is a follow up to the previous post that questioned which approach is most efficient when deleting rows.

The Oracle documentation for 10g R2 states the following

“In certain circumstances, it is better to use IN rather than EXISTS. In general, if the selective predicate is in the subquery, then use IN. If the selective predicate is in the parent query, then use EXISTS.”

“Sometimes, Oracle can rewrite a subquery when used with an IN clause to take advantage of selectivity specified in the subquery. This is most beneficial when the most selective filter appears in the subquery and there are indexes on the join columns. Conversely, using EXISTS is beneficial when the most selective filter is in the parent query. This allows the selective predicates in the parent query to be applied before filtering the rows against the EXISTS criteria.”

The above quotes seems to be similar to the quotes provided in the earlier blog article. Let’s set up the same test tables as were used in the previous blog article:

CREATE TABLE T1 (
  C1 NUMBER,
  FILLER VARCHAR2(300),
  PRIMARY KEY (C1));

CREATE TABLE T2 (
  C1 NUMBER,
  FILLER VARCHAR2(300),
  PRIMARY KEY (C1));

INSERT INTO
  T1
SELECT
  ROWNUM,
  LPAD('A',300,'A')
FROM
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V1,
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V2;

INSERT INTO
  T2
SELECT
  ROWNUM*3,
  LPAD('A',300,'A')
FROM
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=333) V1,
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V2;

COMMIT;

EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER, TABNAME=>'T1')
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER, TABNAME=>'T2')

Next, we will set up to capture a 10053 trace for three SQL statements:

Uncorrelated IN subquery
Correlated IN subquery
EXISTS subquery

ALTER SESSION SET TRACEFILE_IDENTIFIER = 'SELECT_METHODS';
ALTER SESSION SET EVENTS '10053 trace name context forever, level 1';

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SET TIMING ON

SPOOL select_methods.txt

SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2 WHERE T2.C1=T1.C1);

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SELECT C1 FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

SPOOL OFF

ALTER SESSION SET EVENTS '10053 trace name context off';

On Oracle Database 11.1.0.7, I received the following output in the select_methods.txt file (slightly cleaned up):

SQL> SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

Elapsed: 00:00:43.54

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;
SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

SQL> SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2 WHERE T2.C1=T1.C1);

Elapsed: 00:00:43.18

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;
SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

SQL> SELECT C1 FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

Elapsed: 00:00:43.33

The above output seems to indicate that the correlated IN subquery completed the fastest, followed by the EXISTS subquery, and then the uncorrelated IN subquery. Let’s check the 10053 trace file.

******************************************
----- Current SQL Statement for this session (sql_id=19gjmx8y5rcg9) -----
SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2)
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT "T1"."C1" "C1" FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T1"."C1"="T2"."C1"
...
============
Plan Table
============
---------------------------------------------+-----------------------------------+
| Id  | Operation              | Name        | Rows  | Bytes | Cost  | Time      |
---------------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT       |             |       |       |   576 |           |
| 1   |  NESTED LOOPS          |             |  325K | 3252K |   576 |  00:00:07 |
| 2   |   INDEX FAST FULL SCAN | SYS_C0016167|  977K | 4883K |   514 |  00:00:07 |
| 3   |   INDEX UNIQUE SCAN    | SYS_C0016168|     1 |     5 |     0 |           |
---------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
3 - access("C1"="C1")

...
...
...

******************************************
----- Current SQL Statement for this session (sql_id=5hkw5pa3stxvr) -----
SELECT C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2 WHERE T2.C1=T1.C1)
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT "T1"."C1" "C1" FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T1"."C1"="T2"."C1" AND "T2"."C1"="T1"."C1"
...
============
Plan Table
============
---------------------------------------------+-----------------------------------+
| Id  | Operation              | Name        | Rows  | Bytes | Cost  | Time      |
---------------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT       |             |       |       |   576 |           |
| 1   |  NESTED LOOPS          |             |  325K | 3252K |   576 |  00:00:07 |
| 2   |   INDEX FAST FULL SCAN | SYS_C0016167|  977K | 4883K |   514 |  00:00:07 |
| 3   |   INDEX UNIQUE SCAN    | SYS_C0016168|     1 |     5 |     0 |           |
---------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
3 - access("C1"="C1")

...
...
...

******************************************
----- Current SQL Statement for this session (sql_id=4x73yz9t7dazj) -----
SELECT C1 FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1)
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT "T1"."C1" "C1" FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T2"."C1"="T1"."C1"
...
============
Plan Table
============
---------------------------------------------+-----------------------------------+
| Id  | Operation              | Name        | Rows  | Bytes | Cost  | Time      |
---------------------------------------------+-----------------------------------+
| 0   | SELECT STATEMENT       |             |       |       |   576 |           |
| 1   |  NESTED LOOPS SEMI     |             |  325K | 3252K |   576 |  00:00:07 |
| 2   |   INDEX FAST FULL SCAN | SYS_C0016167|  977K | 4883K |   514 |  00:00:07 |
| 3   |   INDEX UNIQUE SCAN    | SYS_C0016168|  108K |  541K |     0 |           |
---------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
3 - access("T2"."C1"="T1"."C1")

Notice anything strange about the “Final query after transformations”? The transformed uncorrelated IN subquery and the transformed EXISTS subquery are identical (if you ignore that T1.C1= T2.C1 is actually listed as T2.C1=T1.C1. Notice also that the transformed queries are now standard joins rather than the IN query being transformed into an EXISTS query, or vice-versa. The transformed version of the correlated IN subquery is a bit odd – it seems that the query optimizer in 11.1.0.7 did not automatically eliminate the duplicate T1.C1= T2.C1 (if I recall correctly 10.2.0.4 does remove the duplicate predicate when writing the “Final query”). OK, the transformed versions of the SQL statements are for all purposes identical. The plan for the EXISTS query shows that the join is a NESTED LOOPS SEMI, while the other plans show a join of NESTED LOOPS.

Let’s look at the DBMS_XPLAN output (note that STATISTICS_LEVEL was set to the default of TYPICAL):

SET TIMING OFF
SET AUTOTRACE OFF

SET PAGESIZE 2000
SET LINESIZE 140

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SPOOL select_methods2.txt

SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2 WHERE T2.C1=T1.C1);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SPOOL OFF

The output of the above is as follows (note that the output has been cleaned up slightly):

SQL_ID  gwd4d6sk75hhk, child number 0                                
-------------------------------------                                
SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2)                          
Plan hash value: 319633161        

---------------------------------------------------------------------------------------------------------
| Id  | Operation             | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
---------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |              |      1 |        |    333K|00:00:00.05 |   64418 |   2547 |
|   1 |  NESTED LOOPS         |              |      1 |    333K|    333K|00:00:00.05 |   64418 |   2547 |
|   2 |   INDEX FAST FULL SCAN| SYS_C0016167 |      1 |   1000K|   1000K|00:00:00.02 |   24052 |   1883 |
|*  3 |   INDEX UNIQUE SCAN   | SYS_C0016168 |   1000K|      1 |    333K|00:00:00.84 |   40366 |    664 |
---------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):                  
---------------------------------------------------                  
   3 - access("C1"="C1")          

-

SQL_ID  fcm7ptb886bzt, child number 0                                
-------------------------------------                                
SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE C1 IN (SELECT C1 FROM T2 WHERE T2.C1=T1.C1)        

Plan hash value: 319633161        

---------------------------------------------------------------------------------------------------------
| Id  | Operation             | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
---------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |              |      1 |        |    333K|00:00:00.09 |   64418 |   2547 |
|   1 |  NESTED LOOPS         |              |      1 |    333K|    333K|00:00:00.09 |   64418 |   2547 |
|   2 |   INDEX FAST FULL SCAN| SYS_C0016167 |      1 |   1000K|   1000K|00:00:00.03 |   24052 |   1883 |
|*  3 |   INDEX UNIQUE SCAN   | SYS_C0016168 |   1000K|      1 |    333K|00:00:01.06 |   40366 |    664 |
---------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):                  
---------------------------------------------------                  
   3 - access("C1"="C1")          

-

SQL_ID  4ym7p8815nquy, child number 0                                
-------------------------------------                                
SELECT /*+ GATHER_PLAN_STATISTICS */ C1 FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1)        

Plan hash value: 2371405353       

---------------------------------------------------------------------------------------------------------
| Id  | Operation             | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
---------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT      |              |      1 |        |    333K|00:00:00.06 |   64418 |   2547 |
|   1 |  NESTED LOOPS SEMI    |              |      1 |    333K|    333K|00:00:00.06 |   64418 |   2547 |
|   2 |   INDEX FAST FULL SCAN| SYS_C0016167 |      1 |   1000K|   1000K|00:00:00.03 |   24052 |   1883 |
|*  3 |   INDEX UNIQUE SCAN   | SYS_C0016168 |   1000K|    110K|    333K|00:00:00.94 |   40366 |    664 |
---------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):                  
---------------------------------------------------                  
   3 - access("T2"."C1"="T1"."C1")

The output captured during the 10053 trace implied: “The above output seems to indicate that the correlated IN subquery completed the fastest, followed by the EXISTS subquery, and then the uncorrelated IN subquery.” The output captured during the DBMS_XPLAN script seems to show that the uncorrelated subquery completed the fastest, followed by the EXISTS subquery, and then the correlated subquery.

OK, so the question remains: Which is most efficient when selecting rows: EXISTS, IN, or a JOIN? It is pretty hard to tell when the optimizer automatically re-writes the queries that we submit.

It might be interesting to recheck the output with a new test case that permits:

NULL values in one or both tables.
Duplicate values in one or both tables.
Larger data sets.

Comments : Leave a Comment »

Categories : Performance, SQL

Which is Most Efficient when Deleting Rows: EXISTS, IN, or a VIEW

29 12 2009

December 29, 2009

In the process of reading the book “Oracle SQL Recipes” I encountered a couple interesting statements. One such statement is found on page 68 in recipe 3-5 Removing Rows Based on Data in Other Tables. The book states the following:

“Whether you use IN or EXISTS depends on the sizes of the driving table (the outer table referenced in the SELECT, UPDATE, or DELETE) and the size of the result set in the subquery. Using IN is most likely better if the results of the subquery are small or is a list of constants. However, using EXISTS may run a lot more efficiently since the implicit JOIN may take advantage of indexes.”

The above comment sounds a bit like this one from the book “Expert Oracle Database 11g Administration” on page 1067 that might be describing just the behavior of SELECT statements, or might also be describing the behavior of DELETE statements:

“Subqueries perform better when you use IN rather than EXISTS. Oracle recommends using the IN clause if the subquery has the selective WHERE clause. If the parent query contains the selective WHERE clause, use the EXISTS clause rather than the IN clause.”

Both are interesting statements. The “Oracle SQL Recipes” book provided three different SQL statements for removing rows in one table based on the contents of another table – those SQL statements used the demo data available for Oracle 11g. I do not have the demo data loaded, so I wondered – what if I set up a simple test case?

Which method do you think will execute faster using the supplied test case?

The sample data, 1,000,000 rows in table T1 and 333,000 rows in T2:

CREATE TABLE T1 (
  C1 NUMBER,
  FILLER VARCHAR2(300),
  PRIMARY KEY (C1));

CREATE TABLE T2 (
  C1 NUMBER,
  FILLER VARCHAR2(300),
  PRIMARY KEY (C1));

INSERT INTO
  T1
SELECT
  ROWNUM,
  LPAD('A',300,'A')
FROM
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V1,
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V2;

INSERT INTO
  T2
SELECT
  ROWNUM*3,
  LPAD('A',300,'A')
FROM
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=333) V1,
  (SELECT
    ROWNUM NR
  FROM
    DUAL
  CONNECT BY
    LEVEL<=1000) V2;

COMMIT;

EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER, TABNAME=>'T1')
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER, TABNAME=>'T2')

The sample DELETE statements:

DELETE FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

DELETE FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

DELETE (SELECT C1 FROM T1 JOIN T2 USING (C1));

——–

Do not scroll down – you will spoil the follow up. Which do you think is the most efficient method based on what is written above, and what you know about the behavior of recent Oracle releases (say 10.1 and later)?

—

Let’s say you ran the following script:

ALTER SESSION SET TRACEFILE_IDENTIFIER = 'DELETE_METHODS';
ALTER SESSION SET EVENTS '10053 trace name context forever, level 1';

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SET TIMING ON

SPOOL delete_methods.txt

DELETE FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

ROLLBACK;

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

DELETE FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

ROLLBACK;

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

DELETE (SELECT C1 FROM T1 JOIN T2 USING (C1));

ROLLBACK;

SPOOL OFF

ALTER SESSION SET EVENTS '10053 trace name context off';

Now which do you think is most efficient? Do not scroll down – you will spoil the follow up.

—

Let’s say that in the Oracle Database 11.1.0.7 10053 trace file created by the above script, you find the following:

******************************************
----- Current SQL Statement for this session (sql_id=7mc4cvm67pns3) -----
DELETE FROM T1 WHERE C1 IN (SELECT C1 FROM T2)
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT 0 FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T1"."C1"="T2"."C1"
...
============
Plan Table
============
-------------------------------------------+-----------------------------------+
| Id  | Operation            | Name        | Rows  | Bytes | Cost  | Time      |
-------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT     |             |       |       |  2025 |           |
| 1   |  DELETE              | T1          |       |       |       |           |
| 2   |   NESTED LOOPS       |             |  325K | 3252K |  2025 |  00:00:25 |
| 3   |    INDEX FULL SCAN   | SYS_C0030271|  977K | 4883K |  1892 |  00:00:23 |
| 4   |    INDEX UNIQUE SCAN | SYS_C0030272|     1 |     5 |     0 |           |
-------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
4 - access("C1"="C1")

...
...
...

******************************************
----- Current SQL Statement for this session (sql_id=ahq5s54mw5a2g) -----
DELETE FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1)
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT 0 FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T2"."C1"="T1"."C1"
...
============
Plan Table
============
-------------------------------------------+-----------------------------------+
| Id  | Operation            | Name        | Rows  | Bytes | Cost  | Time      |
-------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT     |             |       |       |  2025 |           |
| 1   |  DELETE              | T1          |       |       |       |           |
| 2   |   NESTED LOOPS SEMI  |             |  325K | 3252K |  2025 |  00:00:25 |
| 3   |    INDEX FULL SCAN   | SYS_C0030271|  977K | 4883K |  1892 |  00:00:23 |
| 4   |    INDEX UNIQUE SCAN | SYS_C0030272|  108K |  541K |     0 |           |
-------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
4 - access("T2"."C1"="T1"."C1")

...
...
...

******************************************
----- Current SQL Statement for this session (sql_id=61r3vxah952fu) -----
DELETE (SELECT C1 FROM T1 JOIN T2 USING (C1))
*******************************************
...
Final query after transformations:******* UNPARSED QUERY IS *******
SELECT 0 FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T1"."C1"="T2"."C1"
...
============
Plan Table
============
-------------------------------------------+-----------------------------------+
| Id  | Operation            | Name        | Rows  | Bytes | Cost  | Time      |
-------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT     |             |       |       |  2025 |           |
| 1   |  DELETE              | T1          |       |       |       |           |
| 2   |   NESTED LOOPS       |             |  325K | 3252K |  2025 |  00:00:25 |
| 3   |    INDEX FULL SCAN   | SYS_C0030271|  977K | 4883K |  1892 |  00:00:23 |
| 4   |    INDEX UNIQUE SCAN | SYS_C0030272|     1 |     5 |     0 |           |
-------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
4 - access("T1"."C1"="T2"."C1")

Well, that is odd, three different queries sent in to the optimizer, and each time the optimizer showed the final query after transformation as the following:

SELECT 0 FROM "TESTUSER"."T2" "T2","TESTUSER"."T1" "T1" WHERE "T1"."C1"="T2"."C1"

Now which do you think is most efficient?

The plans written in the 10053 trace file for Oracle 10.2.0.4 were slightly different:

Current SQL statement for this session:
DELETE FROM T1 WHERE C1 IN (SELECT C1 FROM T2)

============
Plan Table
============
---------------------------------------------+-----------------------------------+
| Id  | Operation               | Name       | Rows  | Bytes | Cost  | Time      |
---------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT        |            |       |       |  1132 |           |
| 1   |  DELETE                 | T1         |       |       |       |           |
| 2   |   NESTED LOOPS          |            |  328K | 3283K |  1132 |  00:00:14 |
| 3   |    INDEX FAST FULL SCAN | SYS_C009783|  328K | 1642K |   173 |  00:00:03 |
| 4   |    INDEX UNIQUE SCAN    | SYS_C009782|     1 |     5 |     1 |  00:00:01 |
---------------------------------------------+-----------------------------------+

Predicate Information:
----------------------
4 - access("C1"="C1")

...
...
...
Current SQL statement for this session:
DELETE FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1)

============
Plan Table
============
----------------------------------------------+-----------------------------------+
| Id  | Operation                | Name       | Rows  | Bytes | Cost  | Time      |
----------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT         |            |       |       |  1687 |           |
| 1   |  DELETE                  | T1         |       |       |       |           |
| 2   |   NESTED LOOPS           |            |  328K | 3283K |  1687 |  00:00:21 |
| 3   |    SORT UNIQUE           |            |  328K | 1642K |   173 |  00:00:03 |
| 4   |     INDEX FAST FULL SCAN | SYS_C009783|  328K | 1642K |   173 |  00:00:03 |
| 5   |    INDEX UNIQUE SCAN     | SYS_C009782|     1 |     5 |     1 |  00:00:01 |
----------------------------------------------+-----------------------------------+

Predicate Information:
----------------------
5 - access("T2"."C1"="T1"."C1")

...
...
...
Current SQL statement for this session:
DELETE (SELECT C1 FROM T1 JOIN T2 USING (C1))

============
Plan Table
============
---------------------------------------------+-----------------------------------+
| Id  | Operation               | Name       | Rows  | Bytes | Cost  | Time      |
---------------------------------------------+-----------------------------------+
| 0   | DELETE STATEMENT        |            |       |       |  1132 |           |
| 1   |  DELETE                 | T1         |       |       |       |           |
| 2   |   NESTED LOOPS          |            |  328K | 3283K |  1132 |  00:00:14 |
| 3   |    INDEX FAST FULL SCAN | SYS_C009783|  328K | 1642K |   173 |  00:00:03 |
| 4   |    INDEX UNIQUE SCAN    | SYS_C009782|     1 |     5 |     1 |  00:00:01 |
---------------------------------------------+-----------------------------------+
Predicate Information:
----------------------
4 - access("T1"."C1"="T2"."C1")

Do not scroll down – you will spoil the follow up.

—

Let’s say that you test the performance on a system that uses direct, asynchronous I/O (no OS caching) with DBMS_XPLAN using the following script:

SET TIMING OFF
SET PAGESIZE 2000
SET LINESIZE 140

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SPOOL delete_methods.txt

DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

ROLLBACK;

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

ROLLBACK;

ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

DELETE /*+ GATHER_PLAN_STATISTICS */ (SELECT C1 FROM T1 JOIN T2 USING (C1));

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

ROLLBACK;

SPOOL OFF

Here is the output when I ran the above script (slightly cleaned up):

SQL> DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE C1 IN (SELECT C1 FROM T2);

333000 rows deleted.

SQL>
SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT           
-------------------------------------------------------------------------------------------------------
SQL_ID  3ng6d061qazr1, child number 0
-------------------------------------
DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE C1 IN (SELECT C1 FROM T2)                    

Plan hash value: 1028502382 

-------------------------------------------------------------------------------------------------------
| Id  | Operation           | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
-------------------------------------------------------------------------------------------------------
|   0 | DELETE STATEMENT    |              |      1 |        |      0 |00:00:15.67 |     424K|  46351 |
|   1 |  DELETE             | T1           |      1 |        |      0 |00:00:15.67 |     424K|  46351 |
|   2 |   NESTED LOOPS      |              |      1 |    333K|    333K|00:00:01.10 |    7484 |   2577 |
|   3 |    INDEX FULL SCAN  | SYS_C0016151 |      1 |   1000K|   1000K|00:00:00.07 |    1877 |   1920 |
|*  4 |    INDEX UNIQUE SCAN| SYS_C0016152 |   1000K|      1 |    333K|00:00:01.08 |    5607 |    657 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):              
---------------------------------------------------              
   4 - access("C1"="C1")    

SQL> ROLLBACK;

Rollback complete.

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

System altered.

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

System altered.

SQL> DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1);

333000 rows deleted.

SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT           
-------------------------------------------------------------------------------------------------------
SQL_ID  1138j52zs4rqw, child number 0                            
-------------------------------------                            
DELETE /*+ GATHER_PLAN_STATISTICS */ FROM T1 WHERE EXISTS (SELECT 1 FROM T2 WHERE T2.C1=T1.C1)  

Plan hash value: 4152631912 

-------------------------------------------------------------------------------------------------------
| Id  | Operation           | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
-------------------------------------------------------------------------------------------------------
|   0 | DELETE STATEMENT    |              |      1 |        |      0 |00:00:15.69 |     424K|  46355 |
|   1 |  DELETE             | T1           |      1 |        |      0 |00:00:15.69 |     424K|  46355 |
|   2 |   NESTED LOOPS SEMI |              |      1 |    333K|    333K|00:00:01.38 |    7484 |   2577 |
|   3 |    INDEX FULL SCAN  | SYS_C0016151 |      1 |   1000K|   1000K|00:00:00.02 |    1877 |   1920 |
|*  4 |    INDEX UNIQUE SCAN| SYS_C0016152 |   1000K|    110K|    333K|00:00:01.11 |    5607 |    657 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):              
---------------------------------------------------              
   4 - access("T2"."C1"="T1"."C1")                               

SQL> ROLLBACK;

Rollback complete.

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

System altered.

SQL> ALTER SYSTEM FLUSH BUFFER_CACHE;

System altered.

SQL> DELETE /*+ GATHER_PLAN_STATISTICS */ (SELECT C1 FROM T1 JOIN T2 USING (C1));

333000 rows deleted.

SQL> SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

PLAN_TABLE_OUTPUT           
-------------------------------------------------------------------------------------------------------
SQL_ID  c4xz24vawqqbw, child number 0                            
-------------------------------------                            
DELETE /*+ GATHER_PLAN_STATISTICS */ (SELECT C1 FROM T1 JOIN T2 USING (C1))                       

Plan hash value: 1028502382 

-------------------------------------------------------------------------------------------------------
| Id  | Operation           | Name         | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |
-------------------------------------------------------------------------------------------------------
|   0 | DELETE STATEMENT    |              |      1 |        |      0 |00:00:15.90 |     424K|  46355 |
|   1 |  DELETE             | T1           |      1 |        |      0 |00:00:15.90 |     424K|  46355 |
|   2 |   NESTED LOOPS      |              |      1 |    333K|    333K|00:00:03.69 |    7484 |   2577 |
|   3 |    INDEX FULL SCAN  | SYS_C0016151 |      1 |   1000K|   1000K|00:00:00.02 |    1877 |   1920 |
|*  4 |    INDEX UNIQUE SCAN| SYS_C0016152 |   1000K|      1 |    333K|00:00:01.13 |    5607 |    657 |
-------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):              
---------------------------------------------------              
   4 - access("T1"."C1"="T2"."C1")                               

SQL> ROLLBACK;

Rollback complete.

So, the most efficient execution plan is the first one (by 0.02 seconds), and the least efficient is the last one (0.21 seconds slower than the second SQL statement). Oh, but there is a problem – compare the steps in the first and last execution plans. OK, now compare the steps in the first and second execution plans.

OK, so the question remains: Which is most efficient when deleting rows: EXISTS, IN, or a VIEW? It is pretty hard to tell when the optimizer automatically re-writes the queries that we submit.

Comments : 2 Comments »

Categories : Performance, SQL

Extract the First 4400 Images from Excel 2003 (and Above) and Transfer to a Database Table

28 12 2009

December 28, 2009

This code sample is an Excel 2003/2007 macro that extracts all of the named toolbar button pictures from Excel 2003/2007, transfers those pictures to an Oracle database (stored in a BLOB), and then retrieves each of the pictures and displays the pictures on an Excel worksheet.

First, we need to create a table to hold the pictures:

CREATE TABLE EXCEL2003_TOOLBAR_PICTURES (
  PICTURE_NAME VARCHAR2(60),
  PICTURE_SIZE NUMBER,
  PICTURE BLOB,
  PRIMARY KEY(PICTURE_NAME));

See the instructions in the previous blog article to enable macro support in Excel 2007 and add a reference to the ActiveX Data Objects in Excel 2003 and above.

The Excel macro code follows:

Private Sub ExtractAllImages2003()
    'Adapted from an example by John Walkenbach
    '    http://www.dailydoseofexcel.com/archives/2006/11/16/displaying-commandbar-faceid-images/
    'See also http://support.microsoft.com/kb/286460
    Dim i As Integer
    Dim intResult As Integer
    Dim intFileNum2 As Integer
    Dim lngNumPics As Long
    Dim sglX As Single
    Dim sglY As Single
    Dim strName As String
    Dim strSQL As String
    Dim strDatabase As String
    Dim strUserName As String
    Dim strPassword As String
    Dim bytPicture() As Byte

    Dim tbNewToolbar As CommandBar
    Dim tbcNewControl As CommandBarButton
    Dim picPicture As stdole.IPictureDisp
    Dim picMask As stdole.IPictureDisp
    Dim snpData As New ADODB.Recordset
    Dim dynData As New ADODB.Recordset
    Dim dbDatabase As New ADODB.Connection

    On Error Resume Next

    'Remove all of the previously created sheet
    Application.DisplayAlerts = False
    Sheets("BuiltInImages2003").Delete

    Sheets.Add
    ActiveSheet.Name = "BuiltInImages2003"

    'Delete existing TempFaceIds toolbar if it exists
    Application.CommandBars("TempFaceIds").Delete

    'Add an empty toolbar
    Set tbNewToolbar = Application.CommandBars.Add(Name:="TempFaceIds")

    'Create an object to act as a command bar control
    Set tbcNewControl = tbNewToolbar.Controls.Add(Type:=msoControlButton)

    Application.DisplayAlerts = True

    If Len(Dir("C:\ExcelBuiltInImages2003", vbDirectory)) < 4 Then
        'Create the folder to hold the exported pictures
        MkDir "C:\ExcelBuiltInImages2003"
    End If

    Err = 0

    strDatabase = "MyDB" 'From tnsnames.ora
    strUserName = "MyUserID"
    strPassword = "MyPassword"

    'Connect to the database
    'Oracle connection string
    dbDatabase.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUserName & ";Password=" & strPassword & ";ChunkSize=1000;FetchSize=100;"

    dbDatabase.ConnectionTimeout = 40
    dbDatabase.CursorLocation = adUseClient
    dbDatabase.Open

    If (dbDatabase.State <> 1) Or (Err <> 0) Then
        intResult = MsgBox("Could not connect to the database.  Check your user name and password." & vbCrLf & Error(Err), 16, "Excel Demo")
    End If

    'The table definition
    'CREATE TABLE EXCEL2003_TOOLBAR_PICTURES (
    '  PICTURE_NAME VARCHAR2(60),
    '  PICTURE_SIZE NUMBER,
    '  PICTURE BLOB,
    '  PRIMARY KEY(PICTURE_NAME));

    If Err = 0 Then
        'Remove pictures that were previously brought in
        strSQL = "DELETE FROM EXCEL2003_TOOLBAR_PICTURES"
        dbDatabase.Execute strSQL

        dbDatabase.BeginTrans

        'Prepare to add the new pictures to the database
        strSQL = "SELECT"
        strSQL = strSQL & "  PICTURE_NAME," & vbCrLf
        strSQL = strSQL & "  PICTURE_SIZE," & vbCrLf
        strSQL = strSQL & "  PICTURE" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  EXCEL2003_TOOLBAR_PICTURES"
        dynData.Open strSQL, dbDatabase, adOpenKeyset, adLockOptimistic, adCmdText

        For i = 1 To 4400  'Maximum in Excel 2003 is around 10033
            'Set the picture on the command bar control to picture number i
            tbcNewControl.FaceId = i

            'Transfer the pictures from the command bar control to local variables
            Set picPicture = tbcNewControl.Picture
            Set picMask = tbcNewControl.Mask

            'Save the pictures to disk
            strName = "FaceID " & Format(i, "0000") & ".bmp"
            Application.StatusBar = strName
            stdole.SavePicture picPicture, "C:\ExcelBuiltInImages2003\temp.bmp"

            'Create a new row in the table
            dynData.AddNew
            dynData("picture_name") = strName
            dynData("picture_size") = FileLen("C:\ExcelBuiltInImages2003\temp.bmp")

            'Read the picture into the table
            intFileNum2 = FreeFile
            Open "C:\ExcelBuiltInImages2003\temp.bmp" For Binary As #intFileNum2
            'Prepare a variable of byte data type to hold the picture read from disk
            ReDim bytPicture(FileLen("C:\ExcelBuiltInImages2003\temp.bmp"))
            Get #intFileNum2, , bytPicture
            Close #intFileNum2

            'Write the picture into the table and save the row
            dynData.Fields("picture").AppendChunk bytPicture
            dynData.Update

            'Save the mask picture for the toolbar button
            stdole.SavePicture picMask, "C:\ExcelBuiltInImages2003\temp.bmp"
            strName = "FaceID " & Format(i, "0000") & " Mask.bmp"

            'Create a new row in the table
            dynData.AddNew
            dynData("picture_name") = strName
            dynData("picture_size") = FileLen("C:\ExcelBuiltInImages2003\temp.bmp")

            'Read the picture into the table
            intFileNum2 = FreeFile
            Open "C:\ExcelBuiltInImages2003\temp.bmp" For Binary As #intFileNum2
            'Prepare a variable of byte data type to hold the picture read from disk
            ReDim bytPicture(FileLen("C:\ExcelBuiltInImages2003\temp.bmp"))
            Get #intFileNum2, , bytPicture
            Close #intFileNum2

            'Write the picture into the table and save the row
            dynData.Fields("picture").AppendChunk bytPicture
            dynData.Update

            'Free the memory ffrom the local variables
            Set picPicture = Nothing
            Set picMask = Nothing

            'Allow Excel to process events every 160 pictures
            If i Mod 160 = 0 Then
                Application.ScreenUpdating = True
                DoEvents
                Application.ScreenUpdating = False
            End If
        Next i

        dbDatabase.CommitTrans
        dynData.Close

        'Delete the picture from the folder
        Kill "C:\ExcelBuiltInImages2003\temp.bmp"

        'Retrieve the pictures from the database and display in Excel
        i = 0
        strSQL = "SELECT" & vbCrLf
        strSQL = strSQL & "  PICTURE_NAME," & vbCrLf
        strSQL = strSQL & "  PICTURE_SIZE," & vbCrLf
        strSQL = strSQL & "  PICTURE" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  EXCEL2003_TOOLBAR_PICTURES" & vbCrLf
        strSQL = strSQL & "ORDER BY" & vbCrLf
        strSQL = strSQL & "  PICTURE_NAME"
        snpData.Open strSQL, dbDatabase

        If Not (snpData.EOF) Then
            Do While Not (snpData.EOF)
                i = i + 1
                Application.StatusBar = snpData("picture_name")

                'Retrieve the picture from the database and write to a file
                intFileNum2 = FreeFile
                ReDim bytPicture(snpData("picture_size"))
                bytPicture = snpData("picture")

                Open "C:\ExcelBuiltInImages2003\" & snpData("picture_name") For Binary As #intFileNum2
                Put #intFileNum2, , bytPicture
                Close #intFileNum2

                'There will be 20 pictures across the page, so identify the top left position of the picture
                sglX = ((i - 1) Mod 40) * 18
                sglY = Int((i - 1) / 40) * 18

                'Create the shape object and load the picture that was saved from the image object
                With Sheets("BuiltInImages2003").Shapes.AddShape(Type:=msoShapeRectangle, Left:=sglX, Top:=sglY, Width:=16, Height:=16)
                    .Line.Visible = False
                    .Fill.UserPicture "C:\ExcelBuiltInImages2003\" & snpData("picture_name")
                    .AlternativeText = snpData("picture_name")
                End With

                'Allow Excel to refresh the screen as every four rows complete
                If i Mod 160 = 0 Then
                    Application.ScreenUpdating = True
                    DoEvents
                    Application.ScreenUpdating = False
                End If

                snpData.MoveNext
            Loop
            snpData.Close
        End If
    End If

    Application.CommandBars("TempFaceIds").Delete

    Application.ScreenUpdating = True
    Application.StatusBar = ""
    dbDatabase.Close

    'Clean up
    Set snpData = Nothing
    Set dynData = Nothing
    Set dbDatabase = Nothing
End Sub

When the macro runs, it saves each of the built-in toolbar icons to a file named temp.bmp, and then inserts a row into the database table with the temp.bmp picture and the mask for the temp.bmp picture that will allow creating a semi-transparent button image. Once all of the pictures are stored in the database, a query is run to retrieve each of the pictures, create a file in the ExcelBuiltInImages2003 folder for that picture, and then display the picture in Excel. The screen is refreshed after every 160 pictures are displayed. Note that most of the built-in toolbar icons are designed to be viewed at a size of 16 pixels by 16 pixels, so the icons are extracted at that size. The picture filenames are written to the Alt Text property of each picture, which may be viewed by right-clicking a picture and selecting Size and Properties…

After the macro runs, the new worksheet will look something like the picture below:

Since the pictures are also saved to a file, they are ready to be used for other purposes:

Comments : 3 Comments »

Categories : Excel

Extract 1834 Images from Excel 2007 and Transfer to a Database Table

28 12 2009

December 28, 2009

(Portions of this code are adapted from the book “Excel 2007 Power Programming with VBA” )

This code sample is an Excel 2007 macro that extracts all of the named toolbar button pictures from Excel 2007, transfers those pictures to an Oracle database (stored in a BLOB), and then retrieves each of the pictures and displays the pictures on an Excel worksheet.

First, we need to create a table to hold the pictures:

CREATE TABLE EXCEL2007_TOOLBAR_PICTURES (
  PICTURE_NAME VARCHAR2(60),
  PICTURE_SIZE NUMBER,
  PICTURE BLOB,
  PRIMARY KEY(PICTURE_NAME));

Download the ExcelImageList.doc file and save it using Microsoft Word as a Plain Text File with the name ExcelImageList.txt in the root of the C:\ drive.

We need to make certain that macro support is enabled in Excel 2007, by default it is disabled. Follow steps 1 through 3 to verify that the Developer tab appears at the top of the screen:

On the Developer tab, click the Visual Basic button. We need to add a reference to the Microsoft ActiveX Data Objects to allow the macro to interact with an Oracle database. From the Tools menu, select References…

Locate one of the recent releases of Microsoft ActiveX Data Objects, select it, then click OK.

Finally, we create the following macro:

Private Sub ExtractAllImages()
    Dim i As Integer
    Dim intResult As Integer
    Dim intFileNum As Integer
    Dim intFileNum2 As Integer
    Dim sglX As Single
    Dim sglY As Single
    Dim strName As String
    Dim strSQL As String
    Dim strDatabase As String
    Dim strUserName As String
    Dim strPassword As String
    Dim bytPicture() As Byte

    Dim imgPicture As OLEObject
    Dim snpData As New ADODB.Recordset
    Dim dynData As New ADODB.Recordset
    Dim dbDatabase As New ADODB.Connection

    On Error Resume Next

    'Remove all of the previously created sheet
    Application.DisplayAlerts = False
    Sheets("BuiltInImages").Delete

    Sheets.Add
    ActiveSheet.Name = "BuiltInImages"

    DoEvents
    Application.DisplayAlerts = True

    If Len(Dir("C:\ExcelBuiltInImages", vbDirectory)) < 4 Then
        'Create the folder to hold the exported pictures
        MkDir "C:\ExcelBuiltInImages"
    End If

    'Create a temporary image object to hold the pictures from GetImageMso
    Set imgPicture = Sheets("BuiltInImages").OLEObjects.Add(classtype:="Forms.Image.1", Left:=800, Top:=1, Width:=32, Height:=32)
    With imgPicture.Object
        .AutoSize = True
        .BorderStyle = 0
    End With

    Err = 0
    intFileNum = FreeFile
    Open "C:\ExcelImageList.txt" For Input As #intFileNum

    If Err <> 0 Then
        intResult = MsgBox("Could not open the image list file." & vbCrLf & Error(Err), 16, "Excel Demo")
        Exit Sub
    End If

    strDatabase = "MyDB" 'From tnsnames.ora
    strUserName = "MyUserID"
    strPassword = "MyPassword"

    'Connect to the database
    'Oracle connection string
    dbDatabase.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUserName & ";Password=" & strPassword & ";ChunkSize=1000;FetchSize=100;"

    dbDatabase.ConnectionTimeout = 40
    dbDatabase.CursorLocation = adUseClient
    dbDatabase.Open

    If (dbDatabase.State <> 1) Or (Err <> 0) Then
        intResult = MsgBox("Could not connect to the database.  Check your user name and password." & vbCrLf & Error(Err), 16, "Excel Demo")
    End If

    'The table definition
    'CREATE TABLE EXCEL2007_TOOLBAR_PICTURES (
    '  PICTURE_NAME VARCHAR2(60),
    '  PICTURE BLOB,
    '  PRIMARY KEY(PICTURE_NAME));

    If Err = 0 Then
        'Remove pictures that were previously brought in
        strSQL = "DELETE FROM EXCEL2007_TOOLBAR_PICTURES"
        dbDatabase.Execute strSQL

        dbDatabase.BeginTrans

        'Prepare to add the new pictures to the database
        strSQL = "SELECT"
        strSQL = strSQL & "  PICTURE_NAME," & vbCrLf
        strSQL = strSQL & "  PICTURE_SIZE," & vbCrLf
        strSQL = strSQL & "  PICTURE" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  EXCEL2007_TOOLBAR_PICTURES"
        dynData.Open strSQL, dbDatabase, adOpenKeyset, adLockOptimistic, adCmdText

        'Process each of the toolbar picture names from the text file
        Do While Not (EOF(intFileNum))
            i = i + 1
            'Read a toolbar picture name from the text file
            Line Input #intFileNum, strName
            Application.StatusBar = strName

            'Set the picture in the temporary image object, and then save to disk
            imgPicture.Object.Picture = Application.CommandBars.GetImageMso(strName, 32, 32)
            SavePicture imgPicture.Object.Picture, "C:\ExcelBuiltInImages\temp.bmp"

             'Create a new row in the table
            dynData.AddNew
            dynData("picture_name") = strName & ".bmp"
            dynData("picture_size") = FileLen("C:\ExcelBuiltInImages\temp.bmp")

            'Read the picture into the table
            intFileNum2 = FreeFile
            Open "C:\ExcelBuiltInImages\temp.bmp" For Binary As #intFileNum2

            'Prepare a variable of byte data type to hold the picture read from disk
            ReDim bytPicture(FileLen("C:\ExcelBuiltInImages\temp.bmp"))
            Get #intFileNum2, , bytPicture
            Close #intFileNum2

            'Write the picture into the table and save the row
            dynData.Fields("picture").AppendChunk bytPicture
            dynData.Update

            'Delete the picture from the folder
            Kill "C:\ExcelBuiltInImages\temp.bmp"
        Loop
        Application.StatusBar = ""

        dbDatabase.CommitTrans

        dynData.Close

        Close #intFileNum

        'Remove the temporary image object
        imgPicture.Delete

        'Retrieve the pictures from the database and display in Excel
        i = 0
        strSQL = "SELECT" & vbCrLf
        strSQL = strSQL & "  PICTURE_NAME," & vbCrLf
        strSQL = strSQL & "  PICTURE_SIZE," & vbCrLf
        strSQL = strSQL & "  PICTURE" & vbCrLf
        strSQL = strSQL & "FROM" & vbCrLf
        strSQL = strSQL & "  EXCEL2007_TOOLBAR_PICTURES" & vbCrLf
        strSQL = strSQL & "ORDER BY" & vbCrLf
        strSQL = strSQL & "  PICTURE_NAME"
        snpData.Open strSQL, dbDatabase

        If Not (snpData.EOF) Then
            Do While Not (snpData.EOF)
                i = i + 1
                Application.StatusBar = snpData("picture_name")

                'Retrieve the picture from the database and write to a file

                ReDim bytPicture(snpData("picture_size"))
                bytPicture = snpData("picture")
                intFileNum2 = FreeFile
                Open "C:\ExcelBuiltInImages\" & snpData("picture_name") For Binary As #intFileNum2
                Put #intFileNum2, , bytPicture
                Close #intFileNum2

                'There will be 20 pictures across the page, so identify the top left position of the picture
                sglX = ((i - 1) Mod 20) * 36
                sglY = Int((i - 1) / 20) * 36

                'Create the shape object and load the picture that was saved from the image object
                With Sheets("BuiltInImages").Shapes.AddShape(Type:=msoShapeRectangle, Left:=sglX, Top:=sglY, Width:=30, Height:=30)
                    .Line.Visible = False
                    .Fill.UserPicture "C:\ExcelBuiltInImages\" & snpData("picture_name")
                    .AlternativeText = snpData("picture_name")
                End With

                'Allow Excel to refresh the screen as every four rows complete
                If i Mod 80 = 0 Then
                    Application.ScreenUpdating = True
                    DoEvents
                    Application.ScreenUpdating = False
                End If

                snpData.MoveNext
            Loop
            snpData.Close
        End If
    End If
    Application.ScreenUpdating = True
    Application.StatusBar = ""
    dbDatabase.Close

    'Clean up
    Set snpData = Nothing
    Set dynData = Nothing
    Set dbDatabase = Nothing
    Set imgPicture = Nothing
End Sub

When the macro runs, it saves each of the built-in toolbar icons to a file named temp.bmp, and then inserts a row into the database table with the temp.bmp picture. Once all of the pictures are stored in the database, a query is run to retrieve each of the pictures, create a file in ExcelBuiltInImages folder for that picture, and then display the picture in Excel. The screen is refreshed after every 80 pictures are displayed. Note that most of the built-in toolbar icons are designed to be viewed at a size of 16 pixels by 16 pixels, rather than 32 pixels by 32 pixels – so the images may appear a bit “blocky”. The picture filenames are stored in the Alt Text property of each picture, which may be viewed by right-clicking a picture and selecting Size and Properties…

After the macro runs, the new worksheet will look something like the picture below:

Since the pictures are also saved to a file, they are ready to be used for other purposes:

Comments : 5 Comments »

Categories : Excel

High Value for MBRC Causes High BCHR, High CPU Usage, and Slow Performance

27 12 2009

December 27, 2009

What follows is part of a presentation that I gave in 2008.

A sudden unexpected spike in server CPU usage was noticed while running a batch process in an ERP package that normally has most of its time captured in the SQL*Net message from client wait event. My Toy Project for Performance Tuning was actively logging database performance when the problem was noticed. Below is a marked up view of the program as it captured the spike in CPU activity:In this 83 second time period, the database instance performed 7.8 million consistent gets, and not a single physical read. The database had a 100% buffer cache hit ratio during this time period, and that is good, right? The database instance read 505.6 million rows by full table scan in the 83 seconds with relatively few rows read by index lookup, which might be a sign of a problem.

Clicking on the Rows Read by Table Scan statistic, we see the running history of the number of rows read by a full table scan, with the current statistic value in blue:If we dig a bit more, knowing that the number of rows retrieved by table scan seems high, we see that we had a quiet period, and then the number of rows read by full table scan jumped considerably. At the same time, the CPU usage also jumped. We should dig into the system a bit more deeply to understand why this is a problem.

Switching over the the SQL Monitor, we start to investigate:If we look at the SQL statements as they are executed, we see one SQL statement that seems to be performing a large number of buffer gets, or logical reads – about 5.1 million per minute when fetching 123 times. The execution plans for the SQL statement are shown in the bottom pane. Note that there are two execution plans for this SQL statement, with the second created due to a bind mismatch – the initial parse call did not initialize the bind variables which caused Oracle to treat the bind variable as a VARCHAR2(2000), the second parse initialized the bind variable as a CHAR.

Zooming in on the grid at the top:If we look at the SQL statements as they are executed, we see one SQL statement that seems to be performing a large number of buffer gets, or logical reads – about 5.1 million per minute when fetching 123 times with 120 executions. Something seems a bit odd.

Zooming in on the execution plan at the bottom:

Taking a look at the execution plan, we see that the query optimizer decided to perform a full table scan when executing this SQL statement, rather than using the selective index on the PART_ID column. Oracle is applying a filter against the table rows to find only those with a specific PART_ID having a QTY greater than the COSTED_QTY, and with a TYPE of O.

If we submit the SQL statement with the STATISTICS_LEVEL parameter set to ALL at the session level, and retrieve the actual execution timing by passing in ‘ALLSTATS LAST’ as the format parameter for DBMS_XPLAN.DISPLAY_CURSOR, we see the following:The above slide shows that a single execution of the query required roughly 0.5 seconds (10% of the estimated time), required 42,751 consistent gets, and returned 1 row when the optimizer predicted that 943 rows would be returned. The good news is that the BCHR is 100%, or maybe that is not good news, depending on who you ask.

The final slide shows the expected execution plan output by DBMS_XPLAN.DISPLAY_CURSOR:The above output shows that the query should be executing in 0.01 seconds or less with the same value specified for the PART_ID, with just 16 consistent gets. While not checked, the clustering factor for the X_INV_TRANS_1 index on the PART_ID column will certainly be high, perhaps close to the number of rows in the table.

What caused the above problem? Assume that the OPTIMIZER_INDEX_COST_ADJ parameter is left at its default value of 100, and system statistics were collected with a command like the following:

EXEC DBMS_STATS.GATHER_SYSTEM_STATS('interval',interval=>30)

The above command captures system (CPU) statistics with a 30 minute time interval. The statistics describe the performance of the server, including CPU speed (possibly a measure of the number of consistent gets per second), single block physical read time, multi-block physical read time, average number of blocks read by multi-block reads, as well as several other statistics – these statistics are visible in the SYS.AUX_STATS$ view. Assume that the data is stored in ASSM tablespaces with AUTO segment space management, which gradually ramps up the extent size from 64KB (for the first 1MB of data) to 1MB and beyond. If DB_FILE_MULTIBLOCK_READ_COUNT is set to allow 1MB multi-block reads (or auto-tuned to allow 1MB multi-block reads), it is quite possible that the MBRC statistic in SYS.AUX_STATS$ could be set to a very high value, possibly close to 100, if system statistics are gathered shortly after bouncing the database. It is also possible that a clever DBA might have executed something like the following to force the MBRC statistic to a value that is close to what would be expected if full table scans involving physical reads might be required (DB_FILE_MULTIBLOCK_READ_COUNT was auto-tuned to 128):

EXEC DBMS_STATS.SET_SYSTEM_STATS('MBRC',100)

Since the optimizer assumes that every block needed for a query will result in a physical read, odd side-effects, such as high CPU usage with long execution times, may happen when every block needed is satisfied by blocks already in the buffer cache, and a popular value (possibly identifying 0.98% of the rows in the table) is specified on the initial hard parse when bind peeking is enabled (the default starting with Oracle 9.0.1).

Comments : 6 Comments »

Categories : Performance

SQL – RANK, MAX Analytical Functions, DECODE, SIGN

26 12 2009

December 26, 2009

A couple years ago the following question appeared on the comp.databases.oracle.misc Usenet group:
http://groups.google.com/group/comp.databases.oracle.misc/browse_thread/thread/22b36e3ca18490db

Table Structure:
Table1
symbol  orders  ttime
Requirement: Want to arrange all records, symbolwise, based on orders (asc order).
Among that, if a particular symbol have records in the range TTIME BETWEEN 9300 AND 1530, then I want to extract MIN(TTIME) within that range else MIN(TTIME) of whatever available records.

I want to achieve this using a single query.

Example:
Table1
symbol  orders  ttime
A       2       9300
A       2       9450
A       2       1030
A       2       9451
A       2       1530
A       2       1600
A       2       1700
B       3       9300
B       4       1600
C       3       1600
I want to get all records with maximum orders (in desc order) for each symbol.
Output:
Symbol  Orders  ttime
A       2       9300
A       2       9450
A       2       9451
A       2       1030
A       2       1530
A       2       1600
A       1       9300
A       2       1700
B       4       9300
B       4       1600
C       3       1600
Out of this subset I want to get all records with ttime falling between 9450 to 1530 to appear first in asc. if there is no record within this range then I want to go for normal asc order on ttime.
Ouput:
Symbol  Orders  ttime
A       2       9450
A       2       1030
A       2       1530
A       2       1600
A       2       9300
B       4       9450
B       4       1030
B       4       1600
C       3       1600
Finally I want to extract only first record
Final output:
A       2       9450
B       4       9300
C       3       1600

Are we dealing with time here? When is 9300 less than 1530? Why is 1 included in the above?

The set up:

CREATE TABLE T1 (
  SYMBOL CHAR(1),
  ORDERS NUMBER(10),
  TTIME NUMBER(10));

INSERT INTO T1 VALUES('A',2,9300);
INSERT INTO T1 VALUES('A',2,9450);
INSERT INTO T1 VALUES('A',2,10300);
INSERT INTO T1 VALUES('A',2,9451);
INSERT INTO T1 VALUES('A',2,15300);
INSERT INTO T1 VALUES('A',2,16000);
INSERT INTO T1 VALUES('A',2,17000);
INSERT INTO T1 VALUES('B',3,9300);
INSERT INTO T1 VALUES('B',4,16000);
INSERT INTO T1 VALUES('C',3,16000);

First, let’s find the maximum value for ORDERS for each SYMBOL:

SELECT
  SYMBOL,
  MAX(ORDERS) OVER (PARTITION BY SYMBOL) ORDERS,
  TTIME TTIME
FROM
  T1;

SYMBOL ORDERS  TTIME
A           2   9300
A           2   9450
A           2  10300
A           2   9451
A           2  15300
A           2  17000
A           2  16000
B           4  16000
B           4   9300
C           3  16000

You stated that if TTIME is between 9450 and 1530 (should it be 15300?), that it should take priority over other values of TTIME. The DECODE below determines if TTIME is between 9450 and 15300, if so it assigns a value of 10 to EXTRA_SORT, otherwise it assigns a value of 1 to EXTRA_SORT:

SELECT
  SYMBOL,
  MAX(ORDERS) OVER (PARTITION BY SYMBOL) ORDERS,
  TTIME TTIME,
  DECODE(SIGN(TTIME-9449.999),1,DECODE(SIGN(TTIME-15300.0001),-1,10,1),1) EXTRA_SORT
FROM
  T1;

SYMBOL ORDERS  TTIME  EXTRA_SORT
A           2   9300           1
A           2   9450          10
A           2  10300          10
A           2   9451          10
A           2  15300          10
A           2  17000           1
A           2  16000           1
B           4  16000           1
B           4   9300           1
C           3  16000           1

If we slide the above into an inline view, we can then rank the rows when sorted first on EXTRA_SORT and then on TTIME:

SELECT
  SYMBOL,
  ORDERS,
  TTIME,
  RANK() OVER (PARTITION BY SYMBOL ORDER BY EXTRA_SORT DESC,TTIME) POSITION
FROM
  (SELECT
    SYMBOL,
    MAX(ORDERS) OVER (PARTITION BY SYMBOL) ORDERS,
    TTIME TTIME,
    DECODE(SIGN(TTIME-9449.999),1,DECODE(SIGN(TTIME-15300.0001),-1,10,1),1) EXTRA_SORT
  FROM
    T1);

SYMBOL ORDERS  TTIME  POSITION
A           2   9450         1
A           2   9451         2
A           2  10300         3
A           2  15300         4
A           2   9300         5
A           2  16000         6
A           2  17000         7
B           4   9300         1
B           4  16000         2
C           3  16000         1

We can again slide the above into an inline view and extract only those with a POSITION value of 1:

SELECT
  SYMBOL,
  ORDERS,
  TTIME
FROM
  (SELECT
    SYMBOL,
    ORDERS,
    TTIME,
    RANK() OVER (PARTITION BY SYMBOL ORDER BY EXTRA_SORT DESC,TTIME)
POSITION
  FROM
    (SELECT
      SYMBOL,
      MAX(ORDERS) OVER (PARTITION BY SYMBOL) ORDERS,
      TTIME TTIME,
      DECODE(SIGN(TTIME-9449.999),1,DECODE(SIGN(TTIME-15300.0001),-1,10,1),1) EXTRA_SORT
    FROM
      T1)
  )
WHERE
  POSITION=1;

SYMBOL ORDERS  TTIME
A           2   9450
B           4   9300
C           3  16000

Comments : 5 Comments »

Categories : Analytic Functions, SQL

SQL – DENSE_RANK, PERCENT_RANK, and COUNT Analytical Functions

26 12 2009

December 26, 2009

A couple years ago the following question appeared on the comp.databases.oracle.misc Usenet group:
http://groups.google.com/group/comp.databases.oracle.misc/browse_thread/thread/9af4117466316d9a

I have to following problem:

I get X rows from a statement, these are sorted by a certain column, let’s say a numerical value.
Now I want to calculate the average of this numerical value, but the 10% with the lowest and the 10% with the highest value shall not be included in this calculation. So for example, if I get 20 rows, I need the average of the value in rows 3 to 18.

Currently I solved this with a very complicated statement, but I don’t know the built-in Oracle mathematical functions so I hope that there could be a way to do this with a better performance.

Let’s set up a short experiment:

CREATE TABLE T1 (C1 NUMBER(4));

INSERT INTO T1
SELECT
  ROWNUM
FROM
  DUAL
CONNECT BY
  LEVEL<=20;

We now have a table with 20 rows with numbers between 1 and 20.

Assuming that you are running a version of Oracle that supports analytical functions, the following returns the twenty rows with the relative ranking of each row, if the rows are sorted by C1 in descending order:

SELECT
  C1,
  DENSE_RANK() OVER (ORDER BY C1 DESC) DR,
  COUNT(C1) OVER (PARTITION BY 1) R
FROM
  T1;

        C1         DR          R
---------- ---------- ----------
        20          1         20
        19          2         20
        18          3         20
        17          4         20
        16          5         20
        15          6         20
        14          7         20
        13          8         20
        12          9         20
        11         10         20
        10         11         20
         9         12         20
         8         13         20
         7         14         20
         6         15         20
         5         16         20
         4         17         20
         3         18         20
         2         19         20
         1         20         20

A slight modification of the above, dividing the value of DENSE_RANK by the value of COUNT, and also including a PERCENT_RANK for comparison:

SELECT
  C1,
  DENSE_RANK() OVER (ORDER BY C1 DESC) DR,
  (DENSE_RANK() OVER (ORDER BY C1 DESC))/(COUNT(C1) OVER (PARTITION BY 1)) DRP,
  PERCENT_RANK() OVER (ORDER BY C1 DESC) PR
FROM
  T1;

        C1         DR        DRP         PR
---------- ---------- ---------- ----------
        20          1        .05          0
        19          2         .1 .052631579
        18          3        .15 .105263158
        17          4         .2 .157894737
        16          5        .25 .210526316
        15          6         .3 .263157895
        14          7        .35 .315789474
        13          8         .4 .368421053
        12          9        .45 .421052632
        11         10         .5 .473684211
        10         11        .55 .526315789
         9         12         .6 .578947368
         8         13        .65 .631578947
         7         14         .7 .684210526
         6         15        .75 .736842105
         5         16         .8 .789473684
         4         17        .85 .842105263
         3         18         .9 .894736842
         2         19        .95 .947368421
         1         20          1          1

The final cleanup is performed when the above is slid into an inline view, by using a WHERE clause:

SELECT
  SUM(T.C1) S
FROM
  (SELECT
    C1,
    DENSE_RANK() OVER (ORDER BY C1 DESC) DR,
    (DENSE_RANK() OVER (ORDER BY C1 DESC))/(COUNT(C1) OVER (PARTITION BY 1)) DRP,
    PERCENT_RANK() OVER (ORDER BY C1 DESC) PR
  FROM
    T1) T
WHERE
  T.DRP>0.1
  AND T.DRP<=0.9;

S
———-
168

A version that uses the PERCENT_RANK value:

SELECT
  SUM(T.C1) S
FROM
  (SELECT
    C1,
    DENSE_RANK() OVER (ORDER BY C1 DESC) DR,
    (DENSE_RANK() OVER (ORDER BY C1 DESC))/(COUNT(C1) OVER (PARTITION BY 1)) DRP,
    PERCENT_RANK() OVER (ORDER BY C1 DESC) PR
  FROM
    T1) T
WHERE
  T.PR BETWEEN 0.1 AND 0.9;

S
———-
168

Comments : 2 Comments »

Categories : Analytic Functions, SQL

SQL – COUNT Analytical Function, GROUP BY, HAVING

26 12 2009

December 26, 2009

A couple years ago the following question appeared on the comp.databases.oracle.misc Usenet group: http://groups.google.com/group/comp.databases.oracle.misc/browse_thread/thread/93bf8d1e75033d4c

I have a book table and in that table it has the book tile, publisher, and type of book it is. example mystery, scifi, etc…

I am trying to write a query that brings back a list of every pair of books that have the same publisher and same book type. I have been able to get the following code to work:
select publisher_code, type
from book
group by publisher_code, type
having count(*) > 1;
which returns the following results:
PU TYP
-- ---
JP MYS
LB FIC
PE FIC
PL FIC
ST SFI
VB FIC
I can not figure out how to get the book title and book code for the books that this result list represents, everything I have tried throws out an error.

My initial response follows:

I see two possible methods:

Slide the SQL statement that you have written into an inline view, join the inline view to your book table, and then use the publisher_code, type columns to drive back into your book table. The join syntax may look like one of the following: (publisher_code, type) IN (SELECT…) or b.publisher_code=ib.publisher_code and b.type=ib.type
Use analytical functions (COUNT() OVER…) to determine the number of matches for the same publisher_code, type columns. Then slide this SQL statement into an inline view to retrieve only those records with the aliased COUNT() OVER greater than 1. This has the benefit of retrieving the matching rows in a single pass.

—

The original poster then attempted to create a query to meet the requirements, but the query generated an error:

SQL> select title
  2  from book
  3  where publisher_code, type in
  4  (select publisher_code, type
  5  from book
  6  group by publisher_code, type
  7  having count(*) > 1);
where publisher_code, type in
                    *
ERROR at line 3:
ORA-00920: invalid relational operator

My reponse continues:

Very close to what you need. However, Oracle expects the column names to be wrapped in () … like this: where (publisher_code, type) in

The above uses a subquery, which may perform slow on some Oracle releases compared to the use of an inline view. Assume that I have a table named PART, which has columns ID, DESCRIPITION, PRODUCT_CODE, and COMMODITY_CODE, with ID as the primary key. I want to find ID, DESCRIPTION, and COMMODITY_CODE for all parts with the same DESCRIPTION and PRODUCT_CODE, where there are at least 3 matching parts in the group:

The starting point, which looks similar to your initial query:

SELECT
  DESCRIPTION,
  PRODUCT_CODE,
  COUNT(*) NUM_MATCHES
FROM
  PART
GROUP BY
  DESCRIPTION,
  PRODUCT_CODE
HAVING
  COUNT(*)>=3;

When the original query is slid into an inline view and joined to the original table, it looks like this:

SELECT
  P.ID,
  P.DESCRIPTION,
  P.COMMODITY_CODE
FROM
  (SELECT
    DESCRIPTION,
    PRODUCT_CODE,
    COUNT(*) NUM_MATCHES
  FROM
    PART
  GROUP BY
    DESCRIPTION,
    PRODUCT_CODE
  HAVING
    COUNT(*)>=3) IP,
  PART P
WHERE
  IP.DESCRIPTION=P.DESCRIPTION
  AND IP.PRODUCT_CODE=P.PRODUCT_CODE;

Here is the DBMS_XPLAN:

-------------------------------------------------------------------------------------------------------------------
| Id  | Operation             | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
-------------------------------------------------------------------------------------------------------------------
|*  1 |  HASH JOIN            |      |      1 |   1768 |  11525 |00:00:00.21 |    2748 |  1048K|  1048K| 1293K (0)|
|   2 |   VIEW                |      |      1 |   1768 |   1156 |00:00:00.11 |    1319 |       |       |          |
|*  3 |    FILTER             |      |      1 |        |   1156 |00:00:00.11 |    1319 |       |       |          |
|   4 |     HASH GROUP BY     |      |      1 |   1768 |  23276 |00:00:00.08 |    1319 |       |       |          |
|   5 |      TABLE ACCESS FULL| PART |      1 |  35344 |  35344 |00:00:00.04 |    1319 |       |       |          |
|   6 |   TABLE ACCESS FULL   | PART |      1 |  35344 |  35344 |00:00:00.04 |    1429 |       |       |          |
-------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   1 - access("IP"."DESCRIPTION"="P"."DESCRIPTION" AND "IP"."PRODUCT_CODE"="P"."PRODUCT_CODE")
   3 - filter(COUNT(*)>=3)

The query format using the subquery looks like this:

SELECT
  P.ID,
  P.DESCRIPTION,
  P.COMMODITY_CODE
FROM
  PART P
WHERE
  (DESCRIPTION,PRODUCT_CODE) IN
  (SELECT
    DESCRIPTION,
    PRODUCT_CODE
  FROM
    PART
  GROUP BY
    DESCRIPTION,
    PRODUCT_CODE
  HAVING
    COUNT(*)>=3);

The DBMS_XPLAN, note that Oracle 10.2.0.2 transformed the query above:

-----------------------------------------------------------------------------------------------------------------------
| Id  | Operation             | Name     | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------
|*  1 |  HASH JOIN RIGHT SEMI |          |      1 |      1 |  11525 |00:00:00.21 |    2748 |  1048K|  1048K| 1214K (0)|
|   2 |   VIEW                | VW_NSO_1 |      1 |   1768 |   1156 |00:00:00.12 |    1319 |       |       |          |
|*  3 |    FILTER             |          |      1 |        |   1156 |00:00:00.12 |    1319 |       |       |          |
|   4 |     HASH GROUP BY     |          |      1 |   1768 |  23276 |00:00:00.09 |    1319 |       |       |          |
|   5 |      TABLE ACCESS FULL| PART     |      1 |  35344 |  35344 |00:00:00.04 |    1319 |       |       |          |
|   6 |   TABLE ACCESS FULL   | PART     |      1 |  35344 |  35344 |00:00:00.01 |    1429 |       |       |          |
-----------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   1 - access("DESCRIPTION"="$nso_col_1" AND "PRODUCT_CODE"="$nso_col_2")
   3 - filter(COUNT(*)>=3)

Without allowing the automatic transformations in Oracle 10.2.0.2, the query takes _much_ longer than 0.21 seconds to complete.

The method using analytical functions starts like this:

SELECT
  P.ID,
  P.DESCRIPTION,
  P.COMMODITY_CODE,
  COUNT(*) OVER (PARTITION BY DESCRIPTION, PRODUCT_CODE) NUM_MATCHES
FROM
  PART P;

Then, sliding the above into an inline view:

SELECT
  ID,
  DESCRIPTION,
  COMMODITY_CODE
FROM
  (SELECT
    P.ID,
    P.DESCRIPTION,
    P.COMMODITY_CODE,
    COUNT(*) OVER (PARTITION BY DESCRIPTION, PRODUCT_CODE) NUM_MATCHES
  FROM
    PART P)
WHERE
  NUM_MATCHES>=3;

The DBMS_XPLAN for the above looks like this:

-----------------------------------------------------------------------------------------------------------------
| Id  | Operation           | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------
|*  1 |  VIEW               |      |      1 |  35344 |  11525 |00:00:00.31 |    1319 |       |       |          |
|   2 |   WINDOW SORT       |      |      1 |  35344 |  35344 |00:00:00.27 |    1319 |  2533K|   726K| 2251K (0)|
|   3 |    TABLE ACCESS FULL| PART |      1 |  35344 |  35344 |00:00:00.04 |    1319 |       |       |          |
-----------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter("NUM_MATCHES">=3)

Note that there is only one TABLE ACCESS FULL of the PART table in the above.  The execution time required 0.31 seconds to complete, which is greater than the first two approaches, but that is because the database server is concurrently still trying to resolve the query method using the subquery with no permitted transformations (5+ minutes later).

Subquery method with no transformations permitted:

---------------------------------------------------------------------------------------
| Id  | Operation            | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
---------------------------------------------------------------------------------------
|*  1 |  FILTER              |      |      1 |        |  11525 |00:46:21.46 |      38M|
|   2 |   TABLE ACCESS FULL  | PART |      1 |  35344 |  35344 |00:00:00.25 |    1429 |
|*  3 |   FILTER             |      |  29474 |        |   6143 |00:46:06.52 |      38M|
|   4 |    HASH GROUP BY     |      |  29474 |      1 |    613M|00:33:24.30 |      38M|
|   5 |     TABLE ACCESS FULL| PART |  29474 |  35344 |   1041M|00:00:02.54 |      38M|
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter( IS NOT NULL)
   3 - filter(("DESCRIPTION"=:B1 AND "PRODUCT_CODE"=:B2 AND COUNT(*)>=3))

Maxim Demenko provided another possible solution for the problem experienced by the original poster.

Comments : Leave a Comment »

Categories : Analytic Functions, SQL

Proving that 1=2, is Oracle Wrong to Short-Circuit an Execution Plan?

25 12 2009

December 25, 2009

Earlier this month I wrote a blog article that showed how Oracle behaves when the WHERE clause includes the predicate 1=2. But is the shortcut implemented by Oracle wrong? Will 1=2 never happen?

I attended a college course that covered just mathematical proofs (the name of the class escapes me at the moment – Discrete Mathematics?). The mathematics professor for that course proved that 1=2. That proof follows (I attempted to align the equal sign to make the proof easier to read):

Assumption: Let a = b
1.  a^2               = a^2
2.  a^2               = b^2
3.  a^2               = b * b
4.  a^2               = a * b
5.  a^2 - b^2         = a * b - b^2
6.  (a + b) * (a - b) = b * (a - b)
7.  a + b             = b
8.  b + b             = b
9.  2b                = b
10. 2 = 1

Mathematical Explanation of the Above Steps:
1.  Truth Statement
2.  Substitution
3.  Factor
4.  Substitution
5.  Subtract b^2 from Both Sides of the Equation
6.  Factor
7.  Divide Both Sides of the Equation by (a - b)
8.  Substitution
9.  Simplification
10. Divide Both Sides of the Equation by b

I think that there is a lesson in the above that may be applied to the understanding of Oracle databases. Let me ponder the lesson while you review the Faulty Quotes series of blog articles.

A couple references for mathematical proofs follow – how might the techniques of mathematical proofs be applied to understanding the logic built into Oracle Database?http://www.eiu.edu/~mathcs/mat2345/index/Webview/Slides/handout-Week02-2×3.pdf
http://www.math.harvard.edu/archive/23b_spring_05/proofs.pdf
http://en.wikibooks.org/wiki/Category:Mathematical_Proof
http://wapedia.mobi/en/Inductive_proof

Comments : 5 Comments »

Categories : Faulty Quotes

Miscellaneous Metalink Performance Articles 2

24 12 2009

12/24/2009

(Back to the Previous Post in the Series)

It appears that my previous post of this topic was well received (thanks Doug). My search through Metalink in July/August found quite a number of performance related articles – many more than I expected to find on the support site. Below are a couple more of those articles – some of these articles are offered as suggested further reading items for the topics in the chapters that I co-authored.

Doc ID 179668.1 “TROUBLESHOOTING: Tuning Slow Running Queries”
Doc ID 296377.1 “Handling and resolving unshared cursors/large version_counts”
Doc ID 43214.1 “AUTOTRACE Option in sqlplus” – lists all of the various SQL*Plus options for controlling the output of expected execution plans for SQL statements, and the runtime statistics.
Doc ID 215187.1 “SQLT (SQLTXPLAIN) – Enhanced Explain Plan and related diagnostic information for one SQL”
Doc ID 235530.1 “Methods for Obtaining a Formatted Explain Plan” – shows several ways to retrieve execution plans, including DBMS_XPLAN.DISPLAY_CURSOR.
Doc ID 41634.1 “TKProf Basic Overview” – quick reference guide for enabling a basic SQL trace, processing the trace file with TKPROF and understanding the contents of the resulting report.
Doc ID 39817.1 “Interpreting Raw SQL_TRACE and DBMS_SUPPORT.START_TRACE output” – shows how to decode a raw 10046 extended SQL trace file, includes keywords added in 11g.
Doc ID 779226.1 “Troubleshooting Oracle Net”
Doc ID 219968.1 “SQL*Net, Net8, Oracle Net Services – Tracing and Logging at a Glance”
Doc ID 216912.1 “How to Perform Client-Side Tracing of Programmatic Interfaces on Windows Platforms”
Doc ID 438452.1 “Performance Tools Quick Reference Guide”
Doc ID 394937.1 “Statistics Package (STATSPACK) Guide” – the 26 page reference for creating and understanding Statspack reports in 9.2.0.1 through 11.1.0.6.
Doc ID 215858.1 “Interpreting HANGANALYZE trace files to diagnose hanging and performance problems”
Doc ID 423153.1 “Understanding and Reading Systemstates” – shows how to read the contents of systemstate dumps .
Doc ID 370363.1 “CASE STUDY: Using Real-Time Diagnostic Tools to Diagnose Intermittent Database Hangs”
Doc ID 362791.1 “STACKX Core / Stack Trace Extraction Tool” – Unix/Linux utility that pulls the call stack information from core dump files.
Doc ID 352363.1 “LTOM – The On-Board Monitor User Guide” – tracing in the database and in the operating system.
Doc ID 301137.1 OS Watcher for Unix/Linux, Doc ID 433472.1 OS Watcher for Windows.
Doc ID 377152.1 “Best Practices for automatic statistics collection on Oracle 10g”
Doc ID 149560.1 “System Statistics: Collect and Display System Statistics (CPU and IO)”

Comments : Leave a Comment »

Categories : Performance

SORT_AREA_SIZE Affecting Sorts to Disk with PGA_AGGREGATE_TARGET Set 2?

23 12 2009

December 23, 2009

This is a follow-up to my post from a couple days ago (back to the previous post).

A small confession may be found below…

The good news is that I was able to reproduce the behavior on 64 bit Oracle 11.1.0.7 with an initial PGA_AGGREGATE_TARGET of 1.8GB. Below is the test script that I created – it took about 2 hours for SQL*Plus to scroll all of the returned data up the screen, regardless of how quickly the DBMS_XPLAN outputs indicate that the SQL statements executed.

SET PAGESIZE 2000
SET LINESIZE 140
SET ARRAYSIZE 100

SPOOL SortToDiskTest11.1.0.7-1.txt

DROP TABLE T1 PURGE;

CREATE TABLE T1 (
  C1 VARCHAR2(20),
  C2 NUMBER(12),
  C3 NUMBER(12));

/* Fill the test table to 1,000,000 rows: */
DECLARE
  X NUMBER;
BEGIN
  FOR X IN 1 .. 1000000
  LOOP
    INSERT INTO T1 VALUES (
      TO_CHAR(MOD( X ,100))||'-'||TO_CHAR(MOD( X ,100)),
      MOD( X ,500),
      X);
  END LOOP;
END;
/

COMMIT;

/* Gather statistics on the table: */
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T1');

/*+ Let's see where the sort statistics are right now: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/*+ Let's turn on a 10053 trace so that we can see what the cost-based optimizer is willing to report: */
ALTER SESSION SET TRACEFILE_IDENTIFIER = 'Default_Opt';
ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT 'STARTING POINT' FROM DUAL;

SPOOL OFF

/*+ Run a simple SQL statement to generate a sort, and gather statistics for DBMS_XPLAN: */
SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

/*+ Determine the DBMS_XPLAN for the query: */
SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

/* Let's check the sort statistics: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's look at the plan statistics for the SQL statement: */
SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Let's try again, this time with a 200MB PGA_AGGREGATE_TARGET rather than a 150MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 200' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

/*+ Let's check the sort statistics: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/*+ Let's look at the plan statistics for the SQL statement: */
SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Since we are still sorting to disk, let's increase the PGA_AGGREGATE_TARGET again and repeat the test: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=300M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 300' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's continue the test, dropping PGA_AGGREGATE_TARGET back to 200MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 200' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;
SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Let's drop the PGA_AGGREGATE_TARGET to 150MB and see if we have an optimal sort (no sort to disk): */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=150M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 150' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/*+ Let's again increase the PGA_AGGREGATE_TARGET to 200MB and see if we have an optimal sort (no sort to disk): */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 200' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Now, let's see if changing the SORT_AREA_SIZE helps remove the sort to disk: */
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=41943040;

SELECT 'ADJUST SAS' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Now, let's try a similar test with optimizer_features_enable set to 10.2.0.3 */
ALTER SESSION SET TRACEFILE_IDENTIFIER = 'OFE_10.2.0.3';
ALTER SYSTEM SET optimizer_features_enable='10.2.0.3';

ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT 'OFE_10.2.0.3' FROM DUAL;

SPOOL OFF

/* Let's first dump a DBMS_XPLAN with the estimated statistics. */
SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,NULL));

SPOOL OFF

/* Let's repeat, this time requesting the additional statistics from DBMS XPLAN: */
SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Repeat with pga_aggregate_target at 200MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 200' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Repeat with pga_aggregate_target at 100MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=100M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's try again, this time bumping the SORT_AREA_SIZE to roughly 40MB: */
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=41943040;

SELECT 'SAS' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's try again, this time using the optimizer setting for Oracle 10.2.0.2: */
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET TRACEFILE_IDENTIFIER = 'OFE_10.2.0.2';
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.2.0.2';

SELECT 'OFE_10.2.0.2' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's try again, this time using the optimizer setting for Oracle 10.1.0.4: */
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET TRACEFILE_IDENTIFIER = 'OFE_10.1.0.4';
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.1.0.4';

SELECT 'OFE_10.1.0.4' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's try again with pga_aggregate_target at 300MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=300M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 300' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Try again, reseting SORT_AREA_SIZE: */
ALTER SESSION SET SORT_AREA_SIZE=65536;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=0;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SAS 64KB' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Just to confirm, one more time: */
ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.1.0.4';
ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT 'OFE_10.1.0.4' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Try again, setting PGA_AGGREGATE_TARGET back to 200MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 200' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Trying again with PGA_AGGREGATE_TARGET at 100MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=100M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 100' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Let's be cruel to Oracle to see what happens with PGA_AGGREGATE_TARGET at 20MB: */
ALTER SYSTEM SET PGA_AGGREGATE_TARGET=20M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'PGA 20' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-1.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

CONNECT /@OR11 AS SYSDBA

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Let's see the hidden parameters (adapted from a script on Jonathan Lewis' website): */
SELECT
  UPPER(NAM.KSPPINM) NAME,
  VAL.KSPPSTVL VALUE,
  VAL.KSPPSTDF ISDEFAULT,
  DECODE(BITAND(VAL.KSPPSTVF,7),
    1,'MODIFIED',
    4,'SYSTEM MODIFIED',
    'FALSE') ISMODIFIED
FROM
  X$KSPPI NAM,
  X$KSPPSV VAL
WHERE
  NAM.INDX = VAL.INDX
  AND UPPER(NAM.KSPPINM) IN ('_SMM_MIN_SIZE','_SMM_MAX_SIZE')
ORDER BY
  UPPER(NAM.KSPPINM);

SPOOL OFF

The output of the above script follows, slightly cleaned up:

NAME             VALUE        
-------------- -------        
sorts (disk)      0        
sorts (memory) 3746        
sorts (rows)  33118        

'STARTINGPOINT             
--------------             
STARTING POINT             

PLAN_TABLE_OUTPUT          
----------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         0        
sorts (memory)    6291        
sorts (rows)   1196353        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000           

'PGA200                    
-------                    
PGA 200                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         0        
sorts (memory)    8948        
sorts (rows)   2217360        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000           

'PGA300                    
-------                    
PGA 300                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:03.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         0        
sorts (memory)   11357        
sorts (rows)   3367784        

'PGA200                    
-------                    
PGA 200                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         0        
sorts (memory)   13757        
sorts (rows)   4388065        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000           

'PGA150                    
-------                    
PGA 150                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.62 |    2715 |   2927 |   2927 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.62 |    2715 |   2927 |   2927 |    25M|  1845K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         1        
sorts (memory)   17889        
sorts (rows)   5558136        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               27058176                1889280         31464448 1 PASS                 23552                    
         1000000           

'PGA200                    
-------                    
PGA 200                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.44 |    2715 |   2927 |   2927 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.44 |    2715 |   2927 |   2927 |    25M|  1843K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         2        
sorts (memory)   20280        
sorts (rows)   6578105        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               26975232                1887232         31464448 1 PASS                 23552                    
         1000000           

'ADJUSTSAS                 
----------                 
ADJUST SAS                 

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         2        
sorts (memory)   22832        
sorts (rows)   7729292        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000           

'OFE_10.2.0.               
------------               
OFE_10.2.0.3               

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 1                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

-----------------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |       |       |       |  5700 (100)|          |
|   1 |  SORT ORDER BY     |      |  1000K|    13M|    45M|  5700   (1)| 00:01:09 |
|   2 |   TABLE ACCESS FULL| T1   |  1000K|    13M|       |   757   (1)| 00:00:10 |
-----------------------------------------------------------------------------------

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 1                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         2        
sorts (memory)   23828        
sorts (rows)   9864585        

'PGA200                    
-------                    
PGA 200                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         2        
sorts (memory)   26015        
sorts (rows)  10884097        

'SA                        
---                        
SAS                        

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.50 |    2717 |   2928 |   2928 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.50 |    2717 |   2928 |   2928 |    25M|  1845K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         3        
sorts (memory)   29925        
sorts (rows)  11926332        

'OFE_10.2.0.               
------------               
OFE_10.2.0.2               

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.41 |    2717 |   2928 |   2928 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.41 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         4        
sorts (memory)   32075        
sorts (rows)  13076448        

'OFE_10.1.0.               
------------               
OFE_10.1.0.4               

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.50 |    2717 |   2928 |   2928 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.50 |    2717 |   2928 |   2928 |    25M|  1845K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         5        
sorts (memory)   34911        
sorts (rows)  14099400        

'PGA300                    
-------                    
PGA 300                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.41 |    2717 |   2928 |   2928 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.41 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         6        
sorts (memory)   40285        
sorts (rows)  15276005        

'SAS64KB                   
--------                   
SAS 64KB                   

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.31 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.31 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         6        
sorts (memory)   42443        
sorts (rows)  16295460        

'OFE_10.1.0.               
------------               
OFE_10.1.0.4               

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.31 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.31 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         6        
sorts (memory)   44626        
sorts (rows)  17315395        

'PGA200                    
-------                    
PGA 200                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         6        
sorts (memory)   48666        
sorts (rows)  18487624        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   48210000               40624128                2264064         36109312 OPTIMAL             
         1000000           

'PGA100                    
-------                    
PGA 100                    

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.56 |    2717 |   2928 |   2928 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.56 |    2717 |   2928 |   2928 |    25M|  1844K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE        
-------------- -------
sorts (disk)         7        
sorts (memory)   58650        
sorts (rows)  22038592        

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   48210000               27012096                1888256         21013504 1 PASS                 23552                    
         1000000           

'PGA20                     
------                     
PGA 20                     

PLAN_TABLE_OUTPUT          
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                  
-------------------------------------                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3             

Plan hash value: 2148421099

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.90 |    2746 |   3816 |   3816 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.90 |    2746 |   3816 |   3816 |    25M|  1848K| 4141K (3)|   26624 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000           
         1000000   48210000               27150336                1892352          4240384 3 PASSES               26624                    
         1000000           

NAME            VALUE ISDEFAULT ISMODIFIED             
--------------- ----- --------- ----------  
_SMM_MAX_SIZE    4096 TRUE      FALSE            
_SMM_MIN_SIZE     128 TRUE      FALSE

Note in the above that the sorts to disk did not happen when PGA_AGGREGATE_TARGET was adjusted to 200MB, then to 300MB, and then to 200MB. The first sort to disk happened when the parameter was set to 150MB, and the second sort to disk happened when the parameter was bumped back up to 200MB.

Prior to posting the original blog article “SORT_AREA_SIZE Affecting Sorts to Disk with PGA_AGGREGATE_TARGET Set?”, I read a document titled “Advanced Management of Working Areas In Oracle 9I/10G” that was written by Joze Senegacnik in 2004 (http://joze-senegacnik.blogspot.com/). When I originally created the test case a couple years ago, I suspected that there was a delayed reaction when the PGA_AGGREGATE_TARGET parameter is adjusted, but I did not know what caused that delayed reaction. Joze’s article seems to indicate that the delay is caused by the CKPT process publishing the memory bounds only every three seconds. So, I re-ran the test, adding the following command after each adjustment of the PGA_AGGREGATE_TARGET parameter:

host sleep 30

The above command executes the operating system’s sleep command, causing SQL*Plus to pause for 30 seconds. The results (just displaying the 200MB to 150MB to 200MB section of the output)?

'PGA200  
-------  
PGA 200  

PLAN_TABLE_OUTPUT 
----------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0         
-------------------------------------         
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3    

Plan hash value: 2148421099                   

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE
-------------- -------
sorts (disk)         0
sorts (memory)   14062
sorts (rows)   4390586

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000  
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000  

'PGA150  
-------  
PGA 150  

PLAN_TABLE_OUTPUT 
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0         
-------------------------------------         
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3    

Plan hash value: 2148421099                   

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.47 |    2715 |   2927 |   2927 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.47 |    2715 |   2927 |   2927 |    25M|  1846K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME             VALUE
-------------- -------
sorts (disk)      1
sorts (memory)16553
sorts (rows)5541481

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000  
         1000000   24110000               27076608                1890304         31464448 1 PASS                 23552                    
         1000000  

'PGA200  
-------  
PGA 200  

PLAN_TABLE_OUTPUT 
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0         
-------------------------------------         
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3    

Plan hash value: 2148421099                   

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.31 |    2712 |       |       |          |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.31 |    2712 |    38M|  2211K|   34M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

NAME             VALUE
-------------- -------
sorts (disk)      1
sorts (memory)18956
sorts (rows)6561447

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000  
         1000000   24110000               40624128                2264064         36109312 OPTIMAL             
         1000000

As can be seen from the above, pausing 30 seconds after each adjustment of the PGA_AGGREGATE_TARGET parameter provides CKPT enough time to perform its processing (3 seconds likely would have been sufficient).

—

I recall reading an article on Jonathan Lewis’ site about a SORT_AREA_SIZE bug (search for SAS Bug on his site). I then wondered if adjusting the SORT_AREA_SIZE parameter when WORKAREA_SIZE_POLICY = MANUAL at the session level is also subject to the roughly 3 second delay. I put together the following test case, which differs quite a bit from the one referenced on the asktom.oracle.com site:

SET PAGESIZE 2000
SET LINESIZE 140
SET ARRAYSIZE 100

SPOOL SortToDiskTest11.1.0.7-3.txt

DROP TABLE T1 PURGE;

CREATE TABLE T1 (
  C1 VARCHAR2(20),
  C2 NUMBER(12),
  C3 NUMBER(12));

/* Fill the test table to 1,000,000 rows: */
DECLARE
  X NUMBER;
BEGIN
  FOR X IN 1 .. 1000000
  LOOP
    INSERT INTO T1 VALUES (
      TO_CHAR(MOD( X ,100))||'-'||TO_CHAR(MOD( X ,100)),
      MOD( X ,500),
      X);
  END LOOP;
END;
/

COMMIT;

/* Gather statistics on the table: */
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T1');

/*+ Let's see where the sort statistics are right now: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/*+ Let's turn on a 10053 trace so that we can see what the cost-based optimizer is willing to report: */
ALTER SESSION SET TRACEFILE_IDENTIFIER = 'SORT_AREA';
ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT 'STARTING POINT' FROM DUAL;

SPOOL OFF

/*+ Run a simple SQL statement to generate a sort, and gather statistics for DBMS_XPLAN: */
SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

/*+ Determine the DBMS_XPLAN for the query: */
SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

/* Let's check the sort statistics: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's look at the plan statistics for the SQL statement: */
SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

ALTER SESSION SET WORKAREA_SIZE_POLICY = MANUAL;

/* Let's try again, this time with a 20MB SORT_AREA_SIZE: */
ALTER SESSION SET SORT_AREA_SIZE=20971520;
ALTER SYSTEM FLUSH SHARED_POOL;
host sleep 30

SELECT 'SORT_AREA_SIZE=20971520' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

/*+ Let's check the sort statistics: */
SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/*+ Let's look at the plan statistics for the SQL statement: */
SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

/* Since we are still sorting to disk, let's increase the SORT_AREA_SIZE to 25MB again and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=26214400;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SORT_AREA_SIZE=26214400' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Since we are still sorting to disk, let's increase the SORT_AREA_SIZE to 30MB again and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=31457280;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SORT_AREA_SIZE=31457280' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's increase the SORT_AREA_SIZE to 40MB again and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SORT_AREA_SIZE=41943040' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's change the SORT_AREA_SIZE to 20MB again and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=20971520;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SORT_AREA_SIZE=20971520 #2' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's change the SORT_AREA_SIZE to 40MB again and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT 'SORT_AREA_SIZE=41943040 #2' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's change the SORT_AREA_SIZE to 20MB again, sleep, and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=20971520;
ALTER SYSTEM FLUSH SHARED_POOL;
host sleep 30
SELECT 'SORT_AREA_SIZE=20971520 #3' FROM DUAL;

SPOOL OFF

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

/* Let's change the SORT_AREA_SIZE to 40MB again, sleep, and repeat the test: */
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SYSTEM FLUSH SHARED_POOL;
host sleep 30

SELECT 'SORT_AREA_SIZE=41943040 #3' FROM DUAL;

SPOOL OFF
SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SPOOL SortToDiskTest11.1.0.7-3.txt APPEND

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

SPOOL OFF

The output of the above script follows:

NAME              VALUE            
--------------- -------
sorts (disk)          0            
sorts (memory)     3591            
sorts (rows)      33734            

'STARTINGPOINT                     
--------------                     
STARTING POINT                     

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

-------------------------------------------------------------------------------------------------------------------------                  
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |  OMem |  1Mem | Used-Mem |                  
-------------------------------------------------------------------------------------------------------------------------                  
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.50 |    2712 |   2707 |       |       |          |                  
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.50 |    2712 |   2707 |    38M|  2211K|   34M (0)|                  
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.03 |    2712 |   2707 |       |       |          |                  
-------------------------------------------------------------------------------------------------------------------------                  

NAME              VALUE            
--------------- -------
sorts (disk)          0            
sorts (memory)     6019            
sorts (rows)    1066679            

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000                   
         1000000   24110000               40624128                2264064         36109312 OPTIMAL                                         
         1000000                   

'SORT_AREA_SIZE=2097152            
-----------------------            
SORT_AREA_SIZE=20971520            

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.50 |    2715 |   2926 |   2926 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.50 |    2715 |   2926 |   2926 |    25M|  1844K|   18M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME              VALUE            
--------------- -------            
sorts (disk)          1            
sorts (memory)     8435            
sorts (rows)    2086785            

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000                   
         1000000   24110000               27002880                1888256         18884608 1 PASS                 23552                    
         1000000                   

'SORT_AREA_SIZE=2621440            
-----------------------            
SORT_AREA_SIZE=26214400            

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.44 |    2715 |   2926 |   2926 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.44 |    2715 |   2926 |   2926 |    25M|  1843K|   23M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME              VALUE            
--------------- -------
sorts (disk)          2            
sorts (memory)    10870            
sorts (rows)    3107315            

'SORT_AREA_SIZE=3145728            
-----------------------            
SORT_AREA_SIZE=31457280            

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.47 |    2715 |   2926 |   2926 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.47 |    2715 |   2926 |   2926 |    25M|  1843K|   27M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME              VALUE            
--------------- -------
sorts (disk)          3            
sorts (memory)    13542            
sorts (rows)    4127908            

'SORT_AREA_SIZE=4194304            
-----------------------            
SORT_AREA_SIZE=41943040            

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

----------------------------------------------------------------------------------------------------------------                           
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |                           
----------------------------------------------------------------------------------------------------------------                           
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |                           
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|                           
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |                           
----------------------------------------------------------------------------------------------------------------                           

NAME              VALUE            
--------------- -------
sorts (disk)          3            
sorts (memory)    16027            
sorts (rows)    5148428            

'SORT_AREA_SIZE=20971520#2         
--------------------------         
SORT_AREA_SIZE=20971520 #2         

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.44 |    2715 |   2926 |   2926 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.44 |    2715 |   2926 |   2926 |    25M|  1843K|   18M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME              VALUE            
--------------- -------
sorts (disk)          4            
sorts (memory)    18474            
sorts (rows)    6168736            

'SORT_AREA_SIZE=41943040#2         
--------------------------         
SORT_AREA_SIZE=41943040 #2         

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

----------------------------------------------------------------------------------------------------------------                           
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |                           
----------------------------------------------------------------------------------------------------------------                           
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |                           
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|                           
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |                           
----------------------------------------------------------------------------------------------------------------                           

NAME              VALUE            
--------------- -------
sorts (disk)          4            
sorts (memory)    22413            
sorts (rows)    7206482            

'SORT_AREA_SIZE=20971520#3         
--------------------------         
SORT_AREA_SIZE=20971520 #3         

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.53 |    2715 |   2926 |   2926 |       |       |          |         |
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.53 |    2715 |   2926 |   2926 |    25M|  1843K|   18M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

NAME              VALUE            
--------------- -------
sorts (disk)          5            
sorts (memory)    24788            
sorts (rows)    8226615            

'SORT_AREA_SIZE=41943040#3         
--------------------------         
SORT_AREA_SIZE=41943040 #3         

PLAN_TABLE_OUTPUT                  
--------------------------------------------------------------------------------------------------------------------------------------------
SQL_ID  auy15crnrmkc3, child number 0                                                  
-------------------------------------                                                  
SELECT /*+ GATHER_PLAN_STATISTICS */   * FROM   T1 ORDER BY   C2,   C3                 

Plan hash value: 2148421099        

----------------------------------------------------------------------------------------------------------------                           
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |                           
----------------------------------------------------------------------------------------------------------------                           
|   0 | SELECT STATEMENT   |      |      1 |        |   1000K|00:00:00.28 |    2712 |       |       |          |                           
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:00.28 |    2712 |    38M|  2211K|   34M (0)|                           
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    2712 |       |       |          |                           
----------------------------------------------------------------------------------------------------------------                           

NAME              VALUE            
--------------- -------
sorts (disk)          5            
sorts (memory)    27252            
sorts (rows)    9247027            

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE                    
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------                    
         1000000                   
         1000000                          40624128                2264064         36109312 OPTIMAL                                         
         1000000

The output shows that at least in Oracle 11.1.0.7 there is no delay in the implementation of the SORT_AREA_SIZE at the session level when WORKAREA_SIZE_POLICY = MANUAL is set at the session level and the SORT_AREA_SIZE parameter is adjusted. The test query continued to perform a sort to disk with SORT_AREA_SIZEs OF 30MB or smaller. Every time the SORT_AREA_SIZE was set to 40MB without a delay, Oracle switched to an in-memory optimal sort. I might need to repeat the final test with an older release of Oracle to see if the same behavior is present (I will update this post if I see a change).

Back on the original topic, I now wonder if the occasional sorts to disk that I saw when experiementing with the SORT_AREA_SIZE parameter with PGA_AGGREGATE_TARGET set (and not being modified) in a production environment could have been a side-effect of the “CKPT process publishing the memory bounds only every three seconds” as was described in Joze’s paper. In short, the behavior was an unrelated cause and effect.

Comments : 3 Comments »

Categories : Performance

Faulty Quotes 4 – Buffer Cache Hit Ratio (BCHR)

22 12 2009

December 22, 2009

(Back to the Previous Post in the Series) (Forward to the Next Post in the Series)

Over the years a lot of conflicting advice for maintaining Oracle databases has surfaced in books, magazine articles, websites, Usenet (and other discussion forums), and presentations. Even as the year 2009 draws to a close there are Oracle forum posts asking for help in improving the buffer cache hit ratio (BCHR).

Here is an interesting history of recommendations for the buffer cache hit ratio (BCHR), as presented in user conferences for a particular ERP platform:

1999:

Find the statistics in V$SYSSTAT or REPORT.TXT
  Logical reads = db block gets + consistent gets
  Buffer cache hit ratio (%) = ((logical reads - physical reads)/logical reads)*100
Ratio should be above 90%, if not increase the shared_pool_size

——————–

2001:

HIT_RATIO=((DB BLOCK GETS + CONSISTENT GETS - PHYSICAL READS)/
           (DB BLOCK GETS + CONSISTENT GETS))*100
90 – 100 % Few physical reads. Current size is optimal if not a bit high. OK to remove some buffers if memory needed elsewhere.
70 – 89 % Buffer cache has low to moderate number of physical reads. Consider resizing if there is a serious problem with memory on the Oracle database.
0 – 69 % Buffer cache is experiencing moderate to high number of physical reads. Consider adding more buffers to the buffer cache.

——————–

2004:

Be aware that the Cache Hit Ratio isn’t necessarily the definitive answer; although it can be a good indication that something is up.

——————–

2006:

Bad performance indicators – Poor cache hit ratios

——————–

2008:

The buffer cache hit ratio was never mentioned.

——————–
If the above is any indication, recommending the use of the buffer cache hit ratio as a performance metric is on the decline. This seems to be confirmed by the helpful replies that typically follow requests for improving the buffer cache hit ratio in various forum threads. But, there is a twist. Do we know what the buffer cache hit ratio is supposed to measure? A search of the Internet, including Metalink, finds a number of formulas for calculating the buffer cache hit ratio. Some of those forumulas follow:

(logical reads – physical reads)/logical reads

——

1 – (physical reads)/(consistent gets + db block gets)

——

(logical reads)/(logical reads + physical reads)

——

1 – (physical reads – physical reads direct – physical reads direct (lob))/(consistent gets + db block gets)

——

1 – (physical reads – (physical reads direct + physical reads direct (lob)))/(db block gets + consistent gets – (physical reads direct + physical reads direct (lob)))

So many formulas, and then we have this one from the 11g R2 Performance Tuning Guide:
http://download.oracle.com/docs/cd/E11882_01/server.112/e16638/memory.htm#g61128

1 - (('physical reads cache') / ('consistent gets from cache' + 'db block gets from cache')

Nice… and a bit confusing. Pick a formula, any formula. But, what is it measuring?

A search of the Internet finds a large number of articles discussing the benefits of using the buffer cache hit ratio as a performance metric. For example:

www.cryer.co.uk/brian/oracle/tuning_bchr.htm

“If the cache-hit ratio goes below 90% then:
* For Oracle 8 and earlier: increase the initialisation parameter DB_BLOCK_BUFFERS.
* For Oracle 9 onwards: increate the initialisation parameter DB_CACHE_SIZE.”

——————–

publib.boulder.ibm.com/tividd/td/oracle2/GC32-0454-00/en_US/HTML/oracle20rgf2k81.htm

“In general, if the hit ratio is below 90%, and the dictionary cache has been tuned, increase the init.ora parameter DB_BLOCK_BUFFERS to increase the hit ratio.”

——————–

oracle-training.cc/teas_elite_troub_5.htm

“One must use delta statistics over time to come up with a meaningful value for the ratio, and high logical I/O values can definitely be a leading cause of bad execution times. However, when properly computed, the buffer cache hit ratio is an excellent indicator of how often the data requested by users is found in RAM instead of disk, a fact of no small importance.”

——————–

www.iselfschooling.com/mcpfcd1/iscdpfh05vrl.htm

“As a DBA, you are responsible for monitoring and calculating the Buffer Cache Hit Ratio in the SGA memory in case of performance problems…

— If the Buffer Cache Hit Ratio is more than 90% then there is no problem.
— If the Buffer Cache Hit Ratio is between 70% and 90% then there could be a problem.
— And if the Buffer Cache Hit Ratio is less than 70%, there is definitely a problem and the Buffer Cache size needs to be increased.”

——————–

praetoriate.com/t_%20tuning_data_buffer_hit_ratio.htm

“To summarize, our goal as the Oracle DBA is to allocate as much RAM as possible to the data buffers without causing the database server to page-in RAM. Whenever the hourly data buffer hit ratio falls below 90 percent, we should add buffers to the block buffers.”

——————–

dba-oracle.com/art_dbazine_bust_ora_myth_bust.htm

“New myth – Ratio-based Oracle tuning is meaningless…
However, most OLTP systems and systems in which the working set of frequently-referenced data are not cached will greatly benefit from ratio-based tuning. Oracle Corporation recognizes that monitoring SGA usage ratios and adjusting the size of the SGA regions can have a dramatic impact on system performance, and this is the foundation of Oracle10g Automatic Memory Management (AMM) in which Oracle calculates the point of diminishing marginal return from adding data buffers”

——————–

oracle-training.cc/t_viewing_sga_performance.htm

“The data buffer hit ratio can provide data similar to v$db_cache_advice , and most Oracle tuning professionals use both tools to monitor the effectiveness of data buffers and monitor how AMM adjusts the sizes of the buffer pools.”

——————–

oracle-training.cc/t_allocating_oracle_buffer_caches.htm

“The DBHR is a common metric used by Oracle tuning experts to measure the propensity of a row to be in the data buffer. For example, a hit ratio of 95 percent means that 95 percent of row requests were already present in the data buffer, thereby avoiding an expensive disk I/O. In general, as the size of the data buffers increases, the DBHR will also increase and approach 100 percent.”

——————–

oracle-training.cc/phys_55.htm

“In order for the DBA to determine how well the buffer pools are performing, it is necessary to measure the hit ratio at more frequent intervals. Calculating the DBHR for Oracle8 and beyond is more complicated than earlier versions, but the results enable the administrator to achieve a higher level of tuning than was previously possible.”

——————–

dba-oracle.com/t_buffer_cache_hit_ratio_value.htm

“But the question remains about the value of this metric to the DBA.

Once I’ve tuned and stabilized my systems, I notice that the metrics create repeatable ‘signatures’, patterns of usage that form the baselines for the exception alerts.

First, we establish a ‘exception threshold’ for the BCHR, (e.g. +- 20%), and compare that deviation to the historical average, normalized by the historical average per hour and the day-of-the-week.”

——————–

http://forums.oracle.com/forums/thread.jspa?messageID=1701819

“Many folks misunderstand that bit about ‘setting your own BHR’, and falsely conclude that it’s a useless metric. It’s not useless.

Of course, doing lots pre-buffered I/O (consistent gets) will increase the BHR, that’s the whole point, right?

That does not mean that the BHR is useless, it just means that it’s not a panacea.

The BHR remains very useful for detecting ‘undersized’ data buffers, where the working-set is not cached and Oracle is forced to do extra physical reads. . . .

If the BHR was totally useless, why does Oracle continue to include it in OEM alert thresholds, and STATSPACK and AWR reports?

The BHR is just like any other Oracle metric, you must understand its limitations and proper usage. It’s just one of many tools…”

——————–

http://groups.google.com/group/comp.databases.oracle.server/browse_thread/thread/1a946dbe8dcfa71e (Many, many pros and cons)

“BCHR can be manipulated. That is nothing new. All stats can be inflated in similar manners. But that doesn’t make them all meaningless. Given everything else being equal, high BCHR is always better than low BHCR… BCHR alone is not meant to tell performance. If it does, we would not have to look at anything else… BCHR alone does not tell you about overall performance. It simply tell you the disk I/O percentage. It is an indicator, a very meaningful one.”

A number of examples advocating the use of the buffer cache hit ratio also exist in Oracle books, many of which may be viewed, in part, through Google searches:

“Creating a self-tuning Oracle database: automating Oracle9i Dynamic SGA”
http://books.google.com/books?id=_rQ-WyEJTgkC&pg=PA51

“The data buffer hit ratio (DBHR) measures the propensity for a block to be cached in the buffer pool, and the goal of the DBA is to keep as many of the frequently used Oracle blocks in buffer memory as possible…”

——————–

“Oracle9i High-Performance Tuning with STATSPACK”
http://books.google.com/books?id=gsFC1D1LmvQC&pg=RA1-PA264

“From this chart we can clearly see that the DBHR dropped below the recommended value of 90 percent at 3:00 A.M., 4:00 A.M., and 10:00 A.M. each day… The problem here is that the DBHR is dropping low at 10:00 A.M., a prime-time online period.”

——————–

“Oracle Tuning: The Definitive Reference”
http://books.google.com/books?id=bxHDtttb0ZAC&pg=PA217

“A query like the one that follows can be used to see a metric’s behavior for the recent time period. For example, the following query shows data buffer hit ratio history for the last hour.”

——————–

“Expert Oracle Database 11g Administration” – Page 190

“this is why the buffer cache hit ratio, which measures the percentage of time users accessed the data they needed from the buffer cache (rather than requiring a disk read), is such an important indicator of performance of the Oracle instance.”

The author provides a link on page 1161 to an article authored by Cary Millsap which discusses why a higher buffer cache hit ratio may not be ideal. This is definitely a step in the right direction regarding the buffer cache hit ratio, but it might be better to simply ignore the statistic.

——————–

“Oracle Database 10g Performance Tuning Tips & Techniques”
http://books.google.com/books?id=omq9aRx8s0EC&pg=P750
http://books.google.com/books?id=omq9aRx8s0EC&pg=PT806

“Some DBAs (usually those trying to sell you a tuning product) minimize the importance of hit ratios (proactive tuning) and focus completely on waits (reactive tuning), since focusing on waits is a great way to quickly solve the current burning problems. By monitoring the Instance Efficiency section (and using all of STATSPACK and Enterprise Manager), the DBA will combine reactive and proactive tuning and will find some problems before the users scream or wait events hit the top 5 list. Hit ratios are one important piece of the puzzle (so are waits).”

“Hit ratios are a great barometer of the health of your system. A large increase or drop from day to day is an indicator of a major change that needs to be investigated.”

“Hit Ratio = (Logical Reads – Physical Reads) / Logical Reads” “The buffer hit ratio should be above 95 percent. If it is less than 95 percent, you should consider increasing the size of the data cache by increasing the DB_CACHE_SIZE initialization parameter (given that physical memory is available to do this).”

——————–

There are, of course, very strong counter-points to using the buffer cache hit ratio (BCHR) as a tuning metric. The first two are very well written, detailed articles:

http://jonathanlewis.wordpress.com/2007/09/05/hit-ratios-2/ (a very detailed blog entry)

“Ratios are highly suspect for monitoring purposes. If you think a ratio is helpful, think carefully about whether you should be monitoring the two underlying values instead.

The buffer cache hit ratio (BCHR) is a particularly bad example of the genre as there are so many events that make the attempt to correlate BCHR and performance meaningless.”

——————–

http://richardfoote.wordpress.com/2007/12/16/buffer-cache-hit-ratios-useful-or-not/ (a very detailed blog entry)

“The biggest problem of all with the BCHR is that regardless of it’s values, or whether it goes up or down or remains unchanged, we need to perform precisely the same database checks regardless as it doesn’t tell us whether the ‘health’ of the database has improved, got worse or remains unchanged.”

——————–

http://www.orafaq.com/wiki/Improving_Buffer_Cache_Hit_Ratio

“Many DBAs do their best to get a 99% or better buffer cache hit ratio, but quickly discover that the performance of their database isn’t improving as the hit ratio gets better.”

——————–

http://books.google.com/books?id=dN805AoUyc8C&pg=PA365#v=onepage&q=&f=false

“The evidence that hit ratios are unreliable is overwhelming, and similar ratio fallacies occurring in other industries are well documented.”

——————–

“Oracle Performance Troubleshooting: With Dictionary Internals SQL & Tuning”
http://books.google.com/books?id=ELVz1O8Z0qoC&pg=PA38

“For example, there have been a number of benchmarks done to prove that a 99-100% buffer cache hit ratio does not mean that a database is running well. A high cache hit ratio can be observed while the database is literally at a standstill.”

——————–

Cary Millsap’s “Why a 99%+ Database Buffer Cache Hit Ratio is Not Ok”
http://www.oradream.com/pdf/Why%20a%2099%20Cahe%20Hit%20Ratio%20is%20Not%20OK.pdf

“Many tuning professionals and textbook authors sell advice encouraging their customers to enjoy the performance virtues of Oracle database buffer cache hit ratios that approach 100%. However, database buffer cache hit ratio is not a reliable system performance metric. Buffer cache hit ratios above 99% usually indicate particularly serious SQL inefficiencies”

——————–

http://forums.oracle.com/forums/thread.jspa?messageID=3492715

“The data buiffer hit ratio has limited value”

——————–

http://books.google.com/books?id=OOJ5qMytsoYC&pg=PA561

“This ratio should not be used as an indicator of database performance health. Much Oracle software documentation touts the database buffer cache hit ratio as being one of the most important tuning metrics. In my opinion and that of many others this statement is often a complete and absolute falsity.”

——————–

http://www.jlcomp.demon.co.uk/Cache_advice.html

“So if someone tells you that the buffer cache hit ratio must be a good thing because Oracle has based their v$db_cache_advice technology on it, then they are displaying a lack of understanding about the deficiencies of the buffer cache hit ratio in particular, and how LRU (least recently used) caching mechanisms work in general.”

——————–

http://groups.google.com/group/comp.databases.oracle.server/browse_thread/thread/7ef93e3ef8963a29 (Howard J. Rogers)

“Who cares?

Are your users complaining about performance?

If not, then your hit ratio is just fine and dandy, whatever it happens to be.

The more general point here is that, *** and *** notwithstanding, the buffer cache hit ratio is an absolutely abysmal way of tuning anything. It can sometimes offer a useful corollary to other statistics; to allow you to distinguish between two otherwise equally plausible causes for, for example, free buffer waits. But as a performance tuning goal in its own right? Furgedaboudit.”

——————–

http://groups.google.com/group/comp.databases.oracle.server/browse_thread/thread/7508e23d122b27a2/ (Richard Foote)

“And it only take *one* piece of what I technically define as ‘Crap’ code to both inflate the BHR to incredibly high levels whilst at the same time killing or impacting *database* performance.

I’m probably more sympathetic to BHRs than many. However, it provides only one very small piece in the tuning puzzle, one that needs to be put into perspective. It can be used as an indicator of whether the buffer cache is set way to low/high and nothing more. And what it’s actual *value* is of little consequence, there is no such thing as an ideal value x.

Does a 99.9% BHR mean the database/buffer cache/sql is well tuned. Possibly.

Does a 99.9% BHR mean the database/buffer cache/sql is poorly tuned. Possibly.

So what does a 99.9% BHR actually mean and represent? Without being able to answer this question in it’s fullness, the figure is meaningless.

You get the point.”

——————–

“Tuning Oracle Without Cache-Hit Ratios”
http://www.quest.com/whitepapers/TuningOracleWithoutCacheHit_new.pdf

“From time immemorial, Oracle performance tuning has the infamous label of witchcraft, secretly practiced by an elite group of individuals, who allegedly use voodoo to cure the performance problems of an Oracle system. To compound this misperception there exists many thousands of pages of published material that propagate the idea of tuning Oracle with cache-hit ratios.”

“Bottom line, Oracle tuning efforts need to be based on isolating and pinpointing bottlenecks (the disease) not cache-hit ratios (the symptoms).”

——————–

Of course, tools are available to help correct a low buffer cache hit ratio:

From the book “Optimizing Oracle Performance”
http://books.google.com/books?id=dN805AoUyc8C&pg=PA368

Connor McDonald’s “Choose any hit ratio”
http://www.oracledba.co.uk/tips/choose.htm

Jonathan Lewis’ “A higher buffer hit ratio is a good thing. (17-Jan-2001)” script
http://www.jlcomp.demon.co.uk/myths.html

Comments : 6 Comments »

Categories : Faulty Quotes

Faulty Quotes 3 – Contradictory Information

21 12 2009

December 21, 2009

(Back to the Previous Post in the Series) (Forward to the Next Post in the Series)

Frustrated by some of the confusing and contradictory information I have encountered on the Internet over the years, I put together a “cheat sheet” to help identify useful information. The “cheat sheet” lists several questions that one might consider when reviewing books and web articles prior to changing parameters (or implementing other changes) based on the information found in those sources:

Is a specific Oracle release mentioned in the book or article? What was true, or thought to be true, with release 8.0.5 might not be true or even a good idea with release 11.2.0.1.
Does the article have a publication date, and is there a revision history that identifies the date and what modifications were made to the article? Articles which change from one day to the next without knowing what changed, and why the article changed, are difficult to use as justification for changes to the initialization parameters.
Are there any articles by other authors on the Internet which agree with the author’s suggestions or sharply disagree with the author’s suggestions? If Oracle’s official documentation strongly disagrees with the contents of the article, which of the two sources are correct? Should the advice be deemed an over-generalization which worked as a fix for a one time problem that is now advertised as something all DBAs should do as a first step in performance tuning?
Is there reproducible evidence that supports the claims made? Or, is the majority of the justification similar to “I have seen it a 100 times” or “a DBA at a fortune 50 company said to do this” or “I have been doing this for 25 years, and you should too”?
Does the parameter actually control the behavior which it is purported to control, and are there any potential side effects from modifying the parameter?

Comments : Leave a Comment »

Categories : Faulty Quotes

Miscellaneous Metalink Performance Articles

21 12 2009

December 21, 2009

(Forward to the Next Post in the Series)

A couple months ago I scanned through Metalink looking for interesting articles. I found a couple that seem to be well written, most with recent modification dates, that someone out there might enjoy reading when troubleshooting performance problems. Hopefully, the documents still exist on the Metalink replacement.

Doc ID 233112.1 “START HERE – Diagnosing Query Tuning Problems” – basically a click to jump to the specific problem being experienced.
Doc ID 745216.1 “Query Performance Degradation – Upgrade Related – Recommended Actions” – a tree like structure for performance tuning.
Doc ID 398838.1 “FAQ: Query Tuning Frequently Asked Questions” – another tree like structure.
Doc ID 223806.1 “Query with Unchanged Execution Plan is Slower than Previously” – another tree like structure.
Doc ID 387394.1 “Query using Binds is Suddenly Slow”
Doc ID 604256.1 “Why is a Particular Query Slower on One Machine than Another?” – another tree like structure.
Doc ID 372431.1 “Troubleshooting: Tuning a New Query”
Doc ID 163563.1 “Troubleshooting: Advanced Query Tuning” – another tree like structure
Doc ID 122812.1 “How to Tune a Query that Cannot be Modified”
Doc ID 67522.1 “Diagnosing Why a Query is Not Using an Index”
Doc ID 69992.1 “Why is my hint ignored?”
Doc ID 163424.1 “How to Identify a Hot Block within the Buffer Cache”
Doc ID 223117.1 “Tuning I/O-related waits” – another tree structure
Doc ID 402983.1 “Database Performance FAQ” – mentions pstack, system state dumps, 10046 traces, AWR/Statspack
Doc ID 66484.1 “Which Optimizer is Being Used”
Doc ID 271196.1 “Automatic SQL Tuning – SQL Profiles”
Doc ID 276103.1 “Performance Tuning Using 10g Advisors and Manageability Features”
Doc ID 463288.1 “How to generate an outline with a good plan loaded into the shared_pool”
Doc ID 43718.1 “View: V$SESSION_WAIT Reference”

Comments : 4 Comments »

Categories : Performance

SORT_AREA_SIZE Affecting Sorts to Disk with PGA_AGGREGATE_TARGET Set?

20 12 2009

December 20, 2009

(Forward to the Follow-Up Post)

A couple of years ago I was under the impression that increasing the SORT_AREA_SIZE parameter could reduce the number of sorts to disk in Oracle 10g R2 with the PGA_AGGREGATE_TARGET parameter set, using dedicated server processes, and with the WORKAREA_SIZE_POLICY set to AUTO (yes, confirmed to be set in the session). Why? Because I saw it happen a couple of times when troubleshooting performance problems. But, the sorts to disk still happened on occasion when there seemed to be no reason why Oracle would not be able to perform an optimal, in-memory workarea execution.

Thankfully, someone questioned my earlier testing that produced results which were inconsistent with the documentation. I then attempted to produce a test case to prove myself wrong. I set out to trigger a query to perform a sort to disk, and then by changing only the SORT_AREA_SIZE parameter, have Oracle switch to perform an in-memory workarea execution for the query. Once the test case produced that result, I attempted to identify the trigger that caused Oracle to switch. The test case appears in the following Usenet thread:
http://groups.google.com/group/comp.databases.oracle.server/msg/e0b8bea300e45ae4

Line wrapping is a problem in the above thread, so I have reproduced the test case below:

I was able to produce a test case on the base patch of Oracle 10.2.0.2 on Win 32. The following required roughly 10 hours of testing and analysis. I built a new database instance with the following pfile:

############################################
aq_tm_processes=1
background_core_dump=partial
cluster_database=FALSE
compatible=10.2.0.1.0
control_files=("C:\oracle\OraData\LT\ctlLT01.ctl", "C:\oracle\flash_recovery_area\LT\ctlLT02.ctl")
control_file_record_keep_time=7
cursor_sharing=FORCE
cursor_space_for_time=true
db_block_size=8192
db_cache_advice=on
db_block_checking=false
db_block_checksum=typical
db_domain=world
db_files=200
db_file_multiblock_read_count=16
db_flashback_retention_target=1440
db_name=LT
db_keep_cache_size=400M
db_recycle_cache_size=10M
db_recovery_file_dest_size=14000M
db_recovery_file_dest=C:\oracle\flash_recovery_area
db_unique_name=LT
db_writer_processes=1
global_names=false
instance_name=LT
java_pool_size=1M
job_queue_processes=10
log_archive_format=arc%s_%r.%t
log_buffer=1048576
log_checkpoint_interval=65536
log_checkpoint_timeout=3600
log_checkpoints_to_alert=false
max_dump_file_size=202400
nls_language=american
nls_territory=america
O7_DICTIONARY_ACCESSIBILITY=TRUE
open_cursors=1000
open_links=4
optimizer_dynamic_sampling=2
optimizer_features_enable=10.1.0.4   #Needed to overcome ORA-600 on production DB
optimizer_index_caching=0
optimizer_index_cost_adj=100
optimizer_mode=ALL_ROWS
pga_aggregate_target=150M
plsql_code_type=INTERPRETED
processes=210
query_rewrite_enabled=FALSE
query_rewrite_integrity=TRUSTED
recyclebin=ON
remote_login_passwordfile=EXCLUSIVE
service_names=LT
sessions=236
session_cached_cursors=200
sga_max_size=1100M
sga_target=900M
star_transformation_enabled=FALSE
statistics_level=typical
timed_statistics=true
transactions=259
transactions_per_rollback_segment=5
undo_management=AUTO
undo_retention=1800
undo_tablespace=ROLLBACK_DATA
workarea_size_policy=auto
background_dump_dest=C:\oracle\product\10.2.0\admin\LT\bdump
core_dump_dest=C:\oracle\product\10.2.0\admin\LT\cdump
user_dump_dest=C:\oracle\product\10.2.0\admin\LT\udump
utl_file_dir=C:\oracle\product\10.2.0\admin\LT\udump
############################################

Create a new user named TESTING

Create a table for testing and fill in the table:

CREATE TABLE T1 (
  C1 VARCHAR2(20),
  C2 NUMBER(12),
  C3 NUMBER(12));

DECLARE
  X NUMBER;
BEGIN
  FOR X IN 1 .. 1000000
  LOOP
    INSERT INTO T1 VALUES (
      TO_CHAR(MOD( X ,100))||'-'||TO_CHAR(MOD( X ,100)),
      MOD( X ,500),
      X);
  END LOOP;
END;
/

COMMIT;

EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T1');

Let’s see where the sort statistics are right now:

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                   VALUE
----------------- ----------
sorts (disk)               0
sorts (memory)          2092
sorts (rows)            8342

Let’s turn on a 10053 trace so that we can see what the cost-based optimizer is willing to report:

ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

Run a simple SQL statement to generate a sort, and gather statistics for DBMS_XPLAN:

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

Determine the DBMS_XPLAN for the query:

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:04.19 |    2709 |   5627 |   2926 |    25M|  1843K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.02 |    2706 |   2701 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

This query used the temp tablespace as indicated by the Used-Tmp column.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          43 Area size:      156672 Max Area size:     7864320
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         4 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH DEFAULT VALUES
  _smm_min_size = 153 KB
  _smm_max_size = 30720 KB

Note:
Max Area size is reported as 7,864,320 (7680 KB), while _smm_max_size is reported as 30720 KB. On Oracle 10.2.0.3, when optimizer_features_enable is set to 10.2.0.3, both will be reported with the same values.
5% of 150MB = 7.5MB = 7864320 bytes
150MB/5 = 30MB = 31457280 bytes = 30720 KB

From my notes recorded during the second read through of “Cost-Based Oracle Fundamentals”:
sort width = same as the “max intermediate sort width” from a 10032 trace file
area size = amount of memory available for processing data – number reported will be smaller than the SORT_AREA_SIZE due to overhead
max area size = maximum memory available for sorting
degree = degree of parallelism for the query
blocks to sort = row_size*rows/db_block_size
row size = estimate of the average row size in bytes
rows = computed (filtered) cardinality of the table
initial runs = optimizer’s estimate of the number of sort runs that will be dumped to disk
merge passes = always at least one, even for an in-memory sort, counts the number of times the entire data set will be written to and read from disk in the event of a disk sort
IO cost/pass = cost of doing a single merge pass
total IO sort cost – combines the cost per pass with the number of passes
total CPU cost – CPU component of the cost – measured in CPU operations
total temp space used = estimated amount of temporary space needed for the sort operation

Let’s check the sort statistics:

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)                1
sorts (memory)           3557
sorts (rows)          1015256

Let’s look at the plan statistics for the SQL statement:

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48407000               26966016                1887232         31472640 1 PASS                 23552

From the Oracle documentation:
TEMP_SPACE: Temporary space usage of the operation (sort or hash-join) as estimated by the optimizer’s cost-based approach. For statements that use the rule-based approach, this column is null.
ESTIMATED_OPTIMAL_SIZE: Estimated size (in KB) required by this work area to execute the operation completely in memory (optimal execution). This is either derived from optimizer statistics or from previous executions.
ESTIMATED_ONEPASS_SIZE: Estimated size (in KB) required by this work area to execute the operation in a single pass. This is either derived from optimizer statistics or from previous executions.
LAST_MEMORY_USED: Memory size (in KB) used by this work area during the last execution of the cursor
LAST_EXECUTION: Indicates whether this work area ran using OPTIMAL, ONE PASS, or under ONE PASS memory requirement (MULTI-PASS), during the last execution of the cursor
LAST_TEMPSEG_SIZE: Temporary segment size (in bytes) created in the last instantiation of this work area. This column is null if the last instantiation of this work area did not spill to disk.

The SQL statement required a 1 pass sort to disk.

Let’s try again, this time with a 200MB PGA_AGGREGATE_TARGET rather than a 150MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:01.82 |    2709 |   2926 |   2926 |    25M|  1843K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

This query used the temp tablespace as indicated by the Used-Tmp column.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          58 Area size:      208896 Max Area size:    10485760
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         3 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH DEFAULT VALUES
  pga_aggregate_target = 204800 KB
  _smm_min_size        = 204 KB
  _smm_max_size        = 40960 KB

Note that the Area Size, Max Area Size, _smm_min_size, and _smm_max_size parameters increased, yet a sort to disk was still required.

Let’s check the sort statistics:

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)            2
sorts (memory)       4504
sorts (rows)      2022536

The sorts (disk) value increased again.

Let’s look at the plan statistics for the SQL statement:

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48407000               26966016                1887232         31472640 1 PASS                 23552

The SQL statement required a 1 pass sort to disk.

Since we are still sorting to disk, let’s increase the PGA_AGGREGATE_TARGET again and repeat the test:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=300M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:01.55 |    2706 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |       |       |          |
----------------------------------------------------------------------------------------------------------------

The Used-Tmp column did not print, so no sort to disk was required.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          88 Area size:      314368 Max Area size:    15728640
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH DEFAULT VALUES
  pga_aggregate_target = 307200 KB
  _smm_min_size        = 307 KB
  _smm_max_size        = 61440 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)                2
sorts (memory)           6223
sorts (rows)          3035178

As shown above, no sort to disk was needed with PGA_AGGREGATE_TARGET at 300MB, but a sort to disk was required at 200MB.

Let’s continue the test, dropping PGA_AGGREGATE_TARGET back to 200MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:01.55 |    2706 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |       |       |          |
----------------------------------------------------------------------------------------------------------------

The Used-Tmp column did not print, so no sort to disk was required. So, increasing the PGA_AGGREGATE_TARGET from 150MB to 200MB apparently forces a sort to disk, while decreasing the PGA_AGGREGATE_TARGET from 300MB to 200MB does not require a sort to disk.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          58 Area size:      208896 Max Area size:    10485760
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         3 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH DEFAULT VALUES
  pga_aggregate_target = 204800 KB
  _smm_min_size        = 204 KB
  _smm_max_size        = 40960 KB

The values are the same as for the first run with PGA_AGGREGATE_TARGET at 200MB.

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                   VALUE
----------------- ----------
sorts (disk)               2
sorts (memory)          7224
sorts (rows)         4043937

The sorts (disk) value did not increase.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48407000               35688448                2136064         31722496 OPTIMAL

Note that LAST_TEMPSEG_SIZE is null in the above, and that the last execution was OPTIMAL.

Let’s drop the PGA_AGGREGATE_TARGET to 150MB and see if we have an optimal sort (no sort to disk):

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=150M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_SESSION_NUM SQLADDR     SQLHASH SQL_ID        TABLESPACE     CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#    EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ------------------- -------- ---------- ------------- -------------- --------- --------- ---------- ---------- ---------- ---------- ----------
TESTING  TESTING  54C631C8          1 507DB66C 2019831986 4ww44m1w68c5k TEMPORARY_DATA1TEMPORARY SORT             201       2825         23       2944          1

(2944 blocks * 8KB block size = 24,117,248 bytes)

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:02.85 |    2709 |   2926 |   2926 |    25M|  1843K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

This query used the temp tablespace as indicated by the Used-Tmp column.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          43 Area size:      156672 Max Area size:     7864320
      Degree:               1
      Blocks to Sort:       1 Row size:           76 Total Rows:              1
      Initial runs:         1 Merge passes:        0 IO Cost / pass:          0
      Total IO sort cost: 0      Total CPU sort cost: 11511282
      Total Temp space used: 0

PARAMETERS WITH DEFAULT VALUES
  pga_aggregate_target                = 153600 KB
  _smm_min_size                       = 153 KB
  _smm_max_size                       = 30720 KB

Problem in the 10053 trace file? Note the “Total Temp space used: 0”

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                   VALUE
----------------- ----------
sorts (disk)               3
sorts (memory)          9997
sorts (rows)         5068937

The sorts (disk) value did increase.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48407000               26966016                1887232         31472640 1 PASS                 23552

Let’s again increase the PGA_AGGREGATE_TARGET to 200MB and see if we have an optimal sort (no sort to disk):

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_SESSION_NUM SQLADDR     SQLHASH SQL_ID        TABLESPACE CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#    EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ------------------- -------- ---------- ------------- -------------------------------  --------- --------- ---------- ---------- ----------
TESTING  TESTING  54C631C8          1 507DB66C 2019831986 4ww44m1w68c5k TEMPORARY_DATA1  TEMPORARY SORT       201          9         23       2944          1

The above shows a sort to disk in progress.

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:01.80 |    2709 |   2926 |   2926 |    25M|  1843K|   30M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

The DBMS_XPLAN shows that a sort to disk was required.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          58 Area size:      208896 Max Area size:    10485760
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         3 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH DEFAULT VALUES
  pga_aggregate_target = 204800 KB
  _smm_min_size        = 204 KB
  _smm_max_size        = 40960 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)                4
sorts (memory)          11776
sorts (rows)          6081798

The sorts (disk) value did increase.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48407000               26966016                1887232         31472640 1 PASS                 23552

The above shows a 1 pass sort to disk was required.

Now, let’s see if changing the SORT_AREA_SIZE helps remove the sort to disk, as I suggested in my previous post:

ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=41943040;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

(No rows)

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1004K|   1000K|00:00:01.57 |    2706 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1004K|   1000K|00:00:00.01 |    2706 |       |       |          |
----------------------------------------------------------------------------------------------------------------

No sort to disk was required.

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          58 Area size:      208896 Max Area size:    10485760
      Degree:               1
      Blocks to Sort:    3197 Row size:           26 Total Rows:        1004199
      Initial runs:         3 Merge passes:        1 IO Cost / pass:       1734
      Total IO sort cost: 4931      Total CPU sort cost: 992293034
      Total Temp space used: 48407000

PARAMETERS WITH ALTERED VALUES (for some reason, these did not show the first time the query was parsed - I had to force a second parse):
  sort_area_size                      = 41943040
  sort_area_retained_size             = 41943040

PARAMETERS WITH DEFAULT VALUES
  _smm_min_size                       = 204 KB
  _smm_max_size                       = 40960 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)                4
sorts (memory)          12126
sorts (rows)          7084470

The sorts (disk) value did not increase.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
          113401   48407000               35688448                2136064         31722496 OPTIMAL

Note that LAST_TEMPSEG_SIZE is null in the above, and that the last execution was OPTIMAL. So, as I originally suggested, changing the SORT_AREA_SIZE for the session helped by removing the sort to disk. Or not – I suspect the second time the query needed to be parsed, it would not have required a sort to disk.

—-

Now, let’s try a similar test on Oracle 10.2.0.3 with a pre-existing database that was started with the following parameters:

optimizer_features_enable=10.2.0.3
optimizer_index_caching=0
optimizer_index_cost_adj=100
pga_aggregate_target=300M
query_rewrite_enabled=FALSE
sga_max_size=1100M
sga_target=900M
sort_area_retained_size  not specified
sort_area_size  not specified
workarea_size_policy=auto

ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

Let’s first dump a DBMS_XPLAN with the estimated statistics.

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,NULL));

-----------------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |       |       |       |  5302 (100)|          |
|   1 |  SORT ORDER BY     |      |  1000K|    13M|    46M|  5302   (6)| 00:00:29 |
|   2 |   TABLE ACCESS FULL| T1   |  1000K|    13M|       |   604   (9)| 00:00:04 |
-----------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:         358 Area size:      314368 Max Area size:    62914560
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 307 KB
  _smm_max_size = 61440 KB    {= 62914560 bytes}

Note: 5% of 300MB = 15MB = 15728640 bytes, 300MB/5 = 60MB = 62,914,560

Oracle 10.2.0.3 with optimizer_features_enable set to 10.2.0.3 reports Max Area size at 62914560 bytes, which is identical to _smm_max_size, which is 1/5 of the pga_aggregate_target. In the above, Oracle 10.2.0.2 reported Max Area size at roughly 5% of the pga_aggregate_target.

Let’s repeat, this time requesting the additional statistics from DBMS XPLAN:

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.57 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME              VALUE
-----------------------
sorts (disk)          0
sorts (memory)     4069
sorts (rows)    3027233

No sort to disk was required.

Repeat with pga_aggregate_target at 200MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.56 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:         238 Area size:      208896 Max Area size:    41943040
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 204 KB
  _smm_max_size = 40960 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)                0
sorts (memory)           6283
sorts (rows)          4044290

No sort to disk required.

Repeat with pga_aggregate_target at 100MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=100M;
ALTER SYSTEM FLUSH SHARED_POOL;

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:07.21 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

    SORT resource      Sort statistics
      Sort width:         118 Area size:      131072 Max Area size:    20971520
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 128 KB
  _smm_max_size = 20480 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                        VALUE
---------------------- ----------
sorts (disk)                    1
sorts (memory)               7785
sorts (rows)              5058041

This time, a sort to disk was required.

Let’s try again, this time bumping the SORT_AREA_SIZE to roughly 40MB:

ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=41943040;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:04.34 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:         118 Area size:      131072 Max Area size:    20971520
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size                       = 128 KB
  _smm_max_size                       = 20480 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                        VALUE
---------------------- ----------
sorts (disk)                    2
sorts (memory)               9699
sorts (rows)              6073631

A sort to disk was still required, so changing SORT_AREA_SIZE did not help – of course the SORT_AREA_SIZE would have been double the value of _smm_max_size…

Let’s try again, this time using the optimizer setting for Oracle 10.2.0.2:

ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.2.0.2';

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:04.36 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:         118 Area size:      131072 Max Area size:    20971520
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 128 KB
  _smm_max_size = 20480 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                        VALUE
---------------------- ----------
sorts (disk)                    3
sorts (memory)              11435
sorts (rows)              7088591

Still a sort to disk.

Let’s try again, this time using the optimizer setting for Oracle 10.1.0.4:

ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.1.0.4';

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:04.30 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          28 Area size:      131072 Max Area size:     5242880
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         5 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 128 KB
  _smm_max_size = 20480 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                   VALUE
----------------- ----------
sorts (disk)               4
sorts (memory)         12907
sorts (rows)         8102004

Still a sort to disk, but note that Max Area size no longer matches the value for _smm_max_size.

Let’s try again with pga_aggregate_target at 300MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=300M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.58 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          88 Area size:      314368 Max Area size:    15728640
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 307 KB
  _smm_max_size = 61440 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                   VALUE
----------------- ----------
sorts (disk)               4
sorts (memory)         14532
sorts (rows)         9116171

No sort to disk, but SORT_AREA_SIZE is still set to roughly 40MB.

Try again, reseting SORT_AREA_SIZE:

ALTER SESSION SET SORT_AREA_SIZE=65536;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=0;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.56 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          88 Area size:      314368 Max Area size:    15728640
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 307 KB
  _smm_max_size = 61440 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                  VALUE
---------------- ----------
sorts (disk)              4
sorts (memory)        16400
sorts (rows)       10132013

No sort to disk, changing SORT_AREA_SIZE did not change anything.

Just to confirm, in a new session:

ALTER SYSTEM FLUSH SHARED_POOL;
ALTER SESSION SET OPTIMIZER_FEATURES_ENABLE='10.1.0.4';
ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.56 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          88 Area size:      314368 Max Area size:    15728640
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         2 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 307 KB
  _smm_max_size = 61440 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)            4
sorts (memory)      17859
sorts (rows)     11145435

No sort to disk with no SORT_AREA_SIZE specified.

Try again, setting PGA_AGGREGATE_TARGET back to 200MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.56 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          58 Area size:      208896 Max Area size:    10485760
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         3 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size = 204 KB
  _smm_max_size = 40960 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

-------------- ----------
sorts (disk)            4
sorts (memory)      19443
sorts (rows)     12159195

Still no sort to disk.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48243000               35688448                2136064         31722496 OPTIMAL

This SQL statement used an optimal sort (no sort to disk).

Trying again with PGA_AGGREGATE_TARGET at 100MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=100M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:04.30 |    2717 |   2928 |   2928 |    25M|  1843K|   20M (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:          28 Area size:      131072 Max Area size:     5242880
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:         5 Merge passes:        1 IO Cost / pass:       1270
      Total IO sort cost: 4456      Total CPU sort cost: 981216297
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size  = 128 KB
  _smm_max_size  = 20480 KB

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------------------
sorts (disk)            5
sorts (memory)      27009
sorts (rows)     14227511

A sort to disk was required.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48243000               26984448                1887232         21013504 1 PASS                 23552

A 1 pass sort to disk was required.

Let’s be cruel to Oracle to see what happens with PGA_AGGREGATE_TARGET at 20MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=20M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /*+ GATHER_PLAN_STATISTICS */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(Executed from a different session while the above was running):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_SESSION_NUM SQLADDR     SQLHASH SQL_ID        TABLESPACE CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#   EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ------------------- -------- ---------- ------------- ---------------------------------------- --------- ---------- ---------- ----------
TESTING  TESTING  54C654DC         70 50FD66AC  580783698 98h83a8j9w3kk TEMPORARY_DATA1 TEMPORARY SORT        201     895625         23      2944          1

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

--------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem | Used-Tmp|
--------------------------------------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:12.33 |    2730 |   2931 |   2931 |    25M|  1844K| 4141K (1)|   23552 |
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |      0 |      0 |       |       |          |         |
--------------------------------------------------------------------------------------------------------------------------------------------

From 10053 trace:

ORDER BY sort
    SORT resource      Sort statistics
      Sort width:           4 Area size:      131072 Max Area size:     1048576
      Degree:               1
      Blocks to Sort:    3186 Row size:           26 Total Rows:        1000729
      Initial runs:        25 Merge passes:        3 IO Cost / pass:       1270
      Total IO sort cost: 6996      Total CPU sort cost: 1059674733
      Total Temp space used: 48243000

Under PARAMETERS WITH DEFAULT VALUES:
  _smm_min_size                       = 128 KB
  _smm_max_size                       = 4096 KB

The execution plan shows a sort to disk was required.

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /*+ GATHER_PLAN_STATISTICS */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
         1000000   48243000               27012096                1888256          4240384 1 PASS                 23552

Still a 1 pass sort to disk.

Let’s see the hidden parameters (adapted from a script on Jonathan Lewis’ website):

SELECT
  UPPER(NAM.KSPPINM) NAME,
  VAL.KSPPSTVL VALUE,
  VAL.KSPPSTDF ISDEFAULT,
  DECODE(BITAND(VAL.KSPPSTVF,7),
    1,'MODIFIED',
    4,'SYSTEM MODIFIED',
    'FALSE') ISMODIFIED
FROM
  X$KSPPI NAM,
  X$KSPPSV VAL
WHERE
  NAM.INDX = VAL.INDX
  AND UPPER(NAM.KSPPINM) IN ('_SMM_MIN_SIZE','_SMM_MAX_SIZE')
ORDER BY
  UPPER(NAM.KSPPINM);

NAME            VALUE        ISDEFAULT ISMODIFIED
-------- ------------------------------------------
_SMM_MAX_SIZE   4096         TRUE      FALSE
_SMM_MIN_SIZE    128         TRUE      FALSE

—-

I performed a similar test on Oracle 10.2.0.2 with the July 2006 CPU on Win x64, and the results were similar to those of Oracle 10.2.0.3.

In summary, as the “Oracle Database Performance Tuning Guide 10g Release 2” Pg 7-38 (PDF page 146) documentation states, “sizing of work areas for all sessions becomes automatic and the *_AREA_SIZE parameters are ignored by all sessions running in that mode.” There is apparently an odd quirk that once in a while, the first time a SQL statement is parsed, a sort to disk may be required, at least under the base patch of Oracle 10.2.0.2. This lead me, incorrectly, to believe that setting the SORT_AREA_SIZE to a larger value and re-executing the query actually removed the sort to disk – but it was actually the second parse that resulted in the removal of the sort to disk. This test case disproves my suggestion that the SORT_AREA_SIZE has any impact on Oracle 10.2.0.2 when all sessions are set to auto for the WORKAREA_SIZE_POLICY. It is possible to modify the WORKAREA_SIZE_POLICY at the session level, and then the SORT_AREA_SIZE setting takes effect for that session.

This displays the optimizer’s expected execution statistics:

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,NULL));

-----------------------------------------------------------------------------------
| Id  | Operation          | Name | Rows  | Bytes |TempSpc| Cost (%CPU)| Time     |
-----------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |      |       |       |       |  5302 (100)|          |
|   1 |  SORT ORDER BY     |      |  1000K|    13M|    46M|  5302   (6)| 00:00:29 |
|   2 |   TABLE ACCESS FULL| T1   |  1000K|    13M|       |   604   (9)| 00:00:04 |
-----------------------------------------------------------------------------------

This displays the actual execution statistics for the last run:

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

----------------------------------------------------------------------------------------------------------------
| Id  | Operation          | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |  OMem |  1Mem | Used-Mem |
----------------------------------------------------------------------------------------------------------------
|   1 |  SORT ORDER BY     |      |      1 |   1000K|   1000K|00:00:03.57 |    2712 |    34M|  2086K|   30M (0)|
|   2 |   TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:02.00 |    2712 |       |       |          |
----------------------------------------------------------------------------------------------------------------

Here is a new test run on Oracle 10.2.0.2 July 2006 CPU Win x64. I did not run any explain plans during this test, although I did enable a 10053 trace:

First test at 150MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=150M;
ALTER SYSTEM FLUSH SHARED_POOL;

Current sort statistics:

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                    VALUE
------------------ ----------
sorts (disk)               87
sorts (memory)       28169866
sorts (rows)        675027667

No other sorts in process – an otherwise idle database instance:

SELECT
  *
FROM
  V$SORT_USAGE;

no rows selected

Run the query:

ALTER SESSION SET EVENTS '10053 TRACE NAME CONTEXT FOREVER, LEVEL 1';

SELECT /* FIND_ME */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_ADDR     SESSION_NUM SQLADDR             SQLHASH SQL_ID        TABLESPACE                      CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#     EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ---------------- ----------- ---------------- ---------- ------------- ------------------------------- --------- --------- ---------- ---------- ---------- ---------- ----------
TESTING  TESTING  000007FF9E4C0160       61714 000007FF929494C8 1262288602 0gzdjb55mtzqu TEMPORARY_DATA1                 TEMPORARY SORT             201      37641         23       2944          1

The above shows that the query is actively sorting to the temp tablespace.

The sorts (disk) statistic increased:

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                  VALUE
---------------- ----------
sorts (disk)             88
sorts (memory)     28170875
sorts (rows)      676034807

The execution statistics for the plan shows the estimates and actuals. A 1 pass sort to disk:

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /* FIND_ME */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
                   48522000               26975232                1887232         31472640 1 PASS                 23552

Second test at 200MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=200M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /* FIND_ME */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_ADDR     SESSION_NUM SQLADDR             SQLHASH SQL_ID        TABLESPACE                      CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#     EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ---------------- ----------- ---------------- ---------- ------------- ------------------------------- --------- --------- ---------- ---------- ---------- ---------- ----------
TESTING  TESTING  000007FF9E4C0160       61714 000007FF94EA7488 2019831986 4ww44m1w68c5k TEMPORARY_DATA1                 TEMPORARY SORT             201      29705        23       2944          1

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)           89
sorts (memory)   28172666
sorts (rows)    677048890

The execution statistics for the plan shows the estimates and actuals. A 1 pass sort to disk:

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /* FIND_ME */%'
  AND S.SQL_ID=SP.SQL_ID;

The execution statistics for the plan shows the estimates and actuals. A 1 pass sort to disk the first time this query was parsed with PGA_AGGREGATE_TARGET=200M:

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
                   48522000               26975232                1887232         31472640 1 PASS                 23552

Third test at 200MB (note changing SORT_AREA_SIZE is not the cause of the sort to disk disappearing):

ALTER SESSION SET SORT_AREA_SIZE=41943040;
ALTER SESSION SET SORT_AREA_RETAINED_SIZE=41943040;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /* FIND_ME */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

no rows selected

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)           89
sorts (memory)   28173729
sorts (rows)    678058329

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /* FIND_ME */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
                   48522000               40551424                2262016         36044800 OPTIMAL

Fourth test at 300MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=300M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /* FIND_ME */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

no rows selected

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)           89
sorts (memory)   28175547
sorts (rows)    679074619

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /* FIND_ME */%'
  AND S.SQL_ID=SP.SQL_ID;

LAST_OUTPUT_ROWS TEMP_SPACE ESTIMATED_OPTIMAL_SIZE ESTIMATED_ONEPASS_SIZE LAST_MEMORY_USED LAST_EXECU LAST_TEMPSEG_SIZE
---------------- ---------- ---------------------- ---------------------- ---------------- ---------- -----------------
                   48522000               40551424                2262016         36044800 OPTIMAL

Fifth test at 150MB:

ALTER SYSTEM SET PGA_AGGREGATE_TARGET=150M;
ALTER SYSTEM FLUSH SHARED_POOL;

SELECT /* FIND_ME */
  *
FROM
  T1
ORDER BY
  C2,
  C3;

(EXECUTED IN A DIFFERENT SESSION WHILE THE ABOVE IS EXECUTING):

SELECT
  *
FROM
  V$SORT_USAGE;

USERNAME USER     SESSION_ADDR     SESSION_NUM SQLADDR             SQLHASH SQL_ID        TABLESPACE                      CONTENTS  SEGTYPE     SEGFILE#    SEGBLK#     EXTENTS     BLOCKS   SEGRFNO#
-------- -------- ---------------- ----------- ---------------- ---------- ------------- ------------------------------- --------- --------- ---------- ---------- ---------- ---------- ----------
TESTING  TESTING  000007FF9E4C0160       61714 000007FF94EA7488 2019831986 4ww44m1w68c5k TEMPORARY_DATA1                 TEMPORARY SORT             201      37641         23       2944          1

SELECT
  NAME,
  VALUE
FROM
  V$SYSSTAT
WHERE
  UPPER(NAME) LIKE '%SORT%'
ORDER BY
  NAME;

NAME                VALUE
-------------- ----------
sorts (disk)           89
sorts (memory)   28172666
sorts (rows)    677048890

SELECT
  SP.LAST_OUTPUT_ROWS,
  SP.TEMP_SPACE,
  SP.ESTIMATED_OPTIMAL_SIZE,
  SP.ESTIMATED_ONEPASS_SIZE,
  SP.LAST_MEMORY_USED,
  SP.LAST_EXECUTION,
  SP.LAST_TEMPSEG_SIZE
FROM
  V$SQL S,
  V$SQL_PLAN_STATISTICS_ALL SP
WHERE
  S.SQL_TEXT LIKE 'SELECT /* FIND_ME */%'
  AND S.SQL_ID=SP.SQL_ID;

It might be interesting to see how the test behaves on Oracle database 11.1.0.7 or 11.2.o.1.

Comments : Leave a Comment »

Categories : Performance

Database Inpector Gadget

19 12 2009

December 19, 2009

This example is adapted from an example I created for a presentation a couple months ago. While the example as displayed is specific to displaying ERP data, the example may be easily adapted for use in monitoring Oracle database performance.

—

Vista and Windows 7 offer a set of built in gadgets that perform a variety of tasks, such as displaying calendars, clocks, resource meters, stock quotes, and so on. It might be nice to show a quick overview of various statistics in an ERP system (or in the database itself) so that a determination may be made whether or not a potential problem exists. Such a gadget would need to be able to automatically update its statistics. This example is complicated as it combines a large number of technologies each with their own unique syntax which looks vaguely like English. This example uses VBScript, Oracle database access, cascading style sheets, XML, HTML, DHTML, and coding that is specific to Vista/Windows 7 gadgets.

When the gadget is opened, it determines the statistics that should be displayed for the user, and the order in which the statistics should be displayed. The selected statistics are then retrieved from the database and displayed on the gadget surface in a DIV tag. Every ten minutes the gadget reconnects to the database and displays updated statistics. As the mouse pointer is moved over the gadget, the gadget’s background lights up slightly, and as the mouse pointer passes over the statistics, those also light up (more specifically, the background image for the statistic is changed). When one of the statistics is clicked, a fly-out child window appears on the screen that shows the detailed statistics behind the one line numeric statistic.

The gadget code files should be placed into the C:\Program Files\Windows Sidebar\Gadgets\KMInspector.gadget folder on the computer (note that KMInspector may be specified with a different name). The gadget.xml file in that folder describes the gadget to Vista/Windows 7 and tells Windows where to find the main gadget HTML file, the name of the gadget, and the name of the various icons – this file MUST be saved in a UTF-8 characterset (this is an option when saving with Notepad). The KMInspector.html file is the main gadget page, and the various pages with names beginning with FlyOut are the pages containing the detail information. The security requirement that all Windows gadgets must be signed needs to be disabled.

Below are a couple of screen shots of what we are trying to achieve (a standard Windows sidebar calendar gadget appears above, with the custom developed gadget below):

The top left screen shot shows the custom gadget directly below a built-in Windows calendar gadget. The top right screen shot shows what happens when the mouse pointer passes over the gadget (note that the background becomes lighter in color, and the button under the mouse pointer changes color). The bottom picture shows what happens when one of the buttons in the gedget is clicked.

gadget.xml file (save as in the UTF-8 character set):

<?xml version="1.0" encoding="utf-8" ?>
<gadget>
  <name>KM Inspector</name>
  <version>1.0.0.0</version>
  <icons>
    <icon height="48" width="36" src="InspectorIcon.png" />
  </icons>
  <hosts>
    <host name="sidebar">
      <base type="HTML" apiVersion="1.0.0" src="KMInspector.html" />
      <permissions>Full</permissions>
      <platform minPlatformVersion="1.0" />
      <defaultImage src="Inspector.png" />
    </host>
  </hosts>
</gadget>

Inspector.png is the picture that is displayed when the gadget is dragged from the list of available Windows gadgets to the sidebar. InspectorIcon.png is the picture of the gadget that is displayed in the list of available Windows gadgets. KMInspector.html is the web page that contains the code for the gadget.

Inspector.png:

InspectorIcon.png:

KMInspector.html file:

<html>
<head>
    <title>Visual Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 140px;
            height: 200px;
        }
        #gadgetContent
        {
            margin-top: 0px;
            width: 140px;
            vertical-align: middle;
            text-align: center;
            overflow: hidden;
        }
        .Item {
            background-position: center center;
            padding: 0px;
            font-family: Arial;
            font-size: 8pt;
            color: #FFFFFF;
            border-style: none;
            clip: width: 130px;
            height: 14px;
            position: absolute;
            left: 10px;
            width: 128px;
            background-image: url('InspectorButtonUp.png');
            background-repeat: no-repeat;
         }
        </style>
</head>
<script language="VBScript">
Option Explicit
Dim intMinuteCount       'Number of minutes that have elapsed since the last refresh
Dim intRefreshMinutes    'Frequency of the data refresh, in minutes
Dim strInpector(10)      'Keeps track of which data is displayed in each Inspector position
Dim userEntry
Sub Window_Onload
    Dim lngTimerID
    'Specify the Inspector items to be output, in the order that they should appear
    'Currently Available options
    '  "Negative On Hand"
    '  "Past Due PO Line Count"
    '  "In Use Shop Resources"
    '  "Employees Clocked In"
    '  "Employees Clocked Into Indirect"
    '  "Non Invoiced Shipper"

    strInpector(1) = "Negative On Hand"
    strInpector(2) = "Past Due PO Line Count"
    strInpector(3) = "In Use Shop Resources"
    strInpector(4) = "Employees Clocked In"
    strInpector(5) = "Employees Clocked Into Indirect"
    strInpector(6) = "Non Invoiced Shipper"
    strInpector(7) = ""
    strInpector(8) = ""
    strInpector(9) = ""
    strInpector(10) = ""

    'Hide those Inspector items which are not specified (set = "")
    If strInpector(1) = "" Then
        divInspector1.style.visibility = "hidden"
    End If
    If strInpector(2) = "" Then
        divInspector2.style.visibility = "hidden"
    End If
    If strInpector(3) = "" Then
        divInspector3.style.visibility = "hidden"
    End If
    If strInpector(4) = "" Then
        divInspector4.style.visibility = "hidden"
    End If
    If strInpector(5) = "" Then
        divInspector5.style.visibility = "hidden"
    End If
    If strInpector(6) = "" Then
        divInspector6.style.visibility = "hidden"
    End If
    If strInpector(7) = "" Then
        divInspector7.style.visibility = "hidden"
    End If
    If strInpector(8) = "" Then
        divInspector8.style.visibility = "hidden"
    End If
    If strInpector(9) = "" Then
        divInspector9.style.visibility = "hidden"
    End If
    If strInpector(10) = "" Then
        divInspector10.style.visibility = "hidden"
    End If

    'System.Gadget.onSettingsClosing = "SaveSettings"
    'System.Gadget.onSettingsClosed = "SaveSettingsClosed"
    System.Gadget.settingsUI = "settings.html"

    'Set the refresh frequency in minutes
    intRefreshMinutes = 10

    'Force a refresh on the first execution
    intMinuteCount = intRefreshMinutes

    'Refresh the Inspector items
    RefreshInspector

    'Set a times that fires every 60 seconds that determines if it is time to refresh the data
    lngTimerID = window.SetInterval("RefreshInspector", 60000)
End Sub

Sub RefreshInspector
    Dim i
    Dim lngCount
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strOut

    On Error Resume Next

    'See if the specified number of minutes have elapsed since the last refresh, if not, exit the subroutine
    intMinuteCount = intMinuteCount + 1
    If intMinuteCount < intRefreshMinutes Then
        Exit Sub
    End If

    intMinuteCount = 0

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    'Note that Switch is a function that implments a simple light-weight reversible encryption so that the password does not appear in clear text
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        For i = 1 to 10
            strOut = ""

            Select Case strInpector(i)
                Case "Negative On Hand"
                    strSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  COUNT(*) AS NUM" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  PART" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  QTY_ON_HAND<0"
                    snpData.Open strSQL, dbMyConnection

                    If snpData.State = 1 Then
                        If Not(snpData.EOF) Then
                            strOut = "Negative On Hand: " & cStr(snpData("num"))
                        End If
                        snpData.Close
                    End If
                Case "Past Due PO Line Count"
                    strSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  COUNT(*) AS NUM" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  PURCHASE_ORDER PO," & VBCrLf
                    strSQL = strSQL & "  PURC_ORDER_LINE POL," & VBCrLf
                    strSQL = strSQL & "  PURC_LINE_DEL PLD" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  PO.STATUS IN ('F','R','U')" & VBCrLf
                    strSQL = strSQL & "  AND PO.ID=POL.PURC_ORDER_ID" & VBCrLf
                    strSQL = strSQL & "  AND POL.LINE_STATUS='A'" & VBCrLf
                    strSQL = strSQL & "  AND POL.SERVICE_ID IS NULL" & VBCrLf
                    strSQL = strSQL & "  AND POL.ORDER_QTY>POL.TOTAL_RECEIVED_QTY" & VBCrLf
                    strSQL = strSQL & "  AND POL.PURC_ORDER_ID=PLD.PURC_ORDER_ID(+)" & VBCrLf
                    strSQL = strSQL & "  AND POL.LINE_NO=PLD.PURC_ORDER_LINE_NO(+)" & VBCrLf
                    strSQL = strSQL & "  AND DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(POL.ORDER_QTY,0),NVL(PLD.ORDER_QTY,0)) > " & VBCrLf
                    strSQL = strSQL & "     DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(TOTAL_RECEIVED_QTY,0),NVL(PLD.RECEIVED_QTY,0))" & VBCrLf
                    strSQL = strSQL & "  AND COALESCE(PLD.DESIRED_RECV_DATE,POL.DESIRED_RECV_DATE,PO.DESIRED_RECV_DATE) < SYSDATE"
                    snpData.Open strSQL, dbMyConnection

                    If snpData.State = 1 Then
                        If Not(snpData.EOF) Then
                            strOut = "PO Past Due: " & cStr(snpData("num"))
                        End If
                        snpData.Close
                    End If
                Case "In Use Shop Resources"
                    strSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  COUNT(DISTINCT RESOURCE_ID) AS NUM" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  LABOR_TICKET" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
                    strSQL = strSQL & "  AND HOURS_WORKED IS NULL" & VBCrLf
                    strSQL = strSQL & "  AND RESOURCE_ID IS NOT NULL" & VBCrLf
                    strSQL = strSQL & "ORDER BY" & VBCrLf
                    strSQL = strSQL & "  RESOURCE_ID"
                    snpData.Open strSQL, dbMyConnection

                    If snpData.State = 1 Then
                        If Not(snpData.EOF) Then
                            strOut = "Resources in Use: " & cStr(snpData("num"))
                        End If
                        snpData.Close
                    End If
                Case "Employees Clocked In"
                    strSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  COUNT(DISTINCT EMPLOYEE_ID) AS NUM" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  LABOR_TICKET" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
                    strSQL = strSQL & "  AND HOURS_WORKED IS NULL"
                    snpData.Open strSQL, dbMyConnection

                    If snpData.State = 1 Then
                        If Not(snpData.EOF) Then
                            strOut = "Emp Clocked In: " & cStr(snpData("num"))
                        End If
                        snpData.Close
                    End If
                Case "Employees Clocked Into Indirect"
                    strSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  COUNT(DISTINCT EMPLOYEE_ID) AS NUM" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  LABOR_TICKET" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
                    strSQL = strSQL & "  AND HOURS_WORKED IS NULL" & VBCrLf
                    strSQL = strSQL & "  AND RESOURCE_ID IS NULL"
                    snpData.Open strSQL, dbMyConnection

                    If snpData.State = 1 Then
                        If Not(snpData.EOF) Then
                            strOut = "Emp on Indirect: " & cStr(snpData("num"))
                        End If
                        snpData.Close
                    End If
                Case "Non Invoiced Shipper"
                    STRSQL = "SELECT" & VBCrLf
                    strSQL = strSQL & "  S.PACKLIST_ID" & VBCrLf
                    strSQL = strSQL & "FROM" & VBCrLf
                    strSQL = strSQL & "  SHIPPER S," & VBCrLf
                    strSQL = strSQL & "  SHIPPER_LINE SL" & VBCrLf
                    strSQL = strSQL & "WHERE" & VBCrLf
                    strSQL = strSQL & "  S.INVOICE_ID IS NULL" & VBCrLf
                    strSQL = strSQL & "  AND S.STATUS='A'" & VBCrLf
                    strSQL = strSQL & "  AND S.PACKLIST_ID=SL.PACKLIST_ID" & VBCrLf
                    strSQL = strSQL & "  AND SL.SHIPPED_QTY>0" & VBCrLf
                    strSQL = strSQL & "GROUP BY" & VBCrLf
                    strSQL = strSQL & "  S.PACKLIST_ID"
                    snpData.Open strSQL, dbMyConnection

                    lngCount = 0
                    If snpData.State = 1 Then
                        Do While Not(snpData.EOF)
                            lngCount = lngCount +1

                            snpData.MoveNext
                        Loop
                        strOut = "PLs to Invoice: " & cStr(lngCount)
                        snpData.Close
                    End If
            End Select

            'Output the selections in the correct divInspector box
            Select Case i
                Case 1
                    divInspector1.InnerText = strOut
                Case 2
                    divInspector2.InnerText = strOut
                Case 3
                    divInspector3.InnerText = strOut
                Case 4
                    divInspector4.InnerText = strOut
                Case 5
                    divInspector5.InnerText = strOut
                Case 6
                    divInspector6.InnerText = strOut
                case 7
                    divInspector7.InnerText = strOut
                Case 8
                    divInspector8.InnerText = strOut
                Case 9
                    divInspector9.InnerText = strOut
                case 10
                    divInspector10.InnerText = strOut
            End Select
        Next

        divLastUpdate.InnerText = "Last Update: " & Time

        dbMyConnection.Close
    Else
        divLastUpdate.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

'Function SaveSettings(Event)
'    If Event.closeAction = event.Action.commit Then
'        System.Gadget.Settings.write("settingsSelectionIndex", selUserEntry.selectedIndex)
'    End If
'    'Allow the Settings dialog to close.
'    event.cancel = false
'End Function

'Function SaveSettingsClosed(event)
'    'User hits OK on the settings page.
'    If event.closeAction = event.Action.commit Then
'        userEntry = System.Gadget.Settings.readString("settingsUserEntry")
'        SetContentText userEntry
'    Else
'        If event.closeAction = event.Action.cancel Then
'            'User hits Cancel on the settings page.
'            SetContentText "Cancelled"
'        End If
'    End If
'End Function
Sub ShowBackgroundHighlight
    imgBackground.brightness = 0.0
End Sub

Sub ShowFlyOut(intInspector)
    Dim strPage

    On Error Resume Next

    'Determine which of the flyout web pages to display based on the Inspector loaded into the position
    Select Case strInpector(intInspector)
        Case "Negative On Hand"
            strPage = "FlyOutNOH.html"
        Case "Past Due PO Line Count"
            strPage = "FlyOutPOPD.html"
        Case "In Use Shop Resources"
            strPage = "FlyOutRIU.html"
        Case "Employees Clocked In"
            strPage = "FlyOutECI.html"
        Case "Employees Clocked Into Indirect"
            strPage = "FlyOutEOI.html"
        Case "Non Invoiced Shipper"
            strPage = "FlyOutPLI.html"
    End Select
    If strPage <> "" Then
        System.Gadget.Flyout.file = cStr(strPage)
        System.Gadget.Flyout.Show = true
    End If
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background" onmouseover="imgBackground.brightness = 0.5" onmouseout="imgBackground.brightness = 0.0">
    <g:background id="imgBackground" src="url(Inspectorbackground.png)" brightness="0.0" style="padding: 0px"</g:background>
    <div id="divInspector1" style="top: 10px;"
          onmouseover="divInspector1.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector1.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(1)">Inspector 1</div>
    <div id="divInspector2" style="top: 25px;"
          onmouseover="divInspector2.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector2.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(2)">Inspector 2</div>
    <div id="divInspector3" style="top: 40px;"
          onmouseover="divInspector3.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector3.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(3)">Inspector 3</div>
    <div id="divInspector4" style="top: 55px;"
          onmouseover="divInspector4.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector4.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(4)">Inspector 4</div>
    <div id="divInspector5" style="top: 70px;"
          onmouseover="divInspector5.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector5.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(5)">Inspector 5</div>
    <div id="divInspector6" style="top: 85px;"
          onmouseover="divInspector6.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector6.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(6)">Inspector 6</div>
    <div id="divInspector7" style="top: 100px;"
          onmouseover="divInspector7.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector7.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(7)">Inspector 7</div>
    <div id="divInspector8" style="top: 115px;"
          onmouseover="divInspector8.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector8.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(8)">Inspector 8</div>
    <div id="divInspector9" style="top: 130px;"
          onmouseover="divInspector9.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector9.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(9)">Inspector 9</div>
    <div id="divInspector10" style="top: 145px;"
          onmouseover="divInspector10.style.backgroundImage='url(InspectorButtonMouseOver.png)'"
          onmouseout="divInspector10.style.backgroundImage='url(InspectorButtonUp.png)'"
          onclick="ShowFlyOut(10)">Inspector 10</div>

    <div id="divLastUpdate" style="background-position: center center; padding: 0px; font-family: Perpetua; font-size: 10px; font-weight: bold; position: absolute; top: 169px; left: 12px; color: #000000;">Time</div>  
    <div id="divTitle"
        style="background-position: left center; border-style: none; border-width: 0px; padding: 0px; margin: 0px; position: absolute; top: 178px; left: 3px; height: 25px; width: 135px; background-image: url(Inspector.png); background-repeat: no-repeat;">
        </div>
    <div id="div1"
        style="background-position: left center; border-style: none; border-width: 0px; padding: 0px; margin: 0px; position: absolute; top: 200px; left: 3px; height: 25px; width: 135px; background-image: url(Inspector.png); background-repeat: no-repeat; visibility: hidden;">
        </div>
</body>
</html>

—

FlyOutECI.html:

<html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 655px;
            height: 300px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strHTML

    On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  LT.EMPLOYEE_ID," & VBCrLf
        strSQL = strSQL & "  E.LAST_NAME || ', ' || E.FIRST_NAME AS EMPLOYEE_NAME," & VBCrLf
        strSQL = strSQL & "  LT.RESOURCE_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_BASE_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_LOT_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_SPLIT_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_SUB_ID," & VBCrLf
        strSQL = strSQL & "  LT.OPERATION_SEQ_NO," & VBCrLf
        strSQL = strSQL & "  WO.PART_ID," & VBCrLf
        strSQL = strSQL & "  LT.INDIRECT_ID," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN) AS CLOCK_IN" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  LABOR_TICKET LT," & VBCrLf
        strSQL = strSQL & "  EMPLOYEE E," & VBCrLf
        strSQL = strSQL & "  WORK_ORDER WO" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  LT.TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
        strSQL = strSQL & "  AND LT.HOURS_WORKED IS NULL" & VBCrLf
        strSQL = strSQL & "  AND LT.EMPLOYEE_ID=E.ID" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_TYPE=WO.TYPE(+)" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_BASE_ID=WO.BASE_ID(+)" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_LOT_ID=WO.LOT_ID(+)" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_SPLIT_ID=WO.SPLIT_ID(+)" & VBCrLf
        strSQL = strSQL & "  AND WO.SUB_ID(+)='0'" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  E.LAST_NAME || ', ' || E.FIRST_NAME," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN)"

        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Employees Clocked In</b>" & vbCrLf
            strHTML = strHTML & "<table width=640 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>Emp ID</b></td>"
            strHTML = strHTML & "<td><b>Name</b></td>"
            strHTML = strHTML & "<td><b>Resource ID</b></td>"
            strHTML = strHTML & "<td><b>Work Order Op</b></td>"
            strHTML = strHTML & "<td><b>Part ID</b></td>"
            strHTML = strHTML & "<td><b>Clock In</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                strHTML = strHTML & "<td>" & cStr(snpData("employee_id")) & "</td>"
                If Not IsNull(snpData("employee_name")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("employee_name")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("resource_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("resource_id")) & "</td>"
                    If cStr(snpData("workorder_sub_id")) = "0" Then
                        strHTML = strHTML & "<td>" & cStr(snpData("workorder_base_id")) & "/" & cStr(snpData("workorder_lot_id")) & "/ OP " & cStr(snpData("operation_seq_no")) & "</td>"
                    Else
                        strHTML = strHTML & "<td>" & cStr(snpData("workorder_base_id")) & "-" & cStr(snpData("workorder_sub_id")) & "/" & cStr(snpData("workorder_lot_id")) & "/ OP " & cStr(snpData("operation_seq_no")) & "</td>"
                    End If
                    If Not IsNull(snpData("part_id")) Then
                        strHTML = strHTML & "<td>" & cStr(snpData("part_id")) & "</td>"
                    Else
                        strHTML = strHTML & "<td>&nbsp;</td>"
                    End If
                Else
                    strHTML = strHTML & "<td>Indirect: " & cStr(snpData("indirect_id")) & "</td>"
                    strHTML = strHTML & "<td>&nbsp;</td>"
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("clock_in")) Then
                    strHTML = strHTML & "<td><p align=""right"">" & cStr(snpData("clock_in")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 645px; height: 280px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

FlyOutEOI.html:

<html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 505px;
            height: 300px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strHTML

    On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  LT.EMPLOYEE_ID," & VBCrLf
        strSQL = strSQL & "  E.LAST_NAME || ', ' || E.FIRST_NAME AS EMPLOYEE_NAME," & VBCrLf
        strSQL = strSQL & "  LT.INDIRECT_ID," & VBCrLf
        strSQL = strSQL & "  I.DESCRIPTION," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN) AS CLOCK_IN" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  LABOR_TICKET LT," & VBCrLf
        strSQL = strSQL & "  EMPLOYEE E," & VBCrLf
        strSQL = strSQL & "  INDIRECT I" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  LT.TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
        strSQL = strSQL & "  AND LT.HOURS_WORKED IS NULL" & VBCrLf
        strSQL = strSQL & "  AND LT.EMPLOYEE_ID=E.ID" & VBCrLf
        strSQL = strSQL & "  AND LT.INDIRECT_ID IS NOT NULL" & VBCrLf
        strSQL = strSQL & "  AND LT.INDIRECT_ID=I.ID" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  E.LAST_NAME || ', ' || E.FIRST_NAME," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN)"

        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Employees Clocked Into Indirect</b>" & vbCrLf
            strHTML = strHTML & "<table width=490 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>Emp ID</b></td>"
            strHTML = strHTML & "<td><b>Name</b></td>"
            strHTML = strHTML & "<td><b>Indirect</b></td>"
            strHTML = strHTML & "<td><b>Description</b></td>"
            strHTML = strHTML & "<td><b>Clock In</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                strHTML = strHTML & "<td>" & cStr(snpData("employee_id")) & "</td>"
                If Not IsNull(snpData("employee_name")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("employee_name")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML &  "<td>" & cStr(snpData("indirect_id")) & "</td>"
                If Not IsNull(snpData("description")) Then
                    strHTML = strHTML &  "<td>" & cStr(snpData("description")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("clock_in")) Then
                    strHTML = strHTML & "<td><p align=""right"">" & cStr(snpData("clock_in")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function
</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 495px; height: 280px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

FlyOutNOH.html:

<html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 505px;
            height: 300px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strHTML

    On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  ID," & VBCrLf
        strSQL = strSQL & "  DESCRIPTION," & VBCrLf
        strSQL = strSQL & "  PRODUCT_CODE," & VBCrLf
        strSQL = strSQL & "  COMMODITY_CODE," & VBCrLf
        strSQL = strSQL & "  QTY_ON_HAND," & VBCrLf
        strSQL = strSQL & "  DECODE(PURCHASED,'Y','Purchased','Fabricated') PART_TYPE" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  PART" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  QTY_ON_HAND<0" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  ID"
        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Parts with a Negative QTY On Hand</b>" & vbCrLf
            strHTML = strHTML & "<table width=490 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>Part ID</b></td>"
            strHTML = strHTML & "<td><b>Description</b></td>"
            strHTML = strHTML & "<td><b>P.C.</b></td>"
            strHTML = strHTML & "<td><b>C.C.</b></td>"
            strHTML = strHTML & "<td><b>Qty</b></td>"
            strHTML = strHTML & "<td><b>Type</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                strHTML = strHTML & "<td>" & cStr(snpData("id")) & "</td>"
                If Not IsNull(snpData("description")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("description")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("product_code")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("product_code")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("commodity_code")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("commodity_code")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("qty_on_hand")) Then
                    strHTML = strHTML & "<td><p align=""right"">" & FormatNumber(snpData("qty_on_hand"),4) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("part_type")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("part_type")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 495px; height: 280px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

FlyOutPLI.html:

 <html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 655px;
            height: 300px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strLastPacklist
    Dim strSQL
    Dim strHTML

    On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  S.PACKLIST_ID," & VBCrLf
        strSQL = strSQL & "  SL.LINE_NO," & VBCrLf
        strSQL = strSQL & "  S.BOL_ID," & VBCrLf
        strSQL = strSQL & "  S.SHIPPED_DATE," & VBCrLf
        strSQL = strSQL & "  CO.CUSTOMER_ID," & VBCrLf
        strSQL = strSQL & "  SL.CUST_ORDER_ID," & VBCrLf
        strSQL = strSQL & "  SL.CUST_ORDER_LINE_NO," & VBCrLf
        strSQL = strSQL & "  COL.PART_ID," & VBCrLf
        strSQL = strSQL & "  SL.SHIPPED_QTY" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  SHIPPER S," & VBCrLf
        strSQL = strSQL & "  SHIPPER_LINE SL," & VBCrLf
        strSQL = strSQL & "  CUST_ORDER_LINE COL," & VBCrLf
        strSQL = strSQL & "  CUSTOMER_ORDER CO" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  S.INVOICE_ID IS NULL" & VBCrLf
        strSQL = strSQL & "  AND S.STATUS='A'" & VBCrLf
        strSQL = strSQL & "  AND S.PACKLIST_ID=SL.PACKLIST_ID" & VBCrLf
        strSQL = strSQL & "  AND SL.CUST_ORDER_ID=COL.CUST_ORDER_ID(+)" & VBCrLf
        strSQL = strSQL & "  AND SL.CUST_ORDER_LINE_NO=COL.LINE_NO(+)" & VBCrLf
        strSQL = strSQL & "  AND COL.CUST_ORDER_ID=CO.ID(+)" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  S.PACKLIST_ID," & VBCrLf
        strSQL = strSQL & "  SL.LINE_NO"

        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Pack Lists Not Yet Invoiced</b>" & vbCrLf
            strHTML = strHTML & "<table width=640 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>PL</b></td>"
            strHTML = strHTML & "<td><b>Line</b></td>"
            strHTML = strHTML & "<td><b>BOL</b></td>"
            strHTML = strHTML & "<td><b>Ship Date</b></td>"
            strHTML = strHTML & "<td><b>Customer</b></td>"
            strHTML = strHTML & "<td><b>Cust Order</b></td>"
            strHTML = strHTML & "<td><b>Part ID</b></td>"
            strHTML = strHTML & "<td><b>Qty</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                If strLastPacklist <> snpData("packlist_id") Then
                    'Need to output the pack list ID this time, since it is not the same as the last row returned by the database
                    strHTML = strHTML & "<td>" & cStr(snpData("packlist_id")) & "</td>"   
                    strLastPacklist = snpData("packlist_id")
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "<td>" & cStr(snpData("line_no")) & "</td>"
                If Not IsNull(snpData("bol_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("bol_id")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("shipped_date")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("shipped_date")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("customer_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("customer_id")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("cust_order_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("cust_order_id")) & "/" & cStr(snpData("cust_order_line_no")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("part_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("part_id")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("shipped_qty")) Then
                    strHTML = strHTML & "<td><p align=""right"">" & cStr(snpData("shipped_qty")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 645px; height: 280px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

FlyOutPOPD.html:

<html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 610px;
            height: 500px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strHTML

    'On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  POL.PART_ID," & VBCrLf
        strSQL = strSQL & "  POL.VENDOR_PART_ID," & VBCrLf
        strSQL = strSQL & "  P.DESCRIPTION," & VBCrLf
        strSQL = strSQL & "  POL.PURC_ORDER_ID," & VBCrLf
        strSQL = strSQL & "  POL.LINE_NO AS PURC_ORDER_LINE_NO," & VBCrLf
        strSQL = strSQL & "  PLD.DEL_SCHED_LINE_NO," & VBCrLf
        strSQL = strSQL & "  COALESCE(PLD.DESIRED_RECV_DATE,POL.DESIRED_RECV_DATE,PO.DESIRED_RECV_DATE) AS DESIRED_RECV_DATE," & VBCrLf
        strSQL = strSQL & "  DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(POL.ORDER_QTY,0),NVL(PLD.ORDER_QTY,0)) AS ORDER_QTY," & VBCrLf
        strSQL = strSQL & "  DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(TOTAL_RECEIVED_QTY,0),NVL(PLD.RECEIVED_QTY,0)) AS RECEIVED_QTY" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  PURCHASE_ORDER PO," & VBCrLf
        strSQL = strSQL & "  PURC_ORDER_LINE POL," & VBCrLf
        strSQL = strSQL & "  PART P," & VBCrLf
        strSQL = strSQL & "  PURC_LINE_DEL PLD" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  PO.STATUS IN ('F','R','U')" & VBCrLf
        strSQL = strSQL & "  AND PO.ID=POL.PURC_ORDER_ID" & VBCrLf
        strSQL = strSQL & "  AND POL.LINE_STATUS='A'" & VBCrLf
        strSQL = strSQL & "  AND POL.SERVICE_ID IS NULL" & VBCrLf
        strSQL = strSQL & "  AND POL.ORDER_QTY>POL.TOTAL_RECEIVED_QTY" & VBCrLf
        strSQL = strSQL & "  AND POL.PART_ID=P.ID(+)" & VBCrLf
        strSQL = strSQL & "  AND POL.PURC_ORDER_ID=PLD.PURC_ORDER_ID(+)" & VBCrLf
        strSQL = strSQL & "  AND POL.LINE_NO=PLD.PURC_ORDER_LINE_NO(+)" & VBCrLf
        strSQL = strSQL & "  AND DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(POL.ORDER_QTY,0),NVL(PLD.ORDER_QTY,0)) > DECODE(PLD.PURC_ORDER_LINE_NO,NULL,NVL(TOTAL_RECEIVED_QTY,0),NVL(PLD.RECEIVED_QTY,0))" & VBCrLf
        strSQL = strSQL & "  AND COALESCE(PLD.DESIRED_RECV_DATE,POL.DESIRED_RECV_DATE,PO.DESIRED_RECV_DATE) < SYSDATE" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  POL.PART_ID," & vbCrLf
        strSQL = strSQL & "  POL.PURC_ORDER_ID," & VBCrLf
        strSQL = strSQL & "  POL.LINE_NO," & VBCrLf
        strSQL = strSQL & "  PLD.DEL_SCHED_LINE_NO" & VBCrLf
        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Past Due Purchase Orders</b>" & vbCrLf
            strHTML = strHTML & "<table width=590 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>Part ID</b></td>"
            strHTML = strHTML & "<td><b>Description</b></td>"
            strHTML = strHTML & "<td><b>PO</b></td>"
            strHTML = strHTML & "<td><b>Received</b></td>"
            strHTML = strHTML & "<td><b>Wanted</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                If Not IsNull(snpData("part_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("part_id")) & "</td>"
                Else
                    If Not IsNull(snpData("vendor_part_id")) Then
                        strHTML = strHTML & "<td>" & cStr(snpData("vendor_part_id")) & "</td>"
                    Else
                        strHTML = strHTML & "<td>&nbsp;</td>"
                    End If
                End If
                If Not IsNull(snpData("description")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("description")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("del_sched_line_no")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("purc_order_id")) & "/" & cStr(snpData("purc_order_line_no")) & "/DL " & cStr(snpData("del_sched_line_no")) & "</td>"
                Else
                    strHTML = strHTML & "<td>" & cStr(snpData("purc_order_id")) & "/" & cStr(snpData("purc_order_line_no")) & "</td>"
                End If
                strHTML = strHTML & "<td>" & cStr(snpData("received_qty")) & " of " & cStr(snpData("order_qty")) & "</td>"
                If Not IsNull(snpData("desired_recv_date")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("desired_recv_date")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 590px; height: 480px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

FlyOutRIU.html:

<html>
<head>
    <title>K&M Inspector</title>
    <style type="text/css">
        body
        {
            margin: 0px;
            width: 655px;
            height: 300px;
        }
        <!--table { font-size: 8pt; font-family: Times New Roman } -->
    </style>
</head>

<script language="VBScript">
Option Explicit

Sub Window_Onload
    Dim dbMyConnection
    Dim comData
    Dim snpData
    Dim strUsername
    Dim strPassword
    Dim strDatabase
    Dim strSQL
    Dim strHTML

    On Error Resume Next

    Set snpData = CreateObject("ADODB.Recordset")
    Set dbMyConnection = CreateObject("ADODB.Connection")

    strUsername = "MyUser"
    strPassword = Switch(Chr(112) & Chr(100) & Chr(15) & Chr(10) & Chr(180) & Chr(7) & Chr(206) & Chr(233) & Chr(25))
    strDatabase = "MyDB"

    dbMyConnection.ConnectionString = "Provider=OraOLEDB.Oracle;Data Source=" & strDatabase & ";User ID=" & strUsername & ";Password=" & strPassword & ";"
    dbMyConnection.Open

    If dbMyConnection.State = 1 Then
        strSQL = "SELECT" & VBCrLf
        strSQL = strSQL & "  LT.RESOURCE_ID," & VBCrLf
        strSQL = strSQL & "  SR.DESCRIPTION," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_BASE_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_LOT_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_SPLIT_ID," & VBCrLf
        strSQL = strSQL & "  LT.WORKORDER_SUB_ID," & VBCrLf
        strSQL = strSQL & "  LT.OPERATION_SEQ_NO," & VBCrLf
        strSQL = strSQL & "  WO.PART_ID," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN) AS CLOCK_IN" & VBCrLf
        strSQL = strSQL & "FROM" & VBCrLf
        strSQL = strSQL & "  LABOR_TICKET LT," & VBCrLf
        strSQL = strSQL & "  SHOP_RESOURCE SR," & VBCrLf
        strSQL = strSQL & "  WORK_ORDER WO" & VBCrLf
        strSQL = strSQL & "WHERE" & VBCrLf
        strSQL = strSQL & "  LT.TRANSACTION_DATE>=TRUNC(SYSDATE-180)" & VBCrLf
        strSQL = strSQL & "  AND LT.HOURS_WORKED IS NULL" & VBCrLf
        strSQL = strSQL & "  AND LT.RESOURCE_ID IS NOT NULL" & VBCrLf
        strSQL = strSQL & "  AND LT.RESOURCE_ID=SR.ID" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_TYPE='W'" & VBCrLf
        strSQL = strSQL & "  AND WO.TYPE='W'" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_TYPE=WO.TYPE" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_BASE_ID=WO.BASE_ID" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_LOT_ID=WO.LOT_ID" & VBCrLf
        strSQL = strSQL & "  AND LT.WORKORDER_SPLIT_ID=WO.SPLIT_ID" & VBCrLf
        strSQL = strSQL & "  AND WO.SUB_ID='0'" & VBCrLf
        strSQL = strSQL & "ORDER BY" & VBCrLf
        strSQL = strSQL & "  LT.RESOURCE_ID," & VBCrLf
        strSQL = strSQL & "  NVL(LT.ACT_CLOCK_IN,LT.CLOCK_IN)" & VBCrLf

        snpData.Open strSQL, dbMyConnection

        If snpData.State = 1 Then
            strHTML = strHTML & "<b>Shop Resources Currently in Use</b>" & vbCrLf
            strHTML = strHTML & "<table width=640 border=1>" & vbCrLf
            strHTML = strHTML & "<tr bgcolor=#5555FF>"
            strHTML = strHTML & "<td><b>Resource ID</b></td>"
            strHTML = strHTML & "<td><b>Description</b></td>"
            strHTML = strHTML & "<td><b>Work Order Op</b></td>"
            strHTML = strHTML & "<td><b>Part ID</b></td>"
            strHTML = strHTML & "<td><b>Clock In</b></td>"
            strHTML = strHTML & "</tr>" & vbCrLf

            Do While Not snpData.EOF
                strHTML = strHTML & "<tr>"
                strHTML = strHTML & "<td>" & cStr(snpData("resource_id")) & "</td>"
                If Not IsNull(snpData("description")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("description")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If cStr(snpData("workorder_sub_id")) = "0" Then
                    strHTML = strHTML & "<td>" & cStr(snpData("workorder_base_id")) & "/" & cStr(snpData("workorder_lot_id")) & "/ OP " & cStr(snpData("operation_seq_no")) & "</td>"
                Else
                    strHTML = strHTML & "<td>" & cStr(snpData("workorder_base_id")) & "-" & cStr(snpData("workorder_sub_id")) & "/" & cStr(snpData("workorder_lot_id")) & "/ OP " & cStr(snpData("operation_seq_no")) & "</td>"
                End If
                If Not IsNull(snpData("part_id")) Then
                    strHTML = strHTML & "<td>" & cStr(snpData("part_id")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                If Not IsNull(snpData("clock_in")) Then
                    strHTML = strHTML & "<td><p align=""right"">" & cStr(snpData("clock_in")) & "</td>"
                Else
                    strHTML = strHTML & "<td>&nbsp;</td>"
                End If
                strHTML = strHTML & "</tr>" & vbCrLf

                snpData.MoveNext
            Loop

            strHTML = strHTML & "</table>" & vbCrLf
            snpData.Close

            divOutput.InnerHTML = strHTML
        Else
            divOutput.InnerText = "No Rows Returned"
        End If

        dbMyConnection.Close
    Else
        'Could Not Connect
        divOutput.InnerText = "Could Not Connect"
    End If

    Set snpData = Nothing
    Set dbMyConnection = Nothing
End Sub

Private Function Switch(strValue)
    Dim i
    Dim intOffset
    Dim intTemp
    Dim strTemp
    Dim strKey

    strKey = "OracleDatabaseDoesNotNeedToBeDifficultToUnderstand"
    intOffset = 50

    strTemp = ""
    For i = 1 To Len(strValue)
        intTemp = Asc(Mid(strValue, i, 1)) Xor Asc(Mid(strKey, Len(strValue) + i, 1)) + intOffset
        If intTemp > 255 Then
            intTemp = intTemp - 255
        End If
        strTemp = strTemp & Chr(intTemp)
    Next

    Switch = strTemp
End Function

</script>

<body id="Background">
    <div id="divOutput" style="position: absolute; top: 10px; width: 645px; height: 280px; overflow-y:auto;">&nbsp;</div> 
</body>
</html>

—

Settings.html (this is just a placeholder for configuration settings):

<html >
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=Unicode" />
        <title>Hello World</title>
        <style type="text/css">
        body
        {
            margin: 0;
            width: 130px;
            height: 75px;
            font-family: verdana;
            font-weight: bold;
            font-size: small;
        }
        #gadgetContent
        {
            margin-top: 20px;
            width: 130px;
            vertical-align: middle;
            text-align: center;
            overflow: hidden;
        }
        </style>
        <script type="text/jscript" language="jscript">
            // Initialize the gadget.
            function init()
            {
                var oBackground = document.getElementById("imgBackground");
                oBackground.src = "url(images/background.png)";
            }
        </script>
    </head>

    <body onload="init()">
        <g:background id="imgBackground">
            <span id="gadgetContent">Hello World!</span>
        </g:background>
    </body>
</html>

—

InspectorButtonUp.png:

—

InspectorButtonMouseOver.png:

—

InspectorBackground.png (Background Image for Main Web Page):

Other Backgrounds that May be Used (Note that Power Point is a great tool for creating backgrounds):

—

Resources for Developing Windows Sidebar Applications:
http://msdn.microsoft.com/en-us/library/bb456468(VS.85).aspx
http://www.microsoft.com/uk/msdn/screencasts/screencast/262/building-a-vista-sidebar-gadget-part-1-getting-started.aspx

Resources for VBScript Programming:
http://msdn.microsoft.com/en-us/library/d1wf56tt.aspx
http://www.w3schools.com/vbscript/vbscript_ref_functions.asp

Suggested Book:
“CSS The Definitive Guide” by Eric A. Meyer:
http://www.amazon.com/CSS-Definitive-Guide-Eric-Meyer/dp/0596527330

Comments : 2 Comments »

Categories : HTML, SQL, VBS

Tracking Performance Problems – Inserting a Hint into SQL in a Compiled Program

18 12 2009

December 18, 2009

What follows is part of a presentation that I gave in 2008.

The lead into this slide is that a report in an ERP package was running far slower than expected following an upgrade of the ERP package. Of course the SQL statements are hard coded into the ERP package, so there is not much that can be done, right? I created a 10046 extended SQL trace file at level 12, and then passed the trace file through my Toy Project for Performance Tuning (https://hoopercharles.wordpress.com/2009/12/13/toy-project-for-performance-tuning-2/). One of the outputs of my program’s 10046 trace file parser provides an overview of the trace file, as well as an overview of each SQL statement. A screen shot of that output follows:

The screen shot shows that there were 10,399 execute calls in the trace file that consumed 47.67 seconds of the server’s CPU time, and that the elapsed time/wall clock time from the server’s perspective for executing the SQL statements is 50.32 seconds for the executions. There were 16,146 fetch calls that consumed 263.48 seconds of the server’s CPU time, and the elapsed time/wall clock time from the server’s perspective for fetching the rows for the SQL statements is 263.64 seconds. Note that there were 0 physical reads and 8,804,970 consistent gets during the fetch (what’s that buffer cache hit ratio?). The SQL*Net message from client wait totaled 107.33 seconds, but 42.62 seconds were in a single block of time (likely at the end of the trace file, just before tracing was disabled), so the actual total time waiting for the next request from the client is about 64.7 seconds in 22,825 round-trips.

If we scroll down a bit, we might find a couple of the greatest contributors to the server-side processing:

The screen shot shows that there are two groups of SQL statements that combined contributed to 86% of the total server-side processing time for the report. The first SQL statement group is identified as Cursor 16 Ver 1, and the second is identified as Cursor 17 Ver 1.

We could then search the remainder of this file to locate those identifiers.

The above screen shot shows that there are two identical SQL statements in the first group – the first SQL statement (Cursor 16 Ver 1) was parsed, but never executed. The row source execution plan from the 10046 trace file (in the STAT lines) shows that the optimizer decided to use the index X_RECEIVABLE_3 that is on the column ENTITY_ID (there are two distinct values for this column), rather than the much more selective index on the INVOICE_ID column. Notice the number of consistent gets and the CPU utilization for this one SQL statement that was executed 934 times.

Why was the index selected? Would a DBMS_XPLAN help?

The DBMS_XPLAN output shows that the optimizer estimated that the X_RECEIVABLE_3 index would return a single row, when in fact it returned 66124 rows. A problem where statistics were not collected in the last 15 years? No.

Maybe a 10053 trace will help:

In the above we see that the predicted number of rows returned with the X_RECEIVABLE_3 index will be less than that for the other indexes, and the expected CPU resources will also be slightly less, but something is not right. The optimizer selected not to use the primary key index on the RECEIVABLE table. Note that the calculated cost for all three indexes is 2.

Let’s try an experiment with a NO_INDEX hint to prevent the optimizer from using the X_RECEIVABLE_3 index:

In the above, notice that the primary key index (SYS_C006885) was selected by the optimizer, that the execution time dropped from 0.31 seconds to 0.01 seconds (or less), and the number of consistent gets dropped from 1311 to 5. A rather nice improvement, but how do we force the ERP package to not use the index without the ability to modify the program’s source code?

We could do something like this, if we assume that bind variable peeking is the source of the problem:

CREATE OR REPLACE TRIGGER LOGON_FIX_APP_PERF AFTER LOGON ON DATABASE
DECLARE
  SHOULD_EXECUTE INTEGER;
BEGIN
  SELECT DECODE(SUBSTR(UPPER(PROGRAM),1,2),'VM',1,'VF',1,0)+DECODE(INSTR(PROGRAM,'\',-1),0,0,DECODE(SUBSTR(UPPER(SUBSTR(PROGRAM,INSTR(PROGRAM,'\',-1)+1)),1,2),'VM',1,'VF',1,0)) INTO SHOULD_EXECUTE FROM V$SESSION WHERE SID=(SELECT SID FROM V$MYSTAT WHERE ROWNUM=1);
  IF SHOULD_EXECUTE > 0 THEN
    EXECUTE IMMEDIATE 'ALTER SESSION SET "_optim_peek_user_binds"=FALSE';
  END IF;
END;
/

The above logon trigger results in the following row source execution plans:

But, the above approach may be too broad as it will affect every SQL statement executed by a program that begins with the letters VM or VF. It did help this report:

The time to fetch all rows dropped from 264 seconds to just 11 seconds, and the full report displays in just over a minute.

Let’s see if we are able to trick the optimizer into not using the X_RECEIVABLE_3 index without disabling bind variable peeking. First, we need to enable private outlines in the current session:

ALTER SESSION SET USE_PRIVATE_OUTLINES=TRUE;

For demonstration purposes, I will explicitly tell the optimizer to use the X_RECEIVABLE_3 index when creating the outline (the exact SQL statement used by the program should be used, without the hint):

CREATE OR REPLACE PRIVATE OUTLINE P_RECEIVABLE1 ON
select /*+ INDEX(I X_RECEIVABLE_3) */ sum(a.apply_amount)
from RECV_MEMO_APPLY a, RECEIVABLE i
where a.inv_invoice_id = :1          
and a.apply_date <= :2             
and a.inv_invoice_id = i.invoice_id
and i.status != 'X'
and i.total_amount != 0 and i.recv_gl_acct_id = :3 and ENTITY_ID = :4;

Note that I had to specify a hint in this case as I was at the time testing in Oracle 11g R1 (11.1.0.6), which refused to use the X_RECEIVABLE_3 on its own to reproduce the problem seen with Oracle 10.2.0.x.

Let’s view the hints generated by the optimizer when it created the outline:

SELECT
  HINT#,
  HINT_TEXT
FROM
  OL$HINTS
WHERE
  OL_NAME='P_RECEIVABLE1';

HINT#  HINT_TEXT
    1  USE_NL(@"SEL$1" "A"@"SEL$1")
    2  LEADING(@"SEL$1" "I"@"SEL$1" "A"@"SEL$1")
    3  INDEX(@"SEL$1" "A"@"SEL$1" ("RECV_MEMO_APPLY"."INV_INVOICE_ID" "RECV_MEMO_APPLY"."MEMO_INVOICE_ID"))
    4  INDEX(@"SEL$1" "I"@"SEL$1" ("RECEIVABLE"."ENTITY_ID"))
    5  OUTLINE_LEAF(@"SEL$1")
    6  ALL_ROWS
    7  OPTIMIZER_FEATURES_ENABLE('10.2.0.2')
    8  IGNORE_OPTIM_EMBEDDED_HINTS

We can then verify that in fact the slow execution plan was selected:

EXPLAIN PLAN FOR
select /*+ INDEX(I X_RECEIVABLE_3) */ sum(a.apply_amount)
from RECV_MEMO_APPLY a, RECEIVABLE i
where a.inv_invoice_id = :1          
and a.apply_date <= :2             
and a.inv_invoice_id = i.invoice_id
and i.status != 'X'
and i.total_amount != 0 and i.recv_gl_acct_id = :3                   and ENTITY_ID = :4;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY(NULL,NULL,'BASIC +NOTE'));

------------------------------------------------------------
| Id  | Operation                      | Name              |
------------------------------------------------------------
|   0 | SELECT STATEMENT               |                   |
|   1 |  SORT AGGREGATE                |                   |
|   2 |   TABLE ACCESS BY INDEX ROWID  | RECV_MEMO_APPLY   |
|   3 |    NESTED LOOPS                |                   |
|   4 |     TABLE ACCESS BY INDEX ROWID| RECEIVABLE        |
|   5 |      INDEX RANGE SCAN          | X_RECEIVABLE_3    |
|   6 |     INDEX RANGE SCAN           | X_RECV_MEMO_APP_1 |
------------------------------------------------------------

Next, we will create a stored outline that explicitly specifies not to use the X_RECEIVABLE_3 index:

CREATE OR REPLACE PRIVATE OUTLINE P_RECEIVABLE_TEMP ON
select /*+ NO_INDEX(I X_RECEIVABLE_3) */ sum(a.apply_amount)
from RECV_MEMO_APPLY a, RECEIVABLE i
where a.inv_invoice_id = :1          
and a.apply_date <= :2             
and a.inv_invoice_id = i.invoice_id
and i.status != 'X'
and i.total_amount != 0 and i.recv_gl_acct_id = :3                   and ENTITY_ID = :4;

Let’s view the hints generated by the optimizer when it created the outline:

SELECT
  HINT#,
  HINT_TEXT
FROM
  OL$HINTS
WHERE
  OL_NAME='P_RECEIVABLE_TEMP';

HINT#  HINT_TEXT
    1  USE_NL(@"SEL$1" "A"@"SEL$1")
    2  LEADING(@"SEL$1" "I"@"SEL$1" "A"@"SEL$1")
    3  INDEX(@"SEL$1" "A"@"SEL$1" ("RECV_MEMO_APPLY"."INV_INVOICE_ID" "RECV_MEMO_APPLY"."MEMO_INVOICE_ID"))
    4  INDEX(@"SEL$1" "I"@"SEL$1" ("RECEIVABLE"."INVOICE_ID"))
    5  OUTLINE_LEAF(@"SEL$1")
    6  ALL_ROWS
    7  OPTIMIZER_FEATURES_ENABLE('10.2.0.2')
    8  IGNORE_OPTIM_EMBEDDED_HINTS

We can then verify that the faster execution plan was selected:

EXPLAIN PLAN FOR
select /*+ NO_INDEX(I X_RECEIVABLE_3) */ sum(a.apply_amount)
from RECV_MEMO_APPLY a, RECEIVABLE i
where a.inv_invoice_id = :1          
and a.apply_date <= :2             
and a.inv_invoice_id = i.invoice_id
and i.status != 'X'
and i.total_amount != 0 and i.recv_gl_acct_id = :3                   and ENTITY_ID = :4;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY(NULL,NULL,'BASIC +NOTE'));

-----------------------------------------------------------
| Id  | Operation                     | Name              |
-----------------------------------------------------------
|   0 | SELECT STATEMENT              |                   |
|   1 |  SORT AGGREGATE               |                   |
|   2 |   NESTED LOOPS                |                   |
|   3 |    TABLE ACCESS BY INDEX ROWID| RECEIVABLE        |
|   4 |     INDEX UNIQUE SCAN         | SYS_C0011925      |
|   5 |    TABLE ACCESS BY INDEX ROWID| RECV_MEMO_APPLY   |
|   6 |     INDEX RANGE SCAN          | X_RECV_MEMO_APP_1 |
-----------------------------------------------------------

Next is the potentially dangerous part – refer to Metalink Notes:604022.1, 726802.1, 730062.1, 144194.1, 728647.1, 5893396.8, as well as http://www.jlcomp.demon.co.uk/04_outlines.rtf

We delete the outline hints from the outline with the slow execution plan (the one with the INDEX(I X_RECEIVABLE_3) hint in this case):

DELETE FROM
  OL$HINTS
WHERE
  OL_NAME='P_RECEIVABLE1';

Then we copy the hints from the outline for the fast execution plan (with our NO_INDEX hint):

INSERT INTO
  OL$HINTS
SELECT
  'P_RECEIVABLE1',
  HINT#,
  CATEGORY,
  HINT_TYPE,
  HINT_TEXT,
  STAGE#,
  NODE#,
  TABLE_NAME,
  TABLE_TIN,
  TABLE_POS,
  REF_ID,
  USER_TABLE_NAME,
  COST,
  CARDINALITY,
  BYTES,
  HINT_TEXTOFF,
  HINT_TEXTLEN,
  JOIN_PRED,
  SPARE1,
  SPARE2,
  HINT_STRING
FROM
  OL$HINTS
WHERE
  OL_NAME='P_RECEIVABLE_TEMP';

COMMIT;

Then we instruct Oracle to refresh the private outline:

EXEC DBMS_OUTLN_EDIT.REFRESH_PRIVATE_OUTLINE('P_RECEIVABLE1')

Now, let’s try generating the plan for the original query again (with the forced/hinted X_RECEIVABLE_3 index usage):

EXPLAIN PLAN FOR
select /*+ INDEX(I X_RECEIVABLE_3) */ sum(a.apply_amount)
from RECV_MEMO_APPLY a, RECEIVABLE i
where a.inv_invoice_id = :1          
and a.apply_date <= :2             
and a.inv_invoice_id = i.invoice_id
and i.status != 'X'
and i.total_amount != 0 and i.recv_gl_acct_id = :3                   and ENTITY_ID = :4;

SELECT
  *
FROM
  TABLE(DBMS_XPLAN.DISPLAY(NULL,NULL,'BASIC +NOTE'));

-----------------------------------------------------------
| Id  | Operation                     | Name              |
-----------------------------------------------------------
|   0 | SELECT STATEMENT              |                   |
|   1 |  SORT AGGREGATE               |                   |
|   2 |   NESTED LOOPS                |                   |
|   3 |    TABLE ACCESS BY INDEX ROWID| RECEIVABLE        |
|   4 |     INDEX UNIQUE SCAN         | SYS_C0011925      |
|   5 |    TABLE ACCESS BY INDEX ROWID| RECV_MEMO_APPLY   |
|   6 |     INDEX RANGE SCAN          | X_RECV_MEMO_APP_1 |
-----------------------------------------------------------

Note that even though we hinted/forced Oracle to use the X_RECEIVABLE_3 index, it now selected to use the primary key index SYS_C0011925 due to the hacked private outline. Oracle IGNORED MY HINT (actually, it did exactly as the outline instructed).

Now, let’s make the outline a bit more permanent, converting it from a private outline to a public outline:

CREATE PUBLIC OUTLINE PP_RECEIVABLE1 FROM PRIVATE P_RECEIVABLE1;
ALTER SYSTEM SET USE_STORED_OUTLINES=TRUE;
DROP PRIVATE OUTLINE P_RECEIVABLE1;
DROP PRIVATE OUTLINE P_RECEIVABLE_TEMP;

If we then generate a DBMS_XPLAN for the original query (in my example, the one with the forced index hint), we see the following:

-------------------------------------------------------------------------------------------------------------
| Id  | Operation                     | Name              | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
-------------------------------------------------------------------------------------------------------------
|   1 |  SORT AGGREGATE               |                   |      1 |      1 |      1 |00:00:00.01 |       5 |
|   2 |   NESTED LOOPS                |                   |      1 |      1 |      0 |00:00:00.01 |       5 |
|*  3 |    TABLE ACCESS BY INDEX ROWID| RECEIVABLE        |      1 |      1 |      1 |00:00:00.01 |       3 |
|*  4 |     INDEX UNIQUE SCAN         | SYS_C0011925      |      1 |      1 |      1 |00:00:00.01 |       2 |
|*  5 |    TABLE ACCESS BY INDEX ROWID| RECV_MEMO_APPLY   |      1 |      1 |      0 |00:00:00.01 |       2 |
|*  6 |     INDEX RANGE SCAN          | X_RECV_MEMO_APP_1 |      1 |      1 |      0 |00:00:00.01 |       2 |
-------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   3 - filter(("I"."STATUS"<>'X' AND "I"."TOTAL_AMOUNT"<>0 AND "I"."RECV_GL_ACCT_ID"=:3 AND
              "ENTITY_ID"=:4))
   4 - access("I"."INVOICE_ID"=:1)
   5 - filter("A"."APPLY_DATE"<=:2)
   6 - access("A"."INV_INVOICE_ID"=:1)

Note
-----
   - outline "PP_RECEIVABLE1" used for this statement

But there is a catch. The USE_STORED_OUTLINES parameter cannot be set in the init.ora or spfile, so we need a STARTUP trigger to set the parameter:

CREATE OR REPLACE TRIGGER ENABLE_OUTLINES_TRIG AFTER STARTUP ON DATABASE
BEGIN
  EXECUTE IMMEDIATE('ALTER SYSTEM SET USE_STORED_OUTLINES=TRUE');
END;

After implementing the stored outlines there is a new top SQL statement in the 10046 trace profile file. But, at some point we have to stop (before the effects of compulsive tuning disorder are achieved).

My original idea for hacking the stored outlines came from the book “Troubleshooting Oracle Performance”.

Note that hacking stored outlines should not be the first step. Instead, see if it is possible to modify optimizer parameters at the session level first to achieve the desired execution plan. Once the desired execution plan is achieved, create the stored outline to freeze the execution plan.

Comments : 1 Comment »

Categories : Bind Variable, Performance, SQL

Update Rows in Another Table with the Help of Analytic Functions

17 12 2009

December 17, 2009

In a recent comp.databases.oracle.misc Usenet thread:
http://groups.google.com/group/comp.databases.oracle.misc/browse_thread/thread/42c0d1550278250b

The following question was asked:

I need help with a query which involves the 2 tables defined below. What I need to do is choose the record with the max “Eff Date” from “Table A” for a particular “Emp No.” and update the “Desc” from that record in the field “Desc” of “Table B” for the same “Emp No.”. I am able to choose the max “Eff Date” record for each employee from Table A but somehow not able to updated the same “Desc” in “Table B”.

Request you to please help the query. Any help would be appreciated.
Thanks!
Table A
Emp No. Group   Eff Date        Desc
1234    CI      01/01/1989      X
1234    CI      01/02/2000      X
1234    CI      01/02/2006      A
2345    AF      01/01/1990      X
2345    AF      01/02/2005      A
Table B
Emp No. Group   Desc
1234    CI      X
2345    AF      A
3456    CI      A

I provided the following suggestion:

Watch the query and results closely as one possible solution is built (there are other methods):

CREATE TABLE T1 (
  EMP_NO NUMBER,
  GROUPING VARCHAR2(5),
  EFF_DATE DATE,
  DESCR VARCHAR2(5));

CREATE TABLE T2 (
  EMP_NO NUMBER,
  GROUPING VARCHAR2(5),
  DESCR VARCHAR2(5));

INSERT INTO T1 VALUES (1234,'CI',TO_DATE('01/01/1989','MM/DD/YYYY'),'X');
INSERT INTO T1 VALUES (1234,'CI',TO_DATE('01/02/2000','MM/DD/YYYY'),'X');
INSERT INTO T1 VALUES (1234,'CI',TO_DATE('01/02/2006','MM/DD/YYYY'),'A');
INSERT INTO T1 VALUES (2345,'AF',TO_DATE('01/01/1990','MM/DD/YYYY'),'X');
INSERT INTO T1 VALUES (2345,'AF',TO_DATE('01/02/2005','MM/DD/YYYY'),'A');

INSERT INTO T2 VALUES (1234,'CI','XNN');
INSERT INTO T2 VALUES (2345,'AF','ANN');
INSERT INTO T2 VALUES (3456,'CI','ANN');

COMMIT;

SELECT
  EMP_NO,
  GROUPING,
  DESCR
FROM
  T2;

EMP_NO GROUP DESCR
------ ----- -----
  1234 CI    XNN
  2345 AF    ANN
  3456 CI    ANN

SELECT
  EMP_NO,
  GROUPING,
  EFF_DATE,
  DESCR
FROM
  T1;

EMP_NO GROUP EFF_DATE  DESCR
------ ----- --------- -----
  1234 CI    01-JAN-89 X
  1234 CI    02-JAN-00 X
  1234 CI    02-JAN-06 A
  2345 AF    01-JAN-90 X
  2345 AF    02-JAN-05 A

SELECT
  EMP_NO,
  GROUPING,
  EFF_DATE,
  ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE
DESC) RN,
  DESCR
FROM
  T1;

EMP_NO GROUP EFF_DATE          RN DESCR
------ ----- --------- ---------- -----
  1234 CI    02-JAN-06          1 A
  1234 CI    02-JAN-00          2 X
  1234 CI    01-JAN-89          3 X
  2345 AF    02-JAN-05          1 A
  2345 AF    01-JAN-90          2 X

SELECT
  EMP_NO,
  GROUPING,
  EFF_DATE,
  DESCR
FROM
  (SELECT
    EMP_NO,
    GROUPING,
    EFF_DATE,
    ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,
    DESCR
  FROM
    T1)
WHERE
  RN=1;

EMP_NO GROUP EFF_DATE  DESCR
------ ----- --------- -----
  1234 CI    02-JAN-06 A
  2345 AF    02-JAN-05 A

UPDATE
  T2
SET
  DESCR=(
    SELECT
      DESCR
    FROM
      (SELECT
        EMP_NO,
        GROUPING,
        ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,
        DESCR
      FROM
        T1) T1
    WHERE
      RN=1
      AND T1.EMP_NO=T2.EMP_NO
      AND T1.GROUPING=T2.GROUPING)
WHERE
  (T2.EMP_NO,T2.GROUPING) IN (
    SELECT
      EMP_NO,
      GROUPING
    FROM
      (SELECT
        EMP_NO,
        GROUPING,
        ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,
        DESCR
      FROM
        T1)
    WHERE
      RN=1);

2 rows updated.

SELECT
  EMP_NO,
  GROUPING,
  DESCR
FROM
  T2;

EMP_NO GROUP DESCR
------ ----- -----
  1234 CI    A
  2345 AF    A
  3456 CI    ANN

Note that in the above, I assumed that the combination of EMP_NO and GROUPING had to be the same.

~~~~~~~~~~~~~~~~~~~~~~~

Maxim Demenko provided a very different approach to solving the problem that is both compact and impressive:

SQL> merge into t2 t2
   2  using (
   3    select emp_no,grouping,
   4      max(descr) keep(dense_rank last order by eff_date) descr
   5      from t1 group by emp_no,grouping) t1
   6  on (t1.emp_no=t2.emp_no
   7  and t1.grouping=t2.grouping)
   8  when matched then update set t2.descr=t1.descr
   9  ;

2 rows merged.

Execution Plan
----------------------------------------------------------
Plan hash value: 3235844370

-------------------------------------------------------------------------------
| Id  | Operation              | Name | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------
|   0 | MERGE STATEMENT        |      |     2 |    16 |     8  (25)| 00:00:01 |
|   1 |  MERGE                 | T2   |       |       |            |          |
|   2 |   VIEW                 |      |       |       |            |          |
|*  3 |    HASH JOIN           |      |     2 |   108 |     8  (25)| 00:00:01 |
|   4 |     TABLE ACCESS FULL  | T2   |     3 |    99 |     3   (0)| 00:00:01 |
|   5 |     VIEW               |      |     5 |   105 |     4  (25)| 00:00:01 |
|   6 |      SORT GROUP BY     |      |     5 |   150 |     4  (25)| 00:00:01 |
|   7 |       TABLE ACCESS FULL| T1   |     5 |   150 |     3   (0)| 00:00:01 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
    3 - access("T1"."EMP_NO"="T2"."EMP_NO" AND
               "T1"."GROUPING"="T2"."GROUPING")

~~~~~~~~~~~~~~~~~~~~~~~

Since Maxim provided an execution plan, let’s compare the efficiency of the two methods with a larger test case that uses the same table definitions:

TRUNCATE TABLE T1;
TRUNCATE TABLE T2;

INSERT INTO T1
SELECT
  DECODE(MOD(ROWNUM,10),
         0,0000,
         1,1111,
         2,2222,
         3,3333,
         4,4444,
         5,5555,
         6,6666,
         7,7777,
         8,8888,
         9,9999),
  DECODE(MOD(ROWNUM,6),
         0,'AA',
         1,'BB',
         2,'CC',
         3,'DD',
         4,'EE',
         5,'FF'),
  TRUNC(SYSDATE+SIN(ROWNUM/180*3.141592)*1000),
  UPPER(DBMS_RANDOM.STRING('A',1))
FROM
  DUAL
CONNECT BY
  LEVEL<=1000000;

INSERT INTO T2
SELECT
  DECODE(MOD(ROWNUM,10),
         0,0000,
         1,1111,
         2,2222,
         3,3333,
         4,4444,
         5,5555,
         6,6,
         7,7,
         8,8,
         9,9),
  DECODE(MOD(ROWNUM,11),
         0,'AA',
         1,'BB',
         2,'CC',
         3,'DD',
         4,'EE',
         5,'FF',
         6,'GG',
         7,'HH',
         8,'II',
         9,'JJ',
         10,'KK'),
  UPPER(DBMS_RANDOM.STRING('A',3))
FROM
  DUAL
CONNECT BY
  LEVEL<=100;

COMMIT;

EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T1',CASCADE=>TRUE)
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=>USER,TABNAME=>'T2',CASCADE=>TRUE)

SET PAGESIZE 1000
SET LINESIZE 150
ALTER SESSION SET STATISTICS_LEVEL='ALL';
ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

SPOOL C:\CHECKTIMING.TXT

UPDATE
  T2
SET
  DESCR=(
    SELECT
      DESCR
    FROM
      (SELECT
        EMP_NO,
        GROUPING,
        ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,
        DESCR
      FROM
        T1) T1
    WHERE
      RN=1
      AND T1.EMP_NO=T2.EMP_NO
      AND T1.GROUPING=T2.GROUPING)
WHERE
  (T2.EMP_NO,T2.GROUPING) IN (
    SELECT
      EMP_NO,
      GROUPING
    FROM
      (SELECT
        EMP_NO,
        GROUPING,
        ROW_NUMBER() OVER (PARTITION BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,
        DESCR
      FROM
        T1)
    WHERE
      RN=1);

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  *
FROM
  T2
WHERE
  LENGTH(DESCR)=1
ORDER BY
  EMP_NO;

ROLLBACK;
ALTER SYSTEM FLUSH BUFFER_CACHE;
ALTER SYSTEM FLUSH BUFFER_CACHE;

merge into t2 t2
 using (
   select emp_no,grouping,
     max(descr) keep(dense_rank last order by eff_date) descr
     from t1 group by emp_no,grouping) t1
 on (t1.emp_no=t2.emp_no
 and t1.grouping=t2.grouping)
 when matched then update set t2.descr=t1.descr
 ;

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));

SELECT
  *
FROM
  T2
WHERE
  LENGTH(DESCR)=1
ORDER BY
  EMP_NO;

ROLLBACK;
SPOOL OFF

What is the output of the above?

SQL_ID  8w16pv37zxuh5, child number 0
-------------------------------------
UPDATE   T2 SET   DESCR=(     SELECT       DESCR     FROM       (SELECT
        EMP_NO,         GROUPING,         ROW_NUMBER() OVER (PARTITION
BY EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,         DESCR
FROM         T1) T1     WHERE       RN=1       AND T1.EMP_NO=T2.EMP_NO
     AND T1.GROUPING=T2.GROUPING) WHERE   (T2.EMP_NO,T2.GROUPING) IN (
   SELECT       EMP_NO,       GROUPING     FROM       (SELECT
EMP_NO,         GROUPING,         ROW_NUMBER() OVER (PARTITION BY
EMP_NO, GROUPING ORDER BY EFF_DATE DESC) RN,         DESCR       FROM
      T1)     WHERE       RN=1) 

Plan hash value: 2277482977                                                                                                                          

-----------------------------------------------------------------------------------------------------------------------------------------------
| Id  | Operation                   | Name     | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  | Writes |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------------------------------
|   0 | UPDATE STATEMENT            |          |      1 |        |      0 |00:00:06.12 |   59000 |   3282 |      5 |       |       |          |
|   1 |  UPDATE                     | T2       |      1 |        |      0 |00:00:06.12 |   59000 |   3282 |      5 |       |       |          |
|*  2 |   HASH JOIN SEMI            |          |      1 |      1 |     17 |00:00:03.06 |    3289 |   3280 |      5 |   816K|   816K| 1167K (0)|
|   3 |    TABLE ACCESS FULL        | T2       |      1 |    100 |    100 |00:00:00.03 |       7 |      6 |      0 |       |       |          |
|   4 |    VIEW                     | VW_NSO_1 |      1 |   1000K|     30 |00:00:03.03 |    3282 |   3274 |      5 |       |       |          |
|*  5 |     VIEW                    |          |      1 |   1000K|     30 |00:00:03.03 |    3282 |   3274 |      5 |       |       |          |
|*  6 |      WINDOW SORT PUSHED RANK|          |      1 |   1000K|     67 |00:00:03.03 |    3282 |   3274 |      5 | 36864 | 36864 |   25M (1)|
|   7 |       TABLE ACCESS FULL     | T1       |      1 |   1000K|   1000K|00:00:00.01 |    3275 |   3269 |      0 |       |       |          |
|*  8 |   VIEW                      |          |     17 |  16667 |     17 |00:00:03.06 |   55675 |      0 |      0 |       |       |          |
|*  9 |    WINDOW SORT PUSHED RANK  |          |     17 |  16667 |    566K|00:00:02.53 |   55675 |      0 |      0 |  1541K|   615K| 1369K (0)|
|* 10 |     TABLE ACCESS FULL       | T1       |     17 |  16667 |    566K|00:00:01.13 |   55675 |      0 |      0 |       |       |          |
-----------------------------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):                                                                                                  
---------------------------------------------------                                                                                                  
   2 - access("T2"."EMP_NO"="EMP_NO" AND "T2"."GROUPING"="GROUPING")
   5 - filter("RN"=1)
   6 - filter(ROW_NUMBER() OVER ( PARTITION BY "EMP_NO","GROUPING" ORDER BY INTERNAL_FUNCTION("EFF_DATE") DESC )<=1)
   8 - filter("RN"=1)
   9 - filter(ROW_NUMBER() OVER ( PARTITION BY "EMP_NO","GROUPING" ORDER BY INTERNAL_FUNCTION("EFF_DATE") DESC )<=1)
  10 - filter(("EMP_NO"=:B1 AND "GROUPING"=:B2))

    EMP_NO GROUP DESCR
---------- ----- -----
         0 CC    W   
         0 EE    F   
      1111 BB    F   
      1111 DD    U   
      1111 FF    Y   
      2222 AA    W   
      2222 CC    T   
      2222 EE    K   
      3333 BB    Z   
      3333 FF    I   
      3333 DD    W   
      4444 AA    Z   
      4444 EE    G   
      4444 CC    S   
      5555 DD    L   
      5555 BB    Y   
      5555 FF    X

SQL_ID  93cj2ck69n4kg, child number 0
-------------------------------------
merge into t2 t2  using (    select emp_no,grouping,      max(descr)
keep(dense_rank last order by eff_date) descr      from t1 group by
emp_no,grouping) t1  on (t1.emp_no=t2.emp_no  and
t1.grouping=t2.grouping)  when matched then update set t2.descr=t1.descr

Plan hash value: 4231777338                                                                                                                          

-----------------------------------------------------------------------------------------------------------------------------
| Id  | Operation              | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers | Reads  |  OMem |  1Mem | Used-Mem |
-----------------------------------------------------------------------------------------------------------------------------
|   0 | MERGE STATEMENT        |      |      1 |        |      1 |00:00:01.53 |    3301 |   3277 |       |       |          |
|   1 |  MERGE                 | T2   |      1 |        |      1 |00:00:01.53 |    3301 |   3277 |       |       |          |
|   2 |   VIEW                 |      |      1 |        |     17 |00:00:01.50 |    3282 |   3275 |       |       |          |
|*  3 |    HASH JOIN           |      |      1 |     43 |     17 |00:00:01.50 |    3282 |   3275 |   921K|   921K| 1181K (0)|
|   4 |     VIEW               |      |      1 |     43 |     30 |00:00:01.47 |    3275 |   3269 |       |       |          |
|   5 |      SORT GROUP BY     |      |      1 |     43 |     30 |00:00:01.47 |    3275 |   3269 | 73728 | 73728 |          |
|   6 |       TABLE ACCESS FULL| T1   |      1 |   1000K|   1000K|00:00:00.01 |    3275 |   3269 |       |       |          |
|   7 |     TABLE ACCESS FULL  | T2   |      1 |    100 |    100 |00:00:00.03 |       7 |      6 |       |       |          |
-----------------------------------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):                                                                                                  
---------------------------------------------------                                                                                                  
   3 - access("T1"."EMP_NO"="T2"."EMP_NO" AND "T1"."GROUPING"="T2"."GROUPING") 

    EMP_NO GROUP DESCR
---------- ----- -----
         0 CC    Z   
         0 EE    Z   
      1111 BB    Z   
      1111 DD    X   
      1111 FF    Z   
      2222 AA    Z   
      2222 CC    Z   
      2222 EE    Z   
      3333 BB    Z   
      3333 FF    Z   
      3333 DD    Z   
      4444 AA    Z   
      4444 EE    Z   
      4444 CC    Z   
      5555 DD    Z   
      5555 BB    Z   
      5555 FF    Z

From the point of view of performance, Maxim’s solution is a clear winner. It is interesting to note that the value of the DESCR column in table T2 differs for the two approaches.

Comments : 2 Comments »

Categories : Analytic Functions, SQL

	Charles Hooper on Favorite Free Applications to…
	Zach on Favorite Free Applications to…
	Charles Hooper on Favorite Free Applications to…
	Zach on Favorite Free Applications to…
	Charles Hooper on Huge Presentation – Work…
	Dan on Huge Presentation – Work…
	Charles Hooper on Review of Synology DS415+, How…
	Matthias Clausen on Review of Synology DS415+, How…
	Charles Hooper on _SMALL_TABLE_THRESHOLD Paramet…
	mohd atif on _SMALL_TABLE_THRESHOLD Paramet…
	Charles Hooper on Free ANSI SQL to Oracle Specif…
	Kalin Agrawal on Free ANSI SQL to Oracle Specif…
	Charles Hooper on Large Page Support or Not…
	vaurob on Large Page Support or Not…
	Locating Oracle… on Finding a New Home for a Clien…

Charles Hooper's Oracle Notes

Hard Parses when Using Bind Variables?

Excel – The Graphical Master of Oracle Foreign Keys

Which is Most Efficient when Selecting Rows: EXISTS, IN, or a JOIN

Which is Most Efficient when Deleting Rows: EXISTS, IN, or a VIEW

Extract the First 4400 Images from Excel 2003 (and Above) and Transfer to a Database Table

Extract 1834 Images from Excel 2007 and Transfer to a Database Table

High Value for MBRC Causes High BCHR, High CPU Usage, and Slow Performance

SQL – RANK, MAX Analytical Functions, DECODE, SIGN

SQL – DENSE_RANK, PERCENT_RANK, and COUNT Analytical Functions

SQL – COUNT Analytical Function, GROUP BY, HAVING

Proving that 1=2, is Oracle Wrong to Short-Circuit an Execution Plan?

Miscellaneous Metalink Performance Articles 2

SORT_AREA_SIZE Affecting Sorts to Disk with PGA_AGGREGATE_TARGET Set 2?

Faulty Quotes 4 – Buffer Cache Hit Ratio (BCHR)

Faulty Quotes 3 – Contradictory Information

Miscellaneous Metalink Performance Articles

SORT_AREA_SIZE Affecting Sorts to Disk with PGA_AGGREGATE_TARGET Set?

Database Inpector Gadget

Tracking Performance Problems – Inserting a Hint into SQL in a Compiled Program

Update Rows in Another Table with the Help of Analytic Functions

Categories

Recent Articles

Misc. Articles

Popular Articles

Recent Comments

Blogroll

Hints for Posting Code Sections in Comments