<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:georss="http://www.georss.org/georss" xmlns:geo="http://www.w3.org/2003/01/geo/wgs84_pos#" xmlns:media="http://search.yahoo.com/mrss/"
		>
<channel>
	<title>Comments on: Histograms &#8211; What is Wrong with this Quote?</title>
	<atom:link href="http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/feed/" rel="self" type="application/rss+xml" />
	<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/</link>
	<description>Miscellaneous Random Oracle Topics: Stop, Think, ... Understand</description>
	<lastBuildDate>Mon, 13 May 2013 14:10:06 +0000</lastBuildDate>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.com/</generator>
	<item>
		<title>By: Wolfgang Breitling</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4806</link>
		<dc:creator><![CDATA[Wolfgang Breitling]]></dc:creator>
		<pubDate>Sun, 08 Jul 2012 21:50:14 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4806</guid>
		<description><![CDATA[This is a reply to Mladen&#039;s last reply. For some reason there is no reply &quot;button&quot; on his reply.

If you do a size nnn then that is exactly what gather_stats will do. It could be fewer buckets if there are fewer than nnn distinct values of the columns. If there are more distinct values than nnn then you end up with a height-balanced histogram. In any case, yo WILL get a histogram because that is what you requested. If you ask for a histogram with method_opt=&gt;&#039;... size nnn&#039; then I there is no reason to go with anything but size 254 - unless you deliberately want to force a height-balanced histogram for a column with fewer than 255 distinct values. But in that case you better know exactly what you are doing and you may actually be better off to not gather the histogram in that case but construct it the way you want manually.
If &quot;size auto&quot; gives you only two buckets then that is either because of one of the cases we discussed, e.g. the column of a single-column unique or primary key, where gather_stats determined that a histogram makes no sense. Or the column has never been used in a where clause.]]></description>
		<content:encoded><![CDATA[<p>This is a reply to Mladen&#8217;s last reply. For some reason there is no reply &#8220;button&#8221; on his reply.</p>
<p>If you do a size nnn then that is exactly what gather_stats will do. It could be fewer buckets if there are fewer than nnn distinct values of the columns. If there are more distinct values than nnn then you end up with a height-balanced histogram. In any case, yo WILL get a histogram because that is what you requested. If you ask for a histogram with method_opt=&gt;&#8217;&#8230; size nnn&#8217; then I there is no reason to go with anything but size 254 &#8211; unless you deliberately want to force a height-balanced histogram for a column with fewer than 255 distinct values. But in that case you better know exactly what you are doing and you may actually be better off to not gather the histogram in that case but construct it the way you want manually.<br />
If &#8220;size auto&#8221; gives you only two buckets then that is either because of one of the cases we discussed, e.g. the column of a single-column unique or primary key, where gather_stats determined that a histogram makes no sense. Or the column has never been used in a where clause.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mladen Gogala</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4803</link>
		<dc:creator><![CDATA[Mladen Gogala]]></dc:creator>
		<pubDate>Fri, 06 Jul 2012 21:19:49 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4803</guid>
		<description><![CDATA[Wolfgang, why would I do &quot;size auto&quot; when I don&#039;t know what size of a histogram will Oracle create? I am usually using size 254 or size 127 but never &quot;size auto&quot;. In some releases, SIZE AUTO means that there will be only 2 buckets: minimum and maximum, which is about the same as no histograms at all.]]></description>
		<content:encoded><![CDATA[<p>Wolfgang, why would I do &#8220;size auto&#8221; when I don&#8217;t know what size of a histogram will Oracle create? I am usually using size 254 or size 127 but never &#8220;size auto&#8221;. In some releases, SIZE AUTO means that there will be only 2 buckets: minimum and maximum, which is about the same as no histograms at all.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Charles Hooper</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4802</link>
		<dc:creator><![CDATA[Charles Hooper]]></dc:creator>
		<pubDate>Fri, 06 Jul 2012 12:32:08 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4802</guid>
		<description><![CDATA[There are a lot of excellent comments attached to this blog article.  The linked article written by Wolfgang Breitling is excellent – I am sure that I read the article at least once a couple of years ago (&lt;i&gt;a few changes were made to the query optimizer in Oracle Database over the years, so there are a couple of minor items in the article that no longer work as described in the latest Oracle Database releases&lt;/i&gt;).  I thought that I would share my review comments for this section of the book (the following review comments echo many of the comments provided by other readers of this blog article).

&lt;blockquote&gt;
The quoted section from the book seems to be closely derived from a &lt;a href=&quot;http://docs.oracle.com/cd/A87860_01/doc/server.817/a76992/stats.htm#27069&quot; target=&quot;_blank&quot; rel=&quot;nofollow&quot;&gt;page of the Oracle Database 8.1.7 documentation&lt;/a&gt; which was modified and finally removed from later releases of the documentation.  

Histograms are potentially useful when created on non-indexed columns, with the goal of improving cardinality estimates, which are used to determine join order and join type when multiple tables are referenced in a SQL statement.

Histograms are also potentially useful when the data is not skewed, helping to improve the cardinality estimates of range scans when outlier values are present (such as an out of range value taking the place of NULL values).  See the test case in one of my comments above.

Regarding the statement, “Obviously, if the frequency for a specific value is very high, using the index won’t be the best choice,” the frequency of a value in a column is but one of several factors that the optimizer considers, but a high frequency of a value is not necessarily a problem if the use of the index avoids the need to visit the related table.  

Regarding the bullet points that describes when histograms should not be used, “The column is not used in the WHERE clauses of the queries,” with the introduction of ANSI join syntax in Oracle Database 9.0.1, a column that only appears in JOIN clauses could potentially benefit from a histogram.  

Prior to the introduction of adaptive cursor sharing in Oracle Database 11.1, bind variables when combined with bind variable peeking and histograms could generate execution plans that, while efficient for the first execution, were potentially considerably inefficient when different bind variable values were submitted during later executions of the SQL statement.  Adaptive cursor sharing attempts to address the issues caused by the combination of bind variable peeking and histograms.  In cases where the same value is always submitted for a comparison with a submitted bind variable, a histogram on the comparison column could significantly improve the optimizer’s cardinality estimates such that a more efficient execution plan is developed (consider a case when an index on a STATUS column is present, and that STATUS column contains the value of either PROCESSED or NOT PROCESSED).  The book only mentioned that adaptive cursor sharing would be discussed in a later recipe – there was no comment in this section of the book regarding whether adaptive cursor sharing helped or further hindered the bullet-pointed items in the book.
&lt;/blockquote&gt;]]></description>
		<content:encoded><![CDATA[<p>There are a lot of excellent comments attached to this blog article.  The linked article written by Wolfgang Breitling is excellent – I am sure that I read the article at least once a couple of years ago (<i>a few changes were made to the query optimizer in Oracle Database over the years, so there are a couple of minor items in the article that no longer work as described in the latest Oracle Database releases</i>).  I thought that I would share my review comments for this section of the book (the following review comments echo many of the comments provided by other readers of this blog article).</p>
<blockquote><p>
The quoted section from the book seems to be closely derived from a <a href="http://docs.oracle.com/cd/A87860_01/doc/server.817/a76992/stats.htm#27069" target="_blank" rel="nofollow">page of the Oracle Database 8.1.7 documentation</a> which was modified and finally removed from later releases of the documentation.  </p>
<p>Histograms are potentially useful when created on non-indexed columns, with the goal of improving cardinality estimates, which are used to determine join order and join type when multiple tables are referenced in a SQL statement.</p>
<p>Histograms are also potentially useful when the data is not skewed, helping to improve the cardinality estimates of range scans when outlier values are present (such as an out of range value taking the place of NULL values).  See the test case in one of my comments above.</p>
<p>Regarding the statement, “Obviously, if the frequency for a specific value is very high, using the index won’t be the best choice,” the frequency of a value in a column is but one of several factors that the optimizer considers, but a high frequency of a value is not necessarily a problem if the use of the index avoids the need to visit the related table.  </p>
<p>Regarding the bullet points that describes when histograms should not be used, “The column is not used in the WHERE clauses of the queries,” with the introduction of ANSI join syntax in Oracle Database 9.0.1, a column that only appears in JOIN clauses could potentially benefit from a histogram.  </p>
<p>Prior to the introduction of adaptive cursor sharing in Oracle Database 11.1, bind variables when combined with bind variable peeking and histograms could generate execution plans that, while efficient for the first execution, were potentially considerably inefficient when different bind variable values were submitted during later executions of the SQL statement.  Adaptive cursor sharing attempts to address the issues caused by the combination of bind variable peeking and histograms.  In cases where the same value is always submitted for a comparison with a submitted bind variable, a histogram on the comparison column could significantly improve the optimizer’s cardinality estimates such that a more efficient execution plan is developed (consider a case when an index on a STATUS column is present, and that STATUS column contains the value of either PROCESSED or NOT PROCESSED).  The book only mentioned that adaptive cursor sharing would be discussed in a later recipe – there was no comment in this section of the book regarding whether adaptive cursor sharing helped or further hindered the bullet-pointed items in the book.
</p></blockquote>
]]></content:encoded>
	</item>
	<item>
		<title>By: Wolfgang Breitling</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4801</link>
		<dc:creator><![CDATA[Wolfgang Breitling]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 20:28:58 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4801</guid>
		<description><![CDATA[But you asked it to do that with your method_opt of &#039;... size 254&#039;. Then, of course it does what you asked. It will do the right thing if you use method_opt=&gt;&#039;for all columns size auto&#039;]]></description>
		<content:encoded><![CDATA[<p>But you asked it to do that with your method_opt of &#8216;&#8230; size 254&#8242;. Then, of course it does what you asked. It will do the right thing if you use method_opt=&gt;&#8217;for all columns size auto&#8217;</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mladen Gogala</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4800</link>
		<dc:creator><![CDATA[Mladen Gogala]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 20:22:56 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4800</guid>
		<description><![CDATA[On the other hand, I just checked and here is my result:
SQL&gt; analyze table emp delete statistics;

Table analyzed.

Elapsed: 00:00:00.03
SQL&gt; select column_name,count(*) from user_tab_histograms
  2  where table_name=&#039;EMP&#039;
  3  group by column_name;

no rows selected

SQL&gt; get /tmp/1
  1  begin
  2  dbms_stats.gather_table_stats(
  3  ownname =&gt; user,
  4  tabname =&gt; &#039;EMP&#039;,
  5  method_opt =&gt; &#039;FOR ALL INDEXED COLUMNS SIZE 254&#039;);
  6* end;
SQL&gt; /

colum column_name format a15
SQL&gt; select column_name,count(*) from user_tab_histograms
  2    where table_name=&#039;EMP&#039;
  3   group by column_name;

COLUMN_NAME	  COUNT(*)
--------------- ----------
SYS_NC00009$		12
ENAME			14
EMPNO			14

Elapsed: 00:00:00.02
SQL&gt; 


In other words, Oracle has duly collected stats on the single-column primary key (EMPNO). The version is 11G:

SQL&gt; select * from v$version;

BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 - 64bit Production
PL/SQL Release 11.2.0.3.0 - Production
CORE	11.2.0.3.0	Production
TNS for Linux: Version 11.2.0.3.0 - Production
NLSRTL Version 11.2.0.3.0 - Production

Elapsed: 00:00:00.01
SQL&gt; 

January and April PSU&#039;s are applied. This is as new as it gets.]]></description>
		<content:encoded><![CDATA[<p>On the other hand, I just checked and here is my result:<br />
SQL&gt; analyze table emp delete statistics;</p>
<p>Table analyzed.</p>
<p>Elapsed: 00:00:00.03<br />
SQL&gt; select column_name,count(*) from user_tab_histograms<br />
  2  where table_name=&#8217;EMP&#8217;<br />
  3  group by column_name;</p>
<p>no rows selected</p>
<p>SQL&gt; get /tmp/1<br />
  1  begin<br />
  2  dbms_stats.gather_table_stats(<br />
  3  ownname =&gt; user,<br />
  4  tabname =&gt; &#8216;EMP&#8217;,<br />
  5  method_opt =&gt; &#8216;FOR ALL INDEXED COLUMNS SIZE 254&#8242;);<br />
  6* end;<br />
SQL&gt; /</p>
<p>colum column_name format a15<br />
SQL&gt; select column_name,count(*) from user_tab_histograms<br />
  2    where table_name=&#8217;EMP&#8217;<br />
  3   group by column_name;</p>
<p>COLUMN_NAME	  COUNT(*)<br />
&#8212;&#8212;&#8212;&#8212;&#8212; &#8212;&#8212;&#8212;-<br />
SYS_NC00009$		12<br />
ENAME			14<br />
EMPNO			14</p>
<p>Elapsed: 00:00:00.02<br />
SQL&gt; </p>
<p>In other words, Oracle has duly collected stats on the single-column primary key (EMPNO). The version is 11G:</p>
<p>SQL&gt; select * from v$version;</p>
<p>BANNER<br />
&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8212;&#8211;<br />
Oracle Database 11g Enterprise Edition Release 11.2.0.3.0 &#8211; 64bit Production<br />
PL/SQL Release 11.2.0.3.0 &#8211; Production<br />
CORE	11.2.0.3.0	Production<br />
TNS for Linux: Version 11.2.0.3.0 &#8211; Production<br />
NLSRTL Version 11.2.0.3.0 &#8211; Production</p>
<p>Elapsed: 00:00:00.01<br />
SQL&gt; </p>
<p>January and April PSU&#8217;s are applied. This is as new as it gets.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mladen Gogala</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4799</link>
		<dc:creator><![CDATA[Mladen Gogala]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 20:04:55 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4799</guid>
		<description><![CDATA[Thanks Wolfgang,  this is very good to know.]]></description>
		<content:encoded><![CDATA[<p>Thanks Wolfgang,  this is very good to know.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Wolfgang Breitling</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4798</link>
		<dc:creator><![CDATA[Wolfgang Breitling]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 19:41:12 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4798</guid>
		<description><![CDATA[&quot;About the only thing I concur with is the statement about histograms being unnecessary in case of uniform distribution. That is also my pet peeve with Oracle: it should not create histograms on columns with unique constraints and declared not null. Single column primary keys should not have histograms, period.&quot;
Oracle 11 does not gather a histogram for such columns - unless the column values have huge outliers and in that case it makes sense to have a histogram if you have a range predicate on that column. Where it still gathers a (frequency ) histogram is for columns with a single value for all rows and there I can not see any rationale for having a histogram. I&#039;m not sure right now, would have to test it, but it may be that the optimizer ignores the statistics ( density ) in this case and uses a more sensible density calculated at run time.]]></description>
		<content:encoded><![CDATA[<p>&#8220;About the only thing I concur with is the statement about histograms being unnecessary in case of uniform distribution. That is also my pet peeve with Oracle: it should not create histograms on columns with unique constraints and declared not null. Single column primary keys should not have histograms, period.&#8221;<br />
Oracle 11 does not gather a histogram for such columns &#8211; unless the column values have huge outliers and in that case it makes sense to have a histogram if you have a range predicate on that column. Where it still gathers a (frequency ) histogram is for columns with a single value for all rows and there I can not see any rationale for having a histogram. I&#8217;m not sure right now, would have to test it, but it may be that the optimizer ignores the statistics ( density ) in this case and uses a more sensible density calculated at run time.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Reiner Kuehl</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4795</link>
		<dc:creator><![CDATA[Reiner Kuehl]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 10:34:16 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4795</guid>
		<description><![CDATA[I agree with Mladen: The first statement seems definitely wrong: &quot;Histograms are useful only on indexed columns...&quot; That&#039;s at least what I have learned from Wolfgang Breitling in HISTOGRAMS - MYTHS AND FACTS (http://www.centrexcc.com/Histograms%20-%20Myths%20and%20Facts.pdf). Wolfgang shows clearly that Histograms on non-indexed columns are very useful when evaluating the cost of different join methods and orders.
The second point refers to bind variables and histograms:  It&#039;s true that in the past histograms and bind variables on the same column have been a very dangerous combination. With Adaptive Cursor Sharing (ACS) I would recommend on having a histogram on such columns. In other words: I would regard upon the &quot;Don&#039;t use histogram&quot; for columns with Bind Variables as misleading to say the least.]]></description>
		<content:encoded><![CDATA[<p>I agree with Mladen: The first statement seems definitely wrong: &#8220;Histograms are useful only on indexed columns&#8230;&#8221; That&#8217;s at least what I have learned from Wolfgang Breitling in HISTOGRAMS &#8211; MYTHS AND FACTS (<a href="http://www.centrexcc.com/Histograms%20-%20Myths%20and%20Facts.pdf" rel="nofollow">http://www.centrexcc.com/Histograms%20-%20Myths%20and%20Facts.pdf</a>). Wolfgang shows clearly that Histograms on non-indexed columns are very useful when evaluating the cost of different join methods and orders.<br />
The second point refers to bind variables and histograms:  It&#8217;s true that in the past histograms and bind variables on the same column have been a very dangerous combination. With Adaptive Cursor Sharing (ACS) I would recommend on having a histogram on such columns. In other words: I would regard upon the &#8220;Don&#8217;t use histogram&#8221; for columns with Bind Variables as misleading to say the least.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Charles Hooper</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4794</link>
		<dc:creator><![CDATA[Charles Hooper]]></dc:creator>
		<pubDate>Thu, 05 Jul 2012 02:33:53 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4794</guid>
		<description><![CDATA[Mladen,

Histograms on the 4th of July, probably not the best combination.  ;-)

Nice answers to start the dialog regarding this quote from the book.  In the book, the third bullet point item mentioned that adaptive cursor sharing with bind peeking would be covered in a later recipe, but the bullet point item did not mention whether or not adaptive cursor sharing makes the bullet pointed statement stronger or weaker.

I found something related to the book quote, from the Oracle Database 8.1.7 documentation:
http://docs.oracle.com/cd/A87860_01/doc/server.817/a76992/stats.htm#27069
&lt;blockquote&gt;
&quot;Histograms are not useful for columns with the following characteristics: 
* All predicates on the column use bind variables. 
* The column data is uniformly distributed. 
* The column is not used in WHERE clauses of queries. 
* The column is unique and is used only with equality predicates.&quot; 
&lt;/blockquote&gt;
Equality predicates... that must be the tie in with the 4th of July.

Since &quot;skewed values&quot; is in bold in the book, I wonder if the definition of &quot;skewed&quot; is important.  To me, that term means that some of the distinct values in a column have significant more (or less) rows with that value, when compared to other distinct values in the same column.  Uniformly distributed could mean that all of the distinct values in a column have roughly the same number of rows sharing those distinct values.

Let&#039;s set up a simple test case.  Since I read somewhere that NULL values cause problems, I will use an &quot;out of bound&quot; value of 10,000 in place of any NULL that might naturally appear in a column.  The table built will contain the numbers 0 through 100 in column C2 and 1 through 100 in column C3 with every 101st row considered to have a NULL value in that column (thus the value 10,000):
&lt;pre&gt;
DROP TABLE T2 PURGE;
 
CREATE TABLE T2 AS
SELECT
  ROWNUM C1,
  MOD(ROWNUM,101) C2,
  DECODE(MOD(ROWNUM,101),0,10000,MOD(ROWNUM,101)) C3,
  RPAD(&#039;A&#039;,200,&#039;A&#039;) C4
FROM
  DUAL
CONNECT BY
  LEVEL&lt;=100000;
 
COMMIT;
 
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=&gt;USER,TABNAME=&gt;&#039;T2&#039;,METHOD_OPT=&gt;&#039;FOR ALL COLUMNS SIZE 1&#039;)
&lt;/pre&gt;
In the above, statistics were collected on the table without histograms.  Let&#039;s try a couple of tests.  First, I want to see the rows with either the maximum value in column C3, or the &quot;out of bound&quot; value representing a NULL.
&lt;pre&gt; 
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 IN (100,10000);
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  a7htzq376rch4, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 IN (100,10000)
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;   1980 &#124;00:00:00.04 &#124;    3168 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;   1980 &#124;   1980 &#124;00:00:00.04 &#124;    3168 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter((&quot;C3&quot;=100 OR &quot;C3&quot;=10000))
&lt;/pre&gt;
That worked well without a histogram, the cardinality estimate is exactly correct.  If table T2 were then joined to another table, the accurate cardinality estimate could help the optimizer determine if table T2 should be the driving table, and which type of join method should be most efficient.

Let&#039;s try again with a SQL statement that produces the same resultset as the above SQL statement:
&lt;pre&gt;
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 100 AND 10000;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  g04bcqbxbut9p, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 100 AND 10000
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;   1980 &#124;00:00:00.02 &#124;    3168 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;    100K&#124;   1980 &#124;00:00:00.02 &#124;    3168 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter((&quot;C3&quot;&gt;=100 AND &quot;C3&quot;&lt;=10000))
&lt;/pre&gt;
That did not work so well.  There are only 100,000 rows in the table, and the optimizer is predicting that roughly 100,000 rows will be returned by the query.  If the resultset from table T2 were then joined with another table, the performance could be less than optimal due to the error in the cardinality calculation.

Let&#039;s try again, this time trying to retrieve the values without a &quot;NULL&quot; in column C3:
&lt;pre&gt;
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 1 AND 100;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  6z9pxyd4pguz8, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 1 AND 100
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;  99010 &#124;00:00:00.12 &#124;    9457 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;   1980 &#124;  99010 &#124;00:00:00.12 &#124;    9457 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
--------------------------------------------------
   1 - filter((&quot;C3&quot;&lt;=100 AND &quot;C3&quot;&gt;=1))
&lt;/pre&gt;
The optimizer also had difficulty with the cardinality calculation for this query.  Not a big deal for this query because there is no join, and no index on column C3.  But, what if there were a join or an index on column C3?

Let&#039;s try again with a histogram on column C3:
&lt;pre&gt;
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=&gt;USER,TABNAME=&gt;&#039;T2&#039;,METHOD_OPT=&gt;&#039;FOR COLUMNS C3 SIZE 254&#039;,NO_INVALIDATE=&gt;FALSE)
&lt;/pre&gt;

The first query:
&lt;pre&gt;
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 IN (100,10000);
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  a7htzq376rch4, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 IN (100,10000)
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;   1980 &#124;00:00:00.04 &#124;    3168 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;   2265 &#124;   1980 &#124;00:00:00.04 &#124;    3168 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter((&quot;C3&quot;=100 OR &quot;C3&quot;=10000))
&lt;/pre&gt;
The cardinality estimate is less accurate than before the histogram was built, but the estimate is still reasonably close.

The second query:
&lt;pre&gt;
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 100 AND 10000;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  g04bcqbxbut9p, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 100 AND 10000
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;   1980 &#124;00:00:00.03 &#124;    3168 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;   2265 &#124;   1980 &#124;00:00:00.03 &#124;    3168 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter((&quot;C3&quot;&gt;=100 AND &quot;C3&quot;&lt;=10000))
&lt;/pre&gt;
The cardinality estimate is significantly better than we saw before the histogram was built.

The third query:
&lt;pre&gt;
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 1 AND 100;
  
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,&#039;ALLSTATS LAST&#039;));
 
SQL_ID  6z9pxyd4pguz8, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 1 AND 100
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
&#124; Id  &#124; Operation         &#124; Name &#124; Starts &#124; E-Rows &#124; A-Rows &#124;   A-Time   &#124; Buffers &#124;
------------------------------------------------------------------------------------
&#124;   0 &#124; SELECT STATEMENT  &#124;      &#124;      1 &#124;        &#124;  99010 &#124;00:00:00.12 &#124;    9457 &#124;
&#124;*  1 &#124;  TABLE ACCESS FULL&#124; T2   &#124;      1 &#124;  98863 &#124;  99010 &#124;00:00:00.12 &#124;    9457 &#124;
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter((&quot;C3&quot;&lt;=100 AND &quot;C3&quot;&gt;=1))
&lt;/pre&gt;
Again, the cardinality estimate is significantly better than we saw before the histogram was built.

---

Sorry, I might have skewed too far off topic.  Any other comments?  &quot;What, if anything, is wrong with the above quote from the book?&quot;  I will withhold my comments in this article for at least 24 hours.]]></description>
		<content:encoded><![CDATA[<p>Mladen,</p>
<p>Histograms on the 4th of July, probably not the best combination.  <img src='http://s1.wp.com/wp-includes/images/smilies/icon_wink.gif' alt=';-)' class='wp-smiley' /> </p>
<p>Nice answers to start the dialog regarding this quote from the book.  In the book, the third bullet point item mentioned that adaptive cursor sharing with bind peeking would be covered in a later recipe, but the bullet point item did not mention whether or not adaptive cursor sharing makes the bullet pointed statement stronger or weaker.</p>
<p>I found something related to the book quote, from the Oracle Database 8.1.7 documentation:<br />
<a href="http://docs.oracle.com/cd/A87860_01/doc/server.817/a76992/stats.htm#27069" rel="nofollow">http://docs.oracle.com/cd/A87860_01/doc/server.817/a76992/stats.htm#27069</a></p>
<blockquote><p>
&#8220;Histograms are not useful for columns with the following characteristics:<br />
* All predicates on the column use bind variables.<br />
* The column data is uniformly distributed.<br />
* The column is not used in WHERE clauses of queries.<br />
* The column is unique and is used only with equality predicates.&#8221;
</p></blockquote>
<p>Equality predicates&#8230; that must be the tie in with the 4th of July.</p>
<p>Since &#8220;skewed values&#8221; is in bold in the book, I wonder if the definition of &#8220;skewed&#8221; is important.  To me, that term means that some of the distinct values in a column have significant more (or less) rows with that value, when compared to other distinct values in the same column.  Uniformly distributed could mean that all of the distinct values in a column have roughly the same number of rows sharing those distinct values.</p>
<p>Let&#8217;s set up a simple test case.  Since I read somewhere that NULL values cause problems, I will use an &#8220;out of bound&#8221; value of 10,000 in place of any NULL that might naturally appear in a column.  The table built will contain the numbers 0 through 100 in column C2 and 1 through 100 in column C3 with every 101st row considered to have a NULL value in that column (thus the value 10,000):</p>
<pre>
DROP TABLE T2 PURGE;
 
CREATE TABLE T2 AS
SELECT
  ROWNUM C1,
  MOD(ROWNUM,101) C2,
  DECODE(MOD(ROWNUM,101),0,10000,MOD(ROWNUM,101)) C3,
  RPAD('A',200,'A') C4
FROM
  DUAL
CONNECT BY
  LEVEL&lt;=100000;
 
COMMIT;
 
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=&gt;USER,TABNAME=&gt;'T2',METHOD_OPT=&gt;'FOR ALL COLUMNS SIZE 1')
</pre>
<p>In the above, statistics were collected on the table without histograms.  Let&#8217;s try a couple of tests.  First, I want to see the rows with either the maximum value in column C3, or the &#8220;out of bound&#8221; value representing a NULL.</p>
<pre> 
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 IN (100,10000);
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  a7htzq376rch4, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 IN (100,10000)
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |   1980 |00:00:00.04 |    3168 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |   1980 |   1980 |00:00:00.04 |    3168 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(("C3"=100 OR "C3"=10000))
</pre>
<p>That worked well without a histogram, the cardinality estimate is exactly correct.  If table T2 were then joined to another table, the accurate cardinality estimate could help the optimizer determine if table T2 should be the driving table, and which type of join method should be most efficient.</p>
<p>Let&#8217;s try again with a SQL statement that produces the same resultset as the above SQL statement:</p>
<pre>
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 100 AND 10000;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  g04bcqbxbut9p, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 100 AND 10000
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |   1980 |00:00:00.02 |    3168 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |    100K|   1980 |00:00:00.02 |    3168 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(("C3"&gt;=100 AND "C3"&lt;=10000))
</pre>
<p>That did not work so well.  There are only 100,000 rows in the table, and the optimizer is predicting that roughly 100,000 rows will be returned by the query.  If the resultset from table T2 were then joined with another table, the performance could be less than optimal due to the error in the cardinality calculation.</p>
<p>Let&#8217;s try again, this time trying to retrieve the values without a &#8220;NULL&#8221; in column C3:</p>
<pre>
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 1 AND 100;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  6z9pxyd4pguz8, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 1 AND 100
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |  99010 |00:00:00.12 |    9457 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |   1980 |  99010 |00:00:00.12 |    9457 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
--------------------------------------------------
   1 - filter(("C3"&lt;=100 AND "C3"&gt;=1))
</pre>
<p>The optimizer also had difficulty with the cardinality calculation for this query.  Not a big deal for this query because there is no join, and no index on column C3.  But, what if there were a join or an index on column C3?</p>
<p>Let&#8217;s try again with a histogram on column C3:</p>
<pre>
EXEC DBMS_STATS.GATHER_TABLE_STATS(OWNNAME=&gt;USER,TABNAME=&gt;'T2',METHOD_OPT=&gt;'FOR COLUMNS C3 SIZE 254',NO_INVALIDATE=&gt;FALSE)
</pre>
<p>The first query:</p>
<pre>
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 IN (100,10000);
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  a7htzq376rch4, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 IN (100,10000)
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |   1980 |00:00:00.04 |    3168 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |   2265 |   1980 |00:00:00.04 |    3168 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(("C3"=100 OR "C3"=10000))
</pre>
<p>The cardinality estimate is less accurate than before the histogram was built, but the estimate is still reasonably close.</p>
<p>The second query:</p>
<pre>
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 100 AND 10000;
 
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  g04bcqbxbut9p, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 100 AND 10000
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |   1980 |00:00:00.03 |    3168 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |   2265 |   1980 |00:00:00.03 |    3168 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(("C3"&gt;=100 AND "C3"&lt;=10000))
</pre>
<p>The cardinality estimate is significantly better than we saw before the histogram was built.</p>
<p>The third query:</p>
<pre>
SELECT /*+ GATHER_PLAN_STATISTICS */
  C1,
  C2,
  C3
FROM
  T2
WHERE
  C3 BETWEEN 1 AND 100;
  
SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY_CURSOR(NULL,NULL,'ALLSTATS LAST'));
 
SQL_ID  6z9pxyd4pguz8, child number 0
-------------------------------------
SELECT /*+ GATHER_PLAN_STATISTICS */   C1,   C2,   C3 FROM   T2 WHERE
C3 BETWEEN 1 AND 100
 
Plan hash value: 1513984157
 
------------------------------------------------------------------------------------
| Id  | Operation         | Name | Starts | E-Rows | A-Rows |   A-Time   | Buffers |
------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      |      1 |        |  99010 |00:00:00.12 |    9457 |
|*  1 |  TABLE ACCESS FULL| T2   |      1 |  98863 |  99010 |00:00:00.12 |    9457 |
------------------------------------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter(("C3"&lt;=100 AND "C3"&gt;=1))
</pre>
<p>Again, the cardinality estimate is significantly better than we saw before the histogram was built.</p>
<p>&#8212;</p>
<p>Sorry, I might have skewed too far off topic.  Any other comments?  &#8220;What, if anything, is wrong with the above quote from the book?&#8221;  I will withhold my comments in this article for at least 24 hours.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Mladen Gogala</title>
		<link>http://hoopercharles.wordpress.com/2012/07/04/histograms-what-is-wrong-with-this-quote/#comment-4791</link>
		<dc:creator><![CDATA[Mladen Gogala]]></dc:creator>
		<pubDate>Wed, 04 Jul 2012 18:27:53 +0000</pubDate>
		<guid isPermaLink="false">http://hoopercharles.wordpress.com/?p=6438#comment-4791</guid>
		<description><![CDATA[Well, there are several things here. First, histogram can also be used on a non-indexed column, in a nester loop type join, to determine the access path to the inner table. Also, it may be used to decide between hash join and sort/merge join,
Second, the phrase &quot;do not use histograms in the situations where columns is not used in the where clauses&quot; strikes me as odd. Histograms are not used in the situations, they&#039;re objects, created on columns or, in version 11G, groups of columns or column expressions.
Third, the advice about the bind variables. Oracle has bind variable peeking since version 9i and the first value used is put through the CBO paces, including histograms, to determine the execution plan. Some newer versions have something called &quot;adaptive cursor sharing&quot;, which will cause the statement to be re-parsed if the bind variable values have different characteristics. Again, ACS and bind aware cursors rely on histograms.
About the only thing I concur with is the statement about histograms being unnecessary in case of uniform distribution. That is also my pet peeve with Oracle: it should not create histograms on columns with unique constraints and declared not null. Single column primary keys should not have histograms, period.
Finally, happy 4th of July to everybody. Charles, you should ignore histograms today and devote some attention to BBQ. Ah yes: histograms should not be used on 4th of July.]]></description>
		<content:encoded><![CDATA[<p>Well, there are several things here. First, histogram can also be used on a non-indexed column, in a nester loop type join, to determine the access path to the inner table. Also, it may be used to decide between hash join and sort/merge join,<br />
Second, the phrase &#8220;do not use histograms in the situations where columns is not used in the where clauses&#8221; strikes me as odd. Histograms are not used in the situations, they&#8217;re objects, created on columns or, in version 11G, groups of columns or column expressions.<br />
Third, the advice about the bind variables. Oracle has bind variable peeking since version 9i and the first value used is put through the CBO paces, including histograms, to determine the execution plan. Some newer versions have something called &#8220;adaptive cursor sharing&#8221;, which will cause the statement to be re-parsed if the bind variable values have different characteristics. Again, ACS and bind aware cursors rely on histograms.<br />
About the only thing I concur with is the statement about histograms being unnecessary in case of uniform distribution. That is also my pet peeve with Oracle: it should not create histograms on columns with unique constraints and declared not null. Single column primary keys should not have histograms, period.<br />
Finally, happy 4th of July to everybody. Charles, you should ignore histograms today and devote some attention to BBQ. Ah yes: histograms should not be used on 4th of July.</p>
]]></content:encoded>
	</item>
</channel>
</rss>
