In the follow-up article, table T6 was created with unique (distinct) values in each of the rows – thus there was less chance (maybe no chance) for SQL*Net compression. There is a summary table near the bottom of the follow-up article that compares the transfer time of tables T5 and T6 using 6 different computer configurations and various array fetch sizes.

then, after the first row (which is transmitted in full) subsequent transmissions could take advantage of SQL * Net compression.

An observation:
but if, for example, are transmitted rows with all distinct values (ie no SQL * Net compression) and assuming that the average length of a row is 400 bytes, I assume that SDU (maximum) can transmit 8192/400 = about 20 rows. If this is true I expect that with a value Arraysize greater than 20 , not get additional benefits. this consideration is correct? or there are other factors to consider?

Thanks

The first screen capture shows the first 4 network packets that were received by the client once the specified arraysize of 10 took effect. In the picture, you can see the first row ends at byte 03d2 (hex), the next row ends at byte 0428 (hex), which implies that each row after the first row requires about 86 bytes in the packet. With the array fetch size of 10, the first network packet is, for all intents and purposes, fully used. The second packet contains just 0161 (hex) bytes ~ 353 bytes, compared to the first packet which contained 05F0 (hex) bytes ~ 1520 bytes. The second and third fetch requests with the specified arraysize resulted in packets that were only about 66% filled – 03F2 (hex) bytes ~ 1010 bytes.

Compare the above with the second screen capture that shows the first 4 network packets that were received by the client once the specified arraysize of 5,000 took effect. In the picture, you can see the first row ends at byte 03d2 (hex), the next row ends at byte 0428 (hex), which implies that each row after the first row requires about 86 bytes in the packet. With the array fetch size of 5000, the first network packet is, for all intents and purposes, fully used (1520 bytes). The second, third, and fourth packets are also fully used (1520 bytes).

The timing information appears at the top of each packet.

Trace Time   Time Delta
001.230571
001.230760   0.000189
001.232660   0.001900
001.234423   0.001763
 
Trace Time   Time Delta
428.107961   
428.108409   0.000448
428.108857   0.000448
428.109397   0.000540

By the end of the fourth packet, the array fetch size of:
* 10 just finishes transferring row A0001085, with a total time delta of 0.003852 seconds.
* 5,000 was just starting to transfer row A0001114, with a total time delta of 0.001436.

Sorry for the delay. There is a very long “Network Monitoring Experimentations 6″ blog article that is planned for Friday. I don’t want to give away any of the details about that article just yet, but hopefully it will be useful/helpful.

I did not have a chance to look at a network packet trace yesterday – the packet trace that I captured for tomorrow’s blog article was over 2GB in size, and that caused the 32 bit client to become unstable when loading the trace file at the end of the capture (the entire trace file was lost in the process).

I think that it might be important to mention that the first row transmitted by the server contains values for all columns. It appears that starting with the second row the server starts transmitting only those columns that change in value from the previous row. That means that in this test case fewer rows fit into the first 8192 byte SDU unit compared to the second 8192 byte SDU unit, or for that matter the first network packet compared to the second network packet.

“That I think there is a problem with the test”

I also think there is a problem with my test:)

Calculations made (109 bytes for row) are inaccurate because I considered in my calculations (Wrong) only two columns (not 8 times the column c1 and 8 times for the column c2, as in your example)

Some observations:

1) the size in bytes of the columns c1 and c2:

SQL> select max(vsize(c1)) , min(vsize(c1)), max(vsize(c2)), min(vsize(c2)) from t5;

MAX(VSIZE(C1)) MIN(VSIZE(C1)) MAX(VSIZE(C2)) MIN(VSIZE(C2))
-------------- -------------- -------------- --------------
             9              9            100            100

So, for me every row has 109 bytes (and hence 9 * 8 for column c1 + 100 * 8 for column c2)

the strange thing is that if I run the query in your example:

SQL> SELECT   'A'||TO_CHAR(ROWNUM,'0000000') as c1,  RPAD('A',100,'A') as c2
  2  FROM   DUAL;

C1        C2
--------- ------------------------------------------------------------------------------------------
A 0000001 AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

not seem the same as (a more space):

select 'A' || '0000000' || to_char(ROWNUM) as c1 from dual;

C1
---------------
A00000001

2) the test done on my DB

SET ARRAYSIZE 10;

Elapsed: 00:19:31.28

Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
     106432  consistent gets
          0  physical reads
          0  redo size
   90504856  bytes sent via SQL*Net to client
     699537  bytes received via SQL*Net from client
      99902  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

SQL> SET ARRAYSIZE 50;

Elapsed: 00:05:17.81

Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      27781  consistent gets
          0  physical reads
          0  redo size
   86834056  bytes sent via SQL*Net to client
     140097  bytes received via SQL*Net from client
      19982  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

SQL> SET ARRAYSIZE 100;

Elapsed: 00:03:25.46


Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      17950  consistent gets
          0  physical reads
          0  redo size
   86375174  bytes sent via SQL*Net to client
      70167  bytes received via SQL*Net from client
       9992  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

SQL> SET ARRAYSIZE 500;

Elapsed: 00:02:19.06

Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
      10085  consistent gets
          0  physical reads
          0  redo size
   86009809  bytes sent via SQL*Net to client
      16222  bytes received via SQL*Net from client
       2000  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

SQL> set arraysize 1000;


Elapsed: 00:02:06.73

Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       9102  consistent gets
          0  physical reads
          0  redo size
   85963179  bytes sent via SQL*Net to client
       8230  bytes received via SQL*Net from client
       1001  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

SQL> set arraysize 5000;

Elapsed: 00:02:10.03

Statistiche
----------------------------------------------------------
          0  recursive calls
          0  db block gets
       8315  consistent gets
          0  physical reads
          0  redo size
   85925641  bytes sent via SQL*Net to client
       1830  bytes received via SQL*Net from client
        201  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
     999001  rows processed

For example if I get the last case (ArraySize = 5000) I 85925641/999001, then about 86 bytes transmitted per row. But I know I should have 9 * 8 = 72 bytes for column c1 + 100 * 8 = 800 bytes for column c2, then a total of 872 bytes per row. In conclusion I think that takes advantage of SQL * Net compression, as discussed by J. Lewis the link you highlighted.

But what does this mean? SDU that transmits up to 8192/86, then about 95 rows? (SDU size (8192) / 86 bytes transmitted per row)

thanks

Thank you for posting your analysis. Based on the graphs that were provided, I agree with you that a fetch array size of 500 seems to be ideal.

I think that there is a problem with the test, at least with the design of the data for the test. Each row retrieved was intended to be roughly 875 bytes long (7*8 bytes for column C1 + 100 * 8 bytes for column C2 + overhead). Your observation that each line/row is about 109 bytes disagrees with the intended row length. So, what is happening?

I must admit that I did not catch this earlier. Notice in the third picture (networkmonitoring5-3.jpg) that we see in the packet detail at the bottom of the picture a series of many letter A’s immediately after a number – this is the first row of the fetch. Compare that picture with the fourth picture (networkmonitoring5-4.jpg) which shows one of the later packets in the fetch array size 5000 test. The fourth picture does not show the sequence of the letter A’s – it shows a single letter A, but that is actually part of column C1. Strange… until you read Jonathan Lewis’ recent blog article on the topic of SQL*Net compression. With Jonathan’s post in mind, it now becomes a bit more clear why there was such a significant decrease in the time to fetch 50 rows at a time compared to 10 rows at a time – I would need to double-check the packets, but it looks like 49 of every 50 rows (98%) in each fetch could be compressed when the fetch array size was set to 50, while only 9 of every 10 rows (90%) in each fetch could be compressed when the fetch array size was set to 10. That does not sound like much of a difference until you compare the uncompressed byte size with the compressed byte size. The larger fetch array size also should have reduced the number of partially filled network packets leaving the server.

In short, a smaller fetch array size, such as 100, may be ideal with a different data set that contains less frequently repeated data (thus less compression takes place).

—
Edit May 9, 2010: after reviewing the Wireshark packet capture, the fetch array size did not affect the SQL*Net compression – only the first row in the result set was transmitted in full, with the rest of the rows taking advantage of SQL*Net compression.

So in conclusion, my answer is ArraySize = 500.

But I wanted to make a couple of considerations:
the first, I repeated the test by monitoring elapsed. Elapsed times decrease
with increasing of ArraySize. But if the response time is less, that does not mean that I believe is the most efficient, right?

I could imagine “set always ArraySize 5000 and I solved my problems.” Creed
not work well, because I certainly increase memory usage and then I consider the latency, which is my second consideration.

I think we must take Consider SDU which has a default value of 8K.
In tests I conducted on my DB I know that each line is 109 bytes, then
SDU is filled with rows about 8192/109 = 75 (ie SDU= 8k divided row length). If the values of your tests are not very different from mine, this should explain why it has the lowest latency for ArraySize values between 50 and 100, or wrong ….