4.16 Choosing a blocking factor - an exercise in futility? The definitive answer.

Many of us spent considerable effort in our classic MPE days figuring out optimal blocking factors for files. I remember even writing a program that took record size and number of records and determined optimal blocking factor and total "wasted space". I know, at least intellectually, that on MPE/iX, blacking factor does not matter. But what is the real story. A poster to HP3000-L asked:

I've just been looking over the blocking-factor messages in the 3000-L archives and I'm afraid I'm still somewhat confused. It has been claimed that the best blocking factor can be had by just not specifying one and letting MPE decide. So I did some testing. Looks to me like if you don't specify a blocking factor, MPE divides the sector size (256 bytes) by the record size (adjusted upward to be an even number if necessary). It drops any fractional portion to get the blocking factor and if the calculated number is less than one, it uses one.

But doesn't MPE deal with 4096-byte pages now? Does the blocking factor matter? How does changing it change the efficiency of the storage or performance of the retrieval of the data?"

From Craig Fairchild, file system architect for MPE/iX:

Several people have already answered, so I'll just try to summarize. The blocking factor on MPE/iX is primarily used as a means of compatibility with MPE/V. It comes into play in two situations:

1) When using the STORE ;TRANSPORT option (to create tapes in the MPE/V store tape format)

2) When reading a file that has been opened with either NOBUF or MRNOBUF

In case 2, above, the blocking factor is used to insert fill characters into the 'block' of data that is returned to the caller, so that it looks exactly like it did on MPE/V, when data actually was stored in a block-by-block basis.

On MPE/XL, and now MPE/iX, data is stored with each record laid out 'end-to-end'. The blocking factor is not relevant.

Now to the performance question, the answer is, (drum roll please) 'It depends.'

No, really, it does! For buffered and NOBUF access, it makes no real difference. For MR NOBUF, it can make a difference. The idea behind MR NOBUF is to retrieve multiple blocks of data in a single system call, therefore saving on the overhead associated with multiple calls to FREAD. This is offset by the fact that the file system is doing extra work to pad the data being returned so that it looks as if the data had been stored in blocks. So the larger the number of blocks read (in your MR NOBUF read), and the less efficient the blocking factor, the more overhead that you have per each byte of real data moved.

So in the one case of MR NOBUF access, it is best to try and specify a blocking factor that allows the least amount of 'wasted space' per 'block'.