Why Blocks Shouldn’t Be Reference Counted

Erik Quanstrom

quanstro@quanstro.net

In the kernel, most memory is passed with a Block structure through Queues. Ken’s file server used MsgBufs (but not Queues). Version 7 unix, had struct buf (and struct pack).

The basic block structure looks like the following

    struct Block

    {

        Block*  next;

        Block*  list;

        uchar*  rp; /* first unconsumed byte */

        uchar*  wp; /* first empty byte */

        uchar*  lim;    /* 1 past end of the buffer */

    };

If a Block is read from a Queue, current code assumes that it has exclusive access to the Block; the data, rp, wp and lim may all be modified. If a Block is written to a Queue, the writer loses access.

It is tempting, when building a protocol on unreliable transport, to consider adding reference counting to Blocks, rather than keeping a copy for retransmission. However, adding reference counting (thus allowing multiple concurrent accessors) would not be safe given the current assumption; multiple processes with pointers to the same Block could race in their Block access.

One could argue, that perhaps one could have special knowledge of what goes on on the other end of the Queue or Channel. Making this assumption would make Plan 9 unmodular. Plugging in a different consumer might fail mysterously. For example, it could preclude swapping out network protocols.

Alternatively, one could convert all the code to lock Blocks when modifying data, rp, wp or lim. This lock would need to be an ilock, since Blocks may be accessed in interrupt context. There would be a significant performance cost added to Block access. In addition, this would be a huge and error-prone undertaking. 81 files in the current kernel use Blocks.