NAME

aoe – ATA–over–Ethernet (AoE) interface

SYNOPSIS

bind –a #æ /dev
/dev/aoe/ctl
/dev/aoe/log
/dev/aoe/shelf.slot/config
/dev/aoe/shelf.slot/ctl
/dev/aoe/shelf.slot/devlink/0
...
/dev/aoe/shelf.slot/devlink/i
/dev/aoe/shelf.slot/ident
...

DESCRIPTION

The AoE (ATA–over–Ethernet) interface serves a three–level directory providing control and access to AoE targets. The interface provided is primarily intended for low–level control of the AoE initiator. See sdaoe(3) for the standard interface.

Top–level files
In order to access AoE targets, one or more Ethernet controllers need to be bound to the AoE initiator. By default, the system starts with no interfaces bound. For automatic binding of interfaces on boot, the aoeif configuration variable is set in plan9.ini(8). Ethernet interfaces are specified as ethern, not as #ln. To bind the first and second Ethernet devices on boot, add

aoeif=ether0 ether1
To bind ether1 to a running system:
% echo bind '#l1/ether1' >/dev/aoe/ctl
And to unbind it
% echo unbind '#l1/ether1' >/dev/aoe/ctl
When an interface is unbound, targets depending on that interface are removed.
Each local interface is called a netlink. The mapping of AoE targets to netlinks is called a devlink. Each devlink may see multiple interfaces per target. For example, if the local machine has one Ethernet address bound and the target has two interfaces on the same Ethernet segment, this will result in one netlink and one devlink with two Ethernet addresses. AoE frames are sent in round–robin fashion. Each successive frame is sent on the next address available on the next available devlink (local interface).
Normally the initiator automatically discovers and adds new device directories on startup. New devices are not added except as new interfaces are bound to the initiator. Several messages can be written to /dev/aoe/ctl which alter this behavior:
autodiscover toggle
If toggle is absent, the state of autodiscover is toggled. If it is the string on, it is turned on. Any other string turns autodisover off. This option is not useful after Ethernet devices have been bound.
discover shelf.slot
Attempt to find the named target on all bound interfaces.
remove shelf.slot
The converse of discover: remove the named target if it exists.
rediscover toggle
Allow or disallow rediscovery. This allows for automatic discovery of new targets. Unfortunately, it also allows automatic modification or loss of existing targets. This option is considered dangerous.
Reading /dev/aoe/ctl returns a list of colon–separated lines with keywords and their values:
debug
autodiscover
rediscover   Returns the current state of the variable named by the keyword. Writing the variable's name to the control file toggles the state of that variable.
ifn path      Path to nth bound Ethernet device.
ifn ea        Ethernet address of this device.
ifn flag       A flag of ``Up'' indicates that this interface is available.
ifn lostjumbo   Number of consecutive lost jumbograms.
ifn datamtu    Incorrect and unused.

Target subdirectories
Once configured, each AoE target is accessed via files in the directory named for its shelf and slot. For example, shelf 42, slot 0 would be accessed through the path /dev/aoe/42.0. The ident file contains the read–only, verbatim result of the identify unit ATA command. The config file contains the target's AoE configuration string. Writing to this file sets the targets configuration string.

Reading a target's ctl file returns a list of colon–separated lines with the following keywords and values:
state      ``Up'' or ``down''.
nopen      Number of clients using this target.
nout       Number of outstanding AoE frames.
nmaxout    Maximum number of outstanding frames allowed.
nframes    Maximum number of outstanding frames. Nframes is greater than nmaxout when the initiator is reducing the number of in–flight frames due to packet loss. It is assumed that packet loss is due to an overwhelmed target and not poor network conditions.
maxbcountMaximum number of data bytes per AoE frame. Using standard frames, maxbcount is 1024 or two sectors. AoE ATA headers are 36 bytes.
model
serial
firmware   The respective fields from the ATA identify unit command.
flag       List of flags useful for debugging. The flag jumbo indicates that jumbo frames are accepted, not that they are being used. Maxbcount should be consulted for this purpose.
Writing to the ctl file, the following commands may be issued
failio     fail outstanding i/o.
identify   send an ata identify device command to the target.
maxbno n    set the maximum number of block sent per packet.
mtu n      set the maximum number of bytes (including header) sent per packet.
nofail     never fail this target. This is useful if your root device is on this target.
setsize n   with no arguments, reset the device size to the size claimed. Otherwise, assume the device is the given size.
The data file may be read or written like a normal file except that reads and writes to this file are converted to AoE commands to the target, so transfers should be 512 or 1024 bytes long (or a larger multiple of 512 iff jumbo packets are in use). The size of this file is the usable size of the target.
The devlink directory contains one file for each interface the target was discovered on. The files are numbers from 0 to n and contain a list of colon–separated lines with keywords and their values:
addr       A space–separated list of the target's Ethernet addresses visible from this interface.
npkt       The number of frames sent on this interface.
resent     The number of frames re–sent. Frames are re–sent when they have been outstanding twice the RTT average.
flag       ``Up'' when the netlink is up.
rttavg
mintimer   Minimum timer and RTT average as per Congestion Avoidance and Control.
nl path    Path of the Ethernet device.
nl ea      Ethernet address of the local Ethernet device.
nl flag    ``Up'' if the local interface is up.
nl lostjumbo
Number of consecutive jumbograms lost.
nl datamtu
Unused.

SOURCE

/sys/src/9/port/devaoe.c

SEE ALSO

sd(3), sdaoe(3), vblade(8), snoopy(8)
 http://support.coraid.com/documents/AoEr11.txt
Van Jacobson and Michael J. Karels, ``Congestion Avoidance and Control'', ACM Computer Communication Review; Proceedings of the Sigcomm '88 Symposium in Stanford, CA, August, 1988.

BUGS

There is no raw file for executing arbitrary commands.
This is a fairly primitive interface; sdaoe(3) is usually more suitable.