Hacking and Other Thoughts

Wed, 09 Jul 2008

Grinding away on TX multiqueue

The first batch of TX multiqueue patches went into net-next-2.6 today. Nothing real interesting, just infrastructure pieces. Part of the long road to the destination.

So then I turned towards the task of hardening up the remaining patches and the sequencing isn't all that easy.

F/V Northwestern from Discovery Channel's Deadliest Catch.
It's home dock is down the street from my house.

The existing multiqueue code I want to keep working until I "flip the switch" into the new stuff. But part of what the existing code does is manage the queues in a two-stage manner.

There is a global queue that controls the flow control of the entire device. And underneath there are per-queue controls. Both have to be enabled in order to send packets into a particular queue.

In the new stuff there is just one control per queue for better parallelization and scaling. The new per-queue datastructure holds the flag bits that manage the per-queue flow control state.

First there is a patch I add that transitions over to allocating multiple instances of the new datastructure, one for each TX queue, instead of having just one. But I left the existing multiqueue flow control routines using the egress_subqueue[] state bit array.

At that point the normal netif_stop_queue() et al. functions operate on TX queue zero in the new datastructures. This keeps everything working. And for non-multiqueue aware drivers (every driver we have, sans about 4 or 5) that is exactly what we want those interfaces to do.

But I want to migrate the multiqueue interfaces to use the per-TX state bits too. However, that doesn't work until both the generic networking, and the multiqueue drivers, stop using the compat netif_*_queue() routines. They have to be changed to only operate on the per-queue flow control state bits.

The current plan is to hit the drivers and the core code at the same time, all in one changeset. It's the only way I can come up with that doesn't break things at some intermediate stage.

Another fly in the ointment is the wireless QoS code. It uses the existing multiqueue infrastructure, and it's therefore something else I want to prevent from breaking mid-stream.