VLAN adventures

Posted: Fri, 29 December 2006 | permalink | No comments

Have you ever had a need for a large number of "real" network interfaces on a Linux box? For security reasons, you don't just want to hook everyone up to a single switch and use virtual interfaces (because it's too easy for one of the connected devices to play silly buggers with his neighbours). Up to a certain point, you can stuff more NICs into a box, and when it fills up you can switch to those outrageously expensive 2- and 4-port NICs. But eventually you're going to run out of space[1] or money. At that point, you need something more interesting.

For me, this week, that "more interesting" has been Ethernet VLANs. Linux Journal has a good intro to the technical aspects of how to setup VLANs and Linux.

VLANs are some crazy stuff. They're implemented by wrapping a little number around every Ethernet frame indicating which VLAN the packet is "on", and the switch (you need a higher-end device, appropriately configured) makes sure that packets don't roam onto ports they're not meant to be on. For a single switch, this isn't very interesting, and you don't really need an IEEE standard to make it work -- just a lot of little jumpers. However, VLANs are mostly useful on larger networks, which tend to use more than one switch. So you need a way for your VLANs to roam amongst your switches -- hence the packet tagging. You stick the tag onto the packets and fling them off to another switch, and packets coming in with the tag are corraled into their VLAN.

Naturally, you don't want end-user devices whacking their own VLAN tags onto packets (untrusted data, and all that), so you define most of your ports as being "untagged". Packets that come in on these ports are treated as regular ethernet frames and tagged with the appropriate VLAN ID, and packets that go out have their VLAN IDs stripped before being sent. (This implies that you can tunnel VLANs in VLANs, and I'm pretty sure that is doable in most switches, modulo some nasty problems that can, and probably will, occur). The ports that aren't designated "untagged" spit out tagged packets -- and this is where my interest lies.

Like most every other computer-related standard, Linux can decipher and work with VLAN tags (the 8021q module is what you're looking for). You just load up the module, hook your NIC into the appropriate port on your very expensive switch, define your VLAN tags with vconfig, and suddenly you've got vlanN interfaces, which map to real RJ-45 ports, and you can hang network devices off these and get lots and lots of "real" network interfaces on your Linux router.

It's all so simple, right? Nothing ever is, of course. The trials and tribulations I've had over the past two days have included:

Thankfully, once all those dramas were done, the whole thing just kind of fell into place, and now I've got a machine which has up to 2048(ish) network interfaces (with huge, glaring caveats on that upper bound, of course). With a bit of luck, the live installation in a couple of days will go smoothly.

You can stop sniggering now.

1. In this case, there wasn't a lot of space to begin with, as it's a 1RU box I'm working on, and the 4-port NIC didn't want to place nice with me for some reason anyway.

Post a comment

All comments are held for moderation; markdown formatting accepted.

This is a honeypot form. Do not use this form unless you want to get your IP address blacklisted. Use the second form below for comments.
Name: (required)
E-mail: (required, not published)
Website: (optional)
Name: (required)
E-mail: (required, not published)
Website: (optional)