Nevis Networks and Firewall Restrictions

This web page contains a discussion of the Nevis network environment, including some of the traffic restrictions imposed by the Nevis firewall. If you're configuring a device for one of the networks, here are the network parameters.

The Nevis Public Network (129.236.252 / 24)

The Nevis public network (call it the "inside" network if you're talking to a firewall expert) consists of the IP addresses 129.236.252.XXX, where XXX ranges from 0 to 255. These 256 addresses are a "class C" network, which means only the fourth field, or the final "octet," is variable within the network. This address range was assigned to the particle-physics research group at Nevis by Columbia University Information Technology ("CUIT").

CUIT has permitted Nevis to completely administer this network. This means we operate our own firewall to control access to the public network, and our own DNS servers to assign IP names to individual systems (e.g., 129.236.252.8 has the name franklin.nevis.columbia.edu).

Access to systems on the Nevis network by the rest of the Internet (the "outside") is restricted to some extent by our firewall policy; e.g., we only permit access via ssh to certain systems, notably those on the Linux cluster. However, it is best to be cautious when formulating a network security policy, so you should assume the following:

Any system on the Nevis public network is constantly being probed by the outside world. Every port is checked for vulnerabilities. Attempts are being made to login to every account with a well-known name, with all obvious password combinations.

This means that for a system to be on the public network, it should offer some service outside of Nevis (e.g., the Nevis web server), or there is a need to login directly to the system from the outside. Otherwise it is probably best for the system to be on the private network.

The Nevis Private Network (10.44 / 16)

The Nevis private network (also known as our "local" network) consists of the IP address range 10.44.XXX.YYY, where both XXX and YYY can vary from 0 to 255; this is called a "class B" network, and has 65536 possible IP addresses.

These IP addresses are only visible behind the Nevis firewall; that is, on the public and private networks. The outside world cannot access them directly; a system on the private network can ssh to a system on the outside, but a system on the outside cannot ssh to any of the systems on the private network.

Here's a more specific example. Consider two systems: kolya.nevis.columbia.edu with IP address 129.236.252.83, and eeyore.nevis.columbia.edu with IP address 10.44.40.67. The former system is on the public network, the latter system is on the private network. A user logged into kolya can ssh to eeyore, and vice versa. A user at CERN can ssh to kolya, but not to eeyore; if there was a critical need to login to eeyore from CERN, the user would have to login to kolya first, then login to eeyore from there. (Note that because of automount, there's actually little need to login to eeyore directly; this example would be more relevant for systems that were not part of the Linux cluster.)

When systems on the private network access the outside world (e.g., if someone on eeyore logs into CERN), to the remote systems the access appears to come from address 129.236.255.60, the "outside" IP address of our firewall. This is called "IP masquerading" or "Network Adddress Translation." Outside users cannot login or otherwise access this dummy address; such attempts are blocked by the firewall.

Examples of Nevis systems on the private network are:

• the nodes on the Nevis condor batch farm;
• almost all the offices in the Nevis research and electronics buildings (including the student boxes);
• the systems in the machine shop.

The private network has a limitation: for a machine to be on the private network, it must be connected to a switch that is set up using VLANs for the Nevis networks. That means if that if all the boxes in a room are connected through the same hub, they must all either be on the public or the private network. Offices with a separate ethernet port for each machine (which includes all the faculty and administrative offices) can have each port individually assigned to a network.

Security note

One apparent advantage of the private network is that, since systems on this network cannot be probed by outside attackers, there is less need to be as "paranoid" about security issues on such systems. However, there are still other avenues of attack on the private network:

• A user might read an e-mail with a malicious attachment, or visit a web page that contains a system exploit. (To use colloquial terms, a lock on a door doesn't help if you invite the attacker in.) Obviously nodes on the condor batch farm are not vulnerable to such an exploit (users can't login to them), but Windows systems are.

• A user might bring in an infected laptop with a virus that automatically probes other systems on its network. (Such a system would have to be plugged into the wired network; wireless laptops are in the sandbox network.)

• If either one of the above scenarios results in a Nevis system becoming infected, that infection might spread to other systems here, both on the public and private networks.

Therefore, it's important to maintain an awareness of security on all systems, even those on the private network.

The Nevis Sandbox (10.43 / 16) Network

With both public and private networks, why do we need yet another network at Nevis? The public network is for devices at Nevis that can be accessed by the outside world (e.g., workgroup servers); the private network is for devices that have an individual network identity at Nevis but don't need to be accessed by the outside world (e.g,. compute nodes).

There is another class of device: Machines that are brought into Nevis, but are not maintained at Nevis, that don't require access by the outside world, and for which for security reasons should be kept isolated from the rest of the Nevis networks. Primarily, these are laptops.

Therefore, to the extent that it's possible, we place laptops in the sandbox network. In the sandbox, the laptops can "play with each other" and potentially infect each other if they're not well-maintained. However, these systems have no special access to Nevis services; they see the Nevis networks in the same way as a system outside of Nevis.

In particular, the wireless access points at Nevis are all in the sandbox network. They are assigned addresses in the range 10.43.XXX.YYY, where XXX and YYY can each vary from 0 to 255.

Ping, Traceroute, and the Firewall

Commands such as ping and traceroute use an Internet protocal called ICMP. These commands are handy to tell if a machine is accessible via a network; unfortunately, they are also used by system crackers to identify and probe systems. There have been times when a substantial portion of our network bandwidth has been taken up by ICMP packets from attackers.

To provide a balance between diagnosing network problems and preventing attackers from flooding our network, the following policy is in place:

• Systems on the Nevis public and private networks can ping systems on the outside network.
• Systems outside of Nevis cannot ping systems at Nevis.
• Systems on the public networks can traceroute systems outside Nevis.
• Systems on the private network cannot traceroute systems outside Nevis.
• Systems outside Nevis can traceroute to systems on the public network, but not to the private network (which is invisible to the outside network in any case).
• Systems on the sandbox network can ping and traceroute each other, but that's it.
• Port-scanning programs such as nmap have the same restrictions as traceroute.

What's all this stuff about VLANs?

I assume you got here because you saw a label or notice referring to a VLAN somewhere on the Nevis network, and you want to know what it means.

You can search for information about VLANs from many sources. However, for the Nevis network, the following definition will suffice:

A VLAN is a way for more than one independent network to travel along a device, including a router, switch or even a single ethernet cable.

At Nevis, there are three independent networks: public, private, and sandbox. Therefore, we have three VLANs, numbered 1, 2, and 3 respectively. (Note added 28-Sep-2021: There are other esoteric VLANs on our network: phones (VLAN 7) and management (VLAN 41). These are only used device administration.)

As far as Nevis is concerned, all VLANs are port-based; in turn, a port-based VLAN can be controlled in one of two ways: untagged (no special protocol), or tagged (using the 802.1Q protocol). (There are other forms of VLANs, such as protocol- or MAC-based, but those aren't relevant at Nevis.) All of the managed switches at Nevis (yes, even the really old ones) are capable of handling port-based VLANs. The difference between untagged and tagged VLANs are:

• On a switch with a port with an untagged VLAN, all the traffic on that port is assumed to be on a given VLAN.

  For example, you can define ports 1-12 on a switch to be in VLAN 1,and ports 13-24 to be in VLAN 2. The means that the switch will treat all the network traffic on ports 1-12 to be on its own network, and will not try to transfer any of that traffic to ports 13-24 (and vice versa).

  From the perspective of configuring a switch, this is called "untagged". In the example of the previous paragraph, ports 1-12 are assigned untagged to VLAN 1, and ports 13-24 are assigned untagged to VLAN 2.

• The 802.1Q protocol, or "tagged" access. If a switch or router determines that an Ethernet packet belongs to a particular VLAN, it can include additional information in that packet to identify to which VLAN the packet belongs.

  Let's return to the example in the previous point: a switch with ports 1-12 defined to be in VLAN 1 untagged, and ports 13-24 to be in VLAN 2 untagged. Assume the switch connects to another switch via port 25. You can define that port 25 is "untagged" in VLAN 1, and "tagged" in VLAN 2. This means that Ethernet packets from VLAN 1 will be unchanged, but the switch will modify the VLAN 2 packets to include the tag. Both networks (VLAN 1 and VLAN 2) can now travel unambiguously along the same cable.

  For the sake of the example, let's assume that port 25 of this switch (A) connects to another identically-configured switch (B), perhaps in another building. The cable attached to port 25 of switch A connects to port 25 of switch B. Both switches must have port 25 configured so that VLAN 1 is untagged, and VLAN 2 is tagged. Switch B will then be able to distinguish the two networks on a single port, and send the VLAN 1 traffic to ports 1-12 and VLAN 2 traffic to ports 13-24; it will remove the "tags" for the untagged VLAN 2 traffic.

  This implies that you can have at most one untagged network for a given port on a given switch, but as many tagged networks as the switch allows.

In and of themselves, VLANs provide no network security. The security associated with the Nevis VLANs comes from the firewall restrictions assigned to each network, as described in the sections above.

In order for network traffic to go from one VLAN to another, there must be a router that bridges the two networks. At Nevis, that router is our firewall.

This implies that if the firewall goes down, the different VLANs cannot communicate with each other. (This is not strictly true; most of the workgroup servers have connections to both the public and private networks through their two Ethernet ports.) That is why, in the event of a power outage, the automatic power-monitoring software on the cluster will shut down all the systems if the firewall's power supply is getting low.