13 March, 2015

Starting the Migration from Time Warner High Speed Internet to Verizon FiOS

(By the way...in the following, "TWC" means "Time Warner Cable," if you didn't already think of those letters that way.)

I think the scarier parts are done with as I post this.  I was going to handle Verizon by simply creating another VLAN, but I discovered my particular router platform (a Dell OptiPlex GX1 350MHz Pentium II with some "ancient" NICs) did not support dhclient with VLANs.  Whether this is a problem in the core kernel, the VLAN code, the NIC drivers, or dhclient, it's tough to say.  Using tcpdump, I do see a DHCP server reply...so why this doesn't get back properly to dhclient, I don't know, and I don't much care.  I solved it with hardware, by putting in another NIC.

Why scarey?  The rational mind thinks, powering down a computer, partially disassembling it (OptiPlex GX1s have a riser card with the PCI slots (and ISA slots!)), adding the requisite hardware, reassembling it, and turning it back on should be normal, ordinary, and just work as expected.  But of course the experienced person knows cables (e.g., IDE) get tugged, components get bumped (e.g. RAM modules), hard disks every once in a blue moon refuse to spin up or otherwise refuse to come online, and so on.  This is why using a VLAN interface (e.g. eth2.105, for physical port eth2, VLAN 105...which is the ASCII code for "i", symbolic of "Internet'...I already have a 73/"I" VLAN for TWC) was preferable, because of very few if any hardware unknowns.

Complicating that was the fact that my systems room (a fancy name for what was no doubt intended by the house architect as a bedroom) does not have a whole lot of space, so the "home" for my capable but power-hungry desktop (a Dell Precision 670 which was given to me ) is on top of the router computer.  That one would be expensive to replace if it got munged by moving it.  The video board in it (a PCIe nVidia card) is a bit finnicky, and that was my worst worry; same concerns as twiddling with the router, of HDDs which had their last useful poweron cycle, mainboards getting flexed by the movement just enough to make them fail or become unreliable, etc.  But as it turns out, I'm typing just fine on the '670 to write this post.

I know, I know...I probably sound like Boober Fraggle.  But this is an unfortunate attitude which comes after years and years of experience with stuff failing for no apparent reason at all.  In fact, my very first Linux router was a Dell Dimension XPS P75.  One day, I did something with it, and it had to be powered down.  It would not power back on.  So thankfully that was as easy as taking the parts out of it and jamming them into a Dimension XPS P133.  So not all was bad with that, I actually ended up with a faster router that day, so it could sustain more throughput (which was to be thrown at its way a couple of years later, from 640/90Kbps Verizon ADSL to 10/0.5Mbps Adelphia Power Link).

A certain amount of anxiety can be lessened by thinking of these things and devising, as best as you can, fallback positions or workarounds.  For example, if putting the new NIC in somehow causes an electrical conflict (e.g., IRQ lines), chances are fairly good that if you just remove this NIC, things will go back to the way they were and you can figure out some other strategy (e.g., try a different NIC, or a different manufacturer's NIC).  But of course, nothing like that is absolute.  You may find out inserting that NIC caused a freeze, reset, kernel panic, or whatever, but in the process of removing it, you may have bumped the RAM, and now even with the NIC gone you're still down.  Maybe you just go to the basement and get another computer, and try transplanting the NIC and hard disk.  If the HDD fails, that's a whole new many hour kettle of fish, involving setting up a new one, restoring from backup, and such.  To lessen stress/anxiety, it helps to have a copied and offline tested HDD, but that itself is a lot of work which may be totally unnecessary.  (Yes, I have about a dozen old, old computers in my basement which serve terribly these days as a desktop but function just great as something like a router, or as a source of spare/repair parts.)

You might ask, "why the NIC?"  That's because with most ISPs which use DHCP, the address you obtain is tied to the MAC address of the NIC which requested an IPv4 address.  Not all NICs or NIC drivers support setting an arbitrary MAC address, especially older ones made before people thought of even doing such things.  And if the MAC address changes, the IP address will change, and there will be a whole slew of other things scattered throughout the Internet which must be updated...DNS entries, tunnel endpoints which are administered/work by IP address, ACLs for various bits of software on the router, iptables(8) rules, DNS secondaries which must fetch my masters from a new address, and so on.  So if things go as planned, none of that additional work (and accompanying stress) has to be done to be functional again.  Hopefully if it's the computer dying, the HDD's and NIC's new home will "just work."

Mind you, at some point, the IP address changes will have to happen, it's just that again, there is the fallback position of "TWC still works with the current IPv4 address," and the IP address transition doesn't have to occur ASAP in order to have certain Internet stuff working...such as email.

The first go of it caused quite some tension in me because booting would hang at the eth1 (TWC) initialization.   Oh, great, what now???  Was the new NIC interfering somehow, such as stealing an IRQ?  Was it changing the device names so that eth1 was no longer TWC?  Actually, it was a whole lot simpler.  I thought I knew the color code of the Ethernet cables I used...yellow for Internet (WAN), other (in this case blue) for LAN.  But as I'm staring into space contemplating what was wrong, my gaze was towards my switch.  It is mounted (screwed) to the side of my desk, with two rows of ports running vertically.  The top four ports are natively in VLAN 73.  And as I'm thinking of how to diagnose what's going on, it dawned on me there is a blue cable in port 1/4 and a light gray cable in 1/1.  Notice none of those is yellow.  Ahhhh....LAN is plugged into WAN, and WAN is plugged into LAN.  So trying to do DHCPDISCOVER or DHCPREQUEST on a network segment which has no (other) DHCP server is not going to work out well at all.  (I say "other" because for that segment, the router itself is the DHCP server.)

So right now the new NIC is in, and in fact it's doing DHCP with Verizon instead of Verizon's supplied Actiontec router.  Everything "critical" seems to be working: email, Google Voice, Web surfing, YouTube, etc.  For some oddball reason, my Sipura SPA2000 wouldn't register with PBXes.org; I don't use it that much so I decided to drop that for now as it's not really critical.  I have briefly changed the default route to point out the new FiOS NIC, and it tested at good speeds.  I discovered that sometimes being dually homed like this causes some minor difficulties, such as somehow packets sneak out with the wrong source address, so NAT rules need to be added on each Internet interface which SNAT from the wrong to the right address.  At least for residential class Internet service, reverse path filtering is in full force, and therefore asymmetric routing simply will not work.  I cannot send a packet to Verizon with a TWC source address, nor vice versa.  If this were perhaps a business account, I might be able to do that sort of thing "out of the box," or at least be able to correspond with network engineers at both ISPs to ask them to allow source addresses which I'm using from the "other" ISP.

UPDATE ON 15-Mar-2015: I left the SPA2000 ATA unplugged overnight.  I basically think this was the SPA not sending stuff for many contiguous hours, and PBXes.org not sending any traffic responding to what the SPA was trying to do.  Therefore Linux aged out the NAT entries after PBXes stopped sending stuff.  So it will now function.  I also turned off autoprovisioning via TFTP and connected it for a few minutes directly to TWC.  Next I'll try turning autoprovisioning back on and see if it still will make/receive calls; I suspect that won't be detrimental, but we'll see.

I also might add I had my first "production" use of FiOS this morning.  I had another Linux box set up as a router, and manually pointed my Nexus 7's default route to it so it would mainly use FiOS instead of TWC (I say "mainly" because other things like DNS will still use the old network path).  Since the installation on Wednesday, really all I have done is speed tests at www.speedtest.net and DSLReports.  But today my Titanium Backup backup weighed in at 272 MB, mainly because several large apps got updated (Google+, Angry Birds Rio, and some more).  This would have likely taken the better part of an hour at the 1Mbps speed of TWC to sync to Google Drive.  But with 25 symmetrical FiOS, it was only about 8 minutes.  The speed was almost comparable to the rsync(1) of those files I do to a computer on the LAN!

Well, one of these days, I will finish the transition to FiOS.  The whole goal was to get better Internet at less of a monthly price, and to disconnect TWC Internet (leaving just basic TV).  We'll see how long this takes.


Direct all comments to Google+, preferably under the postabout this blog entry.

English is a difficult enough language to interpret correctly when its rules are followed, let alone when the speaker or writer chooses not to follow those rules.

"Jeopardy!" replies and randomcaps really suck!

Please join one of the fastest growing social networks, Google+!