jump to navigation

TLS circuits and autonegotiation August 30, 2009

Posted by jamesisaac in Uncategorized.
trackback

Our main office and our data center are linked via fiber provided by Cox Communications; we’re on their Metro WAN fiber ring around the city. The service is called “TLS” which apparently stands for Transparent LAN Service circuit. They have gear in our telco room and hand us an ethernet cable; at the data center we have a matching ethernet cable that drops from the raceway into the rack. From our equipment’s perspective, the cable is functionally equivalent to a straight-through ethernet cable.

For all practical purposes, we could just slap a switch into the datacenter rack, plug the TLS cable into it, plug the other end of the TLS cable into our main office switch stack, and have the data center just be another distribution point attached to our switches. This is not really a good way to go, because the TLS segment adds about 2-3 ms of latency, plus it is bandwidth-constrained to 10mb/s. Prior to our routers arriving, I did in fact do this very scenario, and it worked pretty well. I could tell that we were leaking broadcast traffic across the fiber, plus our network analysis tools would complain about unexpected latency.

Once the routers arrived, I set them up back-to-back in our server room using a crossover cable between the FastEthernet0/1 ports (fe0/0 is “inside”, fe0/1 is “outside” on both routers), and made sure that my ip changes would go smoothly. The plan was to take one router over to the datacenter, mount it, then have a guy at the main office and another one at the datacenter, and at a coordinated time, unplug the TLS cable from the respective switches and plug into the routers’ “outside” ports. They worked perfectly back-to-back, so everything should work fine.

Except that it didn’t.

The symptom is this: line is up, protocol is down on both routers.

Well, crap. What’s going on? When the TLS is plugged directly into the switches, it works great. Plugged into router ports, though, it doesn’t work. Up/Down usually means there’s something physically wrong with the line, but I know that both the TLS line and the interfaces work.

So, call the Cox engineers. “Oh, yeah, that’ll happen. You’re not using the right encapsulation. We’ll have our engineer call you back.” Hmm… encapsulation… HDLC? PPP? L2TPv3? I’m not doing any encapsulation on the switches, so why does it work there? This doesn’t make any sense.

Call Cisco. The helpful tech looks at my configs and they look correct. Perhaps it’s something with the interface negotiation. I try autonegotiate. 100/Full. 100/Half. 10/Full. 10/Half. Nothing.

Cox calls back. Ok, it’s not an encapsulation issue per se. We need to be doing 802.1q encapsulation. Their gear is a “layer 1 transport”, and we need to use subinterfaces with vlan tags on our traffic. Ok, great… except I’m already doing that, and I KNOW that my vlans work as they were tested in the back-to-back crossover cable scenario.

The Cox engineer calls to the SOC to get someone on the line who can look at the TLS port itself. He calls back an hour later with interesting news. “The line at the datacenter is down but the one at your office is up.” Strange, it looks down to me. “Well, it’s up physically at the datacenter but since it’s down at your office the link can’t come up so it looks down at both places.” He suggests we check the negotiation. “What are you set for?” “Auto/auto.” “Me too.” “Let’s try hard-coding it.”

And the line comes up.

We bring it down, check it again, bring it up again.

GAAAAHHHH!

Nothing wrong with the configurations, just two devices that didn’t want to negotiate with each other. Hard-code the settings, and they’re fine. But – hard-code on both sides, which means my equipment and the ISP’s equipment. That was the missing step here.

Advertisements

Comments»

No comments yet — be the first.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: