InteropNet Vegas 2009 – Mission Possible II

May 22nd, 2009 by mleonberger

2009 is another successful outing at the InteropNet NOC. Entering the last day of the show there have been no real hardware and/or software issues and only one blunder that caused a brief issue.

This being my second consecutive year working at the InteropNet NOC I have a different prospective and an increased appreciation for how special this event truly is. As you may or may not know the Interop network is staged and built in the SF bay area. Close to 20 vendors descend on a dusty warehouse and begin starting their IT careers over again by un-boxing equipment, running power and network cables, and racking IT gear. By the time all of the vendors have arrived and there is a basic internal/external network in place we are usually 2-3 days in. At this point, all of the vendors are in a mad dash to get their equipment/applications configured to start showcasing why they were chosen as an InteropNet vendor.

The next couple days are consumed with configuration, lots of meet and greets with the various vendors that you have to interface with all while the network is still being built out. This can prove to be very challenging considering there is only about 7-8 full days during “hotstage” and you are trying to work on a network that is usually going up and down a large majority of the time. This is no fault of the network/security vendors; they are merely implementing the changes that the rest of us are requesting.

So with only about 4 or 5 days of having a sporadic network to work with you can see why being selected as a an InteropNet vendor is not only an honor but also a major challenge for some vendors.

Being that we (ScienceLogic) are providing the Network Monitoring and Help Desk solution for the show we have to maximize every second we are here. The problem is it’s usually a hurry up and wait situation considering that enabling SNMP for network monitoring is probably about, oh say, 60th down on the to-do list for most vendors. Once all vendors have their gear stabilized and configured for our liking we usually have about 2 days to deploy a very heterogeneous, high-availability solution that would normally take 5-10x’s the amount of time.

Once hotstage is over, the gear is shipped to Las Vegas to the Mandalay Bay Convention center for the show. There is usually about a month between hotstage and the actual Interop show.

The InteropNet staff shows up about 5 days before the actual start date of the show and picks up where they left off in SF. Although, no major changes are supposed to take place in Vegas, most vendors are still scurrying around to finalize configurations.

Doing the monitoring for this show is quite interesting. Most of the vendors send out pretty senior level engineers but SNMP and network monitoring is not necessarily what they go to sleep thinking about each night. These guys/gals are experts at deploying their solution but most are not used to being in a NOC and having some stranger say “hey look at this error on your application”. So it is something very interesting to see these engineers’ reactions when issues arise.

What makes the InteropNet NOC so special is seeing all of these engineers supporting and reacting to issues with a staff of strangers for the most part. Watching this team work together to resolve issues in such a timely manner is literally like a watching pit crew in action. Just yesterday we had a “ped” (old term for pedestal of computer equipment from Interop in the 80′s, basically a network rack) go down and within seconds EM7 detected the failure and reported it. With the collaboration of the EM7 engineers and the 2 networking vendors (Enterasys and H3C) the issue was traced down to an individual port on a switch within the ped, a radio call was dispatched to a show floor technician and the issue was resolved within minutes. It took until my second year, but stepping back and observing this type of coordination and teamwork has been quite remarkable. Not often or ever do you get to work in a setting where literally each person is an expert in their particular field.

From the top-notch show management orchestrating all vendor activity to the volunteers who literally come from different countries to gain hands-on experience here I have just been blown away at how smooth and seamless things operate come show time considering the timeframe used to build this network/operation.

Long story long, the InteropNet NOC is a one of kind experience that I will NEVER forget. This is just the work part….believe me you don’t want to hear what happens after hours.

Popularity: 2% [?]

Related Posts

• No Related Post

2 comments