Monday, May 28, 2012

Network Chaos & Storm control

I ran into a little bit of network issues about a week ago. The core of the network was strained because of some broadcast storms. Dint realise it initially, it was kind of start-stop. But eventually when I di identify the root cause, I isolated the problem point & applied storm control. Well, if you are reading this & thinking isnt storm control supposed to be part of your basic config for core switches.... It is !! But here ,it was a case of someone stripping the config off and making some bad connection & bad configs(its a big boring story lets forget it for now)

Now the intersting part of storm control is that if you get too aggressive you get no packets flowing !!I wanted not broadcast storms & tried to control the broadcast to 0.1%. But what I realise was that I was telling the switch the number of PPS that it could pass through & when you mention it as 0.1 it takes it as '0'.

So the cli I used was storm-control broadcast level pps 0 which in haste  was  clearly wrong. What this does is that it blocks even the ARP requests that are sent as broadcasts. So eventually your network is alright but no data flows because the arp is not happening.

I eventually modified it to allow the desired level to pass thorugh. It was funny and intersting though how when you get too aggressive with the controls the packet flow is interupted.

Let me know if you have some intersting experiences like this or how you have utilised storm control or other mechanisms in non generic core part of your network.

2 comments:

  1. Many times, we need to configure this function at the time of deployment for switches, Majorly in service provider network this is must to ensure loop free and broadcast free network.... small mistake and chaos.....

    ReplyDelete
  2. I ran into a little bit of network issues about a week ago. The core of the network was strained because of some broadcast storms. Dint realise it initially, it was kind of start-stop. But eventually when I di identify the root cause, I isolated the problem point & applied storm control.

    ReplyDelete

Top 7 popular posts on cciedash !