OSPF ABR-on-a-Stick?

Someone posed the question to me the other day as to whether or not inter-area traffic in OSPF had to traverse the backbone area (Area 0) from a physical perspective. They were listening to a video and heard the instructor say that traffic had to traverse from the non-backbone area, to the backbone area, to the other non-backbone area to conform to OSPF’s loop prevention mechanisms.

My initial thought, was, “Well, from a physical perspective there’s no requirement that I’m aware of…” but I choose not to lean on my own understanding anymore, so I broke out the CSRs and labbed it up to simulate a situation where the traffic traverses an ABR without physical links in Area 0. Let’s take a look at the setup:


Above, you see a typical (broken) design scenario. I have only highlighted the loopback addresses because they are really all that is necessary to show in this particular situation. Note that the physical link between R6 and R7  is in Area 67, and  the physical link between R7 and R9 is in Area 79. R6’s loopback is in Area 67, and R9’s loopback is in Area 79. Currently, R7’s loopback is not assigned to an area. Let’s take a look at the broken design’s database from the different area perspectives:



As you can see, R6 is not learning any routes via OSPF. Its directly connected link to R7 will be in the RIB as a connected route, and remember that R7’s loopback is not currently advertised into OSPF. However, why do we not see R9’s loopback or connected links? Here’s a hint:


There are currently no Area Border Routers. Remember that R9’s loopback is in Area 79, and both R6 and R9 are OSPF Internal Routers (all of their links are in a single area) and both areas are NOT Area 0 (the backbone area).
R9 will show a similar output:


Without an ABR, no Type-3 LSAs will be generated for Inter-Area traffic to function properly. Remember that Type-1 and Type-2 LSAs are limited to local area scope, and Type-3 LSAs are generated by ABRs to hide topology information from the other areas while still allowing inter-area communication to occur.

Also note, that, although this is a “broken” design, R7 will actually have routes to both R6 and R9’s loopback addresses, since it has links in both areas and therefore has Type-1 and Type-2 LSA’s to build the intra-area topology information for both areas without the need for an ABR:


Note the lack of the trailing (IA) for the OSPF routes to R6 and R9’s looback addresses. These are OSPF intra-area routes. Check out the databases for Area 67 and Area 79, respectively:


So, back to our original question–can we fix this “broken” design to allow inter-area communication without the addition of any PHYSICAL links in Area 0, and, if so, will traffic flow as expected without traversing a physical Area 0 link? Let’s take a look. Note that in the below diagram/config, the only thing that has changed is that I have added R7’s loopback0 prefix to Area 0:


The first question–will this make R7 an ABR, even though it’s a logical loopback interface and it has no physical interfaces in Area 0? Let’s see:


Note that, I ran the ‘show ip ospf’ command and piped to include ‘border router’ which tells us whether or not the router is an ABR. Before placing the Loopback0 interface in Area 0, it was not an ABR. After placing the loopback in Area0 as shown above, it now reports that it is, indeed, an ABR. This makes sense from a strictly logical perspective, as a functional ABR is defined as a router with at least one link in a non-backbone area and at least one link in the backbone area (Area 0). You will sometimes see that an ABR is defined more simply as a router with links in at least two areas, which is both correct and incorrect from a functionality standpoint, considering whether or not one of the areas is the backbone area.

Let’s see if R6 and R9 agree that R7 is now an ABR:



Boom. Both do. This means that R7 should be generating Type-3 LSAs for both areas and sending them bidirectionally:



Sure enough, R6 and R9 both have Summary LSAs from R7 ( who is now acting as a functional ABR. We have verified the database, and the last thing to verify is the data plane. Let’s make sure that R6 can reach R9’s loopback0 prefix and vice-versa with a Loopback0-sourced ping and traceroute from R6:


Sure enough ladies and gentlemen, traffic traverses just fine. More importantly, to answer the original question, traffic traverses two links (Gi1.67 and Gi1.79), neither of which belong to Area 0, but both of which are connected to an ABR:


So, to sum it up: the traffic doesn’t have to traverse a physical link in Area 0, but it does traverse a node connected to Area 0, an ABR (which, by its very definition, is connected to Area 0). So, even though the traffic flow from a physical perspective is (Area67->Area79), from a node overview the traffic still traverses the way you would expect it to (Internal Router->ABR->Internal Router) or (Area 67->Area0->Area79) or (R6-R7-R9).

Hopefully this helps you cement some of the finer points of OSPF.


Why Working for $1.25 Per Hour was the Best Career Decision I Ever Made

$1.25 per hour. Five quarters for 60 minutes of work.
That was my hourly wage when I took an internship at a new company–a 91.6% pay cut compared to my former employer.

It was the best career decision I ever made.

My Own Journey

I had a fairly decent job at an IT company–but my growth and advancement opportunities there were extremely limited. Even more frustrating was the fact that I could not specialize in the field I was truly passionate about. I was young and agile, and it was time for a change.

I went to a job fair to speak to a single company. No other company there was even on my radar. I had done my research and admired the portrayed culture, realized they were a place I could follow my passion, and saw the company as a place where I could truly absorb some wisdom from industry experts and jump-start my career.

Then the atom bomb hit:

“We don’t really have any technical openings right now. But if you are interested in marketing or sales, here is some information for you. I’ll take your resume and let you know if anything comes up.”

Devastated, I handed him my resume. I never expected to hear anything back from them. My shot was gone. My resume’s fate was a trashcan or shredder–if it even made it that far.

I walked around the job fair to save face, pretending to be interested in the myriad of other companies there. I left the job fair that day with pamphlets from around a dozen companies interested in my skill set.

I promptly tossed them in the parking deck trashcan–they didn’t make it that far.

And then something weird happened.

 A few weeks later I received an email from the HR director:


I might have an interesting opportunity to talk to you about.  Let me know if you’re around today.”

Had they taken my resume? What was this about?

A technical internship. It didn’t even exist before.

They must have seen something in me–and a worthwhile company will.

This was my chance to jump-start my career–I immediately gave my current employer my two weeks notice.

Perhaps the most enlightening moment about the whole situation was the conversation with my current employer:

“Are you going to make more there?”

“..I think so..”

“You don’t KNOW?”

In this candid moment, I realized I had never even asked what the pay rate would be.The reason being I really didn’t care–this was an opportunity, it wasn’t about the paycheck.

Advancement is Never Ideal

Too often, I hear people uttering: “Can’t get experience without a job, can’t get a job without experience.”

Millennials live by it. Older generations facing age discrimination or belated career changes are victim to it as well.

Often, I think people are unwilling to take a lowly paid (or unpaid) internship or apprenticeship due to pride–“I’m too good, too established (or too old) to go through an internship, I’m worth more than that petty pay.”

Or, on the other end of the spectrum, people lack the confidence–“I may not make the cut, I’m too old to start over, I’m too inexperienced.”

Now, I can certainly understand when you have mouths to feed and bills to pay that a significant pay cut may not be ideal. However, advancement is never ideal. It’s never a strictly linear progression.

Sometimes, you have to step down to take the next step up. Think of it as two steps back, twelve steps forward.

This concept is made evident in the diagram below–my actual pay rates for the 24 months prior to starting my internship at a new company and the subsequent 24 months after being hired on:

Can you guess where the internship pay rate is at?


What if we do have mouths to feed? Bills to pay?

Sacrifices. I’m not telling you to rack up debt. I’m not telling you to let your children starve. Be smart with your money. If you want it bad enough, you will find a way–otherwise, you will find an excuse.

Does it mean taking two crappy jobs so that you can save enough to ‘afford’ an internship pay cut? Possibly.

Does it mean working five fewer hours per week so that you can study up on the profession you really want to be in? Certainly.

I can’t tell you the sacrifices you are going to have to make, but I can almost assure you they will be there. That’s the thing about success–if it were easy, everyone would be successful.

A Leap of Faith

An internship is really more of a leap of faith. Before you take that leap of faith, you really need to realize what that leap of faith is.

First off, you need to recognize than an internship is NOT a paycheck, it’s an opportunity. An opportunity for you to prove your value to the potential employer.

Think of it as an extended interview–the company gets to take you for a test run and see if you are someone they want to stick around. This is the time you should be working your tail off to impress them.

After hours work? Sure. Hours studying up on mastering your profession or ways to benefit the company? It’s necessary.

After all, if you don’t go the extra mile in everything you do–why should they keep you around?

A Worthwhile Company

Notice that I haven’t spoken of an internship as a temporary endeavor. This requires a shift in strategy. It’s far too commonplace to believe an internship is temporary–your goal is to make your spot at that company permanent.

An internship or apprenticeship is really just a stepping stone. You need to make the company believe that you are so indispensable that they cannot afford to lose you when the apprenticeship is over.

A worthwhile company will recognize your value. However, even the finest work and the deepest passion cannot overcome a company or a manager without vision. That’s why choosing the right internship at the perfect company is the most important step.

The Right Internship

Now, here is where things get sticky. You MUST choose a company that will recognize your value. Do your homework on the company. Investigate their owners, their practices, how they treat their existing employees, their personal lives. Invite them to coffee. Chat with them on the phone. If you can’t talk to someone in management or HR, speak to someone lower on the totem pole.

Have they promoted people from within? Are their managers experienced leaders in their field?

If you take an internship for a company that isn’t likely to notice or even care whether you’re the next Richard Branson, don’t be surprised when they happily send you on your way when the internship is over. Try to find a company where their existing culture matches your personality. Then, and only then, will you find somewhere that you can truly flourish.

Following Your Passion

It’s so trite–I’m aware. The truth is, most of the wisdom in the world exists in sayings that are so commonplace we pay them no mind. If you aren’t passionate about the work you will be doing in an internship–don’t do it.

If you take the internship and fail to display passion for your work–you likely won’t get an opportunity to stay. It’s too hard to keep up a lie for that long–a worthwhile employer will recognize your passion and pride in your work. They’ll also recognize disdain and passionless work.

If They Don’t Keep You

The fact is, sometimes there simply won’t be a position available. Headcount, financial limitations, organization size–they can all contribute to you NOT getting a job after your endeavor is over. It is still valuable experience–and it can’t be attained from any textbook or schooling out there

“Experience is simply the name we give our mistakes”

-Oscar Wilde

No experience is bad experience–not in this context, anyways. Take an internship at a company where you hate the culture? Perfect! You’ll know what you don’t like–and you’ll know how to do your research to avoid that type of company next time.

Figure out that doing that thing you wanted to do so bad for so long isn’t all it cracked up to be? Awesome. That’s just about the best experience ever–you may have saved yourself the cost of a degree in that field, all for a few months of lousy pay–and that, my friends, is an incredible ROI.

How to Verify CoPP Policy and Drops in NX-OS

First off, why do you care about CoPP or its counters?

— P.S. I’m now writing on my new blog https://thejordanburnett.com

Cisco Nexus CoPP

On NX-OS, you may find yourself wanting to check Control Plane Policing for drops depending on the policy that you implemented (dense, lenient, strict, moderate, custom) and the performance of the Nexus device in your network.

If you have inexplicable network issues related to certain protocols (DHCP, ARP, ICMP, etc.) you may want to look into the CoPP policy to see if you are running into issues with the default thresholds.

DISCLAIMER: I do not recommend that you modify these values unless you are intimately familiar with your network and have verified that CoPP is actually an issue. CoPP is there for a reason, and just like QoS, drops do not always mean there is an actual problem. In the face of uncertainty, get TAC involved. 

To show which CoPP profile you currently have applied, run the following command:

N9K:# show copp status
Last Config Operation: None
  Last Config Operation Timestamp: None
  Last Config Operation Status: None
  Policy-map attached to the control-plane: new-copp-policy-strict

The policy-map above is the actual CoPP policy applied to your control plane.

** Notice that the one applied here is not the default policy sent from Cisco. We happened to implement this particular switch in an environment in a transitory period where there were a few more clients L2 adjacent to the core than I would prefer–until some other equipment and L3 boundaries were implemented. This required that we modify the default policy and slightly increment the thresholds to account for this rather uncommon deployment scenario.

To show the policy itself (thresholds, actions, etc.), run the following command:

 N9K# show policy-map interface control-plane
  Control Plane
Service-policy input: new-copp-policy-strict
class-map new-copp-class-critical (match-any)
  match access-group name new-copp-acl-bgp
  match access-group name new-copp-acl-rip
  match access-group name new-copp-acl-vpc
  match access-group name new-copp-acl-bgp6
  match access-group name new-copp-acl-ospf
  match access-group name new-copp-acl-rip6
  match access-group name new-copp-acl-eigrp
  match access-group name new-copp-acl-ospf6
  match access-group name new-copp-acl-eigrp6
  match access-group name new-copp-acl-auto-rp
  match access-group name new-copp-acl-mac-l2pt
  match access-group name new-copp-acl-mac-l3-isis
  set cos 7
  police cir 19000 pps , bc 128 packets
  module 1 :
  transmitted 1084573 packets;
  dropped 0 packets;

Bonus: to show the drops only, run the following command:

N9K# show policy-map interface control-plane | grep 'dropped [1-9]'
  dropped 68671 packets;
  dropped 3372 packets;
  dropped 25061 packets;
  dropped 2378 packets;
  dropped 4687 packets;
  dropped 4067 packets;
  dropped 69 packets;
  dropped 1735 packets;
  dropped 5342 packets;
  dropped 1347 packets;
  dropped 336479 packets;
  dropped 28 packets;
  dropped 4723837 packets;
  dropped 69925 packets;
  dropped 76 packets;
  dropped 30 packets;
  dropped 31781871 packets;
  dropped 3916826 packets;
  dropped 34081201 packets;

Extra Bonus: To show the difference between the last time you ran the command and the current output run the following command:

N9K# show policy-map interface control-plane | grep 'dropped [1-9]' | diff
  < dropped 31770215 packets;
  < dropped 3913822 packets;
   dropped 31792911 packets;
  > dropped 3919359 packets;
  > dropped 34083264 packets;
 N9K# show policy-map interface control-plane | grep 'dropped [1-9]' | diff
  < dropped 31792911 packets;
  < dropped 3919359 packets;
   dropped 31798086 packets;
  > dropped 3920358 packets;
  > dropped 34084100 packets;

This will show you the differences between the counters based on the last time you ran the same command. Basically, it will tell you if your counters are going up assuming you haven’t cleared them (*nix folks will be familiar with diff operation).

Heed the command usage, and run ‘diff-clean’ when you are finished–and don’t use on extremely large outputs–under the hood you are creating temp files that store the output.

Until next time.

Have something interesting to add on NX-OS control plane policing? Comment below.

Top 5 Manageability Features Introduced in Cisco ISE 1.3

Cisco ISE 1.3 is finally out, after many software patch releases for ISE 1.2 and 1.2.1. There are a ton of feature improvements and additions including guest enhancements, pxGrid, and others–but this post focuses on the manageability/serviceability side of ISE for those who actually install and administer the appliances. Note, these are not in any particular order of precedence.

1. OVAs for Installation and VM Resource Checks

ISE Finally has pre-configured OVAs for installation.

While not a “WOW” feature for those of us who actually RTFM, pre-packaged OVA installations should save us the pain of decommissioning and rebuilding improperly sized ISE nodes.

I have seen more than one ISE node implode due to improperly sized disks and multiple others complaining about the lack of VM resources which can impact authentication latency and performance, not to mention cause some weird and intermittent issues that are very difficult to troubleshoot.

Another added benefit of the ISE 1.3 upgrade software is that it performs a preliminary verification of the VM’s hardware and stops the installation without making any changes if your resources aren’t up to snuff.

2. Export Policy Configuration

Success Kid - Not having to provide Cisco tac a webex at 8PM to fix suzy in accounting? WINNING
ISE 1.3 finally allows you to export the AAA configuration to an offline XML file for review by your ITSP or Cisco TAC

This is a big feature for those of us who deploy, support, or maintain Cisco ISE. This feature allows you to export the entire authentication and authorization configuration in an XML format for offline review.

With this feature, both your Cisco solution provider and Cisco TAC can review the configuration to check for any obvious anomalies or configuration errors–all without live access to your ISE environment.

3. Test/Preview Portal Feature

ISE Portal Preview
Live portal preview and test URLs–features that have been missing from ISE for a while.

Finally! Make a change to a portal template and want to know what it looks like directly from ISE? ISE 1.3 gives you the ability to test the portals with a portal test URL and adds a WYSIWYG  portal customization page to show you a live preview of the changes you are making on a mobile or desktop device.

4. Regex and Right Click in Live Authentication View

ISE Regex Filter
Cisco finally integrates regular expressions into the filter feature of the live authentication sessions page.

Regular expressions are invaluable with larger deployments where you may have hundreds or thousands of authentications in a very short period of time in the live authentications page.

Having the ability to right-click on a specific authentication session gives you the ability to open the debug tool for the specific session–you can also modify collection filters or bypass suppression filtering.

With the advent of regular expressions in basically all versions of Cisco’s operating systems and the imminent influx of SDN, knowing how to craft regular expressions and coding in general are becoming very useful assets in the IT space.

5. AnyConnect 4.0 Unified Agent for Posture

AnyConnect 4.0 adds the NAC agent as a module.
AnyConnect 4.0 adds the NAC agent as a module.


Using posture to verify your endpoints are up to snuff? You no longer have to deploy a separate NAC agent for more advanced posture policies.

AnyConnect 4.0 integrates the NAC agent functionality as a module–just like NAM or VPN–and allows you to add it on as another supported module. Less management, less hassle.

Cisco has come a long way since ISE 1.0 and these features are but a few of the many that will be consolidated and expanded upon in ISE 2.0 (Everyone cross your fingers for TACACS+!).

Soon to come is an article on the guest improvements in 1.3 and my thoughts on pxGrid which was also introduced in ISE Version 1.3.

What’s your favorite feature of ISE 1.3? Comment below.