Know it’s Broke Before Your Customers Do. David Thomas [email_address]
Acknowledgements <ul><li>Nagios World Conference </li></ul><ul><li>Ethan Galstad </li></ul><ul><li>And of Course all of y...
A little about me, and what I do. David Thomas Director of Information Technology and Emergency Communications Sandoval Co...
Sandoval County <ul><li>Sandoval County a pretty unique place. </li></ul>2011 <ul><li>The county has a total area of 3,714...
Sandoval County <ul><li>A Unique place with Unique Issues: </li></ul><ul><li>Day-to-day operations </li></ul><ul><li>Over ...
Sandoval County Customers <ul><li>So this presentation is called </li></ul><ul><li>So Who we talking about? </li></ul><ul>...
Sandoval County Network 2011 Pretty typical midsize network
Here is where it get’s interesting! 2011 Talk about terrain we start on a hill shoot across the river valley back over th...
So Much for the background. <ul><li>Now for something completely different. </li></ul><ul><li>It’s still about customers! ...
Real emergencies in need of real solutions <ul><li>In 2011 Sandoval County had to activate it’s Emergency Operations Cente...
Regional wide natural gas outage <ul><li>What happened: </li></ul>2011 During a record breaking cold spell demand was such...
Los Conches Fire <ul><li>What happened: </li></ul>2011 Down power line turns into largest wildfire in New Mexico history. ...
Smoke plumes from my backyard. 2011 This image showing the Las Conchas wildfire raging in the Jemez Mountains of the Santa...
The Role Nagios Played and it’s a big role <ul><li>In both situations when the County EOC had to be activated Nagios turne...
Nagios played a vital roll: 2011 Know it’s Broke Before Your Customers Do. Or If it’s frozen or burning down Use advanced ...
of 15

Nagios Conference 2011 - David Thomas - Know Its Broke Before Your Customers Do

Dave Th presentation on Nagios in the datacenter. The presentation was given during the Nagios World Conference North America held Sept 27-29th, 2011 in Saint Paul, MN. For more information on the conference (including photos and videos), visit: http://go.nagios.com/nwcna
Published on: Mar 3, 2016
Published in: Technology      Travel      
Source: www.slideshare.net


Transcripts - Nagios Conference 2011 - David Thomas - Know Its Broke Before Your Customers Do

  • 1. Know it’s Broke Before Your Customers Do. David Thomas [email_address]
  • 2. Acknowledgements <ul><li>Nagios World Conference </li></ul><ul><li>Ethan Galstad </li></ul><ul><li>And of Course all of you! </li></ul>2011
  • 3. A little about me, and what I do. David Thomas Director of Information Technology and Emergency Communications Sandoval County, State of New Mexico Over 20 years in the IT field, certified ITIL, Six Sigma, HDI and a bunch of other stuff.
  • 4. Sandoval County <ul><li>Sandoval County a pretty unique place. </li></ul>2011 <ul><li>The county has a total area of 3,714 square miles. </li></ul><ul><li>5 square miles of water. </li></ul><ul><li>Population of 131, 561 </li></ul><ul><li>12 Indian reservations (pueblos), The second most of any county in the United States. </li></ul><ul><li>Bandelier National Monument </li></ul><ul><li>Cibola National Forest </li></ul><ul><li>Tent Rocks National Monument </li></ul><ul><li>Santa Fe National Forest </li></ul><ul><li>Valles Caldera National Preserve </li></ul>5 National Protected Areas The county is also flanked by two national laboratories <ul><li>Sandia National laboratory to the south </li></ul><ul><li>Los Alamos National laboratory to the north </li></ul>Numerous nuclear waste and weapons transportation routes run through the Sandoval County as well.
  • 5. Sandoval County <ul><li>A Unique place with Unique Issues: </li></ul><ul><li>Day-to-day operations </li></ul><ul><li>Over 70 Buildings throughout the 3,714 square miles of Sandoval County </li></ul><ul><li>County Sherriff facilities including 60 sheriff vehicles (PC and radio functions) </li></ul><ul><li>County Fire including 24 Fire Stations 80 vehicles (PC and radio functions) </li></ul><ul><li>County Detention Center 500+ inmate facility </li></ul><ul><li>County dispatch center </li></ul><ul><li>And all the other functions of county government; Treasure, Assessor, public works, ext. </li></ul><ul><li>County Heath Clinic including dental, dietary, indigent populations operation </li></ul><ul><li>Tribal governments of the 12 pueblos </li></ul>2011
  • 6. Sandoval County Customers <ul><li>So this presentation is called </li></ul><ul><li>So Who we talking about? </li></ul><ul><li>EVERYONE! </li></ul><ul><li>Population of 131, 561 our constituents </li></ul><ul><li>All County Government </li></ul><ul><li>All Municipality Governments </li></ul><ul><li>All 12 Indian Pueblos and their governments </li></ul>2011 Know it’s Broke Before Your Customers Do.
  • 7. Sandoval County Network 2011 Pretty typical midsize network
  • 8. Here is where it get’s interesting! 2011 Talk about terrain we start on a hill shoot across the river valley back over the river valley up to a mountain peak to get to Cuba, NM 75 miles from Bernalillo, NM where we started. Not much of a chose if we are to get high speed networks to the far reaches of the county.
  • 9. So Much for the background. <ul><li>Now for something completely different. </li></ul><ul><li>It’s still about customers! </li></ul><ul><li>And in this case when it broke it broke bad! </li></ul>2011
  • 10. Real emergencies in need of real solutions <ul><li>In 2011 Sandoval County had to activate it’s Emergency Operations Center Twice. </li></ul><ul><li>February 2011 – Regional wide natural gas outage during a record breaking -40 degree temperature outbreak </li></ul><ul><li>June & July 2011 – Los Conches Fire - Largest wildfire fire in New Mexico History </li></ul>2011 Nagios played a pivotal role in the successful emergency operations of Sandoval County during these events!
  • 11. Regional wide natural gas outage <ul><li>What happened: </li></ul>2011 During a record breaking cold spell demand was such that gas companies made the decision to cut gas to less populated areas so urban areas would not be affected. The result was a sudden lost of gas for many areas across the desert southwest. What did Sandoval County Information Technology and Emergency Communications do? <ul><li>Activation of County Emergency Operations Center and information center </li></ul><ul><li>Code Red emergency notification to citizens activated </li></ul><ul><li>24 hour information line set up manned by IT </li></ul><ul><li>Ham radio coordination with county shelters and red cross </li></ul><ul><li>Updated information on County web sites 24/7 </li></ul><ul><li>FEMA and NIMS forms compliance </li></ul><ul><li>Operations lasted 120 hours </li></ul>
  • 12. Los Conches Fire <ul><li>What happened: </li></ul>2011 Down power line turns into largest wildfire in New Mexico history. What did Sandoval County Information Technology and Emergency Communications do? <ul><li>Activation of County Emergency Operations Center and information center </li></ul><ul><li>Code Red emergency notification to citizens activated </li></ul><ul><li>Operations lasted 336 hours </li></ul><ul><li>Mobile emergency radio communications and command unit activated IT manned </li></ul><ul><li>COM/L activation on site communication coordination </li></ul><ul><li>Updated information on County web sites 24/7 </li></ul><ul><li>FEMA and NIMS forms compliance </li></ul><ul><li>Massive evacuations including the city of Los Alamos and LANL </li></ul><ul><li>Ham radio coordination with county shelters and red cross </li></ul><ul><li>Monsoon rains now major flooding. </li></ul>
  • 13. Smoke plumes from my backyard. 2011 This image showing the Las Conchas wildfire raging in the Jemez Mountains of the Santa Fe National Forest in north-central New Mexico the was taken by a crew member aboard the ISS on June 27 Residences: 63 destroyed Outbuildings: 49 destroyed; 2 damaged Injuries to Date: 15 Size: 156,593 acres
  • 14. The Role Nagios Played and it’s a big role <ul><li>In both situations when the County EOC had to be activated Nagios turned into a very useful tool. </li></ul>2011 <ul><li>Nagios was constantly being monitored by EOC staff </li></ul><ul><li>We have remote devices in remote locations that where in the fire </li></ul><ul><li>All fire vehicles and sheriffs Vehicles could be monitored by VPN’ed laptops </li></ul><ul><li>Thanks to IP based radios and repeaters those were monitored too </li></ul><ul><li>Generator power was monitored on remote microwave shelters </li></ul><ul><li>Mobile command truck could monitor and we could monitor it from the EOC </li></ul><ul><li>Don’t forget standard uses for Nagios even more important in an emergency </li></ul><ul><li>All our front line people have access to view including dispatch center </li></ul><ul><li>All of this has to be documented for FEMA, and best of all it can be used as your change management database for ITIL functions. Nagois has all that built in. </li></ul><ul><li>We use Cacti too! </li></ul>
  • 15. Nagios played a vital roll: 2011 Know it’s Broke Before Your Customers Do. Or If it’s frozen or burning down Use advanced monitoring tools to help Save lives and property. David Thomas [email_address] Nagios played a vital roll and will be a tool we use time and again in emergency situations and the day-to-day operations of Sandoval County, NM

Related Documents