“Did you get that new ████████ installed yet?”
Those 8 words have haunted me for the last 80 days.
Last December we started looking to replace our Cisco ASA firewalls with a shiny new NGFW. After talking to a few companies we decided that ████████ looked like the best choice for our environment and price point. The vendor said they will provide free installation services through their SE’s.
After some discussions we diced to install it in “Wire Mode” underneath our current firewalls to let us get some real world experience and get used to the new management features
Getting the ████████ installed has been a nightmare. Here is a running list of what I have been through in the last 80 days:
January 29th: ████████ arrives. I am excited. I get it racked and an email sent off the the SE who is going to help with the install. Setup appointment to install firewall on the 31st.
January 30th: SE emails with the flu and needs to move the install back one week until the following Friday.
February 7th: We configure the firewall with two interfaces set to do wire mode inspection and 1 interface setup to be the WAN interface with an internal IP address to be able to update the firewalls signatures. etc. When we plug it in it kills all the traffic on the network. We unhook it and agree to try it again the following Friday. Failed Install Count: 1
February 11th: I get an email for the SE that he has a meeting in Las Vegas this week and he cant help me install the firewall on the 14th. He will get with me the following week to setup a date.
February 18th: Send the SE an email. Get non-deliverable bounce. Call sales person to learn that my SE was “let go” while in Vegas and she will find me someone else to work with.
March 3rd: Put in contact with a new SE who just inherited 1/3 of the country to try to sort through. We are going to try the install on Friday because it looks right and should work.
March 7th: We try the install basically the same way we did a month earlier with the exact same results. We take a PCAP and she says she will look at it and get back to me. Failed Install Count: 2
March 9th: I get an email suggesting it may be a routing problem and that was can make a few tweaks and get it installed on the 14th.
March 14th: Try the install after the routing changes and it fails again. I send the SE configs from the Firewalls and Switches the ████████ is in between. Failed Install Count: 3
March 17th: Have a meeting with my manager and director about why this install isn’t going well. They suggest pulling out the equipment and sending it back. I talk them into letting us keep in a keep trying because of the midstream SE change. I HATE FAILING.
March 19th: SE sends instructions on how to do a hard reset on the ████████ to start clean.
March 28th: Reconfigure the ████████ the same way it was configured previously and retried the install. It failed. I failed. ████████ failed. Failed Install Count: 4
April 1st: We regroup with a conference call with engineering who ensures us that the way we are installing it is correct and they do installs like this 40 or 50 times a week.
April 11th: SE calls in another SE from the West Coast to look over the install. He says it should work that way but we reset and reconfigure the firewall into layer 2 mode because he is more familiar with that setup and likes it better. I tell him as long as it works we can install it anyway he wants. It doesn’t work. He doesn’t know why. They start suggesting we send the ████████ back. Failed Install Count: 5
April 16th: I discuss next steps with my Boss about sending the ████████ back. I hate to fail. I felt like this was a huge loss for me since I suggested the ████████.
April 21st: We were given the day off work for the day after Easter and I didn’t have anything to speak of going on so I came in early that day and spent the next 4 hours resetting and reinstalling the firewall. After spending some time thinking about it I decide to redo the install without the “WAN” port being an internal address. Once I got the firewall configured this way the installation went well. I am still missing a few logging features I need since there isnt an interface who can talk to my SIEM but it is finally installed.
I call the SE and she is glad it is installed correctly but it still puzzled why it didnt work the way it should. We get the next generation services turned on and she promises to call me back with an explanation on why it didnt work the other way.
What I learned:
You get the installation services you pay for.
I should have taken control of the install way earlier and did it myself.
The SE’s at ████████ arent bad people they are just untrained.
I HATE FAILING.