On The Other Side Of The Rainbow

On-call agent, part 2. The first one was the story; this one goes under the hood, starting with the least glamorous piece, which is also the one that mattered most. Here's a thing nobody tells you about building something with "AI" in it: the part that does the most work often has no AI in it at all.  I spent a while, before any model, before any database, on a function so dumb I hesitated to commit it. It takes an alert and throws most of it away. And it turned out to be the load-bearing wall of the whole project. The problem: a hundred alerts, one problem Our alerts are machine-generated, which means they are precise, and precision is the enemy here. The same underlying failure shows up worded almost identically across every cluster it happens in, differing only in the parts that don't matter. Here's one real alert as it fires in three different environments: ddi-prd-nva1 ipsec Pod not ready alert - ddi-prd-nva1: ipsec Pods are not ready for more than allowed ...

I'm writing this at the tail end of a rotation, which is probably the only honest time to write about one. There's a particular kind of tired that comes from being paged at odd hours: not the clean exhaustion of a long day, but something stranger, because the hours are never quite yours. Our on-call is twelve hours at a stretch, 8 to 8, seven days in a row when it's your turn. And the team lives on opposite sides of the planet, one half in PST, the other in IST. So the pager is really a baton, and by the time it reaches you it's always some odd hour, and it's always still warm from someone else's hands. That handoff is where things go to die. Someone in IST spends an hour chasing an alert: checks a dashboard, tries a restart, rules a thing out, finds the real cause just as their day ends. Then PST wakes up to the same alert firing again, and almost none of that hour survives the crossing. Maybe a one-line note. Maybe nothing. So PST starts over: same dashboard, ...

Setting the Stage: The Cisco C881-K9 Router I wanted to document my experience configuring a Cisco C881-K9 router to set up an IPSec VPN tunnel—not just for myself but also for anyone who might be venturing down this path. Whether you're tackling this for the first time or just need a refresher, I hope this helps. 1. Product Code: C881-K9 2. PoE: 2 port integrated PoE 3. RAM: 256 (default) / 768 MB (Max) 4. Flash Memory: 128 MB 5. Dimensions: 32.5 cm x 24.9 cm x 4.4 cm 6. LAN Interface: 4 x 10Base-T/100Base-TX - RJ-45 7. Management Interface: 1 x console - RJ-45 8. WAN Interface: 1 x 10Base-T/100Base-TX - RJ-45 9. USB: 1 x 4 PIN USB Type A 1. Primary WAN port—FE 2. USB port 3. 4-port 10/100 Ethernet switch 4. Serial port—Console or auxiliary 5. On/Off switch 6. Reset button 7. Power connector 8. Earth ground connection 9. Kensington security slot Why the Cisco C881-K9? I’ll admit, I didn’t plan to splurge on high-end networking gear for my tests. So, when I found a used Cisco C881-...

A few weeks back, while working on Twinstrata project (as a shadow resource), I had some spare time and I wondered if I could provide an SMS notification (in addition to email notification) whenever Nagios detected that the services on the production cluster are down. Hence, I started searching different communication libraries and found Gammu, an open source mobile phone communication library. Below are the steps I took to install Gammu on Linux (CentOS 7) and generate SMS notifications to the development team through it.  First, I had to find a spare mobile phone that could be used for this purpose and a SIM card. I had one Samsung GT-S3653W mobile phone which I wasn't using anymore ever since I bought the smartphone. One of the reasons why I chose Gammu instead of any other library (e.g Gnokki) was their database that indicated that GT-S3653 ( http://wammu.eu/phones/samsung/5152/ ) was compatible with this library. For installation, I downloaded Gammu (version 1.34.0) t...

Update (13th April, 2014) 11:00 AM : I saw my final results today. The large input file of problem 2 and 3 were rejected, while the large input file of problem 4 was accepted. I lost 35 points because of that, and my final score is 55 :( . I do qualify for the next round though. My overall rank dropped to 7920/25462, while Pakistan rank dropped to 42/265. Below is the screenshot now. Hoping to do well in the next round as well which is on 26 April 2014. 12 April, 2014... 6:00 AM: Just woke up and received an email from from Google Code Jam that the contest has been started. Will start competing after breakfast..      6:42 AM: Working on the first problem "Magic Trick". It has only one small input file and is worth 6 points. According to the contest analysis, 88% people who attempted this question got it right.  7:31 AM: Just submitted solution file and source code of the "Magic Trick" problem. The response of the online judge is "Correct...