It's been a while...

Still here

After much procrastination, I was able to get my blog setup working again. I was woefully out of date with both my Hugo and Terraform versions; no one to blame but myself for that. Oddly enough I think I have some blog topics floating around to share in the near future.

I should update my picture, not sure that even looks like me anymore.




Capturing SSH logins with Datadog events

Question Let’s say I wanted to generate a Datadog event every time I use SSH to log into a server. How could this be done? Answer I think one of the quickest ways to accomplish this would be to create a bash script in /etc/profile.d/ called dd-event that calls the Datadog api to generate an event with some identifying information. Quick & Dirty Not too complicated. It logs some pertinent data in the event details. [Read More]

Maximizing Datadog Events

Introduction As a Datadog customer in both my professional and personal life, I think about the service a lot. In today’s world, monitoring and gathering feedback on the performance or health of your software is required. Making design or operational decisions based on metrics makes collaboration easier and more fun. One of my favorite features that Datadog offers is the is the ability to create custom events. Of course Datadog shines with its visualizations, agent and integrations (AWS, Docker, ElasticsSearch, etc), but I have found that the having the ability to generate custom events to be just as valuable. [Read More]

Managing PagerDuty with Terraform

Why didn't I do this sooner?

Introduction Long ago I wrote about blog post describing the usefulness of PagerDuty and the service it provides. Having used this service for over four years on multiple teams, it is still on my list of must have tools. In the past I’ve always configured PagerDuty manually; inviting users, hand crafting escalation policies and schedules. It wasn’t overly laborious but it was still a manual process. I recently switched to a new team that uses both PagerDuty and Terraform. [Read More]

Multi-region Logging with ELK and SQS

This is good voodoo.

Intro Our team has been using the ELK stack since 2014 for managing our system and application logs. Right out of the gate it proved itself to be a transformational tool for troubleshooting issues and providing insight to our team about how our applications and system behave. As we run our applications in the cloud and prefer multi-region deployments, shipping logs around the globe securely and in rapid fashion are top requirements. [Read More]

Technology Over Time

Always be learning.

The other day I was daydreaming and realized that I started my first real job almost 16 years ago. This in turn made me start thinking about all of the different types of software that I have had to use over the different phases of my career. So I made a list. Some of this software is straight up amazing when it comes to functionality and performance. Not surprisingly, a few of these applications were nightmares to use and support. [Read More]

Change of Scenery

It's been a while...

It’s been quite some time since I updated my blog. I figured a change of scenery might be a good motivator for new content and ideas so I decided to move from Medium to a s3 bucket fronted by Cloudflare. Times sure have changed, long ago I can remember writing my own CMS using php. Ew. I could have just created the s3 bucket via the AWS CLI or the AWS console but I wanted to do it via Terraform because it never hurts to try new things. [Read More]

On-call. Don't be scared.

As an engineer for a software development team who participates in an on-call rotation, I am constantly reminded of the following section from “A Tale of Two Cities.” “It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the epoch of belief, it was the epoch of incredulity, it was the season of Light, it was the season of Darkness, it was the spring of hope, it was the winter of despair, we had everything before us, we had nothing before us…” I have had the honor of working with many fine engineers and system administrators over the years, and one universal truth stands out; being on-call can be a real downer. [Read More]

Getting Started with Logstash

Logstash is a powerful open source tool for managing logs and events. The world of log management is highly contested space and Logstash competes against the likes of Splunk, PaperTrail, Loggly and slew of others as a solution for getting more information out of the logs in our lives. Logstash is one component of the ELK Stack - ElasticSearch, Logstash and Kibana. It allows you to: Define multiple inputs for your logs (Files, Redis, Collectd, Twitter) Create filters to help parse or modify your logs Specify data to multiple outputs like ElasticSearch, Statsd or Graphite Having spent some time implementing a new Logstash cluster for my team at work, I wanted to document how to go a few steps beyond the bare essentials and show how to do some testing in a local environment. [Read More]

PagerDuty Incidents Part Two

In my previous post I discussed how to use the PagerDuty Developer API to get data about a specific incident. In this post I will expand on that concept and show some examples about getting a collection of incidents based on a date range and how to handle that with a paginated request. Incidents via Curl The GET Incidents endpoint supports a variety of query parameters and in this example we’ll look at getting all of the resolved incidents for a specific month. [Read More]