Software Engineer - Platform Monitoring (Remote, USA)

This is a remote position and we're considering candidates in Canada only. Residents of Quebec are not eligible for this role. 

Software Engineer - Platform Monitoring

About our Platform (at Grafana Labs):

Grafana Cloud moves millions of metrics, log lines, and traces per second from our customers' environments into a highly available, low-latency stack that processes and stores these data and serves them to dashboards and alerting tools. We aim to grow this to hundreds of millions per second, and it's critical that as we grow, we improve our performance, increase our reliability and security, and do it efficiently and effectively.

The Internal Engineering Platform delivered by the Platform department provides application engineers with the tools, systems, and Kubernetes clusters they need to build, deploy, and run their workloads. Platform roles at Grafana Labs are looking for engineers with a passion for performance and reliability who enjoy taking projects from conception to production. We organize ourselves into squads to allow focus on Cloud Infrastructure, Networking, and Security, engineering Productivity, Monitoring, and US Federal compliance.

Because we deploy production services, we have on-call rotations to ensure the system's health. Everyone at Grafana Labs tries to incorporate and use our product lineup in their day-to-day lives, so being on call is an important way to understand our system and how people use our products.

About the squad & the role:

We are hiring for the Platform Monitoring Squad, which is responsible for managing cloud service provider resources and observability (often referred to as o11y). This includes tasks such as managing costs and creating dashboards. The design for cloud observability incorporates critical elements like Prometheus, global Alertmanager, and meta-monitoring. Additionally, the squad will work on customer stack configuration management libraries and tooling, client tooling for configuration management, and the management of business-critical dashboards. For example, our Crossplane dashboards use the prometheus data sources to create a Grafana dashboards with the heatmap and timeseries panels.

Ultimately, we support teams developing important products like Grafana, Mimir, Loki, and Tempo.

Examples of projects you could be working on:

• Allocation management and utilization monitoring
• customer stack config management libraries and builders tooling
• Terraform and Crossplane Providers for Grafana Cloud

Responsibilities

• Ideally, the successful candidate's responsibilities will center around helping the team to design, compare and choose appropriate solutions for (at least some) of the following things:
  • Cloud service provider resource o11y
  • Create cost alerts
  • Help with improving cloud-cost margins
  • Improve the reliability of autoscaling tools
  • Investigate CSP unallocated resources 
  • Business-critical dashboard management

Requirements:

• Experience working in a Platform group delivering services to internal users and customers
• Experience/interest in implementing, integrating, and maintaining observability systems and processes

Nice to have:

• • Experience with Terraform
  • CLI tooling experience
  • Familiarity with Kubernetes administration
  • Worked on the correlation of application performance to an application technology stack
  • Microservices
  • Telemetry querying
  • Visualization and reporting
  • OpenTelemetry
  • Go would be ideal

In the US, the Base compensation range for this role is $123,933 - $148,719. Actual compensation may vary based on level, experience, and skillset as assessed in the interview process. Benefits include equity, bonus (if applicable) and other benefits listed here.

*Compensation ranges are country-specific. If you are applying for this role from a different location than listed above, your recruiter will discuss your specific market’s defined pay range & benefits at the beginning of the process.