ELK Observability Lead

Job Title: ELK Observability Lead

Location: Chicago, IL (Onsite) / Remote
Experience: 10+ Years

Job Summary

We are seeking an experienced ELK Observability Lead to lead the design, implementation, and optimization of enterprise-wide observability, logging, and monitoring solutions using the Elastic Stack (Elasticsearch, Logstash, Kibana) . The ideal candidate will have deep expertise in Elasticsearch architecture, cluster optimization, data ingestion, observability platforms, and modern DevOps/SRE practices. This role requires technical leadership, platform governance, and close collaboration with DevOps, SRE, and engineering teams to build scalable, secure, and high-performance observability solutions.

Required Skills

Elastic Stack (ELK)

• 10+ years of experience in enterprise observability and monitoring
• Expert-level expertise in:
  • Elasticsearch
  • Logstash
  • Kibana
  • Elastic Stack Administration
  • Elastic Security
  • Elastic Observability

Elasticsearch Architecture

• Distributed Cluster Architecture
• Cluster Design & Administration
• Index Management
• Query DSL
• Aggregations
• Sharding Strategy
• Replica Management
• Data Streams
• Index Lifecycle Management (ILM)
• Cross-Cluster Search
• High Availability

Performance & Scalability

• Elasticsearch Performance Tuning
• Cluster Scaling
• Search Optimization
• High-Volume Data Ingestion
• Storage Optimization
• Capacity Planning
• Root Cause Analysis

Data Ingestion & Pipelines

• Logstash Pipeline Development
• Beats (Filebeat, Metricbeat, Winlogbeat)
• Data Parsing & Enrichment
• Pipeline Optimization
• Log Aggregation
• Data Transformation

Observability & Monitoring

• Enterprise Observability
• Centralized Logging
• Metrics Collection
• Distributed Tracing
• Dashboard Development
• Alerting
• Incident Response
• Operational Analytics
• IT Operations Monitoring

Automation & DevOps

• Python
• Bash
• Terraform
• Ansible
• CI/CD Pipelines
• Infrastructure as Code (IaC)
• Automation Frameworks

Cloud & Infrastructure

• Linux Administration
• Cloud Platforms (AWS, Azure, GCP)
• Kubernetes
• Docker
• Hybrid Infrastructure

Governance & Security

• RBAC
• Access Control
• Data Governance
• Security Best Practices
• Compliance
• Audit Readiness

Key Responsibilities

• Lead the design, implementation, and continuous enhancement of the ELK Stack (Elasticsearch, Logstash, Kibana) for enterprise observability
• Architect and optimize Elasticsearch clusters for scalability, reliability, and high-volume data processing
• Design and govern Logstash pipelines and data ingestion frameworks for multiple enterprise data sources
• Develop intuitive Kibana dashboards, visualizations, alerts, and operational reports for engineering and business stakeholders
• Optimize Elasticsearch performance through index tuning, query optimization, ILM, data streams, and sharding strategies
• Monitor platform health, troubleshoot production issues, and implement proactive performance improvements
• Establish security, governance, access control, and compliance standards across the Elastic ecosystem
• Collaborate with DevOps, SRE, Infrastructure, and Application teams to integrate observability into CI/CD pipelines
• Drive enterprise logging, monitoring, and AIOps initiatives to improve operational visibility and incident response
• Mentor engineers, establish best practices, and serve as the technical Subject Matter Expert (SME) for observability solutions

Preferred Qualifications

• Experience with AIOps, ITOM, or enterprise observability platforms
• Knowledge of OpenTelemetry, Prometheus, Grafana, or similar monitoring solutions
• Experience with Kubernetes and containerized environments
• Hands-on experience with cloud-native observability architectures
• Elastic Certified Engineer certification is a plus
• Experience implementing enterprise logging and monitoring strategies in large-scale environments