Infrastructure Data Engineer

TEAM OVERVIEW  

The ADAPT (AI, Data, and Platform Technologies) Engineering team is integral to KKR's technological strategy, architecting and supporting the firm's foundational data and AI capabilities. This team is recognized as a key enabler for global scale and business transformation, driving excellence by evolving technology into robust, platform-based solutions that enhance agility and deliver material business impact.  

POSITION SUMMARY  

KKR is seeking a senior Engineer to join the core ADAPT Engineering team in Boston. This is a pivotal, hands-on technical leadership role requiring deep technical expertise in modern data engineering and a proven ability to derive critical insights from complex, large-scale financial data. The successful candidate will be instrumental in designing and constructing world-class data engineering capabilities that efficiently process massive data pipelines, leverage state-of-the-art AI-powered insights and document extraction, and integrate seamlessly across diverse cloud-powered databases.  

This role requires defining the technical blueprint for how KKR structures, stores, and leverages data to power its AI and investment platforms, ensuring data integrity, performance, and accessibility for critical firm-wide services.  This is an onsite role, with expectations to be in our Boston offices 4 days per week.

KEY RESPONSIBILITIES  

• Lead the architecture, buildout, and modernization of the firm’s unified data fabric, establishing scalable patterns for data access, interoperability, governance, and productization across business and technology teams. 

• Own the design and evolution of Iceberg-based data platform capabilities, including data ingestion, egress, data replication, data streaming, performance optimization, data lifecycle management, and adoption standards for analytical and operational use cases.

• Define and implement the platform’s compute architecture across stateful, stateless, and distributed processing layers, balancing performance, resiliency, scalability, and cost efficiency.

• Design, implement and drive the adoption of event-driven architecture patterns for real-time ingestion, data movement, and system integration, ensuring low-latency, reliable, and observable data flows.

• Extend core platform capabilities to support the firm’s AI/ML ecosystem, including curated datasets, feature-ready pipelines, training and inference data services, and scalable integration points for model development and deployment.

• Translate business and engineering priorities into a clear technical roadmap, making sound architecture decisions and sequencing platform investments to maximize long-term value.

• Serve as the senior engineering lead across one or more strategic platform domains, partnering closely with application engineering, enterprise architecture, data consumers, and machine learning stakeholders.

• Establish and enforce engineering standards for data quality, observability, lineage, governance, security, and operational excellence across batch and streaming environments.

• Mentor junior engineers, raising the bar on engineering excellence including design rigor, implementation quality, and operational ownership across the team.

• Evaluate emerging technologies and guide proof-of-concept efforts, with accountability for recommending production-ready solutions aligned to the firm’s target-state architecture.  

QUALIFICATIONS  

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related technical discipline; advanced degree preferred. 

• 4+ years of experience in data engineering, distributed systems, or platform engineering, with meaningful experience operating at a senior technical leadership level.

• Platform mindset with proven track record designing and delivering enterprise-scale data platforms or lakehouse ecosystems with strong emphasis on scalability, reliability, governance, and developer enablement.

• Deep hands-on experience with Apache Iceberg and modern open table formats, including data modeling, partitioning, performance tuning, metadata management, and operational best practices.

• Strong understanding of distributed compute architectures, including stateful and stateless processing models, workload orchestration, fault tolerance, and performance optimization.

• Demonstrated experience implementing event-driven and streaming architectures using modern messaging and data movement patterns.

• Experience enabling or extending AI/ML platform capabilities, including pipeline design, feature/data preparation workflows, and integration with model development or production ML systems.

• Strong proficiency in Python and SQL; experience with Java or Scala and modern data processing frameworks is highly desirable.

• Experience working in cloud-native and containerized environments, with familiarity in orchestration, infrastructure automation, and platform observability tooling.

• Ability to operate as a senior technical decision-maker: influencing architecture, driving execution through others, and partnering effectively across engineering, product, and business stakeholders. 

• Experience in AI Harness engineering is highly desirable.

• Strong communication skills, with the ability to articulate complex technical decisions to both technical and non-technical audiences.