Women Impact Tech

Sr. Staff Software Engineer - Systems & Infrastructure (Pinot/Presto)

    Sunnyvale, CA

About the job

We are on a mission to connect every member of the global workforce with economic opportunity, and that starts right here with LinkedIn. LinkedIn is built to help professionals and enterprise companies achieve more. Every day, millions of people use our products to make connections, discover opportunities, and gain insights. Our global reach means we get to make a direct impact on the world’s workforce in ways no other company can.

Searching for your dream job? At LinkedIn, we strive to help our employees find passion and purpose. Join us in changing the way the world works. Explore cutting-edge technology and flex your creativity. Work and learn from the best. Push your skills higher. Tackle big problems. Innovate. Create. Write code that makes a difference in professionals’ lives.

Pinot is part of the Big Data Platform (BDP) organization at LinkedIn. The Big data platform team builds the infrastructure and platforms that powers much of the analytics and data processing at LinkedIn. The team has a wide span and we solve difficult problems in areas as diverse as storage (HDFS) , Resource management (Yarn), Compute platforms (Spark/Map-Reduce), and Analytics query processing (Pinot/Presto). We’ve built and participated in well regarded open source technology for Data management (Apache Gobblin), distributed OLAP datastore (Apache (Incubating) Pinot), Query engine(Presto) as well as internal frameworks for scheduling (Azkaban) and data abstraction (Dali).

Pinot is a realtime distributed OLAP datastore, built to deliver scalable real time analytics with low latency(10s ms) and serves millions of active users serving 1000s of queries per second. It has established itself as the de-facto analytics data store at LinkedIn powering multiple member facing features like Who viewed my Profile, LinkedIn Feed, Jobs you may be interested in, etc. and also powers multiple LinkedIn products like LinkedIn Talent Insights, LinkedIn Audience Engagement Insights, LinkedIn Sales Insight among others. We develop in the Open Source and have a very healthy OS community to work and interact with (Pinot is an Apache Incubating Project and plans to graduate to a top level project soon).

We are working on making Pinot a truly cloud based service and run it as a PaaS offering to easily build analytics products/applications with ultra low latency and support for both realtime and offline data ingestion. We are also working on adding features like Differential Privacy to our offering to protect our member’s identity and more. Pinot powers a large number of analytics use cases and products – Feed, Who viewed my Profile, LinkedIn Talent Insights, LinkedIn Sales Insight and many others. It requires infra to provide capabilities to onboard new use cases quickly and enable product engineers to build analytics and insight products with minimal friction and for scale. We are looking for a Sr. Staff SWE with strong execution and leadership capabilities to help us deliver our charter of building the next generation of low latency analytics for the cloud. As a Sr. Staff Software Engineer, you will be a key technical leader and a role model within the Analytics Infra organization. In this role, you will be responsible for working with other Big Data Platform engineers, managers, and partners to create the vision, technical roadmap and make it a reality.

Responsibilities:

  • You’ll own technical strategy for broad or complex requirements with insightful and forward-looking approaches that go beyond the direct team and solve large open-ended problems.
  • Provide technical leadership and mentorship as we design, build and deploy in the cloud.
  • Lead and execute complex projects and work with partner teams in product, data science and AI to take ideas from conception to production.
  • Build and scale data infrastructure that powers analytics in realtime.
  • Develop scalable, high-volume performing, and reliable cloud service that operates 24x7.
  • You will be responsible for overall system architecture, scalability, reliability, and performance of Pinot and it’s platform offerings.
  • Provide technical leadership and mentorship to junior engineers in the team, review their work, evangelize and drive the best modeling and engineering practices.
  • Work closely with the open source community to participate and influence the feature roadmaps to make Pinot scale and meet the needs of LinkedIn workloads.

Basic Qualifications

  • BA/BS Degree in Computer Science or related technical discipline, or equivalent practical experience.
  • 10+ years experience in software design, development, and large-scale software engineering with at least 3 years of experience in a architect or technical leadership position.
  • Experience in object-oriented and/or functional programming languages(Java/CPP/Scala/etc.) and distributed systems concept.
  • Experience in designing and implementing large scale services and distributed platforms.
  • Hands on experience developing distributed systems, databases, or other large scale data systems.

Preferred Qualifications

  • Understanding of databases (relational, key/value, document, columnar, olap, graph).
  • Experience building and running in production (24x7 environments) distributed large-scale systems.
  • Experience in working on Cloud Distributed Storage/Databases and/or data technologies that power analytics (e.g., Pinot, Druid, Redshift, Hadoop, Spark, Presto, Kafka, Flink, etc. or similar technologies).
  • Experience in Building Micro services & Cloud Platforms on AWS, Azure etc.
  • Experience with industry, open-source projects and/or academic research in data management, relational databases, and/or large-data, parallel and distributed systems.
  • Experience with highly available/fault tolerant systems, and operating complex services running in the cloud are a big plus.
  • Experience with the full lifecycle of building, deploying, and maintaining a large scale public cloud service.
  • Experience with open source project management and governance.
  • Experience leading a large organization.

Equal Opportunity Statement
LinkedIn is committed to diversity in its workforce and is proud to be an equal opportunity employer. LinkedIn considers qualified applicants without regard to race, color, religion, creed, gender, national origin, age, disability, veteran status, marital status, pregnancy, sex, gender expression or identity, sexual orientation, citizenship, or any other legally protected class. LinkedIn is an Affirmative Action and Equal Opportunity Employer as described in our equal opportunity statement here. Please reference the EOCC Document and the EOCC Supplemental Document for more information.

LinkedIn is committed to offering an inclusive and accessible experience for all job seekers, including individuals with disabilities. Our goal is to foster an inclusive and accessible workplace where everyone has the opportunity to be successful.

If you need a reasonable accommodation to search for a job opening, apply for a position, or participate in the interview process, connect with us at accommodations@linkedin.com and describe the specific accommodation requested for a disability-related limitation.

Reasonable accommodations are modifications or adjustments to the application or hiring process that would enable you to fully participate in that process. Examples of reasonable accommodations include but are not limited to:

-Documents in alternate formats or read aloud to you
-Having interviews in an accessible location
-Being accompanied by a service dog
-Having a sign language interpreter present for the interview

A request for an accommodation will be responded to within three business days. However, non-disability related requests, such as following up on an application, will not receive a response.

LinkedIn will not discharge or in any other manner discriminate against employees or applicants because they have inquired about, discussed, or disclosed their own pay or the pay of another employee or applicant. However, employees who have access to the compensation information of other employees or applicants as a part of their essential job functions cannot disclose the pay of other employees or applicants to individuals who do not otherwise have access to compensation information, unless the disclosure is (a) in response to a formal complaint or charge, (b) in furtherance of an investigation, proceeding, hearing, or action, including an investigation conducted by LinkedIn, or (c) consistent with LinkedIn’s legal duty to furnish information.

Pay Transparency Policy Statement
As a federal contractor, LinkedIn follows the Pay Transparency and non-discrimination provisions described at this link: https://lnkd.in/paytransparency.

Global Data Privacy Notice for Job Candidates
This document provides transparency around the way in which LinkedIn handles personal data of employees and job applicants: https://lnkd.in/GlobalDataPrivacyNotice

Meet LinkedIn

LinkedIn is the world’s largest professional network. Our vision is to create economic opportunity for every member of the global workforce.