Women Impact Tech

Senior Cloud Systems Engineer, Data Operations

    New York, NY

Full Time

The Role

As an engineer who is a part of the Production Engineering Team, you will be integral to the design, set up, automation, and maintenance of challenges and solutions the team takes on. The ideal candidate should have effective intercommunication skills to promote collaboration with developers, support engineers, customers, and senior management. They will work closely with development squads, and internal customers, gathering requirements, architecting, and constantly delivering quality improvements to our data pipelines.

As an mParticle Senior Cloud Systems Engineer – Data Operations, you will…

  • Be responsible for the flawless operation of our Spark and Airflow pipelines, working with data engineers and operations to ensure uptime, performance and efficiency of all involved systems.
  • Use your daily interactions with the platform and your experience and skills to constantly improve our environment and ensure that issues do not reoccur
  • Maintain and augment our monitoring systems so that they alert on symptoms, instead of issues
  • Be proactive and take ownership in identifying, raising, and resolving issues or deficiencies you see anywhere in our environment
  • Produce and improve internal documentation and SOPs where they are missing or lacking quality or details
  • Write new Terraform and Ansible code and improve existing codebase to help automate and remove toil from the team
  • Live-debug pipeline issues, and identify, resolve or own resolution for functionality and performance deficiencies
  • Identify, and suggest or resolve performance issues with our data pipelines and their configuration
  • Contribute to our scale goals by identifying areas for improvement that can lead to higher efficiency
  • Help us improve the tools, automation, and monitoring of our data pipelines.
  • Automate yourself out of a job

You will be perfect for this role, if you...

  • Have an academic background in Computer Science, Engineering, Data or similar
  • Comfortably “own” Spark on EMR
  • Comfortably “own” Apache Airflow
  • Are comfortable with DevOps tools related to DataOps (Terraform, Docker, Kubernetes)
  • Have a proactive approach to spotting problems, areas for improvement, and performance bottlenecks
  • Have medium to high mastery coding small solutions in Python
  • Have an eye for edge cases, behaviors, creative solutions
  • Have an unstoppable urge to fix what is broken
  • Efficiently balance speed/iteration and quality

As a Senior Cloud Systems Engineer - Data Operations, we expect you to...

  • Fluently follow existing best practices for maintaining supported application and platform health and writing and testing code
  • Make impactful decisions about your technical contributions
  • Understand how our production systems work
  • Handle vague scope and be autonomous
  • Manage your work with little-to-no supervision
  • Actively collaborate with others through technical documentation
  • Be able to troubleshoot and contribute to resolution of moderate to complex production problems, write post-mortems on them
  • Write SOPs for issues encountered and common tasks
  • Able to automate repetitive tasks using purpose-written code or commercially available tool
  • Detect inefficient common operational patterns and processes
  • Design and implement monitoring solutions for common or critical problems
  • Be empathetic to the organization and team’s pains

As a technical resource and expert, you should be able to...

  • Handle medium complexity issues’ troubleshooting and resolution; be a core resource in troubleshooting and resolving those issues
  • Understand the question behind the question
  • Have sufficient understanding of the mParticle pipeline to be able to assist in troubleshooting medium to complex platform issues
  • Write quality, clean, and maintainable code, following company best practices with minimal guidance
  • Develop sufficient domain understanding to sanity check and ensure the quality of their output, as well as review that of other team members
  • Write custom code of medium to high complexity in at least 2 languages
  • Be the SME engineer for your areas of responsibility
  • Proactively research and keep up to date on the patterns, advancements, and evolutions of tools and technologies used in the mParticle pipeline
  • Identify problematic patterns in the mParticle applications, processes and tools and suggest and implement resolution options
  • Make small design decisions independently, making appropriate tradeoffs between simplicity and performance
  • Follow existing patterns to create new instances of projects, features, or architecture
  • Create novel architectures of small components within your area of expertise This includes diagramming the architecture and assessing trade-offs made and patterns applied, assessing the effort for the change and approximate timeline
  • Understand the flow control of nearly any system including those outside of your area of expertise, though unable to necessarily suggest improvements to systems outside of your area
  • Properly sense when to engage Security for a review of a potential change
  • Understand techniques used to troubleshoot and fix production bugs and issues
  • Develop solutions/code that reduces future operational burden (e.g. by adding appropriate self-healing, high levels of alerting/monitoring/logging, reducing alert noise, etc.)
  • Ensure that infrastructure resources are not wasted by consistently following provided best practices and rightsizing instances, proactively identify areas that can benefit from changes that lead to cost savings
  • Contribute to the build and release tooling and infrastructure
  • Contribute to defining SLAs and SLIs

You should also be able to…

  • Be successful when working on a large feature or improvement of vague scope
  • Identify and push forward new features or enhancements that improve the functioning of a system or feature
  • Identify problems and contribute well-scoped solutions to the team’s roadmap.
  • Focus your work on what is most valuable for the team
  • Make and communicate accurate time estimates for own work, potentially spanning multiple sprints
  • Manage projects that span multiple groups of stakeholders
  • Act as an effective facilitator for team meetings
  • Consistently communicate technical decisions through high-quality design docs, tech talks, and wiki contributions
  • Create documentation, train and mentor others
  • Be the role model for less experienced team members

Lastly, as part of mParticle and our Engineering organization, you should…

  • Participate, own, and improve mParticle technical recruiting, onboarding and branding
  • Act as a brand ambassador for mParticle Engineering
  • Drive the cultural direction of mParticle operations
  • Encourage people to be the best they can

Meet mParticle

At mParticle, we are passionate about building software that empowers our customers to make the most of their data. We count on our operations teams to keep our platform at peak performance and high availability, processing over 1 trillion events a month in near real-time, with no interruptions.

We are growing and expanding the value we bring to our customers, and we are currently seeking an experienced senior engineer to join our data operations team – someone who can bring experience, as well as fresh ideas, demonstrate a unique and informed viewpoint, who enjoys collaborating with a data scientists and end users to achieve the best possible value and quality for the data in our pipelines.