Skip to Main Content

Software Engineer II - Distributed Big Data Platform

New York, New York



At Xandr, our data is our lifeline – we can only be as good as our data, and great data is critical for business success. Our data pipeline processes the tens of billions of log records per day (more than 400TB of data) that are generated by our real-time bidding engines and ad serving platform. Data volume has rocketed over the past year and continues to increase – we have petabytes of data on store at this very moment.

The Data Platform Team develops in-house Big Data processing frameworks built around rapidly evolving product demands. Our platform powers over 1,500 ETL jobs, including batch aggregations and joins, streaming pipelines, and event-level data export to hundreds of clients. These jobs serve client-facing reports with millions of requests per day, billing and invoicing, optimization engines, and data scientists.

We are looking for a team-member to co-own the platform with us and drive investments as we further scale the platforms capabilities to support critical business activities.

About the job:

• Write maintainable platform code that helps engineers efficiently build data pipelines

• Own a client-facing system for exporting event-level data to the cloud

• Work with users to perform research and drive adoption of new big data technologies

• Containerize our applications via Docker and Kubernetes

• Develop tooling for data governance

• Work with product managers to bring high-impact features to market

• Develop monitoring/alerting systems and maintain stability and optimal performance

• Help build a new Data Lake in Azure

• Expand and optimize our current Hadoop data platform and data pipeline architecture


About your skills and experience:

• B.S. degree in Computer Science or related field

• 3+ years of software development experience

• Experience using Java or Scala

• Experience building high-performance distributed applications, preferably in a Linux environment

• Passionate about technology – you keep up to date with the latest technologies

• Excellent Computer Science fundamentals with regards to data structures, algorithms, time complexity, etc.

• Also preferred:

o Experience with Hadoop ecosystem

o Experience with Azure big data frameworks/tools: Azure Data Lake, Azure Data Factory, HDInsight

o Experience with other cloud-based big data architectures

More about you:

• You are focused on building high-quality, intuitive products

• You are passionate about learning and teaching

• You love challenging yourself to constantly improve and sharing your knowledge to empower others

• You view processes as a means and not an end, preferring lean or automated approaches to ensure quality and productivity

• You are not satisfied with the status quo and are always looking to improve how things are done and what is built

• You like to take risks when looking for novel solutions to complex problems. If faced with roadblocks, you continue to reach for solutions to problems

• You care about solving big, systemic problems. You look beyond the surface to understand root causes so that you can build long-term solutions for the whole ecosystem

• You believe in not only serving customers but also in empowering them by providing knowledge and tools

Job ID 1929917 Date posted 06/11/2019
Career Areas


#XandrLife means we’re creating an incredible experience for our people, too. Let our employees show you what it’s really like to work here.

See what it's like here
Back to top