Lead Data Engineer, Data Platform team
Job
Summary:
At Disney Streaming, data is critical to its operations and growth.
The Services & Data Engineering team is responsible for data acquisition, real-time and stream processing, analysis, and distribution of data for operational and analytical uses.
Our platform is built on public cloud and open source technologies and offers transparency into customer behavior and business performance.
The Data Platform team within Services & Data Engineering is hiring a Lead Data Engineer to build a secure data platform that powers our streaming products and can scale with the growth of our data and organization.
You'll be tasked with raising the bar by innovating and adopting new patterns and technologies to keep your teams on the bleeding edge of development.
You'll get to understand the challenges of facilitating a high-quality, flawless media streaming experience across mobile, connected devices, and web, all while using the latest technologies available from AWS and beyond.
At Disney Streaming we feel strongly that teams should own their own processes, decide their own technologies, and design solutions for the long term.
If you're interested in working in a highly collaborative team environment like this, please get in touch - we'd love to hear from you!
Responsibilities:
Work within a cross-functional team of engineers to build software for a large-scale data processing ecosystem supporting real-time and batch data that underpins all analytic and operational data flowsBuild tools and services to support data discovery, lineage, governance, and privacy compliance across the data platformBuild and maintain a CICD process for infrastructure components to ensure high availability while remaining nimbleLeadership and responsibility for technical architecture, reliability, resilience, and scalability of the Disney Streaming data platformInfluence and drive software engineering and architecture best practices and standards within the team and wider organization, along with a culture of quality, innovation, and experimentationWork with other teams:
evangelize the platform, best-practices, data driven decisions; identify new use cases and features and drive adoptionBasic
Qualifications:
6
years of software engineering, ideally Python and Scala4
years' experience engineering big-data solutions using technologies like Databricks, EMR, and Spark3-5 years' experience deploying and running cloud-based data solutions (AWS, GCP, Azure) and familiar with tools such as CloudFormation, Kinesis, ECS, S3 (or equivalent)Knowledge and experience with distributed systems, their development, operations, observability; in-depth understanding of data partitioning and sharding techniquesExperience with streaming data pipelines using tools such (Kinesis, Kafka, Spark, Storm, or Flink) and data warehouses (Redshift, Snowflake)Previous experience taking ownership of features/functionality and delivering projects end-to-end with high quality and robust architecture.
Ability to dive deep into any technical component as well as understand and drive the overall systems architecture.
You're a self-learner, independent, and have excellent problem-solving skills.
You care deeply about craftsmanship in your software, and can work backwards from the customer experience.
You have excellent written and verbal communication skills.
Preferred
Qualifications:
Experience building distributed systems with microservices and/or service-oriented architecturesFamiliarity with workflow scheduler tools such as Airflow Familiarity with metadata management, data lineage, and principles of data governanceExperience building streaming data platforms, solving problems including event definition, evolution, processing, patterns and best practices, scalability, reliability, observability, validation etc.
Required EducationBachelor's degree in Computer Science or related field or equivalent work.
Estimated Salary: $20 to $28 per hour based on qualifications.
Summary:
At Disney Streaming, data is critical to its operations and growth.
The Services & Data Engineering team is responsible for data acquisition, real-time and stream processing, analysis, and distribution of data for operational and analytical uses.
Our platform is built on public cloud and open source technologies and offers transparency into customer behavior and business performance.
The Data Platform team within Services & Data Engineering is hiring a Lead Data Engineer to build a secure data platform that powers our streaming products and can scale with the growth of our data and organization.
You'll be tasked with raising the bar by innovating and adopting new patterns and technologies to keep your teams on the bleeding edge of development.
You'll get to understand the challenges of facilitating a high-quality, flawless media streaming experience across mobile, connected devices, and web, all while using the latest technologies available from AWS and beyond.
At Disney Streaming we feel strongly that teams should own their own processes, decide their own technologies, and design solutions for the long term.
If you're interested in working in a highly collaborative team environment like this, please get in touch - we'd love to hear from you!
Responsibilities:
Work within a cross-functional team of engineers to build software for a large-scale data processing ecosystem supporting real-time and batch data that underpins all analytic and operational data flowsBuild tools and services to support data discovery, lineage, governance, and privacy compliance across the data platformBuild and maintain a CICD process for infrastructure components to ensure high availability while remaining nimbleLeadership and responsibility for technical architecture, reliability, resilience, and scalability of the Disney Streaming data platformInfluence and drive software engineering and architecture best practices and standards within the team and wider organization, along with a culture of quality, innovation, and experimentationWork with other teams:
evangelize the platform, best-practices, data driven decisions; identify new use cases and features and drive adoptionBasic
Qualifications:
6
years of software engineering, ideally Python and Scala4
years' experience engineering big-data solutions using technologies like Databricks, EMR, and Spark3-5 years' experience deploying and running cloud-based data solutions (AWS, GCP, Azure) and familiar with tools such as CloudFormation, Kinesis, ECS, S3 (or equivalent)Knowledge and experience with distributed systems, their development, operations, observability; in-depth understanding of data partitioning and sharding techniquesExperience with streaming data pipelines using tools such (Kinesis, Kafka, Spark, Storm, or Flink) and data warehouses (Redshift, Snowflake)Previous experience taking ownership of features/functionality and delivering projects end-to-end with high quality and robust architecture.
Ability to dive deep into any technical component as well as understand and drive the overall systems architecture.
You're a self-learner, independent, and have excellent problem-solving skills.
You care deeply about craftsmanship in your software, and can work backwards from the customer experience.
You have excellent written and verbal communication skills.
Preferred
Qualifications:
Experience building distributed systems with microservices and/or service-oriented architecturesFamiliarity with workflow scheduler tools such as Airflow Familiarity with metadata management, data lineage, and principles of data governanceExperience building streaming data platforms, solving problems including event definition, evolution, processing, patterns and best practices, scalability, reliability, observability, validation etc.
Required EducationBachelor's degree in Computer Science or related field or equivalent work.
Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
