We are seeking an innovative individual who has a proven track record of building enterprise level platform components to support product development from multiple teams and lines of business. This role is expected to drive innovation through collaboration across our data science teams and business to help push Datos to the next level. Some ideal technologies for this individual would be Spark, Scala/Python, Clickhouse, and AWS.
- Build, automate and enhance data pipelines using batch and real-time streaming technologies/products helping support Datos business initiatives.
- Must be able to contribute to self-organizing teams with minimal supervision working within the Agile / Scrum project methodology
- Build production quality ingestion pipelines with automated quality checks to help enable the business to access all of our data sets in one place
- Participate in the continuous evolution of our schema / data model as we find more data sources to pull into the platform
- Participate in a collaborative, peer review based environment fostering new ideas via cross team guilds / specialty groups
- Maintain comprehensive documentation around our processes / decision making
- Bachelors Degree in Computer science or related field
- 5+ or more years of related information technology experience
- 3 years of experience in Java, Spark, Scala, Python and Data Lake.
- Experience in developing ETL/ELT Processes
- Strong SQL and NoSQL skills to write queries
- Must possess well-developed verbal and written communication skills.