Senior Data Engineer at Andela

We are looking for a talented and passionate Senior Data Engineer with a strong technical and analytical background to build out the Partner’s core data offerings. Data is at the centre of everything our partner does - from developing innovative campaigns to elicit data, to analyzing and processing data to derive insights. As a Data Engineer, you’ll be working with some of the world’s most interesting and difficult to obtain data. As a member of the Data Engineering Team, you will get hands-on experience structuring and modelling data as well as implementing workflows. You will productize cutting-edge machine learning and data science algorithms in service of data analytics solutions requiring a combination of data pipelines, data lakes and data warehouses. You will exercise extensive data processing and advanced ML modeling for such things as harmonisation of place locations, identification of fraudulent behaviour, the tracking and forecasting of reported price data, and derivation of insights from contributor survey responses.

Primary Qualifications

• You are passionate about building and developing a world-class engineering culture
• You are humble and drive positivity
• You value and respect diversity and inclusion
• You are capable of effectively communicating in a data-driven manner

What You Get To Do

• Create innovative on-demand and subscription data products using information collected by partner’s worldwide network of contributors
• Model data and process flows for both live and offline data in a way that maps storage systems to business requirements
• Collaborate with data scientists to productize algorithmic prototypes for statistical analysis and machine learning for prediction and clustering
• Implement data pipelines for data transformation and integration (streaming and batch)
• Develop and improve the current data architecture with an emphasis on data quality, improved monitoring, and high availability
• Analyze trade-offs involving latency, throughput, and transactions for distributed systems
• Champion data governance, security, privacy, quality and retention policies

Your Background Likely Includes

• Expertise designing and maintaining databases (object, columnar, in-memory, relational)
• Proven track record of successful communication of data infrastructure, data models, and data engineering solutions
• Experience with relational data stores as well as one or more NoSQL data stores (e.g. Mongo, Cassandra)
• Prior experience in data warehouse modernization building complete data warehouse solutions, star/snowflake schema designs, infrastructure components, ETL/ELT pipelines, and reporting/analytic tools
• Experience building production-grade data backup/restore, and disaster recovery solutions
• Hands-on experience with batch and streaming data (e.g., Cloud Dataflow, Beam, Spark, Cloud Pub/Sub, Apache Kafka)
• Advanced SQL skills, and proficient in one or more programming languages such as Python
• Familiarity with python data science tooling (pandas, scipy, sklearn)
• Demonstrated proficiency with data structures, algorithms, distributed computing, storage systems
• BS or MS in Computer Science or equivalent work experience & analytical skills
• 5+ years' of overall programming/data engineering experience

Bonus Points

• Knowledge of geospatial processing with PostgreSQL and PostGIS
• Experience designing and maintaining data warehouses
• Developing for Google Cloud Platform/Google Professional Data Engineer Certified
• Experience with Python, Java, Scala
• Experience using ArcGIS or other similar GIS systems

What You’ll Get*

• Full-time compensation
• Full medical coverage
• Beautiful working environment
• Opportunity to work with the brightest minds inside and outside of your field
• A chance to change the world for the better