Senior Data Engineer at PEG Africa

The Position 

The Senior Data Engineer will be responsible for overseeing junior data engineering activities and aiding in building the business’ data collection systems and processing pipelines. As the Senior Data Engineer you will be responsible for building and maintaining optimized and highly available data pipelines that facilitate deeper analysis and reporting by the Data and Analytics department.

You will help in building data processing frameworks that handle the business’s growing database. In this role you will work with other Data and Analytics teams from the business in leveraging data with reporting and scientific tools, for example, Python,Tableau, R, and Spark. As the Senior Data Engineer you are expected to strive towards continuously developing new and improved data engineering capabilities. 

Responsibilities

Management and Strategy:

• This involves overseeing activities of the junior data engineering teams, ensuring proper execution of their duties and alignment with business vision and objectives.
• Providing senior-level contribution to a team that is responsible for the design, deployment, and maintenance of the business’s data platforms.

Collaboration and Support:

• The Senior Data Engineer will work with data analysts, data warehousing engineers, and data scientists in finding and applying best practices within the Data and Analytics department as well as defining the business’s data requirements, which will ensure that the collected data is of a high quality and optimal for use across the department and the business at large.
• The Senior Data Engineer is expected to  participate in designing and providing insights and guidance on database technology and data modeling best practices.

Analytics:

• In your capacity as the Senior Data Engineer you are expected to know how to retrieve and analyze data through the use of SQL, Excel, among other data management systems.
• You will also build data loading services for the purpose of importing data from numerous disparate data sources, inclusive of APIs, logs, relational, and non-relational databases.
• Develop and manage scalable data processing platforms that are used for exploratory data analysis and real-time analytics.
• Oversee, design, and develop algorithms for real-time data processing within the business and to create the frameworks that enable quick and efficient data acquisition.

Knowledge and Opportunity:

• The Senior Data Engineer is tasked with the responsibility of contributing to the continual improvement of the business’s data platforms through his observations and well-researched knowledge.
• He keeps track of industry best practices and trends and through his acquired knowledge, takes advantage of process and system improvement opportunities. 

Required Qualifications for the Senior Data Engineer

Education:

• The Senior Data Engineer must have a bachelor’s degree (masters’ preferred) in Computer Science, Applied Mathematics, Engineering, or any other technology related field.
• An equivalent of this educational requirement in working experience is also acceptable, the experience should be at least 5 years.

Experience:

• A candidate for this position must have had at least 5 years of working experience working in a data engineering department, preferably as a Data Engineer in a fast-paced environment and complex business setting.
• The candidate must have a demonstrated experience in building and maintaining reliable and scalable ETL on big data platforms as well as experience working with varied forms of data infrastructure inclusive of relational databases such as SQL, Hadoop, Spark and column-oriented databases such as Redshift, MySQL, and Vertica.
• The candidate must also have had experience in data warehousing inclusive of dimensional modeling concepts and demonstrate proficiency in scripting languages, for example, Python, Perl, and so forth.
• A suitable candidate will also demonstrate machine learning experience and experience with big data infrastructure inclusive of MapReduce, Hive, HDFS, YARN, HBase, Oozie, etc.
• The candidate will additionally demonstrate substantial experience and a deep knowledge of data mining techniques, relational, and non-relational databases.

Communication Skills:

• Communication Skills for the Senior Data Engineer are just as important as they are for the Data Engineer, both in verbal and written form.
• The Senior Data Engineer oversees and manages junior data engineering teams and to ensure effective management, he must be capable of conveying information and instructions clearly down the line to the junior team.
• As a Senior Data Engineer you will also be required to draft reports and prepare presentations for senior Business Leadership Teams.
• These reports and presentation must be clear, concise, unambiguous, engaging and convincing, which will demand exceptional communication skills on the Senior Data Engineer’s part.

Ms Office/Software:

• The Senior Data Engineer must further have exceptional skills in SQL server reporting services, analysis services, Tableau, Salesforce, integration services, or any other data visualization tools.

Technological Savvy/Analytical Skills:

• The candidate is expected to have strong knowledge in Linux, OS tools, and file-system level troubleshooting.
• The candidate must have substantial experience working with big data infrastructure tools such as Python, SQS, and Redshift. A suitable candidate will also be proficient Scala, Spark, Spark Streaming, AWS, and EMR.

Interpersonal Skills:

• The Senior Data Engineer must have certain preferable personal attributes that will make him that much more suited for the position.
• The Senior Data Engineer will be a result-driven individual, be passionate and a self-starter, be proactive requiring minimal supervision, be highly organized, have an ability to handle multiple tasks and meet tight deadlines, be a creative and strategic thinker, work comfortably work in a collaborative setting, work comfortably with senior departmental leadership, and demonstrate an ability to remain calm during times of uncertainty and stress, inspiring the same in his team.

People Skills:

• The candidate must be a people person who is able to form strong, meaningful, and lasting connections with others, enabling smooth and continued collaborative relationships, earning him the trust of his juniors who will readily follow in his directives, and gaining the confidence of senior data science leadership.

PEG is an equal opportunity employer committed to diversity. All qualified candidates regardless of age, sex, ethnicity, race and religion are encouraged to apply.