Job Code

JD-20227

JOB DESCRIPTION

● 3+ years of experience in data engineering experience, including Kafka on premise. ● Strong knowledge of Kafka, database and other big data technologies ● Strong knowledge of SQL (tuning) and database fundamentals ● Good understanding of distributed systems ● Good communication and interpersonal skills Good to have: ● Experience in Data Warehousing and Big Data. ● Experience in Operational Data Store and Change data capture. ● Expert or Strong knowledge of Debezium

Experience Required

3 - 10 Years

Industry Type

IT

Employment Type

Permanent

Location

Malaysia

Roles & Responsibilities

Responsibilities: ● Design(efficient, adaptable, and scalable), develop, and manage Kafka-based data pipelines ● Develop an expert-level understanding of data migration and CDC as it relates to Kafka using Kafka Connect and Debezium. ● Understand and apply event-driven architecture patterns and Kafka best practices. Enable development teams to do the same. ● Assist developers in choosing correct patterns, event modeling, and ensuring data integrity. ● Continuous learning to be a Confluent/Kafka subject matter expert. ● Work with Kafka and Confluent API's (e.g. metadata, metrics, admin) to provide pro-active insights and automation. ● Prepare documentations – data mapping and data dictionary. ● Design monitoring solutions and baseline statistics reporting to support the implementation ● Level 2 & 3 support BAU and optimize existing Kafka setup

Expertise & Qualification

● Bachelor Degree in Computer Science, related field or equivalent work experience ● 3+ years of experience in data engineering experience, including Kafka on premise. ● Strong knowledge of Kafka, database and other big data technologies ● Strong knowledge of SQL (tuning) and database fundamentals ● Good understanding of distributed systems ● Good communication and interpersonal skills Good to have: ● Experience in Data Warehousing and Big Data. ● Experience in Operational Data Store and Change data capture. ● Expert or Strong knowledge of Debezium