Job Code

JD-20587

JOB DESCRIPTION

Spark Scala Elastic,Network Link Analysis,Spark Scala applications,Spark

Experience Required

2 - 7 Years

Industry Type

IT

Employment Type

Contract

Location

Singapore

Roles & Responsibilities

you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch,You will collaborate with cross-functional teams to build scalable and efficient data processing pipelines and search applications,Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions •t,Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch •t,Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards •tTroubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration •tMonitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components •tStay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation Key Requirements •

Expertise & Qualification

Knowledge and experience in the Compliance / AML domain will be a plus,Experience with Spark Streaming is a plus •tProficiency in Scala programming language and familiarity with functional programming concepts •tIn-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL •tExperience integrating and working with Elasticsearch for data indexing and search applications •tSolid understanding of Elasticsearch data modeling, indexing strategies, and query optimization •tExperience with distributed computing, parallel processing, and working with large datasets •tFamiliarity with big data technologies such as Hadoop, Hive, and HDFS •,Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries •tStrong problem-solving and analytical skills with the ability to debug and resolve complex issues •,Familiarity with version control systems