JD-20587
2 - 7 Years
IT
Contract
Singapore
you will play a crucial role in designing, developing, and optimizing big data solutions using Apache Spark, Scala, and Elasticsearch,You will collaborate with cross-functional teams to build scalable and efficient data processing pipelines and search applications,Collaborate with data engineers, data scientists, and other stakeholders to understand requirements and translate them into technical specifications and solutions •t,Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch •t,Develop and maintain scalable and fault-tolerant Spark applications, adhering to industry best practices and coding standards •tTroubleshoot and resolve issues related to data processing, performance, and data quality in the Spark-Elasticsearch integration •tMonitor and analyze job performance metrics, identify bottlenecks, and propose optimizations in both Spark and Elasticsearch components •tStay updated with emerging trends and advancements in the big data technologies space to ensure continuous improvement and innovation Key Requirements •
Knowledge and experience in the Compliance / AML domain will be a plus,Experience with Spark Streaming is a plus •tProficiency in Scala programming language and familiarity with functional programming concepts •tIn-depth understanding of Apache Spark architecture, RDDs, DataFrames, and Spark SQL •tExperience integrating and working with Elasticsearch for data indexing and search applications •tSolid understanding of Elasticsearch data modeling, indexing strategies, and query optimization •tExperience with distributed computing, parallel processing, and working with large datasets •tFamiliarity with big data technologies such as Hadoop, Hive, and HDFS •,Proficient in performance tuning and optimization techniques for Spark applications and Elasticsearch queries •tStrong problem-solving and analytical skills with the ability to debug and resolve complex issues •,Familiarity with version control systems
I am Nityo Infotech’s virtual assistant. How can I help you today?