Streaming Data Engineer at WB Data Management in Amsterdam

Streaming Data Engineer at WB Data Management

Wholesale Banking Data Lake is on the verge of transitioning from a traditional data warehousing landscape to a big data platform that processes millions of files containing TBs of data with all the necessary (security) controls, monitoring and automation in place that make the solution scalable and in line with banking minimum standards. Within our environment key critical processes run on a daily basis that support strategic ING bank wide initiatives which are used across the world.

To develop this release, we are looking for a Streaming Data Engineer to join our team of experts. Our ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The (Streaming) Data Engineer will active support our software developers on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects.

The candidate will be excited by the prospect of re-inventing and re-designing the data architecture, to support our next generation of our Near Real Time Data Lake and is able to derive a roadmap plan for this based on best practices and experience. You will also drive the implementation of this architecture, including all required data aspects.

Experience in the Agile methodologies would be a perfect fit, able to slice and dice work/ideas in clear stories and tasks.

Responsibilities for Streaming Data Engineer

• Create and maintain, with the team, an optimal data pipeline architecture.
• Define and implement the infrastructure required for optimal loading of data from a wide variety of data sources using big data technologies.
• Assemble large, complex data sets that meet functional / non-functional requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Work with stakeholders including the Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.
• Keep our data separated and secure across international boundaries through multiple data centers
• Work with data and analytics experts to strive for greater functionality in our data systems.

Qualifications for Streaming Data Engineer

• We are looking for a candidate with 5+ years of experience in a Data and/or Software Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.

• Experience with stream-processing systems like Kafka, Kafka Connect and/or Flink.
• Hadoop Eco System (NiFi, Groovy, Hive, HDFS)
• Object-oriented/object function scripting languages: Java, Spark, Scala, Flink and Python.

• Experience building and optimizing ‘big data’ data pipelines, architectures and data sets. In both batch and real-time data integration.
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.
• Experience with ETL processes and tools
• Experience working in a DevOps environment. CI/CD, Azure DevOps, Test automation, Docker
• Excellent communication skills, both verbally and in writing, to be able to align with various stakeholders and cross border squads
• Results oriented, determined and having a self-driven attitude
• Understanding of Agile - Scrum methodology

What do we offer?
 

Working at ING means working in a dynamic and international setting. Individual development of our employees is very important and that is why ING offers excellent courses and programs. You will become a member of the Wholesale Banking team, a prestigious, no-nonsense and high-output data-science & engineering team.

You will work on the most innovative projects within ING. In addition, we offer:
 

· Fun!

· A competitive salary

· Working with highly skilled people

· Working in an area which is of great importance to the strategy of ING

· A relaxed and fun team

· An International atmosphere

· A full-time position (36/40 hour week)

· Great training and education opportunities

Are you keen to know more or apply? 

We are looking forward to getting to know you!