Empresa:

Explore Group

Descrição da Função

Data Engineer GCP Remote in Europe

Role Overview

As a Data Engineer, you will be responsible for building and maintaining scalable Data Platforms. This role focuses on designing, developing, and maintaining the infrastructure required for data storage, processing, orchestration, and analysis. You will implement performant data pipelines and integration solutions to ensure efficient data flow and high quality across the organisation.

Key Responsibilities

• Platform Engineering: Design and build scalable data platforms to support large-scale analytics.
• Pipeline Development: Develop and implement complex data processing workflows using frameworks such as Spark and Apache Beam.
• Data Integration: Extract data from diverse sources, including databases, file systems, and APIs.
• Security and Privacy: Ensure data integrity and security through robust access controls and encryption.
• Optimisation: Monitor system performance and optimise architectures for high availability and scalability.
• Cross-functional Collaboration: Work with data scientists and analysts to enable effective data access and insights.

Technical Requirements

• Cloud Expertise: Proven experience with Google Cloud Platform (GCP) services for data engineering.
• Programming: Proficiency in Python, Java, or Scala.
• Big Data Ecosystem: Hands-on experience with Spark, Flink, Kafka, Elastic Search, Hadoop, and Hive.
• Database Management: Strong SQL skills with experience in both relational and NoSQL databases.
• Data Modeling: Solid understanding of data modelling and database design principles.
• ETL Tools: Familiarity with data integration tools such as Talend or Kafka Connect.
• Version Control: Experience using Git for collaborative development.

Observações

Not Specified (Portugal)