Job Description

Are you passionate about data engineering, Big Data and Cloud architectures? We are looking for a Data Engineer to join a 100% remote technological project, participating in the design, construction and improvement of data solutions on AWS and Azure environments.

What will be your challenge?

You will participate in the design and development of Big Data solutions, construction of ETL/ELT pipelines and evolution of Data Lake/Lakehouse architectures, ensuring the availability, quality, security and governance of data.

Main responsibilities
Design, develop and maintain ETL/ELT pipelines.
Process large volumes of data using Apache Spark / PySpark.
Design and maintain Data Lake / Lakehouse architectures.
Perform data modeling and transformation.
Participate in Government, Quality, Catalog and Data Lineage initiatives.
Analyze and propose improvements to processes, systems and technologies.
Perform testing, troubleshooting and application of correctives.
Participate in DataOps/DevOps initiatives.
Work with AWS and Azure Cloud services.
Requirements
Professional degree in Civil Engineering, Execution Engineering or related field.
Minimum 4 years of professional experience.
Experience as a Data Engineer / Big Data Engineer.
Proficient in Python and advanced SQL.
Experience with Apache Spark / PySpark.
Experience in building ETL/ELT pipelines.
Experience with Data Lake / Lakehouse architectures.
Experience working with AWS and/or Azure.
Technologies

AWS: S3, Glue, Athena, Redshift, Lambda, Step Functions, Kinesis/MSK, CloudWatch, IAM.

Azure: ADLS, Data Factory, Synapse Analytics, Databricks, Azure Functions, Event Hub, Azure Monitor.

Data Engineering: Spark, PySpark, Kafka, Kinesis, Event Hub, Parquet, JSON, Avro.

DataOps / DevOps: Git, CI/CD, Azure DevOps, GitHub Actions, Terraform, Docker and Kubernetes.

Data Governance: Data Catalog, Data Lineage, Data Quality, Glue Catalog and Microsoft Purview.

Opportunity details

Position: Data Engineer
Specialty: Big Data & Cloud
Mode: 100% Remote
Opportunity type: Project
Experience: 4+ years

If you have experience in Data Engineering, Big Data, Spark and Cloud, we want to meet you!",