Job Description

We are looking for a Senior Data Engineer with solid experience in modern data platforms, especially in the Microsoft Fabric and Databricks ecosystems. You will be responsible for designing, building and optimizing robust, scalable and efficient data architectures that support the organization's strategic decision making. We are looking for a technical, autonomous profile with a vision of end-to-end architecture.

Responsibilities:
-Design, build and maintain scalable and reliable ETL/ELT pipelines.
-Implement and manage Microsoft Fabric and components: OneLake, Lakehouse, Warehouse; DataFactory; Synapse (Spark/Notebooks/Data Warehouse); Real-Time Analytics; Power BI (Direct Lake).
-Develop and optimize processing in Databricks: Notebooks and PySpark jobs; Delta Lake/Lakehouse; Unity Catalog; Delta Live Tables and Workflows; cluster optimization (Photon, autoscaling).
-Integrate and optimize sources in Oracle DB (extraction, replication and query tuning).
-Design and implement Medallion architectures (Bronze/Silver/Gold) and dimensional models.
-Ensure quality, governance, security and lineage (Purview/Unity Catalog).
-Optimize performance and costs of data loads.
-Collaborate with analysts, data and business scientists; document architectures and good practices.
-Mentor junior engineers and promote best team practices.

Essentials:
- Verifiable experience (5+ years) in data engineering.
- Solid command of the Microsoft Fabric ecosystem and its components (OneLake, Lakehouse, Data Factory, Synapse, Power BI).
- Advanced experience with Databricks: PySpark, Spark SQL, Delta Lake, Unity Catalog, Workflows and cluster optimization.
- Advanced experience with Oracle Database: SQL, PL/SQL, tuning, partitioning and extraction processes.
- Excellent command of SQL and data modeling (relational and dimensional).
- Solid experience with Apache Spark and PySpark.
- Python programming for data processing and automation.
- Knowledge of Lakehouse / Medallion architectures and formats such as Delta Lake / Parquet.
- Experience with ETL/ELT processes and pipeline orchestration.


Desirable:
- Previous experience with Azure (Data Lake Storage, Azure SQL, Azure Synapse Analytics) and/or Azure Databricks.
- Knowledge in DataOps / CI-CD for data (Azure DevOps, Git).
- Management of Microsoft Purview and/or Unity Catalog for governance and data catalog.
- Experience with other databases (SQL Server, PostgreSQL).
- Knowledge of data streaming (Kafka, Event Hubs, Structured Streaming).
- Certifications: DP-600 (Fabric Analytics Engineer), DP-203, Databricks Certified Data Engineer Associate/Professional or equivalent.
- Familiarity with modeling tools such as dbt.

We offer:
- Participate in data transformation projects with cutting-edge technology.
- Flexible and collaborative work environment.
- Professional growth opportunities and certifications.
- Competitive salary package commensurate with experience.

",