Job Description

Job Description
We are seeking a Senior Data Engineer to design, develop and maintain the data infrastructure that powers the organization's artificial intelligence, machine learning and advanced analytics initiatives. The selected person will work alongside data scientists, machine learning engineers and business teams to build scalable, reliable and production-ready data solutions.
Responsibilities:
Design, develop and maintain scalable data pipelines for analytics and machine learning projects.
Collaborate with data scientists, machine learning engineers and business areas to guarantee the availability and quality of data.
Develop and maintain APIs for data access and integration with internal systems and external services.
Manage and optimize storage solutions for structured and unstructured data, including relational databases, NoSQL, search engines, and object storage.
Implement data monitoring and quality control processes to ensure its accuracy and availability.
Provide technical support and architectural advice on projects related to data and artificial intelligence.
Manage metadata and collaborate on data governance initiatives using specialized tools.
Ensure compliance with security, privacy and data management policies.
Maintain updated technical documentation on architectures, data models and processes.
Participate in the development of real-time data processing solutions and streaming architectures when necessary.
Requirements:
Degree in Computer Science, Engineering, Mathematics or a related discipline, or equivalent professional experience.
8 or more years of data engineering experience.
Proficient in Python, Java and Scala.
Experience managing large volumes of data in terabyte or petabyte environments.
Experience with Big Data technologies, such as Hadoop, Apache Spark, and Apache Flink.
Knowledge of machine learning frameworks such as TensorFlow, PyTorch or similar.
Knowledge of data warehousing, ETL processes and data modeling.
Experience with cloud platforms, especially AWS (S3, Redshift, Glue, Lambda), knowledge of Azure, Google Cloud Platform and VMware will be an advantage.
Experience developing RESTful or GraphQL APIs.
Knowledge of DevOps, CI/CD, Docker and Kubernetes practices.
Familiarity with business intelligence tools such as Metabase and data catalog platforms such as OpenMetadata.
Experience with messaging and streaming platforms such as Kafka or RabbitMQ.

Salary to receive
To agree