Job Description

Accenture is a leading global professional services company, with solutions in strategy and consulting, technology, operations and digital capabilities that drive our clients to reach their full potential.\nIn our Technology team, we accompany organizations from all industries in their digital transformation, helping them adopt emerging technologies that enable innovation, efficiency and real results.\nWe are currently searching for a Senior Big Data & Streaming Data Engineer.\nThe selected person will be responsible for designing, building and optimizing large data pipelines. scale, guaranteeing the efficient processing of information in both batch and real-time environments. You will work in an agile data-oriented cell, building robust, scalable, high-performance cloud solutions.\nRole Responsibilities:\nDesign, develop and optimize large-scale data pipelines using Apache Spark.\nImplement batch and real-time data processing and transformation processes.\nBuild streaming solutions using Apache Kafka, Spark Structured Streaming or Apache Flink.\nIntegrate data platforms with AWS services and cloud architectures scalable.\nDevelop and maintain processing flows on Data Lakes using modern formats such as Parquet, Avro and Delta Lake.\nImplement good practices for performance, monitoring and optimization of distributed jobs.\nOrchestrate data workflows using tools such as Airflow or AWS Step Functions.\nActively participate in agile ceremonies and collaborate with multidisciplinary development, DevOps, QA and business teams.\nEnsure the quality, security and reliability of data in high-demand productive environments.\nJob Requirements:\nAdvanced experience in Apache Spark, especially with PySpark and Spark SQL.\nSolid knowledge in job optimization, DataFrames, Datasets and RDDs.\nPractical experience in Apache Kafka and real-time data processing with Spark Structured Streaming or Apache Flink.\nExperience working with AWS services such as Glue, S3, EMR, Kinesis or MSK and Lambda.\nKnowledge in IAM, VPC and good security practices in cloud environments.\nExperience in orchestrating data flows with Apache Airflow or AWS Step Functions.\nManagement of Git and knowledge of CI/CD processes for data pipelines.\nPreferred knowledge of Terraform or AWS CDK.\nAdvanced level of Python and SQL.\nExperience using data formats such as Parquet, Avro, JSON and Delta Lake.\nMinimum 3 years of experience in Data Engineering.\nVerifiable experience working with Apache Spark in high-volume production environments.\nExperience in real-time data streaming projects.\nExperience in the financial sector, fintech or digital banking is desirable.\nExperience working under agile methodologies such as Scrum or SAFe.\n Important: Knowledge in Apache Spark, Streaming (Kafka/Flink) and AWS is exclusive.\nWe are looking for professionals passionate about technology, data and innovation, with an analytical mindset, flexible and eager to contribute to the continuous growth of our teams. \nAll Accenture decisions will be made without discrimination on the basis of race, religion, color, sex, age, disability, national origin, sexual orientation, gender identity or expression, or any other reason covered by local law.",