Job Description

IDS Comercial is a 100% Mexican company dedicated to IT with 40 years of experience and presence in Mexico, Latin America and the US.\nWe are accredited at Level 5 of CMMI (Capability Maturity Model Integration), a quality model established by the Software Engineering Institute (SEI). Certified by ISO in the ISO/IEC 27001:2013 standard. We have the Socially Responsible Company (ESR) badge.\n\nWE ARE LOOKING FOR YOU!!\nData Engineer -Cloud Native\nModality: 100% Remote\nLocation: Mexico, CDMX\nRequirements:\n3-5 years of experience in data engineering\nBigQuery domain: CREATE TABLE/VIEW, External Tables, Scheduled Queries, optimization of queries\nExperience with Airflow and/or Concord for pipeline orchestration\nAbility to map data dependencies and coordinate with business owners\nAttention to detail in post-migration data integrity validation\nPython for scripting and automation of migration tasks\nExperience in retail or data consolidation projects (desirable)\nTechnical Stack\nCloud & Storage: GCP BigQuery, Cloud Storage, Dataflow, Data Lake (Raw/Catalog/Cmp)\nOrchestration: Apache Airflow, Concord\nSource Control: GitHub / GitHub Enterprise\nLanguages: Advanced SQL (BigQuery), Python (Pandas, PySpark, SQLAlchemy)\nVisualization: Looker, Power BI, Tableau, Dataiku (reconnection, not development)\nManagement: Jira\n\nRequired Skills\nAdvanced SQL / BigQuery\nPython (pandas, PySpark, SQLAlchemy)\nDevelopment of ETL/ELT pipelines (Concord, Airflow or equivalent)\nGoogle Cloud Platform (GCP): BigQuery, Cloud Storage, Dataflow\nGovernance and data quality tools\nVersion control (Git / GitHub)\nExperience in migration projects data (desirable)\n\nResponsibilities\nMigration and CMZ\nRecreate and validate table structures in CMZ\nMigrate historical data by table with integrity validation (volumes, frequencies, business rules)\nReconfigure scheduled queries that generate tables derived from EDW PROD / Data Lake\nIdentify and document upstream/downstream dependencies by domain\nPipelines and Ingestion\nDevelop ingestion pipelines in Airflow to Raw - Catalog - Cmp layers\nConfigure and validate loading processes in the new governed environment\nReconnect system integrations and reports (Looker, Power BI, Tableau) to new tables\nQuality and Governance\nComplete metadata of migrated tables (owner, description, lineage)\nExecute data validation and reconciliation tests against original source\nRaise JIRA tickets for requests for new datasets with justification of business",