Luis Gustavo Saibro

Data Engineer @Indicium AI

Florianópolis, BR

As a Data Engineer, I contribute to multiple high-impact projects, optimize big data environments. My expertise lies in designing, implementing, and maintaining robust data platforms that meet the unique needs of each project.Key Responsibilities and Expertise:Data Orchestration and Workflow Automation: Proficient in using Apache Airflow for data orchestration, deployed on cloud environments with Terraform. I have enhanced platform performance, security, and scalability through targeted updates and optimizations.Data Extraction and Integration: Experienced with tools like Embulk, Meltano, and Airbyte for seamless data extraction and integration across various environments. I have successfully managed and automated data flows, ensuring high reliability and efficiency.Infrastructure as Code (IaC): Skilled in provisioning and managing cloud infrastructure using Terraform, ensuring that the required infrastructure is available, scalable, and secure for all projects.CI/CD Pipelines and Automation: Developed and maintained CI/CD pipelines using GitHub Actions and Bitbucket Pipelines, extensively utilizing shell scripting. I have also created Docker images for various tools within these pipelines, facilitating continuous integration and delivery.Cloud Technologies: Hands-on experience with AWS services such as DMS (Database Migration Service), Kinesis, RDS Aurora, S3, and Redshift. I have architected and optimized data solutions to meet the performance and scalability needs of global clients.Data lakehouse: Projects involving Databricks and Azure Data Factory (ADF) within Azure and GCP environments. I adhere to Databricks best practices to ensure optimal performance and cost-effectiveness of data solutions.Migration & Costs: I have extensive experience in migrating and optimizing Databricks and Apache Airflow across cloud providers, improved scalability, enhanced performance, designing cost-efficient architectures