Find Unlimited Emails.For Free.
Discover prospects, find their business emails, and build your pipeline with Unifers.
Start for FreeIuri Zambotto
Senior Data Engineer @Pier Seguradora
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Pier Seguradora
Own end-to-end data architecture, ensuring high availability, reliability, and up-to-date data across the platform.Monitor and optimize data pipelines, including multi-source ingestion (Stitch) and ETL processes between layers (Airflow + AWS Athena), improving data consistency and performance.Reduced Athena costs by ~40%($2K/month) by refactoring the first-layer ETL into incremental loads and optimizing a critical query used by the software engineering team.Contributed to the migration of the feature store’s cache layer from Redis to PostgreSQL, enhancing scalability. The solution uses FastAPI (Python) and is deployed on Kubernetes.
EDUCATION
Univesp Oficial
Bacharelado, Tecnologia da Informação
Univesp Oficial
Bacharelado, Engenharia da computação
SKILLS
ABOUT IURI ZAMBOTTO
Built and maintained ETLs (data pipelines) in python using Airflow;Created a python package that facilitates data analysts and scientists to access data;Implemented some applications in Kubernetes, such as Apache Airflow, Superset, ELK (Elasticsearch, Logstash, Kibana);Helped to develop a machine learning platform, using tools such as Kubeflow and MLFlow, in addition to helping to design the CI / CD architecture for machine learning models.I have experience working with a system designed to simplify and expedite data creation and provisioning tasks, utilizing a JSON configuration file and SQL queries, managing around 700 materialized tables categorized into datamart, sheets, photos, and consistency types, handling approximately DAGs and tasks (pods in Kubernetes) daily, leveraging Athena, Starburst (Trino), Glue, and Spark technologies, and implementing incremental writing with Iceberg for reduced reading costs.PROGRAMMING SKILLSLanguages: SQL (6 years), Python (5 years).Technologies: Git, Airflow, Linux, Jupyter, Docker, Kubernetes, MongoDB, Elasticsearch, AWS Cloud, Airflow, Pentaho Data Integration, Parquet, Delta format, Iceberg, Glue, Spark, Kubeflow, Datahub.Frameworks: Pandas, Numpy, MLFlow, Pyspark, Great Expectations.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.