Madhavi G

WORK HISTORY

EDUCATION

ABOUT MADHAVI G

I am an experienced data engineer with a passion for leveraging cloud and big data technologies to drive impactful solutions. With over 5 years of expertise in the field, I have a strong background in working with both Azure and AWS platforms, as well as on-premises big data technologies. My technical skills encompass a wide range of tools and languages, including Azure ADF, Databricks, Airflow, AWS, PySpark, SQL, and more.Throughout my career, I have been involved in the entire software development life cycle, from requirement gathering and effort estimation to design, development, quality assurance, and delivery. This end-to-end experience has given me a comprehensive understanding of how data engineering fits into the larger picture of delivering successful projects.I excel in creating robust data pipelines, leveraging my expertise in Azure ADF, Databricks, and other technologies to efficiently process and transform data. I have a deep understanding of data migration, data cleansing, data transformations, data derivations, data mining, and data wrangling techniques. My experience spans across various databases, including Snowflake, SQL Server, Oracle, DB2, Teradata, AWS Redshift, Synapse, and PostgreSQL.Working extensively with cloud technologies, I have utilized Azure Blob Storage, Azure Data Factory (ADF), Azure Functions, Event Hubs, Logic Apps, Synapse, and Azure Data Lake Storage (ADLS). Additionally, I have worked with AWS services such as S3, EC2, EKS, Lambda Functions, as well as Databricks, Delta Lake, and Snowflake. My expertise also extends to the big data ecosystem, including Hadoop, HDFS, and Hive, as well as streaming data processing using Kafka.During my time at Harris Health System, I created ELT/ETL pipelines using AWS Glue, performed data transformations from SFTP server to S3, and leveraged PySpark to process data from Kafka topics. I excelled in writing complex SQL queries and reusable functions, as well as migrating data from Hive to Snowflake.At L&T Technology Services Limited, I gained experience with Informatica, Oracle, PL/SQL, and other tools, creating ETL workflows and implementing business logic through stored procedures and packages.I hold a Bachelor\'s Degree in Electronics and Communication Engineering from Jawaharlal Nehru Technological University, India, which has provided me with a strong technical foundation.