Revanth S

About Revanth S

Data Engineer with a strong background in Data Processing, Analytics, and Data Warehouse Management. Skilled in developing and implementing robust data pipelines, ensuring data consistency, and driving actionable insights for organizations. Proficient in a wide array of technologies and tools including Python, SQL, Databricks, and Azure services.Passionate about leveraging data-driven strategies to support business objectives and drive growth.• Finance, Data Warehouse, Retail, Insurance, E-commerce, Logistics, Sales• Data Streaming and Data Integration of B2BApplications• Azure Data Factory, Azure Databricks, ADLS Gen2• Database: Postgres, MySQL, DB2, SAP HANA, Snowflake• Data Warehouse: MySql, Snowflake• SQL, Python• Spark, SparkSQL,PySpark, CI/CD, Github• ETL Tools: ADF• Visualization tools: Power BI, Grafana

Experience

1. Assistant Manager

   Deloitte

   Feb 2024 — Present · Coimbatore, IN

   Migration of SSIS ETL pipelines to Azure Databricks, significantly enhancing scalability and performance.• Designed and developed a multithreaded dynamic data ingestion framework to process CSV and Parquet files into Delta Lake:o Ensured schema validation and efficient data mapping to handle schema evolution dynamically.o Implemented multithreading to execute the notebook multiple times simultaneously, improving ingestion efficiency.o Parameterized execution for adaptive parallel processing based on workload.• Built an automated data archival and retention strategy, maintaining historical data integrity while optimizing storage costs.• Developed a metadata-driven auditing framework to track changes and maintain data integrity across multiple ingestion layers.• Implemented an event-driven email notification system for critical data processing workflows using Azure Databricks.• Developed a fully automated data purge logic for history tables, ensuring data deletion based on retention periods:o Implemented multithreading to execute the purge function simultaneously across multiple partitions, optimizing performance.o Parameter-driven execution to dynamically adjust retention policies and efficiently manage storage.

Education

• KPR Institute of Engineering and Technology

  Bachelor of Engineering - BE

  2016 — 2020