Dhora Bhabbu Kth

Data Engineer @Wipro

Bengaluru, IN

Proficient in Azure Data Factory (ADF), Azure Databricks, PySpark, Spark SQL and Azure Data Lake Storage (ADLS) for implementing robust batch and semi-structured data workflows to support enterprise-scale data platforms.•Experience in implementing Medallion Architecture by structured data into Bronze, Silver and Gold with Azure Data Lake, Delta Lake, ensuring efficient storage, processing, and analytics.•Experience in developing Data Cleansing Frameworks using Pyspark to automate data validations, ensuring integrity, completeness, and consistency across end-to-end data pipelines.•Expertise in building near real-time data processing workflows using Apache Spark Structured Streaming in Databricks.•Experience in Optimizing Spark Jobs for high-performance data processing and efficient resource utilization in large-scale data environments.•Experience in working with Delta tables and schema evolution in Databricks, ensuring ACID compliance and efficient data versioning in large-scale processing.•Experience in incremental data processing using Databricks Autoloader for efficient and reliable file ingestion.•Experience in implementing Error Logging, Audit Logging, and Data Reconciliation processes, along with automating email notifications for pipeline failures using Azure Logic Apps, to enhance monitoring, troubleshooting, and data reliability.•Hands-on experience in working with multiple data formats such as CSV, JSON, and Parquet for efficient data ingestion and processing across diverse workflows.•Proficiency in writing SQL queries in SQL server for data retrieval and manipulation.•Strong knowledge of data warehousing concepts such as fact and dimension tables, Star and Snowflake schemas, along with data modeling techniques for designing normalized and denormalized structures to support ETL best practices, reporting, analytics, and business intelligence.