Mani Krishna

Azure Data Engineer @Sobeys

CA

I have designed and developed scalable data solutions and automated data pipelines using a modern cloud stack including Azure Databricks, Azure Data Factory, Snowflake, and Apache Spark, ensuring seamless data integration, reliability, and high performance across multiple environments.Developed Databricks Notebooks to apply complex data transformations using Spark SQL, ensuring efficient data flow between diverse systems, databases, and applications. Utilized Apache Hudi to implement incremental data processing and low-latency ingestion, significantly improving query performance and reducing overall storage costs.Constructed and maintained processes for data transformation, data structures, metadata management, dependency control, and workload optimization, ensuring strong data integrity and availability across production environments. Applied PySpark, Scala, and SQL within Databricks for data refinement and performance tuning, achieving faster data processing and optimized analytics workloads.Worked extensively with distributed computing frameworks like Apache Spark to process, clean, and analyze large-scale datasets, demonstrating deep expertise in big data processing, distributed architecture, and performance tuning.Implemented and optimized Snowflake features including Snowpipe, Streams, Tasks, UDFs, Stored Procedures, Zero-Copy Cloning, Time Travel, Micro-partitioning, and External Tables, enabling efficient and automated data ingestion, transformation, and management in the cloud.Automated end-to-end data movement using SnowPipe, Streams, and Tasks, ensuring consistent data synchronization between source systems and the Snowflake data warehouse. Designed robust ETL pipelines to improve data latency, enhance visibility, and ensure high data availability for analytics and reporting.

Vignan's Lara Institute of Technology & Science, Vadlamudi, Chebrolu Mandal, PIN-522213(CC-FE)

Bachelor's degree