Sandeep Reddy Kamidi

About Sandeep Reddy Kamidi

Azure Data Engineer with over 4+years of hands-on experience in designing, deploying, and managing scalable data solutions on Microsoft Azure. Proficient in creating and optimizing data pipelines, implementing robust ETL processes, and ensuring data integrityusing Azure Data Factory, and Databricks. Experienced in large-scale data warehousing, developing data models, and integrating multipledata sources for comprehensive insights. Adept in SQL, Python, and Spark for data transformation, querying, and analysis.

Experience

1. Data Engineer

   Wipro

   Nov 2021 — Present · Hyderabad, IN

   Azure Data Engineer• Collaborated with data analysts and cross-functional teams to deliver data-driven solutions for enhanced modeling, integration, anddecision-making.• Designed and implemented 25+ scalable ELT pipelines using Azure Data Factory, Azure Databricks, Synapse Analytics, and AzureSQL Database.• Orchestrated ingestion from diverse sources (CSV, JSON, SQL) into Azure Data Lake Storage Gen2.• Processed and transformed TBs of daily data using Spark SQL and PySpark in Databricks, improving performance by 30%.• Developed real-time and batch pipelines using ADF, Databricks for streaming and transactional data.• Ensured data governance and compliance using RBAC, Key Vault, and Managed Identities.• Built 10+ end-to-end data flow pipelines to streamline data transformation and integration processes.• Managed relational (SQL Server, MySQL) databases for structured and semi-structured data.• Applied strong skills in SQL and Python for data querying, transformation, and performance tuning.• Utilized Azure DevOps for version control, CI/CD, and deployment automation.• Optimized Azure cloud costs and managed storage, network security, and infrastructure reliability.Data Engineer• Utilized Snowflake and Azure Synapse Analytics to optimize data warehousing processes, enhancing data retrieval speed and efficiency• Developed ETL/ELT processes using SSIS, PySpark, and Python, which streamlined data integration and improved data accuracy• Executed data transformation tasks using Python, PySpark, and SQL, leading to improved data quality and consistency• Processed large datasets using Databricks and Apache Spark, which improved data processing speed and scalability• Managed data storage solutions using Azure Data Lake Storage and Azure Blob Storage, ensuring secure and efficient data access• Managed databases using SQL Server and Azure SQL Database, ensuring data integrity and optimizing performance for improved dataretrieval.

Education

• Osmania University

  Bachelor's degree, Mathematics statistics computerscience

• Birla Institute of Technology and Science, Pilani

  Master of Technology - MTech, software systems