Sowmya Rebelli

About Sowmya Rebelli

Data Engineer specializing in architecting, developing, and optimizing scalable data solutions. With deep expertise in big data technologies—including Hadoop, Apache Spark, Kafka, and cloud platforms like Azure and AWS—I build robust data pipelines that power real-time analytics and drive impactful decision-making. At Microsoft, I lead the creation of high-performance data solutions on Azure, significantly enhancing data processing speeds and optimizing cloud resource utilization. Previously, at Goldman Sachs, I streamlined complex data transformation workflows, improving data quality and enabling predictive analytics for strategic insights. Throughout my career, I\'ve consistently leveraged technologies such as Databricks, Snowflake, Python, SQL, and DevOps tools like Docker and Kubernetes to automate and enhance data engineering processes. My experience spans various industries, empowering organizations with actionable insights derived from clean, reliable, and accessible data. I\'m always eager to connect with professionals passionate about data engineering, analytics, and innovation—let\'s collaborate and unlock the true potential of data!

Experience

1. Senior Data Engineer

   Microsoft

   Jan 2024 — Present · CA, US

   Architected and deployed 10+ end-to-end reusable data pipelines on Azure, leveraging Azure Data Factory, Databricks, and Synapse to process petabytes of data efficiently reducing ETL job completion time by 40% and optimizing cloud resource usage.• Engineered robust data solutions in Databricks using Python, PySpark, and SQL by implementing Auto Loader for real-time data ingestion into Delta Lake, performing data profiling, cleaning, and validation, and achieving a 30% performance improvement.• Built and maintained real-time analytics (KQL engine) in Microsoft Fabric for streaming data ingestion from Event Hubs, IoT, and Kafka.• Utilized Terraform to automate infrastructure provisioning, while CI/CD pipelines in Azure DevOps optimized deployment procedures— cutting deployment time by 40% and improving system dependability.• Optimized SQL queries in Azure Synapse, achieving a 50% improvement in query execution time.• Designed and developed Power BI dashboards for interactive data visualization, leveraging DAX, Power Query and M language for advanced analytics.

Education

• BV Raju Institute of Technology (BVRIT)

  Bachelor of Technology - BTech, Electronics and Communications Engineering

• California State University - East Bay

  Master of Science - MS, Business Analytics