Rishyanth

Experience

1. Senior Azure Data Engineer

   Kraft Heinz

   Nov 2022 — Present · Chicago, IL, US

   Analyzed data from Azure data storages using Databricks and Spark cluster capabilities, extracting valuable insights from large datasets.• Developed and maintained end-to-end operations of ETL data pipeline and worked with large data sets in Azure Data Factory; increased data pipeline throughput by 2x.• Orchestrated complex ETL and ELT workflows using Azure Data Factory, Databricks Jobs, Apache Airflow, and Jenkins pipelines, ensuring automated, reliable, and auditable data movement.• Implemented real-time streaming solutions with Azure Event Hubs, Azure Stream Analytics, Kafka, Spark Streaming, and Flink, enabling low-latency analytics and actionable insights.• Designed scalable, multi-layered data lake architecture in ADLS Gen2, integrating batch and streaming data for high-performance, analytics-ready storage.• Optimized ETL pipelines and SQL-based data integrations across MS SQL Server, Azure SQL DB, MySQL, PostgreSQL, MongoDB, and Cassandra, leveraging PySpark, DBT, T-SQL, and PL/SQL for transformations.• Implemented advanced data modeling and warehouse optimization in Azure Synapse Analytics and Snowflake, including Star and Snowflake schemas, indexing, partitioning, and query performance tuning.• Applied Azure Purview, Unity Catalog, and Azure Key Vault to ensure comprehensive data governance, security, lineage tracking, and compliance with HIPAA and GDPR regulations.• Automated monitoring, issue detection, and remediation workflows using Azure Monitor, Logic Apps, PowerShell scripts, and Azure CLI, reducing incident response time by 50%.• Developed and deployed machine learning and predictive analytics workflows in Azure Databricks and MLflow, integrating PySpark, TensorFlow, Scikit-learn, and Python for model training, evaluation, and deployment.• Implemented Retrieval-Augmented Generation (RAG) pipelines and integrated GenAI APIs (OpenAI, Cohere) into ETL workflows for smarter data enrichment, knowledge extraction, and predictive insights.