Rishitha Repalle
Senior Data Engineer @Harmonic Fund Services
Signup · Get unlimited contacts
WORK HISTORY
Senior Data Engineer @Harmonic Fund Services
Toronto, ON, CA
Designed and implemented cross-cloud data pipelines leveraging AWS (EMR, S3, EC2, Lambda) and Azure (ADF, Synapse, Databricks, ADLS Gen2) to build a unified, scalable data platform supporting enterprise analytics. Developed and optimized PySpark jobs to extract and transform data from S3, Snowflake, REST APIs, and ADLS, integrating structured and semi-structured data (CSV, Parquet, JSON) across both ecosystems. Built modular dbt models and SQL transformations deployed through Airflow DAGs and ADF pipelines, automating batch and streaming workflows. Orchestrated ETL processes using Apache Airflow and Azure Data Factory, handling retries, dependency management, and SLA tracking for hybrid workloads.🧠 Managed and tuned memory-optimized EMR and Azure Databricks clusters, improving runtime performance and reducing costs through resource scaling and caching techniques. Integrated AWS Secrets Manager and Azure Key Vault to manage credentials securely across both cloud environments, ensuring compliance and data protection. Implemented data governance and lineage tracking via Azure Purview and AWS tagging frameworks, ensuring consistency and auditability. Collaborated with cross-functional teams to design end-to-end data pipelines supporting analytics and ML workloads, delivering trusted data for BI dashboards and predictive insights.
EDUCATION
Conestoga College
Postgraduate Degree, Cloud Data Management
Narasaraopeta Engineering College
Bachelor of Technology, Electronics and Communication Engineering
ABOUT RISHITHA REPALLE
I am a Data Engineer and BI Developer with over 8 years of experience designing and implementing scalable, resilient, and high-performance data platforms across Azure and AWS cloud environments. My expertise spans Azure Data Factory, Databricks, Synapse Analytics, ADLS Gen2, Purview, and Key Vault, as well as AWS EMR, S3, EC2, Lambda, Glue, Step Functions, Cloud Watch, Redshift, Secrets Manager, and Airflow, enabling seamless hybrid data solutions for enterprise-scale analytics.I specialize in PySpark, Scala, Python, and SQL for developing complex ETL and ELT pipelines, performing advanced data transformations, and optimizing distributed workloads on both Azure and AWS. My experience includes building Medallion and Lakehouse architectures with Delta Lake, implementing ACID compliance, schema evolution, and time-travel capabilities to ensure data accuracy and reliability.I have extensive hands-on experience with dbt, Terraform, and Kafka/Event Hub, automating infrastructure provisioning, data transformation, and real-time streaming workflows. I also design and orchestrate data workflows with Airflow and ADF, integrating CI/CD pipelines using GitHub Actions and Azure DevOps for automated, version-controlled deployments.As a strong advocate of data governance, observability, and cost optimization, I have leveraged Azure Purview, Unity Catalog, Key Vault, and CloudWatch to establish secure, transparent, and efficient data operations. My work ensures compliance with enterprise and regulatory standards while maintaining performance and scalability.In addition to engineering, I bring a solid background in business intelligence and analytics, developing interactive Power BI, SSRS, and SSAS dashboards that transform complex datasets into actionable insights. I collaborate closely with analytics, product, and engineering teams to deliver business-aligned, high-availability data ecosystems that support analytics, predictive modeling, and strategic decision-making across global organizations.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.