Hari Vignesh M N
Apache Spark | Spark Steaming | SQL | Python | GCP | Airflow | Microsoft Azure | Azure Data Bricks | Azure Data Factory
- Role
- Data Engineer at EY
- Location
- Madurai South, TN, IN
- LinkedIn followers
- 500 followers
Experience
Data Engineer
Jan 2025 — Present · Trivandrum, IN
Developed scalable ETL/ELT pipelines using Azure Data Factory(ADF), improving data availability by 40% for business-criticalanalytics.• Designed event-driven ADF workflows with advanced dependencymanagement, ensuring SLA compliance and resilient dataorchestration.• Built optimized transformation frameworks using PySpark in AzureDatabricks, reducing processing time by 45% for high-volumedatasets.• Implemented schema evolution strategies using Parquet(mergeSchema) to support dynamic data models and productionstability.• Engineered incremental loading frameworks with optimized T-SQLstored procedures, maintaining 100% KPI accuracy with reducedlatency.• Automated CI/CD deployments using Azure DevOps and GitHub,enabling controlled multi-environment releases.• Leveraged GCP (Dataproc – Hadoop, Hive, Spark, BigQuery, Airflow)for data validation, cross-platform reconciliation, and workflowverification.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.