Vivek Singh
Data Engineering Analyst @ Accenture
- Role
- Data Engineering Analyst at Accenture
- Location
- Indore, MP, IN
- LinkedIn followers
- 500 followers
About Vivek Singh
As a data engineering professional, I bring extensive expertise to Accenture, where I\'ve led key initiatives like migrating from Hadoop to BigQuery and optimizing data workflows with Databricks. My role involves the end-to-end development of complex SQL procedures and Python scripts, along with orchestrating automated data pipelines using Cloud Composer and Airflow DAGs. I am committed to maintaining high standards throughout the development lifecycle, including unit and regression testing.Collaboratively, I\'ve enhanced existing systems and streamlined the onboarding process for new analysts by sharing valuable project insights. My educational background in Information Technology from the Oriental Institute of Science & Technology, combined with certifications in cloud engineering and MongoDB, underpins my passion for leveraging technology to solve complex problems. I\'m driven by a commitment to continuous learning and thrive in dynamic, team-oriented environments.
Experience
Data Engineering Analyst
Jul 2021 — Present · Indore, IN
For Data Processing and ETL:Developed and optimized ETL pipelines using Databricks notebooks with PySpark to process large-scale datasets.Built and managed Delta Lake tables for reliable and high-performance data storage and querying.Transformed raw data into structured formats using Databricks to prepare it for analysis and reporting.For Performance and Optimization:Improved query performance by leveraging Databricks\' photon engine and optimizing cluster configurations.Managed Databricks jobs and workflows to schedule and automate data processing tasks efficiently.For Collaborative and Integrated Work:Utilized Databricks notebooks for collaborative development, allowing teams to work on data projects together.Integrated Databricks with other cloud services like Cloud Composer or Airflow to create comprehensive data orchestration solutions.
Education
Oriental Institute of Science & Technology
Bachelor of Technology - BTech, Information Technology
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.