Abhinav Verma

About Abhinav Verma

I have 1.5 years of experience as a Data Engineer at EY GDS, primarily working in the energy sector. My work focuses on building and optimizing data pipelines, managing large-scale datasets, and leveraging cloud technologies to drive insights and efficiency. Domain Knowledge: I have hands-on experience in the energy sector, working with complex data models and processing large volumes of structured and unstructured data. Technical Skills: Data Engineering & Cloud Technologies: Programming: Python, Java, C, C++ Databases: SQL, MySQL, NoSQL Big Data: PySpark Data Modelling: OLTP, OLAP Data Warehouse: Hive Data Lake: Azure Data Lake Storage Gen2 Data Orchestration: Azure Databricks, Azure Data Factory, Azure Functions Spark Processing: Batch ETL/ELT, Near Real-Time & Real-Time Scheduling Tools: Azure Databricks Workflows Data Visualization: Power BI Cloud Platform: Azure Version Control & Project Management: Git, Azure Dev Boards Testing Experience: Unit Testing System Integration Testing User Acceptance Testing Data Science & AI/ML: Data Processing: Data Mining, Data Modelling, Data Wrangling (Pandas, NumPy) Generative AI: LLMs, Lang Chain, Lang Graph, Prompt Engineering, Vector Search, RAG, TAG Azure AI Services: AI Search, Language Service, Text Analytics, OCR, Computer Vision Machine Learning: Natural Language Processing, TF-IDF, Recommendation Systems, Supervised, Unsupervised Learning

Experience

1. Cloud Data Engineer

   EY

   Dec 2023 — Present

   Client - National Grid, United Kingdom (Electricity & Gas Operator)As a Consulting Data Engineer, I am part of the team responsible for shaping the strategy and roadmap of the NESO DAP platform using Azure and Databricks Lakehouse Architecture. We did a comprehensive assessment of the existing data infrastructure, identifying optimization opportunities and designing a scalable ETL framework to process 100+ datasets efficiently. During implementation, I focused on streamlining data movement from acquisition and conformed layers using Azure Data Factory, Azure Functions and Azure Databricks, ensuring seamless data movement, transformation, and performance optimization for scalability and analytics readiness.During the delivery phase, I managed code promotion across DEV, FOF, UAT, and PROD, performed unit and system testing, and documented datasets. I created and executed test plans, identified and resolved bugs, and ensured the solution met quality standards.

Education

• CHANDIGARH UNIVERSITY

  Bachelor of Engineering - BE

  2019 — 2023