Abhinav Verma

About Abhinav Verma

I have 2 years of experience as a Data Engineer at EY GDS, primarily working in the energy sector. My work focuses on building and optimizing data pipelines, managing large-scale datasets and leveraging cloud technologies to drive insights and efficiency.Domain Knowledge:I have hands-on experience in the energy sector, working with complex data models and processing large volumes of structured and unstructured data.Technical Skills:Data Engineering & Cloud Technologies:Programming: Python, Java, C, C++Databases: SQL, MySQL, NoSQLBig Data: PySparkData Modelling: OLTP, OLAPData Warehouse: HiveData Lake: Azure Data Lake Storage Gen2Data Orchestration: Azure Databricks, Azure Data Factory, Azure FunctionsSpark Processing: Batch ETL/ELT, Near Real-Time & Real-TimeScheduling Tools: Azure Databricks WorkflowsData Visualization: Power BICloud Platform: AzureVersion Control & Project Management: Git, Azure Dev BoardsTesting Experience:Unit TestingSystem Integration TestingUser Acceptance TestingData Science & AI/ML:Data Processing: Data Mining, Data Modelling, Data Wrangling (Pandas, NumPy)Generative AI: LLMs, Lang Chain, Lang Graph, Prompt Engineering, Vector Search, RAG, TAGAzure AI Services: AI Search, Language Service, Text Analytics, OCR, Computer VisionMachine Learning: Natural Language Processing, TF-IDF, Recommendation Systems, Supervised, Unsupervised Learning

Experience

1. Data Engineer

   EY

   Dec 2023 — Present

   Client - National Grid, United Kingdom (Electricity & Gas Operator)As a Consulting Data Engineer, I am part of the team responsible for shaping the strategy and roadmap of the NESO DAP platform using Azure and Databricks Lakehouse Architecture. We did a comprehensive assessment of the existing data infrastructure, identifying optimization opportunities and designing a scalable ETL framework to process 100+ datasets efficiently. During implementation, I focused on streamlining data movement from acquisition and conformed layers using Azure Data Factory, Azure Functions and Azure Databricks, ensuring seamless data movement, transformation and performance optimization for scalability and analytics readiness.During the delivery phase, I managed code promotion across DEV, FOF, UAT and PROD, performed unit and system testing and documented datasets. I created and executed test plans, identified and resolved bugs and ensured the solution met quality standards.

Education

• CHANDIGARH UNIVERSITY

  Bachelor of Engineering - BE, Computer Science

  2019 — 2023