Ganesh Singh

About Ganesh Singh

As a dedicated professional, I specialize in cloud data engineering with a focus on Microsoft Azure services and big data tools. My expertise includes-> Azure Services: I have experience working with various Azure services such as Data Factory, Data Lake, Databricks, Synapse, Cosmos DB, Key Vault, Azure DevOps, Azure Automation, and Logic Apps-> Apache Tools: I am proficient in using Pyspark, an open-source distributed computing system, for data processing and analysis-> Programming: I have strong skills in Python, a widely used programming language for data analysis and machine learning-> Databases: My experience includes working with SQL Server, a popular relational database management system-> DevOps: I am familiar with GIT, Azure DevOps and JIRA, commonly used tools in software development and project management-> Visualization: I am skilled in using PowerBI, a business analytics service by Microsoft, for data visualization and reporting-> Agile & Scrum: I am familiar with agile and scrum methodologies, having worked on projects that utilize these frameworks-> Type of Projects worked: I have worked on projects in the healthcare and pharmaceutical industries and FMCG product.As an enthusiastic data science professional, I am passionate about learning and developing efficient solutions to solve business problems.

Experience

1. Data Engineer

   PepsiCo

   Jan 2023 — Present · Gurugram, IN

   Technologies: DATABRICKS, PYSPARK, ADLS, ADF, PYTHON, AZURE DEVOPS, PYTEST, DBT• Acted as a bridge between technical and business stakeholders, actively participating in requirement discussions and delivering critical features like pipeline sustain notifications in agile environment along with managing team of 4 DE and 2 QA’s• Work with different scope 3 emission data sets like Packaging, Agriculture and transport, fuel energy, waste and emission data.• Proficiently handled data ingestion from Teradata EDW, my-EHS and manual files and sphere emission API data, following the Medallion architecture standards of Microsoft. Ensured seamless inbound data flow, maintaining data quality and integrity.• Reduced the effort of manual report generation of emission data for sustainability office by implementing automation that sends aggregated reports to stakeholder over email.• Build an end-to-end pipeline life cycle notification system, which sends a team notification on success or raises a service now ticket which is assigned to production sustain team.• Optimised the Databricks cost by adding pool clusters and strategic data caching and partitions.• Implemented PY-test and DBT-Databricks as unit testing frameworks, ensuring code quality and reliability as code moved from the development to QA environments.• Improved downstream consumption of data using graph-ql API with help of internal DSI team.

Education

• Kendriya Vidyalaya

  High School, Science

  2012 — 2014

• DIT UNIVERSITY

  Bachelor of Technology (B.Tech.), Computer science engineering

  2014 — 2018