Sanket J.

About Sanket J.

As an accomplished Data Engineer, my goal is to apply my technical prowess and management abilities to spearhead data-driven excellence. With expertise in designing and implementing data solutions on the cloud, and employing data pipelines, I aim to create impactful data solutions that drive business growth and deliver actionable insights. Let\'s connect and explore opportunities to make data the foundation of informed decision-making and organizational success.

Experience

1. Software Engineer (Data Engineer)

   Tata Consultancy Services

   Mar 2022 — Present

   Designed and developed a Data Lake Storage processing pipeline on the Databricks platform, handling diverse data sources (CSV, JSON, Parquet) using PySpark\'s DataFrame API and Python for multi-layer transformations. Automated data pipeline validation with Pytest and Azure Data Factory, reducing manual effort and expediting validation tasks. Developed Python logging scripts to track data pipeline activities and configured alerting systems for proactive issue response. Orchestrated data workflows with Azure Data Factory, AWS Glue, automating scheduling and execution of pipelines. Created and maintained technical documentation for data pipelines, including code flow diagrams, data dictionaries, testing procedures, and troubleshooting guides. Contributed to an agile team delivering automated CI/CD pipelines for code deployment. Collaborated with various business verticals (Downstream Insurance, audit team, SAP data team, IT team) to meet client requirements.

Education

• Great Lakes Institute of Management

  PGP in Data Engineering & Data Analytics

  2019 — 2019

• Maharashtra Institute of Technology Aurangabad

  Diploma In Mechanical Engineering, Mechanical Engineering

  2012 — 2014

• Dr. Babasaheb Ambedkar Marathwada University, Aurangabad

  Bachelor of Engineering - BE Mechanical

  2015 — 2017