Pradeep V.

About Pradeep V.

With over six years of experience as an Assistant Consultant at Tata Consultancy Services, I specialize in data migration and leveraging tools like Azure Databricks and Autosys. My work supports efficient and seamless transitions of data systems, ensuring accuracy and consistency. I am passionate about delivering data-driven solutions, enabling organizations to optimize their technological processes. My work reflects a commitment to utilizing advanced tools and strategies to meet business goals effectively.

Experience

1. Assistant Consultant

   Tata Consultancy Services

   Jan 2025 — Present

   Worked within the Medallion Architecture to perform operations across Raw (Raw), Harmonised (Cleansed), and Curated (Aggregated) layers in Delta Lake, ensuring proper data transformations, removal of duplications, and implementation of business logic using Databricks.Collaborated with data analysts and business stakeholders to gather requirements, design solutions, and implement effective data integration processes.Utilized Apache Spark Framework and SQL for data manipulation, aggregation, transformation, sorting, partitioning, and validation to ensure accuracy and quality.Designed transformation logic to standardize data flow between source and target systems, ensuring consistency and reliability.Worked on bug fixing, gathering requirements, and designing innovative solutions to address complex business needs.Conducted performance tuning, data cleansing, and analysis using SQL and other tools to improve data quality and efficiency.Designed and launched new data extraction, transformation, and loading processes to support scalable and efficient data solutions.Developed data loading and processing pipelines using Apache Spark, incorporating advanced operations like shuffling, sorting, partitioning, merging, and concatenation.Applied deduplication & incremental data loading techniques using window functions, merge, and watermarking strategies.Implemented slowly changing dimension (SCD Type 2) logic using Delta Lake MERGE operations for dimensional data management.Optimized Spark jobs by tuning shuffle partitions, broadcast joins strategies.Supported UAT and production deployments, including pipeline monitoring and issue resolution.

Education

• Vellore Institute of Technology, Vellore

  Master's degree, Information Technology

• Vellore Institute of Technology

  Master's degree, Information Technology

  2009 — 2011