Biswarup Nandi

About Biswarup Nandi

As a Data and ML Engineer with over 9 years of experience, I specialize in delivering scalable and efficient Data and MLOps solutions across industries such as BFSI, Healthcare, and Banking. My expertise encompasses AWS, Azure, Databricks, Apache Spark, Python, and SQL, allowing me to design, develop, and optimize data systems tailored to modern organizational needs. I have led several Databricks integrations, enhancing ETL pipelines using Delta Live Tables, Databricks Asset Bundle, Databricks Workflows, and Unity Catalog, ensuring seamless data operations and strong governance. Whether implementing batch processing or real-time streaming, I have successfully executed data engineering solutions that drive impactful business insights. I have extensive experience in developing ETL pipelines using AWS MSK (Kafka), Kinesis, Lambda, Glue, Redshift, and Snowflake, ensuring efficient data ingestion and transformation across systems. My proficiency in MLOps allows me to automate machine learning model deployment and monitoring using DVC, Dagshub, Databricks MLFlow and AWS SageMaker, accelerating the delivery of data-driven applications. I have good experience in Azure Data Lake Storage (ADLS), Azure Data Factory. Additionally, I have developed and deployed data assets in the Life Sciences domain using Apache NiFi, Livy, PySpark, Glue, Athena, and Redshift, offering real-time insights that empower decision-making. With a solid foundation in DevOps, I have designed CI/CD pipelines using Jenkins, AWS CodePipeline, Git, SonarQube, CheckMarx, and JFrog Artifactory to streamline automated deployments and ensure continuous delivery of quality solutions. My skills in SQL, Python, and PySpark have enabled me to handle complex data transformations and integrations, ensuring systems remain scalable and high-performing. I am well-versed in Data Warehousing, including OLAP/OLTP, Lambda/Kappa architectures, Change Data Capture (CDC), and Data Modeling techniques like Star and Snowflake schemas, as well as Slowly Changing Dimensions (SCD). I have a proven track record in implementing cloud security using IAM, KMS, and encryption technologies to safeguard sensitive data and ensure compliance with industry regulations. I thrive in Agile/Scrum environments, collaborating with cross-functional teams to deliver high-performance solutions that meet business goals on time and within budget.

Experience

1. Lead Software Engineer - Mlops Engineer

   Mastercard

   Jul 2025 — Present · Pune, IN

   As part of the Foundry R&D team, I’m currently working as an MLOps Engineer focused on building scalable, secure, and automated machine learning pipelines on the Databricks platform. My role involves: Designing and implementing end-to-end MLOps workflows using Databricks MLflow, Unity Catalog, and Delta Lake for efficient model lifecycle management. Collaborating with data scientists and ML engineers to automate model training, validation, deployment, and monitoring across multiple environments. Enabling reproducibility and governance by integrating feature stores, lineage tracking, and CI/CD with tools like Jenkins, Git, and Databricks Asset Bundles. Driving infrastructure as code (IaC) for Databricks workspace configuration using Terraform/CloudFormation, ensuring consistent and compliant deployments. Partnering across teams to scale AI/ML implementations for business-critical use cases while aligning with Mastercard’s data and model governance standards.

Education

• St. Xavier's College (Autonomous), Kolkata

  Bachelor of Science - BS, Computer Science

  2010 — 2013

• Rajabazar Science College (UCSTA) - University of Kolkata

  Bachelor of Technology - BTech, Computer Science and Engineering

  2013 — 2016