Soumyadip Mitra

About Soumyadip Mitra

Experienced professional with more than 14 years of experience in the Banking, Finance, Retirement Services & Telecom industry, working in the field of Data Analytics/MLOps/Data Engineering/Generative AI.Creative problem solver specializing in Data Ingestion Frameworks, Model Deployment Pipelines, Data Visualization, Analytics and Story telling.Analyzing complex issues striving to deliver every output in a high standard. Holds strong knowledge of Software Engineering. Applies analytic techniques and creativity in effective problem solving. Performs with a strong sense of responsibility and is comfortable in a dynamic environment.

Experience

1. Machine Learning Engineer

   Compunnel Inc.

   Aug 2021 — Present · Charlotte, NC, US

   Designed & Developed SPARK jobs in Databricks with Scala & PySpark to ingest data from OLTP databases & persist in AWS S3 & run AWS Glue Crawler & then run SQL queries on them using Apache Presto/Trino- Fine-tuned pretrained Huggingface Generative AI models using full fine-tuning and parameter efficient fine-tuning (PEFT) and evaluate them using ROUGE and BLEU scores on zero shot, one shot & few shot prompts in AWS Sagemaker- Worked with Feast feature store templates, Github and Snowflake for creating offline and online data stores- Created data aggregation MLOps pipeline using Apache Airflow, Data Build Tool (DBT), Dask & Snowflake to aggregate data from multiple S3 files for Anomaly detection pipeline- Aggregated data from multiple sources in Snowflake using DBT and Airflow- Created anomaly detection MLOps pipeline to run timeseries anomaly detection algorithms on the aggregated data & persist the results in AWS RDS (Postgres) db using Python, Snowflake connector & SQLAlchemy for alerts and reporting- Created Plotly Dash, Tableau and ThoughtSpot Dashboards from AWS S3 (Presto), Snowflake and Postgres for Descriptive and Predictive insights- Deployed ML models with Flask, Docker and Kubernetes cloud services (AWS EKS or Kubeflow) to create prediction endpoints- Dynamically triggered Lambda from AWS Step Function for copying tables from Snowflake to Postgres- Automate data transformation and loading to Snowflake using Data Build Tool (DBT) and Airflow- Run data integrity and data quality tests with DBT- Write unit test, integration test and end to end test with PyTest and Playwright for code and test coverage- Brainstormed new ML use cases aimed at increasing customer satisfaction and revenue- Designed and Developed Model Monitoring and Model Scoring MLOps pipelines with Airflow, Flask, Snowflake, Docker and Kubernetes- Leverage prompt engineering with openai API for sentiment analysis, text analysis, reporting and modeling

Education

• University of North Carolina at Charlotte

  Master's degree, Data Science and Business Analytics

  2019 — 2021

• West Bengal University of Technology, Kolkata

  Bachelor of Technology - BTech, Electronics and Communication Engineering (ECE)

  2004 — 2008