Babu M
Senior Data Engineer| Senior Machine Learning Engineer| Gen AI| Azure AI |AWS| Azure | Microsoft Fabric| Snowflake|Databricks| Dbt| Pyspark|SQL
- Role
- Senior Data Engineer Data Scientist at Discover
- Location
- Charlotte, NC, US
- LinkedIn followers
- 500 followers
About Babu M
At Discover Financial Services, my expertise in Azure and PySpark has been instrumental…
Experience
Senior Data Engineer Data Scientist
Sep 2022 — Present · Cary, NC, US
Developed the Spark PySpark scripts and UDF\'s to read from AWS S3 bucket to perform transformations on large datasets using AWS Databricks.Developed the scalable data ingestion pipelines on AWS EMR Spark cluster using Spark SQL. Also Worked with Cosmos DB.Developed the robust ETL pipeline in AWS GLUE to integrate data from both on-premises to cloud and applied transformations using PySpark to load the enrich data to AWS REDSHIFT.Create pipelines in AWS Glue using linked services to extract, transform and load data from multiple sources like AWS RDS, and AWS Redshift.Developed robust and scalable data integration pipelines to transfer data from S3 bucket to Redshift database using Python and AWS Glue.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.