Sangamesh Kotgi
Software Engineer at Mphasis | Data Engineer | SQL | Python | Pyspark | Azure data Fundamental | Databricks
- Role
- Data Engineer at Mphasis
- Location
- Kalaburagi, KA, IN
- LinkedIn followers
- 500 followers
About Sangamesh Kotgi
With 4 years of hands-on experience, I specialize in designing, implementing, and optimizing data solutions using SQL, Python, PySpark, and a suite of Azure services including Databricks and Azure Data Fundamentals. My background in these technologies has equipped me with the skills to handle complex data engineering tasks, ensuring efficient data processing and analysis for informed business decisions.I am passionate about leveraging data to drive strategic insights and business growth, continuously learning new technologies to stay ahead in the ever-evolving field of data engineering
Experience
Data Engineer
Dec 2021 — Present
Client: Marsh & Mclegan (Migration Project)Project: Revenue Master MigrationTechnologies: Databricks, PySpark, Informatica, Oracle, AWS DMS, Delta Lake• Contributed to the migration of enterprise data pipelines from Informatica to Databricks, modernizing legacy ETL processes into scalable, cloud-native solutions.• Implemented data ingestion from Oracle databases to Databricks using AWS DMS, leveraging source-to-target mappings to ensure accurate and efficient data replication.• Developed PySpark-based transformation logic in Databricks to process, cleanse, and enrich migrated data for downstream analytics.• Designed and maintained structured data flow across Landing → Bronze → Silver → Gold layers, following medallion architecture best practices.• Performed detailed schema mapping and data validation between Informatica mappings and Databricks PySpark transformations to ensure functional parity.• Conducted data reconciliation and row-level comparisons between source (Oracle/Informatica) and target (Databricks) systems to validate data completeness and accuracy.• Implemented Delta Lake tables to enable ACID transactions, time travel, and optimized storage for analytical workloads.• Optimized Spark jobs through partitioning strategies, efficient joins, and cluster configuration tuning to improve performance and reduce execution time.• Collaborated with business analysts and QA teams to resolve data discrepancies and ensure successful migration of critical revenue datasets.Client: Merlin Entertainments, UKProject: Run and Change Data Services | Domain: Entertainment | Tech: ADF, Databricks, PySpark, Delta LakeSupported and monitored ADF pipelines delivering critical business and analytical data.Debugged pipeline failures using PySpark, SQL, and Azure logs, ensuring high availability of data workflows.Performed data quality checks, validation, cleansing, and reconciliation during daily pipeline runs.Worked on building and optimizing Databricks Delta La
Education
APPA
Bachelor of Engineering, E&CE
Appa Institute of Engineering and Technology
Bachelor of Engineering - BE, Electronics and Communications Engineering
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.