Sri Karthikeyan

About Sri Karthikeyan

Data Engineer with hands-on experience in building and supporting Data pipelines development and production environments.Experience with:• Cloud Ecosystem : Expertise in building and deploying scalable data on Microsoft Azure.• Data Integration : Developed Robust, automating ETL pipelines using Azure Data Factory.• Data Processing : Proficient in leveraging Azure Databricks and Pyspark for data processing and workflow optimization.• Core Programming : Strong foundation in Python and SQL for complex data manipulation and querying.• Modern Architecture : Hands - on experience in implementing data lakehouse principles including Delta Lake storage, for data reliability and compute performance.I\'m eager to collaborate with visionary teams and organizations passionate about leveraging data to tackle challenging business problems. My goal is to deepen my expertise in cloud-native architectures, real-time analytics, and Al-driven data solutions, continuously pushing the boundaries of what\'s possible with data.

Experience

1. Data Engineer

   Tata Consultancy Services

   Mar 2024 — Present · Chennai, IN

   Build and Managed daily ETL pipelines using ADF to ingest data from various sources into Azure Data Lake.Handled SCD using Pyspark and Databricks SQL to keep track of historical data changes.collaborated directly with business stakeholders and data analysts to get the business requirements for production-ready outcomes.Managed cloud computing resources and optimized Databricks configurations which reduced operational cost by 15%. Tuned and optimized slow-running Spark jobs by managing partitions, cutting down the execution time by 25% to 30%.setup basic monitoring and alerts for our ADF pipeline to catch and fix failures.Implemented data quality checks within the pipelines to drop or flag bad records before reaching prod tables.set up and maintained Delta Lake tables, for faster querying and to easily handle data updates(upsert) and deletes.wrote complex SQL queries to pull, filter and join specific data sets for daily and weekly reports.