Mani Shankar Reddy
Data Engineer @S&P Global
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @S&P Global
US
I work as a Data Engineer focused on building scalable, reliable, and cloud-based data pipelines to support enterprise analytics and reporting needs.Designed and built scalable data pipelines using Python, SQL, and Azure Data Factory to ingest and process data from APIs, databases, and flat files (5M+ records daily).Developed batch and near real-time data ingestion workflows using Azure Data Factory and Kafka to enable continuous data processing and timely data availability.Processed and transformed large datasets using PySpark and Databricks to support large-scale analytics and data aggregation.Designed and maintained Snowflake data models to improve query performance and support high-volume analytical workloads.Implemented end-to-end orchestration using Azure Data Factory, including scheduling, dependency management, and monitoring of workflows.Built monitoring and logging solutions using Azure monitoring tools and alerts, improving pipeline reliability and reducing failure resolution time.Optimized Spark jobs and SQL queries using partitioning and performance tuning techniques to improve processing efficiency.Implemented data quality checks and validation frameworks to ensure consistency and reliability of production data.Collaborated with business and analytics teams to deliver curated datasets and Power BI dashboards for reporting and insights.Worked on CI/CD pipelines using Git and Jenkins to support automated deployment and version control of data workflows.
EDUCATION
CVR College Of Engineering
Bachelor of Technology - BTech, Electronics and communication engineering
University of Central Missouri
Master's degree, Computer Science
ABOUT MANI SHANKAR REDDY
Data Engineer with 3+ years of experience designing and building scalable ETL pipelines and real-time data processing systems. Currently working at S&P Global, handling high-volume data pipelines processing millions of records daily using Azure Data Factory, Kafka, and Databricks. Strong expertise in Python, SQL, Spark, AWS, and Azure, with hands-on experience in Snowflake, data modeling, and performance optimization. Proven ability to improve pipeline reliability, reduce processing time, and deliver high-quality data for analytics and reporting. Experienced in end-to-end data pipeline development, including ingestion, transformation, orchestration, and deployment using Airflow and CI/CD tools. Open to new opportunities and available for immediate discussion.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.