Srikanth Pagidimunthala

About Srikanth Pagidimunthala

8 years of well-versed experience in Software development life cycle process makes ease in understanding different business requirements, analyzing, designing & developing applications on all platforms, Developing interface and implementation of Object Oriented Paradigm in real time services.• Software Development Methodologies like Waterfall, TOGAF, Iterative and Agile Software Development will be delivered with better solutions using agile concepts like SCRUM, Spiral Model etc. • Built Spark Applications using Spark SQL and Data Frame API extensively• Hands on experienced with Different SPARK Modules like SPARK core, SPARK RDD\'s, SPARK Data frame and SPARK SQL• Designed and developed custom UDF’s in Hive to enhance the performance while working on hive datasets• Having good hands on experience in creating queries in different database technologies like Spark SQL, HQL (Hive Query Language), CQL (Cassandra Query Language) to retrieve data from Cassandra cluster.• Developed infrastructure using the AWS Lambda, EC2, EMR, S3 and RDS as well as configurations and subscription using IAM, Security Groups, Rout 53 and SNS Topics. • Worked on CICD tools (GitHub, Jenkins) for better application deployments and configured ELK for Jenkins build logs and application logs• Ability to work under deadlines, pressure handling techniques to maintain better communication even at deadlines, inter disciplinary skills to communicate with all the team. • Excellent Analytical Skills, Problem solving methods, Quick learner and Profound communication skills to lead teams.

Experience

1. Senior Data Engineer

   Capital One

   Oct 2024 — Present · Richmond, VA, US

   Played keyrole in debugging spark applications to make sure the data load to s3 services as expected based on the customer requirements. Used Spark Scala to load data using Json, excel, csv and text file structure and implement functionalities required code development. Implemented data pipelines to handle data loads as part of team\'s inbuilt frameworks and fixed multiple data load bugs that can corrupt data. Used data bricks to read data from s3 buckets and implemented transformations to handle the business logics and write back to s3 buckets. Performed debugging for data written using pandas, NumPy and Spark to perform dataload and preprocessing on huge datasets. Prototyped analysis and joining of customer data using Spark in Scala. Implemented Spark and Spark SQL for faster testing and processing of data. Developed Spark/Scala, Python for regular expression (regex) projects for big data resources. Stored and retrieved data from data-warehouses using Presto and Snowflake. Developed UDF in Spark Scala and PySpark to handle transformations seamlessly in frameworks to meet business requirements. Built disaster recovery applications in the cloud to prevent data loss. Using Pandas and NumPy for reading data from XLSX, CSV for pivoting and transforming the data according to the business needs. Experienced building strategy, planning, and configuration in Amazon Web Services (AWS), and cloud services such as EMR, EC2, S3, RDS Instances, lambdas, step functions, redshift, snowflake. Experience writing streaming applications with Spark Streaming / Kafka.SSH into EMR AWS instances to handle any One Lake s3 instances. Handling schema changes in the data stream using Kafka. Using shell commands to push the environment and test files AWS using Jenkins automated pipelines also on EC2 instances and S3’s to retrieve data from various sources.

Education

• Kakatiya Institute of Technology & Science, Yerragattu Hillocks, Bheemaram, Hasanparthy, Warangal

  Bachelor of Technology - BTech, Information Technology

• Harrisburg University of Science and Technology

  Master's degree, Computer Science

  2016 — 2018