Gopi Goud

Senior Big Data Engineer @First Citizens Bank

Raleigh, NC, US

Designed and implemented a scalable data architecture on AWS, using Amazon S3, AWS Glue, and Redshift to build a centralized data lake, optimizing storage and analytics for large, diverse datasets.•Designed and implemented a scalable data platform on AWS (S3, Glue, Redshift), creating a centralized data lake that streamlined analytics and optimized storage for large, diverse datasets.•Developed automated ETL pipelines using AWS Glue, Apache NiFi, PySpark, and Lambda, which reduced manual processing by 70%, significantly improving the efficiency of data ingestion, transformation, and movement.•Led data processing efforts with PySpark and Databricks, ensuring large-scale transformations were optimized for business intelligence and real-time analytics.•Built real-time streaming data solutions using AWS Kinesis and PySpark Streaming, reducing latency and enabling quick, actionable insights to drive business decisions.•Integrated chatbot functionality with AWS Lex and Python, automating routine tasks and improving customer interaction efficiency.•Optimized data warehouses on AWS Redshift and Snowflake, improving query performance and ensuring scalability for large-scale analytics.•Implemented cost-saving measures through AWS best practices like reserved instances and auto-scaling, reducing infrastructure costs by 20% while maintaining scalability.•Enabled self-service analytics by integrating AWS Athena and QuickSight, empowering business teams to access and interpret data independently, leading to better decision-making.•Ensured data security and compliance by setting up AWS IAM, CloudTrail, and KMS, safeguarding sensitive data and aligning with industry standards.•Developed and managed CI/CD pipelines with Jenkins, Docker, and Kubernetes, enabling faster and more reliable deployment cycles.•Developed batch and real-time data pipelines using PySpark and AWS Glue, ensuring smooth data ingestion from multiple sources and enabling high-performance processing.