Lahari Gupta

About Lahari Gupta

Over 8+ years of experience in Data Engineering, Data Pipeline Design, Development, and Implementation as a Sr. Data Engineer/Data Developer and Data Modeler. • Expertise working with AWS cloud services like EMR, S3, Redshift, EMR cloud watch, for big data development. • Hands-On experience on Spark Core, Spark SQL, Spark Streaming, and creating the Data Frames handle in SPARK with Scala. • Working experience in developing applications involving Big Data technologies like Map Reduce, HDFS, Hive, Sqoop, Oozie, HBase, Pig, NiFi, Spark, Scala, Kafka, and ETL(DataStage). • Experience in Data ingestion to one or more Azure services (Azure Data Lake, Azure Storage, Azure SQL, Azure DW) and processing the data in Azure Databricks. • Worked with GCP and created GCS Bucket, G-Cloud functions, used gsutil command line. • Designing and Developed Oracle PL/SQL and Shell Scripts, performed Data Import/Export, Data Conversions, and Data Cleansing. • Strong experience in Software Development Life Cycle (SDLC) including Requirements Analysis, Design Specification, and Testing as per Cycle in both Waterfall and Agile methodologies. • Ability to work effectively in cross-functional team environments, excellent communication, and interpersonal skills. Email: l••••••••@gmail.com PH: 81•••••55

Experience

1. Data Engineer

   USAA

   Mar 2022 — Present · San Antonio, TX, US

   Developed Spark applications using PySpark and Spark-SQL for data extraction, transformation, and aggregation from multiple file formats. •Worked on the Reconcile Process where detailed metrics on the aggregate of the calls received and all the issues that happened during the month are reported and sent to the Business. •Handling data from multiple third-party sources and combining them on specific criteria and applying necessary transformations in PySpark using pandas, NumPy, io, re, and custom-built packages. •Worked on POC where specific AD group for users is removed so that we can import appropriate data into our Database.

Education

• Jawaharlal Nehru Technological University, Hyderabad

  Bachelor of Technology - BTech, Information Technology

  2009 — 2013