Pooja B.

About Pooja B.

As a senior data engineer at Freddie Mac, I have over nine years of experience in developing scalable and reliable data pipelines using cutting-edge technologies. My mission is to enable data-driven decisions and insights that support Freddie Mac\'s vision of making home possible for millions of families and individuals. I leverage my skills in SQL, Python, Azure Databricks, and other tools to process various formats and sources of data, such as XLS, JSON, TXT, RDBMS, and Swift Object Store. I also use Snowpark, PySpark, MapReduce, NiFi, Kafka, Spark, Hive, and Scala to ingest, transform, and analyze data in HDFS, AWS S3, Elastic search, Redshift, and Snowflake. I bring diverse perspectives and experiences to the team, as I have worked in different domains, such as banking, retail, and mortgage, and have a master\'s degree in computer science. Tech Stack: Python, SQL, Scala, Spark, Snowflake, Hadoop, Hive, ADF(Azure Data Factory), Azure, AWS, Power-BI, GCP. Database, PostgreSQL Database, Oracle, Teradata, SQL SERVER. AWS: EMR, Glue, Lambda, CloudFormation, DynamoDB, Athena, EC2. Project Management: Agile, JIRA, Confluence.

Experience

1. Senior Data Engineer

   Capital One

   Nov 2023 — Present · McLean, VA, US

   Worked on enhancing the features for the circuit breaker functionality on the Overdraft (OD) UI using Python and Scala. Ensured seamless data integration and feature updates to improve user experience and system reliability.• Loaded and managed data in DynamoDB, ensuring high availability and performance for real-time data access and updates.• Utilized AWS Step Functions to orchestrate various processes, including inclusion, exclusion, and segmentation steps. This ensured efficient workflow management and automation of complex business logic.• Developed Scala-based projects to perform aggregation of various rules and check if user data falls under specific metrics. • This included Implementing aggregation logic to evaluate complex rule sets.• Ensuring scalability and performance of rule evaluations, Integrating with AWS services for data processing and storage.• Leveraged AWS Glue jobs for ETL processes and AWS Lambda for serverless compute to run real-time data processing tasks. This included:• Creating Glue jobs to transform and load data efficiently.• Using Lambda functions to trigger specific steps within workflows.• Ensuring seamless integration between Glue and other AWS services.• Enhanced the circuit breaker feature to dynamically handle system loads and prevent failures, ensuring robustness and reliability of the application.• Employed a comprehensive technology stack including Python, Scala, AWS Step Functions, DynamoDB, AWS Glue, and AWS Lambda to deliver high-quality, scalable solutions.

Education

• Monroe College

  Master's degree

• Alamuri Ratnamala Institute of Engineering and Technology

  Bachelor's degree

Skills

• Apache Zookeeper
• Linux
• Mysql
• Javascript
• Professional Ethics & Leadership
• Junit
• Javaserver Pages (Jsp)
• Spark Sql
• Unix
• Shell Scripting
• Apache Spark
• Flume
• Web Design
• Hdfs
• Centos
• Sqoop
• Java
• Microsoft Sql Server
• Scala
• Spark Streaming
• Apache Kafka
• Ajax
• Big Data
• Intellij Idea
• J2ee
• Amazon Ec2
• Spring
• Hbase
• Project Management
• Hadoop
• Eclipse