Uma Shankar Kota

About Uma Shankar Kota

Experienced Data Engineer with 5+ years of expertise in building scalable data pipelines, optimizing ETL processes, and leveraging cloud platforms like AWS and GCP. Proficient in Python, SQL, and big data technologies, with a passion for transforming complex datasets into actionable insights. Skilled in data integration, analytics, and driving impactful business decisions through efficient and innovative solutions.

Experience

1. Data Engineer

   Avalon Bay

   Oct 2023 — Present · VA, US

   Created ETL transformation scripts using dbt. Managed data ingestion from multiple sources into Google Cloud Storage (GCS) and BigQuery, transforming data using PySpark scripts in Databricks. Set up, troubleshot, and managed GCP infrastructure using Compute Engine, IAM, Persistent Disks, Cloud Functions, Cloud SQL, Cloud Storage, BigQuery, and Dataflow. Experience with Database Migration Service (DMS) and Data Transfer Service. Created BigQuery authorized views for row-level security and exposing data to other teams. Experienced with workflow managers like Apache Airflow and Google Cloud Composer. Automated data ingestion pipelines to streamline data flow between GCP and Palantir Foundry Heavily involved in testing BigQuery to understand the best possible way to use the cloud resources. Participated in a collaborative team designing software and developing a BigQuery data warehouse within GCP. Implemented feature engineering techniques within Vertex AI for improved model performance Developed Java-based microservices using Spring Boot for data processing and integration tasks within the GCP environment. Develop and deploy the outcome using spark and Scala code in Hadoop cluster running on GCP. Internally liaised on enterprise GCP migration, database migration, offshore resource Oracle forms work, and timeline deliverables. Automated data extraction from Lawson using GCP Dataflow for seamless integration Install and configure Apache Airflow for S3 bucket and Snowflake data warehouse and created dags to run the Airflow. Worked on GCP components such as Elasticsearch, Dataproc, Dataflow, Pub/Sub, Cloud Functions, Cloud Composer, Secret Manager, and BigQuery. Experienced in Cloud Development using GCP CDK, Pub/Sub, Cloud Functions, Dataflow, Cloud Composer, and Secret Manager. Used Python\'s google-cloud library to configure the services GCP Compute Engine and Cloud Storage.

Education

• Vignan's Foundation for Science, Technology & Research

  Bachelor's degree, Electrical and Electronics Engineering

• Christian Brothers University

  MS Computer Information Systems, Computer and Information Sciences and Support Services