Chaithanya Thota
NetWork Engineer
- Role
- Data Engineer at Capital One
- Location
- North Plains, OR, US
- LinkedIn followers
- 500 followers
About Chaithanya Thota
I am a Network and Systems Engineer with designing and managing secure, scalable, and…
Experience
Data Engineer
Apr 2023 — Present · US
Worked on a cloud-based data engineering project focused on building scalable data pipelines and enabling advanced analytics using big data technologies. The project involved ingesting, transforming, and analyzing large datasets from multiple sources to support business intelligence and reporting needs across enterprise systems.Key Responsibilities & Tools Used:Gathered and analyzed business requirements to design efficient data workflows.Developed Spark applications (RDD, DataFrames, Datasets) and custom UDFs for data transformation and cleansing.Built ETL pipelines using Apache Spark, Scala, Python, and Apache Airflow for orchestrating batch jobs across HDFS, Hive, and Oracle.Integrated Azure Data Factory with Databricks, Blob Storage, Azure Data Lake, and Azure SQL DW for end-to-end data movement and transformation.Implemented data processing and streaming workflows using Kafka, Apache Iceberg, and Spark Streaming.Designed and optimized complex Spark SQL queries and performed data profiling using SQL on Oracle, Netezza, and Teradata.Built Tableau dashboards for internal and external stakeholders to derive insights from big data platforms.Applied data compaction strategies in Apache Iceberg for large-scale dataset optimization.Worked with HBase, MongoDB, Docker, and Git for storage, containerization, and version control.Followed Agile methodology, contributing to sprint planning, development, and UAT.
Education
Badruka College Commerce Arts
Bachelor's degree, Business/Commerce, General
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.