Krishna Chaitanya
Data Engineer | Master\'s in Information Technology
- Role
- Data Engineer at Td Ameritrade
- Location
- Memphis, TN, US
- LinkedIn followers
- 500 followers
About Krishna Chaitanya
I am a results-driven MSBI / Power BI Developer with 5+ years of experience building…
Experience
Data Engineer
Jan 2024 — Present · CT, US
Managed data transfer between AWS compute and storage services utilizing AWS Data Pipeline to streamline data workflows.• Developed Scala scripts and UDFs in Spark for data aggregation, querying, and storage into S3 buckets, leveraging both Data Frames/SQL and RDDs for efficient data handling.• Executed ETL processes with ingested data by developing and running Scala programs in Apache Spark, optimizing data transformation and loading.• Configured AWS Data Pipeline for efficient data transfers from S3 to Redshift, enhancing data warehousing processes.• using Apache Flink to handle high-throughput event streams, enabling real-time analytics and decision-making for mission-critical business processes.• Designed and implemented data ingestion processes into Cosmos DB from multiple sources, streamlining data integration and accessibility.• Extracted, aggregated, and merged Adobe data within AWS Glue using PySpark, demonstrating proficiency in complex data processing.• Implemented AWS Lambda functions for event-driven script execution in response to Amazon DynamoDB table changes, S3 bucket events, and HTTP requests via Amazon API Gateway, optimizing system responsiveness and scalability.• Integrated Apache Flink with Kafka for real-time event ingestion and processing, enabling the company to react to live events and perform near real-time analytics.• Implemented complex mathematical functions and statistical operations using NumPy.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.