Abdul Samad Shaji
Looking for Sr Data Engineer positions I Spark SQL I Azure database SQL I AWS I Hadoop I Scala I Kubernetes I Python I Kafka
- Role
- Sr Data Engineer at Exabeam
- Location
- New York, NY, US
- LinkedIn followers
- 500 followers
About Abdul Samad Shaji
Above 9 years experience in handling large data sets of both Structured and Unstructured data. Skilled in Data Acquisition, Data Validation, Predictive modeling, Statistical modeling, Data modeling, and Data Visualization. Proficient in using R, Python, SAS, and Apache Spark, including Big Data technologies such as Hadoop, Hive, and Pig
Experience
Sr Data Engineer
Nov 2022 — Present · Foster City, CA, US
Develop Spark applications using PySpark and Spark-SQL to extract, transform, and aggregate data from multiple file formats-Implement Spark streaming applications to read raw packet data from Kafka topics, convert it to JSON, and store it back in Kafka for future use cases-Optimize ETL workflows, ensuring optimal performance and efficiency, and conduct monitoring and performance tuning-Manage Kubernetes clusters and orchestrate containerized applications to ensure high availability, scalability, and efficient resource utilization-Build and optimize data processing pipelines that extract, transform, and load data from various sources into data storage systems, including Amazon S3, Redshift, RDS, and DynamoDB-Design and develop data pipelines and ETL workflows in Python for efficient data extraction, transformation, and loading-Implement scalable and cost-effective data solutions on AWS, following best practices and security standards-Automate data ingestion and processing tasks using Python scripts, resulting in significant time and resource savings.
Education
Osmania University
Bachelor's degree, Computer Science
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.