Sundeep Kumar G.
Sr. Data Engineer @Abercrombie & Fitch Co with 8 Years of Expertise in Big Data Tools and Cloud Services #AWS #Snowflake #Spark #Scala #python #PySpark #Java
- Role
- Senior Data Engineer at Abercrombie & Fitch Co.
- Location
- Richmond, VA, US
- LinkedIn followers
- 500 followers
About Sundeep Kumar G.
8 years of experience as a software developer, with strong emphasis on data engineering,…
Experience
Senior Data Engineer
Aug 2023 — Present · New Albany, OH, US
Worked on AWS resources for ETL jobs using PySpark, Airflow for data orchestration and snowflake WH for building queries based on requirement.• Involved in Created AWS glue jobs, data pipelines and parallelization tasks based on performance and time complexity issues using Multithreading in python.• Involved in loading data from different sources to Aws and Snowflake via SFTP sever using Python script.• Worked on loading transformed data from S3 to Redshift using AWS data pipelines.• Extensive experience in managing and configuring AWS EMR clusters for large-scale data processing and analytics.• Proficient in leveraging EMR to perform data transformations, aggregations, and complex analytics tasks using frameworks such as Hadoop and Spark.• Skilled in optimizing EMR clusters for enhanced performance and cost efficiency through cluster sizing, instance type selection, and data partitioning.• Worked both with batch and real time streaming data sources.• Writing custom Kafka consumer code and modifying existing producer code in Python to push data to Spark-streaming jobs.• Using the spark framework Enhanced and optimized product Spark code to aggregate, group, and run data mining tasks.• Involved in writing python scripts for loading customer data and online shopping event data from various external sources like SFTP servers, DB2 to S3 buckets.• Involved in migrating existing Teradata Datawarehouse to AWS S3 based data lakes.• Involve developing scripts and workflows using tools such as Python, Shell, and Teradata BTEQ.• Responsible for testing and troubleshooting ETL processes to ensure they are running smoothly.• Performed ETL process with Python-SQL Server pipelines/framework to perform data analytics and visualization in Python, NumPy, SciPy, Pandas.• Analyzed the SQL scripts and designed the solution to implement using Scala.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.