Praneeth Reddy Polepally
Senior Data Engineer at Matson, Inc.
- Role
- Data Engineer at Matson, Inc.
- Location
- Fremont, CA, US
- LinkedIn followers
- 500 followers
About Praneeth Reddy Polepally
Proficient with AWS (Analytics, DevOps), Hadoop, Open source big data ecosystems. • Adequate knowledge in technologies like RDBMS, ETL (talend), BI (tableau). • Experience in building an end to end analytics platform. • Experienced in cluster administration like hadoop installation, recovery and maintenance. • Good understanding of application hardware and architecture. • Good at management, communication and interpersonal skills.
Experience
Data Engineer
Apr 2017 — Present · Oakland, CA, US
Matson, Inc. is a public shipping company, founded in 1882. With such a long history, Matson IT holds 135 years of data. To make use of such volume of data for analytics, machine learning, we are working on building a data platform (MDAAS: Matson Data & Analytics as Service) using AWS cloud services and open source applications. • POC on comparing Cloudera, Hortonworks, and EMR platforms. • Designed and built MDAAS platform, s3 data lake. • Used lambda functions, cloud watch streams, kinesis and talend to load data to data lake. • Ingested Cloud Trail, AWS Conf, VPC flow, billing, ELB, APP, WEB, AWS RDS, DB, DC, multiple security logs to data lake to perform analytics. • Developed ETL jobs to use EMR for operations and S3 for data storage. • Used sqoop to efficiently transfer data between RDBMS and other platforms to AWS EMR Cluster. • Talend, AWS Glue and oozie workflow engine are used to schedule our ETL workflows which contains transformations, data import & export actions, hive actions and shell scripting actions. • Created Athena-Hive tables for data on top of S3 with appropriate static and dynamic partitions. • Developed hive scripts for transformations and aggregations on data for analytics. • Performance tuning on Hive Queries. • Built a common data catalog in AWS Glue to use by EMR, Athena, glue jobs and triggers. • Worked on authentication and authorization process for MDAAS using AWS Simple AD and Apache Ranger hive policies. • POC’s for visualization tool selection (Tableau/ Qlik/ MS Power BI). • Implemented tableau online and centralized tableau bridge servers. • Provided tableau training for internal team and power users as part of BRIO migration. • Developed tableau reports for multiple new projects and migration. • Used tabcmd & Jenkins for scheduling tableau refresh extracts and tableau exports. • Used AWS ECS, EKS to productionize analytics processes in a containerized environment. • POC on Alation for enterprise data catalog, governance etc
Education
Kakatiya University
Bachelor’s Degree, Electronics & Instrumentation, 3.3
2007 — 2011
Northwest Missouri State University
Master of Science (M.S.), applied computer science, 3.3
2013 — 2014
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.