Kalpesh Agrawal
Big Data & Abinitio Lead Developer
- Role
- Big Data Lead Developer and Consultant at Travelers
- Location
- Hartford, CT, US
- LinkedIn followers
- 500 followers
About Kalpesh Agrawal
8 years of professional experience in building data warehousing and business intelligence applications. Expertise in Big data and analytics tools. Currently working as Lead developer & consultant at Travelers BIG Data COE (Center of Excellence) team.Big Data tools: Hive, Spark, Spark SQL, Pyspark, Atlas, Solr, Pig, Sqoop, Nifi,HBase, Oozie, Ambari, Zeppelin,Hcatalog, Kerberos, yarn, Map Reduce,Tez. ETL tools: Ab Initio,SAS, R studio, PIG, Microsoft SSIS.Reporting tools: Qlikview, Tableau.Analytical techniques: Linear regression,Logistic regression, Annova, Gradient Descent, Test of hypothesis,Deep learning.RDMBS: SQL Server, Teradata, Oracle, DB2, MS Access.Programming Lang: python,shell scriptin,awk,pearl.
Experience
Big Data Lead Developer and Consultant
Dec 2016 — Present · Hartford, CT, US
Working on enterprise level big data POCs• Working frequently with Hortonworks to resolve critical technical issues and implement fixes for bugs identified in hadoop stack tools• Creating best practice guidelines and design document for hadoop stack developers across travelers• Providing technical solution and resolving issues of hadoop developers from different line of business in travelers• Developing generic utilities using big data tools (spark, hive, solr, atlas, oozie etc.) to automate manual task and ease work for technical and business users• Working on integration of big data tools like Hbase-hive and Hive-Solr integration• Created custom data type in Apache Atlas to store and merge business metadata to technical metadata• Designed and created Solr collection and indexed data from multiple sources• Defined and tuned Solr schema for efficient search• Built tool to collect and store hadoop cluster level usage and job execution metrics• Developed scripts to make rest api calls to resource manager and get cluster level metrics and parse data to hive tables• Built interactive Qlikview dashboards to broadcast cluster usage and metrics details to senior management• Mentoring and conducting hadoop and big data technologies related training sessions for new team members• Leading Big Data COE team in travelers• Managing a team of 2-3 offshore developers
Education
Jawahar Navodaya Vidyalaya, Tuljapur
HSC, Science
1996 — 2002
Shivaji University
Bachelor of Engineering - BE, Biotechnology
2004 — 2008
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.