Manish Kumar Pandya

About Manish Kumar Pandya

Expertise in Big data technology Spark, Kafka, Flume, Yarn, Hive, Parquet, Knox, Ranger…

Experience

1. Associate Director of Big Data

   Synechron

   Jun 2021 — Present · Mumbai, IN

   Key Result Areas:• Facilitating discussions to elicit & understand business needs, requirements & priorities, driving prioritization and end-to-end project management with senior stakeholders and providing insights for design & business process.• Interfacing with infrastructure, network, database, application, and business intelligence teams to guarantee high data quality and ensure business applications availability and performing within agreed service levels.• Implemented and configured Data Mesh and Iceberg v2 on CDP Public cloud.• Build an upgrade roadmap plan for various Data Ingestion pipelines for migration on Iceberg and DBT technology.• Build an upgrade roadmap plan and implement major and minor CDP Public cloud platform for BCP (Business continuous plan).• To migrate all Data Services like Impala,Hive,spark etc from On premise to Auto workload using EKS cluster• To build/support various types of data like text,csv,json,xml,parquet and delta file etc. for data ingestion project. • Implemented and configured CDP Public Cloud,Data Lake Platform and DeltaLakeHouse on AWS Cloud for various internal project.• Implemented auto TLS encryption to secure data in transit within Cloudera clusters, enhancing overall security posture.• Implemented and configured User management in CDP Public cloud with Microsoft Azure AD IAM. • Implemented and configured CDP Data Hub Clusters for various internal projects as per their use cases.• Implemented and configured CDP Data WareHouse(Impala/Hive),CDP Data Flow on EKS Kubernetes cluster with Autoscaling.• Implemented and configured Data Catalog for Data profiling, Asset Collection,Lineage and impact, Security and Classification.• Implemented and configured Ranger for Authorisation, Audit on various CDP/Big Data Tools,• Implemented Monitoring solution using Prometheus and Grafana to monitor the health of CDP Data Lake,Data Hub Cluster,Nifi,Kakfa and Data Service as per agreed KPI’s for various internal projects.

Skills

• Nfs
• Raid
• Hdfs
• Vmware Infrastructure
• Ldap
• Hdp
• Vmware Server
• Linux System Administration
• Datacenter Virtualization
• Dns
• Vmware Esx
• Apache
• High Availability
• Vmware
• Hadoop
• Itil Process
• Ftp
• Rhel
• Lvm
• Vendor Management
• Research
• Customer Service
• Red Hat Linux
• Rhce
• Hbase
• Load Balancing Cluster
• Strategic Planning
• Squid
• English
• Performance Tuning
• Cluster
• Hive
• Editing