Venkatesh M

About Venkatesh M

About 8 years of experience in the design and implementation of client-server technologies, web applications, enterprise application development, and web programming using Java and big data technologies.• Have five or more years of in-depth experience working as a data engineer, big data developer, and Hadoop developer. Knowledge of the Hadoop ecosystem and architecture, including HDFS, MapReduce, Pig, Hive, Sqoop Flume, and Oozie.• A thorough understanding of the MRV1 and YARN architecture, as well as Job Tracker, Task Tracker, Name Node, and Data Node daemons in Hadoop.• Possesses a strong technical skill set, including proficiency in languages such as Python, Java, and SQL, as well as expertise in data storage systems like Hadoop, AWS S3, and Cassandra.• Demonstrates extensive experience in designing, implementing, and managing ETL processes, as well as integrating disparate data sources and creating data pipelines.• Has a track record of developing and optimizing data architectures to enable scalable, reliable, and secure data management.• Is skilled in working with both structured and unstructured data, including experience with NoSQL databases and data warehousing tools such as Redshift and Snowflake.• Has experience working with data visualization tools such as Tableau, Looker, or Power BI, and has a deep understanding of how to transform data into insights and actionable business intelligence.• Possesses excellent communication and collaboration skills, with a proven ability to work cross-functionally and lead projects involving multiple stakeholders.• Is committed to staying up-to-date with emerging technologies and trends in the data engineering space, and has a passion for continuous learning and development.• Knowledge of how to set up, administer, support, and monitor Hadoop clusters utilizing a variety of distributions, including Apache Hadoop, Cloudera Hortonworks, and several cloud providers, like Amazon and GCP.• Knowledge in installing and configuring the MapReduce, HDFS, Hive, Pig, Sqoop, Flume, Oozie, and Zookeeper components of the Hadoop stack. Custom writing expertise

Experience

1. Senior Data Engineer

   American Express

   Jun 2021 — Present · NY, US

   I took part in all stages of data mining, including data collection, data cleaning, model development, model validation, visual analysis, and gap analysis. Developed different Machine Learning Algorithms in Python using Pandas, NumPy, Seaborn, SciPy, Matplotlib, and Sci-kit-Learn.Used a variety of data pre-processing techniques to work with unstructured, structured, and imbalanced data as well as data imputations. Used Scikit-learn pre-processing to participate in features engineering tasks such feature intersection generation, feature normalisation, and label encoding.Using machine learning methods such linear regression, classification, multivariate regression, naive bayes, random forest, k-means clustering, kNN, PCA, and regularisation, predictive models were developed and put into practise.Construct a highly scalable NLP system for handling a lot of unstructured data. The emotion about the company was discovered using Text Mining and NLP approaches. We could classify it effectively with RNN.

Education

• Osmania University

  Bachelor of Engineering - BE, Computer Science