Mahesh M

About Mahesh M

I am a Senior Data Scientist with 6+ years Experience, with 4 years focused on Gen AI…

Experience

1. Senior Data Scientist

   eBOS

   Apr 2022 — Present · Chicago, IL, US

   In this project, I developed data storage strategies and ETL processes using Azure technologies, including Azure Blob Storage, Azure Data Lake, and Azure Data Factory, to create cost-effective and scalable solutions for data retention and processing. I built Spark jobs and PySpark scripts for efficient data loading, transformation, and processing, ensuring data confidentiality and compliance with security regulations. I also optimized and automated data pipelines for structured and semi-structured data, utilizing tools like Apache NiFi and Kafka for real-time ingestion and data flow management.I implemented advanced data processing techniques using Hadoop, Hive, and Informatica to support data warehousing, lakes, and marts. I also used Kafka for message routing and real-time analysis. Additionally, I designed workflows using Apache Airflow for improved pipeline reliability and configured YARN clusters to enhance system performance. I managed NoSQL databases like MongoDB and Cassandra, ensuring high availability and efficient handling of large-scale data loads.In the data visualization space, I developed interactive Power BI reports and dashboards, converting existing reports into dynamic visualizations while ensuring real-time updates using the Power BI gateway. I integrated CI/CD tools with GitHub to automate build, test, and deployment pipelines, enhancing workflow efficiency and supporting agile development methodologies. I utilized various tools, including Tableau, Power BI, and advanced features of T-SQL, to create effective data models and reports that aligned with business requirements.