Nilesh Rajput

About Nilesh Rajput

AWS Data EngineerI am an AWS Certified Data Engineer with over 6 years of experience in designing, building, and optimizing scalable cloud and big data solutions. My expertise spans AWS, Azure, Snowflake, Python, and modern data engineering frameworks. I specialize in end-to-end data pipelines, cloud migrations, automation, and building reliable data ecosystems that support business intelligence and analytics.In my current role, I have led a team of 5–6 engineers, guiding them on technical implementations, reviewing deliverables, ensuring best practices, and successfully handling all assigned tasks and deadlines. I enjoy mentoring teams, solving complex problems, and driving high-quality outcomes. Key Skills & ExperienceCloud & Data Engineering: Strong hands-on experience with AWS (S3, Lambda, Glue, EMR, Redshift, Step Functions, Kinesis, EC2, IAM, DynamoDB, API Gateway) and Azure (ADLS, ADF, Synapse, HDInsight).Big Data & ETL Pipelines: Skilled in PySpark, Python, Delta Lake, Snowflake, and Databricks for building scalable batch and streaming pipelines.Migration Projects: Successfully led the migration of on-prem systems to Snowpark, rewriting and optimizing Scala, Python, and Java-based transformations in Snowflake.Automation & Scripting: Developed automated data extraction and scraping scripts using Python, BeautifulSoup, and Requests—reducing manual effort by up to 30%.Data Processing: Experience parsing, cleaning, and structuring HTML/web datasets with 95%+ accuracy.Azure DevOps: Built and managed automated CI/CD pipelines for data workflows.Big Data Tools: Practical exposure to Hive, HDFS, Oozie, and MapReduce for distributed data processing.Reporting & Dashboards: Supported creation of analytical datasets, reports, and dashboards across BI teams.Cloud Support & Deployment: Background in cloud support, troubleshooting, deployments, and customer assistance.Leadership & Collaboration: Strong communicator who works independently and collaboratively—drives clarity, mentors team members, and ensures high-quality delivery.Fast Learner: Continuously learning and adapting to new cloud, data, and automation technologies.I am passionate about leveraging cloud and data engineering to solve real business problems, create scalable solutions, and help teams deliver their best work.

Experience

1. Consultant

   Deloitte

   Sep 2022 — Present · Pune, IN

   Designed and implemented end-to-end AWS data pipelines using S3 (Bronze/Silver/Gold), Databricks Autoloader, PySpark, and Redshift, improving data flow efficiency by 45%.• Built a multi-layered data lake architecture (Bronze for raw, Silver for transformed, Gold for business-ready data), improving accessibility by 50%.• Developed real-time Databricks Autoloader ingestion framework with schema evolution and drift handling, ensuring 99% reliability.• Implemented PySpark transformations for validation, cleansing, and enrichment, producing consistent, analytics-ready datasets.• Optimized Redshift loading pipelines, accelerating query performance by 30% for downstream analytics.• Automated ingestion, transformation, and loading pipelines, reducing manual intervention by 60%.• Ensured robust error handling, data quality checks, and compliance, maintaining 99% data accuracy.

Education

• Kamla Education Society's Pratibha College of Commerce & Computer Studies

  Master's degree, Computer Science

  2016 — 2018

• Savitribai Phule Pune University

  Bachelor of Computer Science, Information Technology