Nakul Ramesh Varma

About Nakul Ramesh Varma

I am a data-focused engineer with a strong track record in large-scale web scraping, data extraction, and real-time digital commerce analytics. Currently working at NielsenIQ, I help power one of the most advanced platforms in the digital shelf space, processing billions of data points daily from global e-commerce platforms.My core expertise lies in developing distributed crawlers, overcoming anti-bot mechanisms, and engineering resilient, scalable scraping systems. I’m passionate about building data pipelines that are not only fast and efficient but also intelligent — using classification, deduplication, and clustering to generate actionable insights.Beyond scraping, I bring a solid foundation in Python programming, Linux systems, network protocols, and both structured and unstructured data stores. I thrive in fast-moving, innovation-driven environments where I can contribute to solving real-world business challenges through technology.

Experience

1. Engineer

   NielsenIQ

   Aug 2024 — Present · Pune, IN

   Key Responsibilities:1. Architect and implement scalable distributed crawlers to extract data from web and mobile platforms.Develop sophisticated anti-bot evasion techniques to bypass web scraping protections (CAPTCHAs, JavaScript rendering, IP bans).2. Design intelligent data capture systems with components for deduplication, classification, clustering, and filtration.3. Build resilient and efficient data pipelines for parsing, cleaning, and storing high-volume datasets.4. Monitor, maintain, and improve crawler systems with automated alerting and anomaly detection mechanisms.5. Collaborate with cross-functional teams to align technical solutions with business goals in the retail analytics domain.6. Contribute to the strategic design of crawling infrastructure, including IP rotation, proxy management, and scheduling optimizations.Technologies Used:Python, Scrapy, Selenium, Playwright, Linux, Docker, Proxies/IP Pools, Google Cloud Storage, ZyteKey Achievements:1. Enabled scalable crawling of over 60 billion data points monthly across multiple geographies.2. Reduced system downtime and improved data freshness with proactive crawler monitoring and self-healing techniques.3. Developed domain-specific anti-crawling countermeasures, increasing coverage of major retail websites.

Education

• CHRIST COLLEGE, IRINJALAKKUDA

  Bachelor's degree

  2018 — 2021

• St. Joseph's University

  Master's degree

  2021 — 2023