Yashi Shukla

Data Engineer | Social Impact | AI | GenAI

Role: Data Engineer at IDinsight
Location: New Delhi, DL, IN
LinkedIn followers: 500 followers

Information TechnologyView LinkedIn profile

About Yashi Shukla

6 years of experience in data science, engineering, and analytics across government…

Experience

Data Engineer
IDinsight
Jan 2022 — Present · Delhi, IN
Orchestrated large-scale data ingestion, transformation, and dashboard development initiatives for organizations in the development and non-profit sectors. Worked on 10+ workstreams delivering scalable and data-driven solutions. Cloud Architecture & Scale: Designed Redshift-based architecture for Niti Aayog’s NDAP, ingesting terabytes of clickstream data, and built a serverless data quality tool with <$20 monthly cost. Developed robust, scalable solutions using AWS (Lambda, Step Functions, Redshift, Athena, Glue, Fargate, ECS) and Azure technologies. AI & LLM Solutions: Deployed GCP pipeline integrating Gemini LLM to process 10M+ PDFs, automating OCR, classification, and metadata generation; developed transliteration module for 8 Indian languages. Automation & MLOps: Refactored Airflow DAGs into modular workflows; built dbt + Airbyte automated reporting on GCP, reducing finance reporting time by 30%; engineered Cypress regression test suite reducing QA by 30%. Additionally, designed and implemented automated workflows—including API integrations and web scraping—to reduce manual processing time significantly. Dashboards & Decision Systems: Delivered Superset, Looker Studio, Power BI, and Athena dashboards for UNICEF Malawi, Indus Action, and state governments, supporting policy and monitoring. Backend Development & Testing: Built and optimized backend APIs with Python (Pandas, NumPy, FastAPI) and implemented automated testing frameworks to ensure product reliability. DevOps & Infrastructure Management: Leveraged containerization (Docker) and infrastructure-as-code tools (Terraform, AWS SAM, GitHub Actions) to streamline deployments and enhance system efficiency.

Contact information

Verified work emails, personal emails, and direct dials — unlocked in one click.

Work email
•••••@idinsight.org
Personal email
•••••@•••••.com
Direct dial
+•• ••• ••• ••••

Unlock verified contacts

Free to start · No credit card required

Current company

IDinsight

Research Services · 452 employees

IDinsight helps leaders combat poverty worldwide by designing, deploying and promoting evidence-generating tools. We tailor the best methodologies to partner needs and constraints to fuse evidence with action. We serve governments, NGOs, foundations and social businesses across Africa and Asia in all major program areas including health, education, agriculture, livelihoods, finance, energy and governance. Our approach rests on four pillars: 1. Rigorous: We develop and use a wide-range of cutting edge data and evidence tools, including experimental evaluations, monitoring systems, data analytics and visualization, process evaluations, machine learning and more. 2. Cost-effective: Every dollar spent is justified by expected impact. If funds could be better used in another way, we say so. 3. Timely: Actionable information is delivered in time for client decision-making deadlines. 4. Demand-driven: We deploy solutions tailored to partner contexts and needs with no competing agendas. Our diverse, growing team of over 200 outstanding colleagues operate in nearly two dozen countries around the world. Learn more at www.IDinsight.org.

idinsight.org

Find verified contacts for anyone on LinkedIn

Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.

Start for free See how it works

Free plan included · No credit card required

This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.