Aditya Shet

About Aditya Shet

enabling organizations to transform raw data into scalable, AI-ready insights. My current focus is on Architecting solutions with BigQuery, Vertex AI, Dataflow, and LLM automation (Gemini API), where I build end-to-end pipelines that power analytics, real-time decision-making, and LLMOps workflows- on AWS, leveraging Kafka, EMR, and PySpark to process large-scale datasets. This experience gave me a strong foundation in batch + streaming architectures, data lake design, and secure handling of sensitive information through PII masking and governance frameworks, I ensure best practices in data pipeline design, cost optimization, and performance tuning, while mentoring peers to grow their technical expertise. My work blends technical depth with strategic leadership, ensuring that every data solution is reliable, efficient, and business-aligned: Google Cloud (BigQuery, Cloud Composer, Vertex AI, Pub/Sub, Cloud Storage, Cloud Run), AWS (EMR, Kafka, S3, Hadoop, DeltaLake): PySpark, Spark SQL, Airflow, SQL, DBT, GitHub/GitHub Actions : Data Lakes, Data Warehousing, Real-Time Pipelines, ETL/ELT, Governance & LLMOps : Team Mentoring, Project Ownership, Cross-Functional CollaborationI am always interested in discussing new technologies, sharing knowledge, and collaborating with professionals passionate about building impactful products.

Experience

1. Senior Solutions Architect Gcp Cloud Architect & Llmops

   Tata Consultancy Services

   Mar 2025 — Present · Bengaluru, IN

   Engineered a cross-cloud migration strategy to ingest petabyte-scale telemetry from AWS (S3) to GCP (GCS), architecting a Medallion Architecture (Bronze/Silver/Gold) within BigQuery to ensure data lineage and high-fidelity analytics: Developed a unified orchestration framework using Cloud Composer and dbt, managing complex dependency graphs for 50+ production pipelines with automated backfilling and state-tracking via custom metadata logging tables: Built high-throughput bi-directional messaging systems using Pub/Sub and Cloud Run, acting as both publisher and subscriber to facilitate real-time feature engineering for live model inference: Orchestrated end-to-end ML lifecycles by integrating Kubeflow with Vertex AI Pipelines, enabling seamless model preprocessing, training, and deployment for data science teams. & : Architected enterprise-grade GenAI solutions by integrating the Gemini API (Vertex AI) into production workflows; engineered and deployed internal GenAI-powered web applications for business intelligence and developer-facing automation: Engineered a proactive data governance framework using Dynamic Programming to traverse deeply nested JSON/Avro schemas, automatically identifying and masking PII/Sensitive data before BigQuery persistence./ & : Standardized deployment via GitHub Actions, implementing multi-stage workflows for automated unit/integration testing, and blue-green deployments across distributed environments: Optimized ingestion gateways for heterogeneous sources including APIs, SFTP, Oracle, and Amazon S3 into a centralized GCS bucket.

Education

• DKTE'S Textile and Engineering Institute - An Empowered Autonomous Institute

  Bachelor of Technology - BTech