Dhananjay Gupta
Data Engineer @FOOTASYLUM
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @FOOTASYLUM
Manchester, GB
Led the Unity Catalog migration of 50+ Delta tables from legacy mount-based storage across dev and prod, implementing environment aware catalog resolution in PySpark to eliminate recurring deployment failures and reduce environment-related incidents by an estimated 70%. Diagnosed and resolved a critical Unicode encoding issue affecting Nike and Converse product codes across ~200K SKU records in the Silver layer, applying a targeted PySpark cleansing function that restored downstream inventory accuracy and unblocked stalled reporting pipelines. Engineered production-grade Source-to-Raw and Raw-to-Staging pipelines in Azure Databricks for multiple Business Central entities, processing 100K+ purchase records per cycle with soft-delete detection, SCD logic, staging views, and Unity Catalog integration, reducing source-to-mart data latency by an estimated 40%. Designed and deployed an agentic multi-agent analytics app on Databricks, combining Databricks Genie with Claude Sonnet to enable a conversational interface over 3M+ customer and sales records, allowing C-suite stakeholders to self-serve live Gold-tier data and reducing analyst turnaround time by an estimated 60%. Built a production Power BI monitoring dashboard with the BI team, integrating Azure Log Analytics and KQL to track health, failure rates, and SLA breaches across 600+ nightly ADF pipelines, reducing analyst investigation time by an estimated 50% and cutting ad-hoc support requests.
EDUCATION
SRM IST Chennai
BTech - Bachelor of Technology, Computer Science
University of Glasgow
Master's degree, Computer Science
ABOUT DHANANJAY GUPTA
Hello, I\'m Dhananjay, a passionate and results-driven Data Engineer, currently based in London, UK. I\'m actively seeking Data Engineer or Azure Data Engineer opportunities across the UK, Europe, and India — eager to contribute my cloud data engineering and Generative AI experience to forward-thinking, high-impact teams! About Me:I specialize in designing and optimizing scalable ETL/ELT pipelines, implementing cloud-native solutions, and integrating Generative AI for enhanced data interaction. My work is driven by solving complex data problems, improving system efficiency, and enabling smarter business decisions. I thrive in fast-paced Agile environments and enjoy collaborating across teams to drive measurable impact. Professional Experience:Across roles at Cube & Mart Services, Open Insight, and Mastech Infotrellis, I have:Designed and deployed retrieval-augmented generation (RAG) architectures using LangChain, OpenAI, and Qdrant for internal document Q&A systems.Engineered 40+ KPIs over 40M+ records with PySpark in Azure Databricks, boosting revenue by £1M through actionable insights.Migrated over 150M records across global regions with 99.9% accuracy, improving data quality and reporting across teams.Optimized ETL runtimes from 35+ to under 12 hours, saving compute costs and enabling near real-time analytics.Automated deployments with GitLab CI/CD, reducing manual tasks by 50%. Key Skills & Technologies:Data Engineering: ETL/ELT, PySpark, SQL, Data PipelinesCloud & Big Data: Azure Data Factory, Azure Databricks, Azure SQL, Delta Lake, Microsoft AzureGenAI & LLMs: LangChain, OpenAI, Qdrant, Prompt EngineeringDevOps & Automation: GitLab CI/CD, Git, Docker, Agile/Scrum, Azure DevOpsAnalytics & Reporting: Power BI, SQL Server, JSON SchemasFamiliarity: Snowflake, FastAPI, Hugging Face Transformers, Scikit-learn Key Achievements:Reduced Spark job runtimes by 70%, enabling faster analytics deliveryImplemented RAG-based chatbot system serving 500+ usersIncreased data migration accuracy from 80% to 99.9%Delivered KPI dashboards driving £1M+ revenue growth for business teams Open to Work:I\'m actively exploring Data Engineer and Azure Data Engineer roles in UK, Europe, and India. If you’re hiring or know of exciting opportunities, I’d love to connect!Let’s build powerful data systems that deliver real-world value.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.