Sayantan Bhattacharya
Over 7 years of experience with Spark, AZURE, Databricks, Scala, Hadoop, Advanced SQL,Hive, Query Optimization.
- Role
- Data Engineer at Capgemini
- Location
- Kolkata, WB, IN
- LinkedIn followers
- 500 followers
About Sayantan Bhattacharya
Data Engineering professional with a B.Tech degree in Electronics and Communication…
Experience
Data Engineer
May 2024 — Present · Kolkata, IN
Responsible for designing, developing, and maintaining data pipelines and infrastructure using Microsoft Azure Data Fabric platform, with a particular focus on Azure Synapse Analytics and Spark Data Pools. Working as data architects, data scientists, and closely with other stakeholders to ensure that our data systems are robust, scalable, and optimized for performance.Key Responsibilities:I. Data Pipeline Development: Design, implement, and manage end-to-end data pipelines using Azure Data Factory, Synapse Analytics, and other Azure services.II. Spark Data Pools: Leverage Spark Data Pools within Azure Synapse Analytics to process and analyse large-scale datasets in a distributed Data Integration: Develop solutions to integrate data from various sources (structured, semi-structured, and unstructured) into a centralized data lake or warehouse.IV. Data Modelling: Collaborate with data architects to create efficient data models that support business analytics and reporting needs.V. Performance Optimization: Monitor and optimize the performance of data pipelines, ensuring efficient data processing and storage, particularly when using Spark in a big data environment.VI. Azure Synapse Analytics: Utilize Azure Synapse Analytics for complex data integration, big data processing, and advanced analytics solutions.VII. Data Governance: Implement best practices for data management, including data quality, metadata management, and security Collaboration: Work closely with cross-functional teams including data scientists, analysts, and business stakeholders to understand data needs and deliver solutions that meet business objectives.IX. Automation: Develop and maintain scripts and tools to automate data processing tasks and improve workflow efficiency, especially in Spark and Synapse environments.X. Documentation: Create and maintain comprehensive documentation for data architecture, processes, and systems.
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.