Sandeep S

WORK HISTORY

EDUCATION

ABOUT SANDEEP S

Im a Senior Data Engineer with strong experience in designing, building, and optimizing scalable data platforms across AWS, Azure, and Google Cloud Platform (GCP). I specialize in developing end-to-end data pipelines, modern data architectures, and real-time streaming solutions that transform raw data into actionable business insights.I have hands-on expertise in ETL and ELT development using Apache Spark, PySpark, and Spark SQL, along with cloud-native tools such as Azure Data Factory, AWS Glue, and Apache Airflow. I have worked extensively with large-scale structured and semi-structured datasets, ensuring high performance, reliability, and data quality across data pipelines.My core strength lies in building data lakes and lakehouse architectures using Azure Databricks and AWS EMR. I am experienced in optimizing Spark jobs through partitioning, caching, and performance tuning, enabling efficient processing of high-volume data. I also have strong experience in real-time data streaming using Apache Kafka, Spark Streaming, and Google Cloud Dataflow, supporting event-driven architectures and near real-time analytics.I bring solid expertise in data modeling and data warehousing, including designing star and snowflake schemas, building fact and dimension tables, and working with platforms such as Snowflake, Amazon Redshift, Azure Synapse Analytics, and BigQuery. I am highly proficient in SQL and have optimized complex queries to improve performance and support analytics workloads.In addition, I have experience with DevOps and automation tools such as Jenkins, Azure DevOps, and GitHub Actions, along with containerization technologies like Docker and Kubernetes. I have implemented CI/CD pipelines to streamline deployments and ensure scalability and consistency across environments.I also prioritize data security and governance, working with tools like AWS IAM, Azure Key Vault, and monitoring systems such as AWS CloudWatch and Azure Monitor to ensure compliance and system reliability.I am passionate about solving complex data challenges, improving pipeline performance, and building scalable, cloud-based data solutions. I enjoy collaborating with cross-functional teams and continuously learning new technologies in the evolving data ecosystem.I am open to connecting with professionals and exploring opportunities in data engineering, big data, and cloud technologies. Feel free to reach out to discuss potential collaborations or opportunities.