Sandeep P

Senior Data Engineer @SiriusXM

TX, US

Implemented medium to large-scale BI solutions on Azure using services like Azure Data Lake, Data Factory, and SQL DW. Led migration efforts for on-premises data to Azure, utilizing Azure Data Factory. Developed custom data movement pipelines and ad-hoc analysis solutions with Azure Data Lake Analytics and HD Insights. Designed and executed data conversion migration to Snowflake database.Created Power BI reports and dashboards for effective data visualization, connecting to various sources. Implemented security measures for Hadoop clusters and developed Kafka data pipelines for real-time streaming. Managed data ingestion into Azure SQL Data Warehouse and optimized performance using PolyBase. Utilized Apache Spark for data validation, standardization, and encryption.Developed purging scripts, Tableau dashboards, and comprehensive analysis reports using Python. Implemented Apache Airflow for Data Pipelines and worked on data migration using SQL, Azure Storage, and SSIS. Converted Hive/SQL queries into Spark transformations and designed DAGs for automating ETL pipelines. Overall, experienced in end-to-end data solutions, from migration to advanced analytics.