Sohail Baig
Data Engineer with B.sc Computer Science a passionate towards Data/AI/ML
- Role
- Data Engineer at Bitwise
- Location
- Aurangabad, MH, IN
- LinkedIn followers
- 500 followers
About Sohail Baig
A results-driven IT professional with over 4+ years of experience, including 3.8+ years specializing in Big Data Development, ETL Development, and Data Engineering. Proficient in leveraging cloud technologies, particularly within Microsoft Azure, including Azure Data Factory, Azure Databricks, and Azure Synapse. Experienced in building, managing, and optimizing data pipelines, batch processing workflows, and data integration solutions. Adept in using tools like Hadoop, Hive, HBase, PySpark, and Spark SQL for processing large datasets, both structured and unstructured. Strong expertise in data extraction, transformation, and integration, including working with Delta Tables and cloud storage solutions like Azure Blob and ADLS Gen2. Expertise in utilizing Hive optimization techniques and working with a variety of file formats, including CSV and Parquet. Demonstrated ability to rapidly learn new technologies and concepts to drive project success.
Experience
Data Engineer
Aug 2024 — Present · IN
Results-driven IT professional with over 4.4+ years of experience, including 3.10+ years specializing in Big Data Development, ETL Development, and Data Engineering. Adept at designing, implementing, and optimizing scalable data solutions using Microsoft Azure cloud technologies, including Azure Data Factory (ADF), Azure Databricks, and Azure Synapse Analytics.Proficient in developing and managing end-to-end data pipelines, integrating structured and unstructured data from diverse sources, and ensuring efficient data processing using PySpark, Spark SQL, Hadoop, Hive, and HBase. Experienced in working with Delta Tables, partitioning strategies, and performance tuning techniques to enhance data retrieval and storage efficiency. Skilled in handling large-scale batch processing workflows, implementing ETL/ELT processes, and leveraging cloud storage solutions such as Azure Blob Storage and ADLS Gen2.Strong expertise in data modeling, transformation, and integration while optimizing Hive queries and Spark jobs for improved performance. Well-versed in working with various file formats, including CSV, Parquet, and Avro, and implementing best practices for data governance, security, and compliance.A highly adaptable and quick learner, continuously upgrading skills to stay ahead in emerging data technologies. Proven ability to collaborate with cross-functional teams, align data strategies with business objectives, and drive high-impact data-driven solutions that enhance operational efficiency and decision-making.Top 5 key skills:Azure Data Factory,Azure Databricks,Pyspark,Spark,MySQL,Python,Hadoop,Hive,HBase
Education
Maulana Azad College of Arts,commerce & Science, Roza Baug
Bachelor of Science - BS, Computer Science
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.