Get your Emails warmed up and ready to send
Build sender reputation and reach more inboxes with automated email warmup.
Warm up your emails
Starting at $49/month
Digamber Kalivemula
Data Engineer @Cisco
Signup · Get unlimited contacts
WORK HISTORY
Data Engineer @Cisco
AL
Collaborated with stakeholders to design and develop a unified architecture for Retail data, resulting in the creation of a centralized Data Lake on Azure Cloud.• Engineered Spark applications using Azure Databricks for efficient data extraction, transformation, and aggregation from diverse systems, significantly improving data processing capabilities and storage on Azure Data Lake.• Designed and implemented PySpark applications to integrate and process data from various sources including FTP and CSV, utilizing Azure Databricks for streamlined data flows into Snowflake.• Developed functions in Spark with Scala to unzip, decode, and parse XML files, ensuring accurate data storage into Azure BlobStorage.• Conducted thorough testing and debugging of DataStage jobs to ensure seamless data integration and transformation processes.• Led the design and development of ETL processes in DataBricks to efficiently migrate Campaign data from Azure DataLake into optimized formats (ORC, Parquet, Text Files).• Optimized PySpark applications on Databricks, achieving significant cost savings through enhanced performance and reduced resource utilization.• Constructed pipelines in Azure Data Factory for seamless data transfer of parquet files from ADLS Gen2 to Azure Synapse Analytics, facilitating advanced data warehousing solutions.• Replaced legacy Hive scripts with Spark DataFrame transformations, accelerating data analysis and enhancing processing efficiency.• Spearheaded the development of PySpark scripts that reduced organizational costs by 30% through the strategic migration of customer data from SQL Server to Hadoop.• Developed and maintained ETL processes using IBM DataStage for extracting, transforming, and loading data from various sources into the enterprise data warehouse.• Utilized Spark for processing streaming data, conducting interactive queries, and integrating with NoSQL databases, accommodating large-scale data handling needs.
EDUCATION
VNR Vignana Jyothi Institute of Engineering and Technology (VNRVJIET)
Bachelor of Technology - BTech, Computer science and engineering
University of Alabama at Birmingham
Master in Data Science, Data Science
ABOUT DIGAMBER KALIVEMULA
Data Engineer with 4+ years of experience in Data Extraction, Data Modelling, DataStage, Statistical Modeling, Data Mining, and Data Visualization.• Working experience with the entire Software Development Life Cycle (SDLC) process and good experience with Agile and Waterfall Methodologies.• Extensive experience in SQL and Python, including NumPy, SciPy, Scikit-learn, Pandas, Matplotlib, and TensorFlow.• Proficiency in the development and design of various scalable systems using Hadoop technologies in various environments.• Ability to analyze data using Hadoop Ecosystems, including HDFS, MapReduce, Hive, and Pig.• Working knowledge of Amazon’s Elastic Cloud Compute (EC2) infrastructure for computational tasks and Simple Storage Service (S3) as a storage mechanism.• Proven experience designing, developing, and deploying SSIS packages in an MS SQL server environment using SSIS.• Well-versed in analyzing data on various IDEs like Jupyter Notebook and Visual Studio Code.• Capable of working with Tableau reports with complex calculations and working on Ad-hoc reporting using Power BI.• Extensive experience in developing and deploying data pipelines in the cloud, such as AWS and GCP.• Expertise in developing and publishing reports of various types using Power BI, SSRS, and Business Objects.• Good Working knowledge of different database systems such as MS SQL Server, MySQL, and PostgreSQL.
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.