Sai K.

About Sai K.

I am a 8-year-old Data Engineer with IT experience in Data Application Development. Throughout my career, I have gained strong development skills with Azure Data Bricks, Azure Data Lake, Azure Data Factory, SQL Data Warehouse, Azure Blob, and Azure Storage Explorer. As a data engineer, I excel in architecting and implementing ETL and data movement solutions using Azure Data Factory (ADF) and SSIS. I have a fluent programming experience with PowerShell, Scala, Python, SQL, T-SQL, and R, enabling me to develop and deploy enterprise-based applications using major Hadoop ecosystem components like MapReduce, YARN, Hive, HBase, Flume, Sqoop, Spark SQL, and Kafka. One of my key strengths lies in developing data pipelines and flow on Cloud services like Azure Data Factory. I am skilled in configuring and installing Hadoop/Spark Ecosystem Components and proficient with Spark Core, Spark SQL, and Spark Streaming for processing and transforming complex data using in-memory computing capabilities written in Scala. Through my work, I have significantly improved the efficiency of existing algorithms using Spark Context, Spark SQL, Data Frame, Pair RDDs, and Spark YARN. My experience includes managing and reviewing Hadoop log files, as well as recreating existing application logic and functionality in the Azure Data Lake, Data Factory, SQL Database, and SQL Data warehouse environment. I have also been involved in data center migration projects and possess a strong virtualization experience, particularly with Big Data platforms deployed in the Azure Cloud. Within Azure, I have implemented Copy activity and Custom Azure Data Factory Pipeline Activities for on-cloud ETL processing. Additionally, I am comfortable wearing multiple hats, taking on responsibilities as an Azure Architect/System Engineer, handling network operations, and performing data engineering tasks. When it comes to Azure, I have a solid background in designing and implementing migration strategies for traditional systems, including the transition from infrastructure as a service (IaaS) applications to platform as a service (PaaS). I have experience deploying Azure Resource Manager JSON Templates from PowerShell and am well-versed in performance tuning and optimization (PTO), as well as Microsoft Hyper-V virtual infrastructure. In summary, my expertise in data engineering and Azure services, coupled with my extensive experience, make me a valuable asset in the field. I am confident in my ability to drive successful data projects and contribute to the growth and success of organizations.

Experience

1. Data Engineer

   Vsoft Technologies Ltd

   Feb 2024 — Present · Miami-Fort Lauderdale, FL, US

   RolesandResponsibilities: Responsible for creating workflows and implementing Data quality checks for finance data using DataIku Managed databases with the knowledge of database administration I gained from my previous experience. Utilized Data Factory to migrate the data from SQL Share-points to Azure SQL Databases. Implemented backup strategies for the Databases for effective restoration of databases. Implemented row- level of security among the databases for the users. Designed a data model and got it approved by the senior management. Documented the process flow for the data model and built a database using it. Created Dashboards for the DBA using the Audit data on the Dev and Prod databases using Power BI. Responsible for implementing Incremental data from a truncate and load strategy data. Managed cost analysis by implementing the effective usage of DTU’s and cut down the cost of pipeline execution by time and money from 130$/Day to 64$/Day and time from 4 hours to 2.5 hours. Split the old single environment into Dev and Test and Prod which helped to manage the data effectively. Extracted the sheet names separately from excel files using Databricks to efficiently ingest data into tables from Data lake and SharePoint. Utilized Power Automate to create cloud flows and automate processes like mail generation and script executions. Working with Api’s and extracting data from APIs to Azure Sql.

Education

• Jawaharlal Nehru Technological University

  Bachelor of Technology - BTech, Computer Science

  2010 — 2014