Najanth Katuri

Data Engineer @Rogers Communications

Toronto, ON, CA

Implemented Azure Data Factory (ADF) extensively for ingesting data from different source systems like relational and unstructured data to meet business functional requirements.> Design and developed Batch processing and real-time processing solutions using ADF, Databricks clusters and stream Analytics.> Created Numerous pipelines in Azure using Azure Data Factory v2 to get the data from disparate source systems by using different Azure Activities like Move & Transform, Copy, filter etc.> Used DDL and DML for writing Triggers, stored procedures, and data manipulation in Snowflake > Used Snow SQL to execute SQL queries and perform all DDL and DML operations, including loading data into and unloading data out of database tables > Maintain and provide support for optimal pipelines, Data flows and complex data transformations and manipulations using ADF and PySpark with Databricks.> Integrated Kubernetes with Cloud-Native services, such as AWS EKS and GCP GKE, to leverage additional scalability and managed services.> Conducted Performance tuning and optimization of Kubernetes and Docker deployments to improve overall system performance.> Automated jobs using different triggers like Events, Schedules and Tumbling in ADF.> Created different Databricks clusters, Notebooks, jobs and autoscaling.> Created several Databricks Spark jobs with Pyspark to perform several tables to table operations.> Performed Data flow transformation using the data flow activity.> Instantiated, created, and maintained CI/CD continuous integration & deployment pipelines and apply automation to environments and applications.> Implemented Azure self-hosted integration runtime in ADF.> Improved performance by optimizing computing time to process the streaming data by optimizing the cluster run time.> Perform ongoing Monitoring, Automation, and Refinement of Data engineering solutions.

Jawaharlal Nehru Technological University Kakinada (JNTUK)

Bachelor's degree