Nikhitha M Madasu

Data Engineer @Legal & General America

Frederick, MD, US

Analysed stories, participated in grooming sessions and point estimation for development according to agile methodology.• Developed Spark jobs using Scala and Python on top of Yarn/ MRv2 for interactive and Batch Analysis.• Worked with the Spark for improving performance and optimization of the existing algorithms in Hadoop using Spark Context, Spark-SQL, PySpark, Pair RDD\'s, Spark YARN, and Spark MLLib.• Involved in implementation of data movements from on-premises to cloud in MS Azure.• Evaluated the performance of Apache Spark in analysing genomic data.• Developed a NIFI Workflow to pick up the data from SFTP server and send that to Kafka broker.• Used Python - NumPy, SciPy, Pandas, NLTK, Matplotlib, Beautiful Soup, and Text Blob to finish the ETL process of clinical data for future NLP analysis.• Migrated data from traditional database systems to Azure SQL databases.• Used different Machine Learning modules for ruining the spark jobs on daily and weekly basis.• Migrated complex Map reduce programs into Spark RDD transformations, actions.• Designed and developed Oozie workflows to orchestrate Hive scripts, Sqoop.• Involved in developing the Spark Framework to provide structure to data on the fly and process the data using Spark core API\'s, Data Frame, Spark-SQL and Scala Evaluated and improved application performance with Spark.• Recreated existing application logic and functionality in the Azure Data Lake, Data Factory, and Azure SQL Database.• Developed Spark scripts to import large files from Azure and imported the data from different sources like HDFS/HBase into Spark RDD.• Extensively worked on Python and build the custom ingest framework.• Design, build and deliver the operational and management tools, frameworks and processes for the Azure Data Lake and drive the implementations into the Azure Data Lake Cloud Operations team.• Involved in working with Spark on top of YARN for interactive and batch analysis.

Acharya Nagarjuna University

Bachelor of Technology - BTech, Information Technology