Jessica Jahan
- Role
- Big Data Engineer Hadoop Developer at Morgan Stanley
- Location
- Queens, NY, US
- LinkedIn followers
- 500 followers
About Jessica Jahan
Close to 8 years of professional IT experience in Big Data technologies, architecture…
Experience
Big Data Engineer Hadoop Developer
Sep 2021 — Present
Involved in importing the data from various data sources into HDFS using Sqoop and applying various transformations using Hive, Apache Pyspark and then loading data into Hive tables or AWS S3 buckets. Proficient in designing, developing, and implementing Ab Initio graphs, plans, and components. Expertise in developing and maintaining data integration applications using Ab Initio GDE. Used Databricks to collaboratively run interactive and scheduled data analysis workloads. Have scheduled daily, weekly & monthly workflows for Pyspark jobs using AWS Glue and created CRON jobs as well in some cases and scheduled. Developed the Pyspark Jobs and processed the data for the different facts and Dimensions on Azure Cloud environment with different complex transformations. Created and maintained visually appealing and interactive dashboards and reports using Tableau, resulting in improved data insights and informed decision-making Proficient in managing and configuring Azure Active Directory (AAD) to ensure seamless user authentication and access control within the organization\'s Azure environment Demonstrated expertise in leveraging Azure Data Factory (ADF) to orchestrate and automate data workflows, enabling efficient data integration, transformation, and loading processes. Extensive experience in designing and implementing applications using Microservices Architecture, facilitating scalability, modularity, and resilience within Azure-based solutions. Processed the data with different aggregations like QTD, YTD, MTD, Year, Month etc. Developed UDFs using Pyspark wherever required during the development. Configuring Splunk alerts in-order to get the log files while execution and storing them to a location in S3 bucket when cluster is running. Involved in Hive/SQL queries performing pyspark transformations using Pyspark RDDs and Python (pyspark).
Find verified contacts for anyone on LinkedIn
Unifers gives sales teams verified emails and direct dials, enriched profiles, and outreach that lands in the inbox.
Free plan included · No credit card required
This profile is compiled from publicly available professional sources. Unifers is not affiliated with or endorsed by LinkedIn. Request removal of this profile.