Ian Lewis

WORK HISTORY

SKILLS

ABOUT IAN LEWIS

Senior Data Engineer, Developer with rich experience at all levels in start-ups.Always pragmatic in my approach to problem solving I aim to develop solutions which are simple to maintain, operate, and configure.RecentlyAt Divido I built the prototype Data Warehouse and Analytics solutions using a set of Python jobs. Soon after the opportunity arose to lead a team of developers to create the next generation of data pipelines using Meltano, dbt and Apache Airflow. Consumer data contains PII so part of my responsibilities as Data Engineer was to create solutions to manage the lifecycle of Personal data. I created a set of rules based routines to correctly implement the retention policies that were in place.PreviouslyI have worked in ticketing and with extensively with music streaming data. Much of the work included building and maintaining ETL processes for billions of lines of daily data. I added new skills to my portfolio and broadened my experience elsewhere.Google Cloud Platform experience 3+ years includes the following- BigQuery - Storage - SQL (Postgres)- Compute Engine - deployments and VM management - Pub/Sub - from Python and Ruby - IAM, understanding setting user and API permissions. Credential mangement- APIs and Services, including YouTube, BigQuery, Cloud Deployment, Cloud Datastore etc - Source Repositories, connecting to BitBucket - Dataflow - Cloud Scheduler - Monitoring - Datastore - App Engine - Cloud Build - Cloud Functions, Python, NodeJSAlso deployment of websites, setting up DNS records, certificate management, general SysAdmin duties.AWS Experience 5+ years- S3, storage and website deployments- Cloudfront, CDN, Site deployments, Website White Labelling, HTTPS configurations- SSL Certificate Manager - Route53, DNS management - EC2, VM deployments - IAM and Permission management for user and machine access.Other areas of knowledge and experience- Elasticsearch - SQL, extensive experience (15+ years).Worked with Big Data from the major music streaming services: Spotify, Apple, Deezer, Soundcloud, Pandora, Google, YouTube etc. This has involved collecting and ingesting data into BigQuery from a number of different APIs and services. The data was then analysed for clients.As a System Admin (10+ years) I have been responsible for setting up websites, email servers, spinning up and maintaining servers. Managing SSL certificates, Working with Apache and Nginx as proxy servers is usual. Firewall management. Email SPAM management. User endpoint security.