eric wuerfel

Eric Wuerfel

Data Engineer

Building quick and reliable data pipelines, automating busy-work and generating insights

I currently live and work in Radford VA with my wife, May, and my cat/coding partner, Theo.

Download PDF

Stack

  • Hadoop, Hive
  • SQL, Presto/Trino
  • Spark, pySpark
  • Bash, Python, Scala
  • Airflow, Oozie
  • Git, Jenkins, CICD
  • JIRA, Confluence

Education

2015-2019

BA in Physics, minor in Econ

Virginia Tech

2019-2020

Certificate in Data Science

Bloom Institute of Technology

Awards

2023

Top Contributor Award

for contributions as a data engineer at Lowe's

2018

ACC Creativity & Innovation Grant

for undergraduate research in multi-messenger astronomy

2016-2019

Daniel C. & Delia F. Grant Scholarship

for academic achievement in physics

Experience

April 2020 - present

Lowe's Companies, Inc.

Data Engineer

Developed new data pipelines; Maintained, enhanced and optimized existing data pipelines

Exported and extracted data to/from 3rd party sources such as AWS RDS & S3, SFTP, other RDBMSs

Migrated pipelines from Oozie to Airflow, SQL to Spark, Apache Hadoop to Cloudera Hadoop

Built alerts in case of failure, and served as on-call support

Onboarded new data engineers, mentored interns and associate engineers

February 2017 - May 2019

Virginia Tech Department of Philosophy

Researcher

Modified a MATLAB library for gravitational wave detection to take HDF5 files as input

Attended LIGO's data workshop at Caltech

Organized a data workshop at Virginia Tech to show others how to read and clean interferometer data

Projects

2023 - Kafka, Spark, Airflow

Kafka Topic Consumption

Setup ingestion of messages from Kafka topics for a quoting system

Parsed necessary details from Kafka messages into Hive tables for front-end consumption

Designed process to monitor streaming jobs' status, restart as needed and send alerts

2022 - Bash, MySQL, HQL, Airflow

Secure AWS RDS to On-Prem Ingestion

Ingested data from an AWS RDS instance with PGP encryption to handle PII

Coordinated with internal and external security team to satisfy all security requirements

Translated reports from MySQL to HQL for on-prem execution

Contact

Richmond, VA

(703)-896-8911

ericwuerfel@protonmail.com