Data engineer well-acquainted with SQL, Python, linux, Batch data pipeline, Streaming data pipeline, MySQL, BigQuery, Tableau, Spark, Dataflow,
Airflow, DataFusion, Google Cloud Platform, Cloud SQL, Google cloud Storage, Pandas, Numpy, Matplotlib, Plotly @Linkedin
brantdzx@gmail.com
I create a ETL batch data pipeline on Google Cloud Platform by using Dataflow and Composer (Airflow). This pipeline can periodicly download updated covid-19 cases data in EU from website to Google Cloud Storage (Datalake), perform data cleaning and transform , and load data to Bigquery (data warehouse) for analysis and data visualization (Tableau).
Build data pipeline on Google Cloud for restaurant reviews data from yelp website. BigQuery for machine learning (Recommendation system), Tableau for visualization.
Streaming data pipeline for a simulated yelp user's reviews data flow, which load data to BigQuery in real time for analysis, and connect Tableau for visualization.
Business inventory data integration by building a pipeline connect from cloud SQL to BigQuery, this pipeline is created on google cloud with Data Fusion.
Let's see what patterns we can find in the data of the past Nobel laureates. What can we learn about the Nobel prize and our world more generally?
Analyze data from Spotify, ELT data pipeline, data transform, cleaning, and data analyze using SQL in BigQuery
Comprehensive analysis of the Android app market by comparing thousands of apps in the Google Play store.