Company's [Open-source Github] and [Commercialized Product]

* AWS: Administrated DataJoint's AWS account and several other customers' AWS accounts. Configured VPC, Subnet, Security Groups, IAM role and policies, S3 lifecycle management, EFS access point, EC2 instances, RDS instances, Lambda triggered by SQS or EventBridge, SNS and SES, CloudWatch metrics and alarms, Route 53 DNS records, Secrets Manager for deployment secrets.

* CI/CD: Developed generic Github Actions reusable workflows used by 30+ repositories followed by Conventional Commits, Release Flow and GitOps best practices, to automate build, test, release, publish private or open-source Python packages[PyPI] or deploy Docker images[Dockerhub].

* Kubernetes: Provisioned Kubernetes clusters hosted on EC2 instances for development, staging and production environments using k3d or kOps. Developed utility bash scripts with helm and kubectl to manage Kubernetes clusters more efficiently, including configuring Nginx ingress controller, cert manager with Let's encrypt issuer, Cillium Container Network Interface(CNI), IAM Roles for Service Account(IRSA), Cluster Autoscaler, AWS Elastic Load Balancer(ELB) or deploying applications like Percona XtraDB Clusters, Keycloak, JupyterHub, Flask and ReactJS based web application, etc.

* Ephemeral Worker Clusters: Designed and developed a worker lifecycle manager using Python within one month to fulfill an urgent business requirement. This development polls jobs from a MySQL database, then provisions and configures ephemeral EC2 instances by Packer(pre-build AMI), Terraform and cloud-init to compute jobs at scale; implemented AWS S3 mount to significantly reduce raw data downloading overhead and added EFS as a file cache for intermediate steps to improve computation failover; configured NVIDIA CUDA toolkit and NVIDIA container runtime for GPU workers.

* Platform Automation: To provision or terminate AWS resources using boto3 or Terraform; manage customers' RBAC permissions using Keycloak and Github REST API; generating usage and billing report with AWS S3 Inventory report, AWS CloudTrail and AWS Cost and Usage report, made a Plotly Dash to analyze cost and usage efficiency.

* Jupyterhub: Configured and maintained Jupyterhub deployment on a Kubernetes cluster with Node Affinity to assign pods onto different nodes by requirements and Cluster Autoscaler along with AWS Auto Scaling Group to accommodate 100+ active users; improved base images' build time and maintenance overhead.

* Observability: Implemented a small part of the metrics and alerts using AWS CloudWatch, and then later integrated Datadog for Kubernetes clusters' and ephemeral EC2 instances' metrics and logging through OpenTelemetry protocol, synthetic API testing, and UI/UX monitoring.

* Security: Set up codebase vulnerability scan with FOSSA; Set up AWS Secrets Manager working with External Secret Store Operator to secure Kubernetes secrets; Deployed and administrated self-hosted Keycloak for RABC authentication, further integrated it with AWS IAM as an identity provider to access AWS resources through STS, enabled OpenID Connect(OIDC) authentication flows such as authorization code flow, client credential flow, password grant flow etc.

* MySQL Database: Maintained a self-hosted Percona XtraDB Clusters on database daily backup stored on S3, mysqldump backup redundancy, Point-in-Time Recovery(PITR), deadlock detection, and slow query log.

Company's commercialized product [DrillVedik]

* Interactive Drilling Dashboard: Developed an enterprise Plotly Dash dashboard in collaboration with two more engineers that visualizes processed data using Bootstrap, CSS media query, Redis and sqlalchemy. Also, implemented a socket service that will notify when Airflow pipeline is finished processing in order to synchronize(refresh) the dashboard's data.

* CI/CD Pipeline: Set up several Azure Pipelines for continuous development, testing and continuous deployment in dev, test and prod stages. Additionally, made a Jenkins pipeline to work with on-premise infrastructures.

* ML Pipeline: Hosted a MLflow server for machine learning experiment logging, parameter tuning, continuous training, model management and model serving.

* ETL Pipeline: Deployed an Airflow server in collaboration with another data engineer, for our data ETL pipeline.

* Prediction Task Manager: Designed and developed a production web application in collaboration with a front-end developer that supports job queuing and parallel processing for drilling speed prediction using JavaScript, flask, sqlalchemy, celery, RabbitMQ, gunicorn, Nginx, supervisord, Docker and AWS EC2, AWS Cognito Authentication, HTTPS

* Drilling Status Detection: Developed two classification models in collaboration with a domain expert for detecting drilling status using Logistic Regression and Random Forest with the convenience of the MLflow server

* Drilling Speed Prediction: In collaboration with a domain expert, applied Gaussian Process Regression for feature synthesis based on geographical information as well as feature engineering based on correlation matrix and F1 score ranking, and built a non-linear regression model using LSTM RNN.

* Company: startup

* User: potential

* Duties: Implemented layouts, activities, customized widgets, screen adaption

* Gains: Git Version control, Android updated version features, more customized widgets

* Self assessment: it's good to work remotely in a professional team included designers and developers, however, I still need more understanding in an entire software development lifecycle, not just coding.

* Supervisor: Dr. Frank Coyle

* CSE 3342 Undergrad: Programming Language

* CSE 8/7345 Grad: Advanced Application Programming

* CSE 8/7320 Grad: Artificial Intelligence

* CSE 8/7347 Grad: XML and the Enterprise

* Company: The largest public construction company in my hometown Qingdao with 91 million dollars total assets

* Users: 10+ construction teams and supervisors

* Functions: Draw lines or marks and pin icons of machine or labor on a 2D map API(Baidu Map API); update each operation and status of machine or labor then calculate and report daily cost in analytic plots; with different four types of permissions accessing individually

* Duties: Implemented layouts, activities, customized widgets, screen adaption, data synchronization and HTTP request/respond between server and users based on Java

* Gains: SVN Version control, How to customize widget, UML is important, Debugging and testing, Prototype Demo Tool is a saver to know client's demand better, code/material reusability, comment is important

* Self assessment: Fresh try in Android after academic classes, having fun and learning stuffs. Switching between Google and coding all the time, however make it work finally anyway. It doesn't look professionally pretty in graphic, but it works functionally well and indeed saves money for the company