Software Engineer
Viberse Technology Pte. Ltd.
-
Reverse Media Search System
Built a reverse media search system using the vector database Milvus, the sentence-transformer machine learning library, and the CLIP model. Implemented functionality to search similar images and videos by text, image or video.
-
Backend Development and ML Model Deployment
Developed and deployed machine learning models on cloud platforms, implemented scalable backend services with a microservices architecture, integrated Redis caching, and configured Nginx for optimized request routing and rate limiting to ensure system stability and low-latency APIs.
-
MySQL CDC Sync to Elasticsearch using Flink DataStream
Developed a CDC connector and Elasticsearch sink using the Java Flink DataStream API, extracted MySQL binlog for real-time synchronization of entire database tables to corresponding Elasticsearch indexes, including newly created tables.
-
Application Services CI/CD
Created CI/CD pipelines using AWS CDK, AWS SAM and Secrets Manager to deploy Api Gateway, SQS, Lambda functions and layers in multiple regions and multi-environment.
April 2024 - September 2024
Software Engineer
ONElab Technology Ltd.
-
Pipeline Service Development and Integration
Developed and maintained robust data pipeline services utilizing Python and .NET C#, integrating data from cloud SQL, on-premises databases, Kafka, and various departmental projects for comprehensive data aggregation and analysis.
-
Data Warehouse Restructuring and Cost Optimization
Led a technical project to restructure the data warehouse, reducing departmental BigQuery costs by 30%, increasing scheduling efficiency and flexibility, and simplifying the development process.
-
Event-Driven Architecture Design and Implementation
Designed and implemented an event-driven architecture, integrating streaming data, AI models, and Telegram API for real-time analysis and automatic ad blocking and alert notifications based on user chat messages.
-
DevOps Proficiency in Software Deployment and Pipeline Management
Implemented and deployed software and services through Kubernetes, cloud functions, and GitLab CI to streamline pipeline construction and enhance operational efficiency.
-
Data-Driven User Behavior Analysis and Metrics Computation
Collected real-time data and developed data models to compute user footprints and various behavioral metrics. Continuously performed multi-dimensional user segmentation and labeling with a CDP.
-
Skype Bot Development
Developed a Skype bot for automated message broadcasting to targeted users based on various indicators.
-
Monitoring and Alert System Establishment
Established a comprehensive monitoring and alert system using Slack Webhook, Cloud Logging, Scheduled Queries, and Looker Studio.
-
Data Modeling-Driven Business Intelligence
Formulated computational logic for various product and operational metrics, and developed Looker Studio dashboards to facilitate data analysis for various departments and project teams, enabling informed decision-making and project monitoring.
October 2022 - March 2024