Yi-Hui (Lily) Lee 李怡慧

Computer Vision from Scratch to Implementation

Built image preprocessing functions such as (adapted) linear scaling and (adapted) histogram equalization from scratch, similar to cv2 function by python. Built a sequential CNN-based classifier test on downsized MNIST by TensorFlow. Fine-tuned parameter with the order of filter size, learning rate and epochs, dropout rate, and final epochs. Designed two different filter size and dropout layers to handle overfitting that the training accuracy 90.74% and testing accuracy 80.84% within 4 mins 23 secs. Designed a blinks detection application by applying MTCNN and cascade technique combined with rule-based algorithms. Improved the detector and achieved 81.36% accuracy by precisely designed its optimal region of interest, preprocessed image by applying blurring and histogram equalization algorithms, and fined-tuned parameters for MTCNN and cascade.

Python, TensorFlow, CV, cv2, CNN, MTCNN, cascade, MNIST

Convolutional Neural Networks from Scratch to Implementation

Built Neural Networks, and Convolutional Neural Networks function similar to PyTorch function by python from scratch. Reproduced and fine-tuned RegNet test on downsized ImageNet by PyTorch. Designed efficient RegNet by small group size and high bottleneck ratio to lower parameters but keep the accuracy. Designed and concatenated different group sizes on the four different stages output similar to GoogLeNet. Designed Attention-Based Bilingual Image2caption Emoji Model (ABBIE) that transforms the image pixels into the bilingual caption and also with the corresponding emoji. Top-down ABBIE into transformer, bilingual language model, and image caption model. Utilized hugging face multiple transformer architecture to pretrain the word and emoji embedding. Fine-tuned the transformer to the image-2-text task.

Python, PyTorch, CNN, RegNet, ImageNet, GoogLeNet, Transformer, Hugging Face Transformer, Image Caption, Word Embedding, Emoji Embedding

RegNet PDF    ABBIE PDF

[LREC' 2020]
Headword-Oriented Entity Linking: A New Entity Linking Task with Dataset and Baseline

We cooperated with PIXNET to help them link the cosmetic product mentioned in the PIXNET blog. We designed a headword-oriented entity linking (HEL), a specialized entity linking problem in which only the headwords of the entities are to be linked to knowledge bases. We developed a product embedding model to solve the entity linking problem in cosmetic domain. Besides, to increase training data, we propose a special transfer learning framework in which distant supervision with heuristic patterns is first utilized, followed by supervised learning using a small amount of manually labeled data. The experimental results show that our model provides a strong benchmark performance on the special task. Our model raised accuracy from baseline 64% to 83.4%. We published this work in LREC' 2020.

PIXNET, NLP, Corpus, Information Extraction, Distant Supervision, Encoder Modules, Cosmetic Domain, Word Segmentation

PDF    CODE

Define IoT Device Naming Pattern in Domain Name

Researched domain name segmentation algorithms and deep neural networks for analyzing the domain name naming rules of IoT devices in IPv6. Collected 11, 182, 640 domain names in IPv6. Preprocessed leftmost-level domain name by adapted word segmentation by adding unknown words N-gram. Created association rules between substrings. Formulated naming pattern rules for the IoT device from embedding-based clustering algorithm.

Python, spaCy, Gensim, AllenNLP, Stanford CoreNLP, IoT Domain Name, IPv6, NLP, Regular Expressions, Word Segmentation, N-gram, Association Rules, Cluster

PDF    CODE