Zhi-Yi Chin

Hi, I’m a research assistant at National Yang Ming Chiao Tung University (NYCU) in Taiwan, working with Wei-Chen (Walon) Chiu. I also collaborate closely with Pin-Yu Chen from IBM and Mario Fritz from CISPA.

My research interests lie in the application of multimodal generative models, with a specific focus on ensuring their trustworthiness. Additionally, I am conducting interpretability analyses to understand the mechanisms behind these attacks better, as it is crucial to not only identify vulnerabilities but also to comprehend the underlying causes. Beyond my work in the trustworthy ML/ AI safety domain, I am interested in multimodal/ LLM focus applications and alignment. I aim to design an improved evaluation method for assessing video-text alignment, where standard metrics currently fall short.

Previously, I completed my master’s degree in computer scient under the guidance of Professor Wei-Chen (Walon) Chiu. Prior to that, I earned my bachelor degree in computer science from National Chung Cheng University, where I had the privilege of being advised by Professor Chen-Kuo (Adrian) Chiang.

Please feel free to explore my academic journey and research interests on this website, including my Curriculum Vitae. I’m currently seeking a Ph.D. position in multimodal generative model applications: trustworthy, alignment, interpretability. Should my research resonate with you, I welcome you to reach out at joycenerd.cs09[AT]nycu.edu.tw for any potential collaboration or discussion.

Publications

For a comprehensive list of my publications, please refer to my Curriculum Vitae (CV).
($^\dagger$ indicates equal contribution)

	Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts Zhi-Yi Chin^†, Chieh-Ming Jiang^†, Pin-Yu Chen, Ching-Chun Huang, Wei-Chen Chiu In ICML 2024
	Realizing Video Summarization from the Path of Language-based Semantic Understanding Kuan-Chen Mu, Zhi-Yi Chin, Wei-Chen Chiu Preprint
	Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where Zhi-Yi Chin^†, Chieh-Ming Jiang^†, Pin-Yu Chen, Ching-Chun Huang, Wei-Chen Chiu In WACV 2024
	Multi-Camera Tracking by Candidate Intersection Ratio Tracklet Matching Yun-Lun Li, Zhi-Yi Chin, Ming-Ching Chang, Chen-Kuo Chiang. In CVPR Workshop 2021

Projects

	3D Point Cloud Data Augmentation via Scene Representation Network Pei-Tse Chiang, Meng-Hsun Tsai, Zhi-Yi Chin, Chieh-Ming Jiang. 2021 MediaTek Research Project We design a 3D point cloud augmentation based on a novel view synthesis method, scene representation networks, and use PointNet to evaluate our augmented point clouds quality. We replace instance object id with image features from ResNet to apply our method on unseen objects and do interpolation later on. Our method is successful in ModelNet10 and generates the augmented data by intra-class interpolation with ShapeNet in the latent space of SRN encoder.
	RSNA Pneumonia Detection Zhi-Yi Chin, Chieh-Ming Jiang. Final project in Setected Topics in Visual Recognition Using Deep Learning 2021 Fall We design a two stage method for RSNA Pneumonia detection challenge held on Kaggle. We get the best results by using EfficientNet as classification model with 0.2 classification probability threshold when testing, and YOLOR as detection model. At last, we boost the final accuracy 2% by resizing the predicted bounding box to 87.5% of the original size.
	Generative Models as a Data Augmentation for Classification Zhi-Yi Chin, Chieh-Ming Jiang. Final project in Deep Learning and Practice 2021 Summer We investigate image transformation by exploring walks in the latent space of GAN, which is called GAN steer. We conclude that GAN steerability is a better data augmentation technique compare to transformation done in the data space.
	Reimplemenatation Challenge -- Maximum a Posteriori Policy Optimisation Zhi-Yi Chin,Yi-Hsin Chen, Yu-Hsuan Li, Yu-Jie Chen. Reimplementation project in Reinforcement Learning 2021 Spring Apart from replicating the algorithm from the paper, we also apply numerical tricks to stabilize the training process. Moreover, We are considering improving the method by modifying the E-step.
	Lane Detection Zhi-Yi Chin, Shao-Yu Weng, Bo-Yu Cheng. Final project in Computer Vision 2021 Spring We modify two traditional methods and successfully detect more than 2 lanes with accuracy over 70%. We reach high accuracy by apply hourglass network and double hinge loss.
	Mango Classification Zhi-Yi Chin, Tzu-Cheng Lin, Kung-Hao Chang, Yu-Chang Chen. Final project in Machine Learning 2020 Spring Achieve accuracy 82.31% on the testing data and rank 8 in the public board in AICUP Mango Image Recognition Challenge: Grade Classification and Defective Classification.
	Face Morphing and Warping Zhi-Yi Chin. Final project in Introduction to Multimedia Technology in Fall 2019 Face swapping from my face to another person's face smoothly without ghost effect by morphing and warping technique.
	Calendar Helper Zhi-Yi Chin, Mi Li, Jhong-Yu Huang. Google CodeU project 2019 It is a multifunctional platform for to-do lists and calendars. Its highlight is that we have added a tagging system to calendar events/tasks.

Honors

Dean’s list (6 times), Computer Science and Information Engineering Dept. at CCU, Fall ‘17, Spring ‘18, Fall ‘18, Spring ‘19, Fall ‘19, Spring ‘20
College Student Research Scholarship, get NT$ 48,000 from Ministry of Science and Technology, 2020
Google Student Travel Scholarship, scholarship to attend Grace Hopper Celebration, 2019

Miscellaneous

Books I enjoy

The Ride of a Lifetime by Robert Iger
Becoming by Michelle Obama
What We Owe the Future by William MacAskill
Atomic Habits by James Clear
Make Time by Jake Knapp and John Zeratsky
Show Your Work by Austin Kleon

TV shows I enjoy

Grey’s Anatomy
Lessons in Chemistry
The Morning Show
Hospital Playlist