joycenerd.cs09[AT]nycu.edu.tw
EC129 Guangfu Campus

Zhi-Yi Chin

Hi! I am a research assistant in the Reinforcement Learning and Bandits Lab (REAL) at National Yang Ming Chiao Tung University (NYCU), working with Ping-Chun Hsieh and collaborating closely with Pin-Yu Chen from IBM and Mario Fritz from CISPA. My research centers on developing trustworthy multimodal generative models, where I have developed red-teaming methods through prompt attacks and am currently focused on interpreting model vulnerabilities. My goal is to not only identify potential security weaknesses but also uncover the fundamental mechanisms that make these systems susceptible to attacks.

I completed my MSc in Computer Science at NYCU supervised by Wei-Chen (Walon) Chiu and received my B.S. in Computer Science from National Chung Cheng University (CCU), where I was advised by Chen-Kuo (Adrian) Chiang.

I am actively seeking research fellowships or internships in multimodal generative model applications, with particular interest in trustworthiness, alignment, interpretability, and generalizability. My detailed experience and publications can be found in my Curriculum Vitae. I'd love to connect and chat about research, feel free to reach out at joycenerd.cs09[AT]nycu.edu.tw for potential collaborations or discussions, or schedule a quick coffee chat via my Calendly.

Links: CV / Twitter / Github / Google Scholar / Linkedin / Instagram / Threads / Facebook

Publications

For a comprehensive list of my publications, please refer to my Curriculum Vitae (CV).
(† indicates equal contribution)

Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts
Zhi-Yi Chin^†, Chieh-Ming Jiang^†, Pin-Yu Chen, Ching-Chun Huang, Wei-Chen Chiu
ICML 2024
Project Page / Code / Dataset
We introduce P4D a white-box red-teaming method for T2I model by correspondence model guidance and token-level prompt optimization technique.
In-Context Experience Replay Facilitates Safety Red-Teaming of Text-to-Image Diffusion Models
Zhi-Yi Chin, Mario Fritz, Pin-Yu Chen, Wei-Chen Chiu
arXiv 2024

We propose ICER, a jailbreaking framework for T2I models that stores past red-teaming attempts as experience replay, employs bandit sampling from this replay buffer to construct LLM priors, and guides LLM generation of fluent jailbreaking prompts through Bayesian optimization, creating a self-improving attack cycle.
Realizing Video Summarization from the Path of Language-based Semantic Understanding
Kuan-Chen Mu, Zhi-Yi Chin, Wei-Chen Chiu
arXiv 2024

We introduce a novel inference-time video summarization framework that combine multiple VideoLLMs' complementary strengths, enabling comprehensive summaries without requiring additional fine-tuning.
Masking Improves Contrastive Self-Supervised Learning for ConvNets, and Saliency Tells You Where
Zhi-Yi Chin^†, Chieh-Ming Jiang^†, Pin-Yu Chen, Ching-Chun Huang, Wei-Chen Chiu
WACV 2024
Code
We propose a saliency-aware masking strategy for SSL in ConvNets that balances mask distribution between foreground and background regions while introducing hard negatives through strategic salient patch masking.
Multi-Camera Tracking by Candidate Intersection Ratio Tracklet Matching
Yun-Lun Li, Zhi-Yi Chin, Ming-Ching Chang, Chen-Kuo Chiang
CVPR 2021 AI City Challenge Workshop

Projects

3D Point Cloud Data Augmentation via Scene Representation Network
Pei-Tse Chiang, Meng-Hsun Tsai, Zhi-Yi Chin, Chieh-Ming Jiang
pdf / Code / Slides
We develop a 3D point cloud augmentation pipeline that leverages SRN and image features to generate new 3D shapes through latent space interpolation, demonstrating success on ModelNet10. in 2021 MediaTek Research Project
RSNA Pneumonia Detection
Zhi-Yi Chin, Chieh-Ming Jiang
pdf / Code / Slides
We develop a high-performing pneumonia detection system for the RSNA Kaggle challenge by combining EfficientNet for classification and YOLOR for detection, with optimized prediction thresholds and bounding box refinements that boosted accuracy by 2%. Final project in Setected Topics in Visual Recognition Using Deep Learning 2021 Fall @ NYCU
Generative Models as a Data Augmentation for Classification
Zhi-Yi Chin, Chieh-Ming Jiang
Code / Video / Slides
We explore image transformation through latent space manipulation in GAN steer, demonstrating its superiority over traditional data-space transformations for data augmentation. Final poject in Deep Learning and Practice 2021 Summer @ NYCU
Reimplemenatation Challenge -- Maximum a Posteriori Policy Optimisation
Zhi-Yi Chin,Yi-Hsin Chen, Yu-Hsuan Li, Yu-Jie Chen
pdf / Code / Slides
We extend and enhance the MPO paper's algorithm implementation by incorporating numerical stabilization techniques and exploring E-step modifications for improved performance. Reimplementation project in Reinforcement Learning 2021 Spring @ NYCU
Calendar Helper
Zhi-Yi Chin, Mi Li, Jhong-Yu Huang, JC Chen
Code
We build a comprehensive task management platform that seamlessly blends calendars and to-do lists, featuring an innovative tagging system for smarter organization of events and tasks in 2019 CodeU @ Google
Lane Detection
Zhi-Yi Chin, Shao-Yu Weng, Bo-Yu Cheng
pdf / Code / Video / Slides
We develop an enhanced multi-lane detection system achieving over 70% accuracy by combining hourglass network with double hinge loss. Final project in Computer Vision 2021 Spring @ NYCU
Mango Classification
Zhi-Yi Chin, Tzu-Cheng Lin, Kung-Hao Chang, Yu-Chang Chen
pdf / Code
Achieve accuracy 82.31% on the testing data and rank 8 in the public board in AICUP Mango Image Recognition Challenge: Grade Classification and Defective Classification
Face Morphing and Warping
Code / Videos
Face swapping from my face to another person's face smoothly without ghost effect by morphing and warping technique. Final project in Introduction to Multimedia Technology in Fall 2019 @ CCU

Honors

Dean's list (6 times), Computer Science and Information Engineering Dept. at CCU, Fall '17, Spring '18, Fall '18, Spring '19, Fall '19, Spring '20
College Student Research Scholarship, get NT$ 48,000 from Ministry of Science and Technology, 2020
Google Student Travel Scholarship, scholarship to attend Grace Hopper Celebration, 2019

Services

Reviewer:
ICLR 2025
CVPR 2025

Miscellany

Besides research, I am an opera and classical crossover singer who performs both soprano and alto pieces. The majority of my free time is spent running and have completed three half marathons and one full marathon. I also enjoy reading and exploring dessert and coffee shops in my free time.

Books I enjoy: The Ride of a Lifetime (Robert Iger), Becoming (Michelle Obama), What We Owe the Future (William MacAskill), Atomic Habits (James Clear), Make Time (Jake Knapp and John Zeratsky), Show Your Work (Austin Kleon)
TV shows I enjoy: Grey's Anatomy, Lessons in Chemistry, The Morning Show, Hospital Playlist

This website is built from the source code of Nelson F. Liu's awesome website (https://nelsonliu.me ).