PhD in Adapting Transformer Models for Defect Detection with Limited Data

The mission of the Department of Electrical Engineering is to acquire, share and transfer knowledge and understanding in the whole field of Electrical Engineering through education, research and valorization. We work towards a ‘Smart Sustainable Society’, a ‘Connected World’, and a healthy humanity (‘Care & Cure’). Activities share an application-oriented character, a high degree of complexity and a large synergy between multiple facets of the field.

Research is carried out into the applications of electromagnetic phenomena in all forms of energy conversion, telecommunication and electrical signal processing. Existing and new electrical components and systems are analyzed, designed and built. The Electrical Engineering department takes its inspiration from contacts with high-tech industry in the direct surrounding region and beyond.

The department is innovative and has international ambitions and partnerships. The result is a challenging and inspiring setting in which socially relevant issues are addressed.

Introduction

Join the NWO Perspectief FIND program and develop methods to adapt Transformer-based foundation models for defect detection where data is scarce and unlabeled. Explore few-shot learning, self-supervised adaptation, and synthetic data generation to enable robust, scalable AI in semiconductor and printing systems. Work with leading industry partners like Canon and help transform quality inspection in next-generation high-tech equipment!

Job Description

Industrial edge deployments—in semiconductor manufacturing, industrial printing systems, automotive radar, smart mobility cameras, and HealthTech—require on-device AI to ensure low latency, privacy, and resilience. Today’s Transformers models scale poorly and assume abundant cloud resources. The research program FIND aims to deliver architectural and algorithmic breakthroughs that enable foundation models to run predictably and efficiently on embedded processors and accelerators.

FIND is a research program funded by the Dutch government and industry that brings together 5 universities, 11 companies (startups to multinationals), and 2 knowledge institutes to develop foundation models (large AI models) for Dutch high‑tech industry, with strong emphasis on edge deployment, privacy, and timely decision‑making. Partners include ASML, NXP, Canon Production Printing, ASMPT, Technolution, Signify, Shell, Stryker, TNO, and others. A total of 12 PhDs will be employed on the FIND program covering topics from foundation model pre-training and multimodal adaptation to architectures and compression for edge deployment while targeting real-world validation in domains like HealthTech, smart industry, and autonomous mobility.

This PhD position focuses on adapting and fine-tuning Transformer-based foundation models for defect detection in high-tech manufacturing environments where only limited and largely unlabeled defect data is available. Current solutions typically rely on supervised CNN-based models trained on large labeled datasets, which fail when defects are rare, vary across machines, or when labeling is prohibitively expensive. These approaches lack flexibility and generalization, making them unsuitable for dynamic industrial settings with scarce and imbalanced data.

You will also explore few-shot learning, self-supervised adaptation, and multimodal integration techniques to overcome data scarcity and improve robustness. Unlike existing methods that depend on exhaustive annotation or handcrafted features, this research will leverage the rich representations of foundation models and develop strategies for zero-shot or few-shot adaptation. You will investigate domain adaptation, synthetic data generation, and cross-modal learning to enable models that generalize across defect types and machine configurations. This ensures scalable, accurate defect detection even in low-resource industrial contexts.

The resulting