Knowledge Distillation has gained popularity for transferring the expertise of a “teacher” model to a smaller “student” model. Initially, an iterative learning process involving a high-capacity model is employed.  The student, with equal or greater capacity, is trained with extensive augmentation. Subsequently, the trained student expands the dataset through pseudo-labeling new data. Notably, the student…

Adopting finetuned adapters has become a cornerstone in generative image models, facilitating customized image creation while minimizing storage requirements. This transition has catalyzed the development of expansive open-source platforms, fostering communities to innovate and exchange various adapters and model checkpoints, thereby propelling the proliferation of creative AI art. With over 100,000 adapters now available, the…

The rapid evolution in AI demands models that can handle large-scale data and deliver accurate, actionable insights. Researchers in this field aim to create systems capable of continuous learning and adaptation, ensuring they remain relevant in dynamic environments. A significant challenge in developing AI models lies in overcoming the issue of catastrophic forgetting, where models…

Multimodal large language models (MLLMs) integrate text and visual data processing to enhance how artificial intelligence understands and interacts with the world. This area of research focuses on creating systems that can comprehend and respond to a combination of visual cues and linguistic information, mimicking human-like interactions more closely. The challenge often lies in the…

Online text recognition models have advanced significantly in recent years due to enhanced model structures and larger datasets. However, mathematical expression (ME) recognition, a more intricate task, has yet to receive comparable attention. Unlike text, MEs have a rigid two-dimensional structure where the spatial arrangement of symbols is crucial. Handwritten MEs (HMEs) pose even greater…

Within multimedia and communication contexts, the human face serves as a dynamic medium capable of expressing emotions and fostering connections. AI-generated talking faces represent an advancement with potential implications across various domains. These include enhancing digital communication, improving accessibility for individuals with communicative impairments, revolutionizing education through AI tutoring, and offering therapeutic and social support…

Multimodal architectures are revolutionizing the way systems process and interpret complex data. These advanced architectures facilitate simultaneous analysis of diverse data types such as text and images, broadening AI’s capabilities to mirror human cognitive functions more accurately. The seamless integration of these modalities is crucial for developing more intuitive and responsive AI systems that can…

In AI, searching for machines capable of comprehending their environment with near-human accuracy has led to significant advancements in semantic segmentation. This field, integral to AI’s perception capabilities, includes allocating a semantic label to each pixel in an image, facilitating a detailed understanding of the scene. However, conventional segmentation techniques often falter under less-than-ideal conditions,…

The remarkable strides made by the Transformer architecture in Natural Language Processing (NLP) have ignited a surge of interest within the Computer Vision (CV) community. The Transformer’s adaptation in vision tasks, termed Vision Transformers (ViTs), delineates images into non-overlapping patches, converts each patch into tokens, and subsequently applies Multi-Head Self-Attention (MHSA) to capture inter-token dependencies.…

Large Language Models (LLMs) have proven their impressive instruction-following capabilities, and they can be a universal interface for various tasks such as text generation, language translation, etc. These models can be extended to multimodal LLMs to process language and other modalities, such as Image, video, and audio. Several recent works introduce models that specialize in…