The advancement of AI has led to remarkable strides in understanding and generating content that bridges the gap between text and imagery. A particularly challenging aspect of this interdisciplinary field involves seamlessly integrating visual content with textual narratives to create cohesive and meaningful multi-modal outputs. This challenge is compounded by the need for systems that…

Current challenges faced by large vision-language models (VLMs) include limitations in the capabilities of individual visual components and issues arising from excessively long visual tokens. These challenges pose constraints on the model’s ability to accurately interpret complex visual information and lengthy contextual details. Recognizing the importance of overcoming these hurdles for improved performance and versatility,…

Deep convolutional neural networks (DCNNs) have been a game-changer for several computer vision tasks. These include object identification, object recognition, image segmentation, and edge detection. The ever-growing size and power consumption of DNNs have been key to enabling much of this advancement. Embedded, wearable, and Internet of Things (IoT) devices, which have restricted computing resources…

The emergence of Large Vision-Language Models (LVLMs) characterizes the intersection of visual perception and language processing. These models, which interpret visual data and generate corresponding textual descriptions, represent a significant leap towards enabling machines to see and describe the world around us with nuanced understanding akin to human perception. A notable challenge that impedes their…

Diffusion models are a set of generative models that work by adding noise to the training data and then learn to recover the same by reversing the noising process. This process allows these models to achieve state-of-the-art image quality, making them one of the most significant developments in Machine Learning (ML) in the past few…

In the dynamic arena of artificial intelligence, the intersection of visual and linguistic data through large vision-language models (LVLMs) is a pivotal development. LVLMs have revolutionized how machines interpret and understand the world, mirroring human-like perception. Their applications span a vast array of fields, including but not limited to sophisticated image recognition systems, advanced natural…

Natural Language Processing (NLP) is one area where Large transformer-based Language Models (LLMs) have achieved remarkable progress in recent years. Also, LLMs are branching out into other fields, like robotics, audio, and medicine. Modern approaches allow LLMs to produce visual data using specialized modules like VQ-VAE and VQ-GAN, which convert continuous visual pixels into discrete…

Foundational models are large deep-learning neural networks that are used as a starting point to develop effective ML models. They rely on large-scale training data and exhibit exceptional zero/few-shot performance in numerous tasks, making them invaluable in the field of natural language processing and computer vision. Foundational models are also used in Monocular Depth Estimation…

Text-to-image (T2I) generation is a rapidly evolving field within computer vision and artificial intelligence. It involves creating visual images from textual descriptions blending natural language processing and graphic visualization domains. This interdisciplinary approach has significant implications for various applications, including digital art, design, and virtual reality. Various methods have been proposed for controllable text-to-image generation,…

Understanding the world from a first-person perspective is essential in Augmented Reality (AR), as it introduces unique challenges and significant visual transformations compared to third-person views. While synthetic data has greatly benefited vision models in third-person views, its utilization in tasks involving embodied egocentric perception still needs to be explored. A major obstacle in this…