Object detection plays a vital role in multi-modal understanding systems, where images are input into models to generate proposals aligned with text. This process is crucial for state-of-the-art models handling Open-Vocabulary Detection (OVD), Phrase Grounding (PG), and Referring Expression Comprehension (REC). OVD models are trained on base categories in zero-shot scenarios but must predict both…

MLLMs, or multimodal large language models, have been advancing lately. By incorporating images into large language models (LLMs) and harnessing the capabilities of LLMs, MLLMs demonstrate exceptional skill in tasks including visual question answering, instruction following, and image understanding. Studies have seen a significant flaw in these models despite their improvements; they still have some…

3D-aware Generative Adversarial Networks (GANs) have made remarkable advancements in generating multi-view-consistent images and 3D geometries from collections of 2D images through neural volume rendering. However, despite these advancements, a significant challenge has emerged due to the substantial memory and computational costs associated with dense sampling in volume rendering. This limitation has compelled 3D GANs…

In human-computer interaction, the need to create ways for users to communicate with 3D environments has become increasingly important. This field of open-ended language queries in 3D has attracted researchers due to its various applications in robotic navigation and manipulation, 3D semantic understanding, and editing. However, current approaches have limitations of slow processing speeds and…

In 3D scene generation, a captivating challenge is the seamless integration of new objects into pre-existing 3D scenes. The ability to modify these complex digital environments is crucial, especially when aiming to enhance them with human-like creativity and intention. While adept at altering scene styles and appearances, earlier methods falter in inserting new objects consistently…

In the landscape of text-to-image models, the demand for high-quality visuals has surged. However, these models often need to grapple with resource-intensive training and slow inference, hindering their real-time applicability. In response, this paper introduces PIXART-δ, an advanced iteration that seamlessly integrates Latent Consistency Models (LCM) and a custom ControlNet module into the existing PIXART-α…

There’s a burgeoning interest in technologies that can transform textual descriptions into videos. This area, blending creativity with cutting-edge tech, is not just about generating static images from text but about animating these images to create coherent, lifelike videos. The quest for producing high-fidelity, aesthetically pleasing videos that accurately reflect the described scenarios presents a…

Developing large-scale datasets has been critical in computer vision and natural language processing. These datasets, rich in visual and textual information, are fundamental to developing algorithms capable of understanding and interpreting images. They serve as the backbone for enhancing machine learning models, particularly those tasked with deciphering the complex interplay between visual elements in images…

A pressing issue emerges in text-to-image (T2I) generation using reinforcement learning (RL) with quality rewards. Even though potential enhancement in image quality through reinforcement learning RL has been observed, the aggregation of multiple rewards can lead to over-optimization in certain metrics and degradation in others. Manual determination of optimal weights becomes a challenging task. This…

Researchers from Tel-Aviv University and Google Research introduced a new method of user-specific or personalized text-to-image conversion called Prompt-Aligned Personalization (PALP). Generating personalized images from text is a challenging task and requires the presence of diverse elements like specific location, style, or (/and) ambiance. Existing methods compromise personalization or prompt alignment. The most difficult challenge…