Big vision-language models, or LVLMs, can interpret visual cues and provide easy replies for users to interact with. This is accomplished by skillfully fusing large language models (LLMs) with large-scale visual instruction finetuning. Nevertheless, LVLMs only need hand-crafted or LLM-generated datasets for alignment by supervised fine-tuning (SFT). Although it works well to change LVLMs from…
