Trance-0/NoteNextra-origin
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

sad

Browse Source
This commit is contained in:
Trance-0
2025-11-03 23:56:22 -06:00
parent bc44c59707
commit 71ada8d498
2 changed files with 19 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
content/CSE5519/CSE5519_B4.md
View File

@@ -1,2 +1,17 @@
# CSE5519 Advances in Computer Vision (Topic B: 2024: Vision-Language Models)
## Improved Baselines with Visual Instruction Tuning (LLaVA-1.5)
[link to the paper](https://openaccess.thecvf.com/content/CVPR2024/papers/Liu_Improved_Baselines_with_Visual_Instruction_Tuning_CVPR_2024_paper.pdf)
This paper shows that the visual instruction tuning can improve the performance of the vision-language model.
### Novelty in LLaVA-1.5
1. Scaling to high resolution images by dividing images into grids and maintaining the data efficiency.
2. Compositional ability, (use long-form language reasoning together with shorter visual reasoning can improve the model's writing ability)
3. Random downsampling will not degrade the performance.
>[!TIP]
>
> This paper shows that LLaVA-1.5 obeys the scaling law and splitting the high resolution images into grids to maintain the data efficiency. I wonder why this method is not applicable to multi-image understanding tasks? Why we cannot assign index embeddings to each image and push the image sets to the model for better understanding?