Snapshot of salesforce/BLIP: 5.7k★, Jupyter Notebook. PyTorch code for BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation
Snapshot summary built from the project's own GitHub metadata — there's no written TopGit review yet. The page will update automatically when a full review is published.
WHY NO REVIEW YET
TopGit writes full reviews for the most-starred, most-requested repositories. This page is a snapshot until then — see the READ ME tab for the original README in full.
To finetune the pre-trained checkpoint using 8 A100 GPUs, first set 'pretrained' in configs/retrieval_coco.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base.pth". Then run:
To finetune the pre-trained checkpoint using 8 A100 GPUs, first set 'pretrained' in configs/caption_coco.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth". Then run:
To finetune the pre-trained checkpoint using 16 A100 GPUs, first set 'pretrained' in configs/vqa.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base_capfilt_large.pth". Then run:
To finetune the pre-trained checkpoint using 16 A100 GPUs, first set 'pretrained' in configs/nlvr.yaml as "https://storage.googleapis.com/sfr-vision-language-research/BLIP/models/model_base.pth". Then run:
In order to finetune a model with ViT-L, simply change the config file to set 'vit' as large. Batch size and learning rate may also need to be adjusted accordingly (please see the paper's appendix for hyper-parameter details). Gradient checkpoint can also be activated in the config file to reduce GPU memory usage.
Pre-train:
Prepare training json files where each json file contains a list. Each item in the list is a dictonary with two key-value pairs: {'image': path_of_image, 'caption': text_of_image}.
In configs/pretrain.yaml, set 'train_file' as the paths for the json files .
Download MSRVTT dataset following the instructions from https://github.com/salesforce/ALPRO, and set 'video_root' accordingly in configs/retrieval_msrvtt.yaml.
We provide bootstrapped pre-training datasets as json files. Each json file contains a list. Each item in the list is a dictonary with two key-value pairs: {'url': url_of_image, 'caption': text_of_image}.
Image source
Filtered web caption
Filtered synthetic caption by ViT-B
Filtered synthetic caption by ViT-L
CC3M+CC12M+SBU
Download
Download
Download
LAION115M
Download
Download
Download
Citation
If you find this code to be useful for your research, please consider citing.
@inproceedings{li2022blip,
title={BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation},
author={Junnan Li and Dongxu Li and Caiming Xiong and Steven Hoi},
year={2022},
booktitle={ICML},
}
Acknowledgement
The implementation of BLIP relies on resources from ALBEF, Huggingface Transformers, and timm. We thank the original authors for their open-sourcing.
No homepage URL was recorded for salesforce/BLIP in TopGit's last sync. The README tab above frequently contains screenshots and demo links, or check the repository description on GitHub.
How active is development on salesforce/BLIP?
The most recent commit recorded on salesforce/BLIP was 8 months ago, based on the GitHub push timestamp. The repository has 771 forks — one of the better signals of community interest.
How many stars does salesforce/BLIP have?
salesforce/BLIP has 5.7k GitHub stars — refresh the page for the live number, or check github.com/salesforce/BLIP. TopGit mirrors GitHub's count but does not claim minute-by-minute accuracy.
What language is salesforce/BLIP written in?
salesforce/BLIP is written primarily in Jupyter Notebook. GitHub's language field is based on the largest share of bytes in the default branch.
What license does salesforce/BLIP use?
salesforce/BLIP is released under the BSD-3-Clause license. Always verify the LICENSE file directly on GitHub for the authoritative terms — license strings can be edited out of sync with a project's actual stance.
What topics is salesforce/BLIP associated with?
GitHub's repository topics for salesforce/BLIP: "image-captioning", "image-text-retrieval", "vision-and-language-pre-training", "vision-language", "vision-language-transformer", "visual-question-answering", "visual-reasoning". TopGit's editorial category is open-source.
Where do I read more about salesforce/BLIP?
This TopGit page is a snapshot — the READ ME tab shows the project's own README content (links stripped, images preserved). The GitHub repository at github.com/salesforce/BLIP is the definitive source.
Read full README in the tab above.
Is BLIP worth your time?
ChatGPT, Claude and Perplexity can all read this page. Ask one of them what it makes of BLIP.