注：此DINO并非自蒸馏自监督的那个[[DINO]]，而是派生自[[DETR]]
使用的backbone是BERT(通过MLM训练)
该研究认为，image encoder的模型大小应该大于text encoder,所以在text encoder这里，只使用六层self attention来提取特征，剩余六层cross attention用于multi-modal encoder。
,
Archives
- February 202614
- January 20268
- December 20253
- November 20256
- October 20251
- September 20253
- August 20256
- July 20255
- June 20256
- May 202510
- April 202517
- March 202545
- February 202512
- January 202513
- December 202412
- November 20244
- October 202418
- September 202416
- August 202413
- July 20243
- June 20245
- May 202413
- April 202417
- March 20241
- January 20241
- December 20231
- May 202346
- August 20221
- May 20226
- April 20229
