method: DINOv2, ViT-L, 8x8 patch size, linear decoder2024-08-23
Authors: Tommie Kerssies, Daan de Geus, and Gijs Dubbelman
Affiliation: Eindhoven University of Technology
Email: t.kerssies@tue.nl
Description: Fine-tuning for ~40 epochs on Cityscapes, following the setup described in: "How to Benchmark Vision Foundation Models for Semantic Segmentation?" (https://www.tue-mps.org/benchmark-vfm-ss/)
method: DINOv2, ViT-G, 16x16 patch size, linear decoder2024-08-23
Authors: Tommie Kerssies, Daan de Geus, and Gijs Dubbelman
Affiliation: Eindhoven University of Technology
Email: t.kerssies@tue.nl
Description: Fine-tuning for ~40 epochs on Cityscapes, following the setup described in: "How to Benchmark Vision Foundation Models for Semantic Segmentation?" (https://www.tue-mps.org/benchmark-vfm-ss/)
method: DINOv2, ViT-B, 16x16 patch size, linear decoder2024-08-23
Authors: Tommie Kerssies, Daan de Geus, and Gijs Dubbelman
Affiliation: Eindhoven University of Technology
Email: t.kerssies@tue.nl
Description: Fine-tuning for ~40 epochs on Cityscapes, following the setup described in: "How to Benchmark Vision Foundation Models for Semantic Segmentation?" (https://www.tue-mps.org/benchmark-vfm-ss/)
BRAVO Index | Subset H-Means | ACDCfog | ACDCnight | ACDCrain | ACDCsnow | SMIYC | outofcontext | synflare | synobjs | synrain | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Date | Method | bravo | semantic | ood | ACDCfog | ACDCnight | ACDCrain | ACDCsnow | SMIYC | outofcontext | synflare | synobjs | synrain | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_ood | auroc_ood | fpr95_ood | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | auprc_err | auprc_ood | auprc_suc | auroc_ood | auroc | ece | fpr95_ood | fpr95 | miou | auprc_err | auprc_suc | auroc | ece | fpr95 | miou | |||
2024-08-23 | DINOv2, ViT-L, 8x8 patch size, linear decoder | 0.7789 | 0.6981 | 0.8807 | 0.6603 | 0.6808 | 0.6748 | 0.6756 | 0.8991 | 0.7104 | 0.7268 | 0.7666 | 0.7393 | 0.3585 | 0.9886 | 0.8619 | 0.0247 | 0.4451 | 0.7702 | 0.4415 | 0.9911 | 0.9102 | 0.0406 | 0.4767 | 0.6724 | 0.3580 | 0.9939 | 0.9005 | 0.0190 | 0.4097 | 0.7880 | 0.3686 | 0.9910 | 0.8839 | 0.0234 | 0.4242 | 0.7891 | 0.8901 | 0.9727 | 0.1563 | 0.4112 | 0.9962 | 0.9330 | 0.0119 | 0.3565 | 0.7288 | 0.4227 | 0.9966 | 0.9370 | 0.0157 | 0.3507 | 0.7916 | 0.4127 | 0.7442 | 0.9966 | 0.9804 | 0.9348 | 0.0135 | 0.1010 | 0.3328 | 0.7965 | 0.4234 | 0.9976 | 0.9482 | 0.0113 | 0.3057 | 0.8030 | |||
2024-08-23 | DINOv2, ViT-G, 16x16 patch size, linear decoder | 0.7612 | 0.7001 | 0.8340 | 0.6660 | 0.6786 | 0.6766 | 0.6871 | 0.8818 | 0.7100 | 0.7316 | 0.7469 | 0.7317 | 0.3492 | 0.9921 | 0.8877 | 0.0194 | 0.4170 | 0.7845 | 0.4046 | 0.9929 | 0.9141 | 0.0326 | 0.4529 | 0.7102 | 0.3517 | 0.9949 | 0.9118 | 0.0174 | 0.3960 | 0.8002 | 0.3734 | 0.9946 | 0.9153 | 0.0212 | 0.4093 | 0.8052 | 0.8649 | 0.9685 | 0.1757 | 0.4157 | 0.9958 | 0.9319 | 0.0122 | 0.3622 | 0.7207 | 0.4206 | 0.9967 | 0.9404 | 0.0145 | 0.3285 | 0.7980 | 0.4185 | 0.6212 | 0.9964 | 0.9691 | 0.9371 | 0.0149 | 0.1310 | 0.3338 | 0.7803 | 0.4073 | 0.9976 | 0.9474 | 0.0108 | 0.3055 | 0.8087 | |||
2024-08-23 | DINOv2, ViT-B, 16x16 patch size, linear decoder | 0.7554 | 0.7046 | 0.8142 | 0.6903 | 0.6650 | 0.7069 | 0.6795 | 0.8790 | 0.7116 | 0.7277 | 0.7404 | 0.7302 | 0.4063 | 0.9931 | 0.9075 | 0.0194 | 0.4071 | 0.7066 | 0.4596 | 0.9881 | 0.9021 | 0.0455 | 0.5233 | 0.6325 | 0.4140 | 0.9959 | 0.9317 | 0.0173 | 0.3818 | 0.7355 | 0.4058 | 0.9907 | 0.8980 | 0.0267 | 0.4615 | 0.7365 | 0.8440 | 0.9648 | 0.1607 | 0.4451 | 0.9955 | 0.9337 | 0.0121 | 0.3875 | 0.6829 | 0.4573 | 0.9955 | 0.9351 | 0.0175 | 0.3767 | 0.7339 | 0.4395 | 0.5689 | 0.9961 | 0.9671 | 0.9365 | 0.0131 | 0.1414 | 0.3427 | 0.7585 | 0.4382 | 0.9968 | 0.9430 | 0.0125 | 0.3428 | 0.7480 | |||
2024-08-23 | PixOOD YOLO (="Model Selection") | 0.6779 | 0.5706 | 0.8349 | 0.5688 | 0.4895 | 0.6199 | 0.5514 | 0.7487 | 0.6487 | 0.6496 | 0.5638 | 0.6328 | 0.6905 | 0.8930 | 0.7987 | 0.2852 | 0.5393 | 0.3166 | 0.7060 | 0.9099 | 0.8410 | 0.0887 | 0.6267 | 0.2094 | 0.6800 | 0.9261 | 0.8299 | 0.2018 | 0.5079 | 0.3790 | 0.8014 | 0.8365 | 0.8263 | 0.1796 | 0.5598 | 0.2676 | 0.7238 | 0.9332 | 0.3563 | 0.4486 | 0.9848 | 0.8586 | 0.0900 | 0.5117 | 0.5908 | 0.5739 | 0.9719 | 0.8848 | 0.1249 | 0.5064 | 0.4618 | 0.1998 | 0.8557 | 0.9826 | 0.9969 | 0.7863 | 0.0991 | 0.0081 | 0.6673 | 0.7285 | 0.4857 | 0.9753 | 0.8511 | 0.1386 | 0.5415 | 0.5315 | |||
2024-08-23 | DeiT III (IN21K->IN1K), ViT-B, 16x16 patch size, linear decoder | 0.6665 | 0.6649 | 0.6682 | 0.6256 | 0.6120 | 0.6701 | 0.6383 | 0.6820 | 0.6925 | 0.6796 | 0.6954 | 0.6985 | 0.4690 | 0.9755 | 0.8597 | 0.0182 | 0.5856 | 0.5436 | 0.5262 | 0.9759 | 0.8807 | 0.0633 | 0.6113 | 0.4754 | 0.4845 | 0.9879 | 0.9021 | 0.0190 | 0.5110 | 0.5864 | 0.4789 | 0.9791 | 0.8719 | 0.0134 | 0.5778 | 0.5679 | 0.5577 | 0.9014 | 0.3317 | 0.4809 | 0.9931 | 0.9246 | 0.0133 | 0.4462 | 0.5960 | 0.5192 | 0.9885 | 0.9136 | 0.0346 | 0.5220 | 0.5993 | 0.4692 | 0.4242 | 0.9951 | 0.9602 | 0.9347 | 0.0109 | 0.1539 | 0.3990 | 0.6758 | 0.4622 | 0.9941 | 0.9283 | 0.0110 | 0.4323 | 0.6368 | |||
2024-08-23 | DINOv2, ViT-G, 16x16 patch size, Mask2Former decoder | 0.6454 | 0.4968 | 0.9208 | 0.5143 | 0.6134 | 0.4062 | 0.4303 | 0.9437 | 0.4090 | 0.5389 | 0.6432 | 0.6007 | 0.2075 | 0.9877 | 0.8230 | 0.0389 | 0.5707 | 0.7996 | 0.3491 | 0.9931 | 0.9093 | 0.0545 | 0.5730 | 0.7088 | 0.1710 | 0.9930 | 0.8648 | 0.0642 | 0.7759 | 0.8138 | 0.1862 | 0.9907 | 0.8546 | 0.0657 | 0.7561 | 0.8160 | 0.9146 | 0.9849 | 0.0656 | 0.1905 | 0.9910 | 0.8674 | 0.0643 | 0.7922 | 0.7257 | 0.2358 | 0.9944 | 0.9013 | 0.0440 | 0.5968 | 0.7989 | 0.2505 | 0.7677 | 0.9936 | 0.9909 | 0.8981 | 0.0408 | 0.0251 | 0.5508 | 0.7850 | 0.2681 | 0.9955 | 0.9145 | 0.0309 | 0.4723 | 0.8099 | |||
2024-08-23 | Model selection | 0.6349 | 0.6939 | 0.5852 | 0.6882 | 0.6103 | 0.7066 | 0.6598 | 0.7071 | 0.7335 | 0.7765 | 0.5887 | 0.7639 | 0.4127 | 0.9911 | 0.8972 | 0.0385 | 0.4413 | 0.7453 | 0.5033 | 0.9720 | 0.8676 | 0.1050 | 0.6307 | 0.5438 | 0.4814 | 0.9902 | 0.9115 | 0.0499 | 0.4458 | 0.6897 | 0.4375 | 0.9838 | 0.8792 | 0.0583 | 0.5309 | 0.6849 | 0.7981 | 0.9137 | 0.4724 | 0.3961 | 0.9983 | 0.9547 | 0.0182 | 0.2410 | 0.7874 | 0.5483 | 0.9956 | 0.9464 | 0.0397 | 0.3215 | 0.7539 | 0.3646 | 0.2564 | 0.9963 | 0.9611 | 0.9255 | 0.4121 | 0.0654 | 0.3682 | 0.8577 | 0.4395 | 0.9985 | 0.9604 | 0.0182 | 0.2331 | 0.8237 | |||
2024-08-22 | PixOOD w/ ResNet-101 DeepLab | 0.6119 | 0.5866 | 0.6395 | 0.5688 | 0.4895 | 0.6199 | 0.5514 | 0.5318 | 0.6487 | 0.6496 | 0.6320 | 0.6328 | 0.6905 | 0.8930 | 0.7987 | 0.2852 | 0.5393 | 0.3166 | 0.7060 | 0.9099 | 0.8410 | 0.0887 | 0.6267 | 0.2094 | 0.6800 | 0.9261 | 0.8299 | 0.2018 | 0.5079 | 0.3790 | 0.8014 | 0.8365 | 0.8263 | 0.1796 | 0.5598 | 0.2676 | 0.3551 | 0.8430 | 0.3899 | 0.4486 | 0.9848 | 0.8586 | 0.0900 | 0.5117 | 0.5908 | 0.5739 | 0.9719 | 0.8848 | 0.1249 | 0.5064 | 0.4618 | 0.2963 | 0.5869 | 0.9849 | 0.9901 | 0.8268 | 0.1027 | 0.0263 | 0.5748 | 0.6972 | 0.4857 | 0.9753 | 0.8511 | 0.1386 | 0.5415 | 0.5315 | |||
2024-08-22 | PixOOD w/ DeepLab Decoder | 0.5937 | 0.4606 | 0.8349 | 0.3829 | 0.5583 | 0.4894 | 0.5071 | 0.7487 | 0.5615 | 0.4362 | 0.5638 | 0.3640 | 0.1670 | 0.9616 | 0.6659 | 0.2212 | 0.7759 | 0.7167 | 0.3461 | 0.9684 | 0.7937 | 0.0964 | 0.6462 | 0.6126 | 0.2292 | 0.9777 | 0.7735 | 0.1550 | 0.6607 | 0.6898 | 0.2801 | 0.9655 | 0.7431 | 0.1907 | 0.6893 | 0.7016 | 0.7238 | 0.9332 | 0.3563 | 0.3037 | 0.9835 | 0.8200 | 0.0778 | 0.6100 | 0.6630 | 0.2015 | 0.9788 | 0.7738 | 0.1340 | 0.7453 | 0.7161 | 0.1998 | 0.8557 | 0.9826 | 0.9969 | 0.7863 | 0.0991 | 0.0081 | 0.6673 | 0.7285 | 0.1524 | 0.9725 | 0.7236 | 0.1262 | 0.7987 | 0.7145 | |||
2024-08-20 | PixOOD | 0.5347 | 0.4038 | 0.7909 | 0.2588 | 0.5213 | 0.4291 | 0.4235 | 0.6670 | 0.4997 | 0.4837 | 0.4943 | 0.3640 | 0.1245 | 0.9408 | 0.6113 | 0.2961 | 0.8944 | 0.6389 | 0.3734 | 0.9519 | 0.7820 | 0.1105 | 0.7193 | 0.5511 | 0.2145 | 0.9685 | 0.7580 | 0.2177 | 0.7589 | 0.6480 | 0.2323 | 0.9519 | 0.7157 | 0.2965 | 0.7730 | 0.6293 | 0.7332 | 0.9057 | 0.5074 | 0.2598 | 0.9777 | 0.7880 | 0.0929 | 0.6897 | 0.6483 | 0.2611 | 0.9709 | 0.7645 | 0.1478 | 0.7201 | 0.6711 | 0.1723 | 0.9261 | 0.9745 | 0.9976 | 0.7387 | 0.1276 | 0.0064 | 0.7710 | 0.7038 | 0.1574 | 0.9653 | 0.6903 | 0.1226 | 0.8026 | 0.6938 | |||
2024-08-24 | Physically Feasible Semantic Segmentation | 0.3364 | 0.6630 | 0.2253 | 0.6486 | 0.6092 | 0.6610 | 0.6254 | 0.2090 | 0.6851 | 0.6813 | 0.4311 | 0.7096 | 0.3930 | 0.9860 | 0.8769 | 0.0473 | 0.5154 | 0.6949 | 0.4952 | 0.9719 | 0.8664 | 0.1018 | 0.6132 | 0.5136 | 0.4072 | 0.9888 | 0.8896 | 0.0423 | 0.4933 | 0.6813 | 0.4195 | 0.9758 | 0.8503 | 0.0679 | 0.5750 | 0.6317 | 0.4226 | 0.8139 | 0.9070 | 0.3981 | 0.9936 | 0.9120 | 0.0235 | 0.4032 | 0.6930 | 0.4353 | 0.9913 | 0.9099 | 0.0444 | 0.4833 | 0.7043 | 0.4009 | 0.1148 | 0.9941 | 0.8564 | 0.9167 | 0.0278 | 0.5831 | 0.3907 | 0.7449 | 0.4278 | 0.9951 | 0.9290 | 0.0268 | 0.3903 | 0.7306 |