# FollowIR · Instruction Following > Source: Benchize (https://benchize.com) — https://benchize.com/benchmarks/mteb%3AFollowIR. When citing or reusing this content, credit "Benchize" and link to https://benchize.com/benchmarks/mteb%3AFollowIR. Instruction-following retrieval quality, measuring how well models retrieve relevant documents when given detailed natural language instructions alongside queries. Canonical URL: https://benchize.com/benchmarks/mteb%3AFollowIR Last updated: Wed, 21 Oct 2026 18:52:20 GMT Reproduction and quotation must name Benchize, link to the canonical HTML URL, and retain individual source citations. Instruction-following retrieval quality, measuring how well models retrieve relevant documents when given detailed natural language instructions alongside queries. Embeddings · Models tested: 15 · Score publishers: [MTEB](https://arxiv.org/abs/2403.15246) ## Model results | # | Model | Score | | | --- | --- | --- | --- | | 1 | [Qwen3 Embedding](https://benchize.com/models/qwen3-embedding.md) | 0.12source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| 0.11559966666666666 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| 0.11559966666666666 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 1 scoresByTask {"Core17InstructionRetrieval":0.135289,"News21InstructionRetrieval":0.087106,"Robust04InstructionRetrieval":0.124404} scoresByTaskType {"InstructionReranking":0.11559966666666666} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label Qwen/Qwen3-Embedding-4B Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/0` | | | | | 2 | [Qwen3 Embedding](https://benchize.com/models/qwen3-embedding.md) | 0.1source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| 0.10064133333333332 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| 0.10064133333333332 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 3 scoresByTask {"Core17InstructionRetrieval":0.126043,"News21InstructionRetrieval":0.090212,"Robust04InstructionRetrieval":0.085669} scoresByTaskType {"InstructionReranking":0.10064133333333332} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label Qwen/Qwen3-Embedding-8B Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/3` | | | | | 3 | [Gemini Embedding 001](https://benchize.com/models/gemini-embedding-001.md) | 0.05source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| 0.05179366666666666 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| 0.05179366666666666 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 17 scoresByTask {"Core17InstructionRetrieval":0.076909,"News21InstructionRetrieval":0.102558,"Robust04InstructionRetrieval":-0.024086} scoresByTaskType {"InstructionReranking":0.05179366666666666} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label google/gemini-embedding-001 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/16` | | | | | 4 | [all-MiniLM-L12-v2](https://benchize.com/models/all-minilm-l12-v2.md) | -0.00752source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.007523000000000001 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.007523000000000001 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 83 scoresByTask {"Core17InstructionRetrieval":0.016885,"News21InstructionRetrieval":-0.003513,"Robust04InstructionRetrieval":-0.035941} scoresByTaskType {"InstructionReranking":-0.007523000000000001} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label sentence-transformers/all-MiniLM-L12-v2 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/82` | | | | | 5 | [E5-Large-v2](https://benchize.com/models/e5-large-v2.md) | -0.00994source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.009935666666666667 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.009935666666666667 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 78 scoresByTask {"Core17InstructionRetrieval":0.001202,"News21InstructionRetrieval":0.008256,"Robust04InstructionRetrieval":-0.039265} scoresByTaskType {"InstructionReranking":-0.009935666666666667} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label intfloat/e5-large-v2 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/77` | | | | | 6 | [GTE-Large](https://benchize.com/models/gte-large.md) | -0.01source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.012887333333333334 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.012887333333333334 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct -1 ### Reported statistics experiments Not reported rank 86 scoresByTask {"Core17InstructionRetrieval":-0.003123,"News21InstructionRetrieval":0.012578,"Robust04InstructionRetrieval":-0.048117} scoresByTaskType {"InstructionReranking":-0.012887333333333334} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label thenlper/gte-large Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/86` | | | | | 7 | [BGE Large EN V1.5](https://benchize.com/models/bge-large-en-v1-5.md) | -0.02source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.022854666666666665 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.022854666666666665 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 137 scoresByTask {"Core17InstructionRetrieval":-0.01927,"News21InstructionRetrieval":0.003309,"Robust04InstructionRetrieval":-0.052603} scoresByTaskType {"InstructionReranking":-0.022854666666666665} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label BAAI/bge-large-en-v1.5 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/136` | | | | | 8 | [Text Embedding 3 Small](https://benchize.com/models/text-embedding-3-small.md) | -0.02source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.02382847790034975 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.02382847790034975 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct -1 ### Reported statistics experiments Not reported rank 146 scoresByTask {"Core17InstructionRetrieval":0.011536596231932908,"News21InstructionRetrieval":-0.018085227846059017,"Robust04InstructionRetrieval":-0.06493680208692315} scoresByTaskType {"InstructionReranking":-0.02382847790034975} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label openai/text-embedding-3-small Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/145` | | | | | 9 | [GTE-Base](https://benchize.com/models/gte-base.md) | -0.02source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.02449933333333333 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.02449933333333333 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct -1 ### Reported statistics experiments Not reported rank 141 scoresByTask {"Core17InstructionRetrieval":-0.026231,"News21InstructionRetrieval":0.004136,"Robust04InstructionRetrieval":-0.051403} scoresByTaskType {"InstructionReranking":-0.02449933333333333} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label thenlper/gte-base Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/140` | | | | | 10 | [Text Embedding 3 Large](https://benchize.com/models/text-embedding-3-large.md) | -0.03source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.026765020547623527 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.026765020547623527 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct -1 ### Reported statistics experiments Not reported rank 158 scoresByTask {"Core17InstructionRetrieval":-0.0019630682229421848,"News21InstructionRetrieval":-0.02026237736978675,"Robust04InstructionRetrieval":-0.05806961605014164} scoresByTaskType {"InstructionReranking":-0.026765020547623527} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label openai/text-embedding-3-large Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/157` | | | | | 11 | [all-MiniLM-L6-v2](https://benchize.com/models/all-minilm-l6-v2.md) | -0.03source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.028395899216562482 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.028395899216562482 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 159 scoresByTask {"Core17InstructionRetrieval":-0.003451558105842423,"News21InstructionRetrieval":-0.0024698946482787643,"Robust04InstructionRetrieval":-0.07926624489556626} scoresByTaskType {"InstructionReranking":-0.028395899216562482} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label sentence-transformers/all-MiniLM-L6-v2 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/158` | | | | | 12 | [Multilingual-E5-Large](https://benchize.com/models/multilingual-e5-large.md) | -0.03source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.030542 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.030542 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 163 scoresByTask {"Core17InstructionRetrieval":-0.016235,"News21InstructionRetrieval":-0.000595,"Robust04InstructionRetrieval":-0.074796} scoresByTaskType {"InstructionReranking":-0.030542} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label intfloat/multilingual-e5-large Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/162` | | | | | 13 | [All Mpnet Base V2](https://benchize.com/models/all-mpnet-base-v2.md) | -0.03source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.03078166666666667 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.03078166666666667 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 178 scoresByTask {"Core17InstructionRetrieval":-0.007358,"News21InstructionRetrieval":-0.017918,"Robust04InstructionRetrieval":-0.067069} scoresByTaskType {"InstructionReranking":-0.03078166666666667} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label sentence-transformers/all-mpnet-base-v2 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/177` | | | | | 14 | [Bge m3](https://benchize.com/models/bge-m3.md) | -0.03source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.031102328698400197 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.031102328698400197 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 182 scoresByTask {"Core17InstructionRetrieval":-0.012533158568791294,"News21InstructionRetrieval":-0.013922666099468859,"Robust04InstructionRetrieval":-0.06685116142694043} scoresByTaskType {"InstructionReranking":-0.031102328698400197} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label BAAI/bge-m3 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/181` | | | | | 15 | [E5-Base-v2](https://benchize.com/models/e5-base-v2.md) | -0.04source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.03936266666666667 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.03936266666666667 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 199 scoresByTask {"Core17InstructionRetrieval":-0.029186,"News21InstructionRetrieval":-0.020822,"Robust04InstructionRetrieval":-0.06808} scoresByTaskType {"InstructionReranking":-0.03936266666666667} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label intfloat/e5-base-v2 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/198` | | | | | 16 | [BGE Base EN V1.5](https://benchize.com/models/bge-base-en-v1-5.md) | -0.04source score | | | ### Original source values \| Metric \| Value \| Unit \| Scope \| Direction \| \| --- \| --- \| --- \| --- \| --- \| \| `mean_task` \| -0.039828333333333334 \| source_score \| source_published_aggregate \| Higher is better \| \| `mean_task_type` \| -0.039828333333333334 \| source_score \| source_published_aggregate \| Higher is better \| ### Configuration zeroShotPct 100 ### Reported statistics experiments Not reported rank 200 scoresByTask {"Core17InstructionRetrieval":-0.034175,"News21InstructionRetrieval":-0.009961,"Robust04InstructionRetrieval":-0.075349} scoresByTaskType {"InstructionReranking":-0.039828333333333334} trainedOnTasks Not reported ### Source [Original source](https://mteb-leaderboard-backend.hf.space/v1/benchmarks/FollowIR/scores) Source MTEB Original model label BAAI/bge-base-en-v1.5 Observed 2026-10-21T16:40:36.107Z Evidence SHA-256 `9727e715325c0f35c67173e5a59d96566fb3dded3f9f1558f9a7ddf230445ddb` Source locator `/rows/199` | | | | --- Cite as: FollowIR · Instruction Following — Benchize, https://benchize.com/benchmarks/mteb%3AFollowIR (retrieved 2026-10-21)