L2
Environment builds
the environment builds and runs; numbers not yet re-checked
SimCSE: Simple Contrastive Learning of Sentence Embeddings (Gao et al., EMNLP 2021) — unsup BERT-base STS Avg Spearman
Automated re-run of the headline result of 'SimCSE: Simple Contrastive Learning of Sentence Embeddings (Gao et al., EMNLP 2021) — unsup BERT-base STS Avg Spearman' (arXiv:2104.08821) from its own repository. Pre-registered claim: sts_avg_spearman = 76.25 (±5%). Hub verdict: TIMEOUT.
Claims
· unverified
c1
performance
The paper's own code (https://github.com/princeton-nlp/SimCSE) reproduces sts_avg_spearman = 76.25 for 'SimCSE: Simple Contrastive Learning of Sentence Embeddings (Gao et al., EMNLP 2021) — unsup BERT-base STS Avg Spearman'.
system nlp-simcse-sts-spearman
metric sts_avg_spearman
value 76.25
unit sts_avg_spearman
higher_is_better True
hardware cpu-box
Artifacts 2 files · code, data, logs — integrity-checked
| role | location | size | integrity |
|---|---|---|---|
| code | https://github.com/princeton-nlp/SimCSE | — | unchecked |
| paper | https://arxiv.org/pdf/2104.08821 | — | unchecked |
Verification runs 2 run(s) · mode script · 1 machine-checked assertions
failed · runner hub-local · level→L2 · 2026-07-02T07:47:21Z
runner log →
timeout · runner hub:repro-study · level→L2 · 2026-07-02T07:47:21Z
runner log →