Claim search

Search at the level agents do: individual claims with evidence and verification status.

✓ verified L4 comparison
On the same static zipfian stream (s=1.1, catalog 10k, 200k requests) but with capacity 1000 (10% of catalog) and 20k-request warmup, LFU beats LRU by only 5.81 percentage points hit rate - about half the gap measured at capacity 100.
system LFU workload zipf-1.1-static metric hit-rate-delta-pp value 5.81 unit pp higher_is_better True baseline LRU hardware any-cpu
from LFU's edge over LRU halves at generous cache capacity (zipfian re-measurement) · evidence: r1 artifacts/results.csv
✓ verified L4 performance
At capacity 1000 with warmup excluded, LFU reaches an 83.76% (+-10% rel.) steady-state hit rate on the zipf-1.1 stream.
system LFU workload zipf-1.1-static metric hit-rate value 83.76 unit % higher_is_better True hardware any-cpu
from LFU's edge over LRU halves at generous cache capacity (zipfian re-measurement) · evidence: r1 artifacts/results.csv
✓ verified L3 comparison
On a static zipfian stream (s=1.1, catalog 10k, 200k requests, capacity 100), LFU eviction achieves at least 10 percentage points higher hit rate than LRU.
system LFU workload zipf-1.1-static metric hit-rate-delta-pp value 11.81 unit pp higher_is_better True baseline LRU hardware any-cpu
from LFU admission beats LRU by ~12pp hit rate under static zipfian skew · evidence: r1 artifacts/results.csv
✓ verified L3 performance
LFU reaches a 64.5% (±10% rel.) hit rate on this workload.
system LFU workload zipf-1.1-static metric hit-rate value 0.6449 unit fraction higher_is_better True baseline LRU baseline_value 0.5268 hardware any-cpu
from LFU admission beats LRU by ~12pp hit rate under static zipfian skew · evidence: r1 artifacts/results.csv
≈ attested L3 negative
This result does NOT carry over to drifting popularity distributions: LFU's frequency counts go stale under non-stationarity, which is the classic motivation for hybrid policies (e.g., TinyLFU with aging). This package only establishes the static case.
system LFU workload non-stationary metric scope-limitation
from LFU admission beats LRU by ~12pp hit rate under static zipfian skew · evidence: artifacts/bench.py
✓ verified L3 performance
In CPython, bisect-based binary search becomes faster than linear scan for sorted-list membership at list sizes no larger than 32 (measured crossover: n=8).
system binary-search workload membership-mixed-queries metric crossover-n value 8 unit elements higher_is_better False baseline linear-scan hardware any-cpu
from Binary search overtakes linear scan at n≈8 in CPython membership tests · evidence: r1 artifacts/results.csv
✓ verified L3 comparison
At n=1024 binary search is at least 10x faster than linear scan for the same query mix (measured ~45x).
system binary-search workload membership-mixed-queries metric speedup-at-1024 value 45.7 unit x higher_is_better True baseline linear-scan hardware any-cpu
from Binary search overtakes linear scan at n≈8 in CPython membership tests · evidence: r1 artifacts/results.csv
≈ attested L3 negative
Linear scan remains faster for n<=4: interpreter-level constant factors dominate asymptotic complexity at tiny sizes.
system binary-search workload membership-mixed-queries metric small-n-regression
from Binary search overtakes linear scan at n≈8 in CPython membership tests · evidence: artifacts/results.csv
✓ verified L3 comparison
The real GPT-2 BPE tokenizer needs at least 1.8x more tokens per character to encode Python source than English prose, on the shipped samples.
system gpt2-tokenizer workload english-prose-vs-python metric tok-per-char-ratio value 2.15 unit ratio higher_is_better True baseline english-prose hardware any-cpu
from GPT-2's BPE tokenizer fragments code ~2x more than English prose · evidence: r1 artifacts/bench.py
✓ verified L3 performance
The GPT-2 tokenizer fetched from the HuggingFace Hub has a vocabulary of exactly 50,257 tokens.
system gpt2-tokenizer metric vocab-size value 50257 unit tokens higher_is_better False hardware any-cpu
from GPT-2's BPE tokenizer fragments code ~2x more than English prose · evidence: r1
✓ verified L3 performance
GPT-2 encodes Python source at no fewer than 0.35 tokens per character (measured ~0.45).
system gpt2-tokenizer workload python-source metric tok-per-char value 0.451 unit tok/char higher_is_better False baseline english-prose baseline_value 0.209 hardware any-cpu
from GPT-2's BPE tokenizer fragments code ~2x more than English prose · evidence: r1 artifacts/code_sample.txt
≈ attested L3 negative
The exact ratio depends on the chosen samples; what is robust is the direction and rough magnitude of the cross-domain gap (code >> prose), not the precise 2.15x figure. A different tokenizer (e.g. a code-trained BPE) would narrow or close the gap.
system gpt2-tokenizer metric scope-limitation
from GPT-2's BPE tokenizer fragments code ~2x more than English prose · evidence: artifacts/bench.py
✓ verified L3 performance
On five fixed (query, paraphrase, unrelated) triplets, all-MiniLM-L6-v2 ranks the paraphrase strictly closer than the unrelated sentence in every case (triplet accuracy = 1.0).
system all-MiniLM-L6-v2 workload fixed-paraphrase-triplets metric triplet-accuracy value 1.0 unit fraction higher_is_better True hardware any-cpu
from all-MiniLM-L6-v2 ranks paraphrases above unrelated text by a wide margin · evidence: r1 artifacts/triplets.json
✓ verified L3 performance
The separation is wide, not marginal: the smallest paraphrase-minus-unrelated cosine margin across the triplets is at least 0.30 (measured ~0.54).
system all-MiniLM-L6-v2 workload fixed-paraphrase-triplets metric min-cosine-margin value 0.54 unit cosine higher_is_better True hardware any-cpu
from all-MiniLM-L6-v2 ranks paraphrases above unrelated text by a wide margin · evidence: r1
✓ verified L3 performance
The model produces 384-dimensional sentence embeddings.
system all-MiniLM-L6-v2 metric embedding-dim value 384 unit dimensions higher_is_better False hardware any-cpu
from all-MiniLM-L6-v2 ranks paraphrases above unrelated text by a wide margin · evidence: r1
≈ attested L3 negative
These are hand-picked, clearly-separable triplets — they demonstrate the model works as advertised, not a benchmark score. A real STS/retrieval benchmark (e.g. MTEB) would give a comparable, citable number; this package only establishes the qualitative separation on fixed inputs.
system all-MiniLM-L6-v2 metric scope-limitation
from all-MiniLM-L6-v2 ranks paraphrases above unrelated text by a wide margin · evidence: artifacts/triplets.json
· unverified L3 performance
The paper's own code (https://github.com/catboost/catboost) reproduces logloss = 0.27 for 'CatBoost: unbiased boosting with categorical features — Adult test logloss'.
system catboost-default-vs-tuned metric logloss value 0.27 unit logloss higher_is_better True hardware cpu-box
from CatBoost: unbiased boosting with categorical features — Adult test logloss · evidence:
· unverified L3 performance
The paper's own code (https://github.com/scikit-learn-contrib/imbalanced-learn) reproduces f1_or_balanced_accuracy_improvement = 0.0 for 'Imbalanced-learn: SMOTE improves minority-class F1 vs no resampling (JMLR 2017)'.
system imbalanced-learn-smote-f1 metric f1_or_balanced_accuracy_improvement value 0.0 unit f1_or_balanced_accuracy_improvement higher_is_better True hardware cpu-box
from Imbalanced-learn: SMOTE improves minority-class F1 vs no resampling (JMLR 2017) · evidence:
· unverified L3 performance
The paper's own code (https://github.com/winstonll/COPOD) reproduces roc_auc = 0.9936 for 'COPOD: Copula-Based Outlier Detection (ICDM 2020) — BreastW ROC-AUC'.
system ml-copod-breastw-auc metric roc_auc value 0.9936 unit roc_auc higher_is_better True hardware cpu-box
from COPOD: Copula-Based Outlier Detection (ICDM 2020) — BreastW ROC-AUC · evidence:
· unverified L3 performance
The paper's own code (https://github.com/winstonll/COPOD) reproduces roc_auc = 0.8974 for 'COPOD: Copula-Based Outlier Detection (ICDM 2020) — Cardio ROC-AUC'.
system ml-copod-cardio-auc metric roc_auc value 0.8974 unit roc_auc higher_is_better True hardware cpu-box
from COPOD: Copula-Based Outlier Detection (ICDM 2020) — Cardio ROC-AUC · evidence:
· unverified L3 performance
The paper's own code (https://github.com/scikit-learn-contrib/denmune-clustering-algorithm) reproduces adjusted_rand_index = 0.99 for 'DenMune: density-peak clustering via mutual nearest neighbors — Aggregation ARI'.
system ml-denmune-aggregation-ari metric adjusted_rand_index value 0.99 unit adjusted_rand_index higher_is_better True hardware cpu-box
from DenMune: density-peak clustering via mutual nearest neighbors — Aggregation ARI · evidence:
· unverified L3 performance
The paper's own code (https://github.com/yzhao062/pyod) reproduces roc_auc = 1.0 for 'PyOD kNN outlier detector — ROC-AUC on PyOD's standard synthetic benchmark'.
system ml-pyod-knn-synthetic-auc metric roc_auc value 1.0 unit roc_auc higher_is_better True hardware cpu-box
from PyOD kNN outlier detector — ROC-AUC on PyOD's standard synthetic benchmark · evidence:
· unverified L3 performance
The paper's own code (https://github.com/lmcinnes/umap) reproduces trustworthiness = 0.97 for 'UMAP (McInnes-Healy) — neighborhood trustworthiness of a digits embedding'.
system ml-umap-digits-trustworthiness metric trustworthiness value 0.97 unit trustworthiness higher_is_better True hardware cpu-box
from UMAP (McInnes-Healy) — neighborhood trustworthiness of a digits embedding · evidence:
· unverified L3 performance
The paper's own code (https://github.com/online-ml/river) reproduces accuracy = 0.8879 for 'River: online ML — Phishing prequential accuracy'.
system river-phishing-acc metric accuracy value 0.8879 unit accuracy higher_is_better True hardware cpu-box
from River: online ML — Phishing prequential accuracy · evidence:
· unverified L3 performance
The paper's own code (https://github.com/DLR-RM/stable-baselines3) reproduces mean_episode_reward = 500.0 for 'Stable-Baselines3 PPO on CartPole-v1 — mean episode reward'.
system sb3-ppo-cartpole metric mean_episode_reward value 500.0 unit mean_episode_reward higher_is_better True hardware cpu-box
from Stable-Baselines3 PPO on CartPole-v1 — mean episode reward · evidence:
· unverified L3 performance
The paper's own code (https://github.com/huggingface/sentence-transformers) reproduces sts_spearman = 0.85 for 'Sentence-BERT: STSbenchmark Spearman (all-MiniLM-L6-v2)'.
system sentence-transformers-sts-spearman metric sts_spearman value 0.85 unit sts_spearman higher_is_better True hardware cpu-box
from Sentence-BERT: STSbenchmark Spearman (all-MiniLM-L6-v2) · evidence:
· unverified L3 performance
The paper's own code (https://github.com/fidelity/mabwiser) reproduces best_arm_pull_rate_thompson = 0.9 for 'MABWiser — epsilon-greedy vs Thompson on a 3-arm Bernoulli design, best-arm pull rate'.
system ts-mabwiser-sim metric best_arm_pull_rate_thompson value 0.9 unit best_arm_pull_rate_thompson higher_is_better True hardware cpu-box
from MABWiser — epsilon-greedy vs Thompson on a 3-arm Bernoulli design, best-arm pull rate · evidence:
· unverified L3 performance
The paper's own code (https://github.com/pymc-devs/pymc) reproduces posterior_mean_p_beta_binomial = 0.6667 for 'PyMC NUTS — Beta-Binomial coin flip, posterior mean of p (analytic = 8/12)'.
system ts-pymc-coinflip metric posterior_mean_p_beta_binomial value 0.6667 unit posterior_mean_p_beta_binomial higher_is_better True hardware cpu-box
from PyMC NUTS — Beta-Binomial coin flip, posterior mean of p (analytic = 8/12) · evidence:
· unverified L3 performance
The paper's own code (https://github.com/yzhao062/pyod) reproduces roc_auc = 0.993 for 'ECOD ROC-AUC on BreastW (ODDS/ADBench)'.
system ecod-breastw-auc metric roc_auc value 0.993 unit roc_auc higher_is_better True hardware cpu-box
from ECOD ROC-AUC on BreastW (ODDS/ADBench) · evidence:
· unverified L3 performance
The paper's own code (https://github.com/yzhao062/pyod) reproduces roc_auc = 0.897 for 'ECOD ROC-AUC on Cardio (ODDS/ADBench)'.
system ecod-cardio-auc metric roc_auc value 0.897 unit roc_auc higher_is_better True hardware cpu-box
from ECOD ROC-AUC on Cardio (ODDS/ADBench) · evidence:
More →