corpus.blog
41,554 blogs · 4,981,167 posts Claim your blog
Record · corpus.blog/posts/01a087de-bae5-700e-ba31-87a51b04b614

Writing evaluation code for my Masterarbeit

serhii.net · published 28 January 2024

A record is what was published and who pointed at it. The text of the post is not held here: read it at the source, or at its Wayback capture.

Read the post at serhii.net Wayback capture
0blogs citing
First cited
—
Most recent
—
Rank this month
not ranked
External links
48
Words captured
4,977

Cited by

No blog we hold has cited this post yet.

Links from this post

Link
Host
https://github.com/aviaefrat/lmentry/blob/main/lmentry/predict.py
github.com
https://github.com/openai/evals
github.com
https://github.com/EleutherAI/lm-evaluation-harness
github.com
https://github.com/openai/evals?tab=readme-ov-file
github.com
https://github.com/openai/evals/blob/main/docs/build-eval.md
github.com
https://github.com/openai/evals/blob/main/evals/registry/data/README.md
github.com
https://github.com/openai/evals/tree/main/evals/registry/data
github.com
https://github.com/openai/evals/pull/1001
github.com
https://github.com/openai/evals/blob/main/evals/registry/data/ukraine_eit/samples.jsonl
github.com
https://github.com/openai/evals/tree/main/evals/registry/data/ukraine_gec
github.com
https://github.com/openai/evals/blob/main/evals/registry/completion_fns/langchain_llms.yaml
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/new_task_guide.md
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/1710b42d52d0f327cb0eb3cb1bfbbeca992836ca/lm_eval/tasks/toxigen/utils.py
github.com
https://huggingface.co/datasets/swag/viewer/regular
huggingface.co
https://github.com/EleutherAI/lm-evaluation-harness/blob/1710b42d52d0f327cb0eb3cb1bfbbeca992836ca/lm_eval/tasks/swag/swag.yaml
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/1710b42d52d0f327cb0eb3cb1bfbbeca992836ca/lm_eval/tasks/truthfulqa/truthfulqa_mc1.yaml
github.com
https://huggingface.co/datasets/truthful_qa/viewer/multiple_choice?row=0
huggingface.co
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/examples/lm-eval-overview.ipynb
github.com
https://huggingface.co/datasets/cais/mmlu
huggingface.co
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/interface.md
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/decontamination.md
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/lm_eval/evaluator.py
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/scripts/cost_estimate.py
github.com
https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
huggingface.co
https://hub.zenoml.com/project/aeac24ae-82af-467a-bbcc-56d9be0134a5/TruthfulQA/explore?params=eyJtb2RlbCI6IklDQlUtTlBVX19GYXNoaW9uR1BULTcwQi1WMS4xIiwibWV0cmljIjp7ImlkIjoyMTQ2LCJuYW1lIjoiYWNjdXJhY3kiLCJ0eXBlIjoibWVhbiIsImNvbHVtbnMiOlsibWMyIl19LCJjb21wYXJpc29uTW9kZWwiOiJBSURDLWFpLWJ1c2luZXNzX19NYXJjb3JvbmktNzBCLXYxIiwiY29tcGFyaXNvbkNvbHVtbiI6eyJpZCI6ImYwYzk4ODJkLTA4MGMtNGUyMy1hMjUxLWJkYTRiOWE4ZmM5ZCIsIm5hbWUiOiJtYzIiLCJjb2x1bW5UeXBlIjoiRkVBVFVSRSIsImRhdGFUeXBlIjoiQ09OVElOVU9VUyIsIm1vZGVsIjoiSUNCVS1OUFVfX0Zhc2hpb25HUFQtNzBCLVYxLjEifSwiY29tcGFyZVNvcnQiOltudWxsLHRydWVdLCJtZXRyaWNSYW5nZSI6WzAsMC45ODk0MTE2MTAxMjA1MTYxXSwic2VsZWN0aW9ucyI6eyJtZXRhZGF0YSI6e30sInNsaWNlcyI6W10sInRhZ3MiOltdfX0=
hub.zenoml.com
https://platform.openai.com/docs/guides/text-generation/chat-completions-api
platform.openai.com
https://platform.openai.com/playground?mode=chat
platform.openai.com
https://huggingface.co/datasets/cbt
huggingface.co
https://gitlab.kit.edu/udgtb/llm-performance-testing
gitlab.kit.edu
https://github.com/AUGMXNT/llm-experiments
github.com
https://gitlab.kit.edu/udgtb/llm-performance-testing/-/blob/main/LLMPerformanceTest.Dockerfile?ref_type=heads
gitlab.kit.edu
https://github.com/bjoernpl/lm-evaluation-harness-de/blob/master/Dockerfile
github.com
https://github.com/bigcode-project/bigcode-evaluation-harness/blob/main/Dockerfile
github.com
https://github.com/AUGMXNT/llm-experiments/blob/main/01-lm-eval.md
github.com
https://docs.google.com/spreadsheets/d/1kT4or6b0Fedd-W_jMwYpb63e1ZR3aePczz3zlbJW-Y4/edit
docs.google.com
https://bellard.org/ts_server/
bellard.org
https://github.com/EleutherAI/lm-evaluation-harness/issues/1224
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/big-refactor/lm_eval/tasks/xstorycloze/default_ru.yaml
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/big-refactor/lm_eval/tasks/xstorycloze/default_ar.yaml
github.com
https://huggingface.co/datasets/juletxara/xstory_cloze/viewer/en?row=9
huggingface.co
https://github.com/EleutherAI/lm-evaluation-harness/blob/big-refactor/lm_eval/tasks/unscramble/cycle_letters.yaml
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/main/docs/task_guide.md
github.com
https://github.com/EleutherAI/lm-evaluation-harness/issues/1274
github.com
https://github.com/EleutherAI/lm-evaluation-harness/issues/1239
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/25a15379676c8a2fa0b93ca9c4742b156e1fec39/lm_eval/tasks/gsm8k/gsm8k-cot.yaml
github.com
https://github.com/aviaefrat/lmentry/blob/main/lmentry/scorers/first_letter_scorer.py
github.com
https://github.com/robinhad/kruk
github.com
https://github.com/EleutherAI/lm-evaluation-harness/blob/e0eda4d3ffa10e5f65e0976161cd134bec61983a/lm_eval/tasks/bbh/cot_zeroshot/_cot_zeroshot_template_yaml
github.com
Elsewhere on serhii.net (3)