| Benchmark | Reflection 70B | Claude 3.5 Sonnet | Claude 3 Opus | GPT-4o | Gemini 1.5 Pro | Llama 3.1 405B |
|---|---|---|---|---|---|---|
| GPQA | 55.3% (0-shot Reflection) | 59.4%* (0-shot CoT) | 50.4% (0-shot CoT) | 53.6% (0-shot CoT) | — | 50.7% (0-shot) |
| MMLU | 89.9% (0-shot Reflection) | 88.7%** (5-shot) 88.3% (0-shot CoT) | 85.7% (0-shot CoT) | 88.7% (5-shot) 85.9% (0-shot CoT) | 87.3% (5-shot) 88.6% (0-shot CoT) | — |
| HumanEval | 91% (0-shot Reflection) | 92.0% (0-shot) | 84.9% (0-shot) | 90.2% (0-shot) | 84.1% | 89.0% (0-shot) |
| MATH | 79.7% (0-shot Reflection) | 71.1% (0-shot CoT) | 60.1% (0-shot CoT) | 76.6% (4-shot) | 67.7% | 73.8% (0-shot CoT) |
| GSM8K | 99.2% (0-shot Reflection) | 96.4% (0-shot CoT) | 95.0% (0-shot CoT) | — | 90.8% | 96.8% (8-shot CoT) |
| IFEval | 90.13% (0-shot Reflection) | — | — | 85.6% | — | 88.6% |
如何在线使用Reflection 70B模型？
按照以下简单步骤开始与Reflection 70B聊天。
Reflection 70B Features
🧠
Architecture
Built on the Llama-3.1 framework, incorporating special tokens like <thinking>, <reflection>, and <output> to structure the reasoning process.
📊
Training Data
Trained on synthetic data generated by Glaive, utilizing large datasets to enhance performance in natural language processing tasks.
🏆
Performance
Demonstrated superior performance across benchmarks such as MMLU, MATH, IFEval, and GSM8K, outperforming closed-source models like GPT-4o.
🎯
Reduced Hallucinations
Employs stricter control mechanisms during information verification stages to significantly reduce false information, enhancing user trust and reliability.
常见问题
Reflection-70B常见问题解答
- Reflection-70B是一个先进的开源语言模型，采用反思调整技术来减少幻觉并提高生成输出的准确性。
- 反思调整通过引入思考、反思和输出等特殊标记来教模型检测和纠正其推理错误。
- Reflection-70B在多个基准测试中表现出色，包括MMLU、MATH、IFEval和GSM8K，甚至超越了一些闭源模型如GPT-4o。
- 通过在信息验证阶段实施更严格的控制机制，Reflection-70B显著减少了错误信息的生成，提高了用户信任和可靠性。
- Reflection-70B的权重可以在Hugging Face上获取，并将通过Hyperbolic Labs发布API以便于集成到应用程序中。
- 一个更强大的版本Reflection-405B即将发布，预计将显著超越顶级专有模型。