New Sapien Task: Multi-Choice Error Review

Summary

The Sapien community has launched a new task called "Multi-Choice Assistant Error Review." In this task, users will evaluate pre-annotated errors, confirm if the classification is correct, and rate their confidence level across three difficulty settings. This provides new work for Sapiens, ensuring there is sufficient data available to complete over the holidays.

Hey there, Sapiens! 🚀

Just spoke to Santa and he brought us a new task! Get ready for Multi-Choice Assistant Error Review!

In this task you will assess annotated errors, judge whether the error has been classified properly, and rate your confidence.

There’s three difficulty levels!

We will also ensure that there will be enough data to work through across the holidays!

Happy tasking! https://app.sapien.io/t/dashboard

Discord sent us an expired image link.

View on Discord

12 views

The latest from Sapien

Sapien PoQ Challenge: Win a Tesla or $25K

Sapien invites US robotics and AI teams to submit a difficult dataset and define evaluation targets for a Proof of Quality challenge. Five teams will …

Where Human Judgment Matters Most in AI

Sapien reflects on lessons from two years and 195 million tasks measuring data quality, and asks where human judgment can have the greatest impact as …

Proof of Quality Scales AI Evaluation Workflows

Sapien introduces Proof of Quality, a workflow that brings multiple AI reviewers together so teams can compare scores, reasoning, agreement, and disagreement. This helps test …

Proof of Quality for City Intelligence

Sapien explains a key gap in city intelligence: AI can collect vast sensor data but decisions still depend on sparse expert judgment. The announcement links …

Speed Up Robot Training with Proof of Quality

Sapien explains how the review queue between processed robot data and the next model run controls retraining cadence. The announcement covers measuring collection processing and …