Text classification is the single highest-volume task type on trAIn — more submissions flow through it than any other category. It's also where the most avoidable rejections happen. The good news: nearly every rejection traces back to a small set of fixable habits.
This guide covers what separates trainers who pass review on the first try from those who don't.
Read the instructions twice before labeling once
Every campaign comes with its own labeling guidelines, and they override your instincts. A label that feels obvious ("this is clearly negative sentiment") can be wrong if the client defines "negative" differently — for example, counting sarcasm as neutral, or excluding product complaints that mention a competitor.
Before your first submission on any campaign:
- Read the full instruction set, including the examples
- Note every explicit exclusion ("do not label X as Y")
- Check whether uncertain cases get their own label or get skipped
Five minutes of reading saves an hour of reworked submissions.
Consistency beats cleverness
Reviewers (and the inter-annotator agreement checks behind them) care less about whether your label is defensible and more about whether it's consistent. If you labeled "the battery died after two hours" as a complaint about battery life, then "battery life is terrible" must get the same label — even if you personally would phrase the categories differently.
A practical trick: keep a note of the tricky decisions you made early in a campaign. When a similar case appears 40 tasks later, match your own precedent.
The four edge cases that cause most rejections
1. Mixed signals. A review that praises the product but trashes the shipping. The instructions will usually tell you which signal wins — if they don't, look for a "mixed" or "multiple" label rather than guessing.
2. Neutral statements that feel emotional. "I guess it works fine" reads lukewarm, but unless the guidelines count resignation as negative, it's neutral. Label what the text says, not what it implies.
3. Off-topic content. Spam, ads, and unrelated text inside a dataset are usually deliberate quality checks. Label them exactly as the instructions say — these are often the items used to compute your accuracy score.
4. Near-duplicate categories. "Bug report" vs. "feature request," "billing issue" vs. "refund request." When two categories overlap, find the single example in the instructions that separates them and anchor on it.
Speed comes after accuracy, not before
New trainers often rush to build volume. On trAIn, that backfires: submissions are paid when approved, and your accuracy rating determines which higher-paying tasks you unlock. A slower first hundred submissions at 98% accuracy is worth far more than a fast hundred at 85%.
Once your consistency is proven, speed follows naturally — you stop re-reading instructions because the decisions are already made.
What to do when you disagree with a rejection
Rejections happen to everyone. When one lands:
- Re-read the instruction section that covers your case
- If the instructions support the reviewer, adjust and move on
- If they genuinely support your label, flag it through the task's feedback channel with a specific reference to the guideline — "per the examples section, item 3" is far more effective than "I think this was right"
Every campaign you complete cleanly raises your accuracy rating, and your rating is what unlocks the better-paying queues. If you haven't yet, check which certifications open up higher-rate text tasks, and browse what's live on the tasks page.
The trainers earning the most on text classification aren't the fastest — they're the most boring. Same guidelines, same decisions, every time.