If you have 70% accuracy

W&B Help
1

Greetings. Say you have a model with 70% accuracy for your use case:

  1. What are you supposed to do with the other 30%?
  2. What can you say to customers that got the 30%?
  3. I suppose every industry and every company will have their own standards, but among actual data science professionals, is this percentage ‘good enough’ to take into production?
  4. Perhaps that score can be compared to humans performing the same real world task. How much better than humans should it be, and at what cost in pre-training, domain adaptation, and task finetuning compared to the cost of wages & benefits?

I never see anyone talking about these issues in any detail or with any real world experience to back it up. Instead, I see a lot of “who’s biggest” flexing, which is already tiresome. Thanks for sharing your insight.

2

Hi @patientwriter, thank you for writing in and for your insightful questions. Given that this post triggered a support request but is more intended for an open conversation with the community, I will mark this as closed from the support side and keep it open for the community to provide feedback/input.