In a support-classification project, we tested how incoming requests should be grouped before touching live routing. Reading the full issue text exposed decisions that short summaries had hidden. Some requests fit no available category, so leaving them unclassified was the correct result.
We revised the instructions, re-ran the same cases, and kept the workflow disabled. That proved a narrow point: the revised rules handled those development cases more consistently. It did not prove production accuracy. The next step was a reviewed pilot, not automatic action on customers.
Your first improvement can be a better decision rule and a clear test. You do not have to switch everything on to make progress.
From our article Where to start with AI automation.