Once upon a time I was working on a system that was deployed on customers sites. That meant we had no regular access to the logs for the system which made getting feedback difficult.
One of the developers on the team added an exception logging table which would capture every exception raised by the system along with the full stack trace.
In order to debug a certain error the customer provided us with a full backup of their DB. In addition to fixing the specific issue I managed to get 2 days to look at the exceptions.
By looking at the frequencies I was able to put in place measures to handle 99% of these errors. Now most of them were transitory problems that the customer had never reported.
When the fixed version of the product was returned to the customer they were happy with it, but could not explain why.
In facts they were so happy that it took 2. years to convince them to take a new version (we had a 6 month release cycle at this time).