The Value of Automated Error Reporting

Published on by Nick Bradbury

The past few releases of FeedDemon have an included an error reporting feature which captures unexpected problems and sends us detailed reports on what went wrong.  If you’ve never seen this feature in action (and I hope you haven’t), it looks like this:

 

I’ve been going over the past few months of error reports, and much to my surprise, I discovered that the top three most common problems were never reported in our support forums.  If I didn’t add error-reporting to FeedDemon, it’s possible that I’d never have known about these bugs – so if you’re a software developer and you don’t have a similar feature, perhaps that will convince you to add one!

The downside to having an error reporting feature is the dismay you’ll feel if you discover – as I did – that a huge number of problems aren’t your fault, but are instead caused by third-party software (especially anti-virus programs, buggy graphics drivers, and flaky web browsers).   These are the most frustrating bugs because your code isn’t wrong, and you can only hack together a workaround rather than truly fix them.

Anyway…thanks to the error reporting feature, I’ve identified and fixed the top 10 most reported errors.  All of these fixes are included in FeedDemon 2.8 Beta 2, which was released today.

14 thoughts on “The Value of Automated Error Reporting”

  1. You have mentioned previously that you use EurekaLog, but I wondered if there was any additional detail that you would be willing to share about implementing error reporting and some of your challenges with that.

  2. @Nino: Really, the biggest challenge has been how to manage the incoming error reports. The way we set it up, FeedDemon’s error reporting uses an email account that’s tied to FogBugz – this way, the error report automatically gets a case#, etc., in our bug-tracking system.
    After an error report is added to FogBugz, it shows up in the RSS feed I use to track these cases. I don’t usually deal with them as they arrive, but that does enable me to get a feel for what the biggest problems are.
    Then every couple of weeks I’ll go through all the open error reports, figure out which problems have already been fixed, which ones still need to be fixed, etc.

  3. You’re absolutely right about the value of crash report handling – anybody shipping client software with out it should be run out of town! I too am continually amazed at the amount of code I’ve had to write over the years just to work around bugs in anti-virus software, graphics drivers, Microsoft libraries, etc.

  4. That’s great – I haven’t seen it, but it’s nice to know it’s happening. Of all the apps I use yours is way out in front when it comes to what you do to develop and maintain the users’ confidence in the developer, partly with the forum but mainly with this blog. That will be important if the app ever goes through any phases when it doesn’t work so well in the future – when I’ve had that problem with other apps in the past and I’ve abandoned them it’s not been because of the inconvenience or the length of time of the problems, but because I’ve lost confidence that they were going to get fixed any time soon. This is compounded when the developers take a ‘nothing to do with me’ attitude to problems caused by third-part software, basically saying ‘it’s not my fault that my product is a drag to use for you now, get rid of that other software, you should love mine more’. It’s amazing how many take that attitude, and I stop using their products, because there are others that don’t have the same problems, perhaps because they incorporate workarounds like you do.
    It might be interesting to hear what browsers, antivirus and graphics drivers are problematic some time, if they’re persistent. For instance my av is Avast, and that definitely has occasional flaky periods, although not severe enough or long enough for me to try something else before they get fixed, and it’s had a flaky period recently, although it’s just had an update, so fingers crossed.

  5. Done the same myself, it’s exceedingly useful. One of the things I love about FeedDemon is that I haven’t seen that dialog yet ;-)
    I’d be interested to know what information gets sent and what you find useful. I’ve done my own hand-rolled .NET exception logger and the stack trace is generally the most useful information but it’d be interesting to know if I’m missing something that could be useful.
    Also, I don’t see a way to provide an email address, do you not find you see errors and have no idea how they happened and can be reproduced?
    And finally, how does the error get sent? I’ve opted for posting the error to a website that then emails it to me, since configuring email on the user’s machine seems like a right pain.

  6. @Doogal: The stack trace is certainly the most useful piece of info, but it’s also helpful to know things like available memory, Windows/IE version and the last browsed URL. The screenshot has also been surprisingly helpful.
    I also popup a small dialog asking for a description of what the customer was doing when the error occured (most people don’t fill this out, but enough do to make it worthwhile).
    The way I did it, the error report gets sent using your default email client, so I don’t have to futz around with configuring SMTP. You also get to see exactly what you’re sending, and we know how to contact you if we need more info.

  7. Some time ago we added logging of all exceptions, warning messages and their contexts to an external txt file that periodically gets uploaded during the daily update check, with the customer’s permission and without customer information other than the serial number. Should have done it a very long time ago. Among other things, it has helped in identifying problems induced by third-party vendors for which we took the hit.

  8. @Keir: It’s hard to say which AV software causes the most problems since that’s probably determined more by the popularity of the AV software than by the bugs themselves. Likewise with graphics cards: NVIDIA and ATI are among the most popular, so they most likely result in the most problem reports.
    Browser-wise, the biggest problem is buggy IE add-ons, followed by people using beta versions of IE and thinking the IE beta bugs are FeedDemon’s (since FeedDemon embeds IE). For example, I added page transitions to v2.8, but there’s a bug in the IE8 beta that causes page transitions to fail, which has resulted in a number of bug reports coming my way.
    As an aside, one of the more annoying problems with AV software is false positives, where a virus definition update incorrectly flags our software, leading to people thinking our download is infected.

  9. Is that a Google Chrome like theme for Windows you’re using for the screenshot, or is FeedDemon switching to Chrome as its embedded browser (which probably still wouldn’t explain the UI).

Comments are closed.