One of the things I despise about Windows is how much time developers have to spend working around problems in third-party firewall and anti-virus software.
Case in point: I’m seeing quite a few error reports about the new FeedDemon 3.0 beta that are caused by AV software locking files that FeedDemon is trying to access. Many AV programs monitor the file system for changed files, and when they find one, they scan it for viruses – temporarily locking it in the process. This has always been a problem, but it’s now an even bigger problem since FeedDemon stores your feed content in a single SQLite database file. Every time that database file changes, your AV software scans it to make sure it’s not infected.
In some cases this causes file permission errors when FeedDemon tries to access its database, but more often than not it simply slows FeedDemon down since it has to wait for the file lock to be released.
Most people report that the new FeedDemon is a lot faster than previous versions, so if you’re finding it slower, chances are your AV software is the culprit.
The good news is that you can probably stop this from happening. FeedDemon’s database uses the file extension .FDB, so if your AV software has an “exclusions” list, try adding the file extension .FDB to it. If you can’t do that, see if you can add FeedDemon’s cache folder (File | Manage Cache) to its list of excluded folders. Several customers who have done this have reported dramatic speed increases.
Of course, I can’t expect every FeedDemon customer to do this, so I’m looking into ways to work around these issues. But in the meantime, give it a shot, and reply here if it makes a difference.
Maybe AV programs all ship with exclusions for the exact filenames Firefox uses, but despite it being sqlite-happy these days I haven’t heard about people having problems with the bookmarks/history db, or the cookies db, or the cookie permissions db, or the download history db, or the per-site zoom db, or the form history db, or the web app local storage db, or, or… being locked by AV, which makes me wonder if they all ship ignoring *.sqlite and it’s just because they don’t know what an .FDB might be that they’re constantly getting in your way.
I had the same thought as Phil – I can’t believe sqllite is having this problem if it’s so popular. What about adding support for .sqlite extension, just during the beta at least, to see if the problem clears up?
Changing *anything* in the anti-virus software is out of the question on IT-managed systems, like here where we are running TrendMicro.
“Update all feeds” is definitely slower than in 2.8RC2.
What about locking the file (and keeping it locked) yourself?
SSDD. We fight the same battles with the same crappy firewalls and AV products. Drives me nuts. A vary large proportion of all support requests end up being related to firewall or AV products.
For the record; I hate them too!
@Phil: I considered the same thing myself, but I haven’t seen *.sqlite listed as a default exclusion in popular virus software. A number of other popular programs are affected by this (including Google Gears – see http://code.google.com/p/gears/issues/detail?id=370&redir=1 ), so I’m wondering whether Firefox is doing something to specifically handle this problem?
In Firefox’ case, maybe the file access activity is significantly lower than FD’s so it’s not noticeable? Or maybe there’s some caching going on that’s held until the file lock clears?
@Mladen: Locking the DB and journal files might be the best option – this appears to be what Firefox does with places.sqlite. I’ll give this a shot.
@Jack: Not sure. Firefox’s ‘places.sqlite’ seems to get updated quite frequently. As I noted in my previous comment, though, it looks like Firefox places an exclusive lock on that DB, which may be why it doesn’t suffer from the same AV problems I’m seeing.
I’m also using SQLite in one of my product and have the same problem. I ended up with encapsulating all database access in methods that will wait for a little while and retry if they fail with certain errors. Not ideal, but seems to work OK.
Please let us know if your solution with locking the file(s) helps!
Threads are your friend – that is unless the anti virus app actually suspends your app while it does it’s evil deeds. (I hate you Cisco Security Agent)
Suggestions:
* Change the file open flags (share for reading) when you open your DB file
* move all your DB file access into it’s own thread
* move your DB file to an unnamed memory mapped file and then in a worker thread, write it out to disk when it’s data changed.
@Brian: I added “PRAGMA locking_mode=EXCLUSIVE” to the latest beta, and it appears to have made a difference. I have one error report that shows a SQLITE_CANTOPEN error, whereas I have dozens of SQLITE_CANTOPEN errors from the previous build (and that one error may be a red herring).
Good suggestions, Jason. However, moving all DB access to its own thread adds a level of complexity (esp. with regards to UI updates) that would most likely cause more problems that it’s worth.
The memory-mapped file idea could work with non-critical data, but since an app/OS crash could mean that data is never written to the DB, I wouldn’t trust it for critical data. I’d rather have a handful of problems with AV software than hundreds caused by OS instability :)
@Brian: BTW, something I did in previous builds that seemed to help was to use “PRAGMA journal_mode=PERSIST” to tell SQLite to reuse the transaction journal file. This way AV software doesn’t see a new file being created with each SQLite transaction, giving it less reason to get in the way.
Hi Nick,
thank you for the tips, I’m going to give them a try!
Best Regards,
Brian
I got Cyberdefender, and it worked great as a free scanner with spyware and trojan removal. If it finds a virus, you need the upgrade, which I got since I liked the speed and user interface of the scanner. The Cyberdefender anti-virus works great, and the paid version I got also came with 24/7 computer help line with my wife found helpful while I was away. I found out Cyberdefender is a NASDAQ campany and they have a great product.