One of the benefits of using a desktop RSS reader instead of a web-based one is that you can read your feeds offline – a major plus if you’re frequently without a reliable Internet connection. FeedDemon is already a good offline reader, but I’d like to make it a great one.
One thing that’s annoying when working offline in the current version of FeedDemon is that you’re asked whether to go online every time you click a link to an external web page. That’s fine if you can easily go online, but it’s pointless when you don’t have a connection to the Internet (like when you’re on a plane, or attending a conference with spotty WiFi). To remedy this, I’m changing FeedDemon to check whether the page exists in the browser cache, and if it does, it will display the cached page instead of asking to go online.
This ties into a request I’ve received quite frequently: the ability to prefetch (download) links and image in new items as they’re received, so you can work offline and still view external images and links in those items. In the past I’ve shied away from this since I feared it would require too much storage and slow things down, but perhaps I’ve been thinking about it all wrong?
It seems to me that most people wouldn’t really need cached links and images available for every single item – it’s really only the unread items they want to read offline. So, instead of prefetching links and images every time your feeds are updated, what if FeedDemon simply provided a way to prefetch unread items right before going offline?
This is something I’m experimenting with right now, and it seems to work pretty well (and it’s pretty cool being able to visit web pages you haven’t seen before even when working offline). The only stumbling point I’ve found is that some web sites use cache control headers which interfere with offline viewing, but since most sites are cache-aware this doesn’t appear to be a major problem.
So…for all you offline users, would these additions make FeedDemon the perfect offline reader for you, or is there something else I need to do to make it even better for offline use?
I personally would much more like to see real multilevel folders to organize feeds. Not just the single level below “Subscriptions” as it is now.
I think this is a superb idea and I would love to see it in the next version of FeedDemon. I would also love a feature that will update my feeds and grab the content for all items I mark (I would only really want this for several feeds I read so an option to enable it that is not enabled by default would be perfect) when I go online (such as using WiFi at an airport or cafe).
This would be great – when you get it worked out pass the idea along to whoever is developing NetNewsWire for OSX, as that’s what I use on my powerbook and is where I could use the offline caching feature (I use feeddemon on my windows workstation which is never offline).
Yes!
I see a few levels of offline caching available and possibly configurable:
Images. I lose context when trying to read feeds with pictures such as Engadget or a how-to with example screenshots and diagrams.Full page for a summarized item. I often want to read sites with summaries such as NY Times or WSJ.Prefetch every link found.
I currently flag items I want to read later or open a browser window with each story in a separate tab so I can get back to it later. I often don’t get back to it, so offline viewing would be a definite improvement.
I think this will be an incomplete feature if it does not offer the ability to always prefetch web content. I’m not sure that it’s necessary to cache full web pages (that would be nice, of course), but particularly any images should be cached just like text currently is.
I wouldn’t worry about storage being a problem; I currently use an RSS reader on my Pocket PC called Egress that does a fantastic job of this. In fact, Egress would be a good program to investigate when looking at building this feature. It takes a bit longer to grab feeds than most feed readers, but it grabs images asynchronously, so all of the text information comes in immediately even if you don’t have time to let it grab all the images.
Egress also allows me to choose on a per-feed basis how many levels of pages I’d like to cache. So for my feeds that are not full-text feeds, I set it on, but to 0 levels deep. That way I can see the full post by clicking the post heading. Once you get above 1 level deep, you could be looking at some storage issues, but I’ve never bothered with this.
It’s amazing to have a Pocket PC application that can display feeds including images to me faster than any online or offline PC-based RSS feeder that I’ve tried. Of course, Egress is not without shortcomings, and I still also use NewsGator Go!, but the way it handles file caching is definitely worth exploring.
Niall, Engadget is a good example of a feed that would benefit from image caching.
Speaking of images, I’m thinking I might want to avoid caching any of your Flickr photos…
Jason, I see what you mean about Egress – its caching options look very nice, suprisingly so for a mobile app. I can’t say I’m wild about the idea of enabling caching items as they’re received, but it’s not hard to implement, so if enough people want this I’ll add it.
I would love another option — when offline, capture all clicks made to external links into a “load when online” list — when I’m back online, then give me the option to load some or all of them in my external browser (ie FF tabs).
That would be way cool.
This morning: 202 feeds, 11579 items, 2282 unread items and 139 flagged items (- and a low bandwidth GPRS connection ~33MB/day). Most folders update every 4-6 hours and a few every hour.
Bandwidth for FeedDemon/day 8000 – 11000KB with images off (-thanks for that one).
Of the 2282 unread items, I might open the browser for 10-20.
As you can see from the figures above – pre-fetching would not be a desirable. default behaviour for me.
On the other hand it might be fun to set individual feeds, or maybe feeds with flagged items to prefetch.
And as an FD addict, I can appreciate getting a stash in for lean times.
I think this feature would be great – one I’ve been desiring for quite some time. And the way you’re thinking of implemeting it sounds good to me.
I would add that enabling this on a folder or feed level would be CRITICAL. (Feed level preferably).
Most of the feeds I read would not necessarily need pre-fetching, so I would not want to waste time doing this.
For what it’s worth, my personal usage is scenario is: I sync up first thing in the morning before I leave the house. Then, I have a 1 hour bus ride to work where I read my articles. For a lot of these articles, the text is sufficient. For the articles that need to be read online, or for images inline (like Engadget, etc), I copy them to my News Bin, and then go over them once I get to the office and can get connected. I then have to go through my news bin, deleting all of the articles NOT of interest to me.
Thanks, Nick, for constantly looking to make an awesome program even better!
Also, I would like to add that if you’re looking for anyone to beta test this feature…sign me up! I would be using this feature twice a day – going to/coming from work.
If you cached items as they were received (cache all the unread items as they are received, not just when going offline) then eventually you will have all items cached and the user can go offline at any time and have everything available, even older items.
Obviously, this would require a lot of storage and bandwidth so it would probably need to be heavily user configured, but I would suspect users who are comfortable with offline reading would be up to the task of configuration.
Please get Brent to do with with NetNewsWire 3!
Hello Nick,
I’m novice user, still in trial mode, and loving your product.
To have a automatic caching would be great because it would make it faster to read through all the new feeds, without the need to wait for each new post to download every time (what can take some lenghty seconds even with my fast cable connection).
Hi Nick,
It’s been said before, but I’d strongly vote in favor of caching/prefetching all images of the unread items.
I very often travel by train and too often find myself coming back to a post just because the images were missing.
Regards
jr
I would love this functionality as I’m often reading offline and get frustrated that I can’t see the page someone has linked to.
I’ve voted for offline caching on the forum poll, so I’m surprised it is being implemented so quickly :-)
I, personally would suggest following caching levels:
0. No caching
1. Inline images
2. Follow only main link of a newsitem (with its inline styles/images)
2a. Support redirections (e.g. http://feeds.feedburner.com/~r/….)
3. Follow all links of a newsitem (with their inline styles/images) (within feed’s domain)
4. Follow all links of a newsitem (with their inline styles/images) (in any domains)
PS: And what with JavaScript on such pages?
Radek, I am planning to support redirections (I have to, or else links in countless FeedBurner feeds won’t be cached, as you point out), but I’m not planning to support prefetching either JavaScript or CSS, nor am I planning to prefetch anything in linked pages other than the HTML.
All of that is doable, of course, but unless you have a very small set of feeds, it would takes an extremely long time to download everything.
What I’m planning to do is simply enable prefetching links and images in news items, which by itself greatly improves the offline reading experience without taking a huge amount of the user’s time and storage.
Hi Nick – regarding caching images and html each time synchronization takes place, I’d note again that Egress does this asynchronously – so if you need to sync up quickly, at least the text is done fast.
Also, if disk space is an issue, maybe it would be worth letting users decide how much space they’d be willing to devote to caching this content. It would be handled on a FIFO basis. Personally I’d let FeedDemon eat up 10 Gigs of space if it meant that I could know that at any time if I took my computer offline, I would have all of my feeds complete with images ready to take with me!
Now if that isn’t an unnatural form of love for FeedDemon, I don’t know what is… ;-)
Jason, I’m touched by your unnatural love for FeedDemon :)
FeedDemon’s prefetcher also works aysnchronously, so you can keep reading your feeds while it prefetches in the background.
My goal here was to enable quickly prefetching only unread items before going offline – kind of a “pack and go” feature, so you could read your unread items while offline.
I’d really like to avoid making this feature something that’s highly configurable, since that adds complexity to the code and confusion to new users. But based on several of the comments here it looks like I may need to rethink this!
Hi Nick,
The only problem as I see it with the “pack and go” concept is that the last thing I want to have to do is remember to click a button and wait for a sync to happen when it’s time for me to pack up and go somewhere with my laptop. I’d much rather know that everything I need is going to come with me, even if have to pack up and go in a hurry!
I think only caching unread items is fine, but if I have 1000 unread items, it’s going to take a long time to cache them all.
Yay, this is one feature on my wishlist and something I liked while using NewzCrawler (left NC to return to FD since they’re development is heading in directions I care nothing about).
One thing though, caching prior to going offline is a step in the right direction but not as useful for the quick “shut the lid (standby) and go” situations. What would be nice is a per feed setting, which caches for offline just the feeds that require it–NYT, Engadget, WSJ, etc.
I realize that I’m rehashing what others have mentioned, so just consider this a vote in the same line of thinking.
(FWIW, the other big wish on my list is multiple folders/tree structure ;)
Keep up the great work Nick!
Yes – please do – I’ve started a new job that has no reliable wifi (well, no wifi) so I can’t get online at all during the day.
Reading my feeds offline will certainly save me some time – and yes, the images off of Engadget would definitely help – so please make it so images load as well.
Yes yes yes – this would be a great feature if configurable for specific feeds… I would use this on flights, train, etc.