blogbridge
Rss owl
Great news
Fire ant
Blogmatrix sparks!
Blogmatrix jaeger
feed reader
AmpheteDesk
Flock
Sharp reader
Newsgator
Feed demon
Straw
News Monster
aKregator
Netnewswire
and more which I can't remember right now…
None of them quite have all the features need to make me quite stick.
Up till recently I was using Blogmatrix Jaeger but the lack of support from the authors, has me worried plus there are some really silly bugs which are very ignoying. Lets not go there right now. Before that I was using RSSOwl which was good even back in the older versions, but its lack of synchronization drove me spare. Theres nothing worst than reading through a ton of interesting news on one client then go and all of them being highlighted again! Well there is but trust me it becomes a pain after a while switching back and forth between different machines.
Just recently I tried Blogbridge which is ok, but resource hungry and not all that useful with 225 RSS feeds split into about 11 categories. Geez I'm not even close to Robert Scoble and others which have over 400 RSS feeds which need to be sorted, filtered and aggregated. But honestly I have not even started with my search feeds from yahoo, pubsub and blogdigger. So I expect the perfect aggregator will need to support almost 300 feeds without breaking into a sweat when I go for a search.
Attachment support
some other things which are needed in the perfect aggregator. Podcasting support, it needs to be as flexable as jaeger where it doesn't always need the enclosure tag to get stuff. And when I say stuff I mean not just Mp3s but videos like on channel9 etc. Bit torrent support would be cool but not essential. At least pass the Bit torrent files on to Azureus would be very useful.
Currently i'm trying fireANT to deal with all my attachment support but it doesn't do a very job with anything else. It actually kept crashing when loading my OPML. Yes I could just input the rss which supports podcasting, but what happens when someone I usually read starts podcasting? I don't fancy having to copy and paste the url just for a couple of entries. Jaeger works well because it highlights that this rss feed also has attachments (which can be enclosures or simply links to rich media in the post) this is great for example when epic 2015 came out. Someone I read regularly did a review of the changes between epic 2014 and epic 2015. And linked directly to the quicktime version. Jaeger automaticly downloaded because I have an option set to download all without asking me. So there was need to download the movie as it had already been done. This doesn't always work so well however, I'm subscribed to the TED feed and that usually has a lot of pdf attached with badly thoughtout file names. So I end up deleting a load of PDFs every week. talking of
cleaning up files, the perfect aggregator will need a user defined timelimit like in jaeger. This basicly removes all downloaded files unless there marked (maybe tagged, which I'll go into later. – idea being if you tag it, you care enough to keep it?). This stops the hard drive being filled up with junk, and makes synchronization on to a mobile/portable audio device a lot easier. So at the moment I got Jaeger deleting stuff after 2 weeks. Which works well for myself. But you can define from Never to every day. I expect a smarter way would be to have the time limit but also pioritse files by tagging, categories, read/unread status or even files size? I mean seriously if I haven't read or even glanced at the entry, will I be interested at the attachment? I would say no.
On this same thread, I would like to see more TVRSS features in the perfect aggregator. The plugin for Azureus is good for downloading media content, but I think that would be better done in an aggregator and the plugin simply interfaces with it using RSS output of the files to download or picks up torrent files from a certain defined directory. So basicly the perfect aggregator would need a regular experessions engine or hey why not just use the search and smart search (almost playlist) feature which then also generates rss of the output which the TVRSS plugin could use. So for example if Kevin Rose releases another systm or thebroken I would have that feed set to automaticly download the media or torrent file (which would be passed on azureus). If i'm using one of the many RSS feeds with many torrent media links as items, I simply run a regex or simple search on it. Pop that into a smart search folder and tell the aggregator to give me RSS of what evers in that folder. azureus can do the rest.
RSS in and RSS out
Another thing which I rarely see, is RSS output. Now this may sound odd, but really want to beable to subscribe to the output of an aggregated source. For example I have a RSS screen saver on my machines, rather than it going and pulling the exact same feeds once again, wouldn't it be great if it pulled RSS from the perfect aggregator? Yeah starts to make sense right? Also there are other applications like widgets which I would rather pull from my perfect aggregator than from the web again. Currently I have a jabber bot which tells me what's new on slashdot, that's good but a little widget which just runs the latest headline in the corner of my screen would be highly useful. So generally a local RSS aggregator which also server's XHTML and RSS would be ideal. I have found ways to make Jaeger do this, but you can only have one or the other and it only serves to the local machine, aka no one on the same network can access my aggregator. I know Amphedesk does do this well but its highly useable and looks kinda of crap in my own mind. Maybe I should revisit it because its been a while now, and I do believe it supports themes or some kind of skinning.
But on the same front this is a problem when considering the speed issue.
Operating speed and cross platform capabilities
Blogbridge was so laggy when I had 225 feeds in it. RSS Owl copes well for a java application. While Jaeger and my new favorate Great news have no problems with 225 feeds and more. There seems to be a whole rash of .net based aggregators which work ok, but tend to be quite slow too. I do not know why this is, maybe theres some odd configuration on my laptop and desktop machine? It doesn't run as slow as some java aggreagtors but when loading 225 feeds in, its not far off. The compiled windows versions obvioudly run very quick, weirdly having an internal parser seems to be quicker still.
I have another odd requirement which I do not believe the perfect aggregator should service. I own a HP Ipaq (pocketpc) and I do read news on it quite a lot. Pocketpc RSS readers are slowly getting much better but are running into the same problems as there desktop counter-parts. But I don't believe the perfect aggregator should have a pocketpc version. Nono, that would be too much, plus through the steps talked about in this entry, any decent RSS reader on a pocketpc, palm, symbian device or smartphone device should beable to suck down an opml file using synchronization. Its not ideal because the synchronization would work both ways allowing you to also upload read and unread items along with downloading them. This feature seems to be a couple versions off pocketrss at least. I did notice one of its rivals now supports bloglines synchronization which is what Great news also supports well – more on this later.
But generally the perfect aggregator would support aall the main platforms be open source so at least people can port it to platforms like Solaris if needed. I expect when I mean main platfoms I mean Windows, Linux and OSX if your really pushing me. I like the idea of BeOS but how many people still use it? If its open then its at least somewhat portable.
While i'm talking about the nitty grittys of the application it would also support ALL languages in all types of encoding. Most aggregatorss fall back to IE or a mozilla core for display which is ok. But this would be best served as a choice by the user. On the mac obviously Safari would also be a choice too. Ben metcalfe recently linked to a stress test for RSS readers to check if they were subsectical to common dirty html tricks. This includes things like iframes, remote exploits via javascript and other natry things. I did a test with Jaeger and it passed all the test with no problem becaause when ever there was any html type stuff, it would pass that on to firefox which has all the adblocking, popupchecking, etc features I need and have defined already. I have yet to try this on Great news (tried it finally and it crashed Great news – I submitted a bug report, but i'm worried because great news uses IE for its internal browsing and there seems to be no way to change it the firefox core. So I assume a lot of thos dirty html tricks will work in great news? Unless great news doesn't pass any of that crap on to the IE core render?
This may seem the overboard comments of a open souce fan but this is important if like me you haven't really setup IE for your own preferences because your using something else such as Firefox. Security is a large problem and will get worst if the perfect aggregator also because a good way to exploit systems. RSS spam is only the start of things.
Tagging and Categorization
Talking of Great news, I'm really starting to love the abaility to tag and categories rss items. Jaeger has this but its not quite the same. Both also support a feature which I can only call search folders, basicly you define a set search and the aggregator will mark or highlight new items which match the criteria. This is essential and very useful. But tagging is great for reminding yourself of useful things which you may have not searched for. For example I have setup a tag in great news which is simply called blogthis. When I have time during lunch, I quickly run through the tagged entries in blogthis and sometimes blog it. I guess the perfect aggregator would have a compatablity with del.icio.us, technorati, etc so I could use the same tags to categories entries and maybe store them in del.icio.us using its public APIs. I personally think we are only scratching the surface with tagging and categories and maybe something combined between Thuderbirds straight forward search and smart search folders with the tagging ability of technorati and del.icio.us would be useful for making sense of content in an aggregator.
synchronization
Ah my new favor talking point when it comes to RSS aggregator. Like how most RSS readers only read RSS and don't allow any RSS out. Most Aggregators let you input OPML but not export it with equal ease. But lets get the basics right before considering advanced features.
The perfect aggregator should support at least.
OPML input via File and URL.
OPML output via File.
On the next step up in perfection.
OPML input and output via FTP
If you next consider the usage of idisk and .mac drives (better know as WebDav/deltaV) the natrual next step up would be.
OPML input and output via WebDav (unsecure and secure – https)
honestly I do not know any which have WebDav support but it will make sense with more services like idisk and .mac drives. Some support FTP like Jaeger for synchronization and storage.
The final and perfered way of synchronization is via webservice. Generally its using something else to send messages back and forth. There are growing propsals in this area, one I like is attention.xml which has backing from technorati. At this moment most people are using some modified opml to do synchronization which works but is so adhoc its untrue. I have never seen a opml carry what's modified and what's unread information across from one application to another. They tend to only work in there own application domain which sucks. The most used webservice type synchronization is done using bloglines open API service. For example in Great News all I needed to do is enter my email address and password I used to logon to bloglines and it will download not only my subscriptions but which items are unread and read. I do not believe the api supports locked items or any other states but read and unread is usually all you need. When your done with a feed in great news it will send the changes to bloglines. Perfect you would say but what incase I do not want to use bloglines? Well at the moment you could adopt the newsgator package which is a rss aggregator and web service which support syncing between the both of them but outside of that your kinda of stuck.
This maybe an area where Yahoo could really blow away the market. Allowing synchronization with your myyahoo subscribed feeds and maybe further the yahoo!360 service. But it strikes me that myyahoo isn't a bloglines and isn't meant to be either. There are few online aggregators as popular and as powerful as bloglines at this moment, so unless mymsn or myyahoo step up and give real aggregation features, bloglines will be the one and only service to sync with. Maybe in the future others will come along and even adopt the bloglines api in an attept to short cut into the synchronization market?
So in the case of the perfect aggregator, bloglines synchronization is key and needs to be there above syncing over opml.
Other features
Here's a list of key features which are highly recommended
Automatically generated search feeds and Pubsub support (also known as Watch feeds). PocketRSS has this nice feature which allows you to enter a search term and it automatically makes a new RSS search feed and subscribes to it. Practically this is usually a slight url change using a RESTful api but its dead handy when doing research.
The ability to filter a large feed (cited the case for torrents and regex), filtering on top of a search rss feed (like that of yahoo, technorati and blogdigger) is essential because not everything you get through is of interest. I would almost go as far as to say search and torrent feeds should be treated slightly differently. They tend to update quickly and contain many items. It would be good to automaticly remove old items, or archive them away like how azureus rss plugin does.
Comment support. The ability to automatically see the comments to a blog entry or at least click through to see the comments is handy and shouldn't go a miss in the perfect aggregator.
Blog this support. This is usually just a link which sends the permalink url and maybe the content to an external blogging application. I don't think this is quite standardized but this shouldn't be over looked in the perfect aggregator. I would also suggest Email this would be included in this same space.
Search this item. The ability to search all related or at least linked to blogs to that first item is vital when doing research or gaging views on the first item. Blogdigger and technorati do this well allowing you to search for all blogs which link to that first link or search via related terms used in the first item. Having the feedback in the aggregator is tricky but at least having that feature in the perfect aggregator which then sends the results to a browser is good enough. Like the search feed, this is usually just a RESTful method call with the correct URL.
All types of autodiscovery supported. It should support drag and drop from the RSS link and the page link which will cause the aggregator to search for all alternative link feeds in the head of the html page. It should also support feed:// and that weird USM (Universal Subscribe Mechanism) feature.
For display as mentioned before, it should support RSS output but also some simple templating system for displaying the HTML to the local browser or network client. CSS for style should be used while something like VM (velocity), XSLT or Groovy should be used for layout. I'm favouring XSLT. But it should be trivial to add different flavors or templates depending on a simple url query string. Yep this borrows from Blojsom's flavor idea but it works well.
Social aggregation. The idea of what your friends tag and recommend is not exactly new but it would be great to see what your friends recommend directly in the aggregator. Ampheterate was one way of doing this but it requires too much commitment on a user. Attention.xml has ways to deal with this which are well worth checking out.
Microformat and rss extention support via a community. I was orginally thinking just rss extention and microformat support but I'm thinking if the core rss parser engine simply outputs what it gets in, the template engine could deal with the extra rss support. However this is not strictly true. For example Amazons A9 open search is little use after its passed the parser, it would be good for the parser to understand the attributes and elements while it could be argued that Microsofts list extentions should also be parsed at the parser, most of the grunt work would be done at the display/template engine. I 'm not sure how easy it would be to patch the parser but having templates which can be shared around a community certainly makes a lot of sense. Saying that, Flock a decent but underdeveloped aggregator uses XSLT for all its parsing so its possible I guess.
Aggregator engine seperated from the front end. Memory is a worry no matter what operating system you maybe using. It would be real nice to have the actual simple rss catcher/fetcher and parser separate from the actual output and gui. So for example you could settup all your feeds and just let it run every couple of hours and collect a store of RSS and attachments. You could then run the gui to search/view the feeds, add/remove feeds or do anything else like this. I know theres one application which actually does this quite well already. Blogwave is not your typical rss aggregator, it has a front end but that's mainly to setup the actions and the like. Once its setup it will just download rss and run a set process on it if specified. Its good at what it does and I believe it is possible to get another RSS reader/aggregator to read the downloaded feeds but I have not tried this combination yet. I assume its possible to do more with blogwave than I'm suggesting, but why bother? Let something like Jaeger, Great news, Feedreader do the gui stuff and just build links and remote calls to blogwave.
But in the same vein I would like to see more RSS screensavers, widgets, etc which make use of the features of the RSS output. I currently also read RSS news via bloglines on my xbox using xbox media centres bloglines python plugin. Its pretty cool and allows for syncing back and forth as its based on the bloglines api (more on this later) but is it really necessary for each mini application to make calls to bloglines each time? As mentioned I have a bloglines script on the xbox, greatnews installed on 2 machines so far and a RSS screensaver on 2 machines and I haven't even started on the widgets yet. Each one of those are making calls to bloglines and individual blogs/news sites. It would be so much more effective if they all made calls to a proxy of some kind instead? I expect blogwave isn't quite enough for this, but maybe it is?
One, Two and Three pane support. Yeah this is always a conversational firestarter. I like Jaegers one panel solution but I've grown up on three pane rss readers with feedreader, rssowl and now great news. Sparks! has the ability to switch to two pane view if needed. So I pose the question of why not support all three types of RSS reading?
So you can use your browser for RSS reading using the templating engine ala Jaeger and Amphetedesk.
Two and Three pane reading via the browser core ala Sharpreader, feedreader, Feed demon, etc, etc. If anyone knows any other types of mass RSS reading outside of these three methods please let me know.
Automatically change items to read when selected. Unbelievable but I have encountered aggregators which do not support this feature. So you have to manually select read or wait anything up to 10 secs before it counts it as read. Standard feature which should be included.
Alternative grouping. This reflects tagging and categories section but the thrust of this feature is to get away from hiararchical folders to categories rss feeds. It works ok but honestly it gets ignoying when trying to manage the folders. Tagging the feeds means they can come together in groups when ever needed. This should apply to not only the feed but also the individual items.
User selectable views or Stylesheets. Great news does this so well using CSS I can't praise this feature any more, its hard to go back to anything different now.
User configable keyboard navigation. This should allow anyone to setup there own key combinations for going to the next/previous unread/read items.
Some may ask why I don't just build this magic aggregator which supports all these great features. Well honestly I would if I could. I was planning to build something using combinations of other aggregators out there. For example the ability to have just a aggregator engine which simply parses RSS without a front end is kinda of almost there if I use something like Blogwave or even something more complex like Apache Cocoon. Yeah over kill for a lot of programmers but a step in the right direction.
The main purpose of this mini essay/long blog entry was to inform others of the growing conclusions of RSS aggregators. I 'm actually hoping that within a year most of these things I and many others have suggested will be there as standard on most rss aggregators. I expect like the itunes 4.9 intergration of podcasting that longhorn/windows vista will not cover all the bases well first time but it will get better. However there is a burning need for advanced RSS aggregation beyond the usual RSS reader. Even in the naming I have started to make the distiction between the two. RSS readers would include the xbmc bloglines plugin, my pocketpc reader (pocketrss), widgets, screensavers and tons of simple but effective desktop readers. Aggregators on the other hand must have that ability to output content too. So I would include anphetedesk, flock, feeddemon, jaeger, sparks! To this list. I'm also thinking aggregators are much more hackable or remixable using standard technologies such as templating, scripts and other things.