the iCite net > news / blog > a permalink

news and thoughts on and around the development of the iCite net
by Jay Fienberg

Autodiscovery: spider food, interactive query, processing instruction?

posted: Sep 13, 2003 10:44:24 PM

Jeremy Zawodny has jump-started the discussion about RSS feed autodiscovery by proposing what he calls RSS Auto-Discovery 2.0 (which I came across via Russell Beattie's RSS and Commercial Aggregators—itself an interesting post). Many good comments follow Jeremy's post as well.

I have been really interested in this concept of "autodiscovery", partially because I think, in some sense, it is like saying that being able to SELECT TIMESTAMP into aVAR in a SQL stored procedure is a way to autodiscover what time it is ;-)

But, actually, what I think is really interesting is that the autodiscovery interaction between a newsreader and a RSS-enabled website seems like distributed robot / agent processing. (I think that Timothy Appnel's RSS Auto-Discovery 2.0 and WSIL, which is his response to Jeremy's post, is a relevant perspective on picturing RSS feeds as web services.)

Jeremy mentions the idea of having a feed description document along the lines of a robots.txt file (which file is used by search engine spiders, see this excellent tutorial for more info), and also mentions variants on the current RSS autodiscovery usage of the HTML link element (see this post for more info).

What I am working on for the iCite net is looking at the autodiscovery interaction and designing it around what, to me, seems to be the thing really going on: interactive querying and retrieving remote processing instructions, between web client and web server program.

In other words, your newsreader is querying a website for RSS feed information, and the webserver is giving the newsreader a processing instruction it can use to find that RSS feed.

I think that interaction also can (and should be) expressed through files that serve a similar role as robots.txt, that is being (search engine or other) spider food. And, the HTML link element is a natural way to indicate a relationship between the content of a web page and other services around that content.

But, with the query / processing concept, it makes sense to me to allow an extensible domain of queries and an extensible domain of results / processing instructions. For example, why not ask if you (i.e., your agent, the newsreader) can have a RSS 1.0 feed with full contents rather than have to figure out if one is offered or accept the first RSS flavor discovered as the only option?

So, a main piece of the iCite net software is the interface for queries / processing of website content. Initially, that interface will function server-side for interactions between web pages / sites. But, my idea is that, if it turns out to be interesting enough, that it will be natural to use that interface in newsreaders, email clients, web browsers, cell phones, etc.

Note: in July, I posted some related ideas about autodiscovery for RSS-type feeds in Maybe RSS autodiscovery link usage should change (in Echo?) and Mark Pilgrim comments on my RSS/Echo/Atom autodiscovery post.

permalink | comments {0} · trackbacks {2}

also available as: rss · rss2 · rdf · atom

Comments and Tracbacks

trackback from: the iCite net development blog
posted: Oct 17, 2003 7:55:25 PM
title: Metadata URIs and such

Don Park posted about Fixed-URI for Site Metadata, and I posted the following comments . . . I also posted a lot about this topic a couple months ago in Autodiscovery: spider food, interactive query, processing instruction?

trackback from: the iCite net development blog
posted: Oct 21, 2003 1:19:35 PM
title: Agents who do atom might do more too

Danny Ayers has a great post on Atom as Agent Language. This follows on some of the ideas I originally posted in Autodiscovery: spider food, interactive query, processing instruction?

Note: All comments and trackbacks are moderated. Spam is deleted. Other comments are approved as promptly as possible.

Note: Older posts no longer accept new comments or trackbacks.

« prev post
Public parcels of packet publishing

» next post
Recent rants on the RIAA's destruction of value

blog newsfeeds

brief content:

 XML  ·  RSS  ·  RDF  ·  Atom 

full content:

 XML  ·  RSS  ·  RDF  ·  Atom 

blog archive

jan · feb · mar · apr
may · jun · jul · aug 
sep · oct · nov · dec
jan · feb · mar · apr
may · jun · jul · aug
sep · oct · nov · dec

jan · feb · mar · apr
may · jun · jul · aug
sep · oct · nov · dec

may · jun · jul · aug
sep · oct · nov · dec

first post: 
April 30, 2003

highlight views:
Spammers' Choice

Jay elsewhere online
Jay Fienberg - the official home page

Wrong Notes - the music blog of the Ear Reverends

Fine & Full, aka, a fine and full burger

Sociomobilepoetextologia (moblog, currently inactive due to lack of proper mobile)

to enjoy roll
sites I like to read when I start from here

· Anastasia Fuller
· Andy Baio
· Biz Stone
· Boris Mann
· Bre Pettis
· Chris Dent
· Danny Ayers
· Dare Obasanjo
· David Czarnecki
· David Weinberger
· Don Park
· Evan Williams
· Greg Narain
· Jason Kottke
· Jim Benson
· Lucas Gonze
· Marc Canter
· Matt May
· Matt Mullenweg
· Michal Migurski
· Nancy White
· Rebecca Blood
· Reg Cheramy
· Richard MacManus
· Sam Ruby
· Shelley Powers
· Tim Bray
· danah boyd

powered by blojsom

Entries by blojsim