get some news for your site

Bear in mind that the method I am describing here assumes that you want to put headlines on a web site. If you just want to grab them for your own personal use on your desktop, use an aggregator like AmphetaDesk instead. It'll be a whole lot easier and more flexible. However, to put stuff on a site like I've done, read on...

all right then, here we go

Before you start, you obviously want to make sure that you either a) have the cooperation of your webserver's administrator, or b) are your webserver's administrator.

There's really not a whole lot that happens backstage, but it's certainly not just "straight HTML" that makes the headlines magically appear on the news page...

what goes where

Here's the basic order of operations:

  1. Every half-hour, get a cron job to grab the headlines as an RSS (XML) document from the news site, blog, or whatever it is.
  2. Process the RSS file to extract titles and links, and output them to an HTML file.
  3. Do a server-side include in the news file to stuff the generated HTML file in the right place.
  4. Lather, rinse, repeat.

the tricky bits

Other than processing the RSS, the only tricky bits (if you're not the server admin) are getting your cron job added and making sure the webserver (such as apache) is configured to deal with server-parsed HTML for your server-side includes.

Naturally, that step about processing the RSS file is the tricky bit. Actually, it turns out not to be quite as tricky as you might fear. A little Ruby script (that I acquired from Aaron and modified for my purposes) sorts it all out. Here's a script that grabs the headlines from slashdot. The file it generates is here — that's the real thing, updated every 30 minutes, that the news page uses.

Put it all together, and poof! Bits that look (more or less) like this on the server:

    <div class="feed">
      <p class="sitename">
        <a href="http://www.tbray.org/ongoing/">
        Ongoing Tim Bray</a>
      </p>

      <p>
        <!--#include virtual="../headlines/tbray-ongoing.inc" -->
      </p>

      <p class="sitename">
        <a href="http://www.osnews.com/">OS News</a>
      </p>

      <p>
        <!--#include virtual="../headlines/osnews.inc" -->
      </p>

      <p class="sitename">
        <a href="http://www.bbandwire.com/">Broadband Wire</a>
      </p>

      <p>
        <!--#include virtual="../headlines/bbandintel.inc" -->
      </p>
   </div>

Those bits turn out out to look more like the news page after the server parses everything and serves it up to your web browser.

if you thought this was fun...

If you thought this was entertaining, you might be interested in some spam filtering information, too. Be forewarned: just as this page assumes that you're a webserver admin, or at least have access to one, the spam filtering method I outline assumes you run your own mail server. In other words, it's probably not for the faint of heart.



Last updated Tuesday, 02-Sep-2008 22:28:39 MDT

home
sewn goods
blog
for sale
garage
links

validate