Blog

[Intermediate users] If you’ve seen sites like Techmeme and Megite and find them handy, you might be wishing you had something similar for a specific niche. Techmeme’s owners do have similar sites for specific niches (including politics, baseball, celeb gossip), but maybe you want something for a different niche. Since as a freelancer, I cover a number of niches on a regular basis, I’ve got my own personal dashboard tools for monitoring niches. But I’ve been on a quest to find/ build a generic tool similar to Techmeme/ Megite. Fortunately, you can produce something similar: a river of news. It doesn’t offer the topical clustering that makes Techmeme and Megite so useful, but it’s a start.

What’s a River of News, and Why Use It?

A river of news is just a stream of headlines from various websites, possibly with text excerpts. See Techmeme River for an example (a snapshot is below).

Techmeme River does not provide any item excerpts, but that doesn’t mean your river of news cannot. (However, in this post, I’m discussing a headlines-only version.) You can format the text presentation any way you like, as well as insert the information you need to appear.

Ok, Show Me How

Here is the simplest process for building a river of news:

1. Take a bunch of related feeds that you want to monitor, then mash them up. (I like to use Yahoo Pipes for feed mashups because of the relative ease and the great functionality.)
2. Sort the mashed up item stream. You’ll very likely want them presented reverse chronologically (most recent first).
3. Truncate your list to X items – say 25, 50, or whatever.
4. Publish the list to your blog – either in your navigation column, or on a page of its own, or a subdomain. (You can use the new Yahoo Pipes badging or custom code.)

Instead of writing a lengthy explanation of the coding process, I’ll refer you to Splashpress’ own Thord Hendgren and his recent article, Mashing Up Feeds Using Yahoo Pipes. I’ve written similar posts (linked above), but Thord has done a nice job of showing lots of screen snaps. What’s more, he’s provided and explained HTML/ PHP code that you can use in WordPress to parse your mashed feed stream and badge it on your blog.

How Can I Use a River of News For Niche X?

Assuming you apply the instructions that Thord gives, here’s what you might do:

1. Pick out the RSS feeds you want to monitor, including building custom feeds using Google News, Yahoo News, etc.
2. Apply the process I’ve given.
3. Use Thord’s badging process and code.
4. Tweak your WP theme’s CSS stylesheet to present your river of news text in whatever way you like.

The result will look something like this snapshot, below, from Wisdump’s right hand navigation column (depending on your stylesheet settings). You can also modify Thord’s code to add an excerpt, though I’ll cover that later.

Summary

Thord’s code is intended for use in a navigation bar on a WordPress blog. In the next installment of this series, I will discuss presenting a river of news on a separate blog page or on the home page. In a subsequent installment, I might get into a more customized presentation.

[Advanced users] To review, a river of news is a mashup of RSS feeds, arranged in (reverse) chronological order and presented on a web page. Normally, an RSS reader might suffice to monitor a blog niche, but my intent is to produce a Techmeme clone. A step in that direction is a Techmeme River clone, which is what this post is for. Either tool can be immensely useful if you want to keep on top of

Please review Part 1 of this mini-series, which goes over the basics of building a clone of Techmeme River. In that article, we produced an HTML badge that shows a block of headlines collated from multiple source feeds. The intent in this article is to present the headlines block in the content area of a blog instead of in the navigation column.

As with the previous article, I am not supplying 100% of the code, just the basics. This article, unlike the last, is intended for advanced WP users as a stepping stone for producing a Techmeme River clone that lives on its home page. If you’re serious about building your own version, I’m assuming you will likely pursue your own development, based on what I discuss here. (If you simply need such a tool, I’m available for consulting.)

I’ve built a tool, RideSpottr, for monitoring automotive niche blogs, and I’ll use it as an example below.

What You’ll Need

Here’s what you’ll need for this WordPress hack:

1. Theme: You can use any WordPress theme you like, but I chose the Popurls clone theme package from Ericulous. It includes a modified version of the Simplr WP theme, as well as instructions for specific WP Plugins.
2. Plugins: I only used the Exec-PHP plugin, which the Popurls clone theme also uses. The other plugins are not necessary. This plugin, when activated, allows your blog to execute WP/PHP code that is contained in a blog post.
3. Feeds: Mashup your selection of feeds to create a superfeed. As previously mentioned, I prefer using Yahoo Pipes because of the ease with which I can manipulate RSS feeds. You can use whatever web service you prefer. Notes:
   • Be sure that the superfeed you use for monitoring a blog niche is
     actually in the RSS format and not the Atom or RDF feed formats.
   • Make sure that you sort the superfeed’s items reverse chronologically.
4. Code block: The engine for your river of news is a block of PHP code that will only execute properly on a WordPress blog. The code block starts off identical to the code show in Thord Hendgren’s article Mashing Up Feeds Using Yahoo Pipes, which I referred to in the previous article. After that, I apply a serious of tweaks, which are only partially discussed below.
5. WordPress: Of course, you’ll need an installation of WordPress to test this. You might be eager to use the brand spanking new WP 2.5, but I highly recommend that you DO NOT use anything in the WP 2.3.x or higher series, else you might run into technical difficulties. WP 2.3.x (or even slightly earlier) have significant database changes that normally wouldn’t introduced into an intermediate version. (That is, such changes should have been reserved for, say, 3.0.) So you could run into problems like I did.
6. Blueprint CSS Grid Framework code: This is only necessary if you want to pretty up the river of news with some structured layout framework. It is not a Google product but can be downloaded from the Blueprint CSS pages at Google Code. Follow the install instructions discussed there. Do not use the “compressed” version, which typically gets placed in a subdirectory of the WP theme you’re using. I had trouble getting this to work. The default approach allows you to change themes if you like. (The default of Blueprint is 950 px wide pages. So your theme should support this, unless you feel like doing a lot of tweaking. That’s why I used Ericulous’ modified Simplr theme, as mentioned above.)

A Few Cautions

There are a couple of factors that you should be aware of before starting.

1. The WP/PHP code block uses a WP function, fetch_rss(). This function is very sensitive, and if there are any problems with the input superfeed, nothing will show up. I was constantly in a hair-tearing frame of mind because some of the source feeds had a different format for the “pubDate” field (publishing date/ time stamp). Yahoo Pipes would sometimes work and then immediately after would not. There’s absolutely no way a production web service should rely on Yahoo Pipes. Use it as a prototyper only. If you use Pipes in this hack, like I have, expect that your test site’s home page will occasionally show no headline items.
2. More Pipes issues: use only RSS formats, not RDF or Atom. Also make sure that if you plan to display the date/ time stamp in your river of news, that the source feeds use the same format. Else Pipes may not sort properly.
3. The WP/PHP code block I use (presented in full somewhere below) is not robust. It does not have full error handling and is simply meant as a starting point for you to work with.
4. I did manage to get my test sites working on WP 2.3.3, but not without problems. If you apply this approach, you’d better have patience and enjoy a challenge. I have a long way to go before I have robust web service, despite the hours I’ve already poured in.

Just don’t give up on this hack – it’s very useful as a niche monitor. If you have a more reliable way than Yahoo Pipes to “normalize” all the input feeds for the superfeed, then try that. If you have any questions about this WordPress hack, drop a comment and I’ll do my best to answer.

Example: Automotive Blog Niche

I’ve built two test sites in a slightly different manner, to test different theories. The one mentioned below is RideSpottr, which depending on what I’m trying in the code, may show headline items or might not (because of the problems in Yahoo Pipes). RideSpottr’s layout, at the time of this writing, is no work of beauty and is by no means robust. Though I intend to apply the principle of kaizen and improve it by slow degrees over time – in hopes that it might actually be something other people will use.

Stage 1: Raw List

The instructions below assume that you have already installed WordPress, Ericulous’ modified Simplr theme (for his Popurls clone), and the Exec-PHP plugin. (See above for links.)

1. In your WP admin panel, change your “Reading” options to allow only one blog post to display on the home page.
2. Follow Thord’s instructions (linked above) for building a “network feed”. He offers the necessary HTML/ WP/ PHP code necessary to produce an HTML badge.
3. Take the exact code in Thord’s article and paste it into a new blog post in WP. Make sure you have turned off “WYSIWYG” mode in the WP editor.
4. “Publish” the blog post, then view your home page.
5. If you can see headline items, then you can now use your own feed.

Suggestions: Don’t monkey around with Yahoo Pipes to start with. I suggest that you use a single source feed (not a superfeed) while you’re testing, and when you have headline items displaying the way you want, you can use a superfeed.

The result of using Thord’s code will look something like the snapshot below.

Stage 2: Prettying Things Up

From here, you can apply a series of improvements using WP/PHP code tweaks. (My experience over two years is that most performancing members don’t care to see the code step by step. What I’ll do here instead is, as with the last article, describe the steps. There’s also a section somewhere below where you can obtain the final code block and use it for your river of news.)

1. Add a timestamp.
2. Change Feedburner URLs to native URLs. Many sites use Feedburner to “burn” their news feeds. But Feedburner feeds retain the original source URLs, so you can publish these in your river, if you prefer. I did this in Yahoo Pipes.
3. Add floating item excerpt. In an earlier version of my river tool, I did not display item excerpts. Instead, I set up the river so that if a visitor moused over a headline, an item excerpt would appear in a floating box. In later versions, I dropped this. If you want to do this, it is already set up in Ericulous’ modified Simplr theme. You’ll need to study his sample code to understand how this works, as I’m not discussing it here.
4. Insert author name of news item. Some RSS feeds publish this as an “author” field, others as “dc:creator”. Since I’ve used Yahoo Pipes for building the superfeed, I’ve converted all dc:creator values to “author”. But some feeds do not contain any author info, so be warned of this. This does not cause an error in the river of news, though. The author will just show blank.
5. Apply a grid layout. After I got the headlines and related info displaying in my river of news, I applied the Blueprint CSS grid framework to control overall presentation.
6. Add feed sources column. To spiffy things up, an earlier version of my river had a column showing the URLs of blogs/sites used in the superfeed. I later changed the text links to 240×90 banners, though I could also have used screen snapshots of home pages. (If you build a serious, robust tool, you might be able to sell that banner space as advertising.)

Additional Improvements to Consider

Before I get into the actual code used, here are some additional improvements that you might consider trying.

1. Normalize all the date formats. I found that many of the feeds that I wanted to include in the superfeed had different pubDate formats. Some didn’t even use “pubDate”. Without having to do a great deal of hacking in Yahoo Pipes, I simply could not include many feeds. It would be nice to add date format handling to “normalize” item publishing date for any RSS feed.
2. Format the date. The current date/time stamp format isn’t pretty. A nicer format might be “Sun Mar 30, 2008; 10:05:00 am,” or some such.
3. Limit the number of items per source feed. Currently, if one blog has lots of recent items and others have few, the former will dominate in the river of news. Limiting the number of headline items per source feed is one way to resolve this. If you use 50 display slots like I have, thenyou’ll want to trim each input feed to a maximum of X items each. X will decrease as the number of input feeds, N, increases (Xv,N^).
4. Filter for authority sites. Don’t randomly add blogs from a given niche. Pick those that are authority sites.

Just the Code, Please

Okay, if you just want the short summary and code, here it is:

1. Pick an unused domain or subdomain.
2. Install WP. I recommend something before 2.3.
3. Install and activate Ericulous’ modified Simplr theme.
4. Install and activate the Exec-PHP plugin.
5. Install the Blueprint grid framework code. Follow their installation instructions, as there are a couple of lines to add to your header.php file.
6. Set WP to only display one post maximum on the home page.
7. Take the code shown below and paste into a new blog post. Publish the post and refresh your home page.

The HTML/ WP/ PHP Code Block

The code shown below gets pasted into a blog post. You could try pasting it into your index.php file, but I haven’t gone there yet. You’re on your own.

    
RideSpottr - automotive news
    

    items) && 0 != count($rss->items) ) {
          ?>

  

              items = array_slice($rss->items, 0, 50);
              foreach ($rss->items as $item ) {
                  $pubdate = wp_specialchars($item['pubdate']);
                  $desc = wp_specialchars(substr(strip_tags($item['description']), 0,200) );
                  $auth = wp_specialchars($item['author']);
              ?>

    

      

      

        
by 
      
      

        
        
      
    


              

  


           no rss items to display
\n";
          } ?>


  


    

    

      
RideSpottr Sources











    

    


  

Now you’ll need to change the code as follows:

1. Replace the heading text in a way relevant to the niche you want to monitor.
2. Replace the feed URL used in fetch_rss();
3. Change the HTML code where the site banners are displayed.
4. Spiffy up the CSS to make the river of news more attractive than I’ve got it.

Summary

To keep this article relatively readable, I’ve not gone into great detail about the coding. Experiment on your own, and ask questions in the comments, if you like. Good luck.

WordPress 2.5 was just released, and if you read Matt Mullenweg’s post, you’ll see that it has lots of great new features. I’m simply summarizing here.

User Features

Here are features that a user with access to the admin panel will see:

1. Cleaner dashboard.
2. Dashboard widgets, including statistics.
3. Multiple simultaneous file uploading, with progress bars.
4. EXIF extraction for camera metadata.
5. Search that includes both posts and pages. (Previously, search was post-only.)
6. Tag management without plugins.
7. Password strength meter.
8. Edit protection, so that two or more authors cannot simultaneously edit a post or page.
9. Minimized plugin upgrading. This is host-dependent, but WP 2.5 can download and install plugin upgrades for you.
10. Non-desctructive WYSIWYG (visual) editor. The old visual editor mangles your HTML code after saving. The new one uses TinyMCE v3.0 to avoid this issue.
11. Built-in photo galleries.

Developer Features

Developers of WordPress plugins and themes, or those who customize WP installations have these new features to look forwards to:

1. Salted passwords. (If you don’t know, don’t worry.)
2. Secure cookies.
3. Easier URL creation from page taxonomies.
4. Inline documentation.
5. Database optimization. (Matt’s post says they haven’t changed the database table structure with this release, but I’m not clear on this. My experience shows they changed the database schema back around 2.2x. (I don’t remember what X was.). I feel this was a big mistake for a non-major release version. However, it’s possible they changed back in a subsequent release.)
6. Optimized database queries.
7. Media buttons.
8. Shortcode API, for applying custom short notations in posts that expand into content during page render. This is accomplished through custom WP PHP functions you can develop for each shortcode. So you might have a “gallery” shortcode with parameters, “, and then write a PHP function called “gallery_func” to define the shortcode behavior during page render. [Note: this alone will make WP an even more powerful all-purpose CMS for both blogs and websites.]

As you can see, there are a great deal of new features for everyone. I’m going to be working with a WP 2.5 installation, and I’ll blog about positive and negative aspects on an ongoing basis.

Should You Upgrade to WP 2.5?

No one can argue that there are not lots of incredible new features in WP 2.5. Just watch the 4-minute video screencast of the new interface for starters. Just be sure that you’re prepared to upgrade. Here’s a very generic upgrade process:

1. Backup themes, plugins, and uploaded files (e.g., images), in the same directory structure. Use your favorite FTP software.
2. Depending on the version of WP you currently have, backup the blog content or export it. If your WP is very old, you may have some hair-pulling time ahead of you, thanks to database changes since around version 2.2x.
3. Remove your current WP installation and reinstall a new version. Depending on how long you think it’ll take you to re-install, you might publish a temporary index.html that says “Back shortly” or some such message.
4. Reactivate your theme and plugins, and make sure you’re using the same Permalinks options. (There’s a community-built list of plugins that work and don’t work with WP 2.5.)
5. Test everything.
6. Rejoice.

I also wrote a quick and safe WP upgrade approach earlier. I recommend this for those of you that are concerned with messing with your live blog. I’ve been reading since before WP 2.3 of how annoyed some WordPress bloggers have been that older themes and plugins no longer work properly. I’ve experienced this as well and for that reason, I’ve only installed WP 2.3x and up on brand new blogs. (I.e., no time to monkey with upgrading.)