For those not familiar with micro-blogging, it’s pretty much like blogging, but smaller — hence the micro prefix. Micro-blogging posts are very short, 140-160 characters that is. The most popular platform is Twitter, but new platforms are appearing, such as Pownce, Jaiku and Identica. Also, this year’s TechCrunch50 winner is the project oriented micro-blogging platform Yammer (which I didn’t get to try yet).

Back to MicroBlogBuzz, I started this little project part time while testing various APIs, and when stumbled on TwitterBuzz, which presents the most popular links on Twitter. However, TwitterBuzz only present the domain name, which isn’t really meaningful since most micro-bloggers use TinyURL or similar services to shorten their URL. TinyURL was presented as the Top Twitter link. And I thought it was kind of stupid. So I decided to do the same thing, but to follow the HTTP redirections to get to the final URL.

I got surprised by the quantity of data that I would collect. Around 1200 links every 15 minutes, and over 330,000 links and 400,000 comments in five days. I quickly ran into problems, as I built my database on InnoDB and foreign keys to keep things clean, and with this amount of data and the small server it runs on, well it can’t be clean and fast at the same time.

So I switched to MyISAM. But still, it wasn’t enough. And I added caching and smart HTTP headers. But still, it wasn’t enough. And I added preprocessing. And it was ok. At least for now, with my small 500 visitors per day.

Feel free to send me your comments and suggestions, by email, or even better, on Twitter

I’ve stumbled upon Rémi Prévost — who’s blog a follow more or less consistently due to the lack of time — CIOpenID module. It’s basically a CI embed of Janrain’s PHP_OpenID.

His code is great and works well. But one thing I don’t like about it is that it requires a different bootstrap file (a modified version of index.php) and a somewhat hacked version of your typical .htaccess file. The reason is, CodeIgniter annihilates the $_GET variable during initialization, because GET queries aren’t secure (ok, this is overly simplified, but you get the idea).

Being who I am, and somewhat liking to reinvent the wheel during my free time, I used a different approach. Rather than using a different bootstrap file, I rather chose to use the pre_system hook.

The installation process is quite simple:

1. Put my ciopenid.php library in your /system/application/libraries directory.
2. Create /system/application/libraries/openid, and unzip PHP_OpenID 2.x.x’s Auth directory in it.
3. Open your /system/application/config/config.php file, and make sure that hooks are enabled:

   $config['enable_hooks'] = TRUE;

4. Open your /system/application/config/hooks.php file, and add the following lines:

   $hook['pre_system'][] = array(
      'class'    => 'ciopenid',
      'function' => 'pre_system_hook',
      'filename' => 'ciopenid.php',
      'filepath' => 'libraries',
      'params'   => null
      );

To make an authentication query to an OpenID URL in $openid:

$this->load->library('ciopenid');
try {
   $redirect_url = $this->ciopenid->authenticate($openid, site_url($this->uri->uri_string()));
   header("Location: ".$redirect_url);
} catch (Exception $e) {
   die('Error: '.$e->getMessage());
}

And to get the OpenID query response:

$this->load->library('ciopenid');
if (isset($this->ciopenid->response)) {
   switch ($this->ciopenid->response->status) {
      case Auth_OpenID_SUCCESS:
         $openid = $this->ciopenid->response->endpoint->claimed_id;
         print 'Success: '.$openid;
         break;
      case Auth_OpenID_CANCEL:
         print 'User cancelled';
         break;
      default:
         print 'Internal error';
         break;
   }
}

That’s it! Ooops, here’s the file…

ciopenid.zip (3k)

Enjoy!

Schedule aside, there’s an idea that has been on my mind for quite some time. Actually, since I saw the What happens in the Facebook stays in the Facebook Flash presentation, and after my advertising teacher told us something like : “Advertising on the web is great, because you can track the viewers behavior, including pre and post viewing”.

AdServer conspiracy

Well, Facebook knows a lot about you, and in advertising, the more you know about your viewer, the more relevant the ads you present him can be. Now, Facebook has an internal advertising platform, Facebook Social Ads, based on this. Nothing external, such as AdSense, yet.

But lets imagine that Facebook launches an external AdServer and builds a big network of small advertisers, like Google AdSense, or a small network of large advertisers.

First, as Facebook knows a lot about you, it simply has to set a cookie in your browser when you go on your profile, and reuse the cookie when displaying a banner on newyorktimes.com. The AdServer would then know that I’m a 27 years old male in a relationship with a university degree.

The force

This could lead to very interesting possibilities:

• Segmented Ad Performance Calculation : Does you ad under-performs except with single gay females with a college degree? You can’t know now, but a Facebook AdServer could tell you.
• Segmented Ad Presentation: Only present the ad to your target segments. So you could have a Monster ad presented only to unemployed viewers and viewers who didn’t update their work profile for the past 2 years.
• Segmented Ad Selection : Present an ad more specific to the viewer’s segment. Creating various versions of the same banner is cheap, so if you can present a version more specific to a segment. So you could have an Obama ad targeted for:
  • Afro-Americans
  • Hispanic-Americans
  • The bunch

The ethics

Sure, this raises many ethical questions, as it could be misused by the wrong people (you know, the bad guys in 24 and in Michael Moore’s films).

I remember (or I think I do remember — please let me know if I’m misquoting) that, while I was working at ZeroKnowledge Systems way back, Adam Shostack’s moto was: “Someday, soon enough, all databases will be interconnected”.

That’s why ZeroKnowledge developed their Freedom platform, allowing you to browse anonymously. But nobody cared at the time: querying a 500MB database took a few seconds on cluster, so no one could realistically store so much information in a usable way. That’s why ZK’s great idea and platform never really worked, business-wise. Too much, too soon.

But in the 2008 world were Google and Yahoo crawl — and even cache — a huge portion of the web, where 25% of adults in Quebec have a Facebook profile, and where interconnecting platforms is easier than ever with all the APIs around, this could be up and running tomorrow morning. Too little, too late?

Big brother is watching us.