Pushrod

Old dogs, new tricks

Posts Tagged ‘rails

A RubyonRails library for the ebay shopping API

with 21 comments


After the lightweight Facebook library I wrote to scratch my own itch, a couple of days ago I started to look at adding ebay items to Autopendium :: Stuff About Old Cars, the classic car website I run. Users were already shown books from Amazon, appropraite to the content being shown on the page, and it seemed to make sense to show models, cars and parts from ebay, for the vehicle or model being displayed.

Amazon books on Autopendium

I’d had a look at adding ebay functionality quite a while back, when I’d first just started to use Ruby and Rails, and couldn’t quite get to grips with ebay4r, which was at that time the ebay API ruby library. Since I’d last looked, another library had been written, Cody Fauser’s ebayapi, which he introduces with a brief tutorial here, and having a quick look at the code and the tests, it seemed just the job. I then fired up IRB and and gave it a test drive in the console.

It all seemed fine, just rather slow. The problem is, the library uses ebay’s SOAP interface, which is markedly slower than the REST one. And in fact, even the Trading REST interface is slower than the Shopping interface, as a quick and dirty benchmark shows:


  user       system       total      real
0.050000    0.030000    0.080000  (  6.487812) # 10 calls to the shopping REST API
0.130000    0.060000    0.190000  ( 12.517658) # 10 calls to the trading REST API

Now, if you want all the functionality that the Trading API provides — the ability to bid on items, or to list new items — that speed trade-off is no problem, as the user will expect such things to take a couple of seconds.

But if you’re wanting to include items for sale on a page each time it’s displayed (even allowing for caching), each 1/10th of a second counts, and the extra functionality that the Trading API provides is irrelevant.

Unfortunately, there’s no Ruby or Rails library for the Shopping API. So, time to scratch my own itch again. Enter ebay-shopping, a RubyonRails plugin for the ebay Shopping API. It’s a pretty straightforward plugin that was fairly easy to write (the first version, implemented as a basic lib file, was done in an afternoon), and is even easier to use.

To install, from the root of your rails app simply run the usual

script/plugin install http://ebay-shopping.googlecode.com/svn/trunk/ ebay_shopping

Then run

ruby vendor/plugins/ebay_shopping/install.rb

This will copy a basic configuration file into your app’s config directory. This is where you put your ebay settings (Ebay Application id, affiliate info if you have it, etc). Update this with your settings — the only thing you actually need is the app id, which you can get by signing up at http://developer.ebay.com (The code you need is called the AppID — the Auth Token and other stuff is for the Trading API).

Then fire up the Rails console and away we go:


>> request = EbayShopping::Request.new(:find_items, :query_keywords=>"Cadillac")
=> #<EbayShopping::Request:0x246aa54 @affiliate_shopper_id="my_campaign", @affiliate_partner="1",
@site_id=nil,@affiliate_id="foo1234bar", @callname=:find_items, @call_params={:query_keywords=>"Cadillac"},
@app_id="my_ebay_app_id_1234567">

>> response = request.response
=> #<EbayShopping::FindItemsResponse:0x2444520 @request=#<EbayShopping::Request:0x244a36c,
@url="http://open.api.ebay.com/shopping?version=547&appid=my_ebay_app_id_1234567&callname=FindItems&QueryKeywords=Cadillac",
@affiliate_shopper_id="my_campaign", @affiliate_partner="1", @site_id=nil, @affiliate_id=nil, @callname=:find_items,
@call_params={:query_keywords=>"Cadillac"}, @app_id="my_ebay_app_id_1234567",
@full_response={"Version"=>"547", "Timestamp"=>"2008-01-13T13:20:27.641Z", "Build"=>"e547_core_Bundled_5879814_R1",
"Item"=>[{"ShippingCostSummary"=>{"ShippingType"=>"NotSpecified"}, "ListingStatus"=>"Active", "TimeLeft"=>"P20DT16H59M6S",
"PrimaryCategoryName"=>"eBay Motors:Cars & Trucks:Cadillac:STS", "Title"=>"Cadillac : STS",
..."ItemSearchURL"=>"http://search.ebay.com/ws/search/SaleSearch?fsoo=2&fsop=1&satitle=Cadillac",
"Ack"=>"Success", "TotalItems"=>"15580", "xmlns"=>"urn:ebay:apis:eBLBaseComponents"}>

>> response.total_items
=> 15580

To get the items from the response, just ask for them


>> first_item = response.items.first
#<EbayShopping::Item:0x2413a88 @gallery_url="http://thumbs.ebaystatic.com/pict/230212386614.jpg",
@all_params={"ShippingCostSummary"=>{"ShippingType"=>"NotSpecified"}, "ListingStatus"=>"Active",
"TimeLeft"=>"P20DT16H59M6S", "PrimaryCategoryName"=>"eBay Motors:Cars & Trucks:Cadillac:STS",
"Title"=>"Cadillac : STS", "ConvertedCurrentPrice"=>{"currencyID"=>"USD", "content"=>"9500.0"},
"GalleryURL"=>"http://thumbs.ebaystatic.com/pict/230212386614.jpg", "ItemID"=>"230212386614",
"ListingType"=>"FixedPriceItem", "EndTime"=>"2008-02-03T06:19:33.000Z", "PrimaryCategoryID"=>"124117",
"ViewItemURLForNaturalSearch"=>"http://cgi.ebay.com/Cadillac-STS_W0QQitemZ230212386614QQcategoryZ124117QQcmdZViewItem"},
@view_item_url_for_natural_search="http://cgi.ebay.com/Cadillac-STS_W0QQitemZ230212386614QQcategoryZ124117QQcmdZViewItem",
@end_time="2008-02-03T06:19:33.000Z", @primary_category_name="eBay Motors:Cars & Trucks:Cadillac:STS",
@converted_current_price={"currencyID"=>"USD", "content"=>"9500.0"}, @title="Cadillac : STS",
@item_id="230212386614", @time_left="P20DT16H59M6S">

The key attributes for the item are available through ruby-ized version of the ebay Attributes (full documentation for the Shopping API calls and responses)


>> first_item.title # for the Title attribute
=> "Cadillac : STS"
>> first_item.gallery_url # for the GalleryURL attribute
=> "http://thumbs.ebaystatic.com/pict/230212386614.jpg"
>> first_item.view_item_url_for_natural_search # for the ViewItemURLForNaturalSearch attribute
=> "http://cgi.ebay.com/Cadillac-STS_W0QQitemZ230212386614QQcategoryZ124117QQcmdZViewItem"
>> first_item.bid_count
=> nil
>> first_item.primary_category_name
=> "eBay Motors:Cars & Trucks:Cadillac:STS"

As you can see, most of these responses are just strings. For the price, you’ve got a couple of options


>> first_item.converted_current_price
=> #<EbayShopping::Money:0x1410b70 @content=9500.0, @currency_id="USD">
>> first_item.converted_current_price.content
=> 9500.0

or


>> first_item.converted_current_price.to_s
=> "$9500.00"

It’s also worth noting the end time is returned as a Ruby Time object, so you can do calculations against it


>> first_item.end_time
=> Sun Feb 03 06:19:33 GMT 2008
>> first_item.end_time.class
=> Time

Finally, there’s a catch_all [] method which allows you to access other attributes using a familiar hash key notation:


>> first_item["ShippingCostSummary"]
=> {"ShippingType"=>"NotSpecified"}

Other methods and usage are given in the code comments and the fairly extensive test suite (browse the source here). There are also hooks to allow for caching and (separately) error caching, which is necessary if you want to get your app approved as a Compatible Application, which allows you a greater number of API calls per day (I did). I’ll post on usage of these and examples if anyone wants me to.

Tie that into your Rails app, and you’ve got an instant mash-up:
Ebay items on Autopendium

At the moment, the library’s only available as a RubyonRails plugin, rather than a Ruby gem. The only reason for this is that it was extracted from a Rails app, and is slightly structured accordingly (e.g. the YAML config file, and option for different settings in different environments). However, it’s probably not a huge job to package it as a gem, or to use the code as is in a standalone Ruby app.

p.s. Some of the less frequently used API calls haven’t yet been implemented, but are being done bit by bit, and if anyone’s got a crushing need for one of the missing ones, let me know, and I’ll bump it up the priority list.

Written by ctagg

January 13, 2008 at 5:47 pm

Rails 2.0 gotcha: count_from_query plugin

leave a comment »

I’ve got a fairly comprehensive test suite for Autopendium :: Stuff About Old Cars, the classic car community site I run, which makes upgrades of the framework fairly stress-free.

By stress-free, however, I don’t mean trouble-free — there are going to be failing tests, and there are going to be problems. However, I’m fairly confident that if the tests pass, the update to the production server will be without problems (particularly since I’ve started using a staging server).

So it was with upgrading to 1.2.6 — I had only a couple of deprecation warnings, and some failing tests, most of which were due to some problems with my routing.

Upgrading to 2.0.2 has proved a bit trickier however, mainly because the error messages (and they are errors, rather than fails) aren’t helping me in finding the root cause of the problem, only telling me what ultimately brings the whole thing crashing down.

Running the unit tests via the console I get this horror:

Rails 2.0 unit test errors

OK. Let’s take this bit by bit. So I run the unit tests for WikipediaEntry:

Rails 2.0 unit test passes

Hmm. This smells… and the smell is called… fixtures.

Log story short, by trawling through the test logs, using ruby-debug, and getting to grips with the ActiveRecord code, I found out that the count_from_query plugin I use (which makes generating counts from complex custom finders a cinch) wasn’t playing well with the new ActiveRecord behaviour, which has changed a bit since the 1.2 branch, nor with the new faster fixtures.

[As an aside, this sort of thing is why if you’re serious about using Rails you must learn Ruby, and why it’s a good idea only to use lightweight plugins you can understand.]

The offending line in the plugin is towards the bottom, where the plugin’s methods are added to ActiveRecord::Base.

  def self.included(receiver)
    receiver.send :include, ClassMethods
    receiver.extend(ClassMethods)
  end

This method (or callback) is invoked when, to quote the Pickaxe, “the receiver is included in another module or class”. Thus, “receiver.extend(ClassMethods)” extends ActiveRecord::Base with the methods which are contained in the ClassMethods module.

OK, this makes sense, as the methods consist of the count_by_query method and a method_missing, which tests whether the called missing method ends with _count and calls count_by_query if it does. This means if you have a class method called #find_complicated_stuff you can call #find_complicated_stuff_count, which is great and makes will_paginate, for example, much easier in some edge cases.

The problem lies with the previous line: “receiver.send :include, ClassMethods”, which includes the code (and method definitions) in the ClassMethods module, which has the effect of including the methods as instance methods. This doesn’t work with Rails 2.0 for two reasons:

1) If we get a tag cloud, for example, which might look something like this:


    find(:all, {
          :select => "tags.*, COUNT(*) AS tag_count", 
          :joins => "INNER JOIN taggings",
          :conditions => "taggings.tag_id = tags.id", :group => "tags.name",
          :order => "tag_count DESC, name ASC",
          :limit => 10})

The resulting count for each of the returned tags can then be accessed through #tag_count. Except that this will be intercepted by the method_missing in the count_by_query plugin (not sure why this didn’t happen in 1.2 — perhaps to do with the load order of the plugin?)

2) When a failure occurs in the #count_from_query method because of how it works we’ve left #find in an unstable state, which means calling find (which the fixtures code does) results in doing a count. Result: kaboom!

For me the solution was to delete the offending line (looking at the specs this may be intentional behaviour rather than a bug, but I can’t really see a use for it, and certainly don’t need it).

Written by ctagg

December 30, 2007 at 7:08 pm

Facebook, hyperactive APIs and lightweight libraries

with one comment

Probably the single biggest annoyances with writing (and running) a Facebook app is the remorseless changing of the API, to the extent that you begin to wonder sometimes whether it’s worth it.

For those who’ve not yet made the plunge, we’re not talking about minor tweaks, or once-in-a blue-moon essential maintainence that’s introduced over several months.

Oh no. With the Facebook API, you get major, app-breaking changes that are introduced with just a few weeks’ notice. The latest one got people so excited, it even made TechMeme’s front page:

TechMeme Facebook API change

Recent highlights include the deprecation of the whole method of sending invitations to other users, which is a central part of most applications, yet was scheduled to happen in “around 2-3 weeks” after the announcement.

So what’s a person to do? Duck out of the whole Facebook thing? Hope that OpenSocial solves all your problems, providing a standardized, flexible, and stable interface to a who bundle of social networks? Don’t hold your breath — it’s only at version 0.5 at the moment – and they are quite open that it’s going to change, and change rapidly.

The other option is to hang on and ride out the bumps, which is what I’m doing with Autopendium, the old-car community website I run, which gets a steady stream of new users thanks to its Facebook app, which keeps your profile and feed up-to-date with your old cars.

The most recent change I had to sort out was the introduction of templatized (ugh!) feed notifications, which will be replacing feed.publishActionOfUser in time.

In many respects, this was an easy change, certainly much easier than changing the whole invite procedure, which required rewriting the controller actions both for showing friends to invite, and sending the invitations (at least it would have been, if I’d carefully read the API).

The Autopendium app updates users feed whenever they add a new vehicle, or add a running report entry to one of their existing vehicles, using my favourite method of observers, specifically using a general FacebookObserver class, and my lightweight facebook library.

class FacebookObserver < ActiveRecord::Observer   
  observe Vehicle, Post
    def after_create(obj)
      update_facebook_info(obj)
   end  

  private   
  def update_facebook_info(obj)     
    user = obj.user
    return true unless @facebook = user.facebook # instantiate facebook session object from user
    update_profile_box_of(user) # update their profile
    publish_action_about(obj) # and their newsfeed too
  end

  ...    

  def  publish_action_about(obj)
     title = "<fb:userlink uid='#{@facebook.user}'/> added a new vehicle on Autopendium"
     body = "#{obj.title}: #{obj.description[0..80]}..."
     @facebook.fb_feed_publish_action_of_user(:title => title, :body => body)
   end
 end

To change this to the new feed.publishTemplatizedAction you only need to change a couple of line, specifically:

  def  publish_action_about(obj)
     title_template = "{actor} added a new vehicle on Autopendium"
     body_general = "#{obj.title}: #{obj.description[0..80]}..."
     @facebook.fb_feed_publish_action_of_user(:title_template => title_template, 
        :body_general => body_general, :actor_id =&gt @facebook.user)
   end

What makes this such a breeze is having all the Facebook updating done in an observer, rather than in the various controllers, and, I think, the fact that the Facebook library is so transparent, meaning it’s easy to change, add or remove methods as Facebook does.

There is a downside, however, and it’s that fairly eloquently explained by Chad Fowler, when discussing his Facebook library, Facebooker. In brief, “If Facebooker didn’t hide the implementation details of the XML API from its end-users, a change in the XML API would require every application which uses Facebooker to change.”

Essentially, the great thing about it is that the app (and the developer) doesn’t need to worry about Facebook changing its API, because that’s the library’s job. Which is great. We like things that make our life easier. Except…

Except that when something’s changing as much and as fast as the Facebook API, I think it falls down. I think there are three main problems with this ‘black box’ approach:

  1. It assumes the black box is updated as frequently as the API. Now in the case of Chad’s library, that seems to be the case (it’s had well over 100 commits and even has support for templatized actions). However, I got myself into trouble with Ruby/Amazon, which is no longer supported and is stuck at version 3 of Amazon’s API (and thus dead from early next year). I moved to the much more straightforward amazon-ecs, which, ironically, I have abstracted somewhat.
  2. It assumes the API changes can be fully handled by the black box without you having to be aware of them — and I just don’t think that’s the case with Facebook’s API. Even with something as well written and maintained as Facebooker, you still have to subscribe to Facebook’s developer news, and change your code to take account of it, particularly with something like notifications.sendRequest, or feed.publishTemplatizedAction because they force you to change your app or force Facebooker to change its API.
  3. It reduces the time you have to make those changes (again this is not so much of a problem with something like Facebooker, although I’m not sure it’s been formerly released yet) while you wait for the library to catch up, and adds opacity to the process.

So, for me (a relative newbie), a lightweight Facebook library seems the way to go, at least until the API settles down a bit, (and I tend to think it’s not dissimilar to some of the moves Rails has made in moving to 2.0).

Written by ctagg

November 21, 2007 at 9:11 pm

When is a record really updated_at in Ruby on Rails (and the under-overlooked Observer class)?

with 4 comments

Here’s one. When is an item ‘updated’? Well, if you’re using Ruby on Rails, and its automatic timestamps (in particular, updated_at) it’s whenever you save that object. Which is great. One less thing to think about.

However, in the real world, it’s a little more complicated. Take someone’s house. Sure, it changes when the owner changes the core attributes — such as add add another floor to it. But haven’t they also updated it if they give it a makeover, perhaps changed the garden, given it a new front door, painted the windows a different colour, stripped out the mouldings and go all minimalistic inside.

House remodelling —<p> photo by joeltellingPhoto by joeltelling

Depending on how you’ve stored the core attributes of the house (and how much you’ve normalised your models — the house has_many :rooms, has_one :garden, etc), you could completely revamp the house, and it still wouldn’t be updated, at least as far as Rails’ timestamps are concerned. In the case of Autopendium, our old-car website, we’ve had a similar situation. A user’s vehicle has_many :posts (vehicles are essentially blogs devoted to that car, with a few bells and whistles, such as todo lists, associated models and resources, etc).

When would I, as a user, want to be told that a vehicle had been updated? Mainly, when there’s something new been written about it — which, in the case of a restoration, or customisation, usually correlates with something being done to the car. So, we want updated_at to be updated not just when the vehicle record is saved, but also when a new post is created.

Fortunately this is a cinch to solve. If you stop thinking of updated_at as some scary Rails-magic, and think of it as another attribute (albeit one that helpfully gets automatically dealt with when the record is saved), you realise you handle it the same way you’d handle other object whose state depended on that of another one.You could do something like this:

class PostsController < ApplicationController
   def create
     @post = Post.find(params[:id)
     if @post.update_attributes(params[:post])
      @post.vehicle.update_attribute(:updated_at, Time.now)
      ....

A better method is to move it out of the controller and into the Post model. A simple after_create callback should do the trick:

class Post < ActiveRecord::Base
   after_create :timestamp_vehicle
   ....
   private
   def timestamp_vehicle
      vehicle.update_attribute(:updated_at, Time.now) #assumes post belongs_to vehicle and so has a vehicle method
   end

Or you could use the often-overlooked Observer. To quote the Rails API, “Observer classes respond to lifecycle callbacks to implement trigger-like behavior outside the original class.”

So, we create a Observer for the Post model:

class PostObserver < ActiveRecord::Observer
  def after_create(post)
    @post.vehicle.update_attribute(:updated_at, Time.now)
  end
end

In fact, we can slim this down even more, given how the vehicle’s updated_at attribute will be magically updated when we save the vehicle:

class PostObserver < ActiveRecord::Observer
  def after_create(post)
    @post.vehicle.save
  end
end

To activate it we just need to add the following to the environment file in the initialization section:

config.active_record.observers = :post_observer. 

Updated vehicle screenshot

Job done. Both the model callback and the Observer are a helluva lot better than the huge number of SQL craziness when you list a load of vehicles and want to indicate whether each one has been updated (and really is no more an offence against normalisation than counter_cache is).

However, Observers really come into their own when you’ve got multiple models triggering the same behaviour. In the example of a house, you probably want the house to be ‘updated’ when when the colour its painted is changed, when the style is changed, and so on. Here you’d probably wrap it up in a single Observer which watches a whole load of models. Something like:

class UpdatedHouseObserver < ActiveRecord::Observer
  observe Exterior, Style, Garden
  def after_save(record)
    record.house.save
  end
end

I’m also using it in conjunction with my lightweight Facebook library to update a user’s Facebook profile when they add a post or a vehicle (using code in the controller or AR callbacks gets really messy for that).

Update 1: I’ve now posted some details of how I use Observers with the Facebook library.

Update 2: There’s another good post on Observers by Pat Maddox, and specifically a plugin he’s done to make testing with Observers a bit easier.

Written by ctagg

November 9, 2007 at 6:01 pm