>Car or Auto Make-Model-Year Database : For Breakfast

>

Make Model What?

If you like me were tasked with loading a database of recent car makes/models/years, you would start by looking on the web and seeing if someone else just has it out there, readily available, hopefully for free, but perhaps for a tiny nominal fee.?

If only it was that simple…

I looked and looked, and couldn’t find anything that would fit the above requirements. So I thought, who would know about US car models better than Kelly Blue Book? So I went on their site, and sure enough they have a javascript file that lists all known to them makes and models of used cars. Since the file is public, I figured it’s not really “evil” if I scrape and parse it for my own benefit. Disagree? Have a better source? Then leave a comment.

Anyway, to cut the long story short, I’m hoping to save a day or so to someone else who may, like me, be looking for this information. The ruby module shown below retrieves and parses the javascript from KBB site into a Ruby data structure of the following form – basically a hash, keyed on make, then on model with list of years as a value:

>> Constants::Auto::DB.keys.sort[0..5]
=> ["AMC", "Acura", "Alfa Romeo", "Audi", "BMW", "Bertone"]
>> Constants::Auto::DB["Subaru"].keys.sort[0..5]
=> ["B9 Tribeca", "Baja", "DL", "Forester", "GL", "GL-10"]
>> Constants::Auto::DB["Audi"]["A4"]
=> ["1999", "2007", "1998", "2006", "2005", "1996", "2004", "2003", "2002", "1997", "2001", "2000"]
>> Constants::Auto::DB["BMW"]["X5"]
=> ["2003", "2002", "2001", "2000", "2005", "2007", "2006", "2004"]

The idea is that you could load the initial hash:

@models = KBB::Parser.new.to_hash

and then save the output of @models.inspect in your local constants file – hence me using Constants::Auto::DB (I actually have a Rake task for doing this — let me know if I should post it too). Then you would just re-run this every time you think new car models are added/changed on KBB. Realize, that hitting their site every time you need the data is clearly evil. So use this class to load the data initially, save the result of inspect() call into a ruby file, and use that cached version in your app. Re-run the load every time you want to update your database.

Please let me know if you find this code useful, or if you find a better/cleaner/more comprehensive way of maintaining car make/model/year database.

#
# author: Konstantin Gredeskoul © 2008
# license: public domain
#
require 'net/http'
require 'uri'

module KBB
 MODELS_URL = "http://scripts.kbb.com/kbb/ymmData.axd?VehicleClass=UsedCar"

 class Models
   def initialize(js)   
     @models = {}
     @makes = {}
     n = /ymUsed_\[\d{4}\]\s*=\s*'([^']+)'/
     m = /ymmUsed_\["(\d+)~(\d+)"\]\s*=\s*"([^"]+)"/
     js.split(/\n/).each do |line|
       next if line.strip.blank?
       if matched = n.match(line)
         matched[1].split(/,/).each do |token|
           id, name = token.split('|')
           @makes[id.to_i] = name
         end
       end
      
       if matched = m.match(line)
          year, make_id, models = matched[1], matched[2], matched[3]
          models.split(/,/).each do |t| 
            id, model_name = t.split('|')
            make_name = @makes[make_id.to_i]
            @models[make_name] ||= {}
            @models[make_name][model_name] ||= []
            @models[make_name][model_name] << year
          end
        end
      end
    end
    
    def to_hash
      @models
    end
  end

  class Parser
    def initialize
      @m = Models.new(Net::HTTP.get(URI.parse(MODELS_URL)))
    end
    def to_hash
      @m.to_hash
    end
  end

end

About these ads