ActsAsFlinn

JSON-P Rack Handler

Mon, 16 Jun 2008 21:58:00 -0400

Updated 2008-06-19 – better support Halcyon

JSON-P Rack Handler

Juicing Ruby

I’ve been trying to find ways to squeeze all the juice out of Ruby lately. So many blog posts talk about how Rails doesn’t scale, Ruby is slow, blah blah. I had a convo with another developer at work today that went something like this:

Me: if we’re willing to do anything for performance we’d we just switch to Java

Jared: Yeah let’s not do that.

Switching to Java would be a big trade off in performance but also in development time. That’s a big trade off that none of us around here think is worth. That’s where this Rack handler comes in.

JSON-P Caching

I’ve posted before about how to use JSON-P in Rails and how to cache JSON-P in Rails with fairly decent results (500+ reqs/sec) but I felt like I could do better. Action caching always seemed like the best way to cache the full JSON output of a request but the fact that jQuery uses a dynamic callback takes action caching out of the equation.

Speeding up JSON-P

500+ reqs/sec is good and all but I felt like if I could action cache then somehow pad that cached JSON result with the callback I’d get better performance. I thought I’d see what Merb could offer on the JSON-P front. In my quest to juice Ruby my test setup looks like this: Merb, Datamapper, Memcached (the c gem), Memcached (the server) and Ebb. I’m really impressed with Merb and Datamapper in terms of development and I’m equally impressed with Rack and Ebb for performance.

I read recently it’s possible to use Rack to filter results to gzip output, which got me thinking. Why not try to do the same to pad my action cached JSON. Well it is possible and I’m squeezing out JSON-P at ~1200 reqs/sec with the JSON-P Rack Handler with my test stack.

JSON-P Rack Handler


# config/jsonp.rb
class JsonP
  def initialize(app)
    @app = app
  end

  def call(env)
    status, headers, response = @app.call(env)
    request = Rack::Request.new(env)
    response = pad(request.params.delete('callback'), response) if request.params.include?('callback')
    [status, headers, response]
  end

  def pad(callback, response, body = "")
    response.each{ |s| body << s }
    "#{callback}(#{body})" 
  end
end

Config


# config/rack.rb
require 'config/jsonp'
use JsonP
run Merb::Rack::Application.new

In Halcyon


# runner.ru
...
require 'config/jsonp.halcyon'
use JsonP
run Halcyon::Runner.new

Rails Support Soon

One last thing… whenever Rails starts using Rack, you’ll be able to use this in your Rails app.

About Me: I’m a developer with Sports Technologies, a Rails firm specializing in community focused web applications aimed at sports and entertainment. We’re always looking for Rails talent, if you’re looking to work on rewarding high profile projects with a seasoned team of professionals give us a shout.

Cross domain RESTful JSON-P with Rails

Fri, 13 Jun 2008 21:49:00 -0400

Rails based JSON-P

For the last 2 months I've been working on part of a project for a large publisher rolling out some new web services that use Rails based JSON to display comments on static web pages. This project has been a learning experience for nearly everyone involved on the Rails side and on the front end development side. We’ve overcome a number of limitations with the lack of a safe data transport in web browsers and limitations in the way Rails handles JSON-P and cross site REST. Some of the problems are unique to the solution we’re providing but they are no doubt a thorn in other developers’ sides. This is a follow up to a post I made about a month ago called JSON-P on Rails with JQuery. Below I’ve laid out some of the Rails problems and solutions related to JSON-P, cross domain JSON using JSON-P, JSON-P with jQuery and Rails caching JSON-P.

JSON-P with Rails
JSON-P with jQuery
Rails Caching JSON-P
Cross Domain RESTful JSON-P

JSON-P with Rails

My son: “What are you doing?”

Me: “Writing a blog post about JSON-P.”

My son: “Who is Jason P.?”

Me: “Javascript Object Notation with Padding.”

My son: “Oh.”

So as you hopefully know JSON-P isn’t some guy working in the back office of your IT department. JSON-P is a powerful data transport method in use all over the internet by some heavy hitters like Yahoo!, CNN, ESPN and lots more. JSON-P is a hack of sorts to get around a few problems like the dreaded Same Origin Policy. Plain old JSON is great but it can’t do cross site/cross domain. Short of constructing a same origin proxy JSON-P is your best option for cross site AJAX. The best part about JSON-P is you can use it as an alternative to XMLHttpRequest to transport JSON data across sites.

JSON-P Structure

JSON-P can be expressed a number of ways:

Embedded in an HTML request as an invisible iframe that includes a callback to instantiate JSON into the global scope. (clunky)
With a static callback wrapping JSON (good for server cache, bad because of browser cache).
With a dynamic callback wrapping JSON (harder to server cache).

Out of the box Rails best supports option #3 like so.


render :json => @chats, :callback => params[:callback]

This could result in the following JSON-P response, foo being the callback param you submitted as part of your query string.


foo(
 {"chats": [
   {"user": "actsasflinn", "created_on": "June 04, 2008", "id": 109328, "body": "Hey there Jim, I have never seen that kind of hat before!.  Where did you get it?"},
   {"user": "jim", "created_on": "June 04, 2008", "id": 109329, "body": "It's an ass hat. I got it from my ex."}
  ]
 }
)

JSON-P with jQuery

Using the above you can use some built-in jQuery methods to make life easy once again. In case you don’t know, jQuery is the balls! jQuery’s solution to the same origin policy is support for the script tag transport. jQuery will handle the transport magic if you specify a callback=? as a query param (yes question mark - jQuery fills it with a dynamic callback name in for you). This tells jQuery to add a dynamic callback and to use padding and the script tag transport. Check out the getJSON method example using the dynamic callback query param.


$.getJSON("http://example.com/chats.json?callback=?", function(data){
  $.each(data.chats, function(i,chat){
    alert(chat.body);
  });
});

Rails Caching JSON-P

The dynamic callback does a few important things like defeating the browser cache but it kills Rails’ ability to do the same. Action caching the resulting JSON data doesn’t work because jQuery changes the callback name with each request. Overcoming the lack of caches_action isn’ t so bad as long you use data caching within your format.json block. The to_json method is pretty intensive and it’ll kill your reqs/sec so it makes sense to cache the output from to_json. Using cache_fu something like the below will get you rolling with 500+ reqs/sec...


class Chat < ActiveRecord::Base
  acts_as_cached

  def self.recent_chats(format = nil)
    chats = find(:all, :order => "created_on desc", :limit => 10)
    chats.send(format) unless format.blank?
  end
end

class ChatsController < ApplicationController
  def index
    respond_to do |format|
      format.json do
        @json_chats = Chat.caches(:recent_chats, :with => :to_json)
        render :json => @json_chats, :callback => params[:callback]
      end
    end
  end
end

It’s not as great as action cache but it minimizes the db hit and the to_json processing time (which is hefty) reducing the response to simply padding the memcached JSON.

Cross Domain RESTful JSON-P

If you’ve gone this far JSON-P is working wonders... unless of course you want to do AJAX POSTs (argh)! So this next bit of mojo is indeed a hack and not for the faint of heart. As you might know most browsers don’t support the HTTP DELETE and PUT methods so Rails spoofs them by making an POST and passing in a query param called _method. Wonderful but you can’t do an AJAX POST across domains. The AJAX REST Nazi says “no REST for you!” You can of course monkey patch Rails to use _method regardless of the actual HTTP method enabling you to spoof AJAX and REST across domains.


module ActionController
  class AbstractRequest
    def request_method
      @request_method ||= begin
        method = (parameters[:_method].blank? ? @env['REQUEST_METHOD'] : parameters[:_method].to_s).downcase
        if ACCEPTED_HTTP_METHODS.include?(method)
          method.to_sym
        else
          raise UnknownHttpMethod, "#{method}, accepted HTTP methods are #{ACCEPTED_HTTP_METHODS.to_a.to_sentence}"
        end
      end
    end
  end
end

Achtung! Monkey patching with the above will expose your create method without using an actual post. Imho no big deal, others might be more cautious (CAPTCHA is always an option), ymmv. On to the controller...


class ChatsController < ApplicationController
...
  def create
    @chat = Chat.new(params[:chat])
    respond_to do |format|
      if @chat.save
        format.json { render :json => { :chat => @chat }, :callback => params[:callback], :status => :created, :location => @chat }
      else
        format.json { render :json => { :errors => @chat.errors.full_messages }, :callback => params[:callback], :status => :unprocessable_entity }
      end
    end
  end
end

Using jQuery to serialize your form, a spoofed AJAX POST might look something like this.


$(document).ready(function() {
  $('#chat_form').submit(function() {
    var chat = $('#chat_form').serializeArray();
    $.getJSON("http://example.com/chats.json?_method=post&callback=?", chat, function(data){
      if (data.errors == undefined) {
        alert(data.chat.body);
      } else {
        $.each(data.errors, function(error) { alert(error); })
      }
    });
  });
});

Huzah! Easy right? It took quite a while to put all this together into a cohesive process that everyone could work with. At the onset Rails didn’t support spoofing REST, action caching JSON-P or even cross domain JSON but with a little ingenuity we made it happen and you can too. JSON-P is the solution to put your RESTful Cross Site XMLHttpRequest woes to bed.

p.s. none of the above code has been tested so if you copy and paste it and it doesn’t work, don’t complain unless you include working code.

Passenger mod_rails - I'm a believer.

Sun, 01 Jun 2008 02:55:00 -0400

I was initially skeptical of Passenger aka mod_rails (because it seems to good to be true) but tonight I became a believer. We recently finished work on fairly high profile project. The installation seemed to be running fine but after installing monit regular tests revealed mongrel instances were hanging or unresponsive and randomly coming back online. After 3 days of the mysterious mongrel issues, apache tweaking of the proxy config, replacing mod_proxy_balancer with haproxy, and experimenting with other backends like thin and ebb I decided to give Passenger a shot. The gem was super easy and the apache install application was super easy and very helpful. After installing Passenger the new site is rocking. Goodbye mongrel.

CAPTCHA Sucks

Sun, 27 Apr 2008 17:59:00 -0400

I was on Slashdot and there was actually something I cared to comment about… then I got this.

Then I left.

Rails Hosting Options

Mon, 21 Apr 2008 21:26:00 -0400

Rails Hosting

I get a lot of questions about Rails hosting from this blog, so I’ll take a minute to post about some of my experiences.

Site5 Rails Hosting

If you’re looking for hosting for a small website like a typo or mephisto blog Site5 is a great hosting option. I’ve had great luck with them over the last few years. They have a Rails based control panel called backstage which is tied to a customized version of CPanel called SiteAdmin. The engineers at Site5 are Rails guys and they’re always doing some innovative stuff. They also have reseller plans if you need to manage client websites. This blog has been hosted there for a while now and because of that I highly recommend them for Shared Rails hosting.

Slicehost Rails Hosting

When you’re looking for a more dedicated hosting solution for Rails apps Slicehost is the place. You get a virtualized Xen host with the latest stable Ubuntu. They’re perfect for hosting Rails apps that need a little more juice than a shared hosting solution will give you. I’ve had great luck with them in terms of uptime and support. It’s one of those types of places that you just don’t have to worry about once you’ve got an app going. If you’re looking for an excellent Rails hosting environment, especially for development Slicehost is perfect.

Honorable Mentions

I’ve had luck with Rimuhostng in the past. They’re great operation and the support was always really good. They also have an out of the box easy Rails stack install script that makes running a current Rails environment simple.

Non-Starters

I haven’t had a Dreamhost account myself but I have managed an account for a client there and from what I saw the control panel felt hacked, the servers were overbooked and the support wasn’t good. That client switched from them after they wanted more money for a dedicated IP address and more money for support and more money for less value than what other had for the money. I’m sure everyone has heard of the fact that they charged their customers over $7 million after on of the programmers ran a billing script with a bug in it (yikes)!

JSON-P on Rails with JQuery

Sun, 20 Apr 2008 13:22:00 -0400

JSON-P based Comment

Over the last several months I’ve had the pleasure of developer JSON based commenting solutions for publishers. I couldn’t be happier using JSON as a transfer method for moving ActiveRecord based object from the database almost directly to the consumer. I’ve run into some difficulties here and there mainly due to the Same Origin Policy which is a bastard of a browser rule that makes sending JSON across domains difficult. Even though prototype.js seems to support sending requests across domains there is no native script transport in prototype and worse yet it appear some browsers won’t allow AJAX across subdomains to overcome the SOP rules. I searched all over the web and found a few hacks for adding the script tag to prototype.js but I found the best solution was built native into JQuery’s getJSON. I was really taken with JQuery’s ability to handle the script transport method out of the box but I think even more I’m taken with JQuery’s syntax. I had a conversation with Less Everything’s Steven Bristol about 6 months ago and he asked if I had checked out JQuery. I had briefly looked at it but I wasn’t wowed by it (probably because I didn’t use it on a project). Steven: “isn’t it the best thing you’ve ever seen?” Me: It’s ok. (*my answer to everything – ask my wife). Six months later I get to use it on a project and my answer to you Steven is YES it is the greatest.

JSON-P

Back to the subject… So the problem is that a browser doesn’t want to send a request across domains for fear that your secret info will be compromised by some would be hacker. The solution JSON padded with a callback method combined with a plain old script tag aka JSON-P. With JQuery the idea is that you can pass a URL into JQuery’s getJSON method with a ? for JQuery to bind it’s on changing function name that handles a callback. Sound like a hack? Yes but everyone is doing it. I’ve seen some skeptics but overall this is an accepted solution. And until JSONRequest is accepted as a safe cross site transport JSON-P is here to stay.

Does Rails support JSON-P?

Bet your ass it does. Right out of the box Rails supports the callback option to be passed in on a :json render. Like so:


render :json => @comments, :callback => params[:callback]

JSON-P Problems on Rails

So the issue you’ll run into primarily with JQuery is the changing name of the callback. JQuery binds a random function name to the request (presumably to protect from attacks of some sort, maybe for anti caching). The issue is that with action caching caches the entire response body meaning the next call has a stale cache (because of the now changed callback name) or has to generate a new response for a new callback name. While JSON is typically very light weight the to_json method can be expensive and when you’re dealing with high traffic situations you always want to squeeze every bit of performance out that you can. We also do paginated result sets with will_paginate so it means there is a bit of post processing after a plain old find_all_by_blah… The solution I’ve been using is a mix of cache_fu for model level caching along with fragment caching the to_json results then interpolating the cached json into the callback. Sound like a hack? It is but it feels a bit better when you crunch out over 500/reqs/s as opposed to 14/reqs/s.

For the future

Let’s cross our fingers for CouchDB. I’ve been experimenting with Couch over the last week or two and I’m really pleased with the possibility of a RESTful document based database with native JSON and coming support for MapReduce. The future looks bright, I can imagine not so distant support for native ruby object level persistence. Yay!

Handling APIs with Ruby XML Parsing

Fri, 25 Jan 2008 22:08:00 -0500

Have you ever wanted needed to write a Ruby wrapper for an XML based API?

If you have a burning desire to seamlessly exchange data over the web or you just want to use the latest Interweb 2.0 service – you’re most likely contemplating writing your API client in Ruby… yes that’s why you’re here isn’t it?

Why Use Ruby to Parse XML?

It’s a fact – Ruby kicks ass at parsing XML. You can find tons of examples of XML API clients written in Ruby.

ActiveResource parses XML and handles RESTful HTTP

The new ActiveResource Rails gem found in Rails2 makes pretty light work of handling XML APIs via REST. Unfortunately not everyone is rushing to support REST just yet. If you support a big Rails 1.2 app you can’t just run out an add ActiveResource to your project (which is my case). This post is not about REST or ActiveResource so if you’re looking for that, click the link you just ~~clicked~~ skipped over.

Enough with the useful useless Ruby XML facts…

Show me how to write a Ruby XML API wrapper

I’ve been working with an API recently for a third party registration system on a project we’re rolling out soon. The third party provides their API using domain scoped query URLs and HTTP GET params and returns XML documents.

Huh?

http://example.com/aaflinn/lookup?username=billlumberg&password=swingline

XML Messages

When you get a matching user/pass combination you get something like this.

<?xml version="1.0" encoding="ISO-8859-1"?>

<auth>

    <user>

        <username><![CDATA[billlumberg]]></username>

        <fullname><![CDATA[Bill Lumberg]]></fullname>

        <zipcode><![CDATA92131]></zipcode>

        <email><![CDATA[bill.lumberg@initech.com]]></email>

    </user>

</auth>

When you put the wrong password you get an error like so.

<?xml version="1.0" encoding="ISO-8859-1"?>

<auth>

  <error><![CDATA[Invalid username/password combination]]></error>

</auth>

If you don’t enter a username at all you’ll get an error thusly.

<?xml version="1.0" encoding="ISO-8859-1"?>

<auth>

  <error><![CDATA[No username given.]]></error>

</auth>

Pretty easy, no?

API Wrapper Concepts

Here are some concepts that I felt were important when I started writing my wrapper.

Simple – it should be easy to code (I hate writing stupid code)
DRY – if it’s worth writing the wrapper make sure it’s reusable
Self Documenting – it should be as self documenting as possible (rdoc)
“Exceptional Code” – it should raise errors on exceptions and handle errors from the libraries it makes use of
Don’t Spam – Don’t abuse APIs (grrr!)

Requirements

The wrapper should use a GET query string to perform a query to the service provider passing an md5 hashed password and username combination. If the user exists parse the XML result document and return an instantiated user object based on the XML. If no user exists raise some type of rescuable error (RecordNotFound).

Additionally the wrapper should be able to create a new user. This particular service provider uses an HTTP GET query string but in some cases you might find a plain old POST like you’d see in a form or you’ll need to build XML and pass that back to the service provider (which I am not doing here). The wrapper should be able to be able to interpret error messages and determine the status of our create request in a graceful way.

Ruby API Wrapper by Example

The names have been changed to protect the innocent. I’ve edited the wrapper a bit to reduce some complexity and renamed it to hide the actual API provider. Read on in the comments of the code, I’ve done everything on that bullet list and you can read the code as you read the comments setting up just about every line of code written.

# =Example API Ruby Wrapper=

# 

# Usage

#

# === setup ===

#   require ‘ApiExample’

#   ApiExample::account = ‘test’

#   ApiExample::logger = Logger.new(‘example.log’)

#

# ===Find A User===

# user = ApiExample::User.find(‘bill.lumberg’, :password => ‘swingline’)

#

# ===Create A User===

# user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)

#

require ‘base64’

require ‘digest/md5’

require ‘net/http’

require ‘rexml/document’

require ‘cgi’

require ‘logger’

module ApiExample

  # Example Database Host

  HOST = ‘www.example.com’

  # These params are required to register a new user

  REGISTRATION_PARAMS = [ :username, :password, :email ]

  # ApiExample account (brand account not user account)

  @@account = nil

  @@success_url = nil

  @@fail_url = nil

  @@logger = Logger.new(STDERR)

  mattr_accessor :account, :logger, :success_url, :fail_url

  # Exception handling

  class ApiExampleError < StandardError; end

  class UnexpectedError < ApiExampleError; end

  class RegistrationError < ApiExampleError; end

  class RecordNotFound < ApiExampleError; end

  def md5password(password)

    Base64.encode64(Digest::MD5.digest(password)).strip

  end

  # Using Struct here allows us to make our object act similar to an

  # ActiveRecord object.  In this example it’s not so obvious of a pain

  # in the ass it is but the real class has about 20 or so attributes

  # Using means I don’t have to create attribute read and writes

  # attribution – I got this idea from the Ben Vinegar’s

  # <a href="http://rubyforge.org/projects/freshbooks/">freshbooks gem</a>

  User = Struct.new(:username, :email, :password, :fullname, :zipcode)

  # Extend the Struct attributes by adding the class and instance methods we want

  class User

    attr_accessor :attributes, :errors, :new_record

    # class method for create a new user

    # Usage:

    # user = ApiExample::User.create(:username => ‘bill.lumberg’, :password => ‘swingline’, :email => ‘bill.lumberg@initech.com’)

    #

    def self.create(attributes = {})

      object = new(attributes)

      object.create

      object

    end

    # class method for finding an existing user

    # Usage:

    # ApiExample::User.find(‘bill.lumberg’, ‘swingline’) # plain text will be sent md5 hashed rather than clear text

    #

    def self.find(username, password)

      query_params = { :username => username, :password => ApiExample::md5password(password_option[:password]) }

      query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) }.join(’&’)

      # @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable

      uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/lookup", :query => query)

      # I’ve got a bunch of these loggers throughout which are used to debug, remove as you see fit.

      ApiExample::logger.debug("URI: #{uri}")

      # Make the actual HTTP Request

      result = Net::HTTP.get_response(uri)

      # Parse the XML Result of the HTTP Request

      response = REXML::Document.new(result.body)

      ApiExample::logger.debug("RESPONSE: #{response}")

      # Check to see if there is a user node, if there’s not raise an error similar to ActiveRecord

      unless response.elements[’//user’].nil?

        attributes = {}

        # Members is the Struct method for the attributes we setup…they correspond to what

        # we expect the XML to return, so lets only handle what we know

        members.each do |field_name|

          node = response.elements[’//user’].elements[field_name.to_s]

          next if node.nil?

          attributes[field_name.to_sym] = node.text # you can do casting here if you need, I did

        end

        object = allocate

        object.attributes = attributes

        object

      else

        # Raise an error, setting the message to what the API sets as the error in XML

        # if there is no ‘error’ node in the XML set an ‘unknown error’ message

        raise RecordNotFound, (response.elements[’//error’].text rescue "Couldn’t find ApiExample::User with Username: #{username} because of an unknown error!")

      end

    end

    # instance method to setup new object ala ActiveRecord

    # Usage:

    # user = ApiExample::User.new(:username => ‘bill.lumberg’)

    #

    def initialize(attributes = {})

      @new_record = true

      self.attributes = attributes

    end

    # instance method ala ActiveRecord

    #

    def errors

      @errors = [] if @errors.blank?

      @errors

    end

    # instance method to get an attributes hash ala ActiveRecord

    #

    def attributes

      butes = {}

      members.each{ |member| butes[member.to_sym] = self.send(member) } # you can cast here, I did

      butes

    end

    # instance method to set attributes via a hash ala ActiveRecord

    #

    def attributes=(new_attributes = {})

      members.each do |member|

        unless new_attributes.has_value?(member.to_sym)

          # #{member}= because you might possibly write an attribute writer to handle the input to =

          self.send(member+’=’, new_attributes[member.to_sym]) # you can cast here, I did

          ApiExample::logger.debug("#{member}: #{self.send(member).inspect}") # for snooping

        end

      end

    end

    # instance method to create the new instantited object ala ActiveRecord

    # note the difference in create and self.create are the same as in ActiveRecord

    # Usage:

    # user = ApiExample::User.new(:username => ‘bill.lumberg’)

    # user.password = ‘swingline’

    # user.email = ‘bill.lumberg@initech.com’

    # user.create

    #

    def create

      # we pass our success and fail URLs even though we aren’t using them for their intended purpose

      query_params = { :success => ApiExample::success_url, :fail => ApiExample::fail_url }.merge(attributes)

      ApiExample::logger.debug(query_params.inspect) # for snooping

      # check to make sure all required params are included

      if ApiExample::REGISTRATION_PARAMS.all?{ |param| query_params.include?(param) }

        # URL Encode everything

        query = query_params.collect{ |k, v| [k, v].map{ |kv| CGI::escape(kv.to_s) }.join(’=’) unless v.blank? }.compact.join(’&’)

        # @@account doesn’t need to be encoded because we are setting internally, the other stuff is vulnerable

        uri = URI::HTTP.build(:host => ApiExample::HOST, :path => "/#{ApiExample::account}/register", :query => query)

        ApiExample::logger.debug("URI: #{uri}") # for snooping

        # Make the actual HTTP Request

        response = Net::HTTP.get_response(uri)

        ApiExample::logger.debug("RESPONSE: #{response}") # for snooping

        # Check for failure and errors

        if response[‘location’].include?(ApiExample::fail_url)

          # Raise an error for each message

          CGI::parse(URI::parse(response[‘location’]).query)[‘error’].each do |error_message|

            raise RegistrationError, error_message

          end

        elsif response[‘location’].include?(ApiExample::success_url)

          # Change the status to reflect the fact that we’ve saved the object…this could get much more

          # in-depth in terms of ensuring our data was correctly saved… you wouldn’t expect to do

          # that kind of thing in an RDBMS so why here?

          self.new_record = false

        else

          # it wasn’t the fail url, it wasn’t the success url so what the hell was it?

          raise UnexpectedError, "Unknown response URL!"

        end

      else

        # For each missing required param add an error message

        ApiExample::REGISTRATION_PARAMS.each do |missing_param|

          self.errors << "#{missing_param} can’t be blank." unless query_params.include?(missing_param)

        end

      end

    end

  end

end

Rails Deployments Tips

Wed, 16 Jan 2008 10:04:00 -0500

Very often when projects go into production there are a few tweaks needed to make sure the app works correctly in its production environment and stays working. Below are some tips I’ve come up with from my experience rolling rails apps. Most of them are no brainers and some probably apply only to the Rails development we’re doing at work.

Rotate Rails Log Files

Log files get big really quick on high traffic sites. Some server apps rely on the operating system or a log rotating script to archive or delete old logs. Rails has a built in feature thanks to Ruby’s Logger class. The following code will keep 50 archived logs each 1mb in size and automatically rotate the log out once it hits 1mb.

In conf/environments.rb


Rails::Initializer.run do |config|
  # ...
  config.logger = Logger.new(File.join(RAILS_ROOT, 'log', "#{RAILS_ENV}.log"), 50, 1.megabyte)

Ignore Sensitive Parameters in Logs

Sick of seeing your bank password showing up in your Rails based shiny new web 2.0 app? Yeah I know don’t use your bank password but hey if you’ve done it users of your shiny new web 2.0 app probably will too. Keep them safe by keeping their passwords out of your logs. The following hides the password and password_confirmation params from your logs.

In conf/environments.rb


# Include your application configuration below
ActionController::Base.filter_parameter_logging :password, :password_confirmation

Robots.txt

If you have a development site that isn’t password protected to the public for whatever reason you’ll want to make sure the development site isn’t getting indexed by search engines and cluttering up results for your production site.

Create a file called robots-development.txt and on your development site use mod_rewrite to point to the file.


User-agent: *
Disallow: /

Password Protect Mod Proxy

So this goes along with the last topic. If you’re using password protection along with mod proxy you’ll notice that your assets are password protected but page requests still make it to mongrel. Painful on your search results if msn, yahoo or googlebot come to visit.


<Proxy balancer://mongrel_cluster>
    AuthName "Keep Out" 
    AuthType Basic
    AuthUserFile htpasswd.users
    Require valid-user

    BalancerMember http://192.168.1.2:8000
    #...

Turn on /etc/init.d scripts!

This seems like a no-brainer but if you installed apache from source and/or you are using mongrel cluster you’re doing this manually.

Copy the script from the mongrel_cluster resource directory to /etc/init.d


# cp /usr/local/lib/ruby/gems/1.8/gems/mongrel_cluster-1.0.5/resources/mongrel_cluster /etc/init.d/mongrel_cluster
# chmod +x /etc/init.d/mongrel_cluster

On Redhat


# chkconfig --level 345 httpd on
# chkconfig --level 345 mongrel_cluster on

On Debian


# update-rc.d httpd defaults
# update-rc.d mongrel_cluster defaults

More to come…

And We're Back!!!!

Sun, 13 Jan 2008 12:39:00 -0500

Some friends recently reminded me that my blog has been down for almost 2 months. So I’ve managed to spend some time to bring the site back up in Typo 5. It kind of goes without saying that deploying rails apps on shared hosts sucks and I can stand behind that. But that’s OK – right tool for the right job.

In the last two months there’s been a lot written about Ruby. There’s been some bad press (zed and friends) and a lot of good press (Rails Machine, Rails 2, Ruby 1.9). Some crying fowl about Rails not being enterprise ready, the community sucking, it’s hard to deploy or whatever else. Who cares! Most of the complaints and accusations I’ve read are from people that need to point the finger in the mirror.

While my site was down a lot changed. Anyone keeping track of me (yeah lots of you I’m sure) knows my startup was a non-starter and I spent 2 long dreadful PHP filled months at a search marketing company and have since moved on to more reputable endeavors with a Connecticut firm building Rails based entertainment apps. I’m working with a team of experienced producers and engineers and I couldn’t be happier. So you can expect some new Ruby and Rails tidbits here again soon.

One last thing to mention before I sign-off…if you read my post mentioning PHP’s lack of late static binding PHP and ActiveRecord you can rejoice or cringe depending on your perspective because the team is adding the static:: runtime resolution that will get PHP that life extending kidney implant that it so desperately needs. This new feature will allow Zend and friends to build a pragmatic Active Record in PHP. I’ve bolded this because a friend of mine said they are fixing the problem but it’s not a fix – it’s no doubt a feature request from Zend that was originally slated for PHP6. Given the slow adoption rate of PHP5 I’m sure the team decided to add almost all the new goodies originally slated for PHP6 to PHP5.3 in an effort to keep up. With all the new (untested) features I’m sure sysadmins will be rushing to deploy this new version.

Derek Sivers Blaims Rails for Project Mismanagement

Sun, 23 Sep 2007 11:01:00 -0400

Some of you might have seen the O’Reilly article by former Rails trumpeter Derek Sivers entitled 7 reasons I switched back to PHP. Or the Slashdot article, even more ominously named Thinking about Rails? Think Again. In typical fashion, Slashdot ran the link with a suggestive name which I’m sure will effect the credibility of Rails in the eyes of the uninitiated masses of PHP developers trolling the website.

I happened to read the article earlier in the morning before it made it to Slashdot and didn’t think too much of the arguments behind the article. I can honestly say from my own experiences that mismanagement of a project, not limitations of a framework or language are usually to blame for missed deadlines, and project failures. I can say that from experience after developing the same project in both PHP and Rails with about 1/3 of the tables (and probably logic).

Anyone with a lick of common sense can tell you that if you’ve got a legacy application with 90 tables, migrating that data and accompanying application to an MVC framework is going to be a big project. With a 2 person team, 1 of which is the owner of the company (who should be focusing on more important things), the project is going to take a long time, cost a lot of money and the likelihood of failure is high.

I think Derek’s logic and motivations for pursuing the project should be at question, not his choice of programming language or framework. Why do companies go into business? To make money of course. What possible reason does a company then have to rebuild it’s entire infrastructure from the ground up? To either make more money, or save money ie. make more money. The 7 reasons he switched back should have coincided with business considerations before he started the project not after the project failed. I think it’s shameful to place the blame of the project’s failure on the language and framework choice.

I’d love to hear what bitsweat honestly has to say about the project but we can be sure either an NDA or professional courtesy will keep that from coming out.

The comments at the end of the article do this opinionated scapegoat of an article justice.

Useless post without a concrete illustration.

Show us an example of:
1) What you tried to accomplish.
2) How you tried to implement it in rails.
3) The rails code that "didn't work." 
4) The "beautiful" PHP code you created instead.
Richard Hertz | September 22, 2007 08:03 PM

I especially like this one:

I'm a little reluctant to add to the wasteland that is this post and these comments, but here goes.

I'm familiar with the situation here. The deal was this: Derek was not a programmer; he was a musician. He learned some PHP and cobbled together the old CDBaby site by himself. It was good.

Then, he heard about Rails, and became infatuated with it. He proceeded to attempt a rolling rewrite of CDBaby's frontend and backend both (the backend is large, because of inter-label and digital distribution stuff) in Rails.

At this time, Derek had no experience with the following things:

* any language other than PHP
* systems integration and interoperability
* Rails
* object-orientation
* the MVC pattern
* managing a development team

Project fails. All right. As he has learned in #2, legacy compatibility trumps everything. Also, ship early and often.

As you can see in Derek's post about MySQL encodings, he's not always the clearest thinker. Even above he says that REST means POST-only destruction, which misses the point entirely.

His team was fine (mostly just Jeremy, until another developer was hired in the last months). Rails was fine. But there were a lot of things wrong with the project plan ("rewrite everything, eventually") and with the project leader, who was convinced he had found a silver bullet.

No framework saves you from your own inexperience.

Out.
wellwisher | September 23, 2007 01:47 AM