Building Web Apps

Web 2.0 Expo: Social Media

Thu, 09 Apr 2009 04:32:09 GMT

The conference sessions at Web 2.0 Expo had a major emphasis on social media. I only attended a couple of these talks, about which I have brief comments below; at the end of the article, I have links to several others.

Social Media Marketing - why it fails and how to fix it

This one felt somewhat remedial to me, with the essential points being:

Be authentic. You need to be a person, not just a representative of a company.
It's hard to change the marketing culture in an organization to deal well with social media. (A point that I was glad to gloss over, being in my tiny organization that doesn't have this sort of problem.)
It's a conversation. (If in doubt, go read The Cluetrain Manifesto.)

The Whuffie Factor

Taking a completely different approach to similar material, Tara Hunt gave what I found to be the most inspiring and engaging talk of the conference: The Whuffie Factor.

Since both the slides and the video have been posted, I'll let her speak for herself.

Slides

Whuffie at Web 2.0 Expo

View more presentations from Tara Hunt.

Video

The Whuffie Factor: The 5 Keys for Maxing Social Capital and Winning with Online Communities (Tara Hunt) from Steffan Antonas on Vimeo.

Other Social Media Talks

Web 2.0 Expo: Web Developer Tools

Thu, 09 Apr 2009 05:39:08 GMT

Ben Galbraith and Dio Almaer of Ajaxian and the developer tools group at Mozilla gave a wide-ranging talk on web developer tools. They noted that the web often seems more like a hack than a platform — it's amazing that developers are able to make web apps do the things they do. This is especially true with Ajax raising the bar for interactivity, and sites turning into applications instead of documents. They're out to make the web a better platform.

Some future technologies that have them excited:

HTML5 canvas
Fast JavaScript — kicked off by Chrome, Firefox's TraceMonkey
Web workers, giving web apps the ability to have background threads
Desktop integration, such as Fluid, Prism, Adobe Air, and Titanium

As part of its efforts to improve the web as a platform, Mozilla is working on an open web tools directory, as well as a variety of development projects.

Ben and Dio noted that Firebug blew away existing tools and set a new standard for web development tools, providing a comprehensive set of tools in the browser itself. Other browsers have followed with IE8 and Safari 3.0 both providing good tools.

Mozilla Labs has a relatively new project called Bespin, a web-based code editor that is intended to be a showcase for HTML 5. This project is still young, and it won't run on an flavor of IE for a while, but it looks very promising.

Mozilla Labs is also working on a grid-based layout system using JavaScript and HTML5 canvas; so far, there's not much on the web site about this project, but an announcement is due soon.

Web 2.0 Expo: TV and Radio with an API

Thu, 09 Apr 2009 05:41:55 GMT

National Public Radio (NPR) is changing what it means to be a news organization. While traditional news organizations want to keep their content close, NPR has taken the approach of making content widely available via an API – an approach they call "brand and release." They've made 250,000 stories, going back 13 years, available through their API.

Zack Brand from NPR said that the NPR API is getting about 2 million requests per month, and there are 1,300 registrants (you have to register to get access to the API). Content is available in a variety of formats; so far, 51% of requests use XML, 20% RSS, and 23% widgets.

(Another notable new organization that has been making some content available via an API is the New York Times, which has an article search API, among others.)

Also participating in this session was Robin Sloan from Current.tv. Current made interesting use of Twitter streams as part of its presidential debate coverage. The hash tag they created for people to tag their posts, #current, was the third most popular Twitter tag during the debates, bested only by the tags for the candidates themselves.

Current built an application to provide a mash up of various data sources, including Digg, on election day. In true agile fashion, they made multiple software updates even after the show began broadcasting!

Web 2.0 Expo San Francisco Spring 2009

Mon, 06 Apr 2009 22:02:46 GMT

March 31 - April 3, San Francisco

Last week's Web 2.0 Expo seemed like a successful event, with a good-sized crowd despite the economic malaise – due in part, no doubt, to unusually aggressive discounting and promotion. With multiple tracks across four days (one day of workshops and three of conference sessions) there was much more to see than any one person could possibly attend.

Here's a few articles that we've written about talks at the conference:

Many of the presentations are available online, as slides and/or videos:

And to go beyond the presentations and get more flavor of the event:

Enhancing Conditional Routing in Rails

Wed, 27 Aug 2008 20:21:08 GMT

Rails’ routing infrastructure supports the concept of conditional routes: preconditions that must be satisfied before a particular route will trigger. Rails 2.1 supports one built-in condition, HTTP method checking, which is of some use but rather limited. What I needed was to be able to limit certain routes to only trigger when a particular host-name was used to access the application.

I thought I’d have to write messy additional logic until a little comment tucked away in ActionController::Routing::RouteSet and ActionController::Routing::Routing caught my eye. Here I briefly show you how to leverage this functionality for your own purposes.

The Goal — Conditional Routes in routes.rb

Let’s work backwards and see the result I was aiming for. I wanted to expand the existing capabilities of the routing engine and be able to restrict routes to specific hosts. The conditional routing option works by adding a parameter to your route specifications. Here are some examples:

map.with_options(:controller => 'feeds', :conditions => {:hosts => MY_HOSTS}) do |feed|
  feed.feeds_articles '/feeds/articles', :action => 'articles'
  feed.feeds_podcast '/feeds/podcast', :action => 'podcast'
end

map.resources :podcasts, :conditions => {:hosts => MY_HOSTS}, 
   :member => {:show_notes => :get, :transcript => :get},
   :collection => {:admin => :get} do |podcast|
     podcast.resources :comments, :member => {:report_as_ham => :get, :report_as_spam => :get}
   end

or even

map.connect ':controller/:action/:id', :conditions => {:hosts => MY_HOSTS}

In Rails 2.1, however, no such option :hosts exists, only an option to check the HTTP method via :method.

The Implementation

I haven’t really ever needed to use the conditional routing support before, and didn’t really think about it due to it only supporting the HTTP method check. For that reason, I originally thought I’d have to write my own logic, either patching existing Routing routines (nearly right!) or by writing new stuff that could get messy (bad idea).

During a last scan through the code for the keyword “conditions”, I saw this comment:

# Plugins may override this method to add other conditions, like checks on
# host, subdomain, and so forth. Note that changes here only affect route
# recognition, not generation.

Good, a place to start afterall! The solution is elegant as it only requires overriding two simple routines. You can do this in your own app by writing code that gets loaded at startup. Here is one implementation in its entirety:

require 'action_controller'

module ActionController
  module Routing
    class RouteSet
      def extract_request_environment(request)
        { :method => request.method, :host => request.host }
      end
    end

    class Route
      def recognition_conditions
        result = ["(match = #{Regexp.new(recognition_pattern).inspect}.match(path))"]
        result << "conditions[:method] === env[:method]" if conditions[:method]
        result << "conditions[:hosts].include?(env[:host])" if conditions[:hosts]
        result
      end
    end
  end
end

My code is very simplistic and tuned for my needs, but gives you an example of where to patch in. Here, I simply supply a list of host names I care about, and check the incoming host against that list.

Use extract_request_environment to parse out and store any data you will want to use in your conditional checks. This data will be available in the env hash later on.

recognition_conditions generates an Array of String objects that contain the Ruby code that will be used to build dynamic conditional test methods when the routing engine compiles the routes data in routes.rb.

I drop the source file into my project’s pre-existing lib/plugins/action_controller_extensions/lib directory as action_controller_extensions.rb and include an init.rb loader stub in my lib/plugins/action_controller_extensions directory:

require 'action_controller_extensions'

My app deals with loading up such “plugins” at startup. You may have a different set-up. You can get the same effect by putting a require for the main source file in your startup code.

It would be great to see other generally useful conditionals contributed by the community.

Lab 8 (Bonus): A Page Model and Cleanup

Thu, 24 Jul 2008 08:00:46 GMT

This final RailsQuickStart lab write-up describes changes we’ve made to the sample application. Some changes are just code clean-up, while others implement some of the additional features suggested at the end of each lab.

To use the new code, check it out from our shared Subversion repository (see the Lab 7 writeup if you need a refresher on how to do this) and migrate to the current database version (rake db:migrate).

Page model

The most dramatic change is the addition of a Page model, and its associated pages_controller and views. This replaces the old static_controller, which we’ve purged. By using a model to store the content and metadata for each page in the database, an administrator can update page titles and set page description and keyword metadata without having to edit code.

Controller and view code

The page model is administered just like any other, with the usual create and edit views.

To render each page, we added an action called simply “view”. The method in pages_controller is:

  def view
    unless @page = Page.find_by_name(params[:name])
      render :action => 'error' and return
    else     
      login_required if @page.protect
      @current_tab = @page.navtab
    end
  end

We find the page by its name, and if there is no page of a matching name, we render the error page. Assuming it is found, we check its ‘protect’ attribute, and if that is set, we require login to view the page. Finally, we set the @current_tab value, which is used by application layout to highlight the appropriate tab.

The “view” template is very simple, since all it is doing is rendering the contents of the page body attribute:

   <%= render_to_string :inline => @page.body %>

Note that we are not just using <%= @page.body %> here, because we want the body to be intepreted as ERb. Because the render_to_string method is not normally available in views, we need to add this line to pages_controller to make it available as a view helper:

	helper_method :render_to_string

The joys and dangers of ERb

The body for each page, as read from the database, is interpreted as ERb, so you can use helpers and other code. This gives you complete control over what goes on each page; you can use the helpers we wrote for content blocks and images to include items from those database tables, and you can use the usual ERb code to display data from other models.

There is a danger to this, however, which you should be aware of: there is no restriction on what code can go in an ERb block, so it could, for example, delete everything from the database. It can even issue system commands and wipe the file system. With great power comes great responsibility. Presumably, the only people who would be allowed to log in and edit the pages would be people you could trust.

If you want to use a similar approach but in a safer way, take a look at the Liquid templating language. It is similar to ERb for displaying data but does not allow destructive operations or access to system commands.

Creating pages

In addition to the migration that creates the Page model, we added a migration that populates the three pages that used to be handled by the static controller. You can edit the content of these pages by going to the main admin page, clicking on Pages Admin, and then clicking the Edit link for the appropriate page.

If you add new pages, you’ll need to add them to the navigation bar, or put a link on another page somewhere, so users can access them. You could take this design a step further by creating the nav bar from the Page model, so that each new page would automatically get a nav button.

Note that pages that are created from other models, such as the resources page, are independent of the Page model, and for those pages, the page title and metadata are set via instance variables in the controller, just as we did in the static controller. We’ve made changes to the application layout so the description and keyword metadata can be set, in addition to the page title.

Putting metadata in the layout

We want to take the metadata (title, description, and keywords) for each page and put it in the page header, so we need to do that in the layout. These lines go in the head section of views/layouts/application.html.erb:

   <% if @page %>
      <title><%= @page.title %></title> 
      <meta name="keywords" content="<%= @page.keywords %>" />
      <meta name="description" content="<%= @page.description %>" />
   <% end %>

If there is a page object (remember that some pages, like the resources page, are not generated from page objects), then we render the HTML required for each piece of metadata, pulling the contents from the appropriate attribute of the @page object.

Supporting routes

We added a couple of routes to support the Page model:

   map.root :controller => "pages", :action => "view", :name => 'home'
   map.connect ':name', :controller => "pages", :action => 'view'

The first route makes the page named “home” function as the home page of the site.

The second route causes any URL that consists of a single word to invoke the view action in the pages controller, and pass that word as the :name parameter.

Resource page enhancements

We modified the view for the resources page so it doesn’t show a category heading if there are no links for that category. This is just a one-line change in links/resources_page.html.erb:

   <% if category.links.size > 0 %><h2><%= category.name %></h2><% end %>

We also added a route for the resources page so the simple URL “/resources” would work:

   map.resources_page '/resources', :controller => 'links', :action => 'resources_page'

Since we gave the route a name (after “map.”), we can refer to it in other places, such as in the nav link in the application layout, as resources_page_path.

User management

We added an admin page that lists all the users, and allows you to delete users and to change their passwords. Take a look at users_controller and views/users/index.html.erb to see how this works. Here’s the heart of the index view:

<table>
<% @users.each do |user| %>
   <tr>
      <td><%= user.login %></td>
      <td><%= user.email %></td>
      <td><%= link_to "Delete", user, :confirm => 'Are you sure?', :method => :delete %></td>
      <td><%= link_to "Change Password", :controller => 'users', :action => 'change_password', :id => user %></td>
   </tr>
<% end %>
</table>

On each row of the table, we show one user’s login and email, and provide links to delete that user or change their password.

Creating the first user

We had disabled the requirement for login to access the users controller so you could create your first user, but obviously this makes the entire system completely insecure. So we’ve re-enabled the login requirement.

This creates a quandary if you’re starting with an empty database: you can’t log in to create your first user if there are no users. You could use the Rails console to create a user, but to make this easier, we added a migration that creates a user “admin” with the password “seminar” (in the file db/migrate/013_create_admin_user.rb):

   def self.up
      User.create(:login => 'admin', 
                  :email => 'noone@anydomain.com', 
                  :password => 'seminar', 
                  :password_confirmation => 'seminar')
   end

You should either change the password (which you can now do via the Users Admin page) or delete this user if you’re using this code in a production environment.

Making things a little prettier

We’ve added some styling to the forms, so they’re a little more respectable. This includes changes in the html pages, as well as some additional CSS styles. We’ve also linked the standard Rails scaffold.css file in the application layout, which provides styling for validation error messages and highlights fields with errors.

There’s one thing we don’t like about the way the standard Rails error display works. Fields that have errors are wrapped in a div that has an ID of “fieldWithErrors”. This lets the stylesheet apply different styling to fields that have errors.

But a div is the wrong element to use for this; for one thing, it messes up the layout of the forms when there is an error. We added a little code to environment.rb that makes Rails use a span, instead of a div, to avoid this problem:

   ActionView::Base.field_error_proc = Proc.new{ |html_tag, instance| "<span class=\"fieldWithErrors\">#{html_tag}</span>" }

This is a little obscure, but we didn’t have to figure it out ourselves; a little bit of Google searching with a description of the problem revealed this solution.

On the new subscription page, we combined all the error messages into one block by listing all of the models on one line:

   <%= error_messages_for :subscription, :customer, :creditcard %>

Finally, we added some styles to our quickstart.css style sheet that override some of the styles in the standard Rails scaffold.css. We also added a little default styling for the table element so the admin pages would look a little better.

Testing Purchases Through Mock ActiveMerchant Gateway

In the code we developed previously, we weren’t testing the creation of a subscription (where ActiveMerchant gets called). In this bonus lab, we refactored SubscriptionsController.create to lend itself to testing. We needed to make a few changes:

change the create action itself to allow swapping in a test gateway object;
change the environment setup files to use the proper gateway depending on whether you were in test, development, or production;
and add some tests.

Fortunately, ActiveMerchant provides many tools to help with testing. The PeepCode Active Merchant PDF is an excellent resource that you should check out.

Refactored SubscriptionsController.create

We refactored the code to look like this:

def create
  # render :inline => "<%= debug params %>" and return
  @current_tab = "subscribe_tab"
  @page_title = "Please correct the errors below"
  @subscription = Subscription.new(params[:subscription])
  @subscription.amount = Subscription.cost_for_period(@subscription.period)
  @customer = Customer.new(params[:customer])
  # create the creditcard object and get name from customer object
  @creditcard = ActiveMerchant::Billing::CreditCard.new(params[:creditcard])    
  @creditcard.first_name = @customer.first_name
  @creditcard.last_name = @customer.last_name
  # make sure all three models are valid
  @customer.valid?
  @subscription.valid?    
  #
  # Our test code uses bogus credit cards for signaling various conditions to the 
  # bogus gateway. In test mode, the creditcard is never "valid", so work around that
  # here. The better thing to do if you are going to do creditcard stuff in the future
  # is to refactor this logic into separate classes. See PeepCode's ActiveMerchant PDF
  #
  unless (ENV['RAILS_ENV'] == 'test' or @creditcard.valid?) and @customer.errors.empty? and @subscription.errors.empty?
    render :action => :new and return
  end
  # if everything looks ok, send the purchase request to the gateway
  # set up options hash with billing address and ip
  options = {
    :ip => request.remote_ip,
    :billing_address => { 
      :name     => @customer.full_name,
      :address1 => @customer.address,
      :address2 => '',
      :city     => @customer.city,
      :state    => @customer.state,
      :country  => 'US',
      :zip      => @customer.zip,
      :phone    => @customer.phone
    }
  }  
  begin
    response = Subscription.gateway.purchase(@subscription.amount, @creditcard, options)
    # redisplay form if not successful
    unless response.success?
      flash[:notice] = "Transaction failed: #{response.message}"
      render :action => :new
    else
      # save the responses from the transaction in the subscription record
      @subscription.message = response.message
      @subscription.reference = response.authorization
      @subscription.test = response.test?
      @subscription.params = response.params.to_yaml
      @subscription.customer = @customer
      @subscription.save!
      flash[:notice] = "Thanks for subscribing to our site!"
      redirect_to root_path
    end
  rescue ActiveMerchant::ActiveMerchantError => e
    # Consider much better error handling (of ActiveMerchant specific exceptions)
    flash[:notice] = "Transaction failed: #{e}"
    render :action => :new
  end        
end

The unless clause that tests for valid objects changed. Our tests will use a bogus credit card object that ActiveMerchant’s test code knows how to use. The fake credit card encodes whether the mocked merchant gateway should succeed or not. This means we can’t use the validation check when we are in test mode.

We also rewrote the gateway usage itself. Now, we use a gateway that is stored in a class variable of Subscription. The gateway is created at initialization time, and this allows us to swap in different gateways if desired. ActiveMerchant supplies a mocked gateway and we use that when testing.

Note we also put the gateway usage between exception handling code (the begin/rescue/end blocks). If something goes wrong in the gateway during testing, or during real use, we catch that problem and send the user back to the data entry form.

Environment File Change and Subscription Model Tweak

Since we are creating the appropriate gateway at startup, we need to do that in our environment files. The code for test.rb looks like:

config.after_initialize do
  ActiveMerchant::Billing::Base.mode = :test
  Subscription.gateway =
    ActiveMerchant::Billing::BogusGateway.new
end

We are simply putting ActiveMerchant into test mode and setting the class variable in Subscription to hold an instance of the mock gateway object called ActiveMerchant::Billing::BogusGateway.

The Subscription class needed a small change to add the variable:

class Subscription < ActiveRecord::Base

  belongs_to :customer  
  validates_numericality_of :amount, :message => "Choose a subscription amount"
  validates_numericality_of :period, :message => "Choose a subscription amount"
  
  # The payment gateway we want to use is set in our environments, so we can
  # swap it out for testing
  cattr_accessor :gateway
  
  def self.cost_for_period period
    if period == 1
      995     # $9.95 in cents
    elsif period == 12
      9995    # $99.95 in cents
    else
      "invalid"
    end
  end
end

cattr_accessor is a method that creates a class level set of data accessors for a variable, in this case called gateway.

Some Tests

We tweaked the test_helper.rb file to add a helper that makes a creditcard hash object for us:

def credit_card(options = {})
  { :number => '1',
    :first_name => 'Bob',
    :last_name => 'Sample',
    :month => '8',
    :year => "#{ Time.now.year + 1 }",
    :verification_value => '123',
    :type => 'visa'
  }.update(options)
end

and we use it in several functional tests in our subscriptions_controller_test.rb file:

def test_should_create_subscription
  assert_difference('Subscription.count') do
    post :create, :subscription => { :amount => 1, :period => 1 }, 
                  :customer => {:first_name => 'bob', :last_name => 'Sample', :address => '123 main', :city => 'San Jose', :state => 'CA', :zip => '95000'},
                  :creditcard => credit_card({:number => '1'})
  end

  assert_redirected_to root_path
end

def test_should_not_create_subscription_failed_auth
  assert_no_difference('Subscription.count') do
    post :create, :subscription => { :amount => 1, :period => 1 }, 
                  :customer => {:first_name => 'bob', :last_name => 'Sample', :address => '123 main', :city => 'San Jose', :state => 'CA', :zip => '95000'},
                  :creditcard => credit_card({:number => '2'})
  end
  
  assert_response :success
end

def test_should_not_create_subscription_activemerchant_exception
  assert_no_difference('Subscription.count') do
    post :create, :subscription => { :amount => 1, :period => 1 }, 
                  :customer => {:first_name => 'bob', :last_name => 'Sample', :address => '123 main', :city => 'San Jose', :state => 'CA', :zip => '95000'},
                  :creditcard => credit_card({:number => '3'})
  end
  
  assert_response :success
end

The only difference between these three tests is setting the number of the creditcard to 1, 2, or 3. These magic numbers are used by the mock object to signal three cases: 1) good validation; 2) bad credit card validation; and 3) a fault in the gateway. See the ActiveMerchant documentation for more info.

But wait, there’s more!

There’s an assortment of other changes throughout the code: adding validations for most model fields, making the defensio code a little more robust, and fixing some tests.

What’s next?

That’s all, folks. If you find any issues with the code, or want to make any suggestions, please post to the Google group.

Lab 1. Set-Up and Static Pages

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, you’ll create a new Rails site from scratch, building it up step by step until you have a four-page site with navigation. This creates the basis upon which all the other labs build.

1. Set-up development environment

The instructions below assume you have the following software installed:

Ruby 1.8.6
Rails 2.0.2
MySQL database server
NetBeans 6.0.1

You can use another editor or IDE, but then you’re on your own for the specifics of how to accomplish these tasks. We will, however, provide the command-line alternative to each of the NetBeans Rails commands.

If you are using NetBeans, you must have it configured to use your native Ruby interpreter, and not JRuby.

2. Create your Rails project and its associated databases

a. Create your local database

To create your database using the MySQL command-line client, type

	mysql --u root

in a terminal window. (Note: if you installed MySQL using macports, you’ll probably have to type mysql5 instead of mysql.) This assumes this is a new database server installation and you haven’t set a root password. If you have set a root password, enter

	mysql --u root --p

and it will prompt you for the password.

You should now have a >> prompt, which indicates that you’re in the MySQL monitor. Enter the following commands at this prompt:

	create database labs_development;
	grant all on labs_development.* to 'seminar'@'localhost' identified by 'seminar';

The first command creates the database, and the second command creates the user “seminar” with the password “seminar”.

You don’t need this quite yet, but while you’re at it you might as well create the test database, which is used for running tests:

	create database labs_test;
	grant all on labs_test.* to 'seminar'@'localhost' identified by 'seminar';

Finally, you can create a production database if you’d like, in case you later want to try running in production mode on your development machine:

	create database labs_production;
	grant all on labs_production.* to 'seminar'@'localhost' identified by 'seminar';

b. Create a Rails project

If you’re using NetBeans, Choose File > New Project > Ruby on Rails Project and enter “labs” as the name of the project. By default, NetBeans will create a folder called NetBeans Projects in your home folder, and then within that it will create a folder that matches the name of the project. You can change this if you want, but these defaults should be fine. Make sure that the MySQL database is selected.

When you click Next, unless you’ve updated to a newer version of NetBeans, it is likely to tell you that you have Rails 1.2.6, and that your gems are unavailable. Don’t worry, this is a NetBeans bug.

When you click Finish, you should see a bunch of file creation commands scroll by in the Output window (if that window isn’t visible, choose Window > Output > Output), and now in the Project pane on the left you should see your new project with all of its components.

Note that the NetBeans Projects tab does not show the literal folder structure, but rather a conceptual structure that is somewhat flattened relative to the actual folder structure and has more explicit names (e.g., Configuration instead of config). Switch to the Files tab to see the actual folder structure.

If you’re working from the command line instead, make a folder for your Rails project, change directory (cd) into that folder, and type:

	rails -d mysql labs

c. Configure your Rails project to use your database

Open the file database.yml in the config folder (Configuration if you’re in the NetBeans project view).

In the section that starts with “development:”, change the username from root to seminar, and the password from blank to seminar. Make the same changes in the test and projection sections.

We suggest also deleting the socket: line from the database.yml file from each of the three sections.

d. Run the application

You now have a new, empty Rails application. To run the application, if you’re using NetBeans, right-click on the project name in the Projects pane and choose Run. If you’re working from the command line, make sure you’re in the project’s folder, and enter “ruby script/server”. (If you’re using a Mac or Linux system, you can omit the word “ruby” in all such commands, but it won’t hurt, and is required on Windows.)

Now open your browser and enter http://localhost:3000 (that’s port number 3000 on the local server, which goes by the alias localhost). You should see the standard Rails “Welcome Aboard” page. (If not, look at the log in the output window to see what the error messages are.)

In the Welcome Aboard page in your browser, click on the link “About Your Application’s Environment”. You should see a list of versions that shows that you have Ruby 1.8.6 and version 2.0.2 of the various Rails components.

If you see an error message, it probably means that either your database server is not running, or you didn’t create the labs_development database, or you didn’t create the seminar user with the seminar password, or you didn’t edit the database.yml file properly.

Assuming all is well, now you want to get rid of this welcome page, because from this point forward it will only get in your way. Delete the file index.hmtl in your application’s “public” folder. (Or rename it to something like index.html.old if you want to be able to restore it for old time’s sake.)

Now refresh the page in your browser and you should see:

	Routing Error

	No route matches "/" with {:method=>:get}

Congratulations! You’ve gone to a lot of trouble to create an application that does absolutely nothing except display an error message.

But actually you’ve done a great deal. You now have all the infrastructure in place for your Rails application. Now the fun starts, and you’ll be able to start adding real pages very quickly.

This error message is Rails’ way of saying that it doesn’t know what it is supposed to do when you access the root page of your site, because we haven’t defined anything yet and we’ve taken away the default index.html page.

3. Creating your first pages

a. Creating your first controller and views

We’re going to start by doing the absolute minimum of work required to create a few static web pages within the Rails structure. As we progress through the lessons, we’ll evolve these to more sophisticated versions that are more representative of good Rails design practices.

We’re going to create a site with four pages:

Home
About Us
Contact Us
Resources

To do so, we need a controller, which we’re going to call “static” because we’re using it to access these static pages. Each of the four pages will be created by one action in this controller and its corresponding view.

If you’re using NetBeans, right-click on the name of the project in the Projects pane and choose Generate. The Rails Generator dialog opens, and it defaults to the “controller”, option, which is what we want. Enter “static” for the name, and in the Views field enter “home about contact resources”. Click OK, and the generator will create the controller and open it in the editor.

(If you’re not using NetBeans, enter “ruby script/generate controller static home about contact resources” in a terminal window, at the root of your application.)

In the file static_controller.rb, you’ll see four empty methods, one for each action. We can leave them empty for now.

Now look in the Views folder (inside the apps folder if you’re not in the NetBeans project view), and you’ll find a new folder called “static”, and inside that you’ll find the four view files for the pages we’ve defined.

To see the changes in the browser, you need to first stop and then restart the server. Using NetBeans, click the red X in the the output window labeled “Mongrel for labs on 3000” and then click the green double arrow. (If you’re using the console. type control-C to stop the server and then reissue the script/server command.)

Now click the refresh button in the browser. Still an error message! That’s because Rails doesn’t know which of these pages is our home page. We’ll fix that in a minute.

To see the home page, enter this URL:

	http://localhost:3000/static/home

That tells Rails we want it to run the home action in the static controller. You should now see a page with this exciting content:

	Static#home
	Find me in app/views/static/home.html.erb

That’s the default text Rails puts in the empty view file. Open the file views/static/home.html.erb, and you’ll see the HTML that created this page.

You can replace that HTML with whatever you’d like, perhaps:

	<h1>Welcome to RailsQuickStart</h1>
	<p>This is the home page</p>

Now you can refresh the page and you should see the new text. You can do the same for the other three pages, editing their view files, and entering the corresponding action name in the URL to access them.

b. Adding a layout

If you view the source of one of these pages in your browser, you’ll notice that they aren’t well-formed HTML. There’s no doctype, no head section, and so forth — just the raw HTML for the page. We could add that in each view, but that would involve a lot of repetition.

A layout file is the answer. Every view gets wrapped in a layout, so the view only needs to provide the information that is unique to that page.

In your Views folder, you’ll see an empty folder called “layouts”. In the NetBeans projects pane, right-click on layouts, choose New > ERB file, and name the file application.html. NetBeans will append the .erb suffix. (If you’re using another editor, you’ll probably need to specify the full filename, application.html.erb.) This layout file will be used by default for all views, unless there is another layout specific to that controller.

Here’s a minimal HTML wrapper you can paste into that file (first delete the useless boilerplate code that NetBeans inserts):

	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
	<html xmlns="http://www.w3.org/1999/xhtml">
	    <head>
	        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"/>
	        <title>Page Title Goes Here</title>
	    </head>
	    <body>
	        <p>
	            <%= yield %>
	        </p>
	    </body>
	</html>

This is just standard HTML, with the exception of a single line of ERb code:

	<%= yield %>

This line tells Rails that it should insert the code for the current view at this point.

Now you can refresh the browser page, and it should look just the same, except that “Page Title Goes Here” appears in the browser’s title bar. If you view the source, you’ll see that there’s now a proper HTML structure, courtesy of our layout.

c. Add basic navigation links

Now we need a way to navigate to each of our pages without having to enter the full URL. First, let’s add some simple navigation links, using an HTML list (which we’ll style a little later).

Insert the following code in layouts/application.html.erb, immediately after the <body> tag.

	
		<%= link_to ‘Home’, {:controller => ‘static’, :action => ’home’} >
    	<= link_to ‘About Us’, {:controller => ‘static’, :action => ’about’} >
    	<= link_to ‘Contact Us’, {:controller => ‘static’, :action => ’contact’} >
    	<= link_to ‘Resources’, {:controller => ‘static’, :action => ’resources’} %>

Now refresh the page in the browser, and you’ll see four ugly links, which at least allow us to easily navigate to each of the four pages.

d. Make the root URL work, and clean up the others

Now we can get to each of the pages, but we still get an error if we try to just browse to the root of the site. Let’s fix this.

Open the file config/routes.rb (in the Configuration folder in the NetBeans project pane). This default file is mostly full of comments that give you some hints about how to add routes, plus two default routes at the end. The first of those default routes is what has been enabling us to navigate to our pages.

To make our home view work as the site’s home page, add the following line immediately above the first default route (so your new line is now the first uncommented line):

     map.root :controller => "static", :action => "home"

This tells Rails that the root, or home, page of our site is accessed via the home action in the static controller. Save the routes.rb file, and now you should be able to browse to //localhost:3000 and see the home page.

There’s one more bit of cleanup we’ll do while we’re in the routes file. Note that when you click on the links to browse to the other pages, you still see /static/ before the page name. This is ugly, and there’s no need for us to expose the name of our controller to our visitors. Add this line immediately below the map.root line you just added:

     map.connect ":action", :controller => "static"

This tells Rails that if there is only a single word after the host name, then it should use that word as the name of the action, and assume that the controller is “static”.

Save the file, and now when you click on the links for each of the pages, you should see the simpler URL without the word “static”.

e. Add a stylesheet

Now it’s time to make this all a little prettier by adding a CSS stylesheet. You can use any stylesheet you’d like, but to follow along with our example, grab the file “quickstart.css” (it’s in the example code for any of the labs, or right-click here and choose Save Link As…) and save it in the folder /public/stylesheets.

Now you need to add a tag to tell the browser to use this stylesheet. Insert the following line in the head section of layouts/application.html.erb:

	<%= stylesheet_link_tag 'quickstart.css' %>

Note that you don’t need to specify that this file is in the stylesheets folder; Rails assumes that by default.

If you refresh the browser now, you should see that the fonts have changed, but it’s still just as ugly. We need to add some divs in the layout to tell the browser what styles to apply. This is just regular HTML/CSS stuff, nothing to do with Rails.

Just grab the updated layout file from the “labs2_end” sample application code (or right-click here and choose Save Link As…). This layout provides a “pagewrapper” div to give the content a defined width and center it; “header” and “navbar” divs for the top of the page; and a “footer” div for the bottom of the page.

The updated layout also adds an id tag to the navigation list, which we’ll explain in the next section.

With the new application layout in place, refresh the browser, and you should finally have something that looks almost like a respectable, if extraordinarily simple, web site.

f. Adding in page titles

All of our pages have the same HTML title (which appears in the browser’s title bar), which isn’t good. Let’s fix that.

When we slipped in the new layout file in the previous section, the title tag in the head section was changed to this:

	<title>RailsQuickStart: <%= @page_title %></title>

This provides the text “RailsQuickStart:” as the start of the title for every page, follow by a bit of ERb code that simply inserts the value of the @page_title instance variable. Now we need to set that variable somewhere.

You might think that the natural place to do this would be in the view files, but that won’t work; the layout has already rendered the head section of the page by the time it gets to the view file. So we need to set it in the controller.

So far, all of our controller actions are empty. Now add one line to the home action so it looks like this:

	def home
		@page_title = "Welcome"
	end

and make similar changes to the other three actions. Save the file, and now browse to each of the pages. You’ll see the appropriate titles now appear in the browser’s title bar.

g. Make the navigation buttons highlight

We’re almost done. There’s just one little improvement to make. It would be nice if the navigation buttons highlighted to show which page we’re on.

A standard CSS technique to make this happen is to use a descendant selector. We apply an id to the navigation ul element, with the value of the id indicating the current page. We’ve also added an id to each of the navigation links. A CSS rule highlights the navigation link if the navigation link’s parent has an id that corresponds to the link:

	#home_tab a#home_nav, #about_tab a#about_nav, #subscribe_tab a#subscribe_nav, #resources_tab a#resources_nav, #contact_tab a#contact_nav {
	    background-color: #333;
	}

This is just standard CSS, nothing to do with Rails. But we need to somehow change the id for the ul tag that wraps the navigation list, depending on which page we’re on.

This change has already been slipped in to the layout file we swapped in a couple steps ago. The tag that starts the list looks like this:

	<ul id="<%= @current_tab %>">

This sets the id for the ul element to the value of the instance variable @current_tab. Now we just need to set that in static_controller, just as we did for the page title. So the controller now looks like this:

	def home
    @page_title = "Home"
    @current_tab = "home_tab"
  end

  def about
    @page_title = "About Us" 
    @current_tab = "about_tab"   
  end

  def contact
    @page_title = "Contact Us"
    @current_tab = "contact_tab"
  end

  def resources
    @page_title = "Resources"
    @current_tab = "resources_tab"
  end

Now the tabs stay highlighted to indicate the page we’re on.

It may seem clumsy to have to set the tab and the page title in the controller like this, especially since they’re really aspects of the view. In a more elaborate design, we could pull these values from a database, so they could be set by a non-technical site administrator and didn’t clutter up the controller. But for now, this is the simplest way to do it.

Moving On

That concludes Lab 1. You now have a simple static site, which makes use of a few little bits of Ruby to set and use instance variables, and also uses a layout file to provide the parts that are common to all pages. This gives us a core structure that we can easily extend in the later labs.

Lab 2. Building a Simple CMS

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, we’ll create a mini content management system to enable an administrative user to edit the site’s content without having to modify the code.

To start this lab, load the lab2_start folder from the USB Flash drive. And if you have a question about how any of the code should look after completing these steps, take a look at the labs2_end folder.

1. Creating a text management system

a. Using the scaffold generator

We can create the heart of the code we’ll need using the scaffold generator, which creates a database migration, a controller, a model, views, and some test files, all with a single command.

There’s a graphical interface to the generators built in to NetBeans, but the current version of NetBeans doesn’t provide the proper interface for the Rails 2 scaffold. So we need to use the terminal. Open a terminal window, change to the root directory of your application, and enter the following command:

ruby script/generate scaffold content_block name:string body:text

(If you’re not using Windows, you can drop the “ruby” preface.)

This command tells Rails that we want to generate a model called content_blocks, with two fields: one called “name”, of type string, and one called “body”, of type text. (Strings are limited to 255 characters, whereas text fields can be arbitrarily long.)

Take a look at the files that the generator created:

models/content_block.rb — this is empty for now, but inherits a great deal of capability from ActiveRecord::Base
views/content_block/ — there are four view files here that create the scaffolded admin interface
controllers/content_block_controller.rb — the controller that implements the standard RESTful actions to create, edit, delete, and list the content blocks
db/migrate/001_create_content_blocks.rb — the migration file (in the NetBeans project view, find this at Database Migrations/migrate/

There’s more files in the test directory, but we’ll ignore those for now.

b. Creating the database table

The migration file provides the specification for creating the database tables. The t.timestamps line is added automatically, and it creates two fields: created_at and updated_at, which are automatically maintained by Rails. The id field is automatically generated, so it isn’t mentioned here.

To actually create the database table, we need to run the migration. In the NetBeans project pane, right-click on the project name and choose Migrate Database > To Current Version. The output window will show the actions that Rails takes. (If you prefer to work from the command line, the command is “rake db:migrate”.)

If you want, you can now examine the database (either with one of many MySQL graphical user interface programs, or by issuing commands with the mysql monitor) and see the table that was created and its fields.

c. Using the content blocks

Now you can browse to //localhost:3000/content_blocks and you’ll see the list of content blocks, which isn’t too interesting because there aren’t any yet. Click the New Content Block link, and you can now create the first content block. Name it “welcome” and enter some text, and click Create. You’ve now created your first database entry.

Notice that these pages are not using our default layout. That’s because the scaffold generator annoyingly creates a new layout for every model, and if there is a layout whose name matches the model, it is used instead of the default layout (which is named application.html.erb). So delete the automatically generated views/layouts/content_blocks.html.erb file, and access //localhost:3000/content_blocks again. Now the admin pages are using the layout.

Now, how do we use the data from the content block? We could write the code to find a content block by name each time we want to use one, but that would create a lot of redundancy. To avoid this, we’ll create a small view method, called a helper. There’s a helper file for each controller.

Open the file helpers/content_blocks_helper.rb, and add the following code between the two lines that are already there:

	def show_content name
	    content = ContentBlock.find_by_name(name)
	    if content
	        content.body
	    else
	      "No content for #{name}"
	    end
	end

This method first uses the find_by_name method to find the desired content block (Active Record automatically provides such a method for every attribute of every model). If the result is not nil, then the show_content_name method returns the body of the content block. If the result is nil, that means that there was no content block that matched the request name, so a message to that effect is displayed.

Now open the home page (views/static/home.rhtml.erb) and add the following line:

<%= show_content 'welcome' %>

Make sure you’ve saved all the files, and then browse to //localhost:3000. You should now see the content you entered when you created the “welcome” content block.

d. Using Textile markup

We don’t want our administrative users to have to enter HTML code, especially since they could “break” the page if they made an error in the coding. So we’ll use a simple markup language called Textile.

There’s two pieces of Ruby code we use to implement this. The first is the gem RedCloth, which you should already have installed. This is the code that translates Textile into HTML. The second is a Rails plugin called “acts_as_textiled”, which makes it trivially easy to add Textile markup to our content blocks.

To add the plugin, open a terminal window at the root of your application and enter the following command:

script/plugin install svn://errtheblog.com/svn/plugins/acts_as_textiled

How did we know this obscure address to enter? Just do a google search for the name of a plugin, or go to www.agilewebdevelopment.com/plugins, and you can find most anything pretty quickly.

If you look in the vendor/plugins folder (in NetBeans 6.0.1, you need to use the Files pane, not the Projects pane, to see this folder), you’ll see a folder into which this software has been installed.

Now, to turn our content_block body into textile markup, all you need to do is open the file models/content_block.rb, and add this line:

acts_as_textiled :body

That’s it. This plugin is smart enough to automatically display the Textile markup source when you’re displaying the body in a form field, as we are in the admin pages, but to render it into HTML anywhere else you use the body.

You’ll need to stop and restart the server before the plugin will function, though. Plugins add their own initialization code that is run when Rails starts.

Now browse to //localhost:3000/content_blocks, click the Edit link for the welcome block you created earlier, and edit the body to include some Textile markup. For example, surround a word or phrase with asterisks to make it bold, or with underscores to make it italic. Click Update to save the modified content block.

In the list of content blocks, you’ll notice that the HTML markup is now shown. That’s because the code generated by the scaffold to generate the list uses the <%=h %> wrapper for the attributes it display. The “h” causes HTML elements to be rendered as HTML entities, so the HTML text is displayed rather than interpreted by the browser. This is a security measure to avoid cross-site scripting attacks, but it has the side effect here of showing us the HTML markup that acts_as_textiled, with help from RedCloth, generated from the text in our content block.

Now refresh the home page, and you should see your welcome text rendered with bold, or italic, or whatever you’ve specified in the content block’s Textile markup.

2. Adding a File Uploader

a. Creating the asset scaffold

Now we have the basis for a content management system for text, but what about images or other uploaded files? We need a file uploader.

We can start by scaffolding another model, which we’ll call asset:

script/generate scaffold asset name:string

Go ahead and run the scaffold generator, but don’t run the migration yet, because we’re going to need to add some fields to it.

While we’re thinking of it, go ahead and delete views/layouts/assets.html.erb, the pesky layout file that the scaffold generator created but that whose only effect will be to prevent our usual layout from being used.

b. Adding attachment_fu

Uploading files involves a little complexity but, as usual, there’s a plugin that handles most of it for us: in this case, it’s “attachment_fu”. First, install the plugin with the following command in your terminal:

script/plugin install http://svn.techno-weenie.net/projects/plugins/attachment_fu/

This is a very capable plugin, which will take care of generating thumbnails of uploaded images and lots of other things, but we’re going to use it in a pretty basic way.

You can look at the readme file in the vendor/plugins/attachment_fu folder to see what some of its requirements and capabilities are. From this, we can see that we need to provide a number of fields in our database model that is going to manage the uploaded files.

Open the migration file that the scaffold command generated, db/migrate/002_create_assets.rb, and add the following two lines after the line t.string :name:

  t.integer :parent_id, :size, :width, :height
  t.string :content_type, :filename, :thumbnail

This adds four integer fields and three string fields that are required for attachment_fu to do its work. Now run the migration to create the table, and restart the server so the plugin can run its initialization code.

c. Updating the model

Now we need to tell the asset model that we want it to use attachment_fu. Add the following two lines in the file model/asset.rb:

  has_attachment :storage => :file_system
  validates_as_attachment

The first line tells Rails to use attachment_fu for this model, and to store the uploaded assets in the file system (not in the database). The second line tells it to validate the entered data (such as the filename).

d. Updating the views

That’s all it takes for our asset model to deal with uploaded files. Now we need to add a little code to the view files to allow us to choose the filename. Open the file views/new.html.erb. Add the following code after the name field so we’ll have a place to enter the filename:

    <p><b>Asset File</b><br />
       <%= f.file_field :uploaded_data %>
    </p>

The file_field helper will display a text field for the file name and a button that will bring up a file browser dialog so the user can easily pick a file.

This won’t actually work, however, without one more change. When we submit the form, it needs to use a special HTML format called multipart, so the form POST can provide not only the filename but the actual file contents. Edit the form_for line so that it looks like this:

<% form_for(@asset, :html => { :multipart => true }) do |f| %>

Now repeat these changes in the file views/edit.html.erb, so you’ll be able to edit assets after they have been created.

Finally, we want to see the asset after we’ve uploaded it, so open the file views/show.html.erb, and after the line <%=h @asset.name %>, add this line:

<%= image_tag @asset.public_filename %>

This uses the image_tag helper, which generates a standard HTML img tag. We use the method public_filename, which is provided by attachment_fu to give us the externally accessible filename for the image.

Now you can browse to //localhost:3000/assets, click New asset, enter a name (we’ll use “logo” for our example), click the Browse button to locate an image file, and finally click Create. The file is uploaded, and the database record to associate it with the name (and store other information about the file) is created. The image should then be displayed on the “show” scaffold page.

e. Using the image

Now we’ll add a helper, much like the one we created for the content blocks, to make it easy to display images. Add the following code to the file helpers/assets_helper.rb:

   def show_asset name
      asset = Asset.find_by_name(name)
      if asset
         image_tag asset.public_filename
      else
         "No content for #{name}"
      end
   end

Now, assuming you’ve uploaded an image and named the asset “logo”, you can add the following code to the home page:

<%= show_asset 'logo' %>

And the image should be displayed there.

3. Adding an admin page and log-in

a. Creating an admin page

We’ve been entering the full URL to get to the admin pages for content blocks and assets, and we should have an easier way. Let’s create an admin page.

In the file static_controller.rb, add the empty method for an admin page:

	def admin
	end

And then create a file named admin.html.erb in the views/static folder. (In NetBeans, right-click on the views/static folder, choose New > ERB, and enter the name “admin.html”. NetBeans will add the .erb suffix.)

Open the file views/static/admin.html.erb, and add the following:

	<h1>Site Admin Dashboard</h1>
	<ul>
	  <li><%= link_to 'Content Block Admin', content_blocks_path %></li>
	  <li><%= link_to 'Asset Admin', assets_path %></li>  
	</ul>

Now add a link to this admin page to the layout, which you might put in the navigation bar, or in the footer:

<%= link_to 'Admin', :controller => 'static', :action => 'admin' %>

Now you can browse to the admin page when you want to access the content blocks and assets administration pages.

b. Adding authentication

We now have the heart of a content management system. But we’ve also made it possible for any visitor to modify the content of our site! Clearly we need to require log-in (authentication) before accessing these functions.

As usual, there’s a plugin that takes care of all the dirty work for us. Let’s install restful_authentication:

  script/plugin install http://svn.techno-weenie.net/projects/plugins/restful_authentication/

You can view the readme in vendor/plugins/restful_authentication to learn more about it.

This plugin provides a generator of its own to help us set things up, so let’s run it:

	script/generate authenticated user sessions

In this command, “authenticated” is the name of the generator; “user” is the name of the model that will store user names and passwords; and “sessions” is the name of the controller that will manage user sessions.

This generator creates a migration for us, but we need to run the migration, so go ahead and migrate the database to the current version.

Finally, to make the methods provided by this plugin available in your controllers, you need to add the following line to the file controllers/application.rb:

include AuthenticatedSystem

The application controller is “mixed in” to all other controllers, so any code you put here is available to all controllers. So by putting the “include” statement for the authenticated system here, you’re effectively adding that code to every controller.

c. Adding user management

Before we can have log-in, we need to have users, so let’s add some user management.

The authenticated generator we ran in the previous section gave us a users controller and a view for creating users, so browse to //localhost:3000/users/new and create a login for yourself.

The generator also created a sessions controller, which is what handles the actual log in and log out operations. Logging in is creating a new session, and logging out is destroying a session. It’s convenient to have shortcuts we can use to refer to these, so let’s add some named routes. Add the following lines to the top of the config/routes.rb file:

  map.logout '/logout', :controller => 'sessions', :action => 'destroy'
  map.login '/login', :controller => 'sessions', :action => 'new'

This gives us two things. The text ‘/logout’ and ‘/login’ makes those simple URLs work. And the words after the “map.” provide shortcuts we can refer to anywhere in our code when we want to generate that URL, using either login_path, for example, to generate a relative URL, or login_url, to generate a full, absolute URL.

Let’s add a log-in button to the navigation bar. Open views/layouts/application.html.erb, and add this code to the end of the list of links that creates the nav bar:

    <li><% if logged_in? %><%= link_to "Log Out", logout_path %>
    <% else %><%= link_to "Log In", login_path %><% end %></li>

The restful_authentication plugin provides us with a “logged_in?” method that we can use to determine if any user is logged in. If someone is logged in, then we display a log out link; if not, we display a log in link.

d. Using authentication to protect the admin pages

Now we have our entire authentication system in place, but we haven’t actually used it to require login for any pages. We need to invoke the authentication feature for the pages for which we want to require log-in.

Open the file controllers/static_controller.rb, and add the following line in the admin method:

   login_required

That’s all it takes to protect that method. (“login_required” is another method provided for us by the restful_authentication plugin.) Try accessing //localhost:3000/admin, and you should be presented with a login screen. Go ahead and log in with the credentials you created in the previous step, and you should see the Log In button in the nav bar change to Log Out. Log out now so we can continue exercising the authentication.

Note that while we’ve protected the admin page, you can still browse directly to the content blocks and assets pages, as well as create users, if you know the URLs. Let’s fix that.

Open the file controllers/content_blocks_controller.rb, and add the following line right after the first line that defines the controller:

	before_filter :login_required

This invokes the same method that we added to static_controller/admin, but because we’ve put it in a before_filter, it is executed as part of every method in the controller, protecting all of them.

Make the same change to the assets controller and the users controller.

What’s Next?

We’re done! You now have a working, albeit somewhat primitive, content management system. To make this a more full-featured CMS, some things you might want to add include:

User admin pages, so you can list the users, delete users, and enable users to request new passwords
A page database, so the page title, metatags such as description and keywords, tab to be highlighted, and content blocks and images to be displayed, could be stored in a database.
An image processing library, such as Image Magick, so attachment_fu can generate thumbnails and other image sizes from uploaded images.

Lab 3. The Resources Page, with Ajax

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, we’ll create a database-driven Resources page for our site that presents a list of links. We’ll also illustrate how you can use Ajax to allow the sorting of the list to be changed without reloading the page.

1. Creating the links and categories database

a. Creating the models

We’re going to need two models: one for the links, and one for the categories. As in the previous lab, we’ll use the scaffold command to create the models and the first cut at the admin pages for them.

For the links, we need a name, a URL, and a description, so issue the following command (in a terminal window at the root of your application):

	ruby script/generate scaffold link name:string url:string description:text

The categories model needs only a name, so issue this command:

	ruby script/generate scaffold category name:string

b. Creating the join table

A link can have many categories, and a category can have many links, so we need a many-to-many relationship, which we’ll implement with has_and_belongs_to_many. To do so, we need a join table, which records the relationships between the two models.

Rails does not provide a generator for join tables, so we need to create a migration and then enter into the migration file the specifications for the join table. In NetBeans, you can right-click on the project name, choose Generate, and select Migration from the pop-up menu. Then enter the name for the migration; it doesn’t matter what this is, but we’d suggest something like add_link_category_join. (If you want to do this from the terminal, enter the command ruby script/generate migration add_link_category_join.)

Now open the migrations folder (Database Migrations in the NetBeans projects view, or db/migrate in the file system), and edit the empty migration file that the generator created. It should be named something like 006_add_link_category_join.rb. (If you’re using NetBeans, this file will be automatically opened for you.)

In the self.up method, insert the following code:

   create_table :categories_links, :id => false do |t|
      t.integer :link_id
      t.integer :category_id
   end

The first line of this migration code tells Rails the name of the table you want to create, and that you don’t want it to automatically generate an ID field. Join tables don’t have an ID of their own, just the two foreign IDs of the tables they are linking together. The do |t| begins the code block that defines the table, and gives the arbitrary name t to the object that is used to create the table (the internals of this are a little complex, and you can just think of this as a magic incantation that you can use in every migration).

The names of the table and the fields are not arbitrary, but are required for Rails to be able to make all the connections. This is an example of convention over configuration. The table name is the two model names, joined by an underscore, in alphabetical order. The field names are the model names with _id appended.

Now add this line to the self.down method:

	drop_table :categories_links

This allows you to migrate back down if desired.

Now run all three migrations. (In NetBeans, right-click on the project name, choose Migrate Database > To Current Version, or in the terminal enter rake db:migrate).

c. Setting up the association

To make the association work, we need to tell Rails how we want these two models associated. To do so, add this line to models/link.rb:

	has_and_belongs_to_many :categories

and this line to models/category.rb:

	has_and_belongs_to_many :links

That’s all you need to do for Rails to know to use the join table and associate the two models. Whenever you save a Link object for which one or more categories has been specified, the appropriate entries will automatically be written to the join table.

2. Fleshing out the link admin

a. Adding a category picker to the link views

The scaffolded views that Rails produces for us don’t deal with associations between models; they just work with one model at a time. This is fine for the category admin pages, which allow us to create and edit category names. But for the links admin pages, we need to add the ability to choose categories to associate with each link.

We want to add a select element to the new link form that lists all the categories. Here’s the line of code that generates the desired select, which you should add to the file views/links/new.html.erb:

   <p><b>Category</b><br />
   <%= f.collection_select :category_ids, @categories, :id, :name, {}, {:name => 'link[category_ids][]', :multiple => true} %>

This is perhaps the most obscure bit of code we’ve seen yet, and it is something that is not as straightforward in Rails as it should be. The collection_select helper creates a select field, using an array of objects as the source for the select list. It’s a powerful helper, but the parameters are a little obtuse.

The first parameter, “:category_ids”, specifies the field that we want to set. In this case, since what we want to set is actually the id in a join table, the peculiar syntax of “category_ids” instead of “categories” is required. The second parameter, “@categories”, is the name of the array that has the objects used to populate the select list. We’ll set this in the controller. The next two fields specify the attributes of the objects in this array to be used for the value returned by the select list, and for the text to be displayed in the select list. The empty hash is a placeholder for an options parameter that we’re not using.

The final parameter is the HTML options hash. First, we set the name that will be used when posting the form data. This obscure syntax is what is required for the Rails “save” operation (“@link.save” in the controller) to properly update the join table. This is a peculiarity of using a has_and_belongs_to_many relationship with a select list. The final item is setting the standard HTML option to allow multiple items in the select list to be chosen, so a link can be assigned to multiple categories.

How would you figure all this out? It’s tricky. We couldn’t figure it out from the documentation. But some google searching on terms such as “habtm collection_select” found some helpful notes. Sometimes that’s what it takes. You can also ask questions about such things in the rubyonrails-talk Google group.

Add the category selector to the edit view as well, so you can edit existing categories.

Note: in the code on the USB Flash drive, there is an error in this code in the “new” view, but it is correct in the “edit” view.

If you try to use these views now, you’ll get an error, because we haven’t yet set the instance variable “@categories”.

b. Providing the list of categories in the links controller

In the links controller, we need to prepare the list of categories to be used in the select list. Just add this line of code to the “new” method in /controllers/links_controller.rb:

   @categories = Category.find(:all, :order => :name)

This statement finds all the categories, sorts them by name, and puts them (as an array) in the instance variable “@categories”. The select list in the view then draws from this instance variable to populate the select.

Since we use the same instance variable in the edit view, you need to add this line of code to the edit action in the links_controller as well. And there’s two more places where we need it too: at the start of the “else” clause in the create action and the update action. That’s because, if there’s a validation failure when creating or editing the link, the form will be displayed again, and it will need the instance variable to populate the select list.

c. Adding links on the admin pages

To provide easy access to our new link and category admin, let’s add a couple links to our admin page. Add to the list in /views/static/admin.html.erb:

   <li><%= link_to 'Links Admin', links_path %></li>
   <li><%= link_to 'Categories Admin', categories_path %></li>

To test all this, create a few categories, and then create a few links. You should see the list of categories in the select list.

d. Protecting the link and category admin pages

One last step for the link and category admin: we don’t want site visitors to access these admin pages, so just as we did for the other administrative controllers, add this line at the top of the links and categories controllers:

   before_filter :login_required

3. Creating the Resources page from the database

a. Adding the resources_page view

Now our admin features for links are in place, and we need to generate a page that displays them. We’d like to add this to our links controller and views, to keep the link-related functions together. But the view that shows a list of links, “index”, is an admin view. So we need to add a new view for the public part of the site, which we’ll call “resources_page”.

Create a new file in the views/links folder, called “resources_page.html.erb” (remember that NetBeans adds the “.erb” for you when you create a new erb file). The simplest thing we can do is to just list out all the links, assuming we have an instance variable that the controller has prepared for us:

   <h1>Resources</h1>
   <ul>
      <% @resources.each do |resource| %>
         <li><%= link_to resource.name, resource.url %><br />
             <%= resource.description %></li>
      <% end %>
   </ul>

This code uses the “each” iterator to step through each of the items in the @links array, and then uses the “link_to” helper to create a link to each one. We use the name attribute as the link text, and the url attribute as the actual link, and then show the description on the next line. (Note that this code assumes that your links, as entered into the database, include “http://”.)

(An aside: We named the instance variable “@resource” so it didn’t get confusing with too many things called “link”, but you could just as well use “link” everywhere we’ve used “resource”, and in retrospect this might have been better. It is a Link object we’re accessing.)

Now, since we want the resources page to be visible to anyone, even though the other actions, in the resources page are for admin use only, modify the before_filter we added previously to the links_controller as follows:

   before_filter :login_required, :except => 'resources_page'

b. Updating the links controller

Our newly-created resources_page view expects an instance variable called @resources with an array of Link objects, so we need to create that in the controller. We’ve added a new view, so we need to add a new method to the controller. Open the file controllers/links_controller.rb, and add the following method (it can go anywhere, but we’d put it at the end).

   def resources_page
      @page_title = "Resources"
      @current_tab = 'resources_tab'
      @resources = Link.find(:all)
   end

In addition to setting the @resources instance variable, we’ve set the variables that control the navigation tab highlighting and the page title.

c. Enabling the non-RESTful route

We need to do one more thing to make this all work. We’ve added a non-RESTful action (that is, not one of the core group of standard REST actions) to the links controller. The routing for this controller is set with the statement map.resource :links in the routes.rb file, and this only supports the standard routes. So we need to tell the routing that we’ve added another action. Modify that line in config/routes.rb so that it reads:

	map.resources :links, :collection => {:resources_page => :get}

This tells the routing system that we’ve added another action, which acts on the collection of links (rather than on a single link), that the action is called “resources_page”, and that the HTTP method used to access it is “get”. (Alas, our use of the instance variable name “@resources” might cause a little more confusion here; that use of the word resources has nothing to do with “map.resources”. In that context, resources is referring to the standard set of routes for RESTful resources.)

With this change, you should now be able to browse to //localhost:3000/links/resources_page and see the list of links that you entered earlier.

d. Cleaning out the old static page

Our navigation button labeled “Resources” still points to the static page we created in Lab 1, so let’s change that.

Open views/layouts/application.html.erb, and find the “link_to_” statement that creates the “Resources” navigation button. Change that line to read:

	<li><%= link_to 'Resources', {:controller => 'links', :action => 'resources_page'}, :id => 'resources_nav' %></li>

Finally, delete the resources action from the static controller, since it is no longer used. Now you should be able to click on the Resources navigation button and see the dynamically generated list of links, complete with descriptions (assuming you entered some).

(A debugging note: if you find that your resources links don’t work when you click on them, check that you have prefaced them with http:// when you entered them in the database. Otherwise, they will be viewed as relative URLs on your site, not as external sites.)

3. Sorting the list by category

So far, we haven’t done anything with the categories we have assigned to the links. Let’s sort the links by category, and display them under a heading for each category.

Replace the code in views/links/resources_page.html.erb with the following:

   <% @categories.each do |category| %>
      <h2><%= category.name %></h2>
      <ul>
         <% category.links.each do |resource| %>
            <li><%= link_to resource.name, resource.url %><br />
                <%= resource.description %></li>
         <% end %>
      </ul>
   <% end %>

We’ve added an outer loop that iterates through the categories. For each category, we display the category name, and then we iterate through each of the links, using “category.links” to find the links associated with the category. This method is added to the category object because we included the has_and_belongs_to_many :links statement in the category model.

This code needs the list of categories, so add this line to the resources_page action in the links controller:

   @categories = Category.find(:all, :order => :name)

Now refresh the resources page, and you’ll see the links listed by category. If a link is assigned to more than one category, it will appear in more than one place.

3. Adding a touch of Ajax

Suppose you wanted to give the viewer the option of dividing the list by category or not. We could create two different resources pages with the two different views. Or we could pass a query string parameter to the view that would choose one option or the other. But we’ll take a more sophisticated approach, which may be overkill in this case but allows us to illustrate how Ajax works: updating how the list is displayed, based on a check box, without reloading the entire page.

a. Adding a sorting option to the view

As a first step, let’s put in both forms of our view code, with a parameter choosing which is displayed. The new view (except for the H1 heading at the top) looks like this:

   <% if params[:divide_by_category] == '1' %>
      <% @categories.each do |category| %>
         <h2><%= category.name %></h2>
         <ul>
             <% category.links.each do |resource| %>
               <li><%= link_to resource.name, resource.url %><br />
                   <%= resource.description %></li>
	         <% end %>
         </ul>
      <% end %>
   <% else %>
      <ul>
         <% @resources.each do |resource| %>
            <li><%= link_to resource.name, resource.url %><br />
                <%= resource.description %></li>
	     <% end %>
	  </ul>
   <% end %>

This is simply our two versions of the view code, with an if statement choosing which version is used.

Now when you browse to the Resources page, you’ll see the list as we first rendered it, without a division into categories. But if you append the parameter to the URL, so the complete URL is now:

	//localhost:3000/links/resources_page?divide_by_category=1

Then you’ll see the page divided into categories. We haven’t done any Ajax here — we’re just passing a query string parameter (the thing after the question mark in the URL) to the view. Such parameters are provided in the params hash, as you can see in the first line of our updated view.

b. Moving the list into a partial

As a first step toward turning this into an Ajax function, let’s move the generation of the list into a partial. Cut all the text that you just put into the view, so there’s nothing there but the H1 heading. Create a new file in views/links, called “_list.html.erb”. The leading underscore is what identifies this file as a partial, rather than a full view. Now paste the text you just deleted from the links/resource_page view into the new _list view (or copy it from the listing above).

To reference the partial from the resources_page view, add this line after the H1 heading:

	<%= render :partial => 'list' %>

Note that you don’t include the underscore, or the suffixes, when you reference a partial.

Make sure the modified view files are saved, and then refresh the Resources page. It should behave just as before. We haven’t changed the behavior yet, we’ve just moved the rendering of the list into a partial, for reasons that will become apparent shortly.

c. Updating the view for Ajax

To turn this into an Ajax view, we need to do three things:

Add a mini-form with a check box to select the view
Wrap the partial in a div so we can apply an ID and thus be able to modify that div using JavaScript
Add an observer to monitor the checkbox, request an updated list from the server, and generate a new list

Here’s full code for the new view:

   <% form_tag '#' do %>
      <%= check_box_tag 'divide_by_category' %><label>Divide by Category</label>
   <% end %>

   <div id='list'>
      <%= render :partial => 'list' %>
   </div>

   <%= observe_field 'divide_by_category',
       :on => 'click',
       :url => {:controller => 'links', :action => 'get_list' },
       :with => 'divide_by_category',
       :update => 'list' %>

We’ve created a form that has no action associated with it, because the form will never be submitted (it has no submit button). Inside this form, we have just the check box.

Then we have the same partial we used in the previous step, just wrapped in a div so we can replace it using JavaScript.

And finally, the observe_field helper provides all the Ajax magic. This is a very powerful helper, and it can take a lot of parameters. Here’s the explanation for each of the parameters we’ve used:

```
‘divide_by_category’
```
is the ID of the field that we want to observe
```
:on => ‘click’ 
```
specifies that it is the click event on the check box that we want to have trigger the Ajax request.
```
:url => {:controller => ‘links’, :action => ‘get_list’ }
```
specifies the controller and action to which the request should be sent when the field changes
```
:with => ‘divide_by_category’
```
indicates that we want the value of this field sent to the controller when the Ajax request is sent
```
:update => ‘list’
```
indicates that the div with the ID ‘list’ should be updated with the Ajax response

This observe_field helper is our first bit of JavaScript, using the Prototype framework, and we haven’t yet included that code in our HTML. So open up layouts/application.html.erb, and add the following line in the head section:

   <%= javascript_include_tag :defaults %>

This helper creates HTML script tags to include all the standard Rails JavaScript files.

You can try the new page, but the check box won’t do anything. Why? We haven’t yet provided the get_list method that the Ajax action is invoking. If you are using Firefox and have Firebug installed, open the Firebug window and click the Console tab with the Resources page displayed, and then click the check box. You’ll see that a POST action occurred (that’s that XML HTTP request generated by the observe_field helper) but the result was a 404 error, because there is no such action.

d. Adding the Ajax controller action

Now that we have everything in place, we just need an action that will return the list.

Add the following method to the end of the links controller:

   def get_list
      @resources = Link.find(:all)
      @categories = Category.find(:all)
      render :partial => 'list', :layout => false
   end

This is the action that will be invoked by the Ajax request. It sets up the two instance variables required by the list partial, and then renders that partial. We need to add the option :layout => false because we want just the bare results from this view, not wrapped in any layout file, since we aren’t using it to render a page; we’re just returning the HTML as the result of the Ajax request, and it will be used to modify the contents of the list div.

Now try the check box in the Resources page again. It should control whether or not the view is sorted by category or not. Note that the URL does not change, and there is no page refresh. When the get_list action returns the HTML for the list to the JavaScript code generated by the observer_field helper, the JavaScript code replaces the contents of the “list” div with this new HTML. You can use the Firebug console to see the Ajax request and response each time you click on the check box.

Next Steps

You’ve now created a database to manage a set of links, sorted by category, and a page that allows them to be displayed with or without category sorting using Ajax.

Here’s some enhancements you might consider (and which we’ll put into our final version of the sample application):

Don’t display the category name if there are no links in that category
Validate that there is at least a name and a URL for each link
Validate that the category name is not blank
Validate the format of the URL
Make the simple URL “/resources” work for the new resources page

Lab 4. Contact Form, Mailer, and an Anti-Spam Web Service

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, we’re going to create a contact form that visitors to our web site can complete. When they submit the contact form, we’ll both store it in the database and generate an email to ourselves.

1. Creating the contact form

a. Scaffolding the contact model

If we only wanted to send an email when the contact form was filled in, we wouldn’t really need to use an Active Record model and save it to the database. But it’s actually easier to use all the scaffolding and other support that Active Record provides, and it is handy to have the messages stored in the database in case any of them are lost in the email maze.

So let’s start by creating a scaffold for contacts:

	ruby script/generate scaffold contact name:string email:string message:text subject:string

As in the previous labs, this will generate the model, controller, views, and test files.

Now run the migration, as in previous labs, by migrating the database to the current version (rake db:migrate if you’re working from the command line.)

Let’s add some simple validations, since we don’t want any contact messages that don’t have all the fields filled in. Add this line to the file models/contact.rb:

   validates_presence_of :name, :email, :subject, :message

You’d probably want more sophisticated validations in the real world, but this is a start.

b. Hooking up the contact form

First delete the annoying layout file the scaffold generates (views/layouts/contact.html.erb), so the scaffolded views will use our standard layout.

We already have a contact button in the navigation bar, but it is pointing to one of our initial static pages, and now we want it to point to the new contact form. So find the link to “Contact” in views/layouts/application.html.erb, and change it to:

	<li><%= link_to 'Contact', {:controller => 'contacts', :action => 'new'}, :id => 'contact_nav' %></li>

Or, to use the path shortcut that map.resources provides:

	<li><%= link_to 'Contact', new_contact_path, :id => 'contact_nav' %></li>

These both do the same thing. The first is more explicit, the second is the “modern” Rails way of doing things.

Now you should be able to click on the Contact button in the nav bar and see the new contact form. Go ahead and create a couple of messages so we can see them in a minute in the admin interface.

Since we’ve hijacked a form that was meant to be part of the admin interface and are using it for a user-facing form, we need to change some of the linkages. Delete this line from the end of views/contacts/new.html.erb, since we don’t want visitors to try to get to the list of all messages:

	<%= link_to 'Back', contacts_path %>

Finally, when a contact form is successfully saved, we don’t want to redirect to the normal “show” action, as the scaffolded pages are set up to do. Instead, let’s redirect to the home page. Open the file controllers/contacts_controller.rb, and find this section of code in the create action:

   if @contact.save
      flash[:notice] = 'Contact was successfully created.'
      format.html { redirect_to(@contact) }

Let’s provide a more appropriate flash message, and change the redirect, so the code now reads as follows:

   if @contact.save
      flash[:notice] = 'Thanks for your message. It has been delivered.'
      format.html { redirect_to :controller => 'static', :action => 'home' }

c. Setting up the contact admin

We don’t want site visitors being able to view the list of contact messages, even if they guess the URL for that page, so we need to add authentication to the contacts controller. But we do need the “new” action in this controller to be accessible to visitors, as well as the “create” action that is invoked by the form when it is submitted. So add the following line to the start of controllers/contacts_controller.rb:

   before_filter :login_required, :except => [:new, :create]

We also want to be able to easily access the list of contacts from our admin page, so add this line to the file views/static/admin.html.erb:

   <li><%= link_to 'Contacts Admin', contacts_path %></li>

Now you can click the Admin button to access the admin page (you’ll have to log in), and then click the Contacts Admin link to see the list of messages that have been submitted.

2. Sending the messages via email

Now we need to create the linkage to the mail system, so new contact messages will be sent to us via email as well as added to the database.

a. Generating the mailer

We’ve used the scaffold and migration generators in previous labs. There’s another generator for mailers, which is what we’re going to use here. Enter the following command in your terminal, at the root of your app:

   ruby script/generate mailer contact_mailer contact_form

(This generator actually works from within NetBeans, so you can use its GUI version of you prefer.)

This creates a mailer model, called contact_mailer and its associated view, which we’ve called contact_form. You can enter multiple views here, for example if you had different kinds of contact forms and wanted to format each message differently.

b. Configuring the mailer model

Once you’ve run the generator, you’ll find a file contact_mailer.rb in your models folder. Open that file, and this is what you should find there:

   def contact_form(sent_at = Time.now)
      @subject    = 'ContactMailer#contact_form'
      @body       = {}
      @recipients = ''
      @from       = ''
      @sent_on    = sent_at
      @headers    = {}
   end

Note that this method is in a class that inherits from ActionMailer::Base, rather than ActiveRecord::Base like all your other models, so it has a different set of characteristics and capabilities.

We need to fill in some of the blanks in this method. First, we need to pass to this method the parameters from our form, so modify the first line to read:

	def contact_form(name, email, subject, message, sent_at = Time.now)

Note that the one parameter the default method has is sent_at = Time.now. When a parameter has an assignment to a value like this, it makes it optional to actually pass that parameter. So this is set up so we can pass a sent_at value if we want, but if we don’t, the current time will be used (which is just fine in most cases).

Now we need to use these parameters appropriately in the body of the method. First, the subject:

   @subject = subject

Simple enough, we’re just taking the subject from the variable that we passed in as a parameter, and assigning it to the instance variable that will be passed to the view. Mailer views are special, so this subject text will automatically become the actual subject line of the email message.

Now for the body, which is a little more obscure. This line should read:

    @body = {:name => name, :message => message}

Mailers are odd, and this is one of the odd bits of syntax. The @body instance variable is set to a hash, and each key in the hash is turned into an instance variable of that name. So we’re creating two instance variables, for use in the body of the message. You might think that we should be able to just add lines of text to explicitly set the name and message instance variables, just as you would in a controller, but remember this is a mailer model, not a controller. The only variables that will be available to our mailer view for explicit use are the ones we define in this body hash. All the other variables are used to create the message headers.

Now we need to set the recipient:

    @recipients = CONTACT_RECIPIENT

We’ve chosen here to set it to a constant, rather than a literal value, because we don’t want a specific email address, which may change in time, deep in our code files. Add this line in your config/environments/development.rb file:

   CONTACT_RECIPIENT = 'yourname@yourdomain.com'

Another benefit of this approach is that you can set a different address in your production.rb environment file. For example, in development, you’ll probably want contact messages to go to the developer, but in production, they should go to an administrative or sales person.

Note that if you want messages to go to more than one person, you can provide an array of email addresses:

	CONTACT_RECIPIENT = ['email1@domain.com', 'email2@domain.com', 'email3@domain.com']

And all of these recipients will be included on the “to” line. You can also set “cc” recipients by adding “@cc = COPY_RECIPIENT” in the model.

Finally, we need to set the “from” address. That’s been passed in as a parameter from our form, so this line should simply read:

	@from = email

That’s it for the mailer model. All it is really doing it taking parameters that we’ll pass to it from the contact form (when we get to modifying the controller), puts those parameters into the right places for the mailer view, and sets the recipient address. Now on to the view.

c. Creating the mailer view

You’ll find an empty (well, with a couple lines of useless boilerplate) view in views/contact_mailer/contact_form.erb. This is the view that will get invoked when we use the mailer model to request delivery of a message.

The instance variables passed to this view are those we defined in the “@body” variable in the model. Now we just use those however we want, much like in a regular view, but remember that we’re generating a plain-text email here, so there’s no HTML code required. Here’s a simple view:

   Email from your web site

   From: <%= @name %>

   Message: <%= @message %>

c. Configuring how mail is sent

We’re almost ready to generate email, but first we need to tell Rails how it supposed to access the mail system. You can configure it to use an SMTP server, or you can tell it to use sendmail, which is the more common approach if you’re on a unix-type system. The details can be a little tricky, but you should be able to get help from your hosting company or system administrator. Here’s the configuration that works on the Joyent server that you have:

   ActionMailer::Base.delivery_method = :sendmail
   ActionMailer::Base.sendmail_settings = {
      :location  => '/opt/csw/sbin/sendmail',
      :arguments     => '-t'
   }
   ActionMailer::Base.default_charset = 'utf-8'

We’re providing two parameters to make sendmail work. The first is the path to the executable file, which is specific to the way the Joyent servers are set up. The second, which you’d probably want for any system configuration, is the -t parameter, which tells sendmail to extract the recipient addresses from the message headers.

In the sample app, we put this in a new file, called mail.rb, which we put in config/initializers. You might want to to put it in environments/production.rb instead, so you could then put a different configuration in environments/development.rb. For example, here’s the configuration for sending email via SMTP over a Comcast connection:

	ActionMailer::Base.delivery_method = :smtp
	ActionMailer::Base.smtp_settings = {
	  :address => 'smtp.comcast.net',
	  :port => 25,
	  :domain => 'comcast.net'
	}
	ActionMailer::Base.default_charset = 'utf-8'

SMTP servers are picky about what mail they will accept, so you’ll have to be on a Comcast internet connection for the above settings to work. If your SMTP server requires authentication, you may need to add login and password settings to the smtp_settings hash.

If you’re having trouble getting your mail to go out in development mode, you may want to change this line in your development environment:

	config.action_mailer.raise_delivery_errors = false

And set it to true instead. If you don’t do that, deliver failures are silently ignored in development mode.

d. Generating the email from the contact form

With all this setup behind us, actually delivering the mail is easy. Open the file controllers/contacts_controller.rb, and add this line immediately after “if @contact.save” (we only want to deliver the mail if the save was successful, indicating that the validations passed):

   ContactMailer.deliver_contact_form(@contact.name, @contact.email, @contact.subject, @contact.message)

We’re invoking the contact_form method in the ContactMailer class. This is the class we edited at the start of this lab. We pass to that method the data from the form; the order of these parameters is determined by the order in which we listed them in the method definition.

There’s just one strange thing here: the method we’re invoking is “deliver_contact_form”, not “contact_form”. Rails creates this method name for us, and we have to use it. Just one of those oddities to get used to. (This odd syntax exists to support another option: you can call the create_contact_form method, which will produce the mail object, ready to be sent, but won’t actually send it.)

You can now try creating a contact message, and it should be sent. Depending on what system you’re running on, and how your mail delivery is configured, it may not actually go out, but you can look at the Rails log, and it will show the email that was generated, even if it couldn’t contact a mail system to actually send it.

3. Blocking spam

As an example of integrating a third-party web service, we’ll use the Defensio service to check our contact messages to see if they are spam.

a. Installing the plugin

As with most web services, there’s a Rails plugin available that simplifies integrating the Defensio service. Install it as follows:

   ruby script/plugin install http://code.macournoyer.com/svn/plugins/defensio/

b. Setting up the contact model to use Defensio

We need to tell our comment model to use the Defensio plugin. The basic invocation, which you should paste into the file models/contact.rb, is:

   acts_as_defensio_comment :fields => {:content => :message,
                                        :author => :name,
                                        :author_email => :email,
                                        :title => :subject
                                       }

This tells Defensio that the field with the content we want to check is the message, and also points to the other relevant fields.

Because it is designed for use with a blog, there’s a little trick we need to use to make Defensio work in our situation, in which there is no parent article to a message. Add the following code to the contact.rb. model:

   class StubArticle
      attr_accessor :created_at

      def initialize(date = Time.now)
         @created_at = date
      end

      # stubbing acts_as_defensio_article methods 
      def self.defensio_fields(field)
         field
      end
   end

   def article
      StubArticle.new(self.created_at)
   end

What we’ve done here is to create a phony article, with no contents, that will act as the parent article that the Defensio plugin expects to find. We created a class, StubArticle, to provide the behaviors expected of an Article object, and then we defined an article method to create an instance of that class.

c. Using Defensio in the contacts controller

Now that this is in place, all we need to do to check for spam is to modify the section of code following if @contact.save in the create method of the contacts_controller to test the spam? method of the contact object, which is provided by the Defensio plugin, to see if the message is believed to be spam:

   if @contact.save
      if @contact.spam?
         flash[:notice] = 'Your message is Under Review.'
      else
         ContactMailer.deliver_contact_form(@contact.name, @contact.email, @contact.subject, @contact.message)
         flash[:notice] = 'Thanks for your message. It has been delivered.'      
      end
      format.html { redirect_to :controller => 'static', :action => 'home' }
   else ...

If the save is successful, the we test to see if the message has been rated as spam. If so, we don’t generate an email, and we display an appropriate message. If not, we proceed as before.

d. Setting up the Defensio API key

To use the Defensio service, you need to sign up for an API key. For testing, you can use ours. If you’re going to use the service, sign up for your own key at http://defensio.com//, as ours will be deactivated in a month or two.

The Defensio plugin looks for a file called defensio.yml in the config folder. The plugin installs a template version of this file, but the API keys in that version won’t work. So open that file and paste the following information into it, overwriting what’s there:

   development:
      api_key: 47dcc80bc580d10b3c2b3eb4ed2841b1
      owner_url: http://lab.railsquickstart.com/

   test:
      api_key: 47dcc80bc580d10b3c2b3eb4ed2841b1
      owner_url: http://lab.railsquickstart.com/

   production:
      api_key: 47dcc80bc580d10b3c2b3eb4ed2841b1
      owner_url: http://lab.railsquickstart.com/

e. Adding the Defensio columns to the contact model

Defensio needs to store a few things in the model for each contact: a boolean value that indicates whether or not a message is spam, a “spaminess” rating that gives the estimated likelihood that a comment is spam, and a “signature” field that Defensio uses to identify the message on future calls to the service.

To add these fields to your content model, create a new migration, called something like AddDefensioFields, and put the following in the self.up block in the new migration file:

   add_column :contacts, :spam,      :boolean, :default => false
   add_column :contacts, :spaminess, :float
   add_column :contacts, :signature, :string

And to keep things tidy, add the following to the self.down block in the migration file:

   remove_column :contacts, :spam
   remove_column :contacts, :spaminess
   remove_column :contacts, :signature

Now run the migration, and restart your server so the plugin can initialize itself. You should now be able to submit contact messages just as before. You may not notice anything different yet, but when the contact message is saved, it is also sent to the Defensio service, and the service responds with the spam flag and the spamminess value, which are stored in the database. All this happens automatically because of the acts_as_defensio_comment statement that we added to the contact model.

If you get any error messages at this point, check all the Defensio-related code.

f. Adding Defensio support to the contact admin pages

The last piece of the puzzle is displaying the values created by Defensio in the contact admin list, and providing a couple links to send information to Defensio.

Open the file views/contacts/index.html.erb, and add these two lines at the end of the list of headers (the last of which is “Subject” until you make this addition):

	<th>Spam?</th>
    <th>Spam Score</th>

Now we need to add the columns that display this information. In the block that is displaying the information for each contact, add these two lines after the one that displays contact.subject:

	<td><%=h contact.spam? ? "TRUE" : "FALSE" %></td>
    <td><%=h contact.spaminess %></td>

The first of these displays TRUE or FALSE depending on the state of the spam attribute. The second displays the “spaminess” value.

Finally, all the follow lines just before the </tr> element, after the link to the Destroy action:

	<td><%= link_to 'Report Spam', {:action => :report_as_spam, :id => contact} %></td>
    <td><%= link_to 'Report Ham', {:action => :report_as_ham, :id => contact} %></td>

The first link allows us to flag a message as spam; the second allows us to flag it as “ham” (i.e., not spam).

g. Adding the actions

Just one more step. We need to add the actions to the contacts controller to deal with reporting of messages as spam or ham from the admin page we just modified. Add these two methods to the end of controllers/contacts_controller.rb:

   def report_as_spam
      @contact = Contact.find(params[:id])
      @contact.report_as_spam if @contact
      redirect_to(contacts_url) 
   end

   def report_as_ham
      @contact = Contact.find(params[:id])
      @contact.report_as_ham if @contact
      redirect_to(contacts_url)
   end

These are both very simple methods, since all they need to do is invoke the corresponding method on the contact object. Those methods are provided by the Defensio plugin.

h. Testing it out

You’re done! Go ahead and create a few contact messages. The go to the Contacts admin page, and you’ll see how the messages were rates in terms of their spaminess. Click the Report Spam link for one of them, and you’ll see its spam state change to TRUE, and its spaminess score go to 1.0. Click the Report Ham link for another, and its spaminess score will go to zero.

After some training, Defensio will begin recognizing messages as spam and marking them as such automatically when they are saved, which prevents them from being emailed to you.

Next Steps

As you extend this code, there lots of things you can do to:

Improve the contact form validations. You might require a minimum length for the name, subject, and message, and validate the email address with a regular expression that checks its format.
The HTML/CSS layout of the contact form could use a lot of help.
If Defensio marks a message as spam, and you later mark it as Ham in the admin interface, you might want to send it as an email, since the initial sending of the email was inhibited when it was first marked as spam.
If the spaminess value is high enough, you might want to not even store the message in your database, but simply discard it.

Lab 5. Subscriptions and Credit Card Processing

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, we’re going to enable visitors to our web site to subscribe and send us real money! Well, not quite real money, since this is using a test account at the credit card gateway, but all it would take is a real merchant account to make the money real.

1. Installing the Active Merchant plug-in

Because we’re going to need some of its features, even before we get to the gateway interface itself, let’s start by installing the Active Merchant plug-in. This is a mature and widely-used plug-in that supports more then 30 credit card gateways.

The NetBeans plug-in interface seems to have trouble installing this plug-in, so let’s do it from the command line:

   ruby script/plugin install http://activemerchant.googlecode.com/svn/trunk/active_merchant

This is a big plugin with lots of pieces, so the installation may take a minute or two.

2. The subscription and customer models and forms

First, we need a form for the subscriber to complete. This form needs to include information about the subscription, which we’ll store in the subscription model, about the customer, which we’ll store in the customer model, and about the credit card, for which we’ll use a non-database model provided by the Active Merchant plug-in.

a. Creating the customer and subscription scaffolds

As in previous labs, we’ll use the scaffold generator to get a quick start on our models, views, and controllers.

We’ll create a customer model, which has information about the customer but not about their subscription, and a subscription model, which has only the information about the subscription and a foreign key to point to the customer.

Run both of these scaffold commands:

	ruby script/generate scaffold customer first_name:string last_name:string address:string city:string state:string zip:string phone:string
	ruby script/generate scaffold subscription customer_id:integer period:integer amount:integer reference:string message:string params:text test:boolean

The customer model should be self-explanatory. Here’s the description of the fields in the subscription model:

customer_id: foreign key for the associated customer
period: length of the subscription, in months
amount: cost of the subscription, in cents
reference: an identifier for the transaction, provided by the gateway
message: the message provided by the gateway, describing the success or failure of the transaction
params: a holding tank for various parameters that the gateway may return
test: a flag indicating that the gateway was in test mode when the transaction was completed

Now migrate the database to the current version. And delete the pesky layouts that we have no use for, views/layouts/customers.html.erb and views/layouts/subscriptions.html.erb.

b. Creating the subscription form

Now you can take a look at the automatically generated forms:

New customer form: //localhost:3000/customers/new
New subscription form: //localhost:3000/subscriptions/new

The new customer form has most of the information we want for entering a new subscription. The new subscription form is rather worthless, however, since this information mostly comes from the gateway, or from our application logic.

Nevertheless, the form we want is most naturally thought of as a new subscription form, so we’re going to use that one, eviscerate most of the scaffolded content, copy the fields from the customer form, and add a few more details.

Open the file view/subscriptions/new, and delete everything between the form_for statement and the <p> tag that precedes the submit button helper, so the entire file now looks like this:

   New subscription

<%= error_messages_for :subscription %>
<% form_for(@subscription) do |f| %>

<%= f.submit “Create” %>

<% end %>

While we have this nice simple view, change the header text (between the h1 tags) to “Subscribe to Our Site”, and change the submit button label from “Create” to “Subscribe”.

Now we’re going to insert the customer fields, much like they exist in the customer form, in the new subscription form. You can do this with some judicious copy-and-paste and then some editing, but it’s a lot of steps to write out (and to read), so we’ll just show the final result, which you can copy and paste from here:

	<h1>Subscribe to Our Site</h1>

	<%= error_messages_for :subscription %>
	<%= error_messages_for :customer %>

	<% form_for(@subscription) do |f| %>

	<% fields_for(@customer) do |customer| %>
	  <p>
	    <b>First name</b><br />
	    <%= customer.text_field :first_name %>
	  </p>

	  <p>
	    <b>Last name</b><br />
	    <%= customer.text_field :last_name %>
	  </p>

	  <p>
	    <b>Address</b><br />
	    <%= customer.text_field :address %>
	  </p>

	  <p>
	    <b>City</b><br />
	    <%= customer.text_field :city %>
	  </p>

	  <p>
	    <b>State</b><br />
	    <%= customer.text_field :state %>
	  </p>

	  <p>
	    <b>Zip</b><br />
	    <%= customer.text_field :zip %>
	  </p>

	  <p>
	    <b>Phone</b><br />
	    <%= customer.text_field :phone %>
	  </p>

	<% end %>

	  <p>
	    <%= f.submit "Subscribe" %>
	  </p>
	<% end %>

Note that we’ve changed the form_for helper for the customer fields to fields_for. This is necessary because you can’t nest forms, and Rails provides a helper for just this situation. We also changed the block variable in the fields_for helper, f by default, to customer, so we can associate the customer fields with the customer model, and subscription fields with the subscription model.

c. Adding in the subscription period

So far, we have no fields for our subscription model in our new subscription form. The one thing we need the user to choose is the subscription period. We could put in a text field for this, as in the scaffolded form, but we probably don’t want arbitrary subscription lengths. So let’s just provide two options, selected via radio buttons. Insert the following text between the form_for and the fields_for statements:

   <p>
      <b>Subscription Period</b><br />
      <%= f.radio_button :period, 1 %><label>One Month: $9.95</label><br />
      <%= f.radio_button :period, 12 %><label>One Year: $99.95</label>
   </p>

Both radio buttons are associated with the attribute :period. Since only one can be selected, that one will determine the value that is provided.

We’ll take care of figuring the price based on the selected period when we get to the controller.

d. Entering the credit card information

The last chunk of information we need from the user is their credit card information. We’ll use a fields_for helper, just as we did for the customer info. We don’t have a model yet for the credit card, and we’ll see when we get to the controller that this model comes from Active Merchant. For our purposes in the form, it acts just like a normal Active Record model.

Insert the following code just before the “Subscribe” submit button (and its surrounding p tag):

   <% fields_for(:creditcard, @creditcard) do |cc| %>

      <p><label>Card Number</label><br />
         <%= cc.text_field :number %></p>

      <p><label>Card Type</label><br />
         <%= cc.select :type, "Visa", "visa"], ["MasterCard", "master"], ["Discover", "discover"], ["American Express", "american_express" (*ERROR - NO SUCH CATEGORY*) %></p>

      <p><label>Expiration</label><br />
         <%= cc.select :month, (1 .. 12) %>
         <%= cc.select :year, (Time.now.year .. 10.years.from_now.year) %></p>

      <p><label>Verification code</label><br />
         <%= cc.text_field :verification_value %></p>

   <% end %>

You may notice that this fields_for helper provides both the name of the object, as a symbol, and the instance variable used to hold it, whereas in our other examples we needed only the instance variable. We need this because this is not an Active Record model, even though it mostly behaves like one; Active Record provides some magic that eliminates the need for both versions, which Active Merchant doesn’t provide.

Here’s an explanation of the fields:

The :number field is straightforward — just a text field for the credit card number.

:card_type is a select field, and we’re providing an array of arrays for the list. The first element of each component array is the text to be displayed, and the second element is the data to be returned if that item is selected.

The :month field is a select, for which we use the Ruby “range” syntax to easily create the list of month numbers, from 1 through 12.

The :year field is similar, and we again use a range, with the start of the range being the current year, and the end of of the range being 10 years later.

The :verification_value field is for the 3- or 4-digit security code, and it is just a text field.

Finally, at the top of the form, we need to report any validation errors on the creditcard model. So add this line just below the other two error_messages_for helpers:

	<%= error_messages_for :creditcard %>

e. Adding the subscription button

Let’s put a subscription button in the navigation bar so we can display our lovely new form easily. Add this code at whatever point you’d like in the navbar list in views/layouts/application.html.erb:

   <li><%= link_to 'Subscribe!', new_subscription_path, :id => 'subscription_nav' %> </li>

To make the tab highlight work, we need to add the code for this new tab to the CSS. Add these lines to public/stylesheets/quickstart.css:

   #subscribe_tab a#subscribe_nav {
      background-color: #333;
   }

Refresh the page, and you’ll see the Subscribe! button. Click on the button, and you’ll see our nice new form. But it won’t yet do much if you submit it, and the tab highlight won’t work yet, because we have work to do in the controller.

f. Styling the validation errors

Rails includes a default style sheet that provides some styling for the validation error messages, as well as highlights for the fields that have errors. We haven’t included that stylesheet in the HTML until now, even though the stylesheet is in the public/stylesheets folder. So just add this line to the head section of views/layouts/application.html.erb:

	<%= stylesheet_link_tag 'scaffold.css' %>

g. Adding links to the admin pages

The scaffolded customer and subscriber admin pages are useful for looking at the data that has been stored, so let’s add links to them to our admin page. Add these lines to the list of links in views/static/admin.hmtl.erb:

   <li><%= link_to 'Customers Admin', customers_path %></li>
   <li><%= link_to 'Subscriptions Admin', subscriptions_path %></li>

3. The subscriptions controller

a. Setting up the instance variables

Let’s get to work on the file controllers/subscriptions_controller.rb.

In the new method, you’ll see that it sets only one instance variable, @subscription. Since we are using three instance variables in the form, we need to set those too. So after the line that sets the @subscription instance variable, add these two lines:

   @customer = Customer.new
   @creditcard = ActiveMerchant::Billing::CreditCard.new

The first line is just like all the others we’ve used in the past. But the second is a little different. We could have set up a creditcard model, and generated a scaffold, but that would be bad, bad, bad! We don’t want to store credit card information in our database. And we don’t want to have to write the routines to validate the credit card information.

Fortunately, Active Merchant provides us with a very nice solution. Active Merchant enables us to create a credit-card object, and to validate that object, without any possibility of it being stored in our database. Since the CreditCard class is part of the Billing module in the ActiveMerchant plugin, we need to create the creditcard object using the syntax shown above.

While we’re at it, add the two lines required for the tab highlighting and the page title:

   @current_tab = "subscribe_tab"
   @page_title = "Subscribe to Our Site"

Now we’re done with the new method.

b. Creating the new objects

The default create method in the subscriptions controller creates a subscription object in the usual manner, since we started from a subscription scaffold. But it doesn’t know anything about customers, or credit cards, no does it know how to figure out the subscription price from the subscription period, not to mention actually sending the credit card transaction to the gateway.

We need to teach it these things. This requires the biggest chunk of Ruby code we’ve written in the entire seminar, but there’s nothing very complicated here. Let’s take it step by step.

We’re not going to use the standard respond_to block that the scaffolding provided, so go ahead and delete all that code now. This should leave you with an empty method, except for the creation of the subscription object.

First, let’s create the customer object, which works in the usual way:

	@customer = Customer.new(params[:customer])

Remember the fields_for helper that we used to wrap the customer fields? That helper takes care of putting the customer fields into a hash within the params hash, named :customer, so we can easily pull those out and set up the customer object in the usual way.

Now we need to figure the subscription cost. The translation from subscription period to subscription cost is business logic, so it belongs in the model, not in the controller. So we’ll just assume that the model will provide us with a method to take care of this:

   @subscription.amount = Subscription.cost_for_period(@subscription.period)

We just need to remember to create this cost_for_period method in the subscription model.

Now let’s move on to setting up the credit card object. We do this in much the same way as the subscription and customer objects, except that this is not an Active Record model, but rather one provided by Active Merchant:

   @creditcard = ActiveMerchant::Billing::CreditCard.new(params[:creditcard])

And then we need to set the first and last name of the cardholder, which we need to get from the customer object:

   @creditcard.first_name = @customer.first_name
   @creditcard.last_name = @customer.last_name

We now have our customer, subscriber, and creditcard objects set up.

c. Checking for a valid order

Now we need to make sure these objects are all valid. Normally, when saving a single object, we use something like this:

   if customer.save
      # proceed, no validation errors
   else
      # redisplay the form with validation error messages
   end

But in this case, we have three objects, and we don’t want to save any of them unless all three are valid. There’s a few ways to handle this, but here’s one simple approach:

   @customer.valid?
   @subscription.valid?    
   unless @creditcard.valid? and @customer.errors.empty? and @subscription.errors.empty?
      render :action => :new and return
   end

We take advantage of the valid? method that is available on all Active Record objects. This performs the validations but doesn’t save the object to the database. If there are any validation errors, they’re stored in the errors array for that object (which is where the error_messages_for helper in the form gets the messages if there are validation errors).

So we validate both the customer and subscription objects, and then we bail out and redisplay the form unless the credit card is valid and the errors attribute on both the customer object and the subscription object are empty.

If our code makes it past this unless clause, we know that we have a valid subscription to process.

d. Setting up the gateway

Now we need to set up all the information needed by the gateway and perform the actual transaction.

First, let’s create the gateway object:

	gateway = ActiveMerchant::Billing::BraintreeGateway.new(BRAINTREE_CREDENTIALS)

We’re just calling a method in ActiveMerchant that sets up the gateway. If we were using a different gateway, we’d of course use the method for the appropriate gateway, but virtually all of the other code would remain the same regardless of the gateway being used (with the one caveat that some gateways require different parameters).

Note that BRAINTREE_CREDENTIALS is set up as a constant. We set this constant in our development environment (config/environments/development.rb):

	BRAINTREE_CREDENTIALS = {:login => 'demo', :password => 'password'}
	ActiveMerchant::Billing::Base.mode = :test

If you were to use this in production, you’d need to get a real gateway account from Braintree (or another gateway provides), and you’d put the production credentials in config/environments/production.rb.

We also set the gateway mode to :test for good measure, though it doesn’t actually matter because the gateway we’re accessing is only a test gateway.

e. Setting the optional information

Different gateways require different amounts of information, and even within one gateway often some information is optional but may reduce your transaction fee if it is provided and is correct. All this optional information is assembled into on hash. So in preparation for running the purchase transaction, we’ll set up this options hash:

   options = {
      :ip => request.remote_ip,
      :billing_address => { 
         :name     => @customer.full_name,
         :address1 => @customer.address,
         :address2 => '',
         :city     => @customer.city,
         :state    => @customer.state,
         :country  => 'US',
         :zip      => @customer.zip,
         :phone    => @customer.phone
      }
   }

The names of the hash keys are taken from the documentation for the Braintree gateway. First we set the customer’s IP address, which we can get from the request object that is always available in any controller. Then we set the billing_address hash, taking the information from the customer object.

Notice, however, that there is no full_name attribute on the customer object — we have first_name and last_name. We could just concatenate those here, but there’s likely to be other places where we need the full name, so it’s better to invent a method to provide us what we need, and then we’ll add this method to the customer model.

f. Performing the transaction

With all this setup in place, actually performing the transaction is easy:

   response = gateway.purchase(@subscription.amount, @creditcard, options)

We simply call the purchase method on the gateway object that we previously created, passing in as parameters the amount of the subscription, the credit card object, and the options hash. We store the response in a new variable we’ve called simply response.

It is in this single line of code that all the web services interaction occurs. Remember that gateway is an Active Merchant object, and when we invoke the purchase method on that object, the Active Merchant code connects to the Braintree payment gateway web service, sends all the information about the transaction, waits for the response, and then returns the response as the result of executing the method.

g. Processing the response

Now that we’ve attempted the transaction, we need to see if it was successful. If not, we want to redisplay the subscription form with appropriate error messages. If so, then we want to save the information from the gateway for later reference and save the subscription and customer information.

Here’s the code:

   unless response.success?
      flash[:notice] = "Transaction failed: #{response.message}"
      render :action => :new
   else
      # save the responses from the transaction in the subscription record
      @subscription.message = response.message
      @subscription.reference = response.authorization
      @subscription.test = response.test?
      @subscription.params = response.params.to_yaml
      @subscription.customer = @customer
      @subscription.save!
      flash[:notice] = "Thanks for subscribing to our site!"
      redirect_to :controller => 'static', :action => 'home'
   end

First we use the success? method on the response object, which is provided by Active Merchant, to see if all went well. If not, we set an appropriate message and redisplay the subscription form.

If all did go well, we save in the subscription object various pieces of information provided by the gateway response. One thing the response object returns is a params object, which is a hash of various parameters that may change from gateway to gateway. So rather than creating a model field for each of these parameters, we convert the params object to text form, using .to_yaml (this is called serializing the data). This allows us to store any arbitrary set of parameters without requiring any code changes.

Finally, we need to associate the customer with the subscription, and then save the subscription. Saving the subscription will also save the associated customer object.

Note that we use the save! method here, because we’ve already validated all the objects, and the new information we’ve added from the transaction doesn’t need to be validated. So if this save fails, that means that there was a programming error, or something really unexpected (like a database failure) happened. If this save fails, an exception will be raised. Somewhere in our code, we should capture that exception and display a “Sorry, something bad happened” message, but there’s not much else we can do.

Finally, we set the flash message and redirect to the home page.

h. Handling the validation error cases

There’s one problem with our code. Suppose that a validation or transaction error occurs, and we redisplay the form. If this happens, we need to have all the instance variable set up that the new subscription form expects. We’ve already created the instance variables for the subscription, customer, and credit card, but we still need to set the page title and the current tab. Since there’s two paths where this would be needed (validation failure, or transaction failure), it’s easiest to just set these values at the top of the method, even though they won’t be used if the transaction succeeds. So add these two lines at the top of the create method:

    @current_tab = "subscribe_tab"
    @page_title = "Please correct the errors below"

Note that we’ve set the page title to an error message, as one more hint to the customer that there’s something they need to fix.

i. Avoiding credit card leaks

We need to be really careful to avoid credit card numbers leaking out. We don’t save them, but we do process them in memory. And we have, inadvertently, stored credit card numbers: they’re in our application log file for each transaction. We can stop this by adding the following line at the top of the subscriptions controller:

	filter_parameter_logging :creditcard

This tells the logger not to show the value of this parameter. The logs will instead show “[FILTERED]” in place of the actual value.

4. Completing the models

a. The subscription model

We’re done with the controller, but we need to complete the models before you can test the code.

First we need to set the associations. A subscription belongs to a customer; that is, we included the customer ID in the subscription table. So add this line to the file models/subscription.rb:

	belongs_to :customer

Now we have to implement the method that we made up when we were writing the controller, that determines the subscription price based on the period. Add the following code:

   def self.cost_for_period period
      if period == 1
         995     # $9.95 in cents
      elsif period == 12
         9995    # $99.95 in cents
      else
         "invalid"
      end
   end

This method is a class method: it acts not upon an instance of the class, but on the class in general. (In other words, the result is the same for all subscriptions.) We signal this fact by prefixing the class name with self..

The logic is quite simple. If the period is 1, the price is 995 (Active Merchant expects prices in cents, so this is $9.95). If the period is 12, the price is 9995 ($99.95).

What should we do if the period is something else? It shouldn’t be possible, since the value is set by a choice from one of two radio buttons. But suppose some trickster spoofed the POST data from the form? They could put in any value they wanted. So we provide a final else clause that returns the string “invalid”.

Then we need to add a line of validation code to the model:

   validates_numericality_of :amount, :message => "Choose a subscription period"

This catches the situation in which neither radio button is selected, so the period is nil, and it also takes care of the situation in which the period is invalid, since the string “invalid” fails the “numericality” test.

b. The customer model

Now we just need to polish off the customer model. First, let’s make all the fields except the phone number required:

	validates_presence_of :first_name, :last_name, :address, :city, :state, :zip

And we need to define the association with subscriptions:

	has_many :subscriptions

(We could use a has_one relationship here if we wanted to restrict a customer to having a single subscription.)

And finally, we need to provide the full_name method that we invented for use in the controller:

   def full_name
      self.first_name + ' ' + self.last_name
   end

Note that this method acts on an instance of the class, not on the class itself, so we do not use self. in the method name, but we do use self. when accessing the instance’s attributes.

5. Ready to test!

This has been our most complex example yet. Let’s make sure it all works.

You’ll need to restart the server, since we’ve added a constant in the configuration file, which is only read on startup.

Click the Subscribe! button, choose a subscription period, and enter a name and address. For the credit card information, the Braintree gateway documentation provides us with the following test card information:

Card type: VISA
Card number: 4111 1111 1111 1111
Expiration: 10/2010

You can use any first and last name and address.

Enter this information, click subscribe, and if all has gone well you’ll see the home page, with “Thanks for subscribing to our site!” at the top. If you get a Rails error, check that you’ve put in all the proper code in the subscriptions controller, the subscription and customer models, and the development environment file.

Now click on the Admin nav button, and then on the Subscriptions Admin link. This will display the scaffolded subscription admin page, so we can see the information that has been written to the subscription database by our code. You can see the series of parameters that have been stuffed into the params field. You can also look at the Customers Admin page to see the customer information.

To see how the parameters change with different levels of verification, enter a subscription with 77777 in the zip code field. The parameters should then show avsresponse=Z (zip code match) instead of N (no match). To simulate a verification code match, enter 999 in that field, and the parameters will show cvvresponse=M (match) instead of N (none). If you get a real gateway account, you’ll find that you pay different fees depending on whether these items match or not, and you can configure your account to decline transactions that don’t match.

You can show what happens with a bad credit card number by entering any other number. The error message displayed doesn’t come from the gateway; our validation of the creditcard object catches this before we get that far, and Active Merchant provides the error messages.

6. Next Steps

This was by far our most complicated lab, but even so, it falls short of a fully robust solution. A complete ecommerce solution has a lot of components and could easily be a two-day class of its own.

Some things you’d want to consider adding for a production environment:

Validate addresses more thoroughly than simply requiring that they not be blank
Set a default value for the subscription period
Process the exception that could occur if for some reason the final save! failed
Create a user account for the customer, so they can log in and see their subscription status
Improve the layout of the subscription form
Process the error messages, rather than using the standard error_messages_for helper, to display them more cleanly
Improve the customer admin page by showing the subscription date and period
Refactor the subscription/create controller code to push more of the logic into the model
And, or course, we haven’t provided any logic to deal with expiration of subscriptions or renewals

Lab 6. Testing

Thu, 24 Jul 2008 08:00:45 GMT

In this lab, we are going to look at some examples of unit and functional testing in Rails. Many of the tests that Rails generates for us automatically need some fixing up to track the changes we made in the code (such as adding authentication), and in this lab we’ll walk through those changes. Then we’ll take some first steps toward more complete tests.

As we’ve discussed, there are multiple approaches to writing tests. In this lab, we are writing tests after we’ve created logic in our program. We encourage you to take a look at test-driven development as well.

Creating automated tests for a Rails application is easier than with most other technologies, but it is still a relatively complex topic. If you’re early in your Rails learning curve, you many want to skip this material and return to it later.

1. Running tests

Many generator scripts, such the scaffold generator, create placeholder test files at the same time they generate the associated application code. All of these test source files get placed in the test directory hierarchy of the application.

Let’s start by seeing what happens when we run our tests:

rake test

You can run more focused sets of the tests by selecting rake test:units or rake test:functionals for instance.

You can also trigger the tests within NetBeans too: just right-click on the project, select the Run Rake Task menu, and select one of the test options.

Running our tests, we get a lot of feedback. Rake runs the tests in sequence, first the unit tests, then the functionals, and so on.

It looks like we have some work to do.

2. Reading the test results

Let’s take apart the results a little at a time. For the unit tests, we scan down and see the lines:

	/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -Ilib:test "/Library/Ruby/Gems/1.8/gems/rake-0.8.1/lib/rake/rake_test_loader.rb" "test/unit/asset_test.rb" "test/unit/category_test.rb" "test/unit/contact_mailer_test.rb" "test/unit/contact_test.rb" "test/unit/content_block_test.rb" "test/unit/customer_test.rb" "test/unit/link_test.rb" "test/unit/subscription_test.rb" "test/unit/user_test.rb" 
	Loaded suite /Library/Ruby/Gems/1.8/gems/rake-0.8.1/lib/rake/rake_test_loader
	Started
	..E..................
	Finished in 0.19429 seconds.

This block of information tells me what tests file were loaded and run, and using the simple command line result format, we see a series of periods and an “E”.

Recall that tests can have one of three states:

Success, indicated with a period (.), sometimes also called “green”;
Failure, indicated with an F, (called “red”), which means one of the assertions we wrote to test an assumption about our code did not work;
Error, indicated by an E, (also called “red”), which means a program logic problem was detected in our test or application code.

So, apparently, we have a number successful tests from seeing all of those periods, but one errored out.

We look further down in the unit test results and see:

   1) Error:
   test_contact_form(ContactMailerTest):
   ArgumentError: wrong number of arguments (1 for 4)
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `contact_form'
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `__send__'
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:410:in `create!'
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:403:in `initialize'
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `new'
	    /Library/Ruby/Gems/1.8/gems/actionmailer-2.0.2/lib/action_mailer/base.rb:351:in `method_missing'
	    ./test/unit/contact_mailer_test.rb:10:in `test_contact_form'
	    /Library/Ruby/Gems/1.8/gems/activesupport-2.0.2/lib/active_support/testing/default.rb:7:in `run'

This gives us a clue as to what went wrong and where. The ContactMailerTest class’ test_contact_form method had a programming error in it: an incorrect number of parameters were passed in. We’ll look at that in a minute.

Finally, just below the list of diagnostic messages for failing or error state tests, we get the summary:

   21 tests, 33 assertions, 0 failures, 1 errors

Wow, we had 21 tests with 33 assertion calls, and we didn’t write any code. Before we look at the problem in ContactMailerTest, what were all of the successful tests doing?

Recall that unit tests check out our models. If we look in all but ContactMailerTest and UserTest, we see classes that contain this:

   # Replace this with your real tests.
   def test_truth
      assert true
   end

Alas, this is a do-nothing placeholder method, so we will ignore it for now. What we should do is write some tests to exercise each model. We’ll try that later.

Peeking inside UserModel, we see a different story. This class is full of tests. It turns out this test file was generated for us when we used the restful_authentication plugin and associated generator to create our user and login system. UserModel tests most of the functionality of our User model and is a good example of how a model should be tested.

3. Fixing the Unit Tests

For now, though, let’s fix our one error in ContactModelTest. This test is exercising the pseudo model that was created when we generated our contact mailer. The test inside of the test class looks like this:

  def test_contact_form
    @expected.subject = 'ContactMailer#contact_form'
    @expected.body    = read_fixture('contact_form')
    @expected.date    = Time.now

    assert_equal @expected.encoded, ContactMailer.create_contact_form(@expected.date).encoded
  end

Mailer related unit tests are a little strange. What this code is trying to do is test to see if the mail message that gets generated by the ContactMailer.contact_form method is identical to a canned email fixture. @expected is an instance variable created for us by the testing class that embodies a TMailer object and which we’ll load with data and subsequently ask to generate a properly formed email body and headers (that is what encoded does).

Here we see that the other special calling convention on a mailer is being used: create_contact_form. Like deliver_contact_form, this method uses the contact_form method we wrote, but rather than building a mail message and then actually emailing it out, this form just returns a string that contains the body of the email message.

To fix this test, we need to make the generated email body equal to the canned version of the email message we prepare in the @expected instance variable. Let’s do that.

a. Call contact_form correctly

The error we received in the first place indicated that we are calling the underlying contact_form incorrectly (not enough parameters). We need to supply name, email, subject, message, and a sent_at time into that method:

  assert_equal @expected.encoded, ContactMailer.create_contact_form('bob', 'bob@bob.com', 'test', 'testing', @expected.date).encoded

If we run the test again (rake test:units), we see the Error is gone, but now we have a Failure: the messages are not equal. Let’s fix that.

b. Set up fixture data

First, lets create a fixture data file with the general body of the email message. This needs to match the view we’ve created to contain the message body (in contact_form.erb from the previous lab):

   Email from your web site

   From: <%= @name %>

   Message: <%= @message %>

Our fixture (found in the directory test/fixtures/contact_mailer/contact_form, another peculiar convention for mailers) looks like this:

   Email from your web site

   From: bob

   Message: testing

Note that we use hard-coded values in the fixture to make it an exemplar of a known working message body.

We are closer, but this still fails. We need to set up the rest of the data needed for the encoded message we are using as our “correct” email state.

c. Fill in rest of email data

We encode the rest of the values in test_contact_form's @expected variable with the sample data we choose to pass in to our own contact_form call:

   def test_contact_form
      @expected.subject = 'test'
      @expected.body    = read_fixture('contact_form')
      @expected.date    = Time.now
      @expected.to      = CONTACT_RECIPIENT
      @expected.from    = 'bob@bob.com'
    
      assert_equal @expected.encoded, ContactMailer.create_contact_form('bob', 'bob@bob.com', 'test', 'testing', @expected.date).encoded   
   end

We save and run this now, and Success! Our admittedly mostly empty unit tests are all running in the “green” state. Let’s go on to fix up some of our functional tests.

4. Fixing the Functional Tests

We now know how to read the test results, so switching our attention to the functional test block results, we see we have our work cut out for ourselves:

   FFFFFFFFFFFFFFEFFF.FFFFFFFFFF......FFFFFFF..........F......FEEEE

Which in my first run was equivalent to 64 tests, 76 assertions, 36 failures, 5 errors. Ouch.

The Rails scaffold generator does a much better job at creating functional tests than it does with unit tests. Functional tests exercise our controllers. Peeking inside most of the test files in the test/functionals directory, you will note that they follow similar patterns.

So what went wrong? In scanning through the test diagnostics, we see a lot of tests with similar issues. There are many tests with problems with redirects: “Expected response to be a <:success>, but was <302>”. There are many tests where things were not created, updated, or destroyed when expected. Thinking back, the common thread here is that all of these things need you to be logged in, yet, when the tests are automatically generated for us, the generator has no idea about this requirement. Let’s fix that. h3. a. Adding authentication to existing tests

The restful_authentication plugin that we used provides a utility module called AuthenticatedTestHelper. This module contains useful methods that you can mix-in to your test classes to simulate things like logging in, precisely what we need. Since we protected most of our administrative functions with those before_filter :login_required checks, that will cause any of our tests that try to directly exercise an action to instead get redirected to a login screen.

Let’s pick one of our controller’s tests to fix up as our example. The others will follow similar patterns and you can look at the final lab code to see the changes (or try to make the changes yourself as an exercise).

First, add the authentication system’s helper at the top of the test class. While there, add the user fixtures so we have some test user accounts to log in with:

   class ContentBlocksControllerTest < ActionController::TestCase
      include AuthenticatedTestHelper
      fixtures :users

Now, let’s use the new method we have to log in as the user quentin:

   def test_should_get_index
      login_as :quentin
      get :index
      assert_response :success
      assert_not_nil assigns(:content_blocks)
   end

Run the tests again. The redirect failure on test_should_get_index should now be gone.

Go back to each method in this test class and add the login_as :quentin line at the start of each test method. All of your redirect failures should now be cleaned up.

But, there are still failures to fix. Look through the other functional tests and make the same changes to all of the others that are protected with the before_filter (assets_controller_test, categories_controller_test, contacts_controller_test, links_controller_test).

For the controllers that are selective about the login requirement, be sure to not login when it isn’t needed. For instance, contacts_controller_test’s new and create tests are not protected by login. See the controller to confirm this.

Run the tests again. We are getting closer. In my run, the results look like this:

   Started
   FE...EE.......E.............F.......................F......FEEEE
   Finished in 6.451664 seconds.

b. Testing asset uploader

Our asset_controller_test is responsible for a number of those errors and failures, so let’s clean it up next. Recall we are using the attachment_fu plugin to implement our uploader, and as luck would have it, the associated README has some guidance about implementation, but not about testing. We’ll have to dig in to the code to figure that out. We’ve done that for you, so let’s walk through what we need to do.

First, make sure you have a test image file as an asset to play with. This takes the role of a fixture, so make a directory in the test/fixtures directory called “files” and place a test file in there. We used an image called “railsquickstart-train.gif”; you can use your own if you wish.

Next, we add information we may need later in our regular fixture file, assets.yml, which spells out some sample content. This will fix any issues with preloaded data for our Asset model, since the validates_as_attachment call makes sure the minimum needed data for a legal attachment is present:

	one:
	    name: first
	    content_type: 'image/gif'
	    filename: '/files/railsquickstart-train.gif'
	two:
	    name: second
	    content_type: 'image/gif'
	    filename: '/files/railsquickstart-train.gif'

Just a few more things. Recall that asset uploading is done through a multipart form upload. The default RESTful controller tests that are created for us by the generate script do not know about the multipart form technique, so we need to help things along.

In our test_should_create_asset method, we need to use some testing helpers to simulate the file upload and we need to tell the post method that we want to do a multipart form. Cruising the documentation on BuildingWebApps.com, we find the fixture_file_upload method, and drop that in:

  def test_should_create_asset
    login_as :quentin
    image_file = fixture_file_upload('/files/railsquickstart-train.gif', 'image/gif')
    assert_difference('Asset.count') do
      post :create, :asset => { :name => 'railslogo', :uploaded_data => image_file },
                    :html => {:multipart => true }
    end

    assert_redirected_to asset_path(assigns(:asset))
  end

We also know that our update method is going to have the same issue, so let’s fix that at the same time:

  def test_should_update_asset
    login_as :quentin
    image_file = fixture_file_upload('/files/railsquickstart-train.gif', 'image/gif')
    put :update, {:id => assets(:one).id, :asset => { :uploaded_data => image_file }}
    assert_redirected_to asset_path(assigns(:asset))
  end

Run the tests. Indeed, we’ve cleaned up the asset_controller tests.

c. User controller cleanup

We see there are still some issues around the user_controller_test. This was a test file that was generated for us when we used the restful_authentication plugin, so what’s up?

We added a before_filter in the users_controller too, to protect the creation of accounts. Let’s fix that up quickly just like we did earlier. Add the AuthenticatedSystemHelper and use the login_as methods in the tests.

d. Email bits and pieces

Down to the wire, and it looks like we have just a few tests to fix up.

The first appears to be an error in test_should_create_contact(ContactsControllerTest). It can’t seem to locate a constant ContactMailer::CONTACT_RECIPIENT. This is an error, so that means some kind of programming mistake is the cause. If you search the code with your editor, you’ll see that CONTACT_RECIPIENT is indeed defined, but it is in config/environments/development.rb.

Recall that testing runs in its own, completely separate, environment. Here is a case where a constant defined in the environment is just plain missing in the test environment. This is fixed by adding (or copying from development.rb) the CONTACT_RECIPIENT definition to the config/environments/test.rb file at the bottom:

   CONTACT_RECIPIENT = 'tester@buildingwebapps.com'

Unless you have a working email connection, you’ll also want to add a line to ignore errors when trying to connect to the mail system (below the CONTACT_RECIPIENT line is ok):

   config.action_mailer.raise_delivery_errors = false

Now, this is somewhat of a hack. We put mail configuration into the initializer file config/initializers/mail.rb. Initializers run after the environment is loaded, and since we are globally setting up the sendmail environment there, we actually have our delivery_method setting stepped on and changed from :test to :sendmail.

In retrospect, we probably should not have put the setup code in the mail.rb initializer file, and instead put the mail setup code in the environment-specific files. That way, we can get the test code to act properly without ignoring possible exceptions.

e. Customers use validation

In this lab, we don’t have a lot of detailed testing for our merchant-oriented code. Even so, we still have a couple of failures to clean up.

test_should_create_customer fails with a pattern that will become familiar to you as you test code that uses validations. Here, the code is trying to make a new customer object. The test, however, by default tries to create an empty customer. Looking inside of the Customer model, we see a validates_presence_of on :first_name, :last_name, :address, :city, :state, and :zip. We better fill in some values for those:

  def test_should_create_customer
    assert_difference('Customer.count') do
      post :create, :customer => { :first_name => customers(:one).first_name,
                                    :last_name => customers(:one).last_name,
                                    :address => customers(:one).address,
                                    :city => customers(:one).city,
                                    :state => customers(:one).state,
                                    :zip => customers(:one).zip }
    end

    assert_redirected_to customer_path(assigns(:customer))
  end

But, what is this customers(:one) syntax? Those aren’t straight string constants. No, instead, we are using references to data we are storing in our fixtures file. Here is what is going on. First, we need the test controller class to load the fixture data, so we add the fixtures :customers line right after the class declaration:

   ...
   class CustomersControllerTest < ActionController::TestCase
      fixtures :customers

      def test_should_get_index
      ...

Now, in our customers.yml fixtures file, we make sure there is some data:

one:
  first_name: 'First'
  last_name: 'Last'
  address: 'Street'
  city: 'City'
  state: 'State'
  zip: 11111
  phone: '123-456-5678'

As a reminder, YAML files are picky about spacing. The label one: starts in the first column and its attributes (e.g. first_name:) are all tab indented.

When loaded, fixtures can be referenced by name, using the name of the file (“customers”), then in parentheses, the name of the record we want as a symbol (e.g., one: is the name in the fixtures file, but in code, we use the symbol :one; note the way that colon moved!). We are now referring to a loaded instance of a model object, so any data field defined in that model is available to us, referred to by a period followed by the attribute name (e.g., customers(:one).first_name). Now our test data and our test code are each in separate files, which makes it easier to maintain them in the future.

f. Subscriptions are compound objects

Subscriptions have a similar problem but are a bit more complex. The ActiveMerchant code and the logic to stitch together subscriptions, customers, and credit cards involves a dance of three models and some business logic. Our remaining error is in the create method, which is the most complex in the controller. Let’s supply all of the data it needs:

	def test_should_create_subscription
    assert_difference('Subscription.count') do
      post :create, :subscription => { :amount => subscriptions(:one).amount,
                                        :period => subscriptions(:one).period },
                    :customer => { :first_name => customers(:one).first_name,
                                    :last_name => customers(:one).last_name,
                                    :address => customers(:one).address,
                                    :city => customers(:one).city,
                                    :state => customers(:one).state,
                                    :zip => customers(:one).zip }, 
                    :creditcard => { :number => '4111111111111111', 
                                    :type => 'visa', 
                                    :month => '10', :year => '2010', 
                                    :verification_value => '999'}
    end

    assert_redirected_to root_path
  end

Wow, that’s a lot!. Here is what we are doing (using a few different techniques for illustration). First, we are supplying a subscription object. The Subscription model requires a value “amount”, so we pull one from our fixtures (defined at the top of our file as usual with fixtures :customers, :subscriptions since we are also using the customers data).

Next, we need to provide a Customer model object, so we reuse the same fixture data we used earlier for the Customer tests.

Lastly, we need to supply a CreditCard object, as used by ActiveMerchant. We should put that data into its own fixture, but here we spell it out explicitly to show that you can do that, if you need or want to.

The create method is now happy, receiving the three model objects it will need. On success, however, it doesn’t send the user back to a show method, like many RESTful controllers, but rather redirects the user to the home page. We adjust our assertion to point to the root_path, which is defined in our routes.rb file as a named route for “home page”.

Run the tests. Success! No problems.

g. Aside: Write a new functional test

We lost a valuable piece of information when we fixed all of those authentication tests. We really should also test that the right thing happens when someone is not logged in. Let’s write a quick example of a test that succeeds when an unauthenticated person tries to get a protected resource. In ContentBlocksControllerTest, add this test:

   def test_should_not_get_index_not_logged_in
      get :index
      assert_redirected_to new_session_path
   end

Here, we are not logged in, and we are trying to get the list of content blocks. We should not be able to do this, and we expect to be redirected to the log-in screen. The assert_redirected_to new_session_path uses the RESTful route to the session controller’s new action. This is where login is initiated.

Run the test, and success!

5. Writing New Unit Tests

Now that we have our existing tests working, and perhaps you’ve tried your hands at an extra functional test, let’s take a brief moment to look at the unit tests of our models. We’ll build a couple of simple tests to get you started.

a. Fixtures with Associations

First, we’ve mentioned in our lectures that fixtures have the ability to not only capture simple values representative of working and broken versions of our models, but also can represent associations such as has_many, has_one, belongs_to, and has_and_belongs_to_many (HABTM). The fixture implementation in Rails 2 allows us to set these relationships up in a natural way, specifying these associations by name.

Let’s look at the fixtures in two of our models, Links and Categories, to see this in action. These two models have HABTM associations. When you have models instantiated in code, and you have used the association methods to identify the relationships, you gain a number of useful utility methods to address the members of the relation. In much the same way, we can “name names” in fixtures.

Suppose we had this links.yml file:

	apple:
	  name: Apple
	  url: http://www.apple.com/

	google:
	  name: Google
	  url: http://www.google.com/

	learningrails:
	  name: LearningRails PodCast
	  url: http://www.learningrails.com/

and this categories.yml file:

	site:
	  name: Site
	blog:
	  name: Blog
	podcast:
		name: Podcast

To indicate which links are currently categorized, we’ll add an attribute “categories:” and literally list out the name of each category fixture member:

	apple:
	  name: Apple
	  url: http://www.apple.com/
	  categories: site

	google:
	  name: Google
	  url: http://www.google.com/
	  categories: site

	learningrails:
	  name: LearningRails PodCast
	  url: http://www.learningrails.com/
	  categories: podcast, site

Note that for “apple” and “google”, we’ve only associated one category with each entry, but we’ve still named the attribute “categories”. Rails will be able to tie the “site” category fixture entry to the associated “link” fixture when it creates the models for you during tests.

“learningrails”, on the other hand, has two categories associated with it, podcast and site, so we list these out as a comma separated list.

Let’s look at the categories.yml file:

	site:
	  name: site
	  links: apple, google, learningrails

	blog:
	  name: blog

	podcast:
	  name: Podcast
	  links: learningrails

Here we do the same thing, and link up (no pun intended) the association from the other side. We have three instances of a “site”, so we list them out as “links” (apple, google, learningrails). We have one podcast, but we still identify it with the plural form “links”. Note also that we don’t have any “blog” items, so that entry in the fixture doesn’t use the “links” identifier.

Naming conventions are important here, and the capitalization and pluralization we’ve used is critical.

Now that we have a data set set up, let’s test it out.

b. Testing Associations in our Model

We are going to load our fixtures into the CategoryTest class, just like we did for functional tests earlier:

   class CategoryTest < ActiveSupport::TestCase
      fixtures :categories, :links
	
      ...

Note that since there are relationships between the models, we need both fixture data files. If one or the other was missing, this would not work.

Now, let’s write a simple test that confirms the test data we just created (and demonstrates that Rails’ use of Test::Unit, the testing framework behind all of this magic, auto loads our data). This test will first check that our “site” category has three links using the notation we explained earlier for referring to fixture members:

   def test_category_has_links
      catsite = categories(:site)
      assert_not_nil catsite
      assert_equal 3, catsite.links.length
   end

Here, we load a local variable “catsite” with the object defined with the “site” name in the categories.yml file. Recall we set up three links, so after asserting that we actually loaded the object with assert_not_nil, we refer to the HABTM derived method “links”, and check that three objects are in that array. Success!

We could further test that the data is also loaded into the test database by using a finder method:

   def test_category_load_site_category
      catsite = Category.find_by_name('site')
      assert_not_nil catsite
      assert_equal 3, catsite.links.length
   end

Again, we should test and have success!

c. One more unit test

Let’s do a slightly more realistic unit test. Let’s add a validation to one of our models and make sure it is working. In the Category model, let’s validate for the presence of the name attribute:

   class Category < ActiveRecord::Base
      has_and_belongs_to_many :links
      validates_presence_of :name
   end

In our CategoryTest class, let’s add two tests that simply try to create an empty model and another that creates a model with a name. The first should fail and the second should succeed:

  def test_should_create_category_with_name
    assert_difference 'Category.count' do
      Category.create(:name => 'testname')
    end
  end
  def test_should_not_create_category_no_name
    assert_no_difference 'Category.count' do
      Category.create()
    end
  end

The assert_difference and assert_no_difference methods run the code snippet provided as a parameter before and after their block executes (in this case, the code that tries to create a Category object). Category.count queries the database and gets the number of Category objects that currently exist. In the case where a create method works, the count should be different (one greater). In the case where it fails, the count should be identical.

We run out tests and… success? Our new tests are working, but one of our old tests is failing!

When we added the validation in the Category model, the test case test_should_create_category(CategoriesControllerTest) broke, because it tries to create an empty Category. To fix this, we can simply supply a name parameter:

  def test_should_create_category
    login_as :quentin
    assert_difference('Category.count') do
      post :create, :category => { :name => 'testname' }
    end

    assert_redirected_to category_path(assigns(:category))
  end

Run the tests again, and finally, we have everything working again.

6. Further Exercises

Install the ZenTest gem and take autotest for a spin (gem install ZenTest). This will continually run your tests as you code and save files. A real time saver.
Fix the remaining (if any, purposely) broken tests in the lab6 or later source (release 6 in Unfuddle).
Add a few validations to the existing models and create the associated tests to check good and bad data test cases.
links_controller_test is missing a couple of tests for two actions we added later in development. What are they and can you write those tests?
Fix up the email settings by moving the initializer code out of mail.rb and in to the different environment files.
Add the Spider plugin for a simple Integration Test that crawls your site (script/plugin install svn://caboo.se/plugins/court3nay/spider_test).

Lab 7. Deployment

Thu, 24 Jul 2008 08:00:45 GMT

In this final lab, we are going to focus on moving your code into a source code control system, preparing a production server for your application, and finally, conducting an actual deployment.

While this lab will use specific services, tools, and server software, the general concepts we will discuss can be applied to most environments. That said, we will not be going too deeply into system administrative tasks. For instance, we won’t be discussing how to properly configure Apache or how to monitor logs. Deploying and conducting daily operations activities on a public production server is a deep topic and is completely independent of your use of Ruby on Rails. If you are new to server administration, it is definitely well worth your time to explore other books, web sites, and experts who can help you out. You’ll find the links that we’ve collected at www.buildingwebapps.com/topic/3-servers.

Note also that this lab uses files that are available in the sample application available to you in the seminar’s Subversion repository. If you have been following the previous labs and recreating the project from scratch, we recommend that you get a copy of our code from Subversion so you can easily copy files over. In addition, the essential files are attached as resources to this write-up.

1. Putting source code into Subversion

The Ruby on Rails ecosystem has developed a number of tools to help make deployment and maintenance of your application easier and less time consuming. Automation is the mainstay of deployment practices where you want a reproducible and controlled process for moving software from development into production.

Ruby on Rails practice, like many other programming environments, fundamentally assumes that you will use a source code control system (SCCS) such as Subversion as the intermediary between your development computers and your production servers. While other deployment strategies are supported, using a SCCS, especially Subversion, is by far the most common, and we’ll focus on that technique here.

Getting set up with Subversion is a three-step process:

Export the code from our shared repository to a directory on your computer. (You can skip this step if you are using your own code but you’ll want a few of our files.)
Import the code into your personal repository on Unfuddle (uploading from your computer to your repository).
Check out a working copy from your personal repository on Unfuddle to a directory on your computer. This is the copy you will modify as you continue working on the code.

This third step is necessary, even though you already have another copy of the code on your computer from step 3, to create a copy that is linked to your personal repository.

Once you’ve done this, adding updates to the repository is simple, and you’ll have a record of every update you’ve ever done and the ability to go back to any point in time.

If you’re using NetBeans, you can do most of these task using its graphical user interface. For the initial process, though, it’s easiest to use the command line. Then, to commit updates you’ve made, you can right-click on your project name, and choose Subversion > Commit.

a. Grab clean copy of latest project

We are using Subversion as our source code control system. The first thing you need to do is get your project into a new Subversion repository that is yours and yours alone. To do this, however, you need a clean copy of the latest code.

Subversion works by keeping private metadata in the same directories as your source code. This means that you can’t simply copy a directory using your operating system that is checked in to another Subversion repository. If you try to do so, you also bring the old metadata, and if you try to add the copied directory into a new repository, Subversion complains.

Note that there are two cases:

If you have not been typing the code in from scratch during the other labs, you should export a clean copy of the code from the seminar repository, as we will use the fresh copy as the basis for our first check-in to your own repository.
If you have been typing in the code from scratch, you should still get a clean copy of the code and stash it in a temporary directory so it will be easier to copy a few files over to your project when we call them out later in this write-up.

First, go in to your command line and make a new directory. Consult your operating system documentation if you aren’t familiar with this process.

Next, change your current working directory to be in the new directory. List the directory contents. It should be empty.

Now, pull down the latest code with the Subversion command:

svn export --username student http://railsquickstart.unfuddle.com/svn/railsquickstart_labs/trunk/labs

If you don’t remember the password, check the Files area of the private Google group for a document that lists the shared passwords.

You should see various messages fly by. You will see something like this (most of the middle messages were deleted here for space):

A    labs
A    labs/test
A    labs/test/unit
A    labs/test/unit/user_test.rb
A    labs/test/unit/asset_test.rb
...
A    labs/public/.htaccess
A    labs/public/stylesheets
A    labs/public/stylesheets/quickstart.css
A    labs/public/stylesheets/scaffold.css
A    labs/public/favicon.ico
Exported revision 10.

Once the code is downloaded, list your directory again. You should see a labs directory. Change directory into labs.

If you have been typing in your own copy, change directory into that location instead.

Pushing clean copy into your repository

You are now ready to import the clean code base into Subversion. In this lab, we are using the Unfuddle service. Unfuddle provides Subversion hosting as well as a number of other useful project management tools (like notes, todo lists, issue tracking, etc.). You received an Unfuddle account welcome sheet at the seminar, and you’ll need to substitute information from that sheet into the following commands.

Issue the Subversion import command to establish your new repository:

svn import --username rqsXX_deploy http://rqsXX.unfuddle.com/svn/rqsXX_labs/trunk/labs -m "Initial import"

Substitute your Unfuddle account number for all three instances of XX in the above command. On pressing return, you should see a number of messages fly by indicating that your source is being imported. You will be queried for your password. If you need a reminder of the deploy account password, see the document in the Files area of the Google group.

Here is what this step looks like on my machine (again, cutting out a chunk of “Adding” messages in the middle):

Necrons:~/Development/work/labs chaupt$ svn import --username rqsXX_deploy http://rqsXX.unfuddle.com/svn/rqsXX_labs/trunk/labs -m "Initial import"
Authentication realm: <http://rqsXX.unfuddle.com:80> Unfuddle Repository
Password for 'rqs21_deploy': 
Adding         test
Adding         test/unit
Adding         test/unit/user_test.rb
Adding         test/unit/asset_test.rb
Adding         test/unit/customer_test.rb
Adding         test/unit/contact_mailer_test.rb
...
Adding         public/422.html
Adding         public/.htaccess
Adding         public/stylesheets
Adding         public/stylesheets/quickstart.css
Adding         public/stylesheets/scaffold.css
Adding         public/favicon.ico

Committed revision 1.

Once the source is imported into your own repository, you can delete the local version in preparation for checking out a version-controlled copy. If you checked in your own, typed in source, keep the spare copy of the lab source in a temporary directory, you’ll want to refer to it later.

Change directory up one level from the labs (or your own) directory and use the appropriate operating system command to delete the labs directory. If you are worried about doing this, archive a copy in a safe place.

First checkout of version-controlled source

Choose where you are going to keep your development directory. Either stay in the current location or change directory to some other place now. Once ready, issue the Subversion checkout command to get a version controlled copy of your source code:

svn co --username rqsXX_deploy http://rqsXX.unfuddle.com/svn/rqsXX_labs/trunk/labs

Again, remember to substitute your Unfuddle account number for XX.

The results will be a long list of files:

Necrons:~/Development/work chaupt$ svn co --username rqsXX_deploy http://rqsXX.unfuddle.com/svn/rqsXX_labs/trunk/labs
A    labs/test
A    labs/test/unit
A    labs/test/unit/user_test.rb
A    labs/test/unit/asset_test.rb
A    labs/test/unit/customer_test.rb
A    labs/test/unit/contact_mailer_test.rb
...
A    labs/public/.htaccess
A    labs/public/stylesheets
A    labs/public/stylesheets/quickstart.css
A    labs/public/stylesheets/scaffold.css
A    labs/public/favicon.ico
Checked out revision 1.

You have now placed your code into source code control, and you have a copy with all of Subversion’s special metadata on your local disk. From here on out, you need to remember that you must use Subversion commands to manipulate files, and not your operating system commands. The Subversion Book is a free, complete reference book about using Subversion that you might want to consult.

Change directory into the newly checked out code. If you’re using our default name, you would enter the labs directory.

2. Configuring Capistrano

Our next step is to prepare your project for using the Capistrano utility. Capistrano is a tool that can run commands and scripts (sometimes called recipes or tasks) on remote computers. This is particularly handy as you will by issuing the same commands over and over again as you develop your application and it is a pain to continually log in to one or more remote computers, type a bunch of commands, then do it again.

Note that you enter all Capistrano commands in your local terminal, even though the Capistrano scripts perform actions on your server. Capistrano makes an SSH connection to the server and issues commands there.

a. Capify your project

Make sure you are in your project’s directory now. You should be at the top level of the Rails application that you just checked out. Make extra certain you are in that directory or the rest of this lab will not work! If you list the contents of this directory, you should see the folders app, config, etc.

Issue the capify command to create the initial Capistrano configuration files:

capify .

Note that period at the end. If you are using the seminar provided code, you will see some “skip” messages, otherwise you will see a couple of files are created:

Necrons:~/Development/work/test chaupt$ capify .
[add] writing `./Capfile'
[add] writing `./config/deploy.rb'
[done] capified!

b. Copy in lab provided recipes

This step is only for those of you working on your own project or typing in the labs from scratch. If you are using the exported seminar files, you can skip this step.

To complete this step you will need access to the lab files (or the attachments to this write-up).

First, replace the config/deploy.rb file with the one we provide.

Second, create the file config/mongrel_cluster.yml and put the following contents inside:

port: 8000
pid_file: /tmp/`hostname`-mongrel.pid
servers: 4
environment: production
address: 127.0.0.1

The mongrel_cluster.yml file specifies on which ports your application server will listen, waiting for traffic to be forwarded to it from Apache. In the default Joyent configuration we are using, four instances of the app server are expected. By default, the app servers will listen on ports 8000, 8001, 8002, and 8003. The corresponding Apache configuration file that is generated during setup will use those ports too.

If you have another application on the same server, you will want to minimally change the port number to be a set of four ports outside of the 8000-8003 range.

Third, copy the lab’s config/accelerator directory into your config directory. It should contain three files: accelerator_tasks.rb, apache_vhost.erb, smf_template.erb.

Lastly, copy the lab’s config/quickstart directory into your config directory. It should contain one file: quickstart_tasks.rb.

At the command line at the top level of your project, issue the command cap -T. You should see something like this:

cap accelerator:create_smf          # Adds a SMF for the application
cap accelerator:create_vhost        # Creates an Apache 2.2 compatible virtua...
cap accelerator:restart_apache      # Restart apache
cap accelerator:setup_smf_and_vhost # After setup, creates Solaris SMF config...
cap accelerator:smf_delete          # Deletes the configuration
cap accelerator:smf_restart         # Restarts the application
cap accelerator:smf_start           # Starts the application
cap accelerator:smf_stop            # Stops the application
cap accelerator:svcs                # Shows all Services
cap deploy                          # Deploys your project.
cap deploy:check                    # Test deployment dependencies.
cap deploy:cleanup                  # Clean up old releases.
cap deploy:cold                     # Deploys and starts a `cold' application.
cap deploy:migrate                  # Run the migrate rake task.
cap deploy:migrations               # Deploy and run pending migrations.
cap deploy:pending                  # Displays the commits since your last de...
cap deploy:pending:diff             # Displays the `diff' since your last dep...
cap deploy:rollback                 # Rolls back to a previous version and re...
cap deploy:rollback_code            # Rolls back to the previously deployed v...
cap deploy:setup                    # Prepares one or more servers for deploy...
cap deploy:symlink                  # Updates the symlink to the most recentl...
cap deploy:update                   # Copies your project and updates the sym...
cap deploy:update_code              # Copies your project to the remote servers.
cap deploy:upload                   # Copy files to the currently deployed ve...
cap deploy:web:disable              # Present a maintenance page to visitors.
cap deploy:web:enable               # Makes the application web-accessible ag...
cap invoke                          # Invoke a single command on the remote s...
cap quickstart:copy_revision_info   # Make REVISION available from site
cap quickstart:get_db_dump          # Retrieve production DB Dump
cap quickstart:tail_server_logs     # Tail production server log files
cap shell                           # Begin an interactive Capistrano session.

Some tasks were not listed, either because they have no description,
or because they are only used internally by other tasks. To see all
tasks, type `cap -Tv'.

Extended help may be available for these tasks.
Type `cap -e taskname' to view it.

Note the accelerator and quickstart lines. If you don’t see those, you didn’t copy the lab files correctly. Go back and check that they are in your config directory.

c. Update deploy.rb with your info

Open up the deploy.rb file with a programmer’s editor. We are going to change the project metadata so it points to your personal Accelerator and Subversion repository. Each area we need to edit has a TODO comment immediately above it.

First, if you are not using the name “labs” for your application, change this line:

#
# TODO: change to your application (labs)
#
set :application, "labs"

The name of your application is typically the name of the directory your code is sitting in. Rails created this directory if you started your project from scratch. If you peek inside of your config/database.yml file, your application name was also used as the root of the database name (e.g. labs_production).

In this write-up, we assume you are using “labs” for the rest of the exercise. We will point out other places you need change but not always provide full commands in those cases.

Next, change the location of your Subversion repository:

#
# TODO: change to your subversion repository (railsquickstart -> rqsNN) (_quickstart -> _labs)
#
set :repository,  "http://railsquickstart.unfuddle.com/svn/railsquickstart_labs/trunk/#{application}"

Here, you should change the words “railsquickstart” (two of them) to rqsXX, where XX is your Unfuddle number:

set :repository,  "http://rqsXX.unfuddle.com/svn/rqsXX_labs/trunk/#{application}"

Your IP number is on the credentials welcome page you were handed at the seminar. Update the IP number of the Accelerator you were assigned to in the domain setting:

#
# TODO: Put your public IP here
#
set :domain, '8.17.170.221'

Also on the credentials welcome page, find your zone name. This is at the top of the paper and is part of From: line of the email. Look for the string that starts with a “Z” and then followed by a seven digit number (e.g. “z1234567”).

Update the deploy.rb file’s server name and alias lines with your zone name:

#
# TODO: Put your domain name here (zNNNNNAA.textdrive.com)
#
set :server_name, "zNNNNNAA.textdrive.com"
set :server_alias, "*.zNNNNNAA.textdrive.com"

Save the file.

d. Checkin your changes

You need to save the changes you have made into your Subversion repository. If you are working with your own code, you have a few extra steps to add files into the repository. Start by typing:

svn status

You will see a list of files that have been added (an initial question mark is displayed) or files that have been modified (an initial M is displayed):

?      Capfile
?      config/accelerator
?      config/quickstart
?      config/deploy.rb

For all files that are new (need to be added), issue this command (substituting “filename” below with the full path/name string shown above):

svn add filename

Once you have added all files, issue the svn status command again and be certain no question marks show up.

Now, check-in your files with:

svn ci -m "Deployment files"

and you will see something akin to:

Sending        config/deploy.rb
Transmitting file data ...
Committed revision 2.

You should now be all set. If you issue another svn status at this point, no files should be shown.

3. Setting up the server

Before starting the deployment process, you need to tweak a few things on your production server.

a. Logging in to the server shell

Its time to switch over to your Accelerator and make sure you can log in. Using your IP address, fire up your secure shell program and log in (substitute your IP number for the XXX):

ssh deploy@8.17.170.XXX
Password: 
Last login: Wed Mar  5 18:22:49 2008 from 76.14.mmm.nnn
                                __                       __ 
                       __      / /___  __  _____  ____  / /_
                    __/ /___  / / __ \/ / / / _ \/ __ \/ __/
                   /_  __/ /_/ / /_/ / /_/ /  __/ / / / /_  
                    /_/  \____/\____/\__, /\___/_/ /_/\__/  
                                    /____/ Accelerators
[z1234567:~] deploy$

You will need to use the deploy account’s default password (see the Files are of the Google group if you can’t remember what it is). You will want to change it once you have set things up. See the follow-up section at the end of this article.

b. Database setup

If you are not using labs as your application (and database) name, or are not using the default database credentials (username and password: seminar), you need to configure a production database now. From the server’s shell do the following:

mysql -p -u root
create database myapp_production;
grant all on myapp_production.* to ’username’@’localhost’ identified by ‘password’;

When asked for the password, use the password supplied for MySQL root access on your Accelerator welcome paper. Instead of _myapp_production_, replace myapp with the name of the application you specified in the deploy.rb file and your database.yml file, and replace “username” and “password” with whatever you want to use. Remember that these need to match the username and password specified in your database.yml file for the production database.

Here is what my commands looked like:

[z1234567:~] deploy$ mysql -p -u root
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 4
Server version: 5.0.37-log Source distribution

Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

mysql> create database myapp_production;
Query OK, 1 row affected (0.00 sec)

mysql> grant all on myapp_production.* to 'seminar'@'localhost' identified by 'seminar';
Query OK, 0 rows affected (0.00 sec)

mysql> exit
Bye
[z1234567:~] deploy$

c. Update gems

The Accelerators are not currently completely up-to-date with the versions of Ruby gems we need for our projects. Let’s update them now.

While still logged in to your private Accelerator issue this command:

sudo gem install -f gem_plugin rails RedCloth

After a while, you should see results similar to the this:

We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:

#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.

Password:
Updating metadata for 335 gems from http://gems.rubyforge.org
...............................................................................................................................................................................................................................................................................................................................................
complete
Successfully installed gem_plugin-0.2.3
Successfully installed rake-0.8.1
Successfully installed activesupport-2.0.2
Successfully installed activerecord-2.0.2
Successfully installed actionpack-2.0.2
Successfully installed actionmailer-2.0.2
Successfully installed activeresource-2.0.2
Successfully installed rails-2.0.2
Successfully installed RedCloth-3.0.4
9 gems installed
Installing ri documentation for gem_plugin-0.2.3...
Installing ri documentation for rake-0.8.1...
Installing ri documentation for activesupport-2.0.2...
Installing ri documentation for activerecord-2.0.2...
Installing ri documentation for actionpack-2.0.2...
Installing ri documentation for actionmailer-2.0.2...
Installing ri documentation for activeresource-2.0.2...
Installing RDoc documentation for gem_plugin-0.2.3...
Installing RDoc documentation for rake-0.8.1...
Installing RDoc documentation for activesupport-2.0.2...
Installing RDoc documentation for activerecord-2.0.2...
Installing RDoc documentation for actionpack-2.0.2...
Installing RDoc documentation for actionmailer-2.0.2...
Installing RDoc documentation for activeresource-2.0.2...

Note your output may be slightly different if you have used sudo already or otherwise touched your gems.

4. Deployment setup and sanity check

Go ahead and logout out of the secure shell session and make sure you are back in your local development machine’s code directory. We are going to do a few one-time setup tasks to get your application prepared and get Capistrano primed for use.

a. Deploy setup step

Capistrano uses a particular remote directory structure for its deployments. Inside of your application area on the remote server (/var/www/apps with our default settings), Capistrano will create a directory for your application, and within that, it will create three additional directories.

The first will be a releases area. This is where Capistrano checks out each version of your application. Capistrano is smart about keeping versions separate, as well as keeping old versions of the application around. It does this so if something goes wrong with a deployment, it is possible to revert to a prior version quickly.

Second, Capistrano creates a soft link (also known as an alias) called current. Current will always point to the active application directory in the releases area (this will normally be the most recent version). Capistrano manages this link for you.

Lastly, Capistrano will make a shared directory. Shared is used for common directories and files that will be used across deployments. Your log files end up in this directory, for instance.

In addition, for our environment, Capistrano does a couple of other housekeeping tasks for us. For one, it generates an Apache VHost configuration for our application. For another, it links our application with the Solaris System Management Facility (SMF). SMF is the tool that starts and stops our application, both at system startup and shutdown as well as whenever we do a deployment.

To set this all up, issue the setup command:

cap deploy:setup

You only do this one time, and the results should look something like:

  * executing `deploy:setup'
  * executing "umask 02 && mkdir -p /var/www/apps/labs /var/www/apps/labs/releases /var/www/apps/labs/shared /var/www/apps/labs/shared/system /var/www/apps/labs/shared/log /var/www/apps/labs/shared/pids"
    servers: ["8.17.170.221"]
Password: 
    [8.17.170.221] executing command
    command finished
    triggering after callbacks for `deploy:setup'
  * executing `accelerator:setup_smf_and_vhost'
  * executing `accelerator:create_smf'
set variables
    servers: ["8.17.170.221"]
  * uploading /var/www/apps/labs/shared/labs-smf.xml
 ** uploading data to 8.17.170.221:/var/www/apps/labs/shared/labs-smf.xml
  * done uploading data to 8.17.170.221:/var/www/apps/labs/shared/labs-smf.xml
    upload finished
  * executing "sudo -p 'sudo password: ' svccfg import /var/www/apps/labs/shared/labs-smf.xml"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
*** [err :: 8.17.170.221] 
    command finished
  * executing `accelerator:create_vhost'
  * executing "ifconfig -a | ggrep -A1 e1000g0 | grep inet | awk '{print $2}'"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
    servers: ["8.17.170.221"]
  * uploading /var/www/apps/labs/shared/labs-apache-vhost.conf
 ** uploading data to 8.17.170.221:/var/www/apps/labs/shared/labs-apache-vhost.conf
  * done uploading data to 8.17.170.221:/var/www/apps/labs/shared/labs-apache-vhost.conf
    upload finished
  * executing "sudo -p 'sudo password: ' cp /var/www/apps/labs/shared/labs-apache-vhost.conf /opt/csw/apache2/etc/virtualhosts/labs.conf"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
  * executing `accelerator:restart_apache'
  * executing "sudo -p 'sudo password: ' svcadm refresh svc:/network/http:cswapache2"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished

For this and future Capistrano commands, you will often need to supply a password. This is your deploy account’s password (see the Files area of the Google group if you can’t remember the default password. If you change it, remember what you changed it to!).

(Note: There is an error line in the above output. We are tracking that down, but it appears to be benign at the moment.)

b. Check setup

Let’s confirm that the environment is set up as Capistrano expects it should be:

cap deploy:check

A long list of status messages will flow by, but if you have followed all of the steps up to this point, you should get a message You appear to have all necessary dependencies installed. If so, you are ready for the first deployment. If not, go back and check to see what piece is missing.

5. Cold deployment

The very first time you deploy an application with Capistrano, you have to boot-strap the environment. For instance, your database tables don’t exist yet, so you need to be sure migrations have been run. Also, since you haven’t loaded the app yet, the application servers don’t need to be shutdown and restarted. This initial deployment is called the cold deployment. Like all other tasks, Capistrano has a command to help with this case.

a. Going from cold to warm

Like setup, cold deployment only happens once, the very first time you deploy a specific application. To get started, issue this command:

cap deploy:cold

Capistrano proceeds to check out your code from your Subversion repository, sets up the current link to the code, migrates the database, and starts the application. Here is a log of my cold deployment; yours should looks something similar to the following if you are using the seminar code:

  * executing `deploy:cold'
  * executing `deploy:update'
 ** transaction: start
  * executing `deploy:update_code'
  * executing "svn checkout -q --username deploy --password SECRET --no-auth-cache  -r1 http://rqs21.unfuddle.com/svn/rqs21_labs/trunk/labs /var/www/apps/labs/releases/20080306031255 && (echo 1 > /var/www/apps/labs/releases/20080306031255/REVISION)"
    servers: ["8.17.170.221"]
Password: 
    [8.17.170.221] executing command
    command finished
  * executing `deploy:finalize_update'
  * executing "chmod -R g+w /var/www/apps/labs/releases/20080306031255"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
  * executing "rm -rf /var/www/apps/labs/releases/20080306031255/log /var/www/apps/labs/releases/20080306031255/public/system /var/www/apps/labs/releases/20080306031255/tmp/pids &&\n      mkdir -p /var/www/apps/labs/releases/20080306031255/public &&\n      mkdir -p /var/www/apps/labs/releases/20080306031255/tmp &&\n      ln -s /var/www/apps/labs/shared/log /var/www/apps/labs/releases/20080306031255/log &&\n      ln -s /var/www/apps/labs/shared/system /var/www/apps/labs/releases/20080306031255/public/system &&\n      ln -s /var/www/apps/labs/shared/pids /var/www/apps/labs/releases/20080306031255/tmp/pids"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
  * executing "find /var/www/apps/labs/releases/20080306031255/public/images /var/www/apps/labs/releases/20080306031255/public/stylesheets /var/www/apps/labs/releases/20080306031255/public/javascripts -exec touch -t 200803060315.34 {} ';'; true"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
  * executing `deploy:symlink'
  * executing "rm -f /var/www/apps/labs/current && ln -s /var/www/apps/labs/releases/20080306031255 /var/www/apps/labs/current"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
 ** transaction: commit
  * executing `deploy:migrate'
  * executing "ls -x /var/www/apps/labs/releases"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished
  * executing "cd /var/www/apps/labs/releases/20080306031255; rake RAILS_ENV=production  db:migrate"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
 ** [out :: 8.17.170.221] (in /var/www/apps/labs/releases/20080306031255)
 ** [out :: 8.17.170.221] == 1 CreateContentBlocks: migrating ===========================================
 ** [out :: 8.17.170.221] -- create_table(:content_blocks)
 ** [out :: 8.17.170.221] -> 0.0218s
 ** [out :: 8.17.170.221] == 1 CreateContentBlocks: migrated (0.0220s) ==================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 2 CreateAssets: migrating ==================================================
 ** [out :: 8.17.170.221] -- create_table(:assets)
 ** [out :: 8.17.170.221] -> 0.0180s
 ** [out :: 8.17.170.221] == 2 CreateAssets: migrated (0.0181s) =========================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 3 CreateUsers: migrating ===================================================
 ** [out :: 8.17.170.221] -- create_table("users", {:force=>true})
 ** [out :: 8.17.170.221] -> 0.0031s
 ** [out :: 8.17.170.221] == 3 CreateUsers: migrated (0.0033s) ==========================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 4 CreateLinks: migrating ===================================================
 ** [out :: 8.17.170.221] -- create_table(:links)
 ** [out :: 8.17.170.221] -> 0.0069s
 ** [out :: 8.17.170.221] == 4 CreateLinks: migrated (0.0070s) ==========================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 5 CreateCategories: migrating ==============================================
 ** [out :: 8.17.170.221] -- create_table(:categories)
 ** [out :: 8.17.170.221] -> 0.0018s
 ** [out :: 8.17.170.221] == 5 CreateCategories: migrated (0.0020s) =====================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 6 AddLinkCategoryJoin: migrating ===========================================
 ** [out :: 8.17.170.221] -- create_table(:categories_links, {:id=>false})
 ** [out :: 8.17.170.221] -> 0.0014s
 ** [out :: 8.17.170.221] == 6 AddLinkCategoryJoin: migrated (0.0016s) ==================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 7 CreateContacts: migrating ================================================
 ** [out :: 8.17.170.221] -- create_table(:contacts)
 ** [out :: 8.17.170.221] -> 0.0133s
 ** [out :: 8.17.170.221] == 7 CreateContacts: migrated (0.0134s) =======================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 8 AddDefensioColumnsToContacts: migrating ==================================
 ** [out :: 8.17.170.221] -- add_column(:contacts, :spam, :boolean, {:default=>false})
 ** [out :: 8.17.170.221] -> 0.0037s
 ** [out :: 8.17.170.221] -- add_column(:contacts, :spaminess, :float)
 ** [out :: 8.17.170.221] -> 0.0034s
 ** [out :: 8.17.170.221] -- add_column(:contacts, :signature, :string)
 ** [out :: 8.17.170.221] -> 0.0036s
 ** [out :: 8.17.170.221] == 8 AddDefensioColumnsToContacts: migrated (0.0110s) =========================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 9 CreateCustomers: migrating ===============================================
 ** [out :: 8.17.170.221] -- create_table(:customers)
 ** [out :: 8.17.170.221] -> 0.0027s
 ** [out :: 8.17.170.221] == 9 CreateCustomers: migrated (0.0028s) ======================================
 ** [out :: 8.17.170.221] 
 ** [out :: 8.17.170.221] == 10 CreateSubscriptions: migrating ==========================================
 ** [out :: 8.17.170.221] -- create_table(:subscriptions)
 ** [out :: 8.17.170.221] -> 0.0041s
 ** [out :: 8.17.170.221] == 10 CreateSubscriptions: migrated (0.0043s) =================================
 ** [out :: 8.17.170.221] 
    command finished
  * executing `deploy:start'
  * executing `accelerator:smf_start'
  * executing "sudo -p 'sudo password: ' svcadm enable -r /network/mongrel/labs-production"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
*** [err :: 8.17.170.221] 
    command finished
  * executing `accelerator:restart_apache'
  * executing "sudo -p 'sudo password: ' svcadm refresh svc:/network/http:cswapache2"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished

We are very close to testing our app out. One more thing step!

b. Warm to hot

A small quirk of the Accelerator environment and the System Management Facility requires that we restart the mongrel application server to get it going. We can start, stop, and restart our application using Capistrano, of course. For this task, issue:

cap accelerator:smf_restart

Remember to enter this in your local terminal, not in an SSH session.

You will see results something like this:

  * executing `accelerator:smf_restart'
  * executing `accelerator:smf_stop'
  * executing "sudo -p 'sudo password: ' svcadm disable /network/mongrel/labs-production"
    servers: ["8.17.170.221"]
Password: 
    [8.17.170.221] executing command
*** [err :: 8.17.170.221] 
    command finished
  * executing `accelerator:smf_start'
  * executing "sudo -p 'sudo password: ' svcadm enable -r /network/mongrel/labs-production"
    servers: ["8.17.170.221"]
    [8.17.170.221] executing command
    command finished

Again, that error in this case is Capistrano and Joyent misbehaving, and is benign at the moment.

c. Try it!

Fire up your browser and hit the URL http://8.17.170.XXX where XXX is your correct IP number.

Success!

6. Normal deployment

After the initial deployment, your work will typically fall into a pattern of incremental improvements to your code, and then deploying to move that (tested!) code into production. There are two common scenarios.

a. Deploying code only

When you need to get code into production and haven’t made any changes to your database, you use Capistrano’s standard deploy command:

cap deploy

The deploy command checks out your code into the releases directory, stops the application server, switches the current directory link, and finally restarts the application server.

b. Deploying with database changes

If you’ve made changes to your model, and you have migrations that need to be run, Capistrano has a special command for this scenario. Rather than the plain cap deploy command shown above, use:

cap deploy:migrations

Like the plain deploy command, deploy:migrations checks out your code, stops the application server, switches the current link, and starts the server again. In addition, it inserts a step to run any pending migrations after the new code is checked out and the server is stopped.

c. Dealing with slower deployment

The deployment process can be pretty quick, but there may be a small window of time in which your users would see a non-responsive application. If you need a little time to get the deployment done, Capistrano has a built in ability to display a “down for maintenance” page. As this page is served directly by Apache, it isn’t affected by the general deployment process.

To turn on the maintenance page:

cap deploy:web:disable

and then reenable the app:

cap deploy:web:enable

7. Common tasks

a. Checking the production logs

Logs are located in /var/www/apps/labs/shared/log (substitute your application name for labs if need be). You can log in to the server and run:

tail -f production.log

and watch your application in real-time. Alternatively, if you don’t want to log in, we’ve provided a Capistrano task for this case. You can use this within your application directory on your local development machine:

cap quickstart:tail_server_logs

b. Check server to see if it is running

If you are logged in to the server, you can use the SMF command svcs to see a list of server processes that are running. A process in a healthy state will be online:

online         20:18:47 svc:/network/http:cswapache2
online         20:24:58 svc:/network/mongrel/labs-production:default

You can get this list on your local development machine with:

cap accelerator:svcs

If you see an application in maintenance, something is wrong. You can try restarting it with cap accelerator:restart_apache for Apache or cap accelerator:smf_restart for mongrel.

c. Stopping and starting apps on the server

The Capistrano versions of starting or stopping mongrel are cap accelerator:smf_start and cap accelerator:smf_start.

If you need to do start or stop an application while logged in to the server, you need to first locate the SMF URL. Issue the svcs command, and look for the third column that starts with svc. Note the string. To stop an application, use:

sudo svcadm disable svc:/network/http:cswapache2

and to start an application use:

sudo svcadm enable svc:/network/http:cswapache2

Swap the svc string with the appropriate identifier for the target application.

d. Apache configuration

Apache is located in /opt/csw/apache2 in case you need to tweak the configurations. Look inside of the /opt/csw/apache2/etc directory.

If you change the apache config, restart Apache, using Capistrano: cap accelerator:restart_apache

f. Download your production database data for use in local development

We’ve written a custom Capistrano task that you can use to grab a copy of your production data from the MySQL database on the Accelerator:

cap quickstart:get_db_dump

Once you’ve downloaded your data, you can import it into your local MySQL database and use it for local testing using a command such as:

mysql -p -u root labs_development < dumpfilename

substituting the name of your database for labs_development and the actual name of the dump file for dumpfilename.

Further Exercises

Change all of your passwords to make your personal servers secure:
1. Change deployer password in shell;
2. Change deployer password in unfuddle;
3. Change mysql password in shell.
Change the email addresses used by your app and accounts:
1. Change email address at unfuddle;
2. Change the addresses used by the sample app (if you are using it).
Write your own Capistrano tasks. Check out the quickstart_tasks.rb for some examples.

Startup Camp and Foo Camp

Thu, 24 Jul 2008 08:00:49 GMT

I feel lucky to have been able to attend last week’s Startup Camp and Foo Camp. These are unusual, invitation-only events, with extraordinary collections of people. Getting an invitation requires some mix of accomplishment, connections, and luck.

They are exclusive events not for the sake of exclusivity, but because they only work at a limited size. O’Reilly Media, which hosts the events and foots the bill (there’s no registration fee), gets to pick the attendees from its diverse range of colleagues and contacts.

Startup Camp

This was the first-ever Startup Camp, created by O’Reilly’s venture fund, O’Reilly Alpha Tech Ventures. OATV set up a two-day program, with a variety of startup veterans to give talks and lead discussions, and invited startups to apply. We were fortunate to be one of the 7 startups accepted — doubly so because getting into Startup Camp provided a much-sought-after invitation to Foo Camp as well.

The companies invited to Startup Camp spanned an incredibly broad range, from custom jewelry for tweens (WhirlyBelle from Replicator) to open-source server management software (Puppet from Reductive Labs) and open-source synthetic biology (Ginkgo BioWorks).

The presenters included Tim O’Reilly and Dale Dougherty (O’Reilly), Bryce Roberts (OATV), Esther Dyson, Evan Williams (Blogger, Twitter), Marc Hedlund (Wesabe), Michael Arrington (TechCrunch), Mark Fletcher (Bloglines), Dave McClure (500 Hats), Howard Morgan (First Round Capital), and Kathy Sierra (Creating Passionate Users).

Since this was an off-the-record session, you won’t see much reporting of the content, and I can’t add much to that either. Keep it in mind next summer if you find yourself leading a new startup and want to apply.

Dave McClure’s Startup Metrics for Pirates presentation is one talk whose slides have been made public. And the “Entrepreneurial Proverbs” session was inspired by older blog posts by the two presenters, Evan Williams and Marc Hedlund.

Foo Camp

As exciting as Startup Camp was, it was a prelude to the much larger Foo Camp.

Foo Camp was created by Tim O’Reilly and his colleague Sara Winge in 2003, and John Battelle wrote one of the first articles about it. The name nominally stands for Friends Of O’Reilly, and it is also a play on the use of “foo” as a stand-in variable name in programming examples (a practice that, incidentally, dates back to the 1960’s). Tim wrote about why Foo Camp last year.

The business rationale for Foo Camp is that it gives the O’Reilly team the opportunity to talk with hundreds of leading-edge thinkers, as they look for ideas for books and conferences.

There’s actually camping at Foo Camp: while many attendees stay at local hotels, a lot of them camp on the back lawn, and inside the office buildings, at O’Reilly. (For me, it’s only a five-minute drive, since O’Reilly is in my home town of Sebastopol, CA.)

O’Reilly aims to have about 250 people at Foo Camp, which requires turning away a lot of past attendees so they can invite lots of new people each year.

Among the well-known entrepreneurs and technologists attending this year were Jimmy Wales, Joshua Schacter, Steven Souders, Adrian Holovaty, Tom Coates, Scott Berkun, Ze Frank, Dries Buytaert, and Caterina Fake. That’s just a random selection of the better known names. There were dozens of other well-known folks, and dozens more inspired, creative folks who have lower profiles.

Most of O’Reilly’s editors and conference directors were there, as well as many of its authors. OATV’s portfolio companies were also well represented.

Foo Camp popularized the “unconference” format, in which the attendees create the program. The sponsors provide a chart with a grid of rooms and meeting times, and the attendees fill it in as they desire.

The unconference format is best known from the proliferation of Bar Camps, which were inspired by the early Foo Camps. (More geek humor here, as in foobar.)

When this format was new, people were often hesitant to propose sessions, but no longer. The schedule board was 80% full within minutes. There’s so many parallel tracks, and so many fascinating people, that you can only see a fraction of what goes on and meet a scattering of people.

One of my favorite sessions was on The Future of News. Both news, as journalism, and newspapers, as businesses, are in a period of dramatic change and stress. Presenters included Monica Guzman of the Seattle Post-Intelligencer, John Markoff and Nick Bilton from the New York Times, Steven Levy from NewsWeek, and Carl Malamud.

Other interesting sessions I joined covered topics including ubiquitous computing, curation vs. crowd-sourcing, pragmatic thinking and learning (Andy Hunt), personal genomics (Esther Dyson, 23andMe), and aggregation vs. copyright.

Tim O’Reilly debated Michael Arrington on whether it is important for Microsoft to develop its own search technology, or if getting it from Yahoo is a reasonable strategy. Danny Sullivan moderated. And yes, that is an inflatable elephant.

Here’s the pair of blog posts that inspired this session:

Why Search Isn’t the Point, by Tim O’Reilly
The Importance of a Competitive Search Market, by Michael Arrington

Retrospection

I have no doubt that this event serves O’Reilly well, but it nevertheless is a great contribution to the community. Rarely is such a diverse collection of extraordinary people brought together under such casual, low-pressure surroundings.

Looking back on Foo Camp, there are so many people I wish I had been able to spend time with, and sessions I wish I had attended. I’ll just have to hope for another invitation in years to come.

As someone who spent a decade running conferences, I at first found the unconference format unsettling. The completely self-organizing nature of the event means that its quality is determined entirely by the attendees. But with a crowd like this one, that’s a good thing.

More Foo Stuff

Setting up Rails on Tiger (Mac OS X 10.4)

Thu, 24 Jul 2008 08:00:48 GMT

This guide walks you through setup instructions for preparing a Mac OS X 10.4 (aka Tiger) development machine to be used for general Ruby on Rails coding. This baseline setup is what we use for our LearningRails online course.

You will end up with a development machine with the following baseline components:

Ruby and all basic Ruby utilities
Ruby Gems package manager
Subversion client
Git client
Native development tools (Xcode, C compiler)
MacPorts native code package manager
MySQL database client utilities and server
Gems for Ruby on Rails, Capistrano, Mongrel, Mongrel Cluster, and MySQL
Programmer’s editor or IDE

Note: In the command sequences we illustrate here, command line prompts are shown as a dollar sign ($).

Prerequisites

This guide assumes you have a Macintosh computer running the current Tiger operating system with up-to-date System Update patches applied. It also assumes you have not set up alternate Ruby on Rails tools prior to running through this guide. If you have, then small adjustments may be required as you walk through the following instructions.

You will need to have access to an Internet connection to complete various download steps.

You will need to have administrator access to your computer to complete this guide. We will be using the sudo command to run various command line programs and some of the Mac OS X native installers will also ask you for your password.

It is helpful if you have access to your operating system installation discs.

The Recipe

Follow this recipe in sequence. If you have previously installed a particular component, you can usually skip the associated step.

If you are planning on upgrading to Mac OS X 10.5 (Leopard), you should do so now and save some time setting up the Ruby environment. Leopard comes with a current version of Ruby and it makes setting things up much easier. If you follow this guide and then upgrade to Leopard later, it is pretty simple to deactivate (or remove) the redundant components that are no longer needed.

Native Development Tools (X11 and Xcode 2.5)

MacPorts will use native development tools when installing many of the utilities listed in this guide. You will also need a native compiler to build many of the gems’ native libraries you are going to use. Apple provides a free native compiler tool set called Xcode. If you have your Tiger installation DVD, load it now. If you don’t have an installation DVD, you can download the Xcode 2.5 tools at Apple’s Developer Web Site. (Note: Apple developer accounts are free.)

Open the Xcode Tools folder (or DMG file if you downloaded the package). Double click on the XcodeTools.mpkg installer and select a standard install. This will take a few minutes to run:

As an optional step, some tools require that the X11 window manager application be installed. Under Tiger, this is an optional install. You can find X11 on your Tiger installation DVD or download it from Apple.

MacPorts

MacPorts is a native code package manager for Macintosh software. This guide uses MacPorts to setup the Ruby tools, MySQL, and the Git distributed version control system. There are many other tools available in the MacPorts library, so it is well worth checking out.

Download the Tiger Universal version (1.6.0 at the time of this writing) and double click the MacPorts DMG file to open it up. Double click on “MacPorts-1.6.0.pkg” to start the installer and select the default options.

After MacPorts completes installation, you need to adjust your command line PATH environment variable so you can run the port command.

Fire up the Terminal program and enter the command:

  $ open .bash_profile

Note that there is a period in front of “bash_profile”. The bash shell configuration file should open in the TextEdit program. If you don’t have a .bash_profile, you can create a new one in your text editor and save it in your home directory.

(By the way, we recommend iTerm as a nice open source replacement to the Apple Terminal program.)

Inside of .bash_profile, find the line that starts with export PATH=, if present. You are going to insert the new directories used by MacPorts into your path:

  export PATH="/opt/local/bin:/opt/local/sbin:$PATH"

If you don’t have a line that exports your PATH, use the text exactly as above. If you do already have such a line, add the /opt/local/bin:/opt/local/sbin: (note colons) after the first quote, but before any other paths. Here is an example:

  export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:$PATH"

Save the file and close TextEdit. Open a new terminal window and have MacPorts update itself with the command:

  $ sudo port selfupdate

Ruby and Ruby Utilities (irb, ri, rdoc)

Tiger comes pre-installed with a version of Ruby that is out of date for day to day use. You will use MacPorts to upgrade to the latest production version of the Ruby toolset. In a terminal window type:

  $ sudo port install ruby
  Password:
  --->  Fetching ncursesw
  --->  Attempting to fetch ncurses-5.6.tar.gz from http://ftp.gnu.org/gnu/ncurses
  --->  Verifying checksum(s) for ncursesw
  --->  Extracting ncursesw
  --->  Applying patches to ncursesw
  --->  Configuring ncursesw
  --->  Building ncursesw with target all
  --->  Staging ncursesw into destroot
...
  --->  Installing ncurses 5.6_0+darwin_8
  --->  Activating ncurses 5.6_0+darwin_8
  --->  Cleaning ncurses
  --->  Fetching openssl
  --->  Attempting to fetch openssl-0.9.8e.tar.gz from http://www.openssl.org/source/
...
  --->  Fetching ruby
  --->  Attempting to fetch ruby-1.8.6.tar.gz from http://www.ibiblio.org/pub/languages/ruby/1.8
  --->  Attempting to fetch ruby-1.8.6.tar.gz from http://mirrors.sunsite.dk/ruby/1.8
  --->  Verifying checksum(s) for ruby
  --->  Extracting ruby
  --->  Applying patches to ruby
  --->  Configuring ruby
  --->  Building ruby with target all
  --->  Staging ruby into destroot
  --->  Packaging tgz archive for ruby 1.8.6_0+thread_hooks
  --->  Installing ruby 1.8.6_0+thread_hooks
  --->  Activating ruby 1.8.6_0+thread_hooks
  --->  Cleaning ruby

Your output may be slightly different depending on the versions of software at the time you run these commands. The listing here was edited for brevity.

You can check that Ruby is installed with:

  $ which ruby
  /opt/local/bin/ruby
  $ ruby -v
  ruby 1.8.6 (2007-03-13 patchlevel 0) [powerpc-darwin8.11]

You should seem similar responses. The important one is the first one which reports back the path of the program. If you are using the MacPorts Ruby, it will be located in /opt/local/bin/.

Ruby Gems Package Manager

You next need to install the latest version of gem.

  $ sudo port install rb-rubygems rb-termios

Check the version that gets installed:

  $ gem -v
  0.9.4

You need 1.0.1 or newer. If you have an older version, you can update with the command line: sudo gem update --system.

Installing a Subversion client

Subversion will be used to access your source code repository during development and deployments. You will need a current version of the Subversion client and you can use MacPorts to install it:

  $ sudo port install subversion +tools

Installing Git via MacPorts

Git is all the rage in the Rails world now and has pretty much replaced Subversion as the version control system of choice. That said, both are in common use. Currently Leopard doesn’t install git by default, so we’ll use MacPorts to quickly install git:

   $ sudo port install git-core +doc +svn

This command installs the core git tools, the man page documentation, and integration with Subversion. This last is useful if you plan on migrating from or need to work with a legacy Subversion repository. The installation will take a while; it has a lot to load.

Besides your own project version control, you’ll typically use git when loading Rails 2.1 and various 3rd party plugins.

Installing MySQL via MacPorts

By leveraging MacPorts to install MySQL, maintenance of the software is slightly easier, especially when you want to upgrade over time. We are using MySQL on our development machine as we prefer to have identical software across our environments. Rails 2.0.2, 2.1, and newer uses SQLite by default, which is fine for development and experimentation, but not appropriate for production code.

To get started, type into your terminal window:

  $ sudo port install mysql5 +server

This command downloads and installs the baseline MySQL client programs and server software. Next, you want to configure MySQL’s server so it launches when your computer boots up:

  $ sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

launchctl is an Apple tool that administers the system daemon that controls the boot process and background programs. Here you are loading the instructions for how to manage MySQL.

A fresh MySQL installation requires its database storage area to be initialized, so you do that next:

  $ sudo mysql_install_db5 --user=mysql

When you configure the storage area with this command, you’re making sure it is owned by the user “mysql”.

MySQL creates a special system file used for program to program communication, called the “socket” file. By default, the MacPorts installation of MySQL server creates this in the directory “/opt/local/var/run/mysql5/mysqld.sock”. Ruby on Rails applications can deal with this just fine if you change the settings in your database.yml file to include a socket entry that points to the correct place. However, we are going to tweak things so all applications can find the file in a fairly standard place: /tmp/mysql.sock with little or no modification.

First, you need to move the default configuration file, my.cnf, to the correct place:

  $ sudo mv /opt/local/etc/my.cnf /opt/local/etc/mysql5/my.cnf

If the installation program didn’t put a file into /opt/local/etc/, try this instead:

  $ sudo mv /opt/local/share/mysql5/mysql/my-medium.cnf /opt/local/etc/mysql5/my.cnf

Now, you need to edit the configuration file to change where the socket file is stored:

  $ sudo pico /opt/local/etc/mysql5/my.cnf

If you aren’t familiar with the pico command line editor, we explain the few commands you will need here.

Inside of pico, use your arrow keys to move down to the line where you first see “[client]” and make the following changes (this is just a small part of the whole file):

  ...
  # In this file, you can use all long options that a program supports.
  # If you want to know which options a program supports, run the program
  # with the "--help" option.
  
  [mysqld_safe]
  socket          = /tmp/mysql.sock

  # The following options will be passed to all MySQL clients
  [client]
  #password       = your_password
  port            = 3306
  socket          = /tmp/mysql.sock

  # Here follows entries for some specific programs

  # The MySQL server
  [mysqld]
  port            = 3306
  socket          = /tmp/mysql.sock
  ...

You are adding the “[mysqld_safe]” section (2 lines) just above “[client]” and then changing the two instances of the "socket = " lines in the “[client]” and “[mysqld]” sections to be /tmp/mysql.sock. Once done, press Control-X, answer Y when asked to save, and press return to accept the default file name (“my.cnf”).

Now you start the server up manually:

  $ cd /opt/local ; sudo /opt/local/lib/mysql5/bin/mysqld_safe &

You can confirm that MySQL is running by trying to fire it up:

  $ mysql5 -p -u root
  Welcome to the MySQL monitor.  Commands end with ; or \g.
  Your MySQL connection id is 1
  Server version: 5.0.45 Source distribution

  Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

  mysql> exit

When you reboot your computer, MySQL should start automatically in the future.

Gems

Tiger requires that you install the base collection of gems you will be using:

  $ sudo gem install rake rails capistrano mongrel mongrel_cluster

All installed gems (including Ruby on Rails and its dependencies, Rake, Capistrano, Mongrel, and Mongrel_Cluster) will get installed at their latest versions.

The MySQL adapter gem needs to be installed, and it is a little finicky due to our use of MacPorts.

If you are running Tiger on an Intel processor, use this command (on one line):

  $ ARCHFLAGS="-arch i386" sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

and if you are on a PowerPC processor, use this command:

  ARCHFLAGS="-arch ppc" sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

Once the command completes, you should be all set with the baseline gems you will need for the LearningRails courses.

Code Editing Tools

While you can get by with using a plain text editor like TextEdit, or even Apples Xcode IDE, you will have much high productivity if you use a programming editor that is highly tuned to Ruby on Rails development.

We use the commercial TextMate programmer’s editor for much of our day to day work. TextMate is highly extensible through a collection of community supplied “bundles”. Many add-ons accelerate development by enhancing the editor (for example, adding language specific short cuts to reduce your typing) or by tying in to other utilities, such as Subversion or Rake, to allow you to quickly get tasks done without leaving the editor’s environment.

There are a variety of good open source or free programmer editors available too. On the open source side, Leopard comes pre-installed with both vim and emacs. jEdit is a very extensible open source editor written in Java. TextWrangler is a free programmer’s editor from BareBones.

Whatever editor you choose, be certain that it provides easy navigation among a large number of open files. Working with Rails applications generally involves dealing with a lot of small files, and that process needs to be efficient.

If you prefer an all-in-one tool, you should look at one of several integrated development environments that exist for Ruby. We use Netbeans in our LearningRails courses, but you should check out the numerous other options listed at BuildingWebApps.com.

Setting up Rails on Leopard (Mac OS X 10.5)

Thu, 24 Jul 2008 08:00:47 GMT

This guide walks you through setup instructions for preparing a Mac OS X 10.5 (aka Leopard) development machine to be used for general Ruby on Rails coding. This baseline setup is what we use for our LearningRails online course.

You will end up with a development machine with the following baseline components:

Ruby and all basic Ruby utilities
Ruby Gems package manager
Subversion client
Git client
Native development tools (Xcode, C compiler)
MacPorts native code package manager
MySQL database client utilities and server
Gems for Ruby on Rails, Capistrano, Mongrel, Mongrel Cluster, and MySQL
Programmer’s editor or IDE

Note: In the command sequences we illustrate here, command line prompts are shown as a dollar sign ($).

Prerequisites

This guide assumes you have a Macintosh computer running the current Leopard operating system with up-to-date System Update patches applied. It also assumes you have not set up alternate Ruby on Rails tools prior to running through this guide. If you have, then small adjustments may be required as you walk through the following instructions.

You will need to have access to an Internet connection to complete various download steps.

It is helpful if you have access to your operating system installation discs.

The Recipe

Follow this recipe in sequence. If you have previously installed a particular component, you can usually skip the associated step.

Ruby and Ruby Utilities (irb, ri, rdoc)

Leopard comes pre-installed with Ruby 1.8.6 and its associated utilities. You can use these programs as is. To check them out, open a Terminal window and type:

  $ which ruby
  /usr/bin/ruby
  $ ruby -v
  ruby 1.8.6 (2007-09-24 patchlevel 111) [universal-darwin9.0]

You should seem similar responses. The important one is the first one which reports back the path of the Ruby interpreter program. If you are using the built-in Ruby, it will be located in /usr/bin/.

Ruby Gems Package Manager

Leopard comes with a pre-installed version of gem. Be sure you have the latest version:

  $ gem -v
  1.0.1

You need 1.0.1 or newer. If you have an older version, you can update with the command line: sudo gem update --system.

Native Development Tools (Xcode 3.0)

You will need a native compiler to build many of the gems’ native libraries you are going to use. You will also occasionally build other native tools via the MacPorts tool described below. Apple provides a free native compiler tool set called Xcode. If you have your Leopard installation DVD, load it now. If you don’t have an installation DVD, you can download the Xcode 3.0 tools at Apple’s Developer Web Site. (Note: Apple developer accounts are free.)

Open the Optional Installs folder, and then the Xcode Tools folder. Double click on the XcodeTools.mpkg installer and select a standard install. This will take a few minutes to run:

MacPorts

MacPorts is a native code package manager for Macintosh software. This guide uses MacPorts to setup an installation of MySQL and the Git distributed version control system. There are many other tools available in the MacPorts library, so it is well worth checking out.

Download the Leopard Universal version (1.6.0 at the time of this writing) and double click the MacPorts DMG file to open it up. Double click on “MacPorts-1.6.0.pkg” to start the installer and select the default options.

After MacPorts completes installation, you need to adjust your command line PATH environment variable so you can run the port command.

Fire up the Terminal program and enter the command:

  $ open .bash_profile

(By the way, we recommend iTerm as a nice open source replacement for the Apple Terminal program.)

Inside of .bash_profile, find the line that starts with export PATH=, if present. You are going to insert the new directories used by MacPorts into your path:

  export PATH="/opt/local/bin:/opt/local/sbin:$PATH"

  export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/local/sbin:$PATH"

Save the file and close TextEdit. Open a new terminal window and have MacPorts update itself with the command:

  $ sudo port selfupdate

Installing Git via MacPorts

   $ sudo port install git-core +doc +svn

Besides your own project version control, you’ll typically use git when loading Rails 2.1 and various 3rd party plugins.

Installing MySQL via MacPorts

By leveraging MacPorts to install MySQL, maintenance of the software is slightly easier, especially when you want to upgrade over time. We are using MySQL on our development machine as we prefer to have identical software across our environments. Rails 2.0.2, 2.1 and newer uses SQLite by default, which is fine for development and experimentation, but not appropriate for production code.

To get started, type into your terminal window:

  $ sudo port install mysql5 +server

This command downloads and installs the baseline MySQL client programs and server software. Next, you want to configure MySQL’s server so it launches when your computer boots up:

  $ sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist

launchctl is an Apple tool that administers the system daemon that controls the boot process and background programs. Here you are loading the instructions for how to manage MySQL.

A fresh MySQL installation requires its database storage area to be initialized, so you do that next:

  $ sudo mysql_install_db5 --user=mysql

When you configure the storage area with this command, you’re making sure it is owned by the user “mysql”.

First, you need to move the default configuration file, my.cnf, to the correct place:

  $ sudo mv /opt/local/etc/my.cnf /opt/local/etc/mysql5/my.cnf

If the installation program didn’t put a file into /opt/local/etc/, try this instead:

  $ sudo mv /opt/local/share/mysql5/mysql/my-medium.cnf /opt/local/etc/mysql5/my.cnf

Now, you need to edit the configuration file to change where the socket file is stored:

  $ sudo pico /opt/local/etc/mysql5/my.cnf

If you aren’t familiar with the pico command line editor, we explain the few commands you will need here.

Inside of pico, use your arrow keys to move down to the line where you first see “[client]” and make the following changes (this is just a small part of the whole file):

  ...
  # In this file, you can use all long options that a program supports.
  # If you want to know which options a program supports, run the program
  # with the "--help" option.
  
  [mysqld_safe]
  socket          = /tmp/mysql.sock

  # The following options will be passed to all MySQL clients
  [client]
  #password       = your_password
  port            = 3306
  socket          = /tmp/mysql.sock

  # Here follows entries for some specific programs

  # The MySQL server
  [mysqld]
  port            = 3306
  socket          = /tmp/mysql.sock
  ...

Now you start the server up manually:

  $ cd /opt/local ; sudo /opt/local/lib/mysql5/bin/mysqld_safe &

You can confirm that MySQL is running by trying to fire it up:

  $ mysql5 -p -u root
  Welcome to the MySQL monitor.  Commands end with ; or \g.
  Your MySQL connection id is 1
  Server version: 5.0.45 Source distribution

  Type 'help;' or '\h' for help. Type '\c' to clear the buffer.

  mysql> exit

When you reboot your computer, MySQL should start automatically in the future.

Gems

Leopard conveniently pre-installs the base collection of gems you will be using, but you need to make sure they are up to date:

  $ sudo gem update

All installed gems (including Ruby on Rails and its dependencies, Rake, Capistrano, Mongrel, and Mongrel_Cluster) will get updated.

The MySQL adapter gem needs to be installed, and it is a little finicky due to our use of MacPorts.

If you are running Leopard on an Intel processor, use this command (on one line):

  $ ARCHFLAGS="-arch i386" sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

and if you are on a PowerPC processor, use this command:

  ARCHFLAGS="-arch ppc" sudo gem install mysql -- --with-mysql-config=/opt/local/bin/mysql_config5

Once the command completes, you should be all set with the baseline gems you will need for the LearningRails courses.

Code Editing Tools

While you can get by with using a plain text editor like TextEdit, or even Apple’s Xcode IDE, you will be more productive if you use a programming editor that is highly tuned to Ruby on Rails development.

We use the commercial TextMate programmer’s editor for much of our day-to-day work. TextMate is highly extensible through a collection of community supplied “bundles”. Many add-ons accelerate development by enhancing the editor (for example, adding language specific short cuts to reduce your typing) or by tying in to other utilities, such as Subversion or Rake, to allow you to quickly get tasks done without leaving the editor’s environment.

If you prefer an all-in-one tool, ook at one of the integrated development environments for Ruby. We use Netbeans when we aren’t using TextMate, but the numerous other options listed at BuildingWebApps.com are worth a look.

Podcasts and Screencasts on Ruby on Rails

Thu, 24 Jul 2008 08:00:49 GMT

There’s a variety of screencasts and podcasts available for Ruby on Rails developers. Here’s a quick rundown of the major ones, and how they differ.

Podcasts

Audio podcasts are great for listening on the go, and for things such as news and interviews.

Tutorial: Learning Rails

Our own Learning Rails series consists of 8 podcasts that cover the fundamental concepts behind Ruby on Rails.

People: Ruby on Rails Podcast

The longest-running podcast in the Rails world is the “official” Ruby on Rails podcast, first created by Scott Barron but produced for most of its life by Geoffrey Grosenbach. This is a great series of interviews with many of the leading developers in the Ruby and Rails world.

News: Rails Envy

The Rails Envy podcast comes out every week and provides a fantastic summary of the latest developments in the Rails world. It’s put together by Gregg Pollack, Jason Seifer, and Steven Bristol, who diligently track a wide variety of Rails blogs to provide you with a distilled and entertaining weekly update.

Screencasts

Screencasts let you watch what’s going on on the screen while the presenter explains what they’re doing. It’s like looking over the shoulder of a developer while they work. Screencasts are a great teaching tool.

Learning Rails

Our Learning Rails screencast series shows how to build a simple web site using Ruby on Rails, step-by-step. Designed for people who are new to Ruby on Rails.

Envycasts

Envycasts is the latest entrant into the Rails screencasts business. They’re created by the Rails Envy folks, Gregg Pollack and Jason Seifer, who bring their trademark wit and creative production to advanced technical screencasts. There’s nothing else like them. Priced at $9 each.

Railscasts

Ryan Bates has producing more than 100 free Railscasts screencasts. They’re short episodes focused on a single feature of Rails, and are a great way to learn how to do specific things with Rails.

Peepcode

Geoffrey Grosenbach’s Peepcode are the gold standard for Ruby on Rails screencasts. At $9 each, they aren’t free, but they’re still a great value. They’re typically about an hour long and deeply explore specific topics, from Git to rSpec.

Pragmatic Screencasts

The Pragmatic Programmer, well known for their Ruby and Rails books, has teamed with Mike Clark of Pragmatic Studio, a leading provider of Rails training, to produce the Pragmatic Screencasts. The 20 to 30 minutes screencasts cost $5 each. So far, most of them are not about Ruby or Rails, but there is a series by Ryan Bates on “Everyday Active Record.”

RubyPlus

Bala Paranj has produce more than 70 free screencasts on various Ruby and Ruby on Rails techniques at RubyPlus.org.

Coderpath

Miles K. Forrest recorded a few early Rails screencasts at coderpath a couple years ago. He’s been quiet since, but he’s talking about restarting the series.

Zenunit Sensei

A short series of Rails screencasts, free for now with some talk of paid screencasts in the future, at Zenunit Sensei. From Australia, apparently by Julian Leviston.

The Rebuilding and Scaling of YellowPages.com

Thu, 24 Jul 2008 08:00:49 GMT

John Straw, chief software architect at YellowPages.com, gave an excellent talk at RailsConf about YellowPages’ conversion to Rails. We’ve pointed to YellowPages in the past as being one of the highest-traffic Rails sites, proving that Rails can scale.

John’s talk covered the scaling issues, but the talk was just as much about the process of successfully doing a big rewrite of a critical application at a large company. (YellowPages.com is part of AT&T.)

You can download John’s presentation from the RailsConf site.

With 2.5M searches per day and more than 170 million page views per month, YellowPages.com is now the biggest site AT&T runs, generating more traffic than att.com.

Why the rewrite?

The previous version was written in Java in 2004 and 2005 by consultants, and had fundamental design problems. This version had 125K lines of code and no automated tests, making new features hard to implement. The Rails version ended up with fewer than 20K lines of code, including tests.

The Java site also had lots of usability issues and SEO problems. As an example of the difficulty of adding features, John noted that it took three months to add a rating feature.

The requirements for the new version included:

Absolute control of URLs
No sessions
Be agile
Develop easy-to-leverage core business services

The new product was built by a cross-functional team of 20 people, including project managers, user experience experts, advertising people, search experts, and content and community managers. There were never more than 5 developers. The entire team sat together for the duration of the project.

The architecture they designed has three tiers: a web tier that delivers the front-end web experience; a services tier that responds to requests from the web tier; and a search cluster that performs the actual searches.

The team built an initial Rails prototype in early 2007 that looked like the existing site, just to get some experience with Rails. (Only one of the team members had any prior Rails experience.) They also built prototype search code in Python. They then started a new Rails prototype with designs from user experience team. They also built a Django prototype to explore that option, and evaluated EJB3/JBoss as a service tier platform.

Why Rails?

The team rejected Java for the front end in part because of its inadequate URL control. In terms of Rails vs. Django, John called the decision a near toss-up. Platform maturity was an important attribute that led them to choose Rails. They also felt that it had better automated testing integration and a clearer path to moving parts of it to C if necessary for performance. Ultimately, the development team simply felt more comfortable with it.

John had initially expected they would use Java for the service tier, but after an evaluation of EJB3 the team felt there were no real advantages over Ruby or Python for their application. All the reasons for choosing Rails for the web tier applied equally to service tier, and having a single software environment has obvious advantages.

Keeping the project moving

The success of this rewrite had as much to do with adeptly managing the process as with technology. One key to keeping it moving was giving one person decision-making authority. Another important factor was freezing development on the existing site, so they wouldn’t have a moving target. An ad-hoc rule they came up with was that if you can’t figure out quickly how you’d like to change something about the existing site, then don’t change it.

Another important process was early and frequent communication with the sales team. A previous site redesign had nearly failed because of lack of support from sales. The project lead, who came from the user experience team, met with 20 to 40 salespeople a week to review the proposed changes. This gave them the buy-in they needed for the site to be successful from a business perspective.

The site was made available to friends and family as a closed beta on 4/26/07, an open beta on 5/17/07, and then went live at the end of June, less than six months after beginning work on their first Rails prototype. This was the first time AT&T had released a beta of any kind.

A few things were farmed out to other teams, including HTML/CSS coding, rewrite rules for legacy URL translation, and performance evaluation for the production deployment configuration.

System architecture

The web tier, service tier, and search cluster are each fronted by an F5 load balancer. The web tier initially used Apache with 16 Mongrels on each server, while the service tier uses Apache with 30 Mongrels and memcached on each server.

The communication between tiers uses stateless HTTP transactions. The service tier provides a set of RESTful services that return JSON to the web tier. The search cluster uses the FAST ESP search engine.

The entire system consists of 25 machines, each with two dual-core, 64-bit processors. Two of the 25 are used for the database. This compares to 21 servers for the previous Java-based solution. There are two data centers with identical setups; each is designed to handle the entire load, so they provide geographically distributed redundancy.

They run Solaris on the database machines and CentOS 5 on all the others. John called Solaris “a mistake they wouldn’t repeat,” not because of any particular problems with it but because of a lack of system administrators in their organization who had experience with it.

They use Oracle for the database engine, which had issues with the large numbers of connections created by the large number of Mongrels. As a result, memory usage on the database servers was high, and these machines were upgraded to 12G of RAM.

Performance optimization

The performance goals, which were achieved, were:

Sub-second home page load time
4-second average search time
Never dies

In the course of performance optimization, they considered HAProxy and Swiftiply but chose to go with the F5 because they had the hardware and understood how to use it.

They wrote Mongrel handlers to pick requests off from the web tier and send them to the service tier without involving Rails. They also wrote a C library for parsing search cluster responses and turning them into hashes.

In the web tier, they switched to Erubis for rendering views. They found asset download times to be a bigger issue than the speed of the framework. Following the Yahoo performance guidelines, they minified the JavaScript. They also switched from Prototype to jQuery.

They used the asset_packager plug-in (now made obsolete by Rails 2) and moved image serving to an Akamai edge cache.

They found that Apache was slow serving the 42-byte single-pixel GIFs that they use as analytics tags, which drove a shift to Nginx for the web tier.

After this tuning, performance was better than the Java site. In terms of availability, all site availability issues in first six months were due to database problems.

All things considered, it was a very successful rewrite. While the problems with the prior Java site cannot all be blamed on the technology, it is clear that Rails gave the team significant advantages with no loss of performance.

New Crop of Rails 2.0 Books

Thu, 24 Jul 2008 08:00:49 GMT

In 2007, more than a dozen Ruby on Rails books debuted — only to be almost instantly made out-of-date by the release of Rails 2 toward the end of the year. In many respects, books written for Rails 1.2 are usable with Rails 2.0, but they can be problematic. If they use the scaffold generator in their examples, or dynamic scaffolding, the examples simply won’t work as written. And there are many small ways, such as the enhanced syntax for migrations, in which a Rails 1.2 book will teach a way that works, but is no longer optimal.

There are now some great Rails 2.0 books out, making life a lot easier for new and experienced Ruby on Rails programmers. Here’s a quick rundown of the recent books.

For Beginners

If you’re new to Rails and don’t have a lot of programming experience, Patrick Lenz’s Simply Rails 2 provides a great start. It explains all the concepts clearly and doesn’t assume much software development background. The example site built throughout the book is a Digg clone, which serves well to illustrate all the key topics.

Getting Started with Rails, for Experienced Developers

If you have some web application and object-oriented programming experience, then you may want a book that goes deeper and explains things more fully, but doesn’t take time to explain programming basics. Agile Web Development with Rails, which was for some time the only Rails book, is a great book for developers with some experience.

The second edition of this book, which is the newest available in print, is based on Rails 1.2, and the nicely detailed “Depot” shopping-cart example fails under Rails 2.0. The depth of this book also creates many places where the differences between the versions is significant.

Fortunately, the third edition is now available as a beta book. You can download today’s version as a PDF file, in which most of the critical updates have already been made, which comes with frequent updates, and also pre-order a printed copy so you’ll have one as soon it comes out the fall.

Going Deeper

After a slew of tutorial books in 2007, most of which are written for Rails 1.2, there’s been a recent surge in deeper, more advanced books that add a tremendous richness to the available Rails books.

The Rails Way is great reference work for Rails developers. It is not a book to start with, but it’s one I’d want to have by my side for reference once I had my bearings.

Advanced Rails Recipes includes dozens of short, focused articles explaining how to accomplish a particular Rails task. There’s a tremendous amount of accumulated community knowledge in this book, which was written by Mike Clark with contributions from many people in the community.

Practical REST on Rails 2 Projects, by Ben Scofield, is the first book to deeply explore the use of RESTful web services as implemented in Rails 2. It even includes an example of a Facebook application.

Deploying Rails Applications is the first book to really tackle Rails deployment in all of its complexities. With a cast of authors that includes Ezra Zygmuntowicz, Bruce Tate, Clinton Begin, Geoffrey Grosenbach, and Brian Hogan, this book encapsulates a lot of hard-earned knowledge and will save a lot of people a lot of pain.

Easy Text Formatting with Textile

Thu, 24 Jul 2008 08:00:46 GMT

When I’m writing content for the web, I hate dealing with HTML coding. HTML is rather verbose as a markup language, and having to include closing tags is messy and error-prone.

We’re all pretty much stuck with delivering HTML code from our web sites, but that doesn’t mean we have to write in it. There are various markup approaches that are superior to HTML when it comes to content creation, and which can be used as source code from which to create HTML. Two that are widely used are Textile and Markdown.

Textile was originally developed by Dean Allen for the TextPattern content management system and is now widely used in wikis and blogs. Markdown, created by John Gruber and Aaron Swartz, is an alternative that is conceptually similar but with a different syntax. Many of the popular blog engines provide a choice of Textile or Markdown (or HTML) for posts and comments (this site offers Textile).

In this article, I’ll show the basics of Textile markup and how easy it is to use it in Rails applications.

What is Textile?

Textile strives to make text markup as clean and simple as possible. You don’t have to put <p> in front of each paragraph; bare text is automatically surrounded by <p> and </p> tags. To make a first-level heading, just start the line with @h1. @. For example:

h1. This is a heading and will be wrapped in <h1> and </h1> tags

This is a text paragraph and will be wrapped in <p> and </p> tags.

This is another paragraph.

And I bet you can guess how to make a second-level heading, or a third-level heading.

You never need to use close tags; a pair of carriage returns automatically triggers the appropriate end tag. Quotes are automatically converted to open and close quotes; hyphens are converted to n-dashes (if there is a space on either side); and double hyphens are converted to m-dashes.

To make text bold, surround it with asterisks; to make it italic, with underscores:

*This text will render in bold*
_This text will render in italic*
_*This text will render in bold italic*_

To add a link to a piece of text, surround the text in quotes, and follow it by a colon and the URL:

"text to be linked":http://thisistheurl.com

To include an image, type its URL surrounded by exclamation points:

!http://domain.com/path/to/image.jpg!

To create a bullet list, start each line with an asterisk and a space, like this:

* First item
* Second item
* Third item

Want a nested bullet list? Just indent the asterisk by three spaces.

There’s more, but this should give you a feel for it. You can also use any HTML you want for things that Textile doesn’t cover — HTML code is simply passed through. For more details, see the Textile Reference. You can try out Textile online at the Textile home page.

Implementing Textile

There are Textile implementations for various environments. In my case, using Ruby on Rails, Textile is implemented by whytheluckystiff ’s RedCloth gem. (The current implementation does have some flaws, and a much-enhanced version called Super RedCloth is in development.) To install, just enter the following in the console:

gem install RedCloth

With RedCloth installed, you create a new RedCloth object and pass some Textile-marked-up text to it:

textile_styled_text ="h1. This is an example of a heading in Textile"
textile_object = RedCloth.new(textile_styled_text)

Then, when you want to render this as HTML, you simply use:

html = textile_object.to_html

Making Textile Even Easier

That’s pretty easy, but you can make it even easier. There’s a Rails helper “textilize,” but thanks to the acts_as_textiled plug from Chris Wanstrath, it’s even easier than that. Install this plugin:

ruby script/plugin install
 svn://errtheblog.com/svn/plugins/acts_as_textiled

And then all you have to do is declare a model attribute as Textile-formatted in your model class:

acts_as_textiled  :field_name

That’s it — everything just works! (Remember, though, that you need to restart the server for the model change to take effect.) Textile marked up text is stored in the database. Form fields automatically show unrendered textile. But any other use of the model attribute automatically delivers the rendered HTML instead of the Textile source.

Markup without Coding

You can also simplify the entry of Textile markup by using the textile_editor_helper from Dave Olsen and Chris Scharf. This plug-in renders a set of icons for the common markup tasks in your form view, just above the text entry area:

When you click an icon, JavaScript code inserts the appropriate markup into your text.

To use this, first install the plugin:

ruby script/plugin install
 http://svn.webtest.wvu.edu/repos/rails/plugins/textile_editor_helper

rake textile_editor_helper:install

And then replace the text_area tag in the form in which the user enters the Textile-formatted text with textile_editor:

<%= textile_editor 'object', 'field' -%>

textile_editor takes all the same options as text_area.

and at the end of the form add:

<%= textile_editor_initialize -%>

Finally, if you don’t already include prototype.js, you’ll need to add that to your layout:

<%= javascript_include_tag 'prototype.js' %>

Presto! Now you have a row of icons above the text area so your users don’t have to remember any markup at all. They’ll still see the markup in the text area (this is not a wysiwyg editor), which will help them learn the markup code.

Web 2.0 Expo Highlights and Factoids

Thu, 24 Jul 2008 08:00:49 GMT

Last week’s Web 2.0 Expo had an overwhelming amount of content. Like any large conference, the quality was uneven, but there were some great talks and a lot of good ones. Among the main-session talks, Clay Shirky gave a wonderful talk on the implications of the web. I especially liked his view on the “cognitive surplus” created by the shift from rural to urban lifestyles, and how this has historically been consumed by alcohol, and then by television. Now this cognitive surplus fuels wikipedia, and all the other forms of user participation on the web. Where does the time come from? Just a fraction of the time spent watching television.

I’m halfway through his new book, Here Comes Everybody: The Power of Organizing without Organizations, and it promises to be a worthwhile read.

The most entertaining talk of the conference, by a long shot, was from Fake Steve Jobs, also known as Dan Lyons in his day job at Forbes. The Secret Diary of Steve Jobs is one of the best pieces of technology satire around.

Video of Dan’s talk at Web 2.0.

Videos of all the other keynote presentations are also available.

Among the most technical talks, and the only one with any real Ruby on Rails content, was Gregg Pollack’s excellent talk on The Art of Testing Web Applications. (We proposed several Rails-related talks, but they didn’t make the cut; apparently the organizers chose to avoid most technology-specific talks this year.)

Many of the presentation files have been posted, and more are likely to be added soon.

Here’s an assortment of random factoids from my notes:

70% of current.com users are on their computer at the same time they’re watching TV
There’s 2.43 billion photos on flickr
28 million people a month share a video on YouTube
MySQL is downloaded 70,000 times a day

AIM stats:
- Delivers 2 billion instant messages every day
- Has had as many as 14 million simultaneous users
- AIM sees a usage drop when there’s a popular show on TV, and then a peak when it ends

Ning stats:
- Ning has about 250,000 individual social networks
- 70% of the networks are active (used in last 30 days)
- Adding 1 million registered users per month
- Adding 1,500 new networks a day

MySpace stats:
- 117M unique users last month
- 100 billion rows of data in the database
- 85 gigs of bandwidth
- 5 million concurrent users
- 200K to 400K new users per day
- 11% of all on-line minutes are spent on myspace
- 50 million messages sent per day

WordPress stats:
- 54 million unique visitors in the U.S. each month — one in four people
- 99.999% of WordPress blogs receive under 10,000 pageviews a day
- In aggregate, there are 10+ million pageviews a day to permalink pages (45% of traffic)
- An amount of content equivalent to 1.5 wikipedias is created on WordPress each month

Tim Bray: Ruby is the Leading Language for Web Applications

Thu, 24 Jul 2008 08:00:49 GMT

In his keynote talk at the recent Silicon Valley Ruby Conference, Sun’s Tim Bray asserted that “PHP and Ruby are the languages of choice for new web apps.” Coming from most people, this wouldn’t be a remarkable comment, but given that Tim is Sun’s director of web technologies, it is worthy of note. Java wasn’t even on Tim’s list, much less his recommended solution.

Tim raised two major concerns about PHP: security and maintainability. He also pointed out that the object-oriented features “seem bolted on,” and don’t feel right as compared to Ruby. He did note that there are some great PHP applications, such as MediaWiki, WordPress, and Drupal.

Tim asserted that Rails is ahead of Java in terms of maintainability because you have so much less code. And, of course, Rails is best for time to market, which can be all-important to the success of a project.

As for when to avoid Rails, Tim cited these situations:

If the IT group will only deploy Java or .NET
When you need a lot of experienced developers quickly
If you have a complex database schema you don’t control
For an application that does a lot of computation

Tim asserted that “waterfall development just doesn’t work any more.” Some principles for success he cited:

Be human
Update often
Link
Don’t try to be sticky
Look good
Balance hubris and humility
Free the conversation

Validating Email Addresses with Ruby

Thu, 24 Jul 2008 08:00:46 GMT

In any application in which a user enters an email address, there is the very real possibility that the user will make a typo and your application will end up with an invalid address. You can have them enter it twice, but this seems clunky. And you can, of course, send an email with an activation link, which provides the only true validation, but there’s no need to bother sending the email if you know the address is no good. Furthermore, once you’re past the page where the user enters their email address, you’ve missed your chance to tell them there’s something wrong and they should correct it.

So you really should do what you can to validate the address when the user enters it. I recently made a simple addition to my applications that helps a lot: verify that the domain name is valid.

It’s surprisingly easy to do — especially with a little help from Peter Cooper’s excellent Beginning Ruby, which has a very useful chapter on network programming. The following code is adapted from his examples:

require 'resolv'
def validate_email_domain(email)
      domain = email.match(/\@(.+)/)[1]
      Resolv::DNS.open do |dns|
          @mx = dns.getresources(domain, Resolv::DNS::Resource::IN::MX)
      end
      @mx.size > 0 ? true : false
end

This example makes use of the Ruby standard library “resolv”, so you need to require it first.

The first step is to separate the domain name from the rest of the email address. The regular expression captures the part of the string that follows the @ symbol.

Then the code creates a new DNS resolver object and queries the resolver for an MX (mail exchanger) resource at the specified domain. This returns an array, which will be empty if there is no MX record for the domain.

(Note: In a previous version of this article, I used Resolv.getaddress to see if there is a DNS entry for the domain, instead of checking for an MX record. This approach works most of the time, but it rejects any domain for which there is no A record. If a domain is used only for email and not for a web server, there might not be an A record. Also, some domains have an A record only for www.domain.com, which will also fail the simple getaddress test.)

You can use something like the following in the validate method within the appropriate model:

unless email.blank?
    unless email =~ /^[a-zA-Z][\w\.-]*[a-zA-Z0-9]@[a-zA-Z0-9][\w\.-]*[a-zA-Z0-9]\.[a-zA-Z][a-zA-Z\.]*[a-zA-Z]$/
        errors.add(:email, "Your email address does not appear to be valid")
    else
        errors.add(:email, "Your email domain name appears to be incorrect") unless validate_email_domain(email)
    end
end

I first check to make sure the email address is not blank, because that’s detected by a simple validates_presence_of :email statement that produces a different error message.

Then I make sure that the email address is at least syntactically reasonable, with a rather ugly regular expression, before bothering to check the DNS.

It should be noted that the regex I use here isn’t designed to cover all of the RFC2822 cases, nor with other RFC drafts dealing with non-ASCII addressing.

An even better approach would be to use an observer to validate the address with an Ajax call before the user submits the form.

It is possible to take this a step further by sending the SMTP server referenced in the MX record a “RCPT TO:” command. In theory, this would check that the user name is valid as well as the domain name. This takes additional time, however, and I’ve read that the response from mail servers is often not reliable. If anyone has tried this, I’d appreciate any feedback on how well it worked.

Exception Handling and Old URLs

Thu, 24 Jul 2008 08:00:46 GMT

Whenever I deploy a Rails site, I install the exception notification plugin so I get an email if a user provokes a bug I hadn’t found. It’s a piece of cake to install:

1. Install the plugin

ruby script/plugin install exception_notification

2. Include the plugin in your ApplicationController (in the application.rb file)

include ExceptionNotifiable

3. Add one line to your environment.rb file to specify where to send the email

ExceptionNotifier.exception_recipients = %w(person1@domain.com person2@domain.com)

4. Make sure you have ActionMailer configured (to use either sendmail or STMP), which you’ll already have done if your app sends email for any purpose.

What’s this? Errors already?

A few minutes after I first installed the Exception Notification plugin, I checked my inbox and found a dozen error emails from the site! After a brief moment of panic, I realized that all were coming from search engine spiders, and that the URLs were all invalid. I had replaced an old, crufty, static HTML site, and the spiders were rechecking pages they had indexed in the past.

So the next question was what to do with the old URLs. I could remap each URL to the most appropriate page on the new site, but the old site got little traffic and there wasn’t a clear mapping between the two sets of pages, so it hardly seemed worth it. I decided to map index.html to the new home page, since many people might have bookmarked that page, and it was clear what it should map to. As for everything else, I wanted a way to tell the spiders to stop trying to index them, and to tell anyone who accessed them that this was no longer a valid URL and they should explore the new site.

The heart of the fix is to add a few lines to routes.rb. Fortunately, the old site design had put all the HTML pages except for the index page into a directory called html. There were also some PDF files, conveniently in a directory named web_pdfs.

Here’s the routes, which I added at the end just before the default route:


map.connect '/index.html', :controller => 'page', :action => 'home'
map.connect '/html/*any', :controller => 'page', :action => 'oldpage'
map.connect '/web_pdfs/*any', :controller => 'page', :action => 'oldpage'

I have a controller called page_controller that manages the public parts of the site. The first route simply maps index.html to the home page action.

The second route maps any URL in the html directory to a new page, which I called simply “oldpage”. The oldpage.rhtml file simply has some text telling the visitor that this page no longer exists, and gives them some suggestions for exploring the new site.

The “*any” in the route definition absorbs whatever comes after “/html/”. With this technique, I don’t have to worry about whether there might be subdirectories within html, or what the file name extensions might be.

The final piece was to tell the spiders not to bother with these pages any more. I didn’t want to use a redirect, because the page to which I wanted to redirect the user wasn’t the new permanent page, but rather one that explained that the site had been changed. I decided to set the HTTP header to the “gone” status code (410). This just takes one line in the page controller oldpage action:


def oldpage
  render :template => 'page/oldpage', :status => :gone
end

Eventually the spiders will stop spidering these old pages, but since index.html will still lead to the new home page without any error code, they will find all the new pages from there.

Using SSH Keys to Speed Login

Thu, 24 Jul 2008 08:00:46 GMT

With a basic SSH setup, you have to enter your password every time you log in to the server, which is not unreasonable from a security perspective. But if you want to automate tasks and use deployment tools such as Capistrano, you’ll end up typing that password over and over again, even for a single deployment process. Fortunately, there is a mechanism to avoid this while still preserving good security. But, as with most such things in Windows, it takes a little effort to set it up. (Things are a little simple on Macs, which we’ll cover in a future article.)

SSH authentication uses public key cryptography, in which you have a private key available only to you on your local system, and a matching public key that can be published on your server. Authentication software can confirm that the public and private keys match, but hackers cannot derive your private key from your public key. Once you set up a public-private key pair, these keys can be used to authenticate your SSH sessions, and you won’t ever have to type your password again.

There’s a couple different programs you can use to accomplish this; I’m going to explain how to do it with PuTTY and its associated programs, PuTTYgen and Pageant. If you installed the full PuTTY package, you’ll have all three programs already installed. If not, download the installer and run it now. (Be sure to get the full package, under the heading “A Windows installer for everything except PuTTYtel,” and not just putty.exe.)

Creating Your Keys with PuTTYgen

To create your public-private key pair, run PuTTYgen. There’s several types of keys, but SSH-2 RSA is the most common and is selected by default. (If this doesn’t work, you’ll need to check with your host to see what type of key their SSH server is expecting.) The number of bits defaults to 1024, which is fine. So all you have to do in the PuTTYgen window is click the Generate button, and then wiggle the mouse around a bit. The mouse movements generate random data that ensures that your key is unique.

When PuTTYgen is done creating the key, it will show a long string of characters that make up the public key. Select this text and paste it into a file, named something like id.pub (using notepad or any simple text editor). I made a folder at the root level of my C drive called SSH to store these keys and other related info, but you can put it anywhere you can find it later. (Note: you can also click the Save Public Key button and enter a file name, but this file won’t work as an alternative to the id.pub file we generated with cut-and-paste. It includes line break characters that confuse the server-side SSH code.)

Now you need to save your private key. If you just click the Save Private Key button, PuTTYgen will ask if you really want to save it without a passphrase, because we didn’t enter one. Here you have a choice to make between convenience and security.

The passphrase is essentially a password for accessing the key. Once you have your public key uploaded to your server (which we’ll do shortly), anyone who has access to your private key will have access to your server. If you use password protection on your PC, and you’re the only one with access to it, you might be comfortable going without a passphrase. But it is safest to use a passphrase, and we’ll soon see how you can make it so you only need to enter it once each time you boot your system. So to set a passphrase and save the private key:

Enter it twice, once in the Key Passphrase field and once in the Confirm Passphrase field. Keep in mind that this passphrase is essentially the key to accessing your server, so make it a robust password.
Click the Save Private Key button, and enter a file name (no extension) for your private key. The .ppk extension is automatically appended.

You now have your key pair and are done with PuTTYgen. Next you need to upload your public key to your server and set up your PC to access your private key.

Uploading Your Public Key

The details of uploading your public key may vary depending on the server configuration. The instructions below are for Rails Machine and are derived from the Mac and Linux oriented instructions they provide.

Open an SSH session to your server (using PuTTy, or another client if you prefer, as described in my previous post.) You probably have more than one user account; in my case, following the recommended practices from the Rails Machine folks, I have a root account that I never log into directly, and regular user accounts of Michael and Deploy. The Deploy account is the one I use for almost all communication with the server. So log into that account, or its equivalent for your setup. You’ll have to manually enter the password one last time.

Now, in the shell window that is connected to your server, create a directory for the private key file:

mkdir ~/.ssh

This creates a directory named .ssh within your home directory, which is where the SSH server will look for the public key.

Now set the permissions for this directory so you, but only you, have all privileges:

chmod 700 ~/.ssh

Now you have a directory on your server to hold your public key, and you need to move the key up there. There’s various tools you can use to do this. One tool you should become comfortable with is scp, or secure copy. It is not built in to Windows, but there is a version of it that comes with PuTTY, called pscp. If you add the path to the PuTTY program directory to your system path, you’ll be able to use pscp in any command window. (You may also want to install a set of Unix-style utilities; you can install the entire Cygwin environment, or if you want something lighter weight just for SSH-related tasks, get just the OpenSSH utilities. In either case, make sure to add to your Windows system path the folder in which these programs are stored, so you can use them from any command window without having to type their full path.)

To copy the public key, follow these steps:

Open a Windows shell in the folder in which you’ve stored your public key. (If you installed the Command Here utility as I recommended in the previous article, you can just right-click the folder and choose Open Command Window Here.)

In the command window, type

pscp id.pub username@hostname.com:~/.ssh/authorized_keys

(Of course, you’ll need to replace “username” with your actual user name, and “hostname.com” with the name of your server. If you’ve named your public key something other than id.pub, replace that name as well. Finally, if you’re using scp from OpenSSH instead of PuTTY’s pscp, drop the p in the command name.) This will copy your public key to a file called authorized_keys in the .ssh directory in your home directory.

Finally, to make the key file a little more secure, go back to your SSH window (remember, we started there but then switched to the Windows console), and type:

chmod 600 ~/.ssh/authorized_keys

This ensures that only the owner of this file (that’s the user name you began your SSH session with) can read or write it.

Making Your Private Key Available in Windows

OK, we’re almost there. Now we need to enable Windows programs making SSH connections to access your private key file. You could set PuTTY to use the key file, but that doesn’t buy you much, since it will ask for the passphrase every time you open a connection, and it won’t be available to other programs (such as Capistrano). So, you need to use another program called Pageant, which is installed along with PuTTY, to load the key into memory and make it available to other programs.

You can run Pageant directly via Start > All Programs > PuTTY > Pageant, and then you can tell Pageant to load your private key. But assuming you want the private key to always be available, you want it to load automatically upon startup. To do so, create a text file called load_private_key.bat (or whatever), with the following contents:

start “Pageant” “c:/Program Files/PuTTY/Pageant.exe” c:/ssh/id.ppk

Note that you’ll need to change the path to Pageant.exe if you didn’t install PuTTY in its default location. The id.ppk file is the private key file that you generated from PuTTYgen. (Using the “start” command, rather than simply providing the path to Pageant directly, prevents a DOS window from being left on the screen. Thanks to Tim Jervis for this tip.)

Finally, add this batch file to your startup tasks (Click Startup > All Programs > right click on Startup and choose Open, then right-click the load_private_key.bat file, drag it into the startup folder, and choose Create Shortcut from the menu that appears when you release the mouse).

Now, when you reboot your system, the batch file will run, Pageant will load your private key, and you’ll be prompted for the passphrase that you specified when you created the key. Enter this passphrase just this once, and your private key is now available to all SSH functions. When you shut your computer down, everything is secure again.

Setting up Subversion

If you’re using Subversion, you need to take one more step to enable it to use the private key generated by PuTTYgen: adding a line to Subversion’s configuration file.

Subversion’s configuration file is located in the Application Data directory under your user account. The full path is:

C:\Documents and Settings\{your windows user name}\Application Data\Subversion\config

Note that Application Data is a hidden folder, so to locate this file you must have Windows set to show hidden files and folders.

Open the config file in any plain text editor (such as Notepad) and add the following line:

ssh = $SVN_SSH plink.exe

plink.exe is the command-line link setup program that is included with PuTTY.

You’ll also need to make sure that the PuTTY directory is listed in your system’s Path.

Unfortunately, plink insists on popping up a DOS window, which is annoying. If anyone knows how to stop it from doing this, please let me know!

You’re Done!

That was simple, wasn’t it? :-) This may seem like a lot of trouble to go to just to avoid having to type your password, but once you’ve set this up once, you’re done. And if you’re using an automated deployment tool such as Capistrano, you’d have to type your password multiple times for a single deployment (since one deployment involved multiple SSH commands and other actions); with this setup, it can be fully automated.

Building Web Apps

Web 2.0 Expo: Social Media

Social Media Marketing - why it fails and how to fix it

The Whuffie Factor

Slides

Video

Other Social Media Talks

Designing Social Interfaces

Designing Social Websites

Building Sites Around Social Objects

Social Media Buyer's Guide

Beyond Buzz: Measuring a Conversation

Web 2.0 Expo: Web Developer Tools

Web 2.0 Expo: TV and Radio with an API

Web 2.0 Expo San Francisco Spring 2009

March 31 - April 3, San Francisco

Enhancing Conditional Routing in Rails

The Goal — Conditional Routes in routes.rb

The Implementation

Lab 8 (Bonus): A Page Model and Cleanup

Page model

Controller and view code

The joys and dangers of ERb

Creating pages

Putting metadata in the layout

Supporting routes

Resource page enhancements

User management

Creating the first user

Making things a little prettier

Testing Purchases Through Mock ActiveMerchant Gateway

Refactored SubscriptionsController.create

Environment File Change and Subscription Model Tweak

Some Tests

But wait, there’s more!

What’s next?

Lab 1. Set-Up and Static Pages

1. Set-up development environment

2. Create your Rails project and its associated databases

a. Create your local database

b. Create a Rails project

c. Configure your Rails project to use your database

d. Run the application

3. Creating your first pages

a. Creating your first controller and views

b. Adding a layout

c. Add basic navigation links

d. Make the root URL work, and clean up the others

e. Add a stylesheet

f. Adding in page titles

g. Make the navigation buttons highlight

Moving On

Lab 2. Building a Simple CMS

1. Creating a text management system

a. Using the scaffold generator

b. Creating the database table

c. Using the content blocks

d. Using Textile markup

2. Adding a File Uploader

a. Creating the asset scaffold

b. Adding attachment_fu

c. Updating the model

d. Updating the views

e. Using the image

3. Adding an admin page and log-in

a. Creating an admin page

b. Adding authentication

c. Adding user management

d. Using authentication to protect the admin pages

What’s Next?

Lab 3. The Resources Page, with Ajax

1. Creating the links and categories database

a. Creating the models

b. Creating the join table

c. Setting up the association

2. Fleshing out the link admin

a. Adding a category picker to the link views

b. Providing the list of categories in the links controller

c. Adding links on the admin pages

d. Protecting the link and category admin pages