nosewheelie

Technology, mountain biking, politics & music.

Archive for the ‘Ruby’ Category

Scoodi featured on Brisbane Extra

without comments

Scoodi, the Rails app I’ve spent a fair bit of this year working on was featured on Brisbane Extra yesterday.

Brad and Kristian do a lovely job of marketing :)

Written by Tom Adams

August 15th, 2008 at 1:19 pm

Posted in Environment,Ruby,Technology

Tagged with

iPhone sites launch

with 4 comments

Well it’s finally official, and public. After two solid months of development, today marks the launch of the three iPhone sites we’ve been working on:

Screenshots of the sites courtesy of Pete.

These sites may not work on a regular browser, you may be pushed to the regular sites. Tweak your user agent to pretend to be an iPhone for all the goodness.

They’re all developed in Rails, with TrueLocal running under Tomcat using JRuby.

Bring on the 11th!

Written by Tom Adams

July 10th, 2008 at 4:53 pm

Posted in Java,Ruby,Technology

Nobody *ever* uses map, et. al… (example 2)

with 2 comments

Today’s Nobody *ever* uses post.

require "netaddr.rb"
CIDR_ADDRESSES = ["192.0.2.1/24", "..."].map {|address| NetAddr::CIDR.create(address)}
in_range = CIDR_ADDRESSES.any? {|address| address.contains?("192.0.2.1")}

But of course nobody ever does this…

Written by Tom Adams

July 8th, 2008 at 11:04 am

Posted in Functional,Languages,Ruby

Tagged with ,

Nobody *ever* uses map, et. al… (example 1)

with 4 comments

Inspired by Daniel’s comment on Tony’s blog, I figured I’d start documenting the times I use “functional” constructs in my everyday work.

Consider this piece of XML, returned from an external web service:

<suggestions>
  <suggestion>
    <suggestion>paddington, NSW</suggestion>
    <location>true</location>
  </suggestion>
  <suggestion>
    <suggestion>paddington, QLD</suggestion>
    <location>true</location>
  </suggestion>
</suggestions>

Perform the following:

  1. Parse out each suggestion, turn it into an instance of Suggestion;
  2. Capitalise the first letter;
  3. Sort by name;
  4. Remove duplicates;

Here’s some Ruby code that does this, parsing by Hpricot:

class SuggestionsParser
  def self.parse(node)
    suggestions = node.search("//suggestions/suggestion").map do |s|
      Suggestion.new(s.at("suggestion").inner_text.capitalise_first_letter, s.at("location").inner_text == "true")
    end
    suggestions.sort.uniq
  end
end

But of course nobody ever does this…

Written by Tom Adams

July 2nd, 2008 at 2:14 pm

Posted in Functional,Languages,Ruby

Tagged with , , ,

Rails 10x more productive, Scala 2x. Really?

with 18 comments

Via Coderspiel comes a Quick RailsConf Update by Josh Susser.

I’m going to bite and take issue with this point that Josh makes:

I think this is the most likely threat to the Rails surplus, that C# or Scala or something can do a good enough job that people can double their productivity with far less of a change in mindset or tools, and eventually no one will care about the ten times (or whatever) productivity of Rails. “Good enough is good enough.”

Now I’ve not used C# much in anger (I have however seen some of the cool ideas pulled in from F#) so won’t comment on it, but I’ve now developed one large Rails app (multi-year, multi-person), one small Rails app and been peripherally involved in one medium & one small Scala projects (in addition to three Scala OSS projects), preceded by about 10 years of Java/C++/PHP projects.

So, I feel quite qualified to offer an opinion on the contention that Scala’s doubling of productivity is “good enough”. Too summarise this post early, this is just plain wrong [1], the productivity boost that is, not the “good enough” bit. Learning functional techniques will not only make you more productive, but at worst they’ll make you a better “Rails programmer”.

Note that I’m not going to address the “Rails vs. Scala” debate, which would be better rephrased as “Ruby vs. Scala”, or “Rails vs. Lift or a hypothetical Scala web app framework”.

Rails

I’m quite happy for people to claim the now standard “10x” productivity boost for new Rails apps, having been through a couple now, I can see how it gets you up and running quickly, mainly by taking a bunch of decision points away from you (that you’d have to make in a Java project for example, what O/R tool, what directories do I need, etc.) and how it simplifies a lot of the drudgery found in most web projects. Throw in some clunky higher order functions for good measure and you’re away!

Actually, I’m not entirely happy with accepting this, I’ve found that in my experience you save about 25% of total development time for small projects, but the curve flattens out the longer the project goes on (say, after a few months), if you add more people, if the project is complex or it’s not a webapp. If any of these get large, you loose the advantage of Rails. Ruby’s lack of a static type system also leads to issues, however I don’t want to address these in this post.

So I have no real problem with Rails for simple web projects; Rails is optimised for the general case, and when you stick to what it’s good for Rails is fine (ignoring the thorny issues of side-effects & composability). What I do have a problem with is claims that developers would want to settle for Scala, or, any (pure or not) functional language.

Scala

I’m not a functional programming guru, however I know enough to be dangerous and I like to improve the way I develop software. There are many compelling reasons why people love functional languages. They are succinct, elegant and composable. I’ve heard “functional programmers” [2] make claims that they can employ high-level abstractions (with funky names like monad & functor) to get massive increases in productivity (usually one to two orders of magnitude). What’s more, they can back up these claims also!

I can only speak first hand about the tools that I’ve used, but I can certainly see how people can make these claims. In what little Scala I’ve written, I’ve seen how you can drastically reduce the size of your code, increase its readability and improve your ability to reason about what’s going on [3]. And this has helped me a lot in my current Rails project!

I’ll say it in different terms, knowing functional programming techniques means you can write Rails apps faster (yes, better than 10x!) [4]. I’ve experienced this first hand on my current project. I’ve written code that would have taken me half a day in Java in 10 minutes, made possibly by only a miniscule knowledge of functional techniques (e.g. map & fold). There’s another post on countering claims that no-one ever uses these abstractions… I’ve used two folds and a map already this morning.

Also, once you learn the basics, with functional languages you also get a better handle on what it is you are actually doing as the language of the problem is the language you’re coding in. You stop thinking in terms of the machine, and more in terms of the problem you’re trying to solve. For example consider this piece of code from Furnace:

removeHeader(stream).filter(!newLines.contains(_)).take(40)

What does it do? Well it removes the header from a stream of bytes, filters out newlines and takes the first 40 bytes. Does the code look similar to the way I’ve described it?

When I first tried to write this code, I thought about the underlying stream of bytes, pulling them off one by one, what about buffering, what happens if I read too many and need to skip back, etc. My thoughts had been warped by years of imperative thinking. When I discussed this problem in the office, I realised that this can be represented simply, if I simply though about what it was I wanted to do, and the code came very easily after that [5].

Summary

Of course there are tradeoffs here, and learning function techniques is mind-bending (in a good way), you’ll also realise how little you actually know. By using a language like Scala you get to have your cake (e.g. great APIs, type inferencing, DSLs, etc.) and eat it too (static typing). You also get great productivity improvements. Am I willing to put a number on it, no, I don’t have enough experience yet, but I can see it has the potential to be large.

This has been a bit of a rambling post, but my central premise is these two things; 1) Using functional techniques will let you be more productive in the general case (i.e. not just web apps) than Rails makes you for writing web apps; and 2) learning functional techniques will make you a better “Rails programmer”.

Footnotes

[1] To be fair, Josh is probably just throwing out the names of the latest languages that are causing a buzz, so Scala gets lumped into that. Still, this doesn’t make the statement true.
[2] Functional programmers usually don’t like to use this term, but I’ll use it here for quick categorisation.
[3] You also get other nice things like improved testability, increased maintainability, flexibility, etc.
[4] I have come up against some of the barriers to making things even easier, for example using APIs that rely on side-effects and are not referentially transparent breaks your ability to compose functions, which at its least is really annoying, and its at worst has a huge impact on a project.
[5] This code is not without its faults. It builds the stack, but is composable. A solution using iterators won’t build the stack and is more efficient, but is not composable, leading to code that is harder to reason about.

Written by Tom Adams

June 27th, 2008 at 3:46 pm

Posted in Ruby,Scala

Running a Rails app under Tomcat using JRuby

with one comment

A new project has me looking at running a Rails app under Tomcat using JRuby [1]. A quick google revels many conflicting guides, which seem to stem from different versions of JRuby and tools (e.g. GoldSpike vs. Warbler). I ended up using this guide for getting JRuby installed, Warbler for packaging of the WAR file and this guide as a basis for setting up a project to run under Tomcat.

The issues I encountered were mainly around these conflicting guides (I recommend you read the JRuby wiki), figuring out which gems to install, which gems need to be in the WAR file and how to configure the database. Don’t take this as the canonical way to do things, it worked for a simple MySQL-backed application based on our requirements below and is accurate as of 20/05/2008 (i.e. things will probably have moved on when you read this).

I built a simple single-table (MySQL) database-backed application and managed to successfully deploy it under Tomcat as a standard WAR file. This work was intended as a spike to determine whether this was possible using the current state of JRuby and associated tools. Our main focus was to determine if the application could support CSS bundling, action/page/fragment caching, correct cache HTTP headers (ETag, expires, cache-control, etc.), compression and a high YSlow score. We’ve done this work recently for Scoodi and know Rails is capable of supporting these under Mongrel.

The good news is it seems to work as advertised.

The process

  1. Install JRuby (download the zip & extract into /opt), be sure to add it to your PATH.

    Install the following gems (the DB ones are MySQL specific):

    $ jruby -S gem install -r jruby-openssl   # required for correct OpenSSL support, removes annoying nag
    $ jruby -S gem install -r warbler         # creates WAR files
    $ jruby -S gem install -r rails jdbc-mysql activerecord-jdbcmysql-adapter
    

    Your installed Gems should look like:

    $ jruby -S gem list
    
    *** LOCAL GEMS ***
    
    actionmailer (2.0.2)
    actionpack (2.0.2)
    activerecord (2.0.2)
    activerecord-jdbc-adapter (0.8)
    activerecord-jdbcmysql-adapter (0.8)
    activeresource (2.0.2)
    activesupport (2.0.2)
    jdbc-mysql (5.0.4)
    jruby-openssl (0.2.3)
    rails (2.0.2)
    rake (0.8.1)
    rspec (1.1.3)
    sources (0.0.1)
    warbler (0.9.9)
    

    This step actually took quite some time, as there’s many different gems available that look similar, for example if you look at the list on RubyForge with an eye towards MySQL, there’s ActiveRecord-JDBC, activerecord-jdbc-adapter, activerecord-jdbcmysql-adapter and jdbc-mysql. After you install them (or ask via gem) it’s pretty clear that some are dependencies of others, but this isn’t helpful at first.

  2. Create a Rails app. Based on the naming of Warbler, I chose to call it whipbird.
    $ jruby -S rails whipbird
    $ cd whipbird
    
  3. Warble-ise the app (this doesn’t seem to do anything to the application itself, it just creates the WAR contents in tmp, other warbler commands – see below – do allow you to make configuration changes):

    $ jruby -S warble
    
  4. Install Tomcat 6.

    Copy the generated WAR file to <TOMCAT_HOME>/webapps. You should get the following in your /opt/tomcat/logs/catalina.out.

    May 19, 2008 2:56:30 PM org.apache.catalina.startup.HostConfig deployWAR
    INFO: Deploying web application archive whipbird.war
    May 19, 2008 2:56:32 PM org.apache.catalina.core.StandardContext addApplicationListener
    INFO: The listener "org.jruby.rack.rails.RailsServletContextListener" is already configured for this context. The duplicate definition has been ignored.
    

    I haven’t looked into what the last warning means.

  5. Hit http://localhost:8080/whipbird/. You will see the default Rails page.

  6. Open up the project in IntelliJ (create the project, select JRuby as the SDK). If you ask it to, IntelliJ will install and configure RSpec and RSpec on Rails as plugins in the project.

  7. Configure Warbler to pull in the required gems:

    $ jruby -S warble config
    

    Add the following to config/warbler.rb:

    # Gems to be packaged in the webapp.  Note that Rails gems are added to this
    # list if vendor/rails is not present, so be sure to include rails if you
    # overwrite the value
    config.gems = ["actionmailer", "actionpack", "activerecord", "activerecord-jdbc-adapter", "activerecord-jdbcmysql-adapter", "activeresource", "activesupport", "jdbc-mysql", "jruby-openssl", "rails"]
    config.gems["rails"] = "2.0.2"
    
  8. Configure the databases in config/database.yml, the ActiveRecord-JDBC page has more details (there are other ways to do this using a standard JDBC URL, but this is the easiest):

    development:
      adapter: jdbcmysql
      username: root
      password:
      hostname: localhost
      database: whipbird_development
    
    test:
      adapter: jdbcmysql
      username: root
      password:
      hostname: localhost
      database: whipbird_test
    
    production:
      adapter: jdbcmysql
      username: root
      password:
      hostname: localhost
      database: whipbird_production
    
  9. Create the databases:

    $ mysql -u root
    mysql> create database whipbird_development;
    mysql> create database whipbird_test;
    mysql> create database whipbird_production;
    
  10. Point logging at the Tomcat logs, add the following to config/environment.rb:

    # Make Rails logging appear in Tomcat logs.
    config.logger = Logger.new(STDOUT)
    
  11. Go ahead and create some code. I used the scaffold generator, then added migrations, modified the views, etc.:

    $ jruby ./script/generate scaffold Bird
    
  12. I also hacked together theses Rake tasks to simplify deployment of the application into Tomcat, they should really be using Capistrano.

    TOMCAT_HOME="/opt/tomcat"
    WEBAPPS_HOME="#{TOMCAT_HOME}/webapps"
    APPLICATION_NAME = RAILS_ROOT.split(File::SEPARATOR).last unless defined?(APPLICATION_NAME)
    
    desc "Deploys the application, stopping and starting Tomcat"
    task "deploy" => ["deploy:stop", "deploy:app", "deploy:start"]
    
    namespace "deploy" do
      desc "Packages and deploys a WAR file to Tomcat"
      task "app" do
        puts "Creating WAR file"
        puts `jruby -S warble war:clean war`
        puts "Deploying #{APPLICATION_NAME} to #{WEBAPPS_HOME}"
        puts `rm -rf #{WEBAPPS_HOME}/ROOT*`
        puts `mv #{APPLICATION_NAME}.war #{WEBAPPS_HOME}/ROOT.war`
      end
    
      desc "Stop Tomcat"
      task "stop" do
        puts "Stopping Tomcat"
        puts `#{TOMCAT_HOME}/bin/shutdown.sh`
      end
    
      desc "Start Tomcat"
      task "start" do
        puts "Starting Tomcat"
        puts `#{TOMCAT_HOME}/bin/startup.sh`
      end
    end
    

    Note that we’re deploying the application as the Root webapp, see below for details.

  13. Deploy the application:

    $ jruby -S rake deploy
    
  14. Hit http://localhost:8080/. Create some data.

  15. Some other things I did to get prototype working:

    • Made whipbird the default webapp by calling it ROOT.war (there are better ways to do this) and removing the old root webapp.
    • Turned on caching of CSS & JS, in application.erb.html, in the standard Rails way.
    • Turned on GZIP compression in Tomcat.
    • In config/environments/production.rb set
      config.action_controller.page_cache_directory = "#{RAILS_ROOT}/cache/"
      config.action_controller.fragment_cache_store = :file_store, "#{RAILS_ROOT}/cache/"
      

      Added caches_page :index to the top of app/controllers/birds_controller.rb.

  16. Findings

    My findings (based on our initial requirements) are as follows:

    • CSS Bundling – This works, but due to rules inside asset_tag_helper.rb, the rails app needs to run as the root webapp in Tomcat. There may be a way to fix this via config, or, you can always patch the rails source. This is not a Tomcat/JRuby problem, the same thing crops up when running mongrel with a prefix.
    • JavaScript bundling – Works, with the same caveats as CSS bundling (needs to be the root webapp).
    • Page Caching – Works fine. RAILS_ROOT is set to /opt/tomcat/webapps/ROOT/WEB-INF/ by default, so the default cache lives in /opt/tomcat/webapps/ROOT/WEB-INF/public/.
    • Cache headers – All pages/assets contain ETags and can be compressed. They do not have expiry headers, this is usually set up in the front-end web server (e.g. Apache).
    • Cache IDs – As with Mongrel-Rails, cache IDs are based on modification times of files so change when the file changes (usually on re-deployment).
    • YSlow score – A (94)

    Stay tuned for performance stats…

    [1] Rails for (initial) speed of development [2] and feature set. Tomcat as it’s our client’s current deployment container.

    [2] Based on my experience with Rails to date, I don’t believe the Rails story is really as rosy as usually claimed, even for a moderately complex application, i.e. the sweet spot for a Rails app is limited.

Written by Tom Adams

May 21st, 2008 at 9:17 am

Posted in Java,Ruby

Tagged with ,

Serving Javascript using Rails’ asset hosts

without comments

I’ve not seen this anywhere in my various googling, so thought I’d post it in case it’s useful to anyone.

As a graphically rich site, Scoodi serves a lot of static content to a client browser; images, CSS and Javascript. We recently moved to using asset servers in order to speed up the total download time, however this had an unexpected side-effect in some versions of Internet Explorer, we got security warnings from our rich-text widget (TinyMCE), which meant members could not create new items or edit existing ones in IE.

We managed to resolve this by serving Javascript files without a host, meaning that they’d be resolved relative to the page (so inherit its scheme, host, port, etc.), however this involved a change to some internal Rails code, not something we’d like to maintain.

However fixing it in a generic way proved to be quite easy. Rails 2 lets you pass a function as an asset host, which gets invoked when an asset host is needed. This function gets passed the source (the images, etc. requested) and in the HEAD of the trunk of asset_tag_helper.rb it also (optionally) gets passed the request.

So, here’s the quick solution, added to the appropriate environment file. We only serve non-Javascript files from asset servers, and we also don’t use an asset server for SSL requests (avoiding mixed content warnings in some browsers).

config.action_controller.asset_host = Proc.new do |source, request|
  request.ssl? || source =~ /javascripts/ ? "#{request.protocol}#{request.host_with_port}" : "http://asset#{source.hash % 4}.scoodi.com"
end

There is another to give Rails the no-asset server host, by replacing "#{request.protocol}#{request.host_with_port}" with "" (the empty string). This however has a number of problems, 1) you don’t explicitly know which host is being used, which may cause problems over SSL (we haven’t reached an SSL-aware environment at the time of posting, so I can’t confirm this) and 2) it relies on the internals of asset_tag_helper.rb, where "" is treated as “don’t use the host provided”.

config.action_controller.asset_host = Proc.new do |source, request|
  request.ssl? || source =~ /javascripts/ ? "" : "http://asset#{source.hash % 4}.scoodi.com"
end

For both these reasons we originally decided to go with explicitly listing the scheme, host & port to use. However, we’ve had some issues when connecting to the site using an alternate address (say the machine’s IP address), the address that gets returned (request.host_with_port) returns a hostname that makes sense on the machine (running Rails) but may not be resolvable by the client (browser). Not using the hostname (returning the empty string) will cause Rails to not append a host (e.g. src="/javascripts/foo.js"), causing the browser to resolve the assets relative to the page, which will take into account the (correct) address of the server as requested by the browser. Good stuff.

Written by Tom Adams

May 7th, 2008 at 8:32 am

Posted in Ruby

Tagged with , ,

IntelliJ IDEA 8 Roadmap

with 4 comments

Via Talios, the IntelliJ 8 roadmap has been refined a little, notable inclusions are:

  • Scala (finally!)
  • Python
  • git???

I’d be nice to see a preview drop of the Scala plugin, it currently doesn’t build from source.

Written by Tom Adams

April 17th, 2008 at 8:36 am

Browser Detect patch for Safari support

without comments

Scoodi’s picture uploading function is known to cause issues in Safari 2 when Keep-Alive connections are turned on. For performance reasons we want to enable Keep-Alives, however don’t need to inform older Safari users (though I don’t believe we have any current Safari 2 users) of the known issues.

We use Browser Detect within Scoodi for browser specific functions, which means IE at present, to do things such as correctly alpha blend images, etc. Unfortunately the current version of Browser Detect has limited support for Safari versioning, and especially doesn’t like the latest Safari versions (3+) which have slightly changed the user agent string.

Luckily, the details are documented so a patch is easy:


...
else if (this.isSafari) {
  // Correctly identify version, see: http://developer.apple.com/internet/safari/faq.html#anchor2
  if (ua.lastIndexOf('version/') != -1) {
    this.versionMinor = parseFloat(ua.substring(ua.lastIndexOf('version/') + 8));
  } else {
    this.versionMinor = parseFloat(ua.substring(ua.lastIndexOf('safari/') + 7));
  }
}
...

Which makes for some nice Safari detection:


...
this.isSafari1x = (this.isSafari && this.versionMajor >= 125 && this.versionMajor < = 312);
this.isSafari124 = (this.isSafari && this.versionMajor == 125);
this.isSafari132 = (this.isSafari && this.versionMajor == 312);
this.isSafari1up = (this.isSafari && (ua.lastIndexOf('version/') != -1 || this.versionMajor >= 125));
this.isSafari2x = (this.isSafari && this.versionMajor >= 419);
this.isSafari204 = (this.isSafari && this.versionMajor == 419);
this.isSafari2up = (this.isSafari && (ua.lastIndexOf('version/') != -1 || this.versionMajor >= 419));
this.isSafari3x = (this.isSafari && this.versionMajor == 3);
this.isSafari30 = (this.isSafari && this.versionMinor == 3.0);
this.isSafari31 = (this.isSafari && this.versionMinor == 3.1);
this.isSafari3up = (this.isSafari && ua.lastIndexOf('version/') != -1);
...

Here’s the complete file: browser_detect.js.

Hopefully something like this can get added back into the main distribution.

Written by Tom Adams

April 8th, 2008 at 3:54 pm

Posted in Design,Ruby

Tagged with , , ,

Safari’s Web Inspector

with 3 comments

We’ve been performance tuning Scoodi for the last few days, making extensive use of FireBug and YSlow in Firefox. Surely there’d have to be something similar for Safari? Please don’t make me start Firefox when I need to do this sort of work…

I’d heard about Safari’s Web Inspector and Drosera a while ago, but the latest Safari 3.1 makes these easily available in a new “Develop” menu (enabled in Safari’s preferences). The Web Inspector is nothing short of spectacular, it seems to have a lot of FireBug’s features (I’ve not found anything apart from the selection widget that FireBug has over it), wrapped up in the usual Mac eye candy.

Here’s a screen shot of it in action against the Scoodi UAT site, showing HTTP headers.

You can also attach it to the browser window (FireBug style) with the icon on the bottom left. I noticed this morning also that IE 8 now bundles a developer console also.

Written by Tom Adams

April 8th, 2008 at 11:35 am

Posted in Design,Mac,Ruby

Tagged with ,