Roman Le Négrate’s weblog

Since I finally found a use for the Rails metal applications (basically, Rack applications run before going through regular routes), I was surprised for my code not to be reloaded between requests in development mode. This article presents a portable monkey patch that implements this missing behaviour.

Problems

As mentioned in the introduction, metal applications precede the regular request dispatch. But since constants are reloaded just before this dispatch happens, constants are still old at the time metal applications are cascaded through.

Not only that, but Rails::Rack::Metal uses require to load metal application constants instead of letting ActiveSupport::Dependencies handle them like models, controllers and such. And since the Rails initializer attempts to load any existing metal application to determine whether it’s worth adding the Metal middleware to the stack or not, the corresponding constants are initially loaded via require, making them unknown to AS::Dependencies, thus preventing them from being reloaded afterward.

Patch

The patch below circumvents the above problems by:

• reverting the consequences of the calls to require from Metal.metals, and preventing further require’s from happening;

• adding a new step before going through the metal application stack which consists in forcing all constants to be reloaded and rebuilding the list of available metal applications from fresh constants.

Tuck the following chunk of code in config/initializers/reload_metals.rb:

(If the code doesn’t show up, get the gist manually.)

Or place it inside config/environments/development.rb, wrapped in a config.after_initialize do .. end block, and stripped from the bottom if Rails.env.development? statement.

Caveats

• The above patch hasn’t been tested extensively, so use at your own risk. Nothing critical could happen though: I haven’t noticed anything strange yet, but at worst, expect weird constant (re)loading behaviour.

• With this patch applied, constants are reloaded twice if the request goes past the metal application stack: once before the stack, and once before the normal dispatch. This can slow down development mode if you have a lot of models. Ideally, ActionController::Dispatcher#reload_application and #cleanup_application (for Rails up to 2.3.0) should be inhibited if both the Metal middleware and the patch of this article are active.

Compatibility

The patch doesn’t seem to be compatible with edge Rails (> 2.3.2.1) anymore. I won’t look into it for now since I don’t use it anymore but any contribution towards making it compatible with the latest version of Rails would be welcome.

Updates

UPDATED 2009-04-13: Added a note about compatibility.

cgit is an alternative to Gitweb that is (really) fast. While it builds flawlessly on Linux, building it on Mac OS Leopard turned out to be a little more complicated. Here’s how I did it.

Building cgit

Fetching the source

First, get the source code of cgit itself:

$ git clone git://hjemli.net/pub/git/cgit

Then, get the source code of Git, the libraries of which being linked to the cgit executable for performance reasons:

$ cd cgit
$ git submodule init
$ git submodule update

Building the executable

If you try to build the executable as per the README:

$ make

You will get the following error message:

Undefined symbols:
  "_iconv_close", referenced from:
      _reencode_string in libgit.a(utf8.o)
      _reencode_string in libgit.a(utf8.o)
  "_iconv", referenced from:
      _reencode_string in libgit.a(utf8.o)
  "_iconv_open", referenced from:
      _reencode_string in libgit.a(utf8.o)
ld: symbol(s) not found
collect2: ld returned 1 exit status
make: *** [cgit] Error 1

Fixing the error

For some reason, libiconv seems not to be set up to be linked against the final binary file. To tell the compiler to do so, run:

$ env LDFLAGS="$LDFLAGS -liconv" make

If all went well, you should now have a file named cgit in the current working directory.

 Moving the executable to the right place

If you choose to put cgit in ~/Documents/cgit, you need to move some other files along with it:

$ cp cgit cgit.css ~/Documents/cgit
$ cp cgit.png ~/Documents/cgit/git-logo.png

Using cgit

The cgit executable is a stand-alone CGI application that reads its configuration file at /etc/cgitrc by default.

Changing the path to cgitrc

To change the default location of cgitrc:

1. Open the Makefile:

   $ mate Makefile
   

2. Modify the CGIT_CONFIG line:

   CGIT_CONFIG = /Users/roman/Documents/cgit/cgitrc
   

3. Re-run the same make command as earlier:

   $ make clean
   $ env LDFLAGS="$LDFLAGS -liconv" make
   

4. Update the binary file:

   $ cp cgit ~/Documents/cgit
   

Pointing to the repositories

According to the gitrc documentation, the default values seem sensible enough* that cgitrc needs only be told where to find the Git repositories:

repo.url=flucti/manager
repo.path=/Users/roman/Repositories/flucti/manager.git
repo.desc=Flucti system management application
repo.owner=roman.lenegrate@gmail.com

* That is, unless the interface is to be made public, in which case you would want to set clone-prefix too.

Setting up the webserver

Now, you will need a webserver in front of the executable to spawn instances of it upon web requests and return the output to the client:

• Nginx is not well suited for that purpose since it can only connect to FastCGI or plain HTTP backends but not spawn external processes for security reasons.

• Of course, there’s Apache coupled with its mod_cgi module but I wouldn’t want it hogging the memory of my desktop computer.

• Fortunately, Lighttpd is lightweight and handles CGI processes very well.

Here is chunk of lighttpd.conf that’s cgit-specific and actually enough to start lighttpd and serve the web interface:

In ~/Documents/cgit/lighttpd.conf:

server.port           = 8080  # to be able to start the server as a regular user
server.modules        = ("mod_cgi")
server.document-root  = "/Users/roman/Documents/cgit"
dir-listing.activate  = "disable"
server.error-handler-404 = "cgit"
cgi.assign            = ("cgit" => "cgit")

To start the server in a dedicated terminal tab:

$ cd ~/Documents/cgit
$ lighttpd -f lighttpd.conf -D

The web interface should be available at http://localhost:8080:

cgit index

cgit summary

Note

So far, I found one problem with this lighttpd-cgit combination: cgit’s caching mechanism behaved strangely so I had to disable it by commenting out both the cache-root and cache-size lines in cgitrc. It might be due to the way lighttpd is configured but I’m not sure. Anyway, since it’s only a Mac OS installation, it’s likely not for production use, so the lack of caching is actually not an issue.

Good luck!

If you do or want to generate PDF documents from a Rails application using the nice Ruby library Prawn, writing view files named like foo.pdf.prawn, you need a new template handler.

Such a template handler already exists, it’s called prawnto. But once you begin writing view code for prawnto to process, you will find yourself writing pdf.foo all over the place, decreasing the readability of the code. prawnto offers a solution to that, .prawn_dsl, but it uses instance_eval, preventing the view from accessing instance variables set by the controller without passing a hash of locals to render.

Introducing Prawn-handler

I implemented an alternative solution in the form of a new template handler simply named Prawn-handler. For installation and usage instructions, please refer to the README.

Implicit receiver

In order to call methods on that pdf object implicitely, namely without specifying pdf as the receiver of Prawn::Document methods, another solution is to take advantage of method_missing so as to consider pdf the default, implicit receiver of calls to methods that are undefined in the view context. That way, the view code is still evaluated in the regular context which gives access to both instance variables from the controller and helper methods. In addition, just in case you happen to have defined a helper method whose name would collide with one of Prawn::Document’s, you can still explicitely indicate pdf as the receiver.

Hello World

All the above is probably better explained through examples. Take this simple Hello World example view code:

Where pdf is a Prawn::Document, receiver of the call to the text method. An equivalent to the above is:

If no text helper method exists, then it’s forwarded to the pdf object. If you think there’s no big readibility improvement in this small one-liner, read on for the next example.

Larger example

Consider a bigger chunk of code such as the following, based on prawn/examples/bounding_box/bounding_boxes.rb:

See all the occurences of the receiver pdf? You can get rid of them, ending up with a much clearer code:

Now try generating an invoice using Prawn without this convenience: it becomes painful very quickly. That’s in such a huge amount of code that the absence of pdf.foo’s starts to shine.

Limitations

Looking at the source code of Prawn-handler, you can see how tiny it is (only 22 lines of code at the time of writing). That’s because what’s described above is all it does.

The following subsections show how to perform common tasks associated with using that new handler. Though prawnto allows for doing them in a friendlier manner, I prefer doing them the manual way since I don’t feel they’re the place of the handler to support. Not to mention, they are quite simple to do manually. Plus, they can always be refactored into helper methods if their code gets repeated more than twice.

Partials

PDF views being nothing more than Ruby code acting on a pdf object, PDF partials are just regular Ruby source files eval‘d in the context of the parent template. So for now, for lack of a proper implementation, you can do something like this:

In app/views/foos/show.pdf.prawn:

Then, put code extracted from show.pdf.prawn directly in _bar.pdf.prawn as-is.

Download as attachment

To force the browser to download the document file, the HTTP header Content-Disposition needs to be set accordingly. You can call a private method used internally by send_data and send_file:

Passing the :length option is mandatory as send_file_headers! expects it not to be nil, though it can only be set to false since the size of the document is unknown at this point. Beware that doing so will enforce Connection: close but that shouldn’t be a concern for most people.

Repository

The source code is available on GitHub. Feel free to participate to its development: with such a small code base, there’s room for contributions.

PS: this is the first article that I have written both in Markdown and via the Edit in TextMate InputManager. What a great combination! It makes writing articles so much more enjoyable.

UPDATED 2009-02-01: Fixed the code in the Partials subsection. Highlighted the code via GitHub’s Gist.

The previous version of RailsTestServing was meant to be stable. It almost was, by getting rid of load path issues and being compatible with Rails earlier than 2.2.0. But then something else came up: weird failures of assertion related to fixtures after the first run. It has been fixed in the development version and no error has been reported since then. As a result, v0.1.3 is released.

You are invited to (re-)read the updated introduction article for how to install and use this new version.

Thanks to great help from Justin Ko, Jack Chen (chendo) and Dr. Nic Williams, the subtleties of several load path issues that prevented some (most?) systems to use RailsTestServing have been tracked down and worked around.

To illustrate this, here is a quote from the last e-mail I received from Justin Ko, about the main source of problems:

Were you using the version built from the repository or the gem?

repository, although I don’t think that made any difference. I think I could have had it working long ago if that variable was in the bash_profile. I do know that for whatever reason some people use bash_login, and others have/use bash_profile. I’ve noticed this trend when installing Rails and other things.

Version 0.1.2 includes the workarounds, both in code and documentation. Hopefully the load path issues are part of the past now. In addition to that, the new version packs 2 new features, courtesy of chendo:

• Support for running tests from a subdirectory under test/:

  $ cd test
  $ ruby units/user_test.rb

  Note that this assumes test_helper.rb can already be require‘d from the current directory.
• Ability to force require‘d files to be re-require‘d right after the regular constant cleanup. See the Options section of the README for more information.

As before, the introduction article has been updated with the latest instructions and gem version.

$ git clone git://github.com/Roman2K/rails-test-serving.git 
$ cd rails-test-serving && rake clean gem && gem install pkg/rails-test-serving-*.gem

UPDATED 2008-11-21: v0.1.3 has since been released.

⌘R for tests of Rails applications in TextMate has been broken by this commit. This commit did the right thing, but TextMate does not adjust Ruby’s load path so that test_helper.rb can be found.

The fix is rather ugly:

1. Open /Applications/TextMate.app/Contents/SharedSupport/Bundles/Ruby.tmbundle/Support/RubyMate/run_script.rb
2. Find the following line (should be line 22):

   if index = path_ary.rindex("test")

3. Append the following right after it:

   Dir.chdir File.join(*path_ary[0..index-1])

4. Append the following line to ~/.bash_profile:

   export RUBYLIB=".:test:$RUBYLIB"

   If you still get load errors, move that line to ~/.bash_login instead.

Hopefully it will work in your environment like it did in mine. Still not working? You might have more luck following this other guide (thanks to Justin Ko for the link).

UPDATED 2008-11-18: apparently, TextMate and/or bash tend to prefer ~/.bash_login over ~/.bash_profile. The article now reflects this observation (thanks again to Justin Ko for figuring this out).

The first version of RailsTestServing (v0.1.0) has been released yesterday.

Unfortunately, it turned out not to work for everybody. I received feedback from Justin Ko (thanks!) about some issues related to the load path and an incompatibility with Rails earlier than 2.2.0

I am glad to announce the release of v0.1.1 which reportedly fixed those issues. The introduction article has been updated with new instructions, so please re-read them if you had issues with v0.1.0.

UPDATED 2008-11-21: v0.1.3 has since been released.

Every time you run a test in a Rails application, the whole environment is loaded, including libraries that don’t change between two consecutive runs. That can take a considerable amount of time. What if we could load the environment once, and only reload the changing parts before each run? Introducing RailsTestServing.

With RailsTestServing, the run time of a single test file has gone from 8 seconds down to .2 of a second on my computer. That’s a x40 speed improvement. Now, I don’t think twice before hitting ⌘R in TextMate. It feels liberating!

Features

• Minimal reloading time: loads routes and application.rb before the run, starts clearing them in the background afterwards.

• Transparency: test files can still be run from the console and TextMate.

• Graceful fallback to normal operation when the test server is not started.

• Unobtrusive: three lines at the top of test/test_helper.rb.

• Test::Unit options: the full set is still handled.

Usage

1. Install RailsTestServing:

   $ gem install Roman2K-rails-test-serving -v '>= 0.1.4' -s http://gems.github.com
   

   In case you get an error saying the gem can’t be found, then:

   $ curl -O http://roman.flucti.com/wp-content/uploads/2009/02/rails-test-serving-0.1.4.gem
   $ gem install rails-test-serving-0.1.4.gem
   

2. Insert the following at the very top of test/test_helper.rb:

   require 'rubygems'
   require 'rails_test_serving'
   RailsTestServing.boot
   

3. Fix TextMate and/or your Ruby load path, following this short guide.

4. Start the test server (one for each of your Rails applications):

   $ ruby test/test_helper.rb --serve
   

   Wait for the following message to be displayed:

   ** Test server started (#94574)
   

5. Open a second console window and run your tests as usual:

   $ ruby test/unit/account_test.rb
   

   Or in TextMate, open test/unit/account_test.rb and hit ⌘R. Once the test has finished running, you should see the following message in the first window:

   >> test/unit/account_test.rb (709 ms)
   

   If you run the test from TextMate, you may encounter the following error in the RubyMate window:

   /Users/roman/usr/rubygems/gems/builder-2.1.2/lib/blankslate.rb:84:in `blank_slate_method_added': stack level too deep (SystemStackError)
   from /Users/roman/usr/rubygems/gems/builder-2.1.2/lib/blankslate.rb:84:in `blank_slate_method_added'
   from /Applications/TextMate.app/Contents/SharedSupport/Support/lib/builder.rb:86:in `method_added'
   

   This error is not related to RailsTestServing. The fix is to insert the following line at the very top of test/test_helper.rb (even before require 'rubygems'):

   $:.reject! { |e| e.include? 'TextMate' }
   

6. To stop the server, reopen the window you started the server from, and hit ^C.

As of v0.1.2, options are supported. As of v0.1.4, hotkeys have been added too. Please see the README for the lists and documentation on how to use them.

How it works

The server loads the environment and keeps it in memory to avoid having to reload it with every run.

When you run a test from the root of a Rails application, and if the corresponding test server is started, the process spawned acts as a client:

1. this client process hands its arguments over to the server. In turn, the server:
2. loads the routes and application.rb,
3. runs the test by processing the arguments,
4. returns its output to the client,
5. starts clearing every constant defined since step 2, in the background,
6. the client prints out the output.

That way, the client-server interaction is totally transparent to the user, either TextMate or yourself from the console.

Status

• Has worked like a charm for me for over a month.

• Only supports Test::Unit. RSpec already has something, the spec_server command.

  • 100% compatible with Mocha.
  • Appears to be compatible with Shoulda.

• Has only been tested with Rails from 2.1.2 up to 2.0.2 RC2. It is compatible with those as of v0.1.1 and is not guaranteed to work with a version of Rails out of that range.

• I would like to add (or see added) support for Autotest in the future, although I am almost certain it would require patching Autotest to modify the list of arguments passed to the Ruby interpreter. The same goes for tests run from Rake tasks.

Source code

RailsTestServing repository on GitHub.

Article updates

UPDATED 2008-11-17: This article now reflects compatibility fixes applied to v0.1.1. This version fixes load path issues and incompatibility with Rails lower than 2.2.0. Thanks Justin Ko for reporting these!

UPDATED 2008-11-18: There were still issues with the load path and TextMate. Step 3 has been updated to link to the fix.

UPDATED 2008-11-18 #2: Version 0.1.2 has been released after Justin Ko had figured out the key to making it work on his machine, which hopefully is the same for more people. The article over there has been updated according to the discovery: see the link at step 3.

UPDATED 2008-11-19: v0.1.2 turned out not to be as flawless as expected. The next version is in the works with encouraging results. If you get weird assertion failures from the second run onwards, you ought to try the development version:

$ git clone git://github.com/Roman2K/rails-test-serving.git
$ cd rails-test-serving && rake clean gem && gem install pkg/rails-test-serving-*.gem

UPDATED 2008-11-21: v0.1.3 released. This article has been updated accordingly.

UPDATED 2009-02-08: v0.1.4 released. This article has been updated accordingly.

This blog was taken offline for about a week until now. It’s hosted at home so it’s subjected to the risks that come with it…

In this case, believe it or not, my cat took a leak right on my ADSL modem, a Freebox. I guess he found the place warm and comfortable. Anyway, the Freebox couldn’t stand it as it shut down instantly and became unable to boot up again. Free’s hotline lived up to their reputation, providing a ridiculously useless help. That was enough for my father to decide to switch Internet providers and we went with DartyBox. The transition has been very smooth so far!

I apologize for the inconvenience. Hopefully the next article will make up for it.

If you update Firefox to the latest 3.1 unstable release (3.1b1 at the time of writing), you will notice that Firebug is incompatible with it.

Firebug releases are categorized in four release streams including the stable and beta streams. While stable releases don’t work with Firefox 3.1 yet, beta releases do.

Once Firefox 3.1 is installed, in order to install that beta version of Firebug, download-install the extension file (.xpi) from here and restart Firefox.

You can now experiment with Firefox 3.1 beta while not sacrificing Firebug.