>Ruby on Rails Hosting: From HostingRails to RailsMachine in a shake of a tail

>

Rails hosting is a hot subject, and with everyone asking everyone else about their experience, I thought its only fair to share my own experience, even though it is relatively limited.

Introduction

I started with a shared “professional” hosting plan from HostingRails. It was around $30/month, and included non-root SSH access to a shared server, and additional 150Mb of RAM for total of 200Mb (although that’s actually quite misleading, see below). I stayed on this plan for about 3-4 months, and then decided to switch to a virtual hosting plan from RailsMachine – their single server plan, for about $100/month (that includes dedicated 384Mb of RAM and a root access to my virtual server). This post describes the reasons behind switching, and compares pros and cons of each hosting plan. Our needs included the following setup:

• One application running in two instances (one production, and one test)
• Two-mongrel instances per application (so total of 4 mongrels)
• Capistrano based deployment / upgrades / restarts of the servers
• Apache 2+
• PostgreSQL 8.2
• Trac (+ MySQL) served over https
• Subversion repository served over https
• Subversion post-commit hook linked to a build environment, for continuous integration
• Automated remote daily backup of the entire environment

To be fair, the plans described here compare sort of like apples and oranges, so the review has to be taken with a grain of salt. Main differences are, of course, in the fact that one is a shared plan and the other one virtual hosting. People on smaller budget and without tight performance requirements will probably prefer the shared plan, while those with a bit more room in the budget and some UNIX know-how will most likely be much happier with virtual hosting. My hope is that perhaps this review will help someone make the right call for their environment based on their needs, budget and experience.

Detailed Comparison

1. HostingRails

• Pros
  • Cheap, only $30/mo for 4 mongrel instances (200Mb)
  • Easy Trac/SVN install
  • Rudimentary backup can be run from web UI
  • Friendly support staff
  • Timely support, usually within an hour or two
  • Lots of info on forums
• Cons
  • [Update: please see comments section for HostingRails response]. Slow, slow, and slow. We were sharing a server with 4Gb of RAM with another 50+ mongrel instances run by as many users. The server used over 1Gb of swap space and often showed load averages over 30. We were promised numerous upgrades, which did not materialize while we were using the service. Our sites and Trac at times took over 30 seconds to load the first page, probably while the corresponding process was getting un-swapped.
  • Have to email support for any changes to top-level apache configuration, such as adding domain aliases. This applies to setting up Trac, svn and mongrel apps.
  • Provided backup facility does not backup PostgreSQL/mysql DB, had to write our own script (but forums had a good starting point)
  • Our mongrels clearly were not allocated any amount of “reserved” RAM. Looks like buying extra 150Mb in reality means you can run three more mongrel instances, but has nothing to do with how much RAM you actually get, as was witnessed by swap usage stats.
  • As we decided to move because of performance issues, HostingRails refused to refund us the unused portion of the yearly fee we paid ahead as we were past first 30 days.
  • Locked in their choice of Ruby version, gems, PostgreSQL, MySQL, Trac – can not upgrade these individually, as each is shared among many clients.

Overall, I think that HostingRails has to fix performance issues and not pack as many clients on a single server if they want to be a viable solution for any business. Until they fix performance issues I would NOT recommend this service to anyone. If the performance is fixed, I would recommend this plan to less experienced engineers, who may not know how to use root on a UNIX system, or setup Apache, but have some basic understanding of Rails deployment. I would also recommend this plan on those with a really tight budget.

RailsMachine

RailsMachine provides virtual hosting, meaning you get your own dedicated server with root access. The server, OS and pre-installed software details can be found on RailsMachine website. Because you are sort of expected to use root access, some basic UNIX system administration skills are necessary to take advantage of this plan. However, the flexibility and power that come with this solution by far outweigh the additional $60/month we are paying for our own box. Note that we could pay only $75/month for 256Mb of RAM, but considering the list of applications we wanted to run, I chose to upgrade our plan to at least 384Mb.

• Pros
  • Much, much, much, faster. When using command line, sometimes you see server “hanging” momentarily while presumably other virtual machines are being processed, but the overall performance improvements are quite dramatic. Pages (trac/mongrel) load within 1-2 seconds consistently.
  • Fantastic provided Apache configuration, comes with plenty of dynamically compiled modules, template config files and modular structure. Our single apache server is providing umbrella access to both mongrel apps, SVN over https and Trac over https (using mod_wsgi, not CGI).
  • RailsMachine gem allows seamless initial capistrano deployment and subsequent installations
  • Apparently each virtual server is backed up automatically (but we do our remote backups anyway)
  • CentOS Linux provides convenient yum utility for installing pre-built packages. Sometimes it takes a few minutes to find the right package using “yum search” but once found, installation is a breeze.
  • Root access allows installing what you need for your environment, eg – latest PostgreSQL, latest Trac, etc. Was able to install mod_wsgi module to enable much faster Trac access than using traditional CGI access.
  • Our entire setup described above takes up 180Mb of RAM, with another 180Mb left for file system caching. Pre-installed MySQL and Apache are built with multi-threading enabled, so the processes manage their shared RAM very effectively. This means we have room to add another 2-3 mongrel instances to the setup without affecting pricing of our plan.
  • Responsive, knowledgeable support, although just the nature of the setup means you have to ask a lot LESS questions if you know what you are doing, since you can get most of the things installed yourself without bugging them.
• Cons
  • [Update: this has been fixed, see comments section] The server came installed with broken permissions on /etc/resolv.conf and /etc/hosts file. This broke host/domain name resolution from the host itself, until the permissions were set to allow read access to the above files.
  • Slightly more expensive.
  • Occasionally (but mostly rarely), shell access has a latency that appears to be CPU bound by the physical server and not the network.

Overall, I would absolutely recommend RailsMachine to a technically savvy engineers, who want freedom of their own box at a very affordable price. Ability to grow the size of the virtual box dynamically is also great – it’s very nice to know that when you need extra 1Gb of RAM, it’s there waiting for you (and your wallet).

Install Notes

A quick note about SVN/Apache configuration with authorization turned on. This has been mentioned on many SVN forums, but if you are setting it up, please be aware that mod_dav_svn contains memory leak that makes apache seg-fault during checkouts of a large repository tree (we are using authorization as well as authentication). I had to add the following clause to apache config for SVN directory to resolve to issue: SVNPathAuthz off This is what our SVN configuraiton looks like in case someone is looking for a complete config:

<VirtualHost *:443>
 ServerName svn.YOURDOMAIN.com

 SSLEngine On
 SSLCertificateFile /etc/httpd/sslcert/server.crt
 SSLCertificateKeyFile /etc/httpd/sslcert/server.key

 <Location />
       DAV svn
       SVNPath /var/lib/subversion
       AuthzSVNAccessFile /var/lib/subversion/conf/authz
       SVNPathAuthz off

       AuthType Basic
       AuthName "Authorization Realm"
       AuthUserFile /var/lib/subversion/conf/passwd
       Require valid-user
 </Location>
</VirtualHost>

References

1. Memory Leak in mod_dav_svn

Closing Notes

Here are the responses I got from HostingRails and RailsMachine – thanks guys…

Hi Konstantin, The issue with broken permissions on hosts/resolv/others has been fixed. It was due to a small (and short-lived) bug in our provisioning system. Thanks for mentioning us! -Rob at Rails Machine

Hi Konstantin, there’s a couple things I’d like to point out that I hope you’d be willing to correct in your review. I understand that you had a bad experience with us and respect your right to voice it.

1. Running a cluster of 4 Mongrels will only cost $22/mo on our professional plan, and $26, $32/mo on our business and platinum plans, respectively.

2. On our site we outline that we’re selling (physical + virtual) memory in blocks of 50MB on shared servers — when Mongrels or Static FastCGI instances are fired up they consume (’reserve’) these resources persistently.

Overall, and I suppose you’d have to take my word for it, your server ran into an unfortunate resource situation with a number of users who are now either suspended due to malicious behavior or upgraded to dedicated environments due to extraordinary growth of their apps. Most of our clients are having good hosting experiences with us and we’re working hard to maintain fast and reliable shared servers for Rails deployment at the lowest possible price. All the best, ~William at HostingRails

— William, thank you for your corrections and I appreciate that it must not be easy to maintain quality service at such a competitive price. I’ve updated the pricing I mention and added a reference to your comment. I am willing to believe that our server may have been exception rather than a rule, and I hope that in the future you have a way to move clients off loaded machines quickly.

 

Ruby on Rails Hosting: From HostingRails to RailsMachine in a shake of a tail

This posting has moved:

Ruby on Rails Hosting: From HostingRails to RailsMachine in a shake of a tail

 

>On Ruby on Rails with PostgreSQL, and Acts as Paranoid

>

Back a few years ago I was researching differences between PostgreSQL and MySQL databases, and chose PostgreSQL because at the time it supported foreign key constraints and many other fantastic SQL extensions that make developer’s life a lot easier. Today I am sure MySQL is just as functional as PostgreSQL, and it does appear to be a more popular choice as the Rails DB than MySQL. I still prefer PostgreSQL, it just feels more natural to me coming out of Oracle background, so I am probably biased (who isn’t?) Anyway, in the last year and a half or so, I’ve been writing Rails apps that use PG-based databases and found a few random tricks I’d love to share here. Opinions differ here, but just like accountants like the ancient double entry accounting system, I personally prefer a double validation system – where Rails validates my objects before/after updates, but the database double checks this using proper constraints. Rail’s validation system is very robust and extensive, and it is its power. However nobody is protected from human error, and it’s pretty easy to forget certain validations, some validations may be performed conditionally, and in general rails validations are at a higher level. I’ve certainly missed a validation or two in the past myself, or had a bug in my condition. So as a rule, I specifically like to use in all of my migrations:

• Foreign key constraints, with “ON DELETE CASCADE” to allow tests to drop/remove fixture data
• Check constraints to verify values satisfy certain conditions. Use these to validate finite value columns, such as state/status, gender, inheritance discriminator column, or polymorphic table’s type discriminator. Use it to validate pattern based columns, eg if you decide to store SSN as a string of the form XXX-XX-XXXX, a check constraint can ensure it really is.

Foreign Key Constraints

Let’s start with foreign key constraints. They are extremely useful in ensuring your relationships work and do not point to non-existing rows. It is even more important if you bypass ActiveRecord and use direct SQL in certain cases to insert data (as this may be the case with some batched or high-volume data loads). I use foreign keys on ALL of my tables, and haven’t had much trouble with that. The only issue to watch out for, is the order in which your fixtures are loaded. Basically you’d want to load your independent tables first, such as countries/states – those that do not depend on anything, and then load fixtures that depend on them in the order of dependence.

Please see the following post for an opposite opinion, and note that I don’t agree with author’s suggestion not to use foreign keys – read first comment, I too was saved on multiple occasions by having them, especially in the development (since that’s where you’ll be finding most bugs in the first place).

Here’s how to add a foreign key constraint in your migration:

create_table :profiles do |t|
   t.column :user_id,    :integer, :null => false
end
execute "ALTER TABLE profiles 
          ADD CONSTRAINT profiles_fk_user_id
          FOREIGN KEY (user_id) 
          REFERENCES users (id) 
          ON DELETE CASCADE"

Notice how all constraints are named. This is very important, because you want to be able to change/update constraints in the future (especially the check constraints – see below). Not giving a constraint a name explicitly forces DB to come up with an auto-generated name, which is not very useful. Any future migration that needs to change this constraint would be at loss as to how to reference it reliably. Using a proper naming convention is also a good idea: table_fk_field is common naming practice for foreign key constraints.

Check Constraints

Check constraints verify that values in a column satisfy provided conditions. They are extremely useful in ensuring you don’t get “^%$$$” as your gender value for user Bob, or “Frak!” as a status for your order. Database can be manipulated in many ways, and in my experience unless you protect your columns some weird stuff always ends up in there, whatever the greatest validation framework sits in front of it. In this hypothetical example, we need to ensure that the gender column on our frogs table only allows “M”, “F” or “T” as possible gender values (I live in San Francisco, yo!). So in the migration file, specify check constraint similar to how we did this for foreign key constraints:

create_table :frogs do |t|
   t.column :name, :string
   t.column :gender, :string,   :limit => 1, :null => false
end

execute "ALTER TABLE frogs 
          ADD CONSTRAINT frogs_check_gender 
          CHECK (gender IN ('M', 'F', 'T'))"

Please see the following link for more information on PostgreSQL powerful constraints syntax:
PostgreSQL Constraints

Partial Indexes, and Acts as Paranoid

If you had to install acts_as_paranoid plugin, you’d notice that it adds deleted_at column which is going to be NULL for all values that are active in your database. Calling destroy will not physically delete the record, but would merely set deleted_at to the current time stamp. By declaring a model as “acts_as_paranoid” it will also add the following to all queries: and (deleted_at is NULL or deleted_at > ?) First of all, the “deleted_at > ?” clause is meant for allowing models to expire in the future, but I haven’t met a single person who uses this feature. So the first thing I do when I install this plugin is I modify it’s source to remove this comparison. I want my find statements generate the following: and (deleted_at is NULL) Ok, so things are great now, but suddenly I realize that my favorite index on users table and email field, does not really work as well! But of course… I am now querying on both “email = ? and deleted_at is null”. So how do you add deleted_at to the index? Here’s how:

execute "CREATE INDEX users_idx_email 
            ON users (email, deleted_at) 
            WHERE deleted_at IS NULL"

This uses a cool feature of PostgreSQL called partial index to create an index on subset of values.

Comments and thoughts/suggestions are always welcome! References:

1. http://www.oreillynet.com/ruby/blog/2005/11/migrating_to_ruby_on_rails_and.html
2. http://www.postgresql.org/docs/current/static/index.html

 

On Ruby on Rails with PostgreSQL, and Acts as Paranoid

This post has moved:

On Ruby on Rails with PostgreSQL, and Acts as Paranoid

 

Ruby on Rails: Reducing clutter in actions by placing common code in filters

This post has moved:

Ruby on Rails: Reducing clutter in actions by placing common code in filters

 

>Ruby on Rails: Reducing clutter in actions by placing common code in filters

>

This is a tiny but useful tip, that saved many lines of repeated code in my controllers, hence why not share it

Do Not Repeat Thyself?

If you’ve looked at the controller code that’s generated by scaffolding, you’ll find something like this:

 
  def show
    @user = User.find(params[:id])
  end

  def new
    @user = User.new
  end

  def destroy 
    User.find(params[:id]).destroy
  end

Sure, in this case all we are doing repeatedly is instantiating the user by a potentially available parameter value. What if instead we placed these common fetches in a controller filter, which would simply set an instance variable for us? Hell, we could even handle exceptions (such as invalid ID) in only one place this way! What not to like.

When dealing with a more complicated route that has been defined, this instantiation may become quite a bit more elaborate, and the case for a filter is even more justified.

Consider the case of building a collaboration system where you have projects and individual contributions under that project, as well as a producer of the project. We might want to support all project operations under a URL that looks kind of like this:

/user/kigster/projects/MyVacation/contribution/view/34

with a corresponding route in the config/routes.rb file:

# project management route
map.connect '/user/:username/project/:project_name/contribution/:action/:id',
    :controller     => "contribution"

Based on the route defined, Rails would create params[:username], params[:project_name] and params[:id] for contribution id, in addition to the standard action/controller pair.

Now imagine that the controller we are writing has many actions, such as add, edit, view, list, append, preview, post, comment, etc. All of them could use a handle on the project, it’s producer and contribution instances. Ideally – in @project, @producer_user and @contribution respectively.

If we followed the scaffolding example, we’d simply add appropriate lookups at the beginning of each action. But that’s a lot of repeated code!

So let’s use filters instead, to get our common lookups under control.

class ContributeController < ApplicationController
  before_filter :setup_project

  def home
     # do stuff
  end

  private

  def setup_project
    @producer_user = 
      User.find_by_username(params[:username])
    @project = 
      @producer_user.produced_projects.detect do |p|
      p.name.downcase.eql?(params[:project_name].downcase)
    end
    @contribution = 
        @project.contributions.find_by_id(params[:id])

    return true

  rescue Exception => e
    logger.info "can't show project:", e
    flash[:error] = e.message if e
    redirect_to :controller => "home" and return
  end
end    

This is short and elegant, and now every action in this controller (which is not excluded for setup_project filter) will have access to our instance variables. Groovy!

Issues

Of course nothing is free, and in this case we are potentially loosing performance. Perhaps some actions don’t need to know the @producer_user. We’ll be fetching it all the time, which saves time for us – developers. If performance problems occur because of extra unused fetches the filter can be broken up or optimized. Until then – it sure saves me a headache.

That’s it!

Thoughts, comments?

 

>Mac OS-X tips: How to run SSHD on an alternative port

>

This tip falls into one of those “I had to spend more than 10 minutes looking for an answer” category, so it’s a worthwhile subject for a quick blog post.

Why Run SSH?

Running SSH on Mac OS-X allows you to login to your machine remotely, and also copy file securely via SCP command to and from your Mac OS-X host.

But I am behind my own $50 router/firewall. Can I still connect to my computer from outiside?

Yes. Most off the shelf routers and firewalls will allow you to do two things that are needed for this:

1. Assign a permanent IP address to your Mac on a local network (see your router documentation for more details)
2. Create a port forwarding rule on your router. Eg – any request to port 22 on your external IP (provided by your router) can be routed to the specific IP address of your Mac. Exact specifics on this configuration are once again available in your router documentation. Most off the shelf routers support this, including basic Netgear and D-Link.

So let’s assume you’ve set this whole thing up and you are now able to connect from outside to port 22 of your external IP address on your firewall, which is then routed to your Mac port 22. This is great, since now you copy files via SCP from the internet to your computer, connect to it from your work office, etc, etc.

But why would you want to run SSH on alternative port, and not 22?

Simple answer is that port 22 is probably the most obvious port to “probe” from outside. Hackers typically run automated port scans of publicly visible IP addresses, but running it for all 64K ports takes a long time, so typically those scans are just for a small subset of ports. Port 22 is a clear indication of a server system which makes it a likely candidate for an attack. Turn of port 22 and to the outside world your system appears a lot more boring. Boring is good. So I still want to use my SSH connection, but would rather not do it on the default port. More expensive firewalls and routers will allow you to route eg. port 33333 from the outside to port 22 on the inside. But none of the cheap off-the-shelf routers I’ve seen allow you to do that. They just take input port, and destination IP which means the outside port and inside port must be the same. That’s a silly limitation and a small bummer. So to make this work with my cheap a$$ router I have to change the listening port on my Mac. Frankly, this isn’t a bad idea either, since whenever you connect your laptop to a public wi-fi network anyone can directly connect to your Mac and probe the open ports. So keeping port 22 open on your Mac is not that great of an idea. So. In order to change the port on Mac OS-X for your SSH daemon, follow the following steps:

1. Open Terminal and as edit the file /etc/services (as root)
2. Add a line at the bottom: secret-ssh 43539/tcp # secret SSH port
3. Edit file /System/Library/LaunchDaemons/ssh.plist and replace the code:

   <key>SockServiceName </key>
   <string>ssh </string>
   

   with

   <key>SockServiceName </key>
   <string>secret-ssh </string>
   

4. Change your port forwarding configuration on your firewall to route port 43539 instead
5. Open “Sharing” control panel and ensure that “Remote Login” is checked off (if not uncheck it). Then check it again to start with the new configuration.

Now you should be able to run SSH command to your external IP as follows: ssh -p 43539 ip_address_of_your_server -l username To copy files from remote hosts to your Mac desktop, run the following command:

scp -P 43539 local_file.txt username@ip_address_of_your_server:~/Desktop

Note that “scp” command requires an upper case “P”, while regular “ssh” wants a lower case “p” to declare remote server’s port. That’s it, hope it helps!

 

Mac OS-X tips: How to run SSHD on an alternative port

This posting has moved:

Mac OS-X tips: How to run SSHD on an alternative port

 

>New laptop and it’s a showoff: G5 vs Intel

>As I left my most recent permanent job for a private gig, I had to get myself my own new laptop. Of course the machine had to be a Mac, but which one? A laptop? Desktop? I decided on laptop since it’s so much more portable and I do contracting. But then – 17″ or 15″ and what specs, and should I shell out for a brand new model, or get the older one cheaper?

To cut the long story short I went for the top of the line, most recently upgraded model of 17″ MacBook Pro, with 2.4Ghz Intel Core Duo processor, 7200RPM 160Gb drive, 2Gb of RAM and a hi-resolution 1920×1200 screen. This machine on Apple store costs $3800 (with tax and apple care), but I managed to save more than a third by buying it from a dude on Craigslist.

My first impression – wow! The screen is so incredible, there are so many pixels that a window dragged when onto my older SyncMaster 23″ screen looks twice as large! That’s because 23″ screen has only 1600×1200 resolution, but the laptop packs 1920×1200 in a 17″ screen. Even upgrading to 1920 pixel wide screen would still not be adequate since each pixel would be larger than on a macbook. Gotta get used to this zooming effect

That minor inconvenience aside, I decided to run some basic tests on my older PowerMac Dual 2Hgz G5, with 1.5Gb of RAM and thought it’d be fun to share it here:

Test #1: Compile ruby interpreter from scratch:

• 17″ MacBook Pro 2.4Ghz Intel Core Duo:

  real    1m3.398s
  user    0m45.229s
  sys     0m15.373s

• PowerMac Dual 2Ghz G5:

  real    2m0.422s
  user    1m28.266s
  sys     0m29.412s

I’ll add more tests as I run them, but to see an almost 2x difference between 2 year old PowerMac and a new MacBook is simply amazing. Can’t wait to start using the laptop for music!

Meanwhile, PowerMac is available for sale for $1000

Laterz!

 

The KORE Arrives

This posting has moved to:

The KORE Arrives