Google App Engine, JRuby, Sinatra and some fun!

Posted on March 6, 2011 by khelll

Yesterday I was experimenting with Google App Engine for a little project that I was working on. I literally started from ground zero and could do my thing after a long night. I’m blogging about it to share you the idea and save you some time googling solutions. So, here is the thing in brief: I wanted to parse some feed on periodical basis and send an email with new entries.

Looks like a 10 minutes with Nokogiri and cron jobs. Actually that’s true except of the fact that I need to pay for a VPS so that I can run the script with various gems (since I needed to do some work on the feeds before sending, but that’s for another topic) and for using cron jobs. Thus, I decided to try something new this time and I went with GAE since it has memecached that I can use as an LRU cache, also it has cronjobs and finally it has a mail system. I’m using JRuby and Sinatra for this project.

Here are the steps:

Install JRuby, I’m using RVM on my machine:

rvm install jruby-1.5.5
rvm use jruby

Install needed gems, those versions are the ones which worked for me:

gem install jbundle
gem install appengine-sdk -v "1.4.0"
gem install google-appengine -v "0.0.19"

Create a simple app:

appcfg.rb generate_app notifier
cd notifier

Now let’s create the following files (Please note that the following code can be further enhanced):

Gemfile

# Critical default settings:
disable_system_gems
disable_rubygems
bundle_path ".gems/bundler_gems"
 
# List gems to bundle here:
gem "appengine-apis"
gem "appengine-rack"
gem "sinatra"
gem "sinatra-reloader"
gem "jruby-rack", "1.0.4"
gem "nokogiri","1.5.0.beta.3"

config.ru

(fill the sender/receiver emails and the feed URL)

require 'sinatra' 
 
set :sender_email, 'SENDER_EMAIL'
set :receiver_email, 'RCEIVER_EMAIL'
set :feed_url, 'FEED_URL'
 
require 'app'
run Sinatra::Application

app.rb

require 'digest/sha1'
require 'appengine-apis/memcache'
require 'appengine-apis/logger'
require 'appengine-apis/mail'
require 'appengine-apis/urlfetch'
require 'sinatra'
require 'sinatra/reloader' if development?
require 'nokogiri'
 
 
class Fetcher 
 
  def initialize(sender_email,receiver_email,feed_url)
    @sender_email = sender_email
    @receiver_email = receiver_email
    @feed_url = feed_url
  end
 
  def act
    send_email(create_message(get_results))
    "done"
  end
 
  private 
	def memcache
    @memcached ||= AppEngine::Memcache.new(:namespace=>"cache")
  end
 
        # get the new results only
	def filter_results(results)
	  res = []
	  results.each do |hashed_guid,info|
	    if memcache.get(hashed_guid).nil?
	      memcache.set(hashed_guid, 1)
	      res << info
	    end
	  end
	  res
	end
 
        # parse the feed
	def get_results
	  results = {}
	  doc = Nokogiri::XML(AppEngine::URLFetch.fetch(@feed_url).body)
	  doc.css('item').each do |item|
	    title = item.at_css('title').content
	    link = item.at_css('link').content
	    description = item.at_css('description').content
	    guid = item.at_css('guid').content
	    results[Digest::SHA1.hexdigest(guid)] = {:title => title,:description => description, :link => link}
	  end
	  filter_results(results)
	end
 
        # create email message
	def create_message(results)
	  template = ""
	  results.each do |info|
	    template += <<-DOC
 
	      Title: #{info[:title]}
	      Description: #{info[:description]}
	      Link: #{info[:link]}
	      --------------------------------------------------------------------------------------------
	    DOC
	  end
	  template
	end
 
 
	def send_email(message)
    AppEngine::Mail::send(@sender_email,@receiver_email,"New updates for the site",message) if message.length > 0
	end
end
 
 
get '/' do
  "Hello World!"
end
 
 
get '/notify' do
  Fetcher.new(settings.sender_email,settings.receiver_email,settings.feed_url).act
end

cron.xml

(You can set the period here, I’m using 15 minutes intervals)

<?xml version="1.0" encoding="UTF-8"?>
<cronentries>
  <cron>
    <url>/notify</url>
    <description>fetch new data</description>
    <schedule>every 15 minutes</schedule>
  </cron>
</cronentries>

Now, create an application-id at appspot.com. Then go to Administration >> Permissions and make sure the sender/receiver emails play some role in the app. Personally, I granted them the developer role.

Now, go back to the source and modify the application-id in WEB-INF/app.yaml

application your-app-id

We should be ready now, start development server, watch the console and make sure no exceptions are there:

dev_appserver.rb .

If everything is OK, go to http://localhost:8080/notfiy and you should see “Done”.

Let’s go live now:

(the first time you execute this, you will prompted to submit the GAE email/pass combination)

appcfg.rb update .

It should now be running at http://your-app-id.appspot.com, and in few minutes you should be receiving the first email. If not, go to app engine page then to Main >> Logs and check what’s the problem.

I like this stack cause it’s Zero cent cost, Zero deployment time, flexible, and it gives me exactly the freedom I want in terms of needed gems. Give it a shot, you won’t regret it!

This entry was posted in GAE, JRuby, Ruby, Sinatra and tagged , , , , . Bookmark the permalink.