Jonathan Gnagy's Blog

Writing Ruby code by yourself can be fun, but distributing it (either for use or other developers) can be a huge pain, thanks largely to changes in RubyGems and incompatibility between different gems. The Bundler project makes life so much easier. I use it in projects or even simple tools I write, mostly because I want my code to be used. Bundler has been out for a very long while, but I know a few people who haven’t used it much yet, so I figured there might be more of you out there.

So, here’s how we get started:

First, examine all the gems your project requires. This should be pretty easy to figure out if you’re doing things right:

1
2
3
4
5
6
7
8
9
10
11

#!/usr/bin/env ruby
require 'rubygems'
require 'sinatra'
require 'json'
require 'dm-core'
require 'dm-validations'
require 'dm-timestamps'
require 'dm-migrations'
require 'dm-types'

# Ruby Code starts here...

You can see here that I’m requiring a few gems for the Sinatra lightweight web app framework, the JSON gem, and several DataMapper gems. This works great now, but at some point, the latest version of one gem might become incompatible with the latest version of another (say Sinatra and DataMapper). To ensure we always have all of our gems installed at compatible versions, we can create a Gemfile (simply called Gemfile) and place it in the project’s root:

1
2
3
4
5
6
7
8
9
10

source "http://rubygems.org"
gem "json",         "~>1.7"
gem "sinatra",      "~>1.3"
gem "data_mapper",  "~>1.2"
gem "dm-sqlite-adapter"

platforms :jruby do
  gem "do_jdbc"
  gem "jdbc-sqlite3"
end

This file is pretty easy to follow, and just tells Bundler:

1. Where to pull gems from (in this case, rubygems.org)
2. To use a version of the JSON gem at version 1.7 or greater, up to and including version 1.9
3. To use a version of the Sinatra gem at or greater than 1.3, up to and including version 1.9
4. To use a version of the DataMapper gem at or greater than 1.2, up to and including version 1.9
5. To use the latest version of DataMapper’s sqlite adapter
6. On the JRuby platform (meaning when you’re using JRuby) we also:
   1. Use the DataObject JDBC gem for connecting to databases
   2. Use the JDBC SQLite3 gem

We needed to limit the JRuby gems, since they wouldn’t work on standard Ruby.

To use the gems installed by Bundler, we need to modify our application slightly to include another gem:

1
2
3
4
5
6
7
8
9
10
11
12

#!/usr/bin/env ruby
require 'rubygems'
require "bundler/setup"
require 'sinatra'
require 'json'
require 'dm-core'
require 'dm-validations'
require 'dm-timestamps'
require 'dm-migrations'
require 'dm-types'

# Ruby Code starts here...

Notice the subtle difference there? We added require "bundler/setup" which tells the application it will be requiring gems from the Bundler environment.

Now, the moment of truth. To make sure we have all of our gems at the right version, lets test out “Bundler” on a new machine.

First, let’s install the Bundler gem (this may need to be run as root if you’re using your system Ruby):

$ gem install bundler

After that, and after we’ve gotten the code to our test environment, open up a terminal and change directory into our project’s root, then type this:

$ bundle install

You may need to run this command as root or the equivalent using sudo or the like. If we get a message like this:

Your bundle is complete!

Then you’re good. Try running your application at that point:

$ ruby main.rb

You’ll notice that Bundler created a new file in your project directory called Gemfile.lock. This file is used to list out the exact versions of all gems installed by Bundler, including their dependencies. Again, this is useful to make sure everything is installed at the exact and expected versions. You’ll want this file, along with the Gemfile itself to be kept under version control.

Ever want to script the installation of an application that runs on IBM WebSphere Application Server? Well for all of you out there who do (and I know this is some pretty specific geek speak), there are a couple pieces that I just recently ironed out.

Increasing the Deployment Manager’s Max Heap

This is pretty easy to do in Jython in a scriptable manner:

1
2
3
4
5

# Increase the Max Heap of the DMGR
dmgrNode = "someserver-dm"
newHeap = "1024"
AdminTask.setJVMMaxHeapSize('[ -nodeName '+dmgrNode+' -serverName dmgr -maximumHeapSize '+newHeap+' ]')
AdminConfig.save()

Modifying a Port

Sometimes new app servers might have port conflicts with other app servers (or anything else on the machine):

1
2
3
4
5
6
7
8

# Fix a conflicting port
appServerName = "solrServer"
serverName = "someserver"
nodeName = "someserver-node1"
newPort = "9091"
AdminTask.modifyServerPort(appServerName, 
  '[-nodeName '+nodeName+' -endPointName ORB_LISTENER_ADDRESS -host '+serverName+' -port '+newPort+']')
AdminConfig.save()

Create a Cluster from an existing App Server

This is also pretty easy:

1
2
3
4
5
6
7

# Create the Solr cluster
clusterName = "SolrCluster"
appServerName = "solrServer"
nodeName = "someserver-node1"
AdminClusterManagement.createClusterWithFirstMember(clusterName, "APPLICATION_SERVER", 
  nodeName, appServerName)
AdminConfig.save()

Installing an Application into a Cluster

Seems almost too easy:

1
2
3
4
5
6

# Install the CacheMonitor application
clusterName = "SolrCluster"
AdminApplication.installAppWithClusterOption("CacheMonitor", 
  "/opt/IBM/WebSphere/AppServer/installableApps/CacheMonitor.ear", 
  clusterName)
AdminConfig.save()

I’ve got more stuff, but that’ll probably do for now.

I’m now officially hold an Associate of Applied Science in Computer Information Systems from Arizona Western College. It’s official (meaning I “graduated”) in the Fall of 2010, but in reality I haven’t taken any classes in at least 3 years. It took a fair amount of arguing and pleading my case to get AWC to acknowledge that I’ve met their own requirements for graduation, but here I am. Just have to wait a little while to get the actual paper diploma.

Now, on to my Bachelors… University of Toledo, here I come!

I passed the SCJA (Sun / Oracle Certified Java Associate for Java Standard Edition 5 and 6) exam yesterday. I’m excited, being that I took no formal training, and I’m not a Java developer. I didn’t pass with “flying colors”, but I did pass with more than a couple questions to spare. I just figured I’d post about it. Not sure how I’ll apply it to my career or life, but hopefully it’ll make a more valuable engineer.

It’s been a while, but I’m still here. Alive and well. Job’s been going great, and my baby girl is growing like a weed. More like a beautiful rose. Either way, I’m pretty happy.

I’m still doing plenty of development, especially in JRuby and, believe it or not, Java. The little bit of Java development I’ve been doing lately has made me appreciate Ruby that much more.

Since my last post, I’ve gotten a new laptop (13" MacBook Pro), got a Motorola Droid (the original) (and I rooted it and installed UltimateDroid on it), and I’m back to using a Mac at work again.

I’ve got to start working on some additional features on my blog, like facebook integration, since I’ve been using that a lot more lately. Like I said in a previous post, facebook is much more fun when you have friends. Now that I’ve got a few on there, it’s pretty nice.

I’ve heard people say great things Apache Ant and old faithful make, but I haven’t heard enough people talking about how wonderful rake is to work with. I’ve used rake for many simple and complex tasks, from importing YAML files into a Rails project, to managing LDAP users. Rake offers the full power of Ruby, but still offers a fairly simple DSL.

Here’s an excellent railscast about rake: http://railscasts.com/episodes/66-custom-rake-tasks.

Here’s some code that demonstrates creating a simple rake task:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

task :mkdirs do
  # Let's create some directories
  ["lib", "conf", "src"].each do |dir|
    Dir.mkdirs(dir) unless File.directory?(dir)
  end
end

# This task requires the 'mkdirs' task
task :copy_source => :mkdirs do
  require 'ftools'
  # Copy the source into the src directory...
  ["main.rb", "classes.rb"].each do |file|
    File.copy("/path/to/sources/#{file}", "src/#{file}")
  end
end

These two simple tasks create some directories and copy some source files. It also demonstrates task dependencies and pure Ruby file operations.

I’ve successfully unloaded the truck I rented (with the help of a couple good friends). My back hurts, I cut my hand pretty deep, and I’ve got a seemingly never ending list of groceries to buy from the closest store.

That drive seemed a lot longer than I remember it being. Maybe it was the truck, who knows. After all that driving, and all that unloading, I’m still not done. Now I need to unpack all these boxes too, and rebuild my dismantled furniture. I will admit though, that it’s nice to feel like the hard part of the move is over. Now I have to wait a couple weeks for my wife and baby girl to fly up here to live with me. Maybe the hard part isn’t over just yet.

I uploaded a very rough draft of a syslog server written in Ruby. Here’s a link to it. I talked about it in a previous blog post, and I think now its ready to show the world. Here’s a quick breakdown of some of the key features:

• Uses ActiveRecord for DB backend
• Heavily Multithreaded
  • Makes extensive use of Queues and Mutexes
  • Works in Ruby 1.9, JRuby, and Mostly in MacRuby for Native threads. (Obviously) Uses green threads in Ruby 1.8
  • Utilizes Thread Pools to maintain a set number of Threads
• Listens for UDP, TCP, or both.
• Uses a YAML config file
• Fully documented with RDoc
• Performs exceptionally under heavy load. I tested 10,000+ simultaneous connections with the included test client (under /test/).
• Written with MVC in mind. (I say “in mind” because there really is no “view” portion to this app)

That said, its still pretty new, and I’m still working on it. I welcome any comments about the code or any bugs encountered. Keep in mind that this SHOULD NOT BE USED IN PRODUCTION. I offer no warranty, implied or otherwise. I license this code under the GPLv3, even if I forgot to include a copy of the license in the tarball.

All-in-all, its actually a pretty simple app. Not that much actual code involved; just a little cleverness on how to process things one piece at a time. Here’s some of my favorite code, along with some parts that help it make more sense:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

# ...
        # This methods tells the server to pull all logs from the LogQueue and 'process' them.
        # Essentially, this moves logs from the incoming LogQueue to a process ThreadPool for
        # controlled processing. This method prevents an insurge of messages from eating up all
        # the server's CPU by limiting the number of logs processed at a time. After the log is
        # parsed in the process pool, it is queued up for storage in the storage ThreadPool.
        def process_logs
                until @log_queue.empty?
                        @queue_flushed = false if @queue_flushed
                        log = pop_log
                        @process_pool.add_job {
                                puts "Processing Log: #{log.object_id.to_s(16)}"
                                parsed_log = @parser.parse log
                                @storage_pool.add_job {@storage.store_log parsed_log}
                        }
                end
                if !@queue_flushed
                        puts "Log Queue Flush Complete @ #{Time.now}" if !logs_queued?
                        @queue_flushed = true
                end
        end
        
        # Utilizes the LogMutex to synchronize initial storage of log packets in the LogQueue
        def enqueue_log(log_data)
                @mutex.synchronize { @log_queue.push log_data }
        end
        
        # Utilizes the LogMutex to synchronize removing log packets from the LogQueue
        def pop_log
                return @mutex.synchronize { @log_queue.pop }
        end
        
        # Utilizes the LogMutex to synchronize checking how many logs are currently queued in the
        # LogQueue.
        def logs_queued?
                return @mutex.synchronize { @log_queue.size > 0 }
        end
# ...

Hopefully, with more work, this could have some serious potential.

Here’s a picture of Marissa showing off which team she is hoping will win. I’m like her in that I’m not really a big fan of the Saints, so we’ll both be cheering with the Cardinals win.

If you want to see a bigger version of the image, check out the Media Section of my blog.

I’ve had some time to burn since I’m not working for a couple weeks while I’m on paternity leave. During the night, my wife and I take shifts watching Marissa sleep (yeah, we’re paranoid like that). During my last shift (this one), I’ve accomplished quite a bit. I’ve managed to:

• Utilize Google’s Translate javascript API to provide site-wide translations
• Create a fully-functional, albeit simple, Todo / Checklist system and integrate into my blog
  • Uses Scriptaculous effects
  • I serialized the list items into an Array, so I only need a single column for an arbitrary number of list items. Also, no need for a separate “ListItems” model / table.
  • It allows me to mark lists as public, so everyone can see them. By default lists are private.
  • All adding and removing from the list is done via Ajax (with a non-javascript HTTP/1.0 POST fall-through), so it looks pretty snazzy.
  • I’ve started an experimental option to apply my tagging system to lists as well, but I haven’t figured out how I will make it all work, or how to display it.
• Polish up some sections of my site that weren’t quite passing XHTML 1.0 Strict compliance tests. I’m sure there are more out there, so feel free to comment if you happen to come upon a page this isn’t. Come to think of it, I’m sure the list stuff isn’t yet, since I didn’t check it.
• Make more of the site RESTful and support more pure XML operations (for scripting and such).

I’m sure I did more stuff, but I can’t really remember it right now. My eyes are awfully blurry… I’ll try to figure out some code to post tomorrow.